What I want to do for some monitors is getting first an informational alert and after a period of time (let’s say 15min) this alert needs to get to a warning state and after more time (another 15min) it needs to get critical. I was thinking to do that with a cmd notification channel, but don’t know how I had to do this. Any suggestions?
There is no straight forward way to change alert severities based on how long the alert has been open. I’d also question the value in this. Does the underlying issue really become more severe the longer it is unresolved? I could almost argue the other way around with this logic – if an alert is open for 48 hours and nothing “bad” has happened then can we lower the severity?
How does your organisation use SCOM? Is it done mainly by looking in the SCOM console or by receiving (and quite possibly ignoring) emails or is it via a ticketing system. I’d look to identify precisely what you are looking to achieve and why and then build a process around that.
For a ticketing system; there is hopefully some sort of SLA framework in place. So from the moment the ticket is created, the SLA starts ticking and gets closer to breaching that SLA. That way; you can offload all the work to another system.
If you are driven by emails then you could implement a “nag mode” along these lines – https://pavleck.wordpress.com/2008/09/10/scom-snippet-the-hidden-nag-mode/. This would need updating for SCOM 2012 \ 2016 but if this idea appeals we can put some scripts together. The challenge is that if the teams already receive too many emails and ignore them that this won’t change the situation. Unless you decide to escalate at each stage to their manager and then their manager.
If you look at the SCOM console then a view of alerts over x hours might do the trick.
We don’t use a ticketing system with our scom alerting.
I’ve been asked to create this for performance monitors.
By example, for the Tomcat JVM monitor. When the alert come for the first time, it may be informational. But when the alert stays for a longer time it may be critical.
I was thinking of doing this by a notification, because you can send a notification after a certain time. In the notification I can also run a command and I was thinking of changing the severity by this command. Bit I don’t know how to do is and if this is the right way?