What is the best way to do mass maintenance mode?

Every month we have a 4-hour maintenance window for windows updates, software updates and other maintenance tasks that require downtime.

In order to prevent alert storms and errors in reporting I put all our DA’s and servers (with contained objects) in maintenance mode.

It always takes a long time for the scom enviroment to recover after this window and I’m usually left with some grey agents.

Is there a better way to do mass maintenance mode?

2 Likes

You could make a temporary SCOM group with all the objects you wish to place into maintenance mode. Then you can place all your objects into maintenance at once using the group drill down in Squared Up, I find that SCOM seems to recover quite quickly after doing this.

3 Likes

We use the SCOM Powershell module, devices appear to recover from maintenance without issue after this.

Look at Get-Help Start-SCOMMaintenanceMode -Full for information.

You’ll be able to iterate through a group of servers using a For Loop and a list of servers in a txt or csv file quite easily.

1 Like

I use the following method, which works perfectly, and allows me to schedule regular maintenance windows.

  1. Create a group containing the servers you wish to put into maintenance mode (or more than one group if you have a number of schedules )
  2. Create a windows schedule (or schedules) to run a powershell script.
  3. 3. Download the powershell script in
OpsMgr 2012: Group Maintenance Mode via PowerShell (the way it should be)

And edit the variables as per the article to specify the group name and the number of minutes that you wish the group to be put into maintenance mode.

As you can see from the screenshot above, I have a lot of maintenance mode tasks that are run on a daily basis. Also I have five groups that can be used adhoc to schedule maintenance mode on systems/servers that arise out of our CAB meetings. So far this has worked flawlessly.

It also has the advantage that any issues with the servers during the outage window all alert at the same time, i.e. when the window ends.

3 Likes

Personally I did a Orchestrator Runbook that looks at my collection for my maintenance windows and places them into maintenance window on a schedule. It was pretty simple runbook to write and it made it so that as a machine gets added to a maintenance window it automatically gets added to my runbook.

1 Like

We also use Orchestrator, but slightly differently. This specific customer uses scripts patch systems, and before they patch, they place a file with the servername in a specific folder. Orchestrator checks the files in this folder every 30 seconds, picks up the name and puts the specifc server in maintenance mode. If the server is already in maintenance mode, we add 4 hours to that specific maintenance window.

If the server belongs to a cluster, the cluster will be put in maintenance mode as well.

1 Like

We use SCOM 2012 Maintenance Mode Scheduler from http://www.scom2k7.com/

You can use it to put computer groups, classes etc. in Maintenance Mode.

1 Like

Hi

I have written a blogpost on how to Automate Maintenance Mode during patch windows.
SCOM 2012: AUTOMATIC MAINTENANCE MODE DURING PATCH WINDOWS

3 Likes

SCOM 2016 is the best answer for this, but all methods above will do what you require ?

I assume you do patching through configMgr?

Yes we are using Config Manager to do our monthly patching

Slight change that you may be interested in…we use a SCOM Monitor to pick up when the WindowsUpdateClient puts EventID 22 in the event log. This gives the User/OS around 15 minutes for a reboot to occur if your updates are automated.

Then use a Command Channel to run a powershell script on the management server to put the device that generated the alert into Maintenance mode for around 30 minutes.

If you have a set time window, you can obviously modify this to suit your environment. I believe Tao Yang had a guide to this for an older version of SCOM.