Archive

Archive for September, 2008

The Alert Object’s Update method – bug, or intended feature?

September 24th, 2008 jpavleck 1 comment

I’ve talked about this before, calling it the hidden ‘nag mode’ inside of SCOM, but I really need to find out whether it’s intended or not.

Please see the following bug report I filed @Connect.

Essentially, if you grab an alert object via Get-Alert, then call the Update method, one of two things will happen depending on what parameters you fiddled with.

Open up the command console and grab an alert, something like $oneAlert = Get-Alert | Select -First 1
Now, change something in that alert, such as the resolution state ($oneAlert.ResolutionState = 111), then call $alert.Update(”") (Or comment it with $alert.Update(”Changed resolution state”)).

What happens?

As you expect, not much, just the alert resolution state was changed. Now grab another alert and call the update method alone ($alert.Update(”")) or with only a comment ($alert.Update(”Testing an issue”)).

What happens?

Whatever notification channel attached to that alert – usually email – will now fire again. So is it a bug, or not? If it isn’t a bug, then that’s excellent – we have more tools at our disposal and can now easily add that nag mode. If it’s a bug, that means it will be fixed, and as such shouldn’t be used.

There are additional bugs I need to file, regarding how a lot of the OpsMgr Commandlets, while stating they support the common parameters, actually don’t. And how the filter parameter should follow basic syntax and use “-Filter” instead of “-Criteria”. But that’s for another post.

PS: It’s my birthday on Friday. Perhaps you’d like to buy something from my wish list? Or at least enter the contest. The next 2 will have considerably better prizes, I promise!

Things in progress, SCOPE, contests

September 18th, 2008 jpavleck No comments

I’ve been working on several things all in parallel, and I’ll give you a little insight into them all.

But first, don’t forget to enter the Pavleck.NET Contest for your chance to win 1 of two Amazon.Com gift cards. We’re currently at a paltry 10 entries, so your chances are pretty good to say the least. If I get a good turnout for this, I’ll make this a regular event – I already have next months prize ready, too. A rare copy of System Center Operations Manager 2007 Unleashed – signed by all the authors. Not many of these exist, but I have one for you!

What’s I’m working on:

  • Writing a small MOM 2005 to SCOM 2007 migration report script. Examines your agents from MOM and compares them with what’s installed in SCOM.
  • Attempting to write a small service that will handle custom alert notifications by matching alert names to notification groups through the SDK. A simple XML file is used to create the configuration, and it’s as easy to setup as this:
    <SCRAPPI>
    <MS>
    <RMS>draco.pavleck.org</RMS>
    </MS>
    <AlertMatrix>
    <alert matchType="Prefix">
    <alertName>My Custom Alert Prefix</alertName>
    <notiRecip>f6f0278c-9bd7-874a-db18-cf85f2620c4d</notiRecip>
    </alert>
    </AlertMatrix>
    </SCRAPPI>
  • Work on SCOPE continues, including a partial command list – feel free to add to it.
  • As does an article (With a nifty flow chart!) of the steps to take to handle an alert storm, from the console all the way to SQL – get that system back in action!

Creating an audible alert notification in SCOM

September 12th, 2008 jpavleck 8 comments

Edit: 09/13/2008 – On the advice of Pete Zerger, updated script to include a throttling mechanism to prevent an overload if an alert storm occurs. Also changed things around to make it a more generic ‘run remote executable’ instead of run remote sound.

A question was recently asked on the MOM Mailing List over at myITforum.com.

That question was, quite to the point:
How to create a audible alert? I like to create one for the critical alerts..

I’ve been working earlier with a script that would go out and disable the run time tracing, stop it, then delete the log files. So I already had knew what would work – a simple PowerShell script that uses WMI’s process create method on a remote machine.

A caveat lector before I continue; while this solution will technically work, I haven’t tested it formally. Additionally, you’ll need to contend with permission issues that arise as well. If you’re running the OpsMgr services under a named account, you’ll need to give that same account local administrator access on whichever machine you plan to run this call against. If you’re using ‘Local System’ you’ll have to either add the RMS\Local System account to the remote machine’s admin group or embed credentials inside the WMI call((Be careful when doing this. I haven’t included directions for that because it’s just a nightmare waiting to happen. I can give you a jumping off point though.))

First, the script. It’s small and basic. It wants to know the machine you want to run the command on, the command, and because this is a a sound player, the path to the WAV file. It then creates the process via WMI, and decodes the return code. If it’s 0, everything is fine. If it’s anything else, the process creation failed and it writes an event to the Operations Manager event log, which you can create an additional rule to look for.

Download SCOM-RunRemoteExecutable.ps1

# ==============================================================================================
#
# Microsoft PowerShell Source File — Created with SAPIEN Technologies PrimalScript 2007
#
# NAME: SCOM-RunRemoteExecutable.ps1
#
# AUTHOR: Jeremy D. Pavleck , Pavleck.NET
# DATE  : 9/13/2008
#
# COMMENT: This is a Proof Of Concept script written in response to a mailing list request to
#               enable OpsMgr to sound an audible alert on a remote admin PC, such as a console in a
#               NOC.
#               This is to be run as a Notification Command Channel.
#
# NOTES/WARNING: This script uses a remote WMI call to spawn a process on a named server. As
#               such, there are security issues to keep in mind. I haven’t added the code to allow you
#               to use alternate credentials, but use http://poshcode.org/501 as a jumping off point.
#               If you’re running the OpsMgr services under a domain account, add that user to the local
#               administrators group on the machine you want to run this command. If you’re using local
#               system, add RMS\Local System to the admin group.
#               RUN AT YOUR OWN RISK!
#
# VERSIONS:
#               v1.0 – 09/12/2008 – Initial version
#               v1.1 – 09/13/2008 – On the advice of Pete Zerger, added a throttling routine to prevent
#                                                       to many executions during an alert storm
#               v1.2 – 09/13/2008 – Changed variable names to make it a more ‘run remote executable’ script
#
# ==============================================================================================

# User Settings
$remoteMachine = "adminconsole.pavleck.net"
# The location to the executable. This is the path on the REMOTE machine.
$myExe = "C:\Program Files\Real Alternative\Media Player Classic\mplayerc.exe"
$myExeParams = "C:\Windows\Media\tada.exe" # Paramters to pass to the executable, such as the location of the sound file, etc.
                                                                                   # Leave blank if none are needed
# Registry &amp; throttling settings
$myKey = "SCOM_PowerShell_Scripts" # Reg key name to use
$myValueName = "LastRunTime" # Data value
$interval = 5 # Wait at least this long, in minutes
# Initialize a couple things
$firstRun = $False
$throttle = $False

# We use the OpsMgr API only because it’s a very quick and simple way to log to the eventviewer
$momAPI = New-Object -comObject "MOM.ScriptAPI"
# LogScriptEvent Severities
$momErr = 1
$momWarn = 2
$momInfo = 4
# Setup some event ids to use
$errID = 11000
$warnID = 11001
$infoID = 11002

$myName = $MyInvocation.MyCommand.Name # Grab script name

### Registry throttling settings
# First see if our key exists, if not, create it and populate it with the current date/time
# and set $firstRun to $True
If(!$(Test-Path HKLM:\SOFTWARE\$myKey)) {
        New-Item -Path HKLM:\Software\$myKey
        New-ItemProperty -Path HKLM:\SOFTWARE\$myKey -Name $myValueName -Value (Get-Date)
        $firstRun = $True
        }

# If this isn’t the first run, compare previous time with current time – if last run is $interval
# minutes ago or higher, carry on, otherwise exit
If(!($firstRun)) {
        $lastRun = (Get-ItemProperty -Path HKLM:\SOFTWARE\$myKey).$myValueName
        Set-ItemProperty -Path HKLM:\SOFTWARE\$myKey -Name $myValueName -Value (Get-Date)
                If(((Get-Date)[DateTime]$lastRun).TotalMinutes -ge $interval) {
                $throttle = $False
                } else {
                $throttle = $True
                }
        }

# Function DecipherRetCode accepts an integer, and returns the failure assigned to that code.
# This only returns the most common failures, such as permissioning and the like
Function DecipherRetCode([int]$retCode) {
        switch ($retCode) {
                0 {return "Success"}
                2 {return "Access Denied"}
                3 {return "Insufficient Privilege"}
                8 {return "Unknown Failure"}
                21 {return "Invalid Parameter"}
                default {return "$($retCode) is uncommon, and will need to be researched manually. "}
        }
}

# This line is actually the entire script.
If($throttle) {
        # Throttling – cancel response
        $momAPI.LogScriptEvent($myName, $warnID, $momWarn, "Notification Workflow requested that $($myname) run, but last response ran less then $($interval) minutes ago. Exiting.")
        $momAPI = $null
        exit
} else {
$retCode = ([WMICLASS]"\\$remoteMachine\root\cimv2:win32_process").Create("$myExe $myExeParams")
        # If $retCode = 0 ($false) exit the If, anything else is $True, and will log it
        If($retCode) {
                $momAPI.LogScriptEvent($myName, $errID, $momErr, "Error creating process. Error Code: $($retcode) Error Message: $(DecipherRetCode $retCode)")
        }
$momAPI = $null
exit
}

To implement this, open the Operations Console and go to Administration > Settings > Notification
Click on the Command tab, then click on add. Fill it out as you normally would:

Then click on OK, and you’ll see it with the rest of your commands:

Now to finish it up you’ll need to create a new notification recipient. Right-click on Notifications and select new recipient.

Make the display something to designate that it runs a command, I used “Sound Audible Alert”. And because the NOC isn’t manned 24/7, I limited the notification time to weekdays from 8am to 6pm. You can also adjust this from the devices tab, but I’m not going to include an emailing or other devices, so I prefer to set it in the general tab, this way it’s obvious even with a casual glance what the settings are.

After that, click on the “Notification Devices” tab, then click “Add”.
In the resulting popup, select our new notification command and enter anything for the delivery address – I used NA, because for this particular command we don’t require any additional information – but OpsMgr still needs something in there. Hit next, keep the schedule at always unless you’re adding additional channels, next again, name the device – I used “Send Audible Alert”

Click OK, and your set. Treat it like any other notification recipient – either create a new rule just for this, or edit an existing subscription and add our new recipient to it.

As you can see, using PowerShell inside of Operations Manager makes it very flexible and powerful. We can run all manner of things in response to alerts; From running a simple sound file all the way up to initiating disaster recovery scenarios and intense system diagnostics – both things which I’ll be showing you later on as we explore the Notification Command Channel together.

SCOM Snippet: The hidden nag mode

September 10th, 2008 jpavleck 1 comment

When it comes to notifications, we have many options – except one that people have asked about, a nag mode. Something that will re-send an email after a certain amount of time to make sure it’s taken care of.

Well, it does exist in OpsMgr.

Either intentionally or unintentionally as a bug, if you call the Update method on an alert without changing any criteria, the notification bound to the alert will re-fire. This will happen whether you add a comment with the update (Update(”Updating the alert”)) or not (Update(”")).

To enable this secret nag-mode, it’s as simple as writing a Powershell script that runs every X hours. In that script you’ll just need to do a Get-Alert with the criteria you’re looking for – in the example I’m just going to have it return all alerts older then 4 hours, and update them.

It’s very simple though – how simple? Like this:

$oldAlerts = Get-Alert | Where-Object {($_.LastModified -ge [DateTime]::Now.AddHours(-4)) -and ($_.ResolutionState -eq 0)}
ForEach($alert in $oldAlerts) {
$alert.Update("")
}

You can expand this as much as you’d like. Match against NetBiosComputerName to only nag for those critical core servers, match it against the monitoring object to ensure critical monitors are being addressed. Multiple management groups? Match against that. You see where I’m going with this. In fact, you can find out everything you can match against by just running Get-Alert | select -first 1 – there’s all the fields available.

Your chance to win – it’s a reverse birthday!

September 6th, 2008 jpavleck 1 comment

On September 26th, I’ll be turning 32. In celebration, and as a way to play with ContestMachine.Com, I’m giving away two $32 Amazon.com gift cards! So enter between now and September 26th in the widget below, and good luck!

Categories: Pavleck.NET Tags:

Operations Manager: From Start to Finish

September 5th, 2008 jpavleck No comments

I’ve never had a ‘proper’ test/development environment for System Center products. I’ve used both client systems and VMs I’d spin up through VirtualBox. That will be changing.

I placed an order a few days ago for a new server – featuring a 2.8Ghz Quad core Xeon and 12GB of ram, it will begin a brand new environment – and I’ll be screencasting all of the most relevant parts of it.

We’ll cover sizing, install, deployment, security – and move on to extending OpsMgr by utilizing the SDK service, designing custom reports – and a lot more.

In the mean time, while I painfully await the arrival of a shiny box from DHL, I’m working on the framework for 2 new side projects – one of which I think you’ll all be quite happy with; the much awaited SCOPE – System Center OpsMgr Powershell Extentions – a collaborative operation between me, Marco Shaw, Cameron Fuller, Pete Zerger and – you, possibly. We’re at the very early stages of SCOPE, and could definitely use people now and down the road – especially C# programmers and those familiar with creating PowerShell snapins. If you’d like to help, send me an email (jeremy@pavleck.net) and let me know what you can do.

Reminder: System Center Virtual User Group Meeting 2, PowerScripting LIVE tonight

September 4th, 2008 jpavleck 2 comments

Two awesome events are happening tonight. I’ll be at both, come join me!

The first is the second meeting of the System Center Virtual User Group – Register to attend.
It begins 7pm CST (1am GMT) and the agenda is:
7:00PM-7:10PM: Introduction – Pete Zerger (Of SystemCenterForum.org)
7:10PM-7:40PM: Steve Rachui – Targeting in Operations Mananger 2007.
7:40PM-8:10PM: Cameron Fuller – Community Extensions for Operations Mananger 2007.
8:10PM-8:40PM: Pete Zerger – SNMP Device Monitoring in Operations Mananger 2007.
8.40PM-8:55PM: Closing – Pete Zerger

Logo for Google Calendar invite

The invitation has all the other information you need. This is a Live Meeting event, and dial in audio information is provided.

EDITED: Thanks for the additional info, Hal!

The second meeting happening today is episode 40 of the PowerScripting Podcast. This will also be the second (Or is it third?) time Hal and crew stream the show live!
If you can’t make it to the live UStream broadcast, be sure to subscribe to the podcast and get your weekly dose of PoShy goodness.
Along with the usual suspects of Hal, Joel and Jon, they have an extra-special guest this week; none other then the great Jeffrey Snover, one of the PowerShell architects. He’ll be taking your questions, so this would be a great time to ask about what’s new in v2, or why remote registry support sucks.

That all starts tonight at 9PM EST – so get your browsers ready to watch Hal and his beard. And if your place of work doesn’t block social networking sites, be sure to RSVP to the invite on FaceBook.

Logo for Google Calendar invite