Send Active vSphere Alarms to Microsoft Teams

One of the ways that we have been driving adoption of Microsoft Teams at work is by automatically pushing useful content into channels via its webhook support.

I've pushed alarms from our general monitoring system into Teams and it removes the limitation of short text messages while providing more details on an issue before resorting to using the corporate VPN.  I simply arm myself with the Microsoft Teams app on my phone.

I thought about what else might benefit from the ability to see details while on the go and came up with active vSphere alarms.  I wanted to be able to get details on active alarms, the time they started, their status (severity, if you ask me) and the objects and clusters involved.

I started playing around with PowerShell and soon had rudimentary alarm data going into a Microsoft Teams channel.  But what if I am or can be connected to work? 

I added a button to open the vSphere Client and then an alarm-specific button that opens the vSphere Client summary page for the object: 

Example vSphere Alarms Alert

Setting it all up is pretty straightforward: create an incoming webhook connector in Microsoft Teams, save the URL, and schedule the PowerShell to run at a frequency of your choosing.

Adding a Webhook

Open Microsoft Teams, choose or create a new Team, choose or create a new channel within that Team, and then select the three dots to the right of the channel name to create a connector.  The screenshots below will help lead you along, but it's pretty simple.  Be sure to save the link that is generated for your connector.  You can always go back and manage the configured connectors on your channel if you need to get the URL again.

Scheduling

Save the PowerShell at the bottom of this article and save it on a server somewhere so that you can set it up as a Scheduled Task.  Invocation is easy, and you will need to provide your vCenter FQDN and Microsoft Teams webhook URL as part of the program/scripts arguments:

-ExecutionPolicy Bypass -file Send-vSphereAlarms.ps1 -vcenter vcenter-fqdn -TeamsUri uri

I like to configure the task to run every 10 minutes, but pick what works best for you and your environment.  

I'm certain that a lot can be changed in the script. Play around and enjoy!

Send-vSphereAlarms.ps1

vSphere Syslog Slam

Maybe you are like me and you have a new syslog destination (SIEM is grand).  Maybe you want to ensure that all of your vSphere hosts are configured the same.

Automation being king, this will do it.  The general process for each host:

1. Set the syslog destination 
2. Insure a firewall exception 
3. Restart the syslog service.

In vSphere 5.1 the Set-VMHostSysLogServer cmdlet was added to PowerCLI.  While there are other ways to accomplish the same task, I tend to prefer using any method that is more obvious when coding.  Set-VMHostSysLogServer it is.

Cheers!

Windows Registry: Loading and Unloading All User Hives

Today my team was discussing the need to search all user registry hives when scanning for malware.

It's not too bad to do this manually if there are a two user profiles on a system, but it gets considerably more tedious on common-use PCs with more user profiles.  And I'm lazy.

The process might also prove handy for other registry-related tasks, where perhaps can't use Group Policy to be lazy.

Queue Load-UserHives.ps1. This quick-and-dirty PowerShell will search the system for available user profiles and load each key.  Later, you can reverse this with the -Unload switch.  Rather than externally tracking what was loaded prior, a simple naming prefix is used for the unload process.

vSphere Host Logging Levels

If you are configuring vSphere hosts for syslog collection you may be overwhelmed by the amount of data thrown at your collector. By default hostd and vpxa are configured for a logging level of “verbose.”

To view the configuration for all of your vSphere hosts, break out PowerCLI.  I’m forever forgetting how to retrieve information from certain cmdlets and include the host name.  Here it is for me to find again:

 

Setting hostd and vpxa logging levels on all hosts to “info” is pretty easy:

Snazzy-Up Your PowerShell with a GUI

I’d like to say that I work with virtualization every day, and in a way I do.  Virtually every server in my employer’s environment is virtualized (see what I did there?).

That we are a small team in an SME environment dictates that we are I.T. generalists.  That’s not a curse—that’s variety and opportunity alleviating the mundane.  It’s also an opportunity to automate the mundane. 

For the purposes of user provisioning I wanted to use PowerShell for identity management on a shoestring budget.   As with many things, creating is harder than destroying; this provisioning needs a GUI.  Pick an OU, pick a user/template, and copy with new details.  While that’s how I got to the point of driving a GUI with PowerShell, it’s difficult to effectively convey a company-specific workflow.  So I’ll demonstrate with something virtualization-focused.

With a little bit of Googling I found an older option or two along with .NET-based Windows Presentation Foundation (WPF).  .NET this plus .NET that sounded appealing with me, so I dug in further. 

There are solid posts to be found that give you all that you need.  I’m working against some of my musings, so here I share my journey toward the same end result.

At a higher level the process is relatively simple:

• Generate a GUI form in Extensible Application Markup Language (XAML).
• Load the XAML in PowerShell with some minor (automated) transformations.
• Instantiate the WPF form.
• Create variables for working with our GUI elements.
• Define PowerShell event handlers for each GUI element, incorporating our workflow within.

Generating Your GUI’s XAML

To generate the GUI we need something that will generate XAML that WPF can handle.  Anything from Visual Studio 2010 on up will do here.  I used Visual Studio 2015 Community since it is full featured with an investment of $0.

Open Visual Studio and create a new project.  Choose WPF Application under Visual C# Templates and give it a name.  Any name will do—we only want the XAML which this project will generate.

What you will notice is a blank window in the main center pane, labeled MainWindow.xaml.  The pane below contains the XAML in which we are interested. 

I find it best to turn on the toolbox from the View menu and pin it to the side bar this yields an environment tailored for creating window forms.

Let’s use the toolbox to drag and position a few controls:

• A ComboBox selector for Cluster
• A DataGrid to display VMs within a selected Cluster.
• A Button for launching a selected VM’s console.
• A Label or three for clarity.

While we can add these controls in any order, Visual Studio will set the form’s tab order based on the order of definition.  That’s not a particularly big deal because of a neat Visual Studio feature:  the GUI form is tied to the XAML and the XAML is tied to the GUI form. We can adjust the tab order by re-ordering the text inside the XAML pane copy and paste!

With each control use the properties pane and give it a unique name, which we will  use later to bind to PowerShell variables.  I like to start all label control names with “label_” for easier exclusion from this binding process.

Loading the XAML

All we need from Visual Studio is the XAML, which we will copy and paste into a PowerShell here-string.  The string isn’t perfect for PowerShell processing yet, so we’ll also rework it to take care of these issues and cast as XML:

Instantiating the GUI Form

Next we need to load the PresentationFramework assembly and process our form so that it can be presented:

Creating GUI Variables

We need to be able to work with the controls on form that comprise our GUI, and that means variables that let us get/set values and work with other properties and methods afforded to us by WPF.   

We can do this rather handily in PowerShell by invoking Set-Variable for nodes within the XML that comprises our form.  The iteration of those nodes will return Label controls as well, which we’ll ignore based on giving them a prefix of “Label_”, which was mentioned earlier:

Defining Event Handlers

WPF will send asynchronous signals (events) to our script when the GUI is acted upon.  Examples include when a selection is made, a button is clicked, and when the window is instantiated or destroyed. 

Defining event handler functions in our PowerShell lets us dictate our workflow:

• When the window is created, connect to vCenter, get a list of clusters, and populate the cluster ComboBox.
• When a selection is made in the cluster ComboBox, get a list of cluster VMs and populate the details DataGrid.
• When a VM selection is made in the DataGrid, enable the console Button.
• When the console Button is clicked, launch a vSphere Remote Console window.
• When the window is closed, disconnect from vCenter

Anything is possible, of course.  Here’s our event handlers:

Final Bits

There’s not much left!  Define the vCenter server against which we will operate and display the form:

Reveal

Run this little puppy and off we go.  Pick a cluster from the drop-down to see a list of VMs and click the column headers to if you don’t like it ordered by VM name.  Choose a VM and the console button becomes active so long as the VM state is “PoweredOn.”

What else?

• My error checking and validations are non-existent--Add some.
• Add another button to perform another task.
• Change what VM properties appear in the DataGrid.
• Do something else and have fun with it!

Full Code

You can grab the full code from here.

Proper Attribution is Proper

• Stephen Owen’s XAML to GUI
• The Scripting Guys’ I've Got a PowerShell Secret: Adding a GUI to Scripts

Office 365 Staged Migrations with PowerShell

I have a love-hate relationship with PowerShell.  I want to love it and often do--except when my knowledge is lacking or it acts mysteriously (to me).
To aid my memory and perhaps help others, I've decided to post snippets as I continue my journey toward improving my PowerShell skills (I hear that shaking Luc Dekens' and Alan Renouf's hand at the same time increases one's PS Mojo).
With that out of the way, onward and upward.

---

We are in the midst of migrating from Exchange Server 2003 to Office 365 as a Staged Migration.  While not a particularly arduous task, it does involve pulling mailbox statistics and email addresses and using that information for the planning and creation of migration batches. It's tedious and error-prone.
Given a set of Exchange Servers, Domain Controllers, and a Universal Security Group, Get-MailboxSizes.ps1 generates a CSV containing mailbox names, server, size, number of items, office location, department, active mailbox flag, and string suitable for directly pasting into a Staged Migration CSV file.


The ActiveMbx column is TRUE if the user is not enabled or a member of a Universal Security Group used for tracking progress and automating other things.

The odd column header at the last column should be directly pasted into your Staged Migration CSV file as row 1.  For successive entries, copy that column value in filtered rows as you plan the migration.   Please note that this column assumes Active Directory Synchronization with Office 365.

We've been known to paste entries from the last column into Notepad for a quick Search-Replace of ",,FALSE" with ";".  That yields a list that you can paste into the To field of a notification email or add members box of the Universal Security Group.

I’d like to thank Iain Brighton who lent a hand when I was stuck with Custom Objects. Yea Twitter!

Capturing CPU Trends with PowerCLI

Inspired by the creeping CPU that we see in Linux guests and helped greatly by @BoerLowie at his blog, I’ve come up with a little PowerCLI to capture CPU trends of the top consumers per cluster.
This is my first cut and will likely see changes over time, like any script should. HTML output and emailed results are the most likely candidates.
The script should be fairly self-explanatory. For each cluster, traverse all VMs and get their OverallCpuUsage (the number that you see in the vSphere Client when selecting a cluster and then the Virtual Machines tab).  Take the top X consumers based on that number and get their average CPU usage performance statistic for N days back in time and compare it to today’s.
The output looks something like this:



So here you go:

#

#  Produce guest CPU trending from a time period back versus a shorter 

#  more immediate time frame.  e.g. 30 days ago versus past 2 days.

#

param(

    [string] $vCenter

)

$DaysOld = -30        # compare to full day stats this many days back

$DaysRecent = -1    # get stats for this many recent days.

$GetTop = 10        # look at top x CPU consumers

Add-PSSnapin VMware.VimAutomation.Core -ErrorAction SilentlyContinue

#if ($vCenter -eq "") {

#    $vCenter = Read-Host "VI Server: "

#}

#if ($DefaultVIServers.Count) {

#    Disconnect-VIServer -Server * -Force -Confirm:$false

#}

#Connect-VIServer $vCenter

$AllClusters = Get-Cluster

Foreach ($Cluster in $AllClusters) {

    Write-Host "`n$($Cluster.Name)"

    $VMs = Get-Cluster $Cluster | Get-VM | `

        Where-Object { $_.PowerState -eq "PoweredOn" }

    $NumVMs = $VMs.Count

    # Get the Overall CPU Usage for each VM in the cluster.  Then cap that 

    # list at the top $GetTop highest for Overall CPU Usage

    $vm_list = @()

    $Count = 0

    Foreach ($vm in $VMs)

    {

        $Count += 1

        Write-Progress -Activity "Getting VM views" -Status "Progress:" `

            -PercentComplete ($Count / $NumVMs * 100)

        # the vSphere .Net view object has the OverallCpuUsage 

        # (VirtualMachineQuickStats)

        # http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.vm.Summary.QuickStats.html

        $view = Get-View $vm

        $objOutput = "" | Select-Object VMName, CpuMhz

        $objOutput.VMName = $view.Name

        $objOutput.CpuMhz = $view.Summary.QuickStats.OverallCpuUsage

        $vm_list += $objOutput

    }

    # Reduce to our Top X

    $vm_list = $vm_list | sort-object CpuMhz -Descending | select -First $GetTop 

    #

    # For each of those VMs, get the statistics for past and current CPU usage

    $NumVMs = $vm_list.Count

    $Out_List = @()

    $Count = 0

    Foreach ($vm in $vm_list)

    {

        $Count += 1

        Write-Progress -Activity "Compiling CPU stats" -Status "Progress:" `

            -PercentComplete ($Count / $NumVMs * 100)

           [Double] $ldblPerfAged = (Get-Stat -Entity $vm.VMName -Stat cpu.usage.average `

            -Start $((Get-Date).AddDays($DaysOld)) `

            -Finish $((Get-Date).AddDays($DaysOld + 1)) -ErrorAction Continue | `

            Measure-Object -Average Value).Average

        If ($ldblPerfAged -gt 0) {

               [Double] $lblPerfNow = (Get-Stat -Entity $vm.VMName -Stat cpu.usage.average `

                -Start $((Get-Date).AddDays($DaysRecent)) `

                -ErrorAction Continue | Measure-Object -Average Value).Average

            [Int] $lintTrend = (($lblPerfNow - $ldblPerfAged) / $ldblPerfAged) * 100

            $objOutput = "" | Select-Object VMName, CpuMhz, PerfAged, PerfNow, Trend

            $objOutput.VMName = $vm.VMName

            $objOutput.CpuMhz = $vm.CpuMhz

            $objOutput.PerfAged = "{0:f2}%" -f $ldblPerfAged

            $objOutput.PerfNow = "{0:f2}%" -f $lblPerfNow

            $objOutput.Trend = "{0}%" -f $lintTrend

            $out_list += $objOutput

        }

    }

    # Spit 'er out

    Write-Host "Top CPU Consumers Trending, $($DaysOld) days vs today`n"

    $out_list | Format-Table -Property VMName, `

        @{Expression={$_.CpuMhz};Name='CPU Mhz';align='right'}, `

        @{Expression={$_.PerfAged};Name='CPU Aged';align='right'}, `

        @{Expression={$_.PerfNow};Name='CPU Now';align='right'}, `

        @{Expression={$_.Trend};Name='Trend';align='right'}

}