A common problem in virtualization is snapshots. The name “snapshot” makes us (novice or otherwise!) think of a picture in time, which sometimes leads to the belief that the snapshot is “taken” and then stored somewhere, though that’s not how snapshots really work.

In reality, snapshots create a psuedo-consistent state of the virtual disk at that point in time. Subsequent writes in a snapshotted state are redirected to delta files. If you are performing an upgrade, a snapshot is helpful, allowing you to restore the prior system state if there are problems. After a few days, the snapshot loses its value as a restore becomes increasingly unlikely because you would lose the application changes as well. Snapshots also play a role in backups, where they are used temporarily to provide the psuedo-consistent state for the backup utility before the snapshot is deleted.

When a snapshot is deleted, that delta is applied to the base virtual disk(s), playing back through the transactions. Large snapshots take a long time to delete and affect system performance until the consolidation is complete. They can also affect the VM during normal operation as the delta file size increases.

Because of this difference between what we picture and how snapshots work, it is not uncommon to find long-lived snapshots that are gigantic in size and have no actual benefit. Today, we’ll look at determining when you have a snapshot problem in your vSphere infrastructure and how to alert the owners.

PowerCLI to the rescue!

You can manage snapshots many ways, including by right clicking on a VM in the Web or Thick clients and looking at the snapshots for that VM. That’s fine if you have 2 or 3 VMs. Past that, you’ll find that is VERY slow. Let’s use PowerCLI instead. I’ve added a new cmdlet to my github PowerCLI modules repository called Send-SnapshotReports. This builds on some work by Alan Renouf, Luc Dekens, and some others (links at the end) who can do a much better job of explaining it than I can. Here’s the gist though:

• Use Get-VM and Get-Snapshots to pull the list of snapshots over a certain age.
• For each snapshot:
  • Grab some extra info about the snapshot, specifically who created it.
  • Attempt to link the owner to an email address.
  • Email that person and tell them they are horrible and they should feel horrible.

There’s a bit more to it if you want to pick at it, or provide some enhancements. Use Get-Help to get an accurate description of all the CLI arguments and some of the caveats. Create a snapshot of a VM and run the following command, filling in your information as needed. Keep in mind that you will need to connect to your vSphere server first.

PS C:\> Send-SnapshotReports -MailFrom rnelson0@gmail.com -MailDefault rnelson0@gmail.com -SMTPRelay relay.example.com -Retention 0 -LookupOwner

This will email the owner, or me, from my email address for all snapshots over 0 days in age (default: 14). You’ll need a valid smtp relay as well (enhancements needed: SMTP on non-standard ports, TLS). This will work great if you think you have a snapshot program. If you don’t think you have a snapshot problem, or you’re lazy, or on vacation, it doesn’t work very well. It would be best if we could schedule this cmdlet to be run on a frequent basis. We can schedule PowerCLI tasks, but it’s slightly tricky.

Scheduling a PowerCLI Task

Scheduling a task is fairly easy and, thanks to Magnus Andersson, there’s a graphical walkthrough to the process. His walkthrough is pretty good but I ran into a few problems up front. I’ll describe the whole process, but you can reference the walkthrough if you need a visual aid.

The first thing you will need is a place to run the task. I’m using a Windows 2008R2 RDP server that I use to manage my network. I imagine this would look very similar in 2012, but you may find some slight variance. The second thing is that you’ll need an account to run this as. I would recommend a designated service account rather than an individual account, so that it continues to run if your account expires, is suspended, or you leave the job. The next guy after you will appreciate it! However… I have encountered some permissions issues with the files we will create below, so I’ll show you this process with an interactive login. If your service account doesn’t have rights for interactive login, you will have to do some legwork to iron out permission issues.

Log into the RDP server as the account that will run the PowerCLI task. Load the PowerCLI-Administrator-Cmdlets module from GitHub, or you won’t get far. We’re going to need to store credentials for the service account. Fire up Powershell (ISE) and run the following commands:

$Cred = Get-Credential
$Cred.Password | ConvertFrom-SecureString | Set-Content C:\Users\ServiceAccount\Documents\WindowsPowerShell\vspherelocal.cred

The first command will open a dialog box where you can enter credentials for a vSphere login that has the necessary rights to view snapshots and tasks. Enter the username and password and hit enter. I’ve used the administrator@vsphere.local account. The second command will store the password as a securestring in the specified location. While a securestring is reasonably secure, don’t tempt fate, make sure it’s stored in a secure location.

Next, create a Powershell script (.ps1) in the user’s Documents\WindowsPowershell directory. Substitute the appropriate values for your environment below:

$Username = 'administrator@vsphere.local'
$Password = Get-Content 'C:\Users\ServiceAccount\Documents\WindowsPowerShell\vspherelocal.cred' | ConvertTo-SecureString
$Email = 'rnelson0@gmail.com'
$SMTPRelay = 'relay.example.com'
$vCenter = 'vcenter.nelson.va' 

$Credential = New-Object System.Management.Automation.PSCredential -ArgumentList $Username, $Password

Send-SnapshotReports -vSphereServer $vCenter -Credential $Credential -MailFrom $Email -MailDefault $Email -SMTPRelay $SMTPRelay -LookupOwner -Retention 7

We set some variables, including the password by reading the securestring on disk. Next we create a proper credential out of the username/password tuple. Finally, we call the cmdlet Send-SnapshotReports with a few extra arguments than we did before. The first two, vSphereServer and Credential, should be used together and are intended for interactive shells, where the user will not be authenticated to the vSphere server prior to calling the cmdlet. We also overrode the default 14 day retention timeframe and used a 7 day timeframe.

Run this script in PowerShell (ISE) and ensure everything works fine. PowerShell can be really picky about the permissions on the credential file and it’s easier to debug in an interactive shell. Assuming everything works, we just need to schedule it.

Flip back to Magnus’s tutorial and schedule the task. This process is accurate, but there is one tweak since we’re using a cmdlet. When adding arguments, all you need is the command, not the psconsolefile. I.e. “-Command “&{C:\Users\ServiceAccount\Documents\WindowsPowerShell\SnapshotReminder.ps1}”. Choose an appropriate interval. I’ve chosen daily, all days of the week, a happy medium between annoying people about snapshots and becoming so noisy they’ll disregard the emails entirely. Once the task is scheduled, right click on it to run it immediately. You should get another set of emails.

If, for some reason, you did not, it’s a real pain to track down where things are going wrong. The task’s history will likely say it completed successfully, because it successfully called Powershell with the correct arguments. We don’t write any output, so there’s nothing to see. We can add some output, though. Add this line to the top of your script:

Start-Transcript C:\snapshots.txt

Run the task again. All output, including errors, will be written to C:\snapshots.txt (assuming the account has permissions, so choose a location it can write!). When I ran into problems with permissions, it showed up in this file:

Get-Content : Access to the path 'C:\Users\ServiceAccount\Documents\WindowsPowerShell\vspherelocal.cred' is denied.

Once you iron out any kinks, you’ll have your daily reminder in place, and you can run it manually if necessary. Enjoy!

Links

• Thanks to JK-47 (blog) for sanity checking.
• Alan’s PowerCLI Snap Reminder post.
• LucD’s limitations on collectors.
• Dave Garnar’s horrible purple webpage describing snapshot ownership determination.