Cloning VM’s with VirtualBox, LVM and Snapshots!

Recently I wanted to setup a development environment to test some Active Directory setups. Basically I wanted a whole stack of domain controllers and client machines across multiple sites, domains and forests to see how they tick and replicate between each other. Easy enough to do, just create a bunch of virtual machines using your favourite virtualization package, I prefer VirtualBox but you could do the same with VirtualPC or VMWare. But say you want to quickly deploy this and be able to easily tear down machines and bring new ones up, again easy enough just clone the disks and create new VM’s. All pretty straight forward. However there is a much neater and more elegant way of doing this which allows you to quickly snapshot a VM and redeploy in a matter of a seconds rather than minutes to hours copying full disk images around.

So how does one accomplish this task, well it’s really quite simple, LVM snapshots. If you have never heard or used LVM before it is a logical volume manager. It’s job is to take a bunch of physical volumes such as hard disks or raid arrays and combine them into a storage group. From this storage group you can then create multiple logical volumes (similar to luns in the Netapp world) which act like standard disk devices allowing you to easily carve up your storage in a more logical way. Now LVM is really quite a powerful tool and has many extra features which are outside of the scope of this article but see my links at the end to find out more on the specifics.

Anyway the feature we are going to use as I already mentioned is snapshots, what these are is an instantaneous snapshot of a volume which we can then mount separate to the real volume and use as a normal disk. The beauty of this is that your original volume remains completely untouched and any changes that are written to the snapshot are stored as just that changes or differences between that and the original volume. This means for our VM’s that only a few hundred meg or so will change that our disk usage is significantly reduced. You can see now just how useful this can be. The other big advantage is that they are fast, you can create a snapshot in a matter of seconds whereas to copy a vmdk or vdi file it takes a lot longer for it to copy, especially if using a dynamic disk as it locates all the used blocks.

Now lets get down to using it in the real world.

First things first get your VM installed using a raw device mapping.

lvcreate -L20G -nvbox_win2003std_base storage /dev/md2
Logical volume “vbox_win2003_base” created

In this instance I have created a 20gb volume called “vbox_win2003_base” on my storage volume group and forced it to allocate all its extents on /dev/md2. I prefer to do this just to make sure no logical volumes cross over physical volumes unless absolutely needed.

Now you have your volume ready you will need to create a disk for VirtualBox to use.

VBoxManage internalcommands createrawvmdk -filename “win2003_base.vmdk” -rawdisk /dev/storage/vbox_win2003std_base -register
VirtualBox Command Line Management Interface Version 2.2.2
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

RAW host disk access VMDK file win2003_base.vmdk created successfully.

This creates a vmdk file with a raw mapping to the logical volume and automatically registers it for use within VirtualBox.

Now prepare your VM like you would any other Windows machine being cloned. Preinstall all your OS, applications, software, any patches or tweaks and get it configured just the way you like. However try to keep things reasonably simple, this means don’t promote the machine to a DC or any other service which when cloned will break. For my builds I have been testing with Windows 2003 Standard edition. So this means you need to extract the deploy.cab from your Windows 2003 CD to c:\sysprep and run the setupmgr.exe app to generate a sysprep.inf. If you want to avoid problems don’t set a password for the admin account in the sysprep if you have already set one for the machine as it appears to fail when updating it. So you have generated your sysprep.inf, do a quick check to ensure everything is in it’s place and run sysprep.exe –reseal –mini –pnp. The machine will do its bit and then shut down meaning it is ready to be cloned.

Now the fun part, time to make a snapshot of your logical volume.

lvcreate -L5G -s -nvbox_vodka_dc /dev/storage/vbox_win2003std_base
Logical volume “vbox_vodka_dc” created

Now in the blink of an eye it has created a snapshot of your base VM’s logical volume with a maximum growth of 5gb. This means the VM can have a maximum of 5gb of changes before it runs out of space. You will need to fine tune this depending on your VM’s and workload. To keep an eye on it simply use lvdisplay as per normal.

lvdisplay storage/vbox_vodka_dc
— Logical volume —
LV Name                /dev/storage/vbox_vodka_dc
VG Name                storage
LV UUID                IOFUF4-Bxze-Xpij-Qd5b-nH7k-FeEx-k6936B
LV Write Access        read/write
LV snapshot status     active destination for /dev/storage/vbox_win2003std_base
LV Status              available
# open                 0
LV Size                20.00 GB
Current LE             5120
COW-table size         5.00 GB
COW-table LE           1280
Allocated to snapshot  0.00%
Snapshot chunk size    4.00 KB
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:4

Now that we have the snapshot volume all we need to do is create another raw device vmdk.

VBoxManage internalcommands createrawvmdk -filename “Vodka_DC.vmdk” -rawdisk /dev/storage/vbox_vodka_dc
VirtualBox Command Line Management Interface Version 2.2.2
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

RAW host disk access VMDK file Vodka_DC.vmdk created successfully.

And that’s it, you now have a complete clone of that machine ready to load into a new VM within VirtualBox and boot. It will run through the mini-setup as per normal for any cloned machine and that’s it. All done!

With a few batch commands you can quickly automate the creating of a whole fleet of VM’s ready to plug into your dev environment.

Hope this helps a few people out there

Links:

• LVM (wiki)
  
• VirtualBox (wiki)
  

Mmmmmm Beeeer…

Wow it has been a while since I have posted… an in that time I’ve been coding, building a house and brewing some beer For a first time brewer i’ve posted up the basic steps required to get started as it had been a little difficult to find just simple pics and descriptions when I started.

 

This is how you start out with the fermenter ready to go and sanitized! I can not stress this enough SANITISE EVERYTHING that will come in contact with the mix.

And your basic ingredients… time, dextrose, yeast and the magic mix (which you can also make yourself). You may notice the pager so yes I was on call and since I can’t drink on call it was the perfect time to brew. This time I am brewing Ginger Beer which should be nice on a hot sunny day.

This is the mix which you heat up in some hot water before hand then mix with hot water to make it dissolve.

At this stage you have a couple litres of hot liquid you start to mix in your dextrose. Make sure you thoroughly mix this through.

Now at this stage you are basically filling it up till your defined level which in my case was 23L. Make sure if you can to use filtered or boiled (and cooled) water.

  

And now your done you seal it up put water in your air lock (filtered) and then let her sit for a week. I would suggest somewhere that doesn’t get too hot as you want it to stay within your temperature range as much as possible.

 

I’ll do up a post when I am bottling it which should be in a few days hopefully. Next batch will be a keg

0.0.5 Release of CactiWMI

It’s been a while since I released an update and there have been a number of templates since 0.0.4 so packaged them up ready for consumption

Below are some examples of what we now have!

So we have some standard Windows stats such as system calls, context switches and process/processor stats.

We can now also monitor processes themselves on an individual basis.

And some new Exchange graphs including the much requested SMTP graphs.

And here is the download. Any questions or suggestions let me know and feel free to stop by the thread on the Cacti forums (Link!).

  CactiWMI-0.0.5.r50.tar.gz (53.7 KiB, 1,019 hits)

Painting the 300ZX

So I have finally started painting the Zed and figured I would give a bit of an update seeing as after the site redesign I didn’t port any of the old posts across.

Picked up a stack of supplies.

 

Hooked up the air compressor.

 

Got out the sander.

 

And got to work!

 

Then with the help of the big boy 2 stage filtration setup.

 

Put down a nice coat of etching primer.

 

The next stage will be lightly sanding the etching primer and then lay down a few coats of high build primer. Once that’s done I can start blocking it back and get it nice and smooth before the colour and clear.

New release with Exchange Monitoring!

It’s been a little while since I have released anything but after a bit of work I have a new version which now supports monitoring Exchange! Previously I had monitored Exchange myself via WMI using a custom PHP script but now this can all be done with a reasonably straight forward set of templates and the generic wmi.php

Aside from just supporting Exchange here are the changes below:

• Now parses out spaces and replaces with an underscore (needed for anything that has a name with spaces in it)
• Included new debug mode which logs detailed information to a per host text file
• Code cleanup, should make it a bit easier to configure
• General tweaking

The biggest issue with Exchange that I encountered was that the store names could have spaces which would throw off Cacti when it interpreted the data being passed to it. But as you can see from below the changes have fixed this issue and allow for some useful monitoring.

The templates are included in the attached tar file, please note however that the active client logons will need tweaking to suit your system. For me I created two graphs based on the template then removed the template so that the name of each store could be displayed on the graph. You can do this or alternatively do per store graphs however I found it to be overkill and used two graphs of 5 stores each.

The new debug mode logs by default to /tmp but you can adjust this to wherever you like so long as it has the correct permissions. Effectively what you will end up with is a log file per host of the filename dbug_xx.xx.xx.xx where the x’s are your IP. Inside it will contain basically all the variables being passed in and out as well as the direct output from wmic etc. With this you should be able to track whats going on and debug any issues.

So there you have it everything you need to get started. I will have some additional Exchange templates over the next few weeks so stay tuned! For now if you need help leave a message or drop by the Cacti forums. Also one last note you can browse the latest code and check out any build you like from the web based Subversion repository. Check it out at http://svn.parkingdenied.com/

  CactiWMI-0.0.4.r45.tar.gz (39.7 KiB, 417 hits)