Here is a short post on how we built HOL-SDC-1416!
The Hands on Lab build and devlopment process has improved over the years that labs have been created and run at VMworld. I have heard stories from my fellow SEs how, back in the day, they would rock up at VMworld with racks of kit and build up the labs with their bare hands on bare metal the night before the show opened! They always got it up an running on time they tell me 🙂
Over the last few years the labs have become more and more sophisticated and are now available online 365 days a year and 24 hours a day.
To that end the development process has been massively refined and is very slick. As a new Lab Captain this has really helped out.
All the development is done in a VDC cloud devoted to HoL development. The core HoL team have populated the HoL Dev catalogues with stacks and stacks of template Pods and media that the development teams can use. There are close on 50 labs, so I guess 200-300 modules so having these raw materials saves a massive amount of time.
The plan was to stand up some vSphere infrastucture, run some SQL and Exchange workload and monitor with vCOps. We would call this our Collector Core and at some point would simulate some ‘bad stuff’ happening in the lab. We would then make a copy of the lab and put vCOps into historical view mode. The un-needed resources would then be deleted leaving just the vCOps VMs and ControlCenter on which the lab itself is actually operated. We’d leave the Collector Core running in case we wanted to do a V2 of the lab later in the year.
For my lab I picked a single site base Pod our of the catalouge and then added a vCOps instance the core team had created. The base Pod had everything I needed – vCenter, ESXi hosts, AD, DNS, Networks, Storage etc etc All the latest code too.
At that point it became clear there was way more to do than I had anticipated and I was running out of bandwidth so my colleague, Michael Armstrong, joined the team. He looked after building the SQL and Exchange environments out and looking after the whole lot while I swanned away on holiday! You can see Michael’s blog at http://www.m80arm.co.uk/
We had to let this run for a few weeks so that vCOps could baseline the systems and start providing meaningful data. While that was happening we started thinking about how we could simulate some load and I started working on the process for ‘cutting the lab’ into historical mode and publishing it. I also added a few objects into the vCOps environment that would be needed for the Lab Scenarios – a Super Metric, some groups and other stuff that would help with the lab flow. We also integrated VIN and Hyperic as the lab was Applicaiton focussed.
D-Day
June 25th was the day we chose to run the simulated workload and freeze the lab. It took a few hours to get things right but eventually we had everything running how wanted.
So at 11:01 (not 11:00, not 11:02 !) we had both the SQL and Exchange services with errors representing the scenarios the lab manual would reference.
Cut the Lab
I then made a copy of the lab. The process was quite laborious. Along the lines of
- Cleanly shut down Collector core
- Make an identical copy
- Restart Collector Core and all the supporting SQL and Exchange in case we needed to recut
- Start the copy
- Delete the vSphere Hosts (including SQL and Exchange) – we had to make this environment as small as possible
- Configure vCOps into historical mode and stop its collector services etc
- Do lots of tidying up – clear browser caches, unhook networks, delete temp files, tidy up bookmarks, check certificates, change screen resolution, check license, remove VPN code…and about 100 other things.
- Change the Powershell script to check the lab is fully functional (if you do the lab you can take a look at C:\hol to see some of the stuff we do under the hood the make things work well)
The final Pod ended up looking like the above. I had hoped to remove vcsa-01a (the vCenter) but couldn’t work through a startup licensing issue in time so it is there simply to make sure the vCOps appliance thinks it is licensed on startup! Thankfully the core team were OK with that even though my Pod was bigger than I told them it was going to be. If we do a V2 later in the year I may try again to remove it.
Once I did some basic functional testing to make sure the lab would flow as designed I published the lab into the staging catalogue ready to pushed into the HoL cloud. That was it, the lab was finished from a technical and build point of view!!
Manual Development
This turned out to be MUCH harder than I thought it would be!!
Thankfully, again, the HoL Core team made things as easy as they could – we use a product called Screensteps to build the content. It’s a really simple but powerful tool. It was particularly good for capturing the screenshots. It could also be used offline and online so I did quite a lot of writing whilst on trains etc!
Once written the Manual was tested, proof read and went through a peer review. I also had to Americanise the spelling (or should that be Americanize!). Thankfully this blog is mine so I can use the Queen’s English and spell things properly and how I like 🙂
Lab Testing
In order to check the HoL cloud can support VMworld we do a number of internal testing events in the run up to VMworld. At this week’s event we had several hundered people from VMware around the world logged into the VMworld HoL environment to do load testing and check all the labs work well. All went well but I guess we’ll really find out when we open the doors at 9am on the 24th August!!!
HOL-SDC-1416 worked well 🙂
The next article will talk a bit more about how we support Hands On Labs at VMworld – for now thanks to everyone who helped! If you’ve come to this blog post as a result of using the QR code in the Lab Manual I hope you enjoyed the lab and, more importantly, found it useful.
CloudyTechnologist 