VMworld 2009 - specifics

Now to the meat and potatoes (what does that really mean?).

VCO (vCenter Orchestrator) and LCM (Life Cycle Management). This information comes from the LCM Instructor led Lab and the VM2120 session.

• Orchestrator is (one of) the product(s) that VMware got when they acquired Dunes a couple of years ago.
• It would seem that Orchestrator could be used as a general purpose workflow engine, but VMware has bundled it with vCenter 4.0. It includes the 'plug-ins' (I'm not sure if that's the correct terminology) for Active Directory and SSH (Secure SHell) with the potential to add plug-ins for other applications such as EMC Navisphere, HP Remedy, Microsoft Sharepoint, etc.
• The current 1.0.x version are written for the VI3 APIs and that version 1.1 will include the vSphere 4 APIs.
• LCM is a pre-assembled 'package' of Orchestrator logic to handle the provisioning, change management, and deprovisioning of virtual machines. Depending on how much LCM deviates from your current business requirements will determine if it is easier to modify LCM or create a new workflow in Orchestrator. The VM2120 session had a detailed breakdown of this analysis.
• Products such as SRM (Site Recovery Manager) do not leverage Orchestrator, but conceivably SRM could be (re-)written to be an Orchestrator package.

Performance. Most of this is from the Performance and Troubleshooting Instructor led Lab and the Performance Best Practices XenApp breakout sessions.

• Unfortunately, all of the participant spots for the Performance and Troubleshooting lab were full so I was relegated to spectator status.This was not such a bad thing as I got a chance to talk to Irfan Ahmad who is the author of the vscsiStats tool. He was able to confirm that vscsiStatis is *not* available for any version of ESXi, but that he has suggested that the functionality be added directly to vCenter.
• Based on what I saw of the Lab the vscsiStats tool is critical in determining storage bottle-necks, whether its between the virtual machine and ESX, or between ESX and the storage system.
• Lots of other good stuff was covered, some of which I knew already, but most of it new. I got a printout of the presentation.
• It seems that a lot of performance issues start with storage. I suspect this is due to the fact that a lot of VI3/vSphere administrators such as myself are new to dealing with massive amounts of shared storage such as what you find in a SAN.
• Designing, testing and verifying are crucial to implementing a proper vSphere environment, but one thing that was emphasized in the Citrix XenApp (formerly Presentation Manager) presentation is to test using your actual applications under actual workloads. The example repeated during the presentation was that the usual Terminal Services/XenApp benchmarks use Office. What if the application you are deploying is not Office? Then the results you get are going to be different.
• From the sessions I attended it seems that the mantra from VMware when virtualizing applications is to scale-out rather than scale-up. Even though ESX/ESXi 4.0 is capable of scaling up virtual machines, the preference is to create more smaller virtual machines. This is due in great part to the fact that the scheduling of multiple threads/process does not scale linearly in the x86 world for both Windows and Linux. By contrast, I have seen comparisons to Solaris on SPARC hardware with near linear scaling for up to 64 processors. The XenApp presenation made the point that the recommended virtual machine size could be smaller than the bare metal solution it is replacing, but that utlimately you will end up with more users per physical server. This is similar in logic to IBM's configuration of one thier x3850M2 systems with and without ESX 3.5. The result is that ESX allows the same physical server to support more users.

Upgrading. This is from the Things to Consider When Upgrading and Monitoring Hardware with ESXi breakout sessions.

• My biggest concern(s) with moving from ESX to ESXi is the lose of functionality due to the lack of HMA (hardware management agent) such as Dell OpenManage or IBM Director. I have seen the hardware monitoring capability built in to ESXi 3.5 and ESXi 4.0 first hand to know that simple things like a failed (pulled) HDD or failed (pulled) power supply are detected and reported to vCenter. I always wonder how complete the CIM monitoring capabilites are from the hardware vendors -- is it as good as thier native HMA? The bigger issue is the inability to remotely update system firmware such as BIOS, BMC, FC HBA, etc. See http://communities.vmware.com/thread/228037. I have heard from hardware vendors that they are still waiting on VMware to add the necessary capabilities to ESXi to allow them to do remote firmware updates. After the Kroger presentation on thier roll-out of ESX 3.5 I asked them if lack of the ability to perform remote firmware updates would impact thier decision to stay with ESX vs. ESXi and the answer was yes. But it seems that this issue has not escaped VMware as one of thier people hinted that this capability would be added *soon*.
• Rolling upgrade of ESX 3.x to ESX 4.0 was not a problem. My concern here was that in my early ESX 3.0.x days we had a cluster with both 3.0.0 and 3.0.1 and had some problems that cleared up once everything was at the same version. Ever since I always configure all hosts in the same cluster to have the same minor version and patch level for both ESX and firmware. This has not prevented all problems, but I'm sure it has reduced them.
• I had a discussion with another attendee who had a mixed 3.0.2 and 3.5 environment and was debating on whether to upgrade the 3.0.2 hosts to 3.0.3 (EOS in 2011) or upgrade to 3.5. I asked why not upgrade to ESX 4.0? His response was that they did not feel ready. So I suggested that they at least upgrade vCenter to version 4.0 and upgrade to ESX 3.0.3 as this would give them exposure to the new version of vCenter and allow them to use Update Manager to patch all of thier hosts.

Powershell. This is from the PowerShell instructor led lab and the Managing VMware with Powershell breakout session.

• First and foremost, Powershell is wickedly cool. With all of the plugins that vendors, not just Microsoft and VMware, are creating its obvious that Powershell will become the automation tool of choice to replace CMD or VBscript scripts.
• Learn Powershell.
• In the Managing VMware with Powershell session I learned about a couple of tools that should come in handy. The first one is VESI Virtualization EcoShell (http://www.thevesi.org/) which gives you the ability to perform administrative activities and then allows you to copy-and-paste the necessary Powershell code to script those same activies. The other tool (I don't remember the name) acts as a proxy between a vSphere Client and an ESX/ESXi host or vCenter management Server to also generate Powershell code for your administrative tasks. The difference is that this tool will generate the necessary code for the API calls even if the appropriate Powershell plugin is not available. The VESI tool by contract requires a couple of Powershell plug-ins such as the Active Directory plugin from Quest and PowerCLI  from VMware.
• One of the coolest demonstrations of the entire VMworld conference was watching VESI generate a Visio diagram of a simple vSphere environment in real time. You could actually follow the adding and moving of objects in the Visio application. You can see a video of this on the download page under "Video Library".
• I really need to get on that Powershell thing.

VMworld 2009 - general

My thoughts on VMworld 2009 in San Francisco.

The Moscone North and South are a decent size, even more so when you add the West building (which VMworld did not occupy). I did get a little confusing remembering if I was in the North building or South building and where I needed to go for my next session.

There were lots of sessions (Breakout, Super, BoF, Labs, self paced Labs). Too many sessions to squeeze into four days. I wish they made the days longer and added a fith day. I had to skip some important sessions to attend even more important sessions. Yeah, I know, they will all be available online soon. The whole point of attending is being able to ask your question(s) and talk to people.

The little book of session times & locations and maps included with the badge holder was nice, but incomplete. The BoF (bird of a feather) sessions were not listed and only 2/3 to 3/4 of the Breakout sessions were listed.

It seems that just because a session was marked as 'full' on the VMworld web site did not mean that you couldn't get in. I guess people were signing up for everything in sight and then cherry picked once they got there. This worked to my favour as I got into some of these full sessions.

The Instructor led Labs for me turned out OK. I participated in the LCM and PowerShell Labs and spectated the Performance and Troubleshooting lab. There were no crashes or other problems.

The show floor seemed nice. I took me an hour just to walk up and down all of the aisles with only stops at the VMware and Dell booths. Unfortunately, by the time I had a couple of free hours again on Thursday most of it had already been dismantled. See comment above about squeezing too much into four days.

A couple of nice spots to eat are Little Joe's (http://www.little-joes.com/) at 85 5th St & Mission St, and Pier Market (http://www.piermarket.com/) on Pier 39.

Lessons learned.

So, I went to VMworld this year in San Franciso with a few days tacked on to the end for sight seeing plus a three day stayover in Vancouver, and learned a few things about traveling. This is the first time traveling back to the United States in twenty years, and the first time traveling by air in ten years. Researching on the TSA web site (tsa.gov) certainly helped, but other things you just need to learn first hand.

Make sure you register early so that you can get a hotel close to the event. Due to delays out of my control I ended up on 8th Street which is almost a mile (1.6 km) from the Moscone Center. While the walk did me good, the neighbourhood seemed to be a bit 'forgotten' by the rest of the city. The folks on the street seemed harmless enough I did not feel completely at ease.

When confirming that your hotel has Internet service it would probably be a good idea to see if they mean wired or wireless. I'm an really paranoid about wireless security so I brought a laptop with no wireless hardware with the intention of using a standard Ethernet cable --- no such luck. I bought a Linksys USB wireless adapter at OfficeDepot but the setup did not complete successfully. I attribute the setup failure to our standard corporate laptop image which is locked down pretty tight. Even with local Administrator access I could not get the USB wireless adapter working. I did a fresh install of Windows XP Professional with Serivce Pack 3 slipstreamed and the setup completed successfully.

Be wary of vendors that hand out metal drinking bottles. EMC was nice enough to sponser a aluminum drink bottle in the backpack bundle availabe to attendees, but I don't think anyone at VMworld considered airport security.

Get to know the public transit. This seems kinda obvious, but it was not until the conference was over that I got a 3-day Muni pass. This made getting around San Francisco easier. While in Vancouver, the BC Translink site was down on the Sunday which made trying to getting scheduling information difficult. It turns out that he transit information that Google Maps uses is actually reliable, for Vancouver at least.

Keep an eye on the weather. Again, really obvious, but while meandering my way to the Presideo to see the Golden Gate bridge I spent too much time in Beuna Vista park and Haight Astbury so that by the time I got there the fog had rolled in. I have some terrific pictures of the bridge, but only as far as the first support tower.

This brings me to my final lesson learned. Again, super obvious, but I must have been getting tired when I left for Stanley Park in Vancouver on my last day and forgot my camera.

Well, at least I didn't lose my luggage.