VMworld 2009 - specifics

Now to the meat and potatoes (what does that really mean?).

VCO (vCenter Orchestrator) and LCM (Life Cycle Management). This information comes from the LCM Instructor led Lab and the VM2120 session.

• Orchestrator is (one of) the product(s) that VMware got when they acquired Dunes a couple of years ago.
• It would seem that Orchestrator could be used as a general purpose workflow engine, but VMware has bundled it with vCenter 4.0. It includes the 'plug-ins' (I'm not sure if that's the correct terminology) for Active Directory and SSH (Secure SHell) with the potential to add plug-ins for other applications such as EMC Navisphere, HP Remedy, Microsoft Sharepoint, etc.
• The current 1.0.x version are written for the VI3 APIs and that version 1.1 will include the vSphere 4 APIs.
• LCM is a pre-assembled 'package' of Orchestrator logic to handle the provisioning, change management, and deprovisioning of virtual machines. Depending on how much LCM deviates from your current business requirements will determine if it is easier to modify LCM or create a new workflow in Orchestrator. The VM2120 session had a detailed breakdown of this analysis.
• Products such as SRM (Site Recovery Manager) do not leverage Orchestrator, but conceivably SRM could be (re-)written to be an Orchestrator package.

Performance. Most of this is from the Performance and Troubleshooting Instructor led Lab and the Performance Best Practices XenApp breakout sessions.

• Unfortunately, all of the participant spots for the Performance and Troubleshooting lab were full so I was relegated to spectator status.This was not such a bad thing as I got a chance to talk to Irfan Ahmad who is the author of the vscsiStats tool. He was able to confirm that vscsiStatis is *not* available for any version of ESXi, but that he has suggested that the functionality be added directly to vCenter.
• Based on what I saw of the Lab the vscsiStats tool is critical in determining storage bottle-necks, whether its between the virtual machine and ESX, or between ESX and the storage system.
• Lots of other good stuff was covered, some of which I knew already, but most of it new. I got a printout of the presentation.
• It seems that a lot of performance issues start with storage. I suspect this is due to the fact that a lot of VI3/vSphere administrators such as myself are new to dealing with massive amounts of shared storage such as what you find in a SAN.
• Designing, testing and verifying are crucial to implementing a proper vSphere environment, but one thing that was emphasized in the Citrix XenApp (formerly Presentation Manager) presentation is to test using your actual applications under actual workloads. The example repeated during the presentation was that the usual Terminal Services/XenApp benchmarks use Office. What if the application you are deploying is not Office? Then the results you get are going to be different.
• From the sessions I attended it seems that the mantra from VMware when virtualizing applications is to scale-out rather than scale-up. Even though ESX/ESXi 4.0 is capable of scaling up virtual machines, the preference is to create more smaller virtual machines. This is due in great part to the fact that the scheduling of multiple threads/process does not scale linearly in the x86 world for both Windows and Linux. By contrast, I have seen comparisons to Solaris on SPARC hardware with near linear scaling for up to 64 processors. The XenApp presenation made the point that the recommended virtual machine size could be smaller than the bare metal solution it is replacing, but that utlimately you will end up with more users per physical server. This is similar in logic to IBM's configuration of one thier x3850M2 systems with and without ESX 3.5. The result is that ESX allows the same physical server to support more users.

Upgrading. This is from the Things to Consider When Upgrading and Monitoring Hardware with ESXi breakout sessions.

• My biggest concern(s) with moving from ESX to ESXi is the lose of functionality due to the lack of HMA (hardware management agent) such as Dell OpenManage or IBM Director. I have seen the hardware monitoring capability built in to ESXi 3.5 and ESXi 4.0 first hand to know that simple things like a failed (pulled) HDD or failed (pulled) power supply are detected and reported to vCenter. I always wonder how complete the CIM monitoring capabilites are from the hardware vendors -- is it as good as thier native HMA? The bigger issue is the inability to remotely update system firmware such as BIOS, BMC, FC HBA, etc. See http://communities.vmware.com/thread/228037. I have heard from hardware vendors that they are still waiting on VMware to add the necessary capabilities to ESXi to allow them to do remote firmware updates. After the Kroger presentation on thier roll-out of ESX 3.5 I asked them if lack of the ability to perform remote firmware updates would impact thier decision to stay with ESX vs. ESXi and the answer was yes. But it seems that this issue has not escaped VMware as one of thier people hinted that this capability would be added *soon*.
• Rolling upgrade of ESX 3.x to ESX 4.0 was not a problem. My concern here was that in my early ESX 3.0.x days we had a cluster with both 3.0.0 and 3.0.1 and had some problems that cleared up once everything was at the same version. Ever since I always configure all hosts in the same cluster to have the same minor version and patch level for both ESX and firmware. This has not prevented all problems, but I'm sure it has reduced them.
• I had a discussion with another attendee who had a mixed 3.0.2 and 3.5 environment and was debating on whether to upgrade the 3.0.2 hosts to 3.0.3 (EOS in 2011) or upgrade to 3.5. I asked why not upgrade to ESX 4.0? His response was that they did not feel ready. So I suggested that they at least upgrade vCenter to version 4.0 and upgrade to ESX 3.0.3 as this would give them exposure to the new version of vCenter and allow them to use Update Manager to patch all of thier hosts.

Powershell. This is from the PowerShell instructor led lab and the Managing VMware with Powershell breakout session.

• First and foremost, Powershell is wickedly cool. With all of the plugins that vendors, not just Microsoft and VMware, are creating its obvious that Powershell will become the automation tool of choice to replace CMD or VBscript scripts.
• Learn Powershell.
• In the Managing VMware with Powershell session I learned about a couple of tools that should come in handy. The first one is VESI Virtualization EcoShell (http://www.thevesi.org/) which gives you the ability to perform administrative activities and then allows you to copy-and-paste the necessary Powershell code to script those same activies. The other tool (I don't remember the name) acts as a proxy between a vSphere Client and an ESX/ESXi host or vCenter management Server to also generate Powershell code for your administrative tasks. The difference is that this tool will generate the necessary code for the API calls even if the appropriate Powershell plugin is not available. The VESI tool by contract requires a couple of Powershell plug-ins such as the Active Directory plugin from Quest and PowerCLI  from VMware.
• One of the coolest demonstrations of the entire VMworld conference was watching VESI generate a Visio diagram of a simple vSphere environment in real time. You could actually follow the adding and moving of objects in the Visio application. You can see a video of this on the download page under "Video Library".
• I really need to get on that Powershell thing.