Enable ssh on ESXi 3.5

First ESXi shell access;

How to get shell access to your VMWare ESXi server?
By default the VMWare ESXi server don’t offer shell access (through the menu).
But by doing the following, you will be able to access the shell (although its limited):

1. Hook up a screen and a keyboard to the VMWae ESXi server.
2. Press “Alt+F1″
3. Write “unsupported” and press Enter (if you screw up, and think you’ve typed something wrong,
you can use Ctrl+U to clear the input field).
4. Enter the root password, an voila, shell access.

Then enable ssh on ESXi 3.5

By default, ssh access is disabled on VMware ESXi 3.5, so how do i enable ssh on VMware ESXi 3.5?

1. Type “vi /etc/inetd.conf” and press “enter”.

2. Locate the line that starts with “#ssh     stream  tcp     nowait  root    /sbin/dropbearmulti…….”

3. Move the marker over the “#” and press “x”.

4. Press “Escape” and write “:wq”, then press “enter”.

5. Type “/sbin/services.sh restart” and press “Enter”. Note: If you are running ESXi 3.5 Update 2, the services.sh no longer restarts the inetd process. You will have to manually kill the inetd process, in order to restart it and enable ssh access without a reboot.  Type “ps | grep inetd” and press “enter”. You will then see something similiar to “1289 1289 busybox              inetd”. Then write “kill -HUP 1289″, and remember to write the number “ps | grep inetd” returns to you!

Best practices for deploying Citrix on vmware ESX

First and foremost: this tuning list is my own experience and the experience of several users on the VMware forum . Your mileage may vary.

The goods:

  • Virtual Infrastructure 3
  • Windows 2003 Std (or Enterprise) Edition R2 (x86, not x64)
  • Citrix Presentation Server 4.0 (yes, I know, the old one ;))

The tips:

  • First this: it all depends on the applications used! Context switches is the key here…
  • Use Windows 2003 , not Windows 2000
  • Don’t P2V your servers, but use clean templates
  • Make sure the correct HAL (single or multi) is installed in the virtual machine. Otherwise, your vCPU will spike.
  • Always assign 1vCPU . If necessary, add a 2nd vCPU. Do not use 4 vCPUs!
  • Use 2 GB to start. Scale up to +-4 GB of vRAM if necessary
  • Use 1 .vmdk for your system partition (C:\ or other remapped drive letter) and 1 separate .vmdk for your program files.
  • Put the page file on the 2nd .vmdk
  • Important: disconnect any .iso file in your virtual CD-Rom
  • Use roaming profiles and cleanup your profiles at logoff
  • Disable sound for your published apps
  • Install the UPH service (download it here )
  • User sessions: for me, 30 users on a VM is the sweet spot. Do not expect to get as many users on it as on a physical box!
  • Scale out, not up. A major advantage of VM is to clone/NewSID/sysprep existing servers and put them into your existing Citrix farm. Just stop & disable your IMA service , clean up your RMLocalDB (if you use enterprise) and NewSid the thing. Refer to this support article for more info.
  • Use dual core or quad core systems. This because ESX will have more CPU to schedule its vCPUs on.
  • Don’t ever use a 2 vCPU Citrix virtual machine in a 2 pCPU physical machine!
  • Do not install the memory ballooning driver while installing the VMware Tools
  • Do not use a complete installation Vmware tools : there is an issue with roaming profiles and the shared folders component. See my previous article for more info.
  • Disable COM ports, hyperthreading, visual effects & use speedscreen technology where possible.
  • Use snapshots when installing applications or patching your servers (yes! With VMware you can do this!). In case of disaster, you can still revert to the original working server without using backups. Make sure all snapshots are removed ASAP when finished!
  • Always check that there are no snapshot leftovers (f.e. the infamous _VCB-BACKUP_ when using VCB)
  • Don’t forget you can use DRS rules to run your citrix servers on separate physical hosts.
  • Check out this vmworld 2006 presentation
  • And last but not least: do not forget to read ESX’s (excellent) performance tuning white paper .

Continue reading

VMware: Hyper-V on Server Core vs ESXi


Video 1 : http://www.vmware.com/technology/whyvmware/resources/esxi-hyper-v-installation.html

This first video shows every step required to install Hyper-V and ESXi on a fresh machine. We kept count of the elapsed time, reboots, mouse clicks and keystrokes each product needed and it clearly shows the huge advantage the truly thin and OS-free ESXi architecture has in installation speed and simplicity.  ESXi goes from bare-metal to fully installed in one-third the time, half the mouse clicks, hundreds fewer keystrokes and just one reboot vs. seven compared to Hyper-V.  The simplicity of the ESXi wizard-driven installation is striking compared to the arduous process needed to first get the Server Core OS installed and then configure Hyper-V in a command line environment.

Video 2 : http://www.vmware.com/technology/whyvmware/resources/hyper-v-side-by-side-esxi.html

Our second video starts where the first left off and takes Hyper-V and ESXi through the steps needed to configure two iSCSI datastores for VM use. iSCSI setup is a standard task for any virtualization user that wants to take advantage of shared storage for VM migration and high availability.  ESXi’s Windows-based Virtual Infrastructure client makes the iSCSI setup quick and easy.  For Hyper-V, the "Windows you know" is nowhere to be seen.  Instead, working with Server Core requires you to key in a long sequence of obscure commands to configure iSCSI initiators and targets, partitions and file systems.  We generously showed the Hyper-V setup executed with no delays, although it took us hours of digging through Microsoft documents and knowledgebase articles to find the right commands to use when configuring iSCSI in Server Core.

Source: http://blogs.vmware.com/virtualreality/2008/09/esxi-vs-hyper-v.html

VMware accidentally timebombs ESX, causing worldwide mayhem

In an act of “endeavoring to deliver a release with support [that] customers deem important ” VMware accidentally left a licensing timebomb enabled in the build that it shipped to customers about three weeks ago. The timebomb causes all installed licenses for ESX to be regarded as invalid on August 12, 2008. This in turn causes virtual machines to not be allowed to start from a powerdown or suspended state or allow virtual machines to be VMotioned to another ESX host .

VMware provides one way to prevent encountering the problem and one temporary workaround until they can provide a patch: VMware has released express patches to remedy the problem.

Full repeat of VMware’s latest e-mail advisory:

Dear VMware Customers,

We have released the express patches for the product expiration issue. Please go to http://www.vmware.com/go/esxexpresspatches for download and KB articles. Since our last customer email we have completed our verification tests that the express patches we’ve released are fully compatible with the VMware Update Manager. Please see the KB articles for deployment information regarding Update Manager.

The KB articles are kept up-to-date. Please refer to the KB articles for information and updates.

In our last update, we referred to an initiative by our support and engineering teams to find an option to apply the patch without the necessity of entering maintenance mode and VMotion of VM’s to other servers, or VM power-off and re-power-on. Our earlier tests have not found a consistently successful way to address this. We continue to investigate this possibility, as we know that it would reduce the maintenance burden on our customers who may not have a patched server available for VMotion.

We are on target to release updated versions of the ESX/ESXi 3.5 Update 2 patch at 6 PM PST today. This is for customers who have not already upgraded to the previously released version of ESX/ESXi 3.5 Update 2

Thank you,

The VMware ESX Product Team


An issue has been discovered by many VMware customers and partners with ESX Update 2 (build number 103909) and ESXi 3.5 Update 2 (build number 103908) where Virtual Machines fail to power on or VMotion successfully. This problem began to occur on August 12, 2008 for customers that had upgraded to ESX 3.5 Update 2. The problem is caused by a build timeout that was mistakenly left enabled for the release build.

The following message is displayed in the vmware.log file for the virtual machine:

This product has expired. Be sure that your host machine’s date and time are set correctly.
There is a more recent version available at the VMware web site: http://www.vmware.com/info?id=4 .
Module License Power on failed.

Affected Products:

- VMware ESX 3.5 Update 2 & ESXi 3.5 Update 2. Thank you, The VMware ESX Product Team

- The problem will be seen if ESX350-200806201-UG is applied to a system.

- No other VMware products are affected.


VMware Engineering has produced express patches for impacted customers to resolve the issue

Source: http://www.buit.org/

Monitoring VMWare ESX using OpsMgr

Monitoring VMWare ESX with SCOM isn’t a hard thing, however getting correct and relivent information in a clean and easy way can be. This guide I have compiled with VMWare should provide the mechanics of setting up monitoring between VMWare ESX and SCOM.

– UPDATE : If you have trouble with the link its due my limit being reached. I will add a secondary site ASAP. Wasn’t prepared for the responce –
Download Here:
Mirror 1
Mirror 2

Continue reading

VMWare Links !

Virtual Machine Backup Guide – http://www.vmware.com/pdf/vi3_301_201_vm_backup.pdf
Consolidated Backup in Vmware Infrastructure 3 – http://www.vmware.com/pdf/vi3_consolidated_backup.pdf
Vmware Consolidated Backup Best Practices, Tips and Tricks – http://download3.vmware.com/vmworld/2006/bct4540.pdf
Backup and Recovery of Virtual Servers – http://download3.vmware.com/vmworld/2006/mdc9870.pdf
Hot Backups and Restores – http://download3.vmware.com/vmworld/2006/tac9816.pdf
Non-disruptive Backups of Vmware Environments Using Veritas Netbackup – http://download3.vmware.com/vmworld/2006/tac9912.pdf
Perl Backup Script for Vmware ESX 2.x – vmbk.pl ( free ) – http://www.vmts.net/vmbk.htm
Perl Backup Script for Vmware ESX 3.x – vmbk.pl ( free ) – http://www.vmts.net/vmbk3.htm
Backup scripts – http://www.rtfm-ed.co.uk/?p=23+and+http://www.rtfm-ed.co.uk/?p=156
Shell script – http://www.vmware.com/community/thread.jspa?messageID=475244
Perl script – http://www.vmware.com/community/thread.jspa?threadID=70253&start=0&tstart=0
Oracle backups – http://www.vmware.com/community/thread.jspa?threadID=76746&messageID=601703#601703
Vmware Consolidated Backup Technology: Today and Future – http://download3.vmware.com/vmworld/2006/bct4539.pdf
Implementing VMware VCB – http://www.vmware-tsx.com/download.php?asset_id=19
Virtual Infrastructure Scripted Backup Utility (VISBU) – http://www.xtravirt.com/index.php?option=com_remository&Itemid=75&func=fileinfo&id=7

Continue reading

ESX 3.5 & VC 2.5 Released, whats new?

At the VMUG event I learned about a few new features that are available in ESX 3.5 & VC 2.5. Jeremy van Doorn told us about:

  • Distributed Power Management (Still experimental)
  • NPIV support (connect a HBA to a VM instead of an ESX host)
  • VMware update manager (formerly Shavlik)
  • Storage VMotion (VMotion the storage of a VM to a different location, only through command line)
  • Paravirtualization possible on a per VM basis
  • More RAM for a VM, up to 64GB and more RAM for the host, max 128GB
  • SATA drives support (not known which drives are supported)
  • Hardware page tabels for even less overhead when virtualizing memory
  • 32 nodes instead of 16 in a DRS / HA cluster
  • Up to 10 isolation addresses in HA
  • A VM can now have a local swap file
  • Cisco Discovery Protocol
  • Wake On LAN for your nics
  • And a very important VCB feature: restore files on VM

Apart from these items, I discovered some more nice little improvements in the interface. So the next post will be only screenshots and short comments on each new feature or button I discovered :-)

Just some screenshots and comments about new ESX 3.5 & VC 2.5 features.

Logon with the VI Client

Starting the VI Client, I noticed a little change in the text and when you logon using the hostname instead of the FQDN, you’ll receive a certificate warning box

Logon to VC 2.5 Certificate mismatch

Getting Started Wizard

Next thing you’ll immediately notice is the “Getting Started Wizard”. On quite a number of levels, you can find these wizards like Hosts & Clusters view, Datacenter, Host, VM, Resource Pools and Clusters. The wizard helps you with the most common task to get you familiar with VC 2.5 as soon as possible. Did you notice the “Close tab” in the upper right corner? The wizards can be disabled for the advanced users.

Getting Started Wizard Getting Started Wizard VM


In the menu-bar I noticed a plugins option. Clicking it and installing the”VMware Update Manager Extension” and “VMware Converter Enterprise”, gave me two new options in the top bar. The “Consolidate” button and the “Update Manager” button. I’ll explore boh buttons later on.

Hosts & Clusters

To wade through all the options I started at the “Hosts & Clusters” level and the DataCenter level (alsmost identical). The datacenter tab didn’t bring anything new and the virtual machines tab looks the same as in VC2.0. I did notice I can now also view the column DNS Name, not sure if this was in VC2.0 already. On the hosts tab and the task & events tab, I couldn’t find any new info or columns. Except for quite a number of warning messages that my storage for VMware Update Manager is low. It is pointing to c:\documents and Settings\….. have to get an extra disk for this :-). It a shame that you can’t select the warning or error message in the “Event details” screen. Would be much easier for copying and pasting when looking for support. The alarms tab, permissions tab and maps tab look like there hasn’t changed anything. But the “Update Manager” tab is completely new. On a seperate page I’ll write more about the update manager.

Hosts level

At the hosts level, there is ofcourse the “Getting Started” tab again and at the “Summary Tab” there now is an option to make an Annotation. I often use them for my VMs, but not yet for the hosts. We’ll see :-)

The “Resource Allocation” tab showed different values then the one I was used to see. When looking at the cpu reseravtions and shares, I noticed that each VM (all single cpu) still had the value “Normal Shares” but the “Shares Value” now was set to 4000. I edited the cpu resources for a VM and noticed that the values related to the settings Low, Normal, High had changed. They now are: low = 2000, normal = 4000, high = 8000 shares. They doubled each value. Well no big deal I guess, because its all relative :-) When using esxtop at the cos, I noticed that there was a change in how the shares are displayed. In ESX 3.0.2 one could see the number of shares in the “ASHRS” column like 1000, 2000, 2217, etc. But now in ESX 3.5 it shows only a value of -3 in my case. See the screenshots below:

007-esxtop-302.jpg 007-esxtop-35.jpg

For the memory resources there is a change as well. Low memory shares is a value of 327680, normal is 655360 and high is 1 million shares.

Performance graphs

It is now easier to switch from cpu to memory or other standard graphs. In the upper right corner, there is a little pulldown menu for easy switching. It also includes a new graph “Management Agent” which shows the memory usage of the service console. Another new option is “System” which shows quite a number of cpu and memory countes for the host.


I discovered it gets even better. You can save your own views and have them listed in the dropdown menu !!! For example, for the realtime network counters, I would like to add “Network Packets Transmitted” and “Network Packets Received” and view this on a regular basis. You can now save your view by pressing “Save chart settings” and the name of this chart setting will be available in your dropdown list.


Configuration tab

First thing to notice are the extra options “Time configuration” and “Virtual Machine Swapfile location”. When selecting a ESX 3.0.2 host, the “Virtual Machine Swapfile location” option is not displayed, but the “Time configuration” option is. Its just limited in options now.


In the hardware settings for processors, memory and storage I can’t spot any differences with previous versions. In the networking properties there is a little extra balloon showing behind each nic. If you click it, it displays the Cisco Discovery Protocol settings. I haven’t used CDP before so I google a little on it and found this link with some explanation about it: http://www.javvin.com/protocolCDP.html


I browsed through all the other network options, creating vSwitches adding nics etc, but can’t find any news there. I was just wondering, when creating a new vSwitch, is the option “Promiscuous Mode” always set to “Reject” ? Not sure if the default changed here.

Storage adapters

Couldn’t find any new options in here, but I was very pleased to find that my SATA controller showed ass Storage device :-) I have an ASUS P2M5-SAS mainbord which I choose because of the LSI Logic Controller that emulates SCSI for the SAS controller. Now I probably can connect my disks directly to it. Small problem with the LSI controller is that you have to create an array (raid1 or 5) to present the disks to ESX. My guess is that I don’t need to do this anymore with the SATA disks.


Network adapater

A new column here: “Wake on LAN supported”. This feature is needed for Distributed Power Management. DRS is now able to powerdown a host when it doesn’t have a workload, should the host be needed again, then a Wake On LAN signal is needed to power it on again. Jeremy van Doorn (VMware) explained that this feature still experimental because integration with monitoring tools isn’t finished yet. Should DRS decide that an ESX host can be powered down, a lot of alarms will be triggered on severall external tools because they don’t understand why the host went down. Another thing I thought of myself, the HP DL 585 server I’m working with at my customers site, don’t power down. If I enter a “shutdown -h now”, ESX goes down, but finaly holds with the message “You can power down your system now” (or similar :D). My guess is that WOL won’t be able to power on the server when its in this state.

I was pleased to find that suddenly my Intel NIC showed in the Network adapters list. This nic wasn’t supported or found when ESX 3.0.2 was running on it. I also noticed a change in vSwitches. I used to have a vswitch vsw-vm01 to which vmnic0 and vmnic1 were assigned. Now it only has vmnic0 assigned. The intel nic that was left in the system and is recognized now, became vmnic2 and my other nic which used to be vmnic1, has now become vmnic3. So be sure to backup your nic config / vSwitch config before upgrading. Although I guess only few people have non-working nics in their system before upgrading :-)


Licensed Features

At a first glance, nothing new here. But when selecting the license source, there is a different window for selecting the license source. There are 4 options now.

  1. “Use evaluation mode”. This provides unlimited host services during the evaluation period. It may not be selected once the evaluation has expired.
  2. “Use serial number”. License host edition and add-ons using a serial number.
  3. “Use License Server”. Acquire licenses for host edition and add-ons on demand from the following server. VirtualCenter may change this server while this host is under management.
  4. “Use Host License file”. License host edition and add-ons using a file installed on the host.


Time Settings

This option makes it much easier to configure NTP. When you click properties, you first have a window in which you can set the date and time manually, next you can enable or disable the NTP client. Through the “Options” button you can configure the general settings and NTP settings. The general settings let you set the startup behavior of the NTP daemon. You can also start, stop and restart the NTP daemon at this point. The NTP settings let you define the NTP servers you wish to use. Nice feature is that enabling or disabling the NTP client, also updates your security profile and opens or closes the NTP client port (123). Hmmm, this would be nice with iSCSI. I’ve seen a number of failed iSCSI installations because the admin forgot to open iSCSI port on the firewall.


DNS and Routing

No changes here I guess.

Virtual Machine Startup / Shutdown

Again no changes.

Virtual Machine Swapfile location

This is a completely new feature. You can choose if your VM stores its swapfile with the VMX file or on a different datastore. This datastore can also be a local VMFS datastore !!! I remember that when running a Microsoft Cluster within ESX, there is the requirement that the boot-drive of your Windows guest, has to be on local storage. Someone explained me this is because sometimes there might be just too much delay if the guest wants to write to swapfile which might trigger the cluster to failover. I asked Jeremy van Doorn if I would now be able to put the cluster VM on SAN and VMotion it, but he thinks this is not the case yet. Unfortunately :-)


Security Profile

No changes here

System Resource Allocation

Looks like no changes to me

Advanced Settings

I did notice some changes here, but I can’t find all of them. For sure I know that LVM has much less options to configure. There are some new sections called VMKernel and Config section. I’m not sure if options have just moved arround, seemed a bit much to search for now :-)


That’s all for the “Configuration tab” of a host through VC2.5.

Virtual Machines

On the summay tab, I see two small GUI changes. First is that now the DNS name of a VM is mentioned, second it shows a “Memory Overhead” value. Not sure how this value is determined and what it would tell me :-)On the performance tab, I see the same changes as on the upper levels. Although the set of settings you can define yourself, is a new set. So you can have different sets at different levels. On the other tabs there are no new things to be found, so lets edit the VM settings :-)

Virtual Machine Settings: Hardware tab

First thing to notice is the max ammount of RAM I can give to the VM. We can now go to 64Gb RAM assigned to a single VM… that’s realy nice :-) Doubt if I’ll ever need it. There is also a change in the recommended values. There are now 4 values suggested by VC, see the screenshot below.


On the network properites, I see a new box to change the MAC Address for a VM. You can choose Automatic or Manual and when selecting Manual, you can change the MAC Address. Very nice, no more editting the VMX.


In the virtual disk properties, you can resize your VMDK ! That’s realy nice. You can only increase the size, not shrink it.


Virtual Machine settings: Options tab

On this tab I immediately noticed quite a number of extra options. Lets have a look. On the “VMware Tools” item, I see an extra section “advanced”. Here you can check to “Check and upgrade Tools before each power on” and “Synchronize guest time with host”. Next there is a new section called “Power Management”. Here you define how the VM should respond when the guest OS is placed on standby. You can choose to suspend the VM or Put the OS in standby mode and leave the VM powered On. Plus you can define which nic should listen for Wake On LAN signals.


Advanced section

In the advanced section of the options tab, we see a number of unchanged options. One of them is the general section, which only has a different view under the “Configuration Parameters” button. The CPUID section hasn’t changed.

A new option is the “boot” option. Two settings can be made here, first is the Power-On boot delay. Here you can enter the number of miliseconds the boot is delayed after a VM is powered on or reset. Default value is 0 ms. Next option is “Force BIOS Setup”. This is a nice feature. Ever had to reset a VM multiple times because you were too slow pressing ESC or F12 to get into the BIOS? That’s history now, just enable “The next time the VM boots, force entry into the BIOS setup screen”. Now it will go into the BIOS without keypress :-)



YES !!! WOW !!! We can now use the Paravirtualization technique if needed for a VM. Mostly Linux systems with kernel 2.6.21 and up, will be able to benefit from this. As the tab explains:

VMI is a paravirtualization standard supported by some guest operating systems. Guests that recognize VMI will gain significantly improved performance with VMI support. Guest operating systems which do not use VMI will gain no performance benefit from this support.


Fibre Channel NPIV

As the tab says: Virtual Machines running on hosts with Fibre Channel hardware that supports NPIV can be assigned virtual WWNs for advanced features.


Virtualized MMU

For cpu’s that support virtualizing the MMU, you can set if the VM will be using the automatic settings from the ESX host, force the use of this feature or forbid the use of this feature. I’m surprised that this MMU feature can be changed on a VM basis, very nice.


Tip: If you want to learn more about MMU and Paravirtualization, read this pdf:



The last option is the SwapFileLocation of the VM. Here you have three options:

  • default, use the host settings
  • always store with the VM
  • Store in the host’s swapfile datastore

The first two options are self explaining I think, the third one is a bit strange I think. The explanation in the text is: “If a swapfile datastore is specified for the host, use that datastore. Otherwise store the swapfile with the VM.”. Wouldn’t this be the same as “default”. Because setting a swapfile datastore at host level, will also enable it. If there is no datastore, then the VM can’t store the swapfile in a different location.


Today it became official, the release of ESX 3.5 & VC 2.5 & Converter 4.0.

When looking at the UN-official release of last friday, it seems that only the ESX buildnumber has changed.

ESX Server Version 3.5 | 12/10/2007 | Build 64607
VirtualCenter Version 2.5 | 12/10/2007 | Build 64201

Homepage at: http://www.vmware.com/products/vi/

Download at: http://www.vmware.com/download/vi/

Release notes at: http://www.vmware.com/support/vi3/doc/vi3_esx35_vc25_rel_notes.html


VMware “announce” the new Vi3 features

Main thing to say that is this is a just an announcement. There’s nothing new dowload and install yet. In fact some of these products (as far as I am aware yet) aren’t even on a private beta yet, never mind a public beta. Some of this was announced at VMworld in th keynotes. I’ll focus on what’s new:

  • 128GB of RAM for ESX 3.5
  • 64GB of RAM for Virtual Machines
  • Suport for Transparent Paravirtualization support for paravirtualizated enabled Linux 2.6.21 kernels
  • Support for MMU – Memory Management Unit – support for memory page tables handled by the new CPUs
  • Support for NPIV – N-Port ID Virtualization – presenting a WWN to a VM
  • Support for TCP Segment Offload and jumbo frames
  • Suport for local SATA Drives
  • Support for 10GigE
  • Storage VMotion – Moving VMs files around whilst the VM is running
  • Protect against operating system failures with virtual machine failure monitoring (experimental) in VMware HA. (Continuious HA???)
  • One-step restore of VMs using VMware Convertor/VirtualCenter
  • Distributed Power Management (Power down ESX hosts when you don’t need them – scarey or what!)
  • VMware Update Manager – “Windows Update” for ESX Hosts & the Vi Client and integrated with DRS
  • Consolidation Management – Mini-Capcity Planner and VMware Convertor integrated with VirtualCenter
  • Site Recovery Manager
  • ESX3i


Offical Press Release: