Wednesday, January 04, 2006

Clone OS - ESX Server

I found this article how to clone virtual OS within ESX server:

* Don't forget uuid.action = "keep", especially if this is a Win32 system.

The added step for what goes on below is that you'll need to move the VM "personality data" (.vmx, .nvram, etc...) using scp (secure copy, aka copy over SSH) to the remote target system.

I can't recall offhand if I posted this before, so... What follows is a detailed process of how one would clone (and in the process relocate) a VM.

The "uuid.bios" value is used in some cases by the Guest OS (via a DMI/SMBIOS query to the VM's BIOS) as part of determining its unique identity. You should only allow the "uuid.bios" value to change in cases where you wish to trigger a change in personality. In Microsoft Windows-ese, this would be when you intend to run things like SysPrep or NewSID on the Guest OS.

The "uuid.location" value is used to determine the uniqueness of a VM at the VMKernel level, and will change any time the VM's .vmx is renamed or relocated. The "uuid.bios" and "uuid.location" values start off identical, but may differ at some later point in time (per the above conditions).


The basic steps involved in cloning a VM on a standalone ESX Server chassis are as follows:
1) Produce "master" VM image
From this point on, leave the "master" VM image in a powered off state to preserve it.

2) Copy "master" to "clone master"
Preferrably, make a copy of the .dsk file, and create a new directory for copies of the .vmx and nvram files. Adjust the .dsk pointer and description data as appropriate in the .vmx file.
ADD the following line to the .vmx file:

uuid.action = "keep"

This will force the "uuid.bios" value to remain consistent, even if the VM's .vmx file has been relocated or renamed.

It is equivalent to answering the UUID question with "Keep Always".

3) Register "clone master" VM

vmware-cmd -s register /path/to/clone/file.vmx

4) Depersonalize the VM as appropriate for the Guest OS (Win32 SysPrep, etc...)

5) Copy "clone master" for deployment

Make a copy of the .dsk file, and create a new directory for copies of the .vmx and nvram files. Adjust the .dsk pointer and description data as appropriate in the .vmx file.
REMOVE the following lines from the .vmx file:

Ethernet0.generatedAddress = "....."

Ethernet0.generatedAddressOffset = "....."

uuid.location = "....."

uuid.bios = "....."

uuid.action = "keep"

6) Register new VM

vmware-cmd -s register /path/to/new/file.vmx

7) Power on new VM from the MUI directly

A fresh "uuid.location" and "uuid.bios" set will be populated into the .vmx file at this point. These values are then used as part of the hashing algorithm to generate the new ethernet MAC for the VM.

The SMBIOS inside the VM will have a "system serial number" with the new UUID as well, allowing repersonalization as a new unique entity.


Steps involving a "clone master" as a copy are to preserve the "master" VM in case errors are encountered during depersonalization.


Note, you may want to also set and/or adjust two other bits of data: = "....."

checkpoint.cptConfigName = "....."

These two values are concatenated together in order to determine a unique location to store the .vmss VM Saved State file during a "Suspend" operation. The default value (if unset) for "" is the VMFS2 Volume where the lowest numbered virtual disk is stored (scsi0:0 typically). The default value (if unset) for "checkpoint.cptConfigName" will be an arguably-unique hash based on the Virtual Machine's "displayName" variable and ten hexadecimal digits.


If you find yourself in a situation where you cannot "Suspend" an already running VM due to disk-space constraints based on the " = *" mapping in the .vmx configuration file, a workaround exists:

While the VM is running, query for current state:

vmware-cmd /path/to/your/VM.vmx getconfig

Determine where you have sufficient space in a VMFS2 volume to provide for the .vmss VM Saved State file (it will be slightly larger than max mem allocated to that VM).

Things like `vdf -h` will be useful.

Set the new path based on your discovery above:

vmware-cmd /path/to/your/VM.vmx setconfig "/vmfs/VMFS2VolNameHere"

Suspend your VM. Wait until the suspend operation has completed successfully. Detach the VMware Remote Console session(s) for this VM at this time.

Adjust the "" in the .vmx configuration file to match the new value used above.

Reconnect to your VM using the VMware Remote Console and resume the VM at your leisure. Note, reversing to the original "" value requires another suspend/resume cycle, using the same procedure as above.




Relocating a VM from one directory to another (be it on the same physical chassis or not) is reasonably straightforward. The big caveats are that you should be well aware of the risks (re: UUID warnings) and to reconfirm the configuration and device mappings for the VM if it is being moved between chassis.

A single example snippet of shellscript is included below. This example assumes that 'root' will own all VMs. Optimally you should use a 'role account' (or VMware VirtualCenter) rather than 'root' on the ESX Server, so you can better manage access controls / permissions across your pool of Virtual Machines.
Details on structuring 'role accounts' for running in a non-VirtualCenter environment follows below.



The process for splitting .VMX files off into individual directories in your environment is as follows:

* Shut down the VM you are going to move. Perform a clean, Guest-OS level shutdown, letting the VM power down completely.

* Run the bit of shellscript /as root/ on the Console OS for that particular VM (see bottom of email). You will be prompted for the root PW twice (unregister, register). A result code of "1" from the unregister and register commands means "success". The shellscript makes a permissions-matched copy of the configuration (.vmx) file, rather than moving it.
This allows you to maintain a "clean" copy of the .vmx file in case your migration process is interrupted before the unregister/register cycle is completed.

* Start up the VM from the /MUI/ using the Green Triangle "play" button. You will see the VM's status change to "Waiting for Input...".

* Click on "Waiting for Input...", and answer "Always Keep". This will fix the UUID data in the .vmx file to reflect the new location for the configuration file (uuid.location=...), but will -NOT- change the UUID/SMBIOS data for the VM on the Guest OS side (uuid.bios=...).
This is precisely the behavior that you want.

* Confirm that the VM has started up correctly. You'll notice that each VM now has 'vmware.log' and 'nvram' files in their respective directories.

* Optionally, delete the original .vmx file from the original location.

##### ##### #####

cd /root/vmware/

mkdir ./myVMsNewHome

chmod 0755 ./myVMsNewHome

cp --preserve /root/vmware/myVMsOldHome/MyVM.vmx ./myVMsNewHome

/usr/bin/vmware-cmd -U root -s unregister /root/vmware/myVMsOldHome/MyVM.vmx

/usr/bin/vmware-cmd -U root -s register /root/vmware/myVMsNewHome/MyVM.vmx

##### ##### #####

This may not be relevant for this specific chassis, but may be of use for singly deployed chassis. The basic information you're looking for can be found here in the "Authenticating Users" section:

I'll provide you a real world example so you can see how it gets put to use:

First, this example assumes you are using the regular Unix permissions model, with users and groups defined locally on the Console OS.

You have one SuperUser (root). This user should not be used to add Virtual Machines. It may be used to delete Virtual Machines, and make system-wide changes. Unix model: SuperUser.

You have (at least) one 'role' account which handles addition/modification/deletion of the _Virtual Machine_ layer. This 'role' account is rarely used.

Unix model: read-write-execute-own. You have several users who are allowed to control VMs owned by their group's role account as though it were a system on their desk. They can change power state, connect/disconnect devices, etc.... They may not however modify the hardware connected to that VM, which includes modifying what the CDROM and Floppy drives point at. Unix model: read-execute-group.

If you add the 'write' permission as well to the .vmx file, they can modify the VM. You may also have a third category of everyone else (people outside of that group) who are allowed to check system configuration/health data on the Web Management Interface, but are not allowed to make any changes (power state, devices, etc...). Unix model: read-only-other.

This follows the standard Unix model of: User Group Other (root is a special case)

Assuming your shared space is in /home/vm/...

Create a 'role' account (ex: vmadmin). The command is all one line, I've wrapped it for readability. In this example, role type accounts are in the 5xxx range for UID/GID.

useradd -c "VMware Admin Account" -d "/home/vmadmin" -u 5001 -m vmadmin

Set that role account's password:

passwd vmadmin

Add the control users (ex: susan, james, terry). You'll disable their shell access unless they need direct access to the Console OS side. In this example, regular user accounts are in the 6001+ range for UID/GID.

useradd -c "Susan P." -d "/home/susan" -u 6001 -G vmadmin -s "/bin/false" -m susan

useradd -c "James R." -d "/home/james" -u 6002 -G vmadmin -s "/bin/false" -m james useradd -c "Terry F." -d "/home/terry" -u 6003 -G vmadmin -s "/bin/false" -m terry

Then set their passwords... Assuming you have VMs already created, you may need to perform a bit of ownership/permissions shuffling.

chown -R vmadmin.vmadmin /home/vm

chmod 0755 /home/vm chmod 0750 /home/vm/*/.

chmod 0750 /home/vm/*/*.vmx

Then finally add the read-only rights to VMs that you want visible to anyone who can authenticate in the Web Management Interface.

chmod 0755 /home/vm/some-public-vm/.

chmod 0754 /home/vm/some-public-vm/some-public-vm.vmx

Origianal URL:

Tuesday, January 03, 2006

Windows 2003 Group Policy

3 days after installing brand new DC, I have heaps of event errors on my DC:

Windows cannot access the file gpt.ini for GPO CN={31B2F340-016D-11D2-945F-00C04FB984F9},CN=Policies,CN=System,DC=X,DC=local. The file must be present at the location <\\X.local\sysvol\X.local\Policies\{31B2F340-016D-11D2-945F-00C04FB984F9}\gpt.ini>. (Configuration information could not be read from the domain controller, either because the machine is unavailable, or access has been denied. ). Group Policy processing aborted.

This is very annoying for me. However, I found solution from Google:

This is one problem with Multihomed DCs in order to cure this you need to do
a couple of things.

1. Set the binding order, by going into network properties Control panel, in
the Advanced menu select Advanced Settings. Make sure the internal NIC is at
the top of the connections list and the Client for MS networks and File
sharing are only bound on the internal interface.

2. You will need to make registry entries to stop the creation of the blank
records for the external interface for both the domain name and the global
catalog record. You will then have to manually create these two blank
records. There is a KB describing this but I'm unable to find it but here is
the reg entry, you must use regedt32 to make this entry.

Registry value: DnsAvoidRegisterRecords
Data type: REG_MULTI_SZ


3. On the Interfaces tab of the DNS server properties set the DNS listener
address to the internal IP.

4. On an XP Client you need to upgrade the GPO by following this KB article
Upgrading Windows 2000 Group Policy for Windows XP

What happens is DNS returns the IP of the external interface and file
sharing is not enabled on the interface and LDAP won't pass NAT.

Full Article:

My DC has dual NICs card btw, one for Internal LAN, the other one for DMZ (only for Virtual machines)

Monday, January 02, 2006

VMware diskmount utility

VMWare disk mount utility, claimed by VMWare, able to mount virtual disk to local (host) operating system file system. In fact, this utility corrupted one of my VMware snapshots which in turn unable to boot the OS.

The error was:

Unable to open disk "F:\Virtual\Windows XP Pro\Windows XP Pro-000002.vmdk": The parent virtual disk has been modified since the child was created. (18).

Luckily I have more than 1 snapshots, thus rolling back to earlier version bring the OS alive again.