How to mount and unmount CDROMs in ESXi

Not documented, but here’s how you do it:

# vmkload_mod iso9660
Module iso9660 loaded successfully

# /sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba32:C0:T0:L0

# ls -al /vmfs/volumes/CDROM
lrwxr-xr-x    1 root     root                 20 Sep 29 22:39 /vmfs/volumes/CDROM -> mpx.vmhba32:C0:T0:L0
# ls /vmfs/volumes/CDROM
A.Z           DRYRUN.CFG    INSTALL.VGZ   OPEN_SOU.TXT  VMKERNEL.GZ
BOOT.CAT      IENVIRON.VGZ  ISOLINUX.BIN  README.TXT
CIM.VGZ       IMAGEDD       ISOLINUX.CFG  SYS.VGZ
CIMSTG.TGZ    IMAGEDD.BZ2   MBOOT.C32     TBOOT.GZ
CUSTOM.CFG    IMAGEDD.MD5   MENU.C32      VMKBOOT.GZ 

And to unmount it:

# /sbin/vsish -e set /vmkModules/iso9660/umount mpx.vmhba32:C0:T0:L0
# ls /vmfs/volumes
3c3693e8-f77a642a-1910-5c6bdcb26d3a  Hypervisor2
4ca3b6e9-fd9ada85-a674-000c29e3db40  Hypervisor3
70589275-fb41eccd-6632-ac007674ad55  d0900a77-edfa5e08-0fee-561dd908cfcd
Hypervisor1                          datastore1
# vmkload_mod -u iso9660
Module iso9660 successfully unloaded
#
Advertisements

Custom kickstart of ESXi

Steps to create a custom ESXi installation:

Extract the media from the CDROM

# mount -o ro,loop -t iso9660 VMware-VMvisor-Installer-4.1.0-260247.x86_64.iso /mnt
# cp -rp /mnt /extract/
# umount /mnt
# cd /extract

Edit the isolinux.cfg file. Eg:

default menu.c32
menu title VMware VMvisor Boot Menu
timeout 80

label ESXi Installer
menu label ^ESXi Installer
kernel mboot.c32
append vmkboot.gz --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label Kickstart debug
menu label ^ESXi Installer (default KS)
kernel mboot.c32
append vmkboot.gz ks=file:///etc/vmware/weasel/ks.cfg --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label ESXi Kickstart
menu label ^ESXi Kickstart from CD
kernel mboot.c32
append vmkboot.gz ks=cdrom:/CUSTOM.CFG --- vmkernel.gz --- sys.vgz --- cim.vgz --- ienviron.vgz --- install.vgz

label ^Boot from local disk
menu label ^Boot from local disk
localboot 0x80

Create a file called custom.cfg which will be the custom jumpstart file.

NB. Because ESXi mounts this as a ISO9660 filesystem, and not Rock Ridge, files will be accessible as UPPER CASE.

Then recreate the CD:

# mkisofs -o /tmp/custom.iso -b isolinux.bin \
  -no-emul-boot -boot-load-size 4 -boot-info-table -c boot.cat -r .

Booting this CD will show two more options. The first additional option will kickstart with the default options, and the second will use a file on the CDROM for installation.

After installation, the following screen is displayed:

The default file is located in /etc/vmware/weasel/ks.cfg, and by manually specifying it, a log file will be created, which is viewable after booting in /var/log/esxi_install.log

Eg /etc/vmware/weasel/ks.cfg has this contents:

vmaccepteula
rootpw letmein
autopart --firstdisk --overwritevmfs
install cdrom
network --bootproto=dhcp --device=vmnic0

%post --unsupported --interpreter=python --ignorefailure=true
import time
stampFile = open('/finished.stamp', mode='w')
stampFile.write( time.asctime() )

And it will generate the following output file (which can be used to verify keywords etc.) :

/var/log/esxi_install.log :

2010-09-29 21:57:22,924 INFO Got boot cmdline “vmkboot.gz ks=file:///etc/vmware/weasel/ks.cfg ”
2010-09-29 21:57:22,941 INFO Weasel skipped boot command line token (vmkboot.gz)
2010-09-29 21:57:22,953 DEBUG command line options: [(‘-s’, ‘file:///etc/vmware/weasel/ks.cfg’)]
2010-09-29 21:57:22,954 WARNING User has not chosen a Weasel run mode.
2010-09-29 21:57:22,954 INFO Weasel run mode defaulting to Scripted.
2010-09-29 21:57:23,262 DEBUG Querying disks
2010-09-29 21:57:23,265 DEBUG  lun — mpx.vmhba1:C0:T0:L0
2010-09-29 21:57:23,265 DEBUG   Trying /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0
2010-09-29 21:57:24,254 DEBUG Unknown parted file system type (‘unknown’) for partition: /vmfs/devices/disks/mpx.vmhba1:C0:T0:L02
2010-09-29 21:57:24,254 INFO Discovered lun — mpx.vmhba1:C0:T0:L0 (console /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0) — VMware,  VMware Virtual S (40960 MB, mptspi)
2010-09-29 21:57:24,256 INFO Using ScriptedInstall file: file:///etc/vmware/weasel/ks.cfg
2010-09-29 21:57:24,259 INFO Using ScriptedInstall file: file:///etc/vmware/weasel/ks.cfg
2010-09-29 21:57:24,260 DEBUG command: install found
2010-09-29 21:57:24,261 DEBUG Branch Created: {‘cdrom’: None}
2010-09-29 21:57:24,261 DEBUG Validating argument: –dir
2010-09-29 21:57:24,261 DEBUG Validating argument: –server
2010-09-29 21:57:24,262 DEBUG Validating argument: cdrom
2010-09-29 21:57:24,262 DEBUG argument is optional
2010-09-29 21:57:24,262 DEBUG Validating argument: nfs
2010-09-29 21:57:24,262 DEBUG Validating argument: url
2010-09-29 21:57:24,263 DEBUG Validating argument: usb
2010-09-29 21:57:24,263 DEBUG doInstall branch: {‘cdrom’: None}
2010-09-29 21:57:24,263 INFO CD-ROM install specified. CD-ROM is the default.
2010-09-29 21:57:24,367 INFO Loaded iso9660 module
2010-09-29 21:57:24,367 DEBUG executing: /sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba32:C0:T0:L0
2010-09-29 21:57:24,671 DEBUG command exited with status 0
2010-09-29 21:57:24,672 INFO
2010-09-29 21:57:24,677 INFO setting media location
2010-09-29 21:57:24,679 INFO Using ScriptedInstall file: file:///etc/vmware/weasel/ks.cfg
2010-09-29 21:57:24,680 DEBUG command: vmaccepteula found
2010-09-29 21:57:24,680 DEBUG Branch Created: {}
2010-09-29 21:57:24,681 DEBUG command: rootpw found
2010-09-29 21:57:24,682 DEBUG Validating argument: –iscrypted
2010-09-29 21:57:24,683 DEBUG command: autopart found
2010-09-29 21:57:24,684 DEBUG Branch Created: {‘–firstdisk’: ‘local,remote’, ‘–overwritevmfs’: None}
2010-09-29 21:57:24,684 DEBUG Validating argument: –disk
2010-09-29 21:57:24,684 DEBUG Validating argument: –drive
2010-09-29 21:57:24,685 DEBUG Validating argument: –firstdisk
2010-09-29 21:57:24,685 DEBUG argument is optional
2010-09-29 21:57:24,685 DEBUG Validating argument: –overwritevmfs
2010-09-29 21:57:24,686 DEBUG argument is optional
2010-09-29 21:57:24,686 DEBUG checking disk for install eligibility — mpx.vmhba1:C0:T0:L0 (console /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0) — VMware,  VMware Virtual S (40960 MB, mptspi)
2010-09-29 21:57:24,686 DEBUG   disk is eligible
2010-09-29 21:57:24,687 DEBUG   BEGIN firstdisk order
2010-09-29 21:57:24,687 DEBUG     mpx.vmhba1:C0:T0:L0 (console /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0) — VMware,  VMware Virtual S (40960 MB, mptspi)
2010-09-29 21:57:24,687 DEBUG   END firstdisk order
2010-09-29 21:57:24,688 INFO autopart –firstdisk == mpx.vmhba1:C0:T0:L0 (console /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0) — VMware,  VMware Virtual S (40960 MB, mptspi)
2010-09-29 21:57:24,688 DEBUG command: network found
2010-09-29 21:57:24,689 DEBUG Branch Created: {‘–device’: ‘vmnic0’, ‘–bootproto’: ‘dhcp’}
2010-09-29 21:57:24,690 DEBUG Validating argument: –addvmportgroup
2010-09-29 21:57:24,690 DEBUG Validating argument: –bootproto
2010-09-29 21:57:24,690 DEBUG argument is optional
2010-09-29 21:57:24,690 DEBUG Validating argument: –device
2010-09-29 21:57:24,691 DEBUG argument is optional
2010-09-29 21:57:24,691 DEBUG Validating argument: –gateway
2010-09-29 21:57:24,691 DEBUG Validating argument: –hostname
2010-09-29 21:57:24,692 DEBUG Validating argument: –ip
2010-09-29 21:57:24,692 DEBUG Validating argument: –nameserver
2010-09-29 21:57:24,692 DEBUG Validating argument: –netmask
2010-09-29 21:57:24,693 DEBUG Validating argument: –nodns
2010-09-29 21:57:24,693 DEBUG Validating argument: –vlanid
2010-09-29 21:57:24,700 DEBUG command: %post found
2010-09-29 21:57:24,703 DEBUG Branch Created: {‘–ignorefailure’: ‘true’, ‘–interpreter’: ‘python’, ‘–unsupported’: None}
2010-09-29 21:57:24,703 DEBUG Validating argument: –ignorefailure
2010-09-29 21:57:24,704 DEBUG argument is optional
2010-09-29 21:57:24,704 DEBUG Validating argument: –interpreter
2010-09-29 21:57:24,704 DEBUG argument is optional
2010-09-29 21:57:24,705 DEBUG Validating argument: –timeout
2010-09-29 21:57:24,705 DEBUG Validating argument: –unsupported
2010-09-29 21:57:24,706 DEBUG section-contents: “import time\nstampFile = open(‘/finished.stamp’, mode=’w’)\nstampFile.write( time.asctime() )\n”
2010-09-29 21:57:24,707 DEBUG scripts: {‘realPath’: None, ‘script’: “import time\nstampFile = open(‘/finished.stamp’, mode=’w’)\nstampFile.write( time.asctime() )\n”, ‘relativePath’: ‘tmp/ks-script’, ‘interp’: ‘/usr/bin/python’, ‘inChroot’: False, ‘timeoutInSecs’: 0, ‘ignoreFailure’: True}
2010-09-29 21:57:24,707 DEBUG Validating command: %firstboot
2010-09-29 21:57:24,708 DEBUG Validating command: %include
2010-09-29 21:57:24,708 DEBUG Validating command: %packages
2010-09-29 21:57:24,708 DEBUG Validating command: %post
2010-09-29 21:57:24,709 DEBUG command is optional
2010-09-29 21:57:24,709 DEBUG Validating command: %pre
2010-09-29 21:57:24,709 DEBUG Validating command: %vmlicense_text
2010-09-29 21:57:24,710 DEBUG Validating command: accepteula
2010-09-29 21:57:24,710 DEBUG Validating command: announcehost
2010-09-29 21:57:24,710 DEBUG Validating command: auth
2010-09-29 21:57:24,710 DEBUG Validating command: authconfig
2010-09-29 21:57:24,711 DEBUG Validating command: autopart
2010-09-29 21:57:24,711 DEBUG Requires: install
2010-09-29 21:57:24,711 DEBUG Validating command: autostep
2010-09-29 21:57:24,712 DEBUG Validating command: bootloader
2010-09-29 21:57:24,712 DEBUG Validating command: clearpart
2010-09-29 21:57:24,712 DEBUG Validating command: cmdline
2010-09-29 21:57:24,713 DEBUG Validating command: device
2010-09-29 21:57:24,713 DEBUG Validating command: deviceprobe
2010-09-29 21:57:24,713 DEBUG Validating command: dryrun
2010-09-29 21:57:24,714 DEBUG Validating command: esxlocation
2010-09-29 21:57:24,714 DEBUG Validating command: firewall
2010-09-29 21:57:24,714 DEBUG Validating command: firewallport
2010-09-29 21:57:24,715 DEBUG Validating command: firstboot
2010-09-29 21:57:24,715 DEBUG Validating command: harddrive
2010-09-29 21:57:24,715 DEBUG Validating command: ignoredisk
2010-09-29 21:57:24,715 DEBUG Validating command: include
2010-09-29 21:57:24,716 DEBUG Validating command: install
2010-09-29 21:57:24,716 DEBUG command is optional
2010-09-29 21:57:24,716 DEBUG Validating command: interactive
2010-09-29 21:57:24,717 DEBUG Validating command: keyboard
2010-09-29 21:57:24,717 DEBUG Validating command: lang
2010-09-29 21:57:24,717 DEBUG Validating command: langsupport
2010-09-29 21:57:24,718 DEBUG Validating command: lilo
2010-09-29 21:57:24,718 DEBUG Validating command: lilocheck
2010-09-29 21:57:24,718 DEBUG Validating command: logvol
2010-09-29 21:57:24,719 DEBUG Validating command: mouse
2010-09-29 21:57:24,719 DEBUG Validating command: network
2010-09-29 21:57:24,719 DEBUG command is optional
2010-09-29 21:57:24,720 DEBUG Validating command: paranoid
2010-09-29 21:57:24,720 DEBUG Validating command: part
2010-09-29 21:57:24,720 DEBUG Validating command: partition
2010-09-29 21:57:24,720 DEBUG Validating command: raid
2010-09-29 21:57:24,721 DEBUG Validating command: reboot
2010-09-29 21:57:24,721 DEBUG Validating command: rootpw
2010-09-29 21:57:24,721 DEBUG Validating command: serialnum
2010-09-29 21:57:24,722 DEBUG Validating command: skipx
2010-09-29 21:57:24,722 DEBUG Validating command: text
2010-09-29 21:57:24,722 DEBUG Validating command: timezone
2010-09-29 21:57:24,723 DEBUG Validating command: upgrade
2010-09-29 21:57:24,723 DEBUG Validating command: virtualdisk
2010-09-29 21:57:24,723 DEBUG Validating command: vmaccepteula
2010-09-29 21:57:24,724 DEBUG Validating command: vmlicense
2010-09-29 21:57:24,724 DEBUG Validating command: vmserialnum
2010-09-29 21:57:24,724 DEBUG Validating command: vnc
2010-09-29 21:57:24,725 DEBUG Validating command: volgroup
2010-09-29 21:57:24,725 DEBUG Validating command: xconfig
2010-09-29 21:57:24,725 DEBUG Validating command: xdisplay
2010-09-29 21:57:24,725 DEBUG Validating command: zerombr
2010-09-29 21:57:24,727 INFO  * STEP GETBIN
2010-09-29 21:57:24,728 WARNING Running task restarting (GETBIN)
2010-09-29 21:57:24,729 WARNING Attempted to load iso9660 module twice
2010-09-29 21:57:24,730 DEBUG executing: /sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba32:C0:T0:L0
2010-09-29 21:57:24,905 DEBUG command exited with status 0
2010-09-29 21:57:24,905 INFO
2010-09-29 21:57:24,937 WARNING A task finished that has not started
2010-09-29 21:57:24,939 INFO  * STEP Clearpart
2010-09-29 21:57:24,940 INFO  * STEP Checkpart
2010-09-29 21:57:24,942 INFO  * STEP WRITEBIN
2010-09-29 21:57:24,942 WARNING Running task restarting (WRITEBIN)
2010-09-29 21:57:25,033 WARNING Attempted to load iso9660 module twice
2010-09-29 21:57:25,033 DEBUG executing: /sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba32:C0:T0:L0
2010-09-29 21:57:25,168 DEBUG command exited with status 0
2010-09-29 21:57:25,169 INFO
2010-09-29 21:57:25,174 INFO Writing image to disk <vmkctl.DiskLun; proxy of <Swig Object of type ‘HostCtl::Storage::DiskLun *’ at 0x976c718> > at /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0
2010-09-29 21:57:25,180 INFO calculating md5 sum of source file
2010-09-29 21:59:51,231 INFO Rescanning…
2010-09-29 21:59:51,316 WARNING Attempted to load iso9660 module twice
2010-09-29 21:59:51,316 DEBUG executing: /sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba32:C0:T0:L0
2010-09-29 21:59:51,541 DEBUG command exited with status 0
2010-09-29 21:59:51,543 INFO
2010-09-29 21:59:51,552 INFO found bootbank partition 5
2010-09-29 21:59:51,556 INFO found bootbank partition 6
2010-09-29 21:59:51,789 DEBUG Executing: fdisk -u /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0 < /tmp/fdisk_script
2010-09-29 21:59:52,000 INFO
The number of cylinders for this disk is set to 40960.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): Command action
l   logical (5 or over)
p   primary partition (1-4)
Partition number (1-4): First sector (1843200-83886079, default 1843200): Last sector or +size or +sizeM or +sizeK (1843200-83886079, default 83886079):
Command (m for help): Partition number (1-8): Hex code (type L to list codes): Changed system type of partition 2 to 6 (FAT16)

Command (m for help): Command action
l   logical (5 or over)
p   primary partition (1-4)
Selected partition 3
First sector (10229760-83886079, default 10229760): Last sector or +size or +sizeM or +sizeK (10229760-83886079, default 83886079): Using default value 83886079

Command (m for help): Partition number (1-8): Hex code (type L to list codes): Changed system type of partition 3 to fb (VMFS)

Command (m for help): The partition table has been altered!

Calling ioctl() to re-read partition table

2010-09-29 21:59:52,097 DEBUG   using auto-generated vmfs volume name — datastore1
2010-09-29 21:59:52,098 DEBUG executing: /usr/sbin/vmkfstools -C vmfs3 -b 1m -S datastore1 /vmfs/devices/disks/mpx.vmhba1:C0:T0:L0:3
2010-09-29 22:00:10,103 DEBUG command exited with status 0
2010-09-29 22:00:10,104 INFO Checking if remote hosts are using this device as a valid file system. This may take a few seconds…
Creating vmfs3 file system on “mpx.vmhba1:C0:T0:L0:3” with blockSize 1048576 and volume label “datastore1”.
Successfully created new volume: 4ca3b6e9-fd9ada85-a674-000c29e3db40

2010-09-29 22:00:10,105 WARNING A task finished that has not started
2010-09-29 22:00:10,105 INFO  * STEP ROOTPASS
2010-09-29 22:00:10,107 INFO  * STEP KEYBOARD
2010-09-29 22:00:10,108 INFO User has not chosen a keymap. Doing nothing.
2010-09-29 22:00:10,109 INFO  * STEP SERIAL
2010-09-29 22:00:10,110 INFO no license key entered, defaulting to evaluation mode
2010-09-29 22:00:10,110 INFO  * STEP Networking
2010-09-29 22:00:10,278 INFO Connecting vmnic0 via DHCP
2010-09-29 22:00:15,608 INFO Setting the host-wide networking options
2010-09-29 22:00:15,609 DEBUG using choices: {‘nameserver2’: None, ‘nameserver1’: None, ‘hostname’: None, ‘gateway’: None}
2010-09-29 22:00:15,649 INFO Setting hostname to localhost
2010-09-29 22:00:15,673 INFO Removing nameserver 192.168.108.2
2010-09-29 22:00:15,674 WARNING A task finished that has not started
2010-09-29 22:00:15,674 INFO  * STEP Scripts
2010-09-29 22:00:15,739 DEBUG executing: /sbin/vmkfstools -V
2010-09-29 22:00:15,968 DEBUG command exited with status 0
2010-09-29 22:00:15,969 INFO
2010-09-29 22:00:15,971 INFO Reading the boot.cfg file
2010-09-29 22:00:15,990 INFO Writing to the boot.cfg file
2010-09-29 22:00:16,038 INFO Staging the first-boot scripts in /tmp
2010-09-29 22:00:16,039 INFO Writing 61 chars to /tmp/etc/vmware/init/init.d/999.firstboot_remove
2010-09-29 22:00:16,040 INFO Writing 596 chars to /tmp/var/lib/firstboot/999.firstboot_remove.py
2010-09-29 22:00:16,042 INFO Writing 316 chars to /tmp/var/lib/firstboot/999.firstboot_password.py
2010-09-29 22:00:16,044 INFO Writing 63 chars to /tmp/etc/vmware/init/init.d/999.firstboot_password
2010-09-29 22:00:16,046 INFO Copying /etc/resolv.conf to /tmp/etc/resolv.conf
2010-09-29 22:00:16,047 DEBUG executing: /bin/busybox chmod +t /tmp/etc/resolv.conf
2010-09-29 22:00:16,155 DEBUG command exited with status 0
2010-09-29 22:00:16,156 INFO
2010-09-29 22:00:16,157 INFO Copying /etc/hosts to /tmp/etc/hosts
2010-09-29 22:00:16,158 DEBUG executing: /bin/busybox chmod +t /tmp/etc/hosts
2010-09-29 22:00:16,259 DEBUG command exited with status 0
2010-09-29 22:00:16,260 INFO
2010-09-29 22:00:16,261 INFO Copying /etc/vmware/esx.conf to /tmp/etc/vmware/esx.conf
2010-09-29 22:00:16,263 DEBUG executing: /bin/busybox chmod +t /tmp/etc/vmware/esx.conf
2010-09-29 22:00:16,365 DEBUG command exited with status 0
2010-09-29 22:00:16,365 INFO
2010-09-29 22:00:16,367 INFO Copying /var/log/esxi_install.log to /tmp/var/log/esxi_install.log

Tweezing apart ESXi image – Part 1

How to tweeze apart the ESXi 4.1 infrastructure – Part 1: The imagedd.bz2 file

What you need

  • ESXi 4.1 ISO image
  • Linux server, with the following tools:
    1. mount
    2. dd
    3. od
    4. gunzip
    5. bzip2
    6. tar

      Method

      Copy the files from the CD:

      # mount -o ro,loop -t iso9660 VMware-VMvisor-Installer-4.1.0-260247.x86_64.iso /mnt
      # cp -rp /mnt /extract/
      # umount /mnt
      # cd /extract
      # md5sum imagedd.bz2
      17e94a90a1b66c81d2f37718295b75c1  imagedd.bz2
      # cat imagedd.md5
      17e94a90a1b66c81d2f37718295b75c1  VMware-VMvisor-big-260247-x86_64.dd.bz2

      Within the imagedd.bz2 compressed file is a raw dump of a disk. The first sector of which is a partition table (MBR signature 0xaa55 [stored in disk in little-endian format])

      # bzip2 -d < imagedd.bz2 > imagedd
      # od -A d -j 446 -N 66 -t x1 -v imagedd
      0000446 00 00 01 04 05 3f e0 83 00 20 00 00 00 00 1c 00
      0000462 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0000478 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0000494 80 01 01 00 04 3f 20 03 20 00 00 00 e0 1f 00 00
      0000510 55 aa
      0000512

      From this we can see that there are 2 partitions.

      Remember that the top 2 bits of the sector byte are bits 9 and 10 of the cylinder byte. (0xE0 becomes 0x20, and 0x83 becomes 0x383):

      # echo $((0xE0 & 0x3F))
      32
      # echo $((0xE0 >> 6))
      3

      From this we can deduce that the geometry of the disk is 32 sector per track, since MBR partitions are usually whole-cylinder aligned.
      And it also looks like there are 64 heads in total, since the partitions start on head 0, and finish on head 63,. So this makes the CHS geometry of the disk  [nnnn/64/32]. (We don’t know the absolute size of the disk yet, but by looking at the size of the image, and knowing the sectors and heads, we can work out the geometry:

      # ls -al imagedd
      -rw-r--r--  1 root root 943718400 Sep 29 13:57 imagedd
      # echo $((943718400 / 64 / 32 / 512))
      900

      So we get CHS = 900/64/32

      Now onto the partitions:

      (partition 1, non-bootable, partition type 0x05, H/S/C start 0x00/0x01/0x04, H/S/C end 0x3f/0x20/0x383)
      (partition 4, bootable, partition type 0x04, H/S/C start 0x01/0x01/0x00, H/S/C end 0x3f/0x20/0x03)

      To convert to LBA addresses, you use the formula:

      LBA = (CylinderNumber * [HeadsPerCylinder * SectorsPerTrack]) + (HeadNumber * [SectorsPerTrack]) + SectorNumber – 1

      (-1 since blocks are addressed from 0, but all sectors start from 1!)

      So this works out for our partitions:

      Partition 4:
      starting block (0x00 * 64 * 32) + (0x01 * 32) + 0x01 – 1 = 32
      ending block (0x03 * 64 * 32) + (0x3f * 32) + 0x20 -1 = 8191

      Partition 1:
      starting block (0x04 * 64 * 32) + (0x00 * 32) + 0x01 – 1 = 8192
      ending block (0x383 * 64 * 32) + (0x3f * 32) + 0x20 -1 = 1843199

      We can also verify the LBA offsets which occur in the last 8 bytes of each 16 byte partition record.

      20 00 00 00 == 0x00000020 == 32 [start]
      e0 1f 00 00 == 0x00001fe0 == 8160 [length]
      00 20 00 00 == 0x00002000 == 8192 [start]
      00 00 1c 00 == 0x001c0000 == 1835008 [length]

      So the LBA addresses are in agreement with the C/H/S addresses!

      If only that were that easy! Partition 1 is actually a primary partition with partition type 5, which means extended partitions are present

      Dealing with Extended Partitions

      Unlike the Primary Partition Table, The format of Extended Boot Records is such that the record immediately preceeds the data that it points to, followed up another record and so on. Only one extended partition exists per EBR.

      The extended partitions are stored within the data section pointed to by the extended partition. We can display it in exactly the same way:

      # od -A d -j $((8192*512+446)) -N 66 -t x1 -v imagedd
      4194750 00 01 01 04 06 3f 20 fd 20 00 00 00 e0 cf 07 00
      4194766 00 00 01 fe 05 3f 60 f7 00 d0 07 00 00 d0 07 00
      4194782 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      4194798 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      4194814 55 aa
      4194816

      Extended partitions are numbered from 5.
      Going through the same process again, we have:

      (partition 5, non-bootable, partition type 0x06, H/S/C start 0x01/0x01/0x04, H/S/C end 0x3f/0x20/0xfd)
      Start 8224, End 520191 (and number of blocks according to LBA = 0x0007cfe0 == 511968)

      And the next line is not a partition, but a pointer to the next EBR. which is
      (next EBR = H/S/C start 0x00/0x01/0xfe, H/S/C end 0x3f/0x20/0x1f7)
      Start 520192, End 1032191

      Which is not surprising – the next EBR immediately follows partition 5.

      So now we must seek to this EBR partition:

      # od -A d -j $((520192*512 + 446)) -N 66 -t x1 -v imagedd
      266338750 00 01 01 fe 06 3f 60 f7 20 00 00 00 e0 cf 07 00
      266338766 00 00 41 f8 05 3f a0 65 00 a0 0f 00 00 70 03 00
      266338782 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      266338798 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      266338814 55 aa
      266338816

      And the existence of the 0xaa55 marker confirms we have found the next record.

      So this gives us:

      (partition 6 = non-bootable, partition type 0x06, H/S/C start 0x01/0x01/0xfe, H/S/C end 0x3f/0x20/0x1f7)
      Start 520224, End 1032191

      Next EBR = H/S/C start 0x00/0x01/0x1f8, H/S/C end 0x3f/0x20/0x265)
      Start 1032192, End 1257471

      # od -A d -j $((1032192*512 + 446)) -N 66 -t x1 -v imagedd
      528482750 00 01 41 f8 fc 3f a0 65 20 00 00 00 e0 6f 03 00
      528482766 00 00 81 66 05 3f e0 83 00 10 13 00 00 f0 08 00
      528482782 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      528482798 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      528482814 55 aa
      528482816

      This one gives us:

      (partition 7 = non-bootable, partition type 0xfc, H/S/C start 0x01/0x01/0x1f8, H/S/C end 0x3f/0x20/0x265)
      Start 1032224, End 1257471

      Next EBR = H/S/C start 0x00/0x01/0x266, H/S/C end 0x3f/0x20/0x383
      Start 1257472, End 1843199

      # od -A d -j $((1257472*512 + 446)) -N 66 -t x1 -v imagedd
      643826110 00 01 81 66 06 3f e0 83 20 00 00 00 e0 ef 08 00
      643826126 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      643826142 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      643826158 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      643826174 55 aa
      643826176

      And this time, since the next EBR record is blank, this will be the last extended partition.

      (partition 8 = non-bootable, partition type 0x06, H/S/C start 0x01/0x01/0x266, H/S/C end 0x3f/0x20/0x383)
      Start 1257504, End 1843199

      Summary

      So, in summary, the partition table looks like this:

      Partition 1 :    8192 - 1843199        Type 0x05 (Extended Partition)
      Partition 4 :      32 -    8191        Type 0x04
      Partition 5 :    8224 -  520191        Type 0x06
      Partition 6 :  520224 - 1032191        Type 0x06
      Partition 7 : 1032224 - 1257471        Type 0xFC
      Partition 8 : 1257504 - 1843199        Type 0x06
      

      Whilst we could have done this with “fdisk”, it’s nice to see how to do it manually.

      Now we know the partition offsets, we can tweeze apart this file further.

      Installing vCenter Orchestrator 4.1 (vCO)

      The steps to install a vCO standalone server are as follows:

      • Build a Windows 2008R2 VM [64 bit required]
      • Disable the firewall
      • Install the Powershell role
      • Install .NET 3.5SP1
      • Install MS-SQL Express. Use Windows authentication. Set a password for “sa” and root.
      • Create an SQL instance [say called VCO]. Enable TCP, disable dynamic ports, use port 1433.
      • Run the vCenterOrchestrator.exe program
      • Use the web configuration interface [default username vmware, password vmware]:
      • Configure Network
      • Configure LDAP (LDAP hosts, LDAP root path, UserID to search the domain, User Lookup Base, VCO Admin Group
      • Install and configure Database (user, password, database name, instance, Use Windows authentication)
      • Create SSL and server certificates
      • Install vCO licenses (point to vCenter Server)
      • Install vCO plugins (user id and password to vCenter Server)
      • Point vCO to a vCenter Server
      • Configure Start up options (Install vCO Server as a service)

      NB, there are two services, the vCO configuration server, and the vCO Server itself.

      Then login to the vCO server using the GUI. Strangely enough, you can’t use “localhost”

      And now you can create workflows:

      Correctly adding MTU for NAS fileshares

      When adding NFS datastores in ESXi, if NAS is used with jumbo frames, remember to set the MTU correctly at all the levels:

      • NAS interfaces
      • NAS switch ports
      • Intermediate switches [if any]
      • Uplink NIC switch port
      • vSwitch
      • vmk virtual interface

      From an ESXi perspective, the latter two are added as follows.

      Verify the NIC names:

      ~ # vmkchdev -l | grep -i nic
      000:002:00.0 8086:100f 15ad:0750 vmkernel vmnic0
      000:002:02.0 8086:100f 15ad:0750 vmkernel vmnic1
      000:002:03.0 8086:100f 15ad:0750 vmkernel vmnic2
      000:002:04.0 8086:100f 15ad:0750 vmkernel vmnic3
      000:002:05.0 8086:100f 15ad:0750 vmkernel vmnic4
      000:002:06.0 8086:100f 15ad:0750 vmkernel vmnic5
      

      Verify the current MTUs:

      ~ # esxcfg-nics -l
      Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
      vmnic0  0000:02:00.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:41 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      vmnic1  0000:02:02.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:4b 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      vmnic2  0000:02:03.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:55 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      vmnic3  0000:02:04.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:5f 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      vmnic4  0000:02:05.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:69 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      vmnic5  0000:02:06.00 e1000       Up   1000Mbps  Full   00:0c:29:95:93:73 1500   Intel Corporation PRO/1000 MT Single Port Adapter
      
      ~ # esxcfg-vswitch -l
      Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
      vSwitch0         128         3           128               1500    vmnic0
      
        PortGroup Name        VLAN ID  Used Ports  Uplinks
        Management Network    0        1           vmnic0
      

      Create a new switch, set the MTU, and the uplinks:

      ~ # esxcfg-vswitch -a vSwitch1
      ~ # esxcfg-vswitch -m 9000 vSwitch1
      ~ # esxcfg-vswitch -A NFS vSwitch1
      ~ # esxcfg-vswitch -L vmnic4 vSwitch1
      ~ # esxcfg-vswitch -L vmnic5 vSwitch1
      
      ~ # esxcfg-vswitch -l
      Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
      vSwitch0         128         3           128               1500    vmnic0
      
        PortGroup Name        VLAN ID  Used Ports  Uplinks
        Management Network    0        1           vmnic0
      
      Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks
      vSwitch1         128         4           128               9000    vmnic4,vmnic5
      
        PortGroup Name        VLAN ID  Used Ports  Uplinks
        NFS                   0        1           vmnic4,vmnic5
      

      Add the NFS portgroup, either with DHCP or static IP:

      ~ # esxcfg-vmknic -a -i DHCP -m 9000 -p NFS
      ~ # esxcfg-vmknic -a -i 10.16.123.123 -n 255.255.255.0 -m 9000 -p NFS
      ~ # esxcfg-vmknic -l
      Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
      vmk0       Management Network  IPv4      192.168.0.123                           255.255.255.0   192.168.0.255   00:0c:29:95:93:41 1500    65535     true    DHCP
      vmk1       NFS                 IPv4      10.16.123.123                           255.255.255.0   10.16.123.255   00:50:56:7e:b1:88 9000    65535     true    STATIC
      

      Add the NFS share:

      ~ # esxcfg-nas -a -o 10.16.123.100 -s /vol/someshare/somedir friendlyname_01
      Connecting to NAS volume: friendlyname_01
      friendlyname_01 created and connected.
      

      Russian roulette of silent corruption

      Frequently, I see checksum errors in ZFS. This might seem like a bad thing, but actually, I’m pleased.

      I’m pleased that ZFS has detected silent corruption in zpools via its end-to-end checksumming.

      Reading the blurb from Sun, you get :

      With ZFS, all data and metadata is verified using a user-selectable checksum algorithm. Traditional file systems that do provide checksum verification have performed it on a per-block basis, out of necessity due to the volume management layer and traditional file system design. The traditional design means that certain failures, such as writing a complete block to an incorrect location, can result in data that is incorrect but has no checksum errors. ZFS checksums are stored in a way such that these failures are detected and can be recovered from gracefully. All checksum verification and data recovery are performed at the file system layer, and are transparent to applications.

      In addition, ZFS provides for self-healing data. ZFS supports storage pools with varying levels of data redundancy. When a bad data block is detected, ZFS fetches the correct data from another redundant copy and repairs the bad data, replacing it with the correct data.

      Here you can actually see checksum errors. These didn’t come from the disk, but from something in-between.

      On any other filesystem, this would have led to silent corruption (or bit rot), but on ZFS, it has detected and corrected the problem.

      # zpool status 
        pool: rpool
       state: ONLINE
      status: One or more devices has experienced an unrecoverable error.  An
              attempt was made to correct the error.  Applications are unaffected.
      action: Determine if the device needs to be replaced, and clear the errors
              using 'zpool clear' or replace the device with 'zpool replace'.
         see: http://www.sun.com/msg/ZFS-8000-9P
       scan: scrub repaired 896K in 5h29m with 0 errors on Tue Sep 21 15:06:00 2010
      config:
      
              NAME          STATE     READ WRITE CKSUM
              rpool         ONLINE       0     0     0
                mirror-0    ONLINE       0     0     0
                  c8t2d0s0  ONLINE       0     0     0
                  c8t1d0s0  ONLINE       0     0     2
                  c8t0d0s0  ONLINE       0     0     5
      
      errors: No known data errors

      If you think that this is reported from the device directly, you’re wrong! As you can see below, no media, transport, hard or soft errors have been reported:

      c8t0d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
      Vendor: ATA      Product: ST31000333AS     Revision: CC1F Serial No:
      Size: 1000.20GB <1000204886016 bytes>
      Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
      Illegal Request: 7 Predictive Failure Analysis: 0
      
      c8t1d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
      Vendor: ATA      Product: WDC WD10EADS-00L Revision: 1A01 Serial No:
      Size: 1000.20GB <1000204886016 bytes>
      Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
      Illegal Request: 7 Predictive Failure Analysis: 0
      
      c8t2d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
      Vendor: ATA      Product: WDC WD10EADS-00L Revision: 1A01 Serial No:
      Size: 1000.20GB <1000204886016 bytes>
      Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
      Illegal Request: 8 Predictive Failure Analysis: 0
      
      c8t3d0           Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
      Vendor: ATA      Product: ST31000333AS     Revision: CC1F Serial No:
      Size: 1000.20GB <1000204886016 bytes>
      Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
      Illegal Request: 0 Predictive Failure Analysis: 0
      

      One security patch for ESXi only.

      Contrary to some MS zealots view, there has only been a single security patch for ESXi 4.x, and that affects NTP only.

      I’ve filtered out the non-ESXi (ie ESX) patches.