Upgrading Solaris 11 SRU 12.4 to Solaris 11 Update 1

As per these instructions:

http://www.oracle.com/technetwork/articles/servers-storage-admin/howto-update-11dot1-ips-1866781.html

it seemed to work ok.

The only trick was to read the instructions, and remove the DNS service first before upgrading:



root@marvin:~# pkg uninstall pkg://solaris/service/network/dns/bind@9.6.3.7.3,5.11-0.175.0.12.0.4.0:20121002T160623Z
            Packages to remove:  1
       Create boot environment: No
Create backup boot environment: No
            Services to change:  1

PHASE                                        ACTIONS
Removal Phase                                  35/35

PHASE                                          ITEMS
Package State Update Phase                       1/1
Package Cache Update Phase                       1/1
Image State Update Phase                         2/2
root@marvin:~# pkg uninstall pkg://solaris/network/dns/bind
            Packages to remove:  1
       Create boot environment: No
Create backup boot environment: No

PHASE                                        ACTIONS
Removal Phase                                  45/45

PHASE                                          ITEMS
Package State Update Phase                       1/1
Package Cache Update Phase                       1/1
Image State Update Phase                         2/2

root@marvin:~# pkg update --be-name s11.1ga --accept
------------------------------------------------------------
Package: pkg://solaris/consolidation/osnet/osnet-incorporation@0.5.11,5.11-0.175.1.0.0.24.2:20120919T184141Z
License: usr/src/pkg/license_files/lic_OTN

Oracle Technology Network Developer License Agreement

Oracle Solaris, Oracle Solaris Cluster and Oracle Solaris Express

EXPORT CONTROLS
Selecting the "Accept License Agreement" button is a confirmation
of your agreement that you comply, now and during the trial term
(if applicable), with each of the following statements:

-You are not a citizen, national, or resident of, and are not under
control of, the government of Cuba, Iran, Sudan, North Korea, Syria,
or any country to which the United States has prohibited export.

-You will not download or otherwise export or re-export the Programs
(as defined below), directly or indirectly, to the above mentioned
countries or to citizens, nationals or residents of those countries.

-You are not listed on the United States Department of Treasury lists
of Specially Designated Nationals, Specially Designated Terrorists,
and Specially Designated Narcotic Traffickers, nor are you listed on
the United States Department of Commerce Table of Denial Orders.

You will not download or otherwise export or re-export the Programs,
directly or indirectly, to persons on the above mentioned lists.

You will not use the Programs for, and will not allow the Programs to
be used for, any purposes prohibited by United States law, including,
without limitation, for the development, design, manufacture or
production of nuclear, chemical or biological weapons of mass
destruction.

EXPORT RESTRICTIONS
You agree that U.S. export control laws and other applicable export
and import laws govern your use of the Programs, including technical
data; additional information can be found on Oracle's Global Trade
Compliance web site (http://www.oracle.com/products/export).

You agree that neither the Programs nor any direct product thereof
will be exported, directly, or indirectly, in violation of these
laws, or will be used for any purpose prohibited by these laws
including, without limitation, nuclear, chemical, or biological
weapons proliferation.


Oracle Employees: Under no circumstances are Oracle Employees
authorized to download software for the purpose of distributing it to
customers. Oracle products are available to employees for internal
use or demonstration purposes only. In keeping with Oracle's trade
compliance obligations under U.S. and applicable multilateral law,
failure to comply with this policy could result in disciplinary action
up to and including termination.

PLEASE READ THE FOLLOWING LICENSE AGREEMENT TERMS AND CONDITIONS
CAREFULLY BEFORE DOWNLOADING, INSTALLING OR USING THE PROGRAMS.
THESE TERMS AND CONDITIONS CONSTITUTE A LEGAL AGREEMENT BETWEEN YOU
AND ORACLE.

Oracle Technology Network Development License Agreement
"We," "us," and "our" refers to Oracle America, Inc., for and on
behalf of itself and its subsidiaries and affiliates under common
control. "You" and "your" refers to the individual or entity that
wishes to use the Programs. "Programs" refers to Oracle Solaris,
Oracle Solaris Cluster and/or Oracle Solaris Express that you download
from this site (including, any updates and/or additional software
packages that you may receive in the future from the product's
package repository) and related program documentation. "License"
refers to your right to use the Programs under the terms of this
agreement. This agreement is governed by California law, except
for that body of laws related to the conflict of laws. You agree to
submit to the exclusive jurisdiction of, and venue in, the courts of
San Francisco or Santa Clara counties in California in any dispute
arising out of or relating to this agreement.

In order to use the Programs, You must first agree to this License
Agreement by selecting the "Accept License Agreement" button below. If
You do not or cannot agree to this License Agreement, You are not
permitted to download or use the Programs.

LICENSE RIGHTS
Except for any included software package or file that is licensed to
you by Oracle under different license terms, we grant you a perpetual
(unless terminated as provided in this agreement), nonexclusive,
nontransferable, limited License to use the Programs only for the
purpose of developing, testing, prototyping and demonstrating your
applications, and not for any other purpose.

All rights not expressly granted above are hereby reserved.  If you
want to use the Programs for any purpose other than as permitted
under this agreement, including but not limited to distribution of the
Programs or any use of the Programs for your internal business purposes
(other than developing, testing, prototyping and demonstrating your
applications) or for any commercial production purposes, you must
obtain a valid license permitting such use. We may audit your use of
the Programs.   Program documentation, if available, may be accessed
online at http://otn.oracle.com/docs.

Third-Party Technology.  The Programs may contain or be distributed
with certain third-party technology. Oracle may provide certain notices
related to such third-party technology in the program documentation,
or in readme or notice files provided with the Programs.

Third party technology will be licensed to you either under the terms
of this agreement, or, if specified in the program documentation,
readme files or otherwise in writing, under separate license
terms ("Separate Terms") and not under the terms of this agreement
("Separately Licensed Third Party Technology"). Licensee's rights to
use such Separately Licensed Third Party Technology under the Separate
Terms are not restricted or modified in any way by this Agreement.

Information Collection and Registration
Configuration Data and Registration:  The Programs may communicate
configuration data to Oracle. You can register your version of
the Programs to capture this data for your use, or the data is
sent anonymously.  For information about what configuration data is
communicated and how to control this facility, refer to the release
notes or www.oracle.com/goto/solarisautoreg.

Ownership and Restrictions
We retain all ownership and intellectual property rights in the
Programs. The Programs may be installed on one computer only, and
used by one person in the hardware environment identified by us. You
may make one copy of the Programs for backup purposes.

You may not:
- use the Programs for your own internal business purposes (other than
developing, testing, prototyping and demonstrating your applications)
or for any commercial or production purposes;

- remove or modify any program markings or any notice of our
proprietary rights;

- make the Programs available in any manner to any third party;

- use the Programs to provide third-party training;

- assign this agreement or give or transfer the Programs or an interest
in them to another individual or entity;

- cause or permit reverse engineering (unless required by law for
interoperability), disassembly or decompilation of the Programs;

- disclose results of any benchmark test results related to the
Programs without our prior consent.

Export
You agree that U.S. export control laws and other applicable
export and import laws govern your use of the Programs,
including technical data; additional information can be
found on Oracle's Global Trade Compliance web site located at
http://www.oracle.com/products/export/index.html?content.html. You
agree that neither the Programs nor any direct product thereof will be
exported, directly, or indirectly, in violation of these laws, or will
be used for any purpose prohibited by these laws including, without
limitation, nuclear, chemical, or biological weapons proliferation.

Disclaimer of Warranty and Exclusive Remedies
THE PROGRAMS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. WE
FURTHER DISCLAIM ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING
WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT.

IN NO EVENT SHALL WE BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL,
PUNITIVE OR CONSEQUENTIAL DAMAGES, OR DAMAGES FOR LOSS OF PROFITS,
REVENUE, DATA OR DATA USE, INCURRED BY YOU OR ANY THIRD PARTY, WHETHER
IN AN ACTION IN CONTRACT OR TORT, EVEN IF WE HAVE BEEN ADVISED OF
THE POSSIBILITY OF SUCH DAMAGES. OUR ENTIRE LIABILITY FOR DAMAGES
HEREUNDER SHALL IN NO EVENT EXCEED ONE THOUSAND DOLLARS (U.S. $1,000).

Additional Trial Programs
We may include additional trial programs with your download of the
Programs licensed under this agreement. You will have 30 days from
the delivery date to evaluate these additional trial programs. Any
use of these trial programs after the 30-day trial period requires
you to obtain the applicable license.  Any additional trial programs
are provided "as is" and we do not provide technical support or any
warranties for these programs.

No Technical Support
Our technical support organization will not provide technical support,
phone support, or updates to you for the Programs licensed under
this agreement.

End of Agreement
You may terminate this agreement by destroying all copies of the
Programs. We have the right to terminate your right to use the Programs
if you fail to comply with any of the terms of this agreement, in
which case you shall destroy all copies of the Programs.

Relationship Between the Parties
The relationship between you and us is that of
licensee/licensor. Neither party will represent that it has any
authority to assume or create any obligation, express or implied,
on behalf of the other party, nor to represent the other party as
agent, employee, franchisee, or in any other capacity. Nothing in
this agreement shall be construed to limit either party's right to
independently develop or distribute software that is functionally
similar to the other party's products, so long as proprietary
information of the other party is not included in such software.

Open Source Software
"Open Source" software - software available without charge for
use, modification and distribution - is often licensed under terms
that require the user to make the user's modifications to the Open
Source software or any software that the user 'combines' with the
Open Source software freely available in source code form. If you use
Open Source software in conjunction with the Programs (or if you plan
on licensing your own application under an Open Source license), you
must ensure that your use does not: (i) create, or purport to create,
obligations with respect to the Oracle Programs; or (ii) grant,
or purport to grant, to any third party any rights to or immunities
under our intellectual property or proprietary rights in the Oracle
Programs.  For example, you may not develop a software program using
an Oracle program and an Open Source program where such use results
in a program file(s) that contains code from both the Oracle program
and the Open Source program (including without limitation libraries)
if the Open Source program is licensed under a license that requires
any "modifications" be made freely available. You also may not combine
the Oracle program with programs licensed under the GNU General Public
License ("GPL") in any manner that could cause, or could be interpreted
or asserted to cause, the Oracle program or any modifications thereto
to become subject to the terms of the GPL.

Entire Agreement
You agree that this agreement is the complete agreement for the
Programs and licenses, and this agreement supersedes all prior or
contemporaneous agreements or representations, including any and
all clickwrap, shrinkwrap or similar licenses.  If any term of this
agreement is found to be invalid or unenforceable, the remaining
provisions will remain effective.

Last updated: 08/12/10

Should you have any questions concerning this License Agreement,
or if you desire to contact Oracle for any reason, please write:

Oracle America, Inc.
500 Oracle Parkway,
Redwood City, CA 94065

Oracle may contact you to ask if you had a satisfactory experience
installing and using this OTN software download.



            Packages to remove:   2
           Packages to install:  44
            Packages to update: 744
           Mediators to change:   2
       Create boot environment: Yes
Create backup boot environment:  No

DOWNLOAD                                  PKGS       FILES    XFER (MB)
system/management/ocm                   11/790  1864/36401   29.9/772.6
Completed                              790/790 36401/36401  772.6/772.6

PHASE                                        ACTIONS
Removal Phase                              9216/9216
Install Phase                            20162/20162
Update Phase                             29908/29908

PHASE                                          ITEMS
Package State Update Phase                 1532/1532
Package Cache Update Phase                   745/745
Image State Update Phase                         2/2

A clone of test exists and has been updated and activated.
On the next boot the Boot Environment s11.1ga will be
mounted on '/'.  Reboot when ready to switch to this updated BE.

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=REFERENCE&id=1372094.1
---------------------------------------------------------------------------

root@marvin:~# beadm mount s11.1ga /mnt
root@marvin:~# pkg -R /mnt install pkg://solaris/service/network/dns/bind
           Packages to install:  2
       Create boot environment: No
Create backup boot environment: No
            Services to change:  1

DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                                  2/2       30/30      1.8/1.8

PHASE                                        ACTIONS
Install Phase                                  85/85

PHASE                                          ITEMS
Package State Update Phase                       2/2
Image State Update Phase                         2/2
root@marvin:~# beadm umount s11.1ga
root@marvin:~# init 6

ESXi 4.1 booting, extensions, driver installs, manifest files etc

I’ve been wondering – where is the list of drivers, and which file contains the manifest.

I noticed that /tmp/stage is populated, even at boot time. It appears that init calls esxupdate (a python program), which creates a visor filesystem (ramdisk) on /tmp/stage, and then copies /var/db/esxupdate/* and the contents of /bootbank/m.z into it:

# ls -al /var/db/esxupdate
drwxr-xr-x    1 root     root                512 Oct 16 04:12 .
drwxr-xr-x    1 root     root                512 Oct 16 04:12 ..
-rw-r–r–    1 root     root                938 Oct 16 04:10 custom-pkgdb.tgz
-rw-r–r–    1 root     root               1311 Aug 15 05:28 oem-pkgdb.tgz
-rw-r–r–    1 root     root               1221 Aug  2 21:03 pkgdb.tgz

# mount -v | grep /tmp/stage
updatestg on /tmp/stage type visorfs (0,750,01777,updatestg)

~ # ls -al /tmp/stage/customstage/usr/lib/ipkg/info
drwxr-xr-x    1 root     root                512 Oct 16 04:12 .
drwxr-xr-x    1 root     root                512 Oct 16 04:12 ..
-rw-r–r–    1 root     root                310 Jan  4  2011 vmware-esx-drivers-net-be2net.control
-rw-r–r–    1 root     root                171 Oct 16 04:09 vmware-esx-drivers-net-be2net.list
-rwxr-xr-x    1 root     root                599 Jan  4  2011 vmware-esx-drivers-net-be2net.postinst
-rwxr-xr-x    1 root     root                599 Jan  4  2011 vmware-esx-drivers-net-be2net.postrm
-rwxr-xr-x    1 root     root                 87 Jan  4  2011 vmware-esx-drivers-net-be2net.preinst
-rwxr-xr-x    1 root     root                 90 Jan  4  2011 vmware-esx-drivers-net-be2net.prerm

The question is, how did this get here in the first place – lets start with a clean system:

# tar tzvf /bootbank/m.z
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 usr/lib/vmware/vmkmod/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/vmware/init/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/vmware/pciid/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 sbin/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 usr/lib/ext/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 lib/cim/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/cim/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 var/lib/sfcb/registration/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/vmware/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/vmware/init/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/vmware/pciid/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/ipmi/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 usr/lib/pycim/
-rw-r–r– 0/0       157 2012-08-15 05:28:50 var/db/esxupdate/custom-pkgdb.tgz

Put ESXi into maintenance mode, if it’s not already:

# vim-cmd hostsvc/hostsummary | grep -i maintenance
inMaintenanceMode = false,
# vim-cmd hostsvc/maintenance_mode_enter
# vim-cmd hostsvc/hostsummary | grep -i maintenance
inMaintenanceMode = true,

Copy in a VIB file (eg 10GbE driver):

# scp root@172.16.170.1:/tmp/vmware* /tmp

Host ‘172.16.170.1’ is not in the trusted hosts file.
(fingerprint md5 b5:46:81:12:22:3b:1b:6d:d7:9e:11:d4:20:2f:06:c1)
Do you want to continue connecting? (y/n) yes
Login for root@172.16.170.1
Password:
vmware-esx-drivers-net-be2net-400.2.102.554.0 100%   80KB  80.1KB/s   00:00

Install the VIB:

# esxupdate -b /tmp/vmware-esx-drivers-net-be2net-400.2.102.554.0-1vmw.2.17.249663.x86_64.vib update
Unpacking cross_vmware-esx-dr.. ######################################## [100%]

Installing packages :cross_vm.. ######################################## [100%]

Running [/usr/sbin/vmkmod-install.sh]…
ok.
The update completed successfully, but the system needs to be rebooted for the
changes to be effective.

Example of bootbank before reboot is called. Notice that m.z hasn’t changed:

# ls -altr /bootbank/
-rwx——    1 root     root              48339 Aug 14 22:28 b.z
-rwx——    1 root     root              11225 Aug 14 22:28 a.z
-rwx——    1 root     root              41155 Aug 14 22:28 tboot.gz
-rwx——    1 root     root           72786024 Aug 14 22:28 s.z
-rwx——    1 root     root             950084 Aug 14 22:28 oem.tgz
-rwx——    1 root     root                551 Aug 14 22:28 m.z
-rwx——    1 root     root                137 Aug 14 22:28 license.tgz
-rwx——    1 root     root            2489796 Aug 14 22:28 k.z
-rwx——    1 root     root            1162824 Aug 14 22:28 cimstg.tgz
-rwx——    1 root     root           15283377 Aug 14 22:28 c.z
-rwx——    1 root     root                169 Aug 14 22:28 vibddi
-rwx——    1 root     root              10101 Oct 16 03:39 state.tgz
-rwx——    1 root     root                143 Oct 16 03:39 boot.cfg
-rwx——    1 root     root               2511 Oct 16 03:53 pkgdb.tgz

# reboot

NB. If you don’t reboot a host cleanly, the updates will get lost, since m.z is only assembled on reboot

After reboot, m.z has changed substantially:

# ls -altr /bootbank/
-rwx——    1 root     root            2489796 Oct 16 03:53 k.z
-rwx——    1 root     root            1162824 Oct 16 03:53 cimstg.tgz
-rwx——    1 root     root           15283377 Oct 16 03:53 c.z
-rwx——    1 root     root              48339 Oct 16 03:53 b.z
-rwx——    1 root     root              11225 Oct 16 03:53 a.z
-rwx——    1 root     root                169 Oct 16 03:53 vibddi
-rwx——    1 root     root              41155 Oct 16 03:53 tboot.gz
-rwx——    1 root     root              10417 Oct 16 03:53 state.tgz
-rwx——    1 root     root           72786024 Oct 16 03:53 s.z
-rwx——    1 root     root             950084 Oct 16 03:53 oem.tgz
-rwx——    1 root     root              73422 Oct 16 03:53 m.z
-rwx——    1 root     root                137 Oct 16 03:53 license.tgz
-rwx——    1 root     root               2464 Oct 16 03:56 pkgdb.tgz
-rwx——    1 root     root                143 Oct 16 03:56 boot.cfg

And indeed contains the be2net.o driver:

# vdu /usr/lib/vmware/vmkmod/be2net.o
For ‘/usr/lib/vmware/vmkmod/be2net.o’:
tardisk SYS5:       245248

# tar tzvf /bootbank/m.z
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 usr/lib/vmware/vmkmod/
-rw-r–r– 201/201    244877 2011-01-11 01:59:55 usr/lib/vmware/vmkmod/be2net.o
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/init/
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/init/manifests/
-rw-r–r– 201/201        36 2011-01-04 21:10:52 etc/vmware/init/manifests/vmware-be2net.mf
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/pciid/
-rw-r–r– 201/201      1337 2011-01-04 21:10:52 etc/vmware/pciid/be2net.xml
drwxr-xr-x 0/0         0 2012-10-16 03:53:26 sbin/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 usr/lib/ext/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 lib/cim/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/cim/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 var/lib/sfcb/registration/
drwxr-xr-x 201/201         0 2011-01-11 01:59:54 etc/vmware/
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/init/
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/init/manifests/
-rw-r–r– 201/201        36 2011-01-04 21:10:52 etc/vmware/init/manifests/vmware-be2net.mf
drwxr-xr-x 201/201         0 2012-10-16 03:53:25 etc/vmware/pciid/
-rw-r–r– 201/201      1337 2011-01-04 21:10:52 etc/vmware/pciid/be2net.xml
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 etc/ipmi/
drwxr-xr-x 0/0         0 2012-08-15 05:28:50 usr/lib/pycim/
-rw-r–r– 0/0       936 2012-10-16 03:53:26 var/db/esxupdate/custom-pkgdb.tgz

You can now leave maintenance mode, if you wish:

~ # vim-cmd hostsvc/hostsummary | grep -i maintenance
inMaintenanceMode = true,
~ # vim-cmd hostsvc/maintenance_mode_exit
‘vim.Task:haTask-ha-host-vim.HostSystem.exitMaintenanceMode-18’
~ # vim-cmd hostsvc/hostsummary | grep -i maintenance
inMaintenanceMode = false,

And you can verify the files are present in the filesystem, that are listed in the manifest file:

# cat /tmp/stage/customstage/usr/lib/ipkg/info/vmware-esx-drivers-net-be2net.list
/tmp/stage/customstage/etc/vmware/init/manifests/vmware-be2net.mf
/tmp/stage/customstage/etc/vmware/pciid/be2net.xml
/tmp/stage/customstage/usr/lib/vmware/vmkmod/be2net.o

But, if you reset the system configuration, the driver and configs will be lost. (There is no state.tgz, nor local.tgz file):

[After Reset System Configuration] :

~ # ls -altr /bootbank/
-rwx——    1 root     root            2489796 Oct 16 03:53 k.z
-rwx——    1 root     root            1162824 Oct 16 03:53 cimstg.tgz
-rwx——    1 root     root           15283377 Oct 16 03:53 c.z
-rwx——    1 root     root              48339 Oct 16 03:53 b.z
-rwx——    1 root     root              11225 Oct 16 03:53 a.z
-rwx——    1 root     root                169 Oct 16 03:53 vibddi
-rwx——    1 root     root              41155 Oct 16 03:53 tboot.gz
-rwx——    1 root     root           72786024 Oct 16 03:53 s.z
-rwx——    1 root     root             950084 Oct 16 03:53 oem.tgz
-rwx——    1 root     root                137 Oct 16 03:53 license.tgz
-rwx——    1 root     root               2464 Oct 16 03:56 pkgdb.tgz
-rwx——    1 root     root                 92 Oct 16 04:02 m.z
-rwx——    1 root     root                129 Oct 16 04:02 boot.cfg

~ # tar tvzf /bootbank/m.z
-rw-r–r– 0/0         0 2012-10-16 04:02:32 .emptytgz

 

More on how to keep the driver in the next post

Removing phantom vmk nics from Cisco 1000V DVS switches

It’s not possible to use the esxcfg-vmknic command with the “-d” option with Cisco Switches.

The easiest way I found was to edit the /etc/vmware/esx.conf file and then reread:

Before:

~ # esxcfg-vmknic -l
Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
vmk3       366                 IPv4      192.168.230.15                          255.255.254.0   192.168.231.255 00:50:56:7f:56:02 1500    65535     true    STATIC
vmk2       378                 IPv4      192.168.226.15                          255.255.255.0   192.168.226.255 00:50:56:79:68:c6 1500    65535     true    STATIC
vmk1       306                 IPv4      10.47.40.16                             255.255.255.0   10.47.40.255    00:00:c9:c8:64:68 1500    65535     true    STATIC
vmk0       304                 IPv4      10.47.40.15                             255.255.255.0   10.47.40.255    00:50:56:77:75:e8 0       0         false   STATIC

~ # vi /etc/vmware/esx.conf
change:
/adv/Net/ManagementIface = "vmk0"
/adv/Migrate/Vmknic = "vmk1"
/adv/FT/Vmknic = "vmk2"

change the index to go, 0000, 0001, 0002, etc for vmkN:
eg vmk0:
/net/vmkernelnic/child[0000]/connectionId = "1961091145"
/net/vmkernelnic/child[0000]/dhcp = "false"
/net/vmkernelnic/child[0000]/dhcpDns = "false"
/net/vmkernelnic/child[0000]/dhcpv6 = "false"
/net/vmkernelnic/child[0000]/dvpId = "306"
/net/vmkernelnic/child[0000]/dvsId = "de 0b 2e 50 7a 5e 86 3d-91 44 25 e5 34 f3
/net/vmkernelnic/child[0000]/enable = "true"
/net/vmkernelnic/child[0000]/ipv4address = "10.47.40.16"
/net/vmkernelnic/child[0000]/ipv4broadcast = "10.47.40.255"
/net/vmkernelnic/child[0000]/ipv4netmask = "255.255.255.0"
/net/vmkernelnic/child[0000]/mac = "00:00:c9:c8:64:68"
/net/vmkernelnic/child[0000]/macFromPnic = "vmnic4"
/net/vmkernelnic/child[0000]/mtu = "1500"
/net/vmkernelnic/child[0000]/name = "vmk0"
/net/vmkernelnic/child[0000]/routAdv = "false"
/net/vmkernelnic/child[0000]/tsoMss = "0"
/net/vmkernelnic/child[0001]/connectionId = "976695119"
/net/vmkernelnic/child[0001]/dhcp = "false"

After:

~ # esxcfg-vmknic -r
[2012-08-08 04:09:50 'NotifyDCUI' warning] Notifying the DCUI of configuration change
[2012-08-08 04:09:50 'NotifyDCUI' warning] Notifying the DCUI of configuration change
[2012-08-08 04:09:50 'IpSecConfig' warning] Ipv6 not Enabled

~ # esxcfg-vmknic -l
Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type
vmk0       306                 IPv4      10.47.40.16                             255.255.255.0   10.47.40.255    00:00:c9:c8:64:68 1500    65535     true    STATIC
vmk1       378                 IPv4      192.168.226.15                          255.255.255.0   192.168.226.255 00:50:56:79:68:c6 1500    65535     true    STATIC
vmk2       366                 IPv4      192.168.230.15                          255.255.254.0   192.168.231.255 00:50:56:7f:56:02 1500    65535     true    STATIC


Logging into ESXi’s Console and SSH as an Active Directory user

It is possible to login as an AD-user, once ESXi has joined to the domain, and it does use the password stored in AD.

[root@somehost ~]# ssh 10.0.0.115 -l christow@lab.somedomain.com
Password:
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see http://www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ $ id
uid=1409287390(LAB\christow) gid=1409286657(LAB\domain^users)

~ $ cat /etc/security/access.conf
+:dcui:ALL
+:root:ALL
+:vpxuser:ALL
+:vslauser:ALL
+:LAB\christow:ALL
-:ALL:ALL

UPN notation doesn’t work. You have to use NetBIOS style user principal names.

Having “+:christow@lab.somedomain.com:ALL” does not work.

You can use the NetBIOS name to log in via SSH. Protect the backslash within double quotes:

[root@somehost ~]# ssh 10.0.0.115 -l “LAB\christow”
Password:
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see http://www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.
~ $ Connection to 10.0.0.115 closed.

Both UPN style names and NetBIOS names work on the Alt-F1 Recovery Console (no quotes and single backslash for NetBIOS name works fine.

To stop the system from replacing /etc/security/access.conf, the vicfg-user command needs to be run from a VIMA system to give “admin” access:

[vi-admin@labvma01 ~][10.0.0.115]$ vicfg-user -e user -o modify -l LAB\\christow -r admin

Backing up the ESXi hypervisor before playing with custom VIBs.

If you copy the /bootbank files to /altbootbank, you can recover these by pressing Shift-R when ESXi boots.

This reverses the symlinks for /altbootbank and /bootbank:

~ # ls -ald /*bootbank*
lrwxrwxrwx    1 root     root                 49 Jun  2 21:13 /altbootbank 
-> /vmfs/volumes/702aea7e-a9164be0-0df5-0cf76d6e11b9
lrwxrwxrwx    1 root     root                 49 Jun  2 21:13 /bootbank 
-> /vmfs/volumes/2be37f76-1609a97d-31db-e8ed56893060
~ # cp /bootbank/* /altbootbank 

Reboot, and press Shift-R. Press Y to confirm. After boot:

~ # ls -ald /*bootbank* lrwxrwxrwx    1 root     root                 49 Jun  2 21:27 /altbootbank 
-> /vmfs/volumes/2be37f76-1609a97d-31db-e8ed56893060
lrwxrwxrwx    1 root     root                 49 Jun  2 21:27 /bootbank 
-> /vmfs/volumes/702aea7e-a9164be0-0df5-0cf76d6e11b9

Adding non root access to ESXi

ESXi, is POSIX-enough that you can add a user to /etc/shadow, /etc/passwd, and /etc/group.

Console Access:

However, in order to allow the user to login directly the /etc/security/access.conf file has to be edited:

/etc/security/access.conf :

+:dcui:ALL
+:root:ALL
+:vpxuser:ALL
+:vslauser:ALL
+:foo:ALL
-:ALL:ALL

SSH Access:

If passwords are used, then the previous change needs to be made. Alternatively, SSH-keys can be used.

SSH keys are stored in /etc/ssh/keys-<username>/authorized_keys.

However, that file needs to be owned by the user.

eg:

[root@somehost ~]# ssh -v foo@10.0.0.115
OpenSSH_4.3p2, OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 10.0.0.115 [10.0.0.115] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/identity type -1
debug1: identity file /root/.ssh/id_rsa type 1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: loaded 3 keys
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.6
debug1: match: OpenSSH_5.6 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_4.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<2048<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '10.0.0.115' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:53
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/identity
debug1: Offering public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
The time and date of this login have been sent to the system logs.

VMware offers supported, powerful system administration tools.  Please
see www.vmware.com/go/sysadmintools for details.

The ESXi Shell can be disabled by an administrative user. See the
vSphere Security documentation for more information.

~ $ id
uid=502(foo) gid=502(foo)

~ $ ls -al /etc/ssh/keys-foo
drwxr-xr-x    1 root     root                512 Jun  2 10:19 .
drwxr-xr-x    1 root     root                512 Jun  2 10:12 ..
-rw------T    1 foo      foo                 399 Jun  2 08:43 authorized_keys

~ $ exit 

However, the big problem is that ownership of VIB files goes back to root on mount time:

~ # ls -al /etc/ssh/keys-foo
drwxr-xr-x 1 root root 512 Jun 2 21:12 .
drwxr-xr-x 1 root root 512 Jun 2 21:12 ..
-rw------T 1 root root 399 Jun 2 18:43 authorized_keys

Script to create a custom VIB in ESXi 5.0

This script can be used to maintain a list of files to mount automatically at boot time. The list of files must also include the parent directories, if those are not present in the default directory hierarchy.

/createcustom.sh:

#!/bin/sh
# /createcustom.sh
# Chris Wells 20120602
# Put the list of files to include in file /customfiles
# Then add “— custom.v00” to the boot options in /bootbank/boot.cfg
# Test it by:
# vmkramdisk /bootbank/custom.v00

cd /
tar cvf /tmp/custom.tar `cat /customfiles` createcustom.sh customfiles
vmtar -o /tmp/custom.vmtar -c /tmp/custom.tar
rm -f /tmp/custom.tar
gzip -9 < /tmp/custom.vmtar > /tmp/custom.v00
rm -f /tmp/custom.vmtar
cp /tmp/custom.v00 /bootbank
rm -f /tmp/custom.v00

eg, I’ve used this to create ssh keys for a non-root user:

~ # /createcustom.sh 
etc/security/access.conf
etc/ssh/keys-foo/
etc/ssh/keys-foo/authorized_keys
createcustom.sh
customfiles
Follow

Get every new post delivered to your Inbox.

Join 91 other followers