18 Nov 2005

For resource control stuff in particular, as well as lots of insight into Zones, see http://users.tpg.com.au/adsln4yb/zones.html.

The zones do appear to be fairly well seperated; they have their own IP addresses, and appear to be separate domains. You go through the Solaris Install procedure of configuring name services, root password, etc, for each zone. However, there are still a few things shared across the zones.

• Hackability (seems good)
• Patching Considerations (seems bad - not enough patches available to tell)
• IP Addressing (good)
• Name Services (good)
• Passwd / UID Entities (bad)
• Secure Shell (ssh) (good)

Hackability

I am no security expert, but I've got a certain amount of experience. One thing I have never really got around to is breaking out of chroot jails, so I can't say for certain how easy it is for a root user in a zone to get to the global zone. I did try some trickery, though, and was caught out by existing Solaris policies.

If the global admin has given access to the rootdisk, try to mount it:

root@dougal # zonecfg -z dbzone
zonecfg:dbzone> add device
zonecfg:dbzone:device> set match=/dev/dsk/*
zonecfg:dbzone:device> end
zonecfg:dbzone> verify
zonecfg:dbzone> commit
zonecfg:dbzone> exit
root@dougal #

dbzone console # ls -l /dev/dsk/c1t0d0s0
brw-r-----   1 root     sys       32,  0 Nov 17 21:22 /dev/dsk/c1t0d0s0
dbzone console # mount /dev/dsk/c1t0d0s0 /mnt
mount: /dev/dsk/c1t0d0s0 is already mounted or /mnt is busy

I assume that, being root, with write access to the device, an attacker who got access to this zone would be able to bring down the machine by writing trash to the disk device. I haven't tried this yet, as I want to keep experimenting with my configuration!

Patching Considerations

These zones have been installed to a certain patch level, but newer patches are available, so I can test this.

It seems that the Global zone can control package levels, and that Solaris 10 patches are zone-aware, but that a zone cannot control its own patch levels, as the zones have read-only hardlinks to the binaries.

This will presumably be different for patches affecting /etc, or any other writeable areas within the zone (my zones are configured with /usr/local being writeable, and /var is writeable in the dbzone). That could be a problem - patch Sendmail, and you could find that the /etc/mail/sendmail.cf config file is updated, but the matching /usr/lib/sendmail binary is not updated to suit.

Let's pick an outdated patch on my system:

root@dougal # showrev -p |grep 119254
Patch: 119254-06 Obsoletes: 119015-03 Requires:  Incompatibles:  Packages: SUNWswmt, SUNWpkgcmdsu
root@dougal # unzip 119254-09.zip > /dev/null

... confirm that the new file is indeed different from the installed file, both in the Global zone and the dbzone:

root@dougal # sum /usr/bin/pkgadm /zones/dbzone/root/usr/bin/pkgadm 119254-09/SUNWpkgcmdsu/reloc/usr/bin/pkgadm
21549 123 /usr/bin/pkgadm
21549 123 /zones/dbzone/root/usr/bin/pkgadm
21561 123 119254-09/SUNWpkgcmdsu/reloc/usr/bin/pkgadm
root@dougal #
root@dougal # patchadd 119254-09
Validating patches...
Loading patches installed on the system...
Done!
Loading patches requested to install.
Done!
Checking patches that you specified for installation.
Done!
Approved patches will be installed in this order:
119254-09
Preparing checklist for local zone check...
Checking local zones...
This patch passes the local zone check.
119254-09
Summary for zones:
Zone webzone
Rejected patches:
None.
Patches that passed the dependency check:
119254-09
Zone dbzone
Rejected patches:
None.
Patches that passed the dependency check:
119254-09
Patching global zone
Adding patches...
Checking installed patches...
Verifying sufficient filesystem capacity (dry run method)...
Installing patch packages...
 (takes a while)
Patch 119254-09 has been successfully installed.
See /var/sadm/patch/119254-09/log for details
Executing postpatch script...
Patch packages installed:
  SUNWpkgcmdsu
  SUNWswmt
Done!
Patching local zones...
Patching zone webzone
Adding patches...
Checking installed patches...
Verifying sufficient filesystem capacity (dry run method)...
Installing patch packages...
Patch 119254-09 has been successfully installed.
See /var/sadm/patch/119254-09/log for details
Executing postpatch script...
Patch packages installed:
  SUNWpkgcmdsu
  SUNWswmt
Done!
Patching zone dbzone
Adding patches...
Checking installed patches...
Verifying sufficient filesystem capacity (dry run method)...
Installing patch packages...
Patch 119254-09 has been successfully installed.
See /var/sadm/patch/119254-09/log for details
Executing postpatch script...
Patch packages installed:
  SUNWpkgcmdsu
  SUNWswmt
Done!
root@dougal #
root@dougal # sum /usr/bin/pkgadm /zones/dbzone/root/usr/bin/pkgadm 119254-09/SUNWpkgcmdsu/reloc/usr/bin/pkgadm
21561 123 /usr/bin/pkgadm
21561 123 /zones/dbzone/root/usr/bin/pkgadm
21561 123 119254-09/SUNWpkgcmdsu/reloc/usr/bin/pkgadm

Okay, so the patch was applied to all the zones. That's nice and easy. Checking was done per-zone, so presumably that implies that the zones would not apply the patch if it did not apply to them.

Let's try to remove the patch from one zone only:

root@dougal # sum /usr/bin/pkgadm /zones/dbzone/root/usr/bin/pkgadm
21561 123 /usr/bin/pkgadm
21561 123 /zones/dbzone/root/usr/bin/pkgadm
root@dougal #
root@dougal # ls -il /usr/bin/pkgadm /zones/dbzone/root/usr/bin/pkgadm
     22929 -r-xr-xr-x   1 root     bin        62760 Oct  7 01:47 /usr/bin/pkgadm
     22929 -r-xr-xr-x   1 root     bin        62760 Oct  7 01:47 /zones/dbzone/root/usr/bin/pkgadm
root@dougal #

I don't think that actually worked. It just didn't admit to failure.

IP Addressing

IP addresses are unique per zone, and only have access to their own virtual port, as shown. This is a good thing, and in some ways better than an E25K domain, whereby domains may (depending on how you configure it) be able to communicate with each other over the internal network. Of course, normal security considerations apply in all cases.

root@dougal # ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849 mtu 8232 index 1
        zone webzone
        inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849 mtu 8232 index 1
        zone dbzone
        inet 127.0.0.1 netmask ff000000
hme0: flags=1000843 mtu 1500 index 2
        inet 192.168.1.20 netmask ffffff00 broadcast 192.168.1.255
        ether 8:0:20:cf:fd:51
hme0:1: flags=1000843 mtu 1500 index 2
        zone webzone
        inet 192.168.1.21 netmask ffffff00 broadcast 192.168.1.255
hme0:2: flags=1000843 mtu 1500 index 2
        zone dbzone
        inet 192.168.1.22 netmask ffffff00 broadcast 192.168.1.255
root@dougal #

This shows that the global zone can see all allocated addresses - they are, after all, just virtual IPs on the same adapter (it would be possible to allocate other adapters as the primary network port for some zones, if desired). However, as we see below, each zone can only see its own part of the network configuration.

webzone console # ifconfig -a
lo0:1: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
hme0:1: flags=1000843 mtu 1500 index 2
        inet 192.168.1.21 netmask ffffff00 broadcast 192.168.1.255
webzone console #

dbzone console # ifconfig -a
lo0:2: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
hme0:2: flags=1000843 mtu 1500 index 2
        inet 192.168.1.22 netmask ffffff00 broadcast 192.168.1.255
dbzone console #
dbzone console # ping 192.168.1.21
192.168.1.21 is alive
dbzone console #

Note that the zones can communicate with each other, just as two Solaris machines on a network can communicate with each other. This is in-the-box, regardless of physical network connectivity, as you would expect - the communication is simply going down one IP stack and up another.

back to top

Name Services

This is unique per zone - in this example configuration, webzone uses DNS, while dbzone uses files.

The webzone is configured to use DNS, and does so, finding the correct IP address for steve-parker.org (correct at the time of writing):

webzone console # grep ^hosts: /etc/nsswitch.conf
hosts:      files dns
webzone console # telnet steve-parker.org 80
Trying 64.40.100.150...
Connected to steve-parker.org.
Escape character is '^]'.
^C
Connection to steve-parker.org closed by foreign host.
webzone console #

However, the dbzone is configured to use files, and cannot find steve-parker.org until an entry is put into /etc/hosts:

dbzone console # grep ^hosts: /etc/nsswitch.conf
hosts:      files
dbzone console # grep steve-parker.org /etc/hosts
dbzone console # telnet steve-parker.org 80
steve-parker.org: node name or service name not known
dbzone console # echo "192.168.1.227 steve-parker.org" >> /etc/hosts
dbzone console # telnet steve-parker.org 80
Trying 192.168.1.227...
Connected to steve-parker.org.
Escape character is '^]'.

^C
Connection to steve-parker.org closed by foreign host.
dbzone console #

This is Good Stuff, and what you would expect from separate machines or domains.

back to top

/etc/passwd Entries

/etc/passwd entries is one shared item. I assume that if NIS, LDAP, etc, are configured in /etc/nsswitch.conf, then this would be distinct between zones if appropriate, like the DNS vs. files example above. However, using file-based /etc/passwd entries:

root@dougal # useradd -u 100 -d /export/home/steve steve
root@dougal # mkdir -p /export/home/steve
root@dougal # chown steve /export/home/steve
root@dougal # useradd -u 101 -d /export/home/fred fred
root@dougal # mkdir -p /export/home/fred
root@dougal # chown fred /export/home/fred

dbzone console # useradd -u 100 -d /export/home/steve steve
dbzone console # mkdir -p /export/home/steve
dbzone console # chown steve /export/home/steve
dbzone console # useradd -u 101 -d /export/home/bob bob
dbzone console # mkdir -p /export/home/steve
dbzone console # chown steve /export/home/steve

So the dbzone and global zones have a user called "steve" with a UID 100. The dbzone has a user called "bob" with a UID 101, while the global zone has a user called "fred" with a UID 101. What happens when we look at /export/home or /zones/dbzone/root/export/home?

dbzone console # ls -l /export/home/
total 4
drwxr-xr-x   2 bob      root         512 Nov 16 23:39 bob
drwxr-xr-x   2 steve    root         512 Nov 16 23:38 steve
dbzone console #

root@dougal # ls -l /zones/dbzone/root/export/home/
total 4
drwxr-xr-x   2 fred     root         512 Nov 16 23:39 bob
drwxr-xr-x   2 steve    root         512 Nov 16 23:38 steve
root@dougal #

This is not necessarily as straightforward as you might think - okay, so there is one filesystem, and the inodes have a certain UID associated with them.

That does explain it, but think about the fact that it did allow me to create the two users in the two zones. Is that what you would expect, given the potential for conflict?

Now that a user with UID 100 exists Within the dbzone zone, I get the normal Solaris error message if I try to create a second user with UID 100 in that zone, just as I would on any Solaris system:

dbzone console # useradd -u 100 john
UX: useradd: ERROR: uid 100 is already in use.  Choose another.
dbzone console #

Yet it did allow the

dbzone console # useradd -u 100 -d /export/home/steve steve

And I can add a new user within the webzone zone with UID 100:

webzone console # useradd -u 100 albert
webzone console #

With no error messages or warnings, although the "steve" user with UID 100 already existed on the global and dbzone zones.

So - does this matter? Well, let's configure a "apache" user (UID 500) on the webzone, a "mysql" (UID 500) user on the dbzone, and a "admin" user (UID 500) on the global zone, and see what the ps command returns for each zone:

webzone console # ps -eaf|grep http
  apache  2225  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
  apache  2226  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
    root  2224   554   0 00:32:47 ?           0:00 /usr/local/apache2/bin/httpd -k restart
  apache  2227  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
  apache  2228  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
  apache  2229  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
    root  2239  2177   0 00:34:32 console     0:00 grep http
webzone console #

.... Good - the webzone sees the processes running as Apache.

dbzone console # ps -eaf|grep http
    root  2241  2089   0 00:35:06 console     0:00 grep http
dbzone console #

.... Good - the dbzone cannot see the webzone processes.

root@dougal # ps -eaf|grep http
   admin  2225  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
   admin  2226  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
    root  2224     1   0 00:32:47 ?           0:00 /usr/local/apache2/bin/httpd -k restart
   admin  2227  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
   admin  2228  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
   admin  2229  2224   0 00:32:48 ?           0:00 /usr/local/apache2/bin/httpd -k restart
    root  2243  2126   0 00:35:55 pts/2       0:00 grep http
root@dougal #

.... Bad - the global zone believes that the Admin user is running the processes. Maybe it is, I don't know many details about internal kernel behaviour, but the obvious assumption would be that processes are dealt with internally by their UID, not their textual name.

That would mean that if different admins on different zones do not communicate with each other and the global administrator, the "admin" user could have restricted disk quotas or project rights, which would restrict the capabilities of the "apache" and "mysql" users.

I do not know if this is the case, but on first impressions, that appears to be the way the zones work.

back to top

Secure Shell

The Secure Shell (ssh) works as expected if these were separate machines. When I try from my local machine (192.168.1.227), to connect to webzone I get this:

$ ssh root@192.168.1.21
The authenticity of host '192.168.1.21 (192.168.1.21)' can't be established.
RSA key fingerprint is b6:06:8a:a9:69:52:56:e7:bd:b0:ee:a6:57:4b:bb:dd.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.21' (RSA) to the list of known hosts.
Password:
Password:
Password:
Permission denied (gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive).
$

And the webzone console really is a console, just like that of a domain:

webzone console # Nov 17 01:10:02 webzone sshd[2244]: Failed keyboard-interactive for root from 192.168.1.227 port 53189 ssh2

If I then edit /etc/ssh/sshd_config on webzone to enable root logins, and issue a svcadm restart ssh command (the Solaris 10 equivalent of /etc/init.d/sshd restart) then I can get a login, with the webzone root password, not the global root password:

$ ssh root@192.168.1.21
Password:
Last login: Thu Nov 17 01:10:38 2005 from elvis
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
#

So that's Good Stuff.