"That which is overdesigned, too highly specific, anticipates outcome; the anticipation of outcome guarantees, if not failure, the absence of grace."
-- William Gibson, All Tomorrow's Parties
Solaris 10 SOHO Unreview.

I guess almost two months ago now I started playing around with Solaris 10. I spent a lot of time reading up on it, and even ordered a SunFire X2100 because I figured I might actually want to run it in production (things like Zones, DTrace, SMF, etc, just are that awesome). I probably wouldn't have thought of getting the SunFire, but mdxi was very positive about his experience with the machine.

As I get older, I seem to notice the noises computers in my room makes more and more. Whenever my fileserver (which is across the room -- but it's a really small room) runs it cron jobs at night (which are all very heavy I/O) it sounds like a small motor is chunking through a duck or something. Last night I finally got tired enough of it that I cleaned out my closet with the intention of moving any machines that don't require a display in there. I also figured I would take the opportunity to do something productive with Solaris 10: Replace my current OpenBSD Samba server.

The goal for the new fileserver was to have a very sparse global zone with no accounts, then two more zones. One for media serving and the other for pulling backups from various machines out on the Interwebs.

The "reduced network" install is the way to go, and build your system up from that. Very few extraneous services will be installed, and you can turn them all off with the new daemon manager, SMF (Service Management Facility), easy enough. I have a USB keyfrob with all the files I use to set up a system; I'll post it later after cleaning up the shell scripts.

The "new" fileserver is a Dell PowerEdge 1400SC I bought several years ago. Most Dell equipment is fully supported by Solaris 10. I threw in two 200G drives and let the copies run overnight. I didn't realize that to get the system to recognize new hardware, I needed to run devfsadm -vC, boot with -r, or touch /reconfigure. Once I had done that, it saw the drives happily enough and I could configure them with the format command (the system didn't recognize the drives, which are IDE Maxtor pieces of crap, but adding a new entry for them was easy enough). Then I got down to bidness.

Solaris 10 has containers, which they call Zones. They aren't virtual machines, but glorified chroots with resource controls (CPU, RAM, etc) , shared directories via loopbacks from the global zone (which is awesome when updating the system), and a few other cool things.

The really important thing to make Solaris livable (for me, anyway; I'm pretty set in my habits) is BlastWave. Jon and Harry, both old Solaris hands, reco'd it, but it took me a while to catch on. I tried using the OpenPkg stuff, but it was very not-awesome.

You just grab their pkg-get tool and boom, you have access to a lot of useful packages like OpenSSH, Samba, etc. It puts everything in /opt/csw as should be expected, if you're at all used to commercial UNIXes (I'm not overly familiar with Solaris; I've admin'd a few machines over the years, but not very in-depth. They pretty much did what they needed to and never died. I think I installed Sol8 once back in the day for a production system).

The first thing I tried doing with zones was NATing them. For instance, running all the zones on a 10/24 network, and using ipf (which Solaris comes with now) to NAT/port forward the traffic. My idea was to hand out zones to interested people on the SunFire, and allocate large swathes of ports to forward to their zone. Since zones are containers and not virtual machines, they're all sharing the same kernel, routing table, etc. If you're running in a zone, you have a few limitations on what you can configure: You can't add routes, for instance. I couldn't quite get it to work and then stopped screwing with it due to time constraints. There's no reason it shouldn't work, though.

There are quite a few really good zone docs online:

I created the zones as described in the above docs and added the directories I wanted to share with them using via "add fs" from the zonecfg shell.

Note: When configuring the zones using OS X's Terminal.app, if you specify xterm as your TERM, you'll have problems with F2... Use the DEC VT100 TERM emu and fn-F2 works as expected.

Note: Another thing to keep in mind is that while /opt/csw will be copied to your non-global zones, and it will try to start OpenSSH, the sshd user will not exist in the non-global zone's passwd file. Easily enough rectified, of course.

Automating zone creation looks like something massively easy to do anyway (see /etc/zones, the configs are simply XML, and once you push the zone onto the file system, you can easily replace or modify any files from the global zone).

Zones list

[root@gaea]:[~]# zoneadm list -cv
ID NAME STATUS PATH
0 global running /
3 cronus running /zones/cronus
4 oceanus running /zones/oceanus

The Global Zone (gaea)

[root@gaea]:[~]# df -h
Filesystem size used avail capacity Mounted on
/dev/dsk/c1t0d0s0 996M 134M 802M 15% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 748M 344K 747M 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
/dev/dsk/c1t0d0s1 3.9G 172M 3.7G 5% /usr
/usr/lib/libc/libc_hwcap1.so.1
3.9G 172M 3.7G 5% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
/dev/dsk/c1t0d0s3 3.9G 16M 3.9G 1% /var
swap 748M 808K 747M 1% /tmp
swap 747M 20K 747M 1% /var/run
/dev/dsk/c1t0d0s4 2.0G 283M 1.6G 15% /opt
/dev/dsk/c1t0d0s5 996M 1.2M 935M 1% /root
/dev/dsk/c1t0d0s7 2.8G 623M 2.2G 22% /zones
/dev/dsk/c4d0s6 186G 113G 71G 62% /export/backups
/dev/dsk/c4d1s6 186G 66G 118G 36% /export/media
/dev/dsk/c1t0d0s6 2.0G 2.0M 1.9G 1% /export/home

As you can see I don't have much room in /zones, but it should be enough for my purposes.

The Fileserver Zone (cronus)

[root@gaea]:[~]# zonecfg -z cronus
zonecfg:cronus> info
zonepath: /zones/cronus
autoboot: true
pool:
inherit-pkg-dir:
dir: /lib
inherit-pkg-dir:
dir: /platform
inherit-pkg-dir:
dir: /sbin
inherit-pkg-dir:
dir: /usr
fs:
dir: /export/media
special: /export/media
raw not specified
type: lofs
options: [rw,nodevices,noexec]
net:
address: 192.168.1.16
physical: iprb0

[root@gaea]:[~]# ssh cronus
Password:
Last login: Sat Mar 25 18:48:17 2006 from hyperion.int.mi
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
[root@cronus]:[~]# df -h
Filesystem size used avail capacity Mounted on
/ 2.8G 624M 2.2G 22% /
/dev 2.8G 624M 2.2G 22% /dev
/export/media 186G 68G 116G 38% /export/media
/lib 996M 134M 802M 15% /lib
/platform 996M 134M 802M 15% /platform
/sbin 996M 134M 802M 15% /sbin
/usr 3.9G 172M 3.7G 5% /usr
proc 0K 0K 0K 0% /proc
ctfs 0K 0K 0K 0% /system/contract
swap 739M 116K 739M 1% /etc/svc/volatile
mnttab 0K 0K 0K 0% /etc/mnttab
/usr/lib/libc/libc_hwcap1.so.1
3.9G 172M 3.7G 5% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 739M 0K 739M 0% /tmp
swap 739M 12K 739M 1% /var/run

So once that's up, I set my shell the way I like, mkdir /export/home add my non-priv user, the usual.

Then I use pkg-get to install Samba. It pulls its catalog, installs its packages, and I copy the config over from the old Samba fileserver box. Edit the paths, start Samba, and... done.

In fact, since it's still mounting as /Volumes/media on my Mac, iTunes is quite happy.

The Backup Server Zone (oceanus)

[root@gaea]:[~]# zonecfg -z oceanus
zonecfg:oceanus> info
zonepath: /zones/oceanus
autoboot: true
pool:
inherit-pkg-dir:
dir: /lib
inherit-pkg-dir:
dir: /platform
inherit-pkg-dir:
dir: /sbin
inherit-pkg-dir:
dir: /usr
fs:
dir: /export/backups
special: /export/backups
raw not specified
type: lofs
options: [rw,nodevices,noexec]
net:
address: 192.168.1.17
physical: iprb0
[root@gaea]:[~]# ssh oceanus
Password:
Last login: Sat Mar 25 20:05:17 2006 from gaea.int.mirror
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
[root@oceanus]:[~]# df -h
Filesystem size used avail capacity Mounted on
/ 2.8G 624M 2.2G 22% /
/dev 2.8G 624M 2.2G 22% /dev
/export/backups 186G 104G 80G 57% /export/backups
/lib 996M 134M 802M 15% /lib
/platform 996M 134M 802M 15% /platform
/sbin 996M 134M 802M 15% /sbin
/usr 3.9G 172M 3.7G 5% /usr
proc 0K 0K 0K 0% /proc
ctfs 0K 0K 0K 0% /system/contract
swap 732M 124K 732M 1% /etc/svc/volatile
mnttab 0K 0K 0K 0% /etc/mnttab
/usr/lib/libc/libc_hwcap1.so.1
3.9G 172M 3.7G 5% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 732M 0K 732M 0% /tmp
swap 732M 12K 732M 1% /var/run

Install rsync, copy over my backup script and configs... and done.

(Well, actually, I have to generate root SSH keys and copy them to the target systems authorized_keys files, but that's outside the scope of this little post.)

Anywhere, there you have it.

It's pretty freaking sweet. :-)

March 25, 2006 8:09 PM