"That which is overdesigned, too highly specific, anticipates outcome; the anticipation of outcome guarantees, if not failure, the absence of grace."
-- William Gibson, All Tomorrow's Parties
January 22, 2004

I've been thinking recently (again) about how to securely connect to a machine you have to administer.

For users, this isn't a major problem. You can have random passwords for each host you need to connect to as long as you set up ssh keys with passphrases and alternatively, ssh-agent to deal with the annoying parts of actually connecting to the machine. As long as you have "trusted" hosts which house the private ssh keys, you're good.

For admins, it's different.

6:22 AM | Systems Administration | Comments (2)
January 23, 2004

I've been slowing gearing up for a major security kick as of late. So much bad, and not enough good. The SSH Trust Web idea is just part of it. Later today, probably after I get some sleep, I'll write a "secure" server policy, which details mount points, kernel security settings (grsecurity, etc) and the like.

Last week I scheduled downtime for all the production servers at work, as they all need reboots for kernel upgrades.

If I get the security policy written in time for a cursory approval from some of the more security conscious people I know, I'll reinstall the LAN firewall following it. I already know it's going to be somewhat of a pain, as keeping all suid binaries on their own partition tends to be a minor annoyance. However, like all things, it's easily scripted around.

A few months ago I played around with grsecurity, but at the time didn't care enough to consider implementing it on real machines. I guess I care now, and recompiled eos.int.walnutfactory.org's kernel, enabling just about every option that looked sane. I'm curious to see how usable the machine is, for day-to-day use, but none of the PWF kids currently use the machine for anything (as its still new to the Factory).

There are a few things I need to dig into with regards to grsecurity, the most interesting being the "learning mode" for ACLs.

Speaking of which, the most time-intensive aspect of enable grsecurity is going to be writing a sane ACL policy, assuming I don't let it figure it out on its own, and then actually turn ACLs on. In one respect, it's good that the majority of my machines are Debian GNU/Linux. Of course, generally speaking, completely homogenous networks are not the love, but it sure does making it easy when rolling out new technologies scripting for system administration.

Considering the amount of documentation I'm going to have to produce in a short amount of time, I really should consider starting to write it all in LaTeX.

6:33 AM | Systems Administration

After a few minutes of looking around, I've discovered quite a bit of useful documentation, most of it describing the use of shiny things which Just Work.

I've been using ssh forever, and yet never knew about the ForwardAgent feature... because I've never really thought to use ssh-agent. Well, it's nice to think "Hm, it'd be useful if..." and have it already done.

This series of articles written by the Gentoo Linux Chief Architect describe a few nifty tricks with ssh and ssh-agent, using the Gentoo-supplied keychain program.

This howto has a few useful aliases in it for starting keychain on login and sourcing the env vars so you can get access to your agents identities. Don't forget the "-q" switch so you don't get that Gentoo-default green and blue spam screen. What's with Gentoo and green and blue console messages, anyway?

At any rate, all of this will be included in my eventual HOW-TO for work, which will be released here as well.

So much useful software, and so easy to use.

6:43 AM | Systems Administration
January 27, 2004

Note to self: Do not upgrade a Perl distribution by two full dot-releases without also recompiling mod_perl. This can end up causing a minor, ten minute long headache after the machine is rebooted for a kernel upgrade and the httpd is finally flushed out of memory and attempts to restart.

And proceeds to segfault.

Took me a few minutes to remember "EVERYTHING=1", as well, which makes the world just full of bunnies. Blue ones. With wings on.

2:27 PM | Work
January 29, 2004

Work has been very interesting this week.

I was informed, in an extremely off-hand manner, that it was a definite that we would be moving operations of our primary building to another -- probably over to the photography studio building.

That building is mostly factory floor space, filled by a press and a book bindery. The office area is taken up by the photo studio, and several smaller office-worthy (read: carpeted, not near heavy equipment) areas. We have several large printers we'd have to move over, not to mention all the people: Pre-press, CSRs, development.

We aren't even sure where we'd put the servers at this point, but we have a few ideas. Our current building is incredibly nice, and the CTO who was responsible for the goodness there is in charge of this move as well, so I'm not too worried about the final layout being not-usable. Hopefully we can get development cordoned off somewhere, as our other co-workers can be very loud (and engage in tapeball warfare at random times).

While I think that consolidating the business into one building is a good move, this isn't going to be a lot of fun.

If nothing else, it forces me to do a number of things that I've been kind of spinning my wheels on, like moving our MX and DNS machines to our colocation facility.

I also ran a quick inventory today, and it came out to about 49U of rackable gear, with about a dozen mini-to-mid-tower server boxes. That doesn't include the tape robot, or the tape archives themselves.

I guess it's a good thing I refactored the LAN last year. This would have been an enormous pain in the butt with the old setup, instead of just a pretty big one. :-)