"That which is overdesigned, too highly specific, anticipates outcome; the anticipation of outcome guarantees, if not failure, the absence of grace."
-- William Gibson, All Tomorrow's Parties

These last two weeks have not been my superhappyfuntime.

The company I work at is merging with another company and their IT guy, who was not only lazy and shall we say, somewhat cavalier with regards to his duties as a systems administrator, but well... the emphasis here is was.

So at the moment I sort of have two jobs. I'm getting tired of 12-16 hour days.

Not to mention the two 24 hour days.

The biggest problem I have is that their entire shop is Windows-based, except for two Macs in pre-press and two in design. That leaves about two dozen Windows workstations, half of which are infected with various forms of viruses, and three Windows servers.

Including, for some as-yet-to-be-determind, an MSSQL machine.

I suppose that explains the "FixBlaster.exe" binary on the PDC's desktop.

I'm just not used to a Windows environment, I think. Tuesday, network connectivity was being super spotty; "Crap," I think. "That 486 junk firewall I replaced their horrible SonicWall with is dying on me." So I go and steal a disk out of a machine whose processor fan had recently failed, install OpenBSD on it, and waste half an hour of bandwidth and a half hour of my time (counting interruptions to deal with other stuff) that the network is still thrashing.

"Bloody Hell," I says, watching it take six packets to get anything anywhere. "hm, are my pf rules screwy?" pf is turned off and the connection is again happy. "Well, I suppose that hopefully rules out the NICs and the hardware," I thinks to meself.

So I finally do what I should have done in the first place:

tcpdump -eni ep0


"Golly gee, that's a lot of 135 and 445 traffic going to space... space that doesn't exist. Invalid subnets. Damnit!"

And let's not forget the 6667 traffic fleeing outbound to the world, doing gods know what...

So I quickly block all egress traffic save for a few required ports, and connectivity is somewhat happier, though hardly not at all. So I ponder to myself, "Ponder ponder, what's the probl-- oh. Queues."

Yes indeedy. It was taken half a dozen to a dozen packets to fall up the goddamn stack and get routed. Luckily I'm a complainer and Andrew quickly suggested that I just block all non-valid traffic on the internal interface, so the junk never gets processed.

Word to Andrew.

tcpdump -c 50000 -eni xl0 src net and dst net and dst net \! > infected_hosts ; awk '{print $6}' infected_hosts |sed -e /.....$/s///|sort|uniq

(My regexp sucks so much.)

That was an adventure!

And not the only one for that day, but the only one that I can remember, because it involved me being stupid. And I always remember those stories.

Today was also pretty awful, but I got a lot done. It's funny how that works. I spent about an hour swapping machines because one of the managers decided to upgrade a piece of software on an operating system that doesn't support... something or other the new version of the application needs. So yesterday Adam installed it on a Win2k box, which is what it wanted.

Only the guy forgot to mention that some printers needed to sort of be hooked up to that box... "Looks like a job for bda!"

So this new machine is actually one of our old ones, but it's been at the new building for maybe three months. And it was caked with dust. And the older box that I was swapping out? Oh. I think at one point, it was probably that stupid tan color old machines all are. But it was grey.

And my clothes? Well, they were black when I started. By 1130, though, they were white.


Luckily I still had a box of Christmas clothes in my cube at the other building, so I could change and not be covered in goddamn dust all day. Whee!

The only thing I feel even remotely good about is that the new mailserver appears to be operating optimally. There was some issue with IMAP and Mail.app... namely, if you create a folder on the IMAP server, then add a message to it... delete the message... and then delete the folder, Mail.app cries. "Can't SELECT!" Because it doesn't refresh after deleting and before opening again. And it was connecting way too much.

But I realized I was blocking the UDP ports IMAP wants on the box, and that seems to have fixed the issue. I didn't look too much into it... tomorrow I'll see exactly why that might be. It seems... odd to use UDP for those operations. But what do I know.

I'm not even going to get into the dozen or so "omfg!" fires that people came to be about, causing me to not clean the infected Windows machines. argh. You'd think that'd be my priority, and it is, but it still hasn't happened. Gods willing, I'll get to that tomorrow morning and afternoon.

What else, what else.

Apparently the previous IT guy's default responses to anything anyone ever asked him to do were:

  • No.

  • I can't do that.

And if you came to him with something broken?

  • Deal with it.

Needless to say, this did not go over well with the users (You know, his fellow employees? The people he was being paid to assist?), and they are all somewhat shocked, I think, to find Adam (who has been at the new building for a month or so now, and also helping them out) and myself somewhat... helpful.

And pleasant.

And useful.

And they seem truly astounded perhaps not by our annoyance and the broken state of affairs, but by our wanting to make things better.

For instance! Two sales guys have a printer in their office, a big HP 8500. Nice printer. It speaks JetDirect. The two designers, who use Macs find it with no problem. Humans ask the "sysadmin" if they can print to it. He tells them, "No, you can't. Windows can't print to that printer."

A week after his ass gets canned, the matter is brought to Adam's attention, who says "wtf?" and yesterday asks me to take care of it today.

I poke around for a few minutes, having absolutely no idea how to get a printer without a real printserver to work on Windows. In OS X-land, it's trivial to get it working (and, I assume, just as trivial with AppleShare/AppleTalk in OS 9 or whatever, as that's what the designers use). However, I am a somewhat astute observer of human behavior, so I check the "sysadmin's" WindowsXP workstation, which I have access to.

Lo and behold, he has the printer added. I check to see how it's configured, and apparently you add the thing as a local printer, then configure the port via IP... pretty silly, I think to myself, but exceedingly straight-forward.

I go to add the printer on the sales guy's workstations, and one of them tells me that the "sysadmin" had told him once: "Yeah, you can use that printer. You just have to install the drivers and figure out the IP. I'm sure you can do it." And walked away.

This is, of course, while the machine I was touching was pulling the drivers off the fucking printer and installing them.

In all, this process took perhaps fifteen minutes, five of which I had spent poking at the printer itself like a retarded monkey with a dopamine problem.

(And then a Mac OS 9 box ate its "Volume Header", which I presume to be some sort of MBR analogue, and after I screwed with Open Firmware for ten minutes, I got someone to bust out a Norton Utilities CD and that fixed it right up.)

So that's what I'm up against. Years of that kind of "administration." The place is an enormous mess, and I think it's going to drive me insane. That was just an example. I could go into detail about the problems with the network itself... but it would all be stupid stuff like the gateway's IP being

The only lights on my horizon at this point is that I've been promised an Xserve and a terabyte XRAID, with which I can get rid of the NT4 PDC and manage both the Macs (which will outnumber the Windows boxes once my company finally gets into the new building) and the Windows boxes.

Joy. Network authentication and control and gods willing some form of remote patch management.

Also, Hunter and the company librarian (the guy who deals with backups) and I finally managed to get together and have a nice productive meeting about Archivist, the NetBackup replacement (and job management, and archiver, and possibly some form of remote data access and preview functionality stuff) I designed and started writing months ago... and then stopped because this merger started happening. But with Hunter coding, it should actually get somewhere, and become useful, and with Adam driving Hunter, it should get done. The backend stuff is all designed following the Postfix model... which is to say, the UNIX model, which is to say... Hopefully I won't fuck up a good thing.

And it'll be OSS. Yay.

During this meeting the owner was sitting at his desk (his office is in the conference room) and was half-listening to us. At one point we were talking about system failures and he said "Woah, I don't want to hear that talk?" "What?" "I don't tolerate system failures." "No, you plan for them."

"Bah!" says he.

And now? Now I'm going to sleep. Because I deserve it.

(I realize the above examples seem somewhat trivial and probably childish. But fuck you. It's obnoxious. It's Windows. I am a UNIX ADMINISTRATOR DAMNIT. I'll whine if I want to while I'm getting all this Microsoft garbage shoved down my throat.)

/* Oh. And I'm missing HOPE because the things I mentioned are 15% of what I wanted to get done this week, and because I want to start moving into my new apartment with my friend Pete this weekend. Which is also something to look forward to. To put it mildly. */

July 8, 2004 10:00 PM

To put it mildly.

Fuckin' werd, yo.

Posted by: Dan at July 9, 2004 1:16 AM

Fo' sheezy.

Posted by: bda at July 9, 2004 5:10 AM

Yo. Let me know if you need a hand (or more likely, wheels) to help out moving this weekend. You'd be amazed what you can fit in the hatch of an integra.

Posted by: Michelle at July 9, 2004 10:15 AM

Cool beans! I'm going to try and get a truck from work, but that's sort of unlikely now. So uhh. Expect a call maybe. :)

Posted by: bda at July 9, 2004 11:01 AM
Post a comment

Remember personal info?