Remembering Reagan

In response to Mandy’s request for stories about Reagan, here’s a funny story that occurs to me…

When Reagan and I were about 12, I went to his house after a Saturday swim practice. After we stuffed ourselves on a whole box of Golden Grahams, Reagan’s dad took us with him to the office for a while so he could finish up some work. Reagan and I played Flight Simulator (he was a pilot-to-be even then) on the only computer in the office (this was about 1980). Then we decided to take a break and get a snack.

A. D. Williams Engineering had a little kitchen in the office, so we went and grabbed a soda and a chocolate bar each. I unwrapped mine, and set it on the counter while I chewed on the first piece. A wriggling movement on the chocolate bar attracted my attention, and I was disgusted to see that it was riddled through with maggots. I sputtered and spat out the chocolate in my mouth onto the floor. Reagan, surprised by my actions, smacked me and said “what are you doing, you idiot?”

Later he told me that he had written a letter to the chocolate bar company, and sent in the maggoty chocolate bar. They had sent him a whole box of chocolate bars, all of which he had managed to devour before letting me know about them.

Thanks buddy.

More Bad News

After writing this five months ago, I can’t believe what’s happened today. My friend Reagan Williams, who I’ve known since I was 10 years old, and who was one of the three awesome guys who stood up for me as groomsmen at my wedding, died today in a plane crash, in a terrible echo of his own father’s death in October. He was so deeply involved in the lives of so many people, I can’t begin to imagine the impact that this tragedy will have. I’m just sick, as I’m sure everyone who knew him is today.

My chest aches for the pain his family is going to have to go through.

Hmm. Where can I get a bunch of email for this test account?

I’m building a corporate email archiving system for work. It will basically clean out user’s GroupWise accounts into an archive storage system as messages age. We haven’t established firm policies yet about which messages of which age will be migrated into the archive system. First, I’m building a working system that we can demonstrate in our suggested configuration to the managers’ committee, so that they can see what it will look like and set the policies.

To demonstrate the system, I was reluctant to use my own email account, because while I’m working with the system, I don’t want to accidentally bugger up my own email, and also because my mail load doesn’t look much like a typical user’s mail, since I’m not working much as a project engineer. I set up a testing user called Archivist instead.

Next, I needed to get a bunch of mail into the account so that I could demonstrate the archive system in action. Luckily I have a ready source of endless emails. I just reconfigured our Barracuda spam firewall to quarantine all spam into a specific account instead of deleting it, and pointed that account at my test Archivist mailbox. After a few hours, I had hundreds of test messages at my disposal. A quick review to clean out any x-rated subject lines, and I was good to go.

Don’t Do This…

…unless you really want to go snowboarding and are willing to pay the price later.

Today we went to Lake Louise. On the way from the van to the lodge, I slipped on the ice in the parking lot, fell down and dislocated my knee. I felt like an ass. Jennifer and a nice New Zealander who had seen me fall helped me back to the van, where I sat down for 10 minutes, took two super-Motrins, and stopped cursing. Then, I got my board, and Jenn, the kids and I spent the day snowboarding on the steeps at Louise.

Tonight I’m sitting in the hotel with a bottle of Motrin, a couple of bottles of beer, a big bag of ice, and a knee like a pomelo grapefruit, paying the price of my dedication to boarding.

Calgary Open Final Atoms Results

The Edmonton Thunderbirds (including my kids) finished the Alberta Open Water Polo Tournament in second place, for a silver medal. They ended up with three wins, one tie, and one loss in the gold medal game. The winners of the Tournament were the Los Angeles Water Polo selects team. LA also won in several of the other age groups.

Emily was on the leaderboard for scoring finishing sixth or so in the top-scorers list. Mack didn’t see as much playing time as Emily, but he did very well. I think he’s learned a lot this season. The final game was not as close as I thought they could have done. They didn’t quite play as well as they had in the round-robin. Regardless, they can be proud of themselves. It was a good showing overall.

Calgary Open First Atoms Game

We’re at the Calgary Open Water Polo Tournament. Tonight the Thunderbirds played their first game of the tournament against the Fraser Valley team. It was a real nail-biter, but the Thunderbirds won 17-16 in sudden-death overtime. So far, it’s a good start.

Solaris iSCSI Target with ESX 3.02 Server

I have a nice big IBM server box with ESX serving as my entire Engineering lab (Engineering is my name for the IT lab we have here for Star Trek-related historical reasons). The box has a terabyte of RAID5 SATA disks on board for virtual machine storage, but that’s not quite enough for what we’re doing. To get more storage for virtual machines, I decided that I could take advantage of our Sun x4500 server with Solaris, to allocate another terabyte of storage. Solaris supports iSCSI targets in ZFS, so that seemed like the way to go. The x4500 has four gigabit network interface cards on board, three of which I haven’t been using so far, so I decided a good way to go would be to add a gigabit network interface to my IBM ESX server and use a crossover CAT6 cable to direct-connect them and provide a dedicated gigabit storage “network” for iSCSI.

Here are my lab notes from getting that setup.

Setting up iSCSI storage on Solaris and then getting it mounted on ESX Server

First, add a physical nic to VMware Server if required. iSCSI is only supported on Gigabit ethernet in VMware ESX 3.x. If you don’t know how to do this, get your hardware dude to take care of it, and you go take a remedial computer class.

Next, add a physical nic to Solaris box if required. This was not required in my case because my x4500 server had a spare port. Hook the Solaris box nic to the ESX box nic with a crossover cable or via a gigabit switch.

Next configure the nic in Solaris like this: (assuming you know the device name). On the x4500 the built in ethernet interfaces are called e1000g0, e1000g1, e1000g2, and e1000g3. I was already using e1000g0 for the main interface to the network.

ifconfig e1000g1 plumb
ifconfig e1000g1 192.168.254.2 netmask 255.255.255.0 up

Check to see if it worked with this:

ifconfig -a

You should see that e1000g1 now has an IP address assigned to it.

Next, add the file for the new interface’s IP address under /etc

cat > /etc/hostname.e1000g1
192.168.254.2
CTRL-D

Next, edit /etc/inet/hosts and add a line for your new IP address. You might have to chmod +w hosts before editing and chmod -w hosts after. Follow the format in the file. Set a new hostname for your new interface

For example,

192.168.254.2 ss003stor.corporate.ae.ca ss003stor

Make sure the file ends with a blank line.

Next, edit the /etc/inet/netmasks file. You might have to do the chmod ting again because the file is normally set to read only. Add this line:

192.168.254.2 255.255.255.0

Make sure the file ends with a blank line.

Next, add a new zfs volume that we will use as an iSCSI target. This example assumes that the zfs filesystem data/vols already exists.

zfs create -V 1T data/vols/vs-vmfs01

This creates a 1 Terabyte data volume. It is a volume, and not a zfs filesystem, so you can’t see it with the ls command, but you can see it with zfs list -r data/vols.

Next, make the volume shared via iSCSI.

zfs set shareiscsi=on data/vols/vs-vmfs01

Next, create an iSCSI target portal group (tpgt), and set it up to only listen for iSCSI connections on your new dedicated storage network card.

iscsitadm create tpgt 1
iscsitadm modify tpgt -i 192.168.254.2 1

Make sure the group got created:

iscsitadm list tpgt -v
TPGT: 1
      IP Address: 192.168.254.2

Add the target we already created to the group.

iscsitadm modify target -p 1 data/vols/fs-vmfs01

Make sure that worked.

iscsitadm list target -v

Target: data/vols/vs-vmfs01
    iSCSI Name: iqn.bla.bla.bla
    Alias: data/vols/vs-vmfs01
    Connections: 1
        Initiator:
            iSCSI Name: iqn.bla.bla
            Alias: vs200.bla.bla
    ACL list:
    TPGT list:
        TPGT: 1
    LUN information:
        LUN: 0
            GUID: blabla
            VID: SUN
            PID: SOLARIS
            Type: disk
            Size: 1.0T
            Backing store: /dev/zvol/rdsk/data/vols/vs-vmfs01
            Status: online

Now the target is ready to get mounted and formatted by the ESX Server.

The next step is to configure the ESX Server.

First, go to the virtual infrastrucutre client and select the server’s configuraion tab. Click on Network Adapters and make sure your network card shows up in the list. Click on Networking, and click Add Networking.

In the dialog, select the VMkernel radio button and click Next. Select “Create a Virtual Switch” and make sure your new network interface is selected. Click next. Under Network Label, give it a name like vmkernel storage, and select Next and then Finish.

On your new virtual switch, click Properties. In the Ports dialog, click Add, and select VMKernel. Call the network label VMkernel Storage, and give it an IP address on the same subnet as your Solaris dedicated storage server network card, with the same network mask. Click Next and Finish.

Next, go to the console command line via ssh or on the actual console. Login as root. Type the following commands to enable iSCSI software initiator, open a port in the firewall, and setup and scan the iSCSI target:

/usr/sbin/esxcfg-swiscsi -e
(enables iSCSI software initiator)
/usr/sbin/esxcfg-firewall -e swISCSIClient
(opens iSCSI client port in firewall)
/usr/sbin/vmkiscsi-tool -D -a 192.168.254.2 vmhba40
(tells the iSCSI software adapter to request iSCSI targets from the storage box)
/usr/sbin/esxcfg-rescan vmhba40
(tells the iSCSI initiator to scan for new targets)

Next, go to the Storage tab and click Add Storage. Choose the Disk/LUN radio button and click Next. Select your new iSCSI target from the list and click Next. Follow the prompts, and ESX server will format the volume with vmfs and mount it. You should now be able to start using it right away.

Mack Learning to Snowboard

We’re a regular snowboarding family. We like to get out most weekends to our local hill and do a bunch of runs. Both Mack and Emily started off skiing (for reasons we can’t quite recall ourselves). Emily took up snowboarding at the end of the season when she was eight. Mack is now eight, and we’ve managed to convince him to strap on a board instead of skis a couple of times. Last weekend he ventured off the bunny hill on a snowboard for the first time. He was quite proud of himself for not falling while getting off the chairlift. That’s more than I can say about my first time off the chairlift on a board years ago.

He’s already managing heel-edge side-to-side slipping, toe edge, and toe-to-heel turns. I think next year we’ll get him into a lesson or two, and hopefully by the end of next season we’ll be hitting the mountains with the four of us all on snowboards.

No More USB Hard Disks

Since I got some Macs at home I’ve been wanting to use Time Machine systematically for backups. Previously I have had a FreeBSD server at home to store bulk and shared data and backups. Initially, I had anticipated getting more drives for the server, formatting them as ZFS, and using that with Time Machine. Then, when Mac OSX 10.5 Leopard released, it turned out that Time Machine only supports external USB or firewire disks. That meant my server idea wouldn’t work.

I borrowed a USB drive from work to test out with time machine, and it worked for a while, to the point that I even used it to restore a large chunk of my system because I had screwed it up by messing around with it. Then, something funny happened and the USB disk wouldn’t mount anymore. I did a bunch of troubleshooting and discovered that the journal had gotten buggered up on the HFS+ volume on the USB disk. I found a little utility that I ran to disable journaling, which then allowed me to use disk utility to repair the volume.

Recently I was using the USB disk to shuffle some data around, and at the one point that the only copy of some digital video files from the video camera I have were on the disk, it died again. I’ve been trying to repair the volume for four days with various tools. Nothing is working so far. It appears that there is an actual hardware failure somewhere in the device. It’s a Maxtor One Touch III with two hard drives configured in a RAID0, so I can’t recover the data from the disks one at a time by direct-connecting them to another machine.

I was considering going out and buying a big external USB disk for backups until this event occurred. On the other side of my desk, there’s a FreeBSD box with a nice set of mirrored ZFS drives where the data really should be. I’m taking this as a slap upside the head, and I’m not risking any of my data on external USB disks anymore. I’m going to get a couple more disks for the server, RAIDZ them, and use that instead.

I won’t be able to use Time Machine, but what the hell, I can setup a different backup software for Jenn’s and my machines.