Solaris iSCSI Target with ESX 3.02 Server
I have a nice big IBM server box with ESX serving as my entire Engineering lab (Engineering is my name for the IT lab we have here for Star Trek-related historical reasons). The box has a terabyte of RAID5 SATA disks on board for virtual machine storage, but that’s not quite enough for what we’re doing. To get more storage for virtual machines, I decided that I could take advantage of our Sun x4500 server with Solaris, to allocate another terabyte of storage. Solaris supports iSCSI targets in ZFS, so that seemed like the way to go. The x4500 has four gigabit network interface cards on board, three of which I haven’t been using so far, so I decided a good way to go would be to add a gigabit network interface to my IBM ESX server and use a crossover CAT6 cable to direct-connect them and provide a dedicated gigabit storage “network” for iSCSI.
Here are my lab notes from getting that setup.
Setting up iSCSI storage on Solaris and then getting it mounted on ESX Server
First, add a physical nic to VMware Server if required. iSCSI is only supported on Gigabit ethernet in VMware ESX 3.x. If you don’t know how to do this, get your hardware dude to take care of it, and you go take a remedial computer class.
Next, add a physical nic to Solaris box if required. This was not required in my case because my x4500 server had a spare port. Hook the Solaris box nic to the ESX box nic with a crossover cable or via a gigabit switch.
Next configure the nic in Solaris like this: (assuming you know the device name). On the x4500 the built in ethernet interfaces are called e1000g0, e1000g1, e1000g2, and e1000g3. I was already using e1000g0 for the main interface to the network.
ifconfig e1000g1 plumb ifconfig e1000g1 192.168.254.2 netmask 255.255.255.0 up
Check to see if it worked with this:
You should see that e1000g1 now has an IP address assigned to it.
Next, add the file for the new interface’s IP address under /etc
cat > /etc/hostname.e1000g1 192.168.254.2 CTRL-D
Next, edit /etc/inet/hosts and add a line for your new IP address. You might have to chmod +w hosts before editing and chmod -w hosts after. Follow the format in the file. Set a new hostname for your new interface
192.168.254.2 ss003stor.corporate.ae.ca ss003stor
Make sure the file ends with a blank line.
Next, edit the /etc/inet/netmasks file. You might have to do the chmod ting again because the file is normally set to read only. Add this line:
Make sure the file ends with a blank line.
Next, add a new zfs volume that we will use as an iSCSI target. This example assumes that the zfs filesystem data/vols already exists.
zfs create -V 1T data/vols/vs-vmfs01
This creates a 1 Terabyte data volume. It is a volume, and not a zfs filesystem, so you can’t see it with the ls command, but you can see it with zfs list -r data/vols.
Next, make the volume shared via iSCSI.
zfs set shareiscsi=on data/vols/vs-vmfs01
Next, create an iSCSI target portal group (tpgt), and set it up to only listen for iSCSI connections on your new dedicated storage network card.
iscsitadm create tpgt 1 iscsitadm modify tpgt -i 192.168.254.2 1
Make sure the group got created:
iscsitadm list tpgt -v TPGT: 1 IP Address: 192.168.254.2
Add the target we already created to the group.
iscsitadm modify target -p 1 data/vols/fs-vmfs01
Make sure that worked.
iscsitadm list target -v Target: data/vols/vs-vmfs01 iSCSI Name: iqn.bla.bla.bla Alias: data/vols/vs-vmfs01 Connections: 1 Initiator: iSCSI Name: iqn.bla.bla Alias: vs200.bla.bla ACL list: TPGT list: TPGT: 1 LUN information: LUN: 0 GUID: blabla VID: SUN PID: SOLARIS Type: disk Size: 1.0T Backing store: /dev/zvol/rdsk/data/vols/vs-vmfs01 Status: online
Now the target is ready to get mounted and formatted by the ESX Server.
The next step is to configure the ESX Server.
First, go to the virtual infrastrucutre client and select the server’s configuraion tab. Click on Network Adapters and make sure your network card shows up in the list. Click on Networking, and click Add Networking.
In the dialog, select the VMkernel radio button and click Next. Select “Create a Virtual Switch” and make sure your new network interface is selected. Click next. Under Network Label, give it a name like vmkernel storage, and select Next and then Finish.
On your new virtual switch, click Properties. In the Ports dialog, click Add, and select VMKernel. Call the network label VMkernel Storage, and give it an IP address on the same subnet as your Solaris dedicated storage server network card, with the same network mask. Click Next and Finish.
Next, go to the console command line via ssh or on the actual console. Login as root. Type the following commands to enable iSCSI software initiator, open a port in the firewall, and setup and scan the iSCSI target:
(enables iSCSI software initiator)
/usr/sbin/esxcfg-firewall -e swISCSIClient
(opens iSCSI client port in firewall)
/usr/sbin/vmkiscsi-tool -D -a 192.168.254.2 vmhba40
(tells the iSCSI software adapter to request iSCSI targets from the storage box)
(tells the iSCSI initiator to scan for new targets)
Next, go to the Storage tab and click Add Storage. Choose the Disk/LUN radio button and click Next. Select your new iSCSI target from the list and click Next. Follow the prompts, and ESX server will format the volume with vmfs and mount it. You should now be able to start using it right away.