Scott’s Blog

After some testing yesterday, I’m happy to see that the upgrade Bart and I did to TRIM 5.2.4 fixed the bug I talked about here. Thanks to the Tower support people for helping us with this.

I only wish they would fix this bug, which still exists. According to a Tower employee who blogs under the name of Fuzzy, Tower Software has known about the bug since 2003. The funniest part about Tower’s response to it, according to Fuzzy’s blog, is that it isn’t an issue because only users running “secure environments” would run into it. Hello!

Bart and I upgraded TRIM last night to version 5.2.4. The upgrade went smoothly except for two things. The first thing was that I must have misremembered a password for a user that the TRIM services run as. One dialog in the install asked for this user name and password. When the installer got to the point where it needed to use that password to reconfigure the TRIM services on my TRIM server, it realized the password was invalid and rolled back the install. It would have been nice if it had just said the password was invalid and let me re-enter it, but the rollback was a good fallback position as well, because when it had finished rolling back, the old version was still functional.

I verified the passwords and did the install again and this time the upgrade went smoothly. After the upgrade I used the TRIM Enterprise manager to update the schema of the Oracle database holding my TRIM data. That was pleasantly uneventful.

Finally I upgraded the client application on my Windows 2000 Terminal Server, so the users would get the current client. That’s where the other minor glitch occurred. For some reason, the TRIM Context client on the Terminal server forgot where the data set was after the upgrade, so we had to login as each of our 20 users and just run TRIM and point it at the correct location for the data set. I’m almost sure that I caused that problem somehow, and it was a minor issue.

If all installs went that way this job would be easy.

We have a minor but irritating problem with TRIM, our records management software. We use it for keeping track of our paper records (of which we have bazillions). We have a couple of custom fields in our records definitions (TRIM is extensible to allow you to track whatever info you need about your records). For some reason, when our users fill out the custom fields with data in mixed case, and then save it in TRIM and retrieve it later, the custom fields have been converted to upper case. No data gets lost, but sometimes it causes the records workers to have trouble printing folder labels because the upper-cased fields don’t fit on the labels.

Anyways, I was working with one of Tower Software’s good technical support people, and in the process of troubleshooting, it came to light that we were running an older verison. I thought it was somewhat old, being as I installed it about a year ago, but the tech support guy acted like it was prehistoric. Anyways, Tower asked us if we could upgrade to the current release because that would make it very much easier for them to figure out what our problem was. I didn’t think it was too onerous a request, and since we pay for support we don’t have to buy a new version or anything.

I’ve got the latest greatest shiny CDs with TRIM 5.2.4 build 3611 ready to go. I hope that it’s still the current build by the end of the day. There seem to be a lot of builds, because I think our old one is build 2158. I’m sure that 99.5% of those were internal-only builds, but sheesh! Wish me luck.

This site is syndicatable, meaning other websites or users with web tools called feed readers (of which there are many) can subscribe to my weblog, and get notified when I post a new entry. I generally don’t know who reads this weblog, as the access log is huge, and I normally don’t bother looking at it. Today, for fun, I was perusing the access log to see what browsers most of my readers use. I noticed regular occurrances of some feed readers accessing the site on a daily basis. One I saw was Sharpreader, a Windows rss reader, and another was Newsfire, for Mac. A third was Liferea for Linux, and a fourth was Onfolio, a plugin for Internet Explorer in Windows. The interesting thing about all these rss readers was that they all were accessing my site from the same IP address subnet, which means they were probably multiple people from the same company. I did a reverse lookup on one of the IP addresses, and guess what I found?

dig -t ptr 254.68.17.203.in-addr.arpa.

resulting in:

254.68.17.203.in-addr.arpa. 120 IN PTR tower-gw.towersoft.com.au.

That’s right, I have some subscribers from Tower Software, makers of TRIM Context, the enterprise software we use for records management. Hi Tower guys! I knew they periodically looked at my blog, from communications I’ve had with them before, but I didn’t realize there were any subscribers. From my log files, it appears I only have about a dozen subscribers, and at least four of them are from Tower. Wow.

That’s customer attention, all right. I’ve had difficulty with their software, but when you communicate your problem to them, they go out of their way to make sure it gets handled.

Update: Geez, these guys really pay attention: Michael, Lindsay and Gord have already posted about this post.

I put our virtualized records management servers into production last night (and this morning). I had built working virtual servers of our two TRIM servers in Engineering this week, and all I had to do was to add RAM to a server, build a 3-drive RAID 5 array with the built-in RAID controller, install an OS, install VMWare GSX server, and copy over the virtual machines. Hah.

Installing RAM and building the RAID array went fine. Installing the OS (SUSE Linux Pro 9.3) went fine. Installing VMWare went OK, Copying over the virtual machines worked fine. Everything ran, except my third virtual machine, the OpenBSD transparent firewall I had built, would not route the packets. I’m not sure why, and even though I spent a couple of hours on it, I couldn’t get it to work. I ended up just bridging the TRIM virtual machines directly to the physical network the way they were on the physical machines, until I figure out the problem.

Then, when I rebooted the server, VMWare GSX server wouldn’t start up properly, or sometimes it would start but the web interface wouldnt’ start, or the management console would only connect from the local host but not over the network. I spent some time troubleshooting this, and decided to screw it, because I had a reliable working GSX server in Engineering running on SLES9 / Open Enterprise Server, so I just re-installed the OS as SLES9. I was pleased that I had decided to store the virtual machines on a separate partition, because I didn’t have to re-copy them over. After getting everything working again, the firewall still wouldn’t route packets, but the server would start cleanly, the virtual machines would start properly, and the remote console would connect as expected.

My final gotcha was that after the OS rebuild, the TRIM application wouldn’t connect to Oracle. That only took me 10 minutes to figure out though. I went over to the Oracle server, and looked at the crontab file. At 10:00 PM there was a script to stop the Oracle services. At 2:00 AM there was a script to start it up again. I looked at my watch and it was 1:50 AM. I waited 15 minutes to let Oracle start properly, and then did some testing. Everything worked properly, so I packed it in and left some time around 3:00 AM.

I am building virtual machines (VMs) from my TRIM servers, so that I can run two servers on one physical computer. The TRIM servers at present run on two existing physical machines. If you want to convert a real computer into a VMWare virtual machine, there are several ways to do it.

• Build the VM from scratch and configure it the same as the real machine.
• Use VMWare’s P2V Assistant to directly copy a physical machine’s hard drive into a virtual drive
• Take an image of the physical machine’s drive using a tool like Ghost and then manually reconfigure it into a VM
• Take a virtual disk restored from a Ghost image and use P2V Assistant to convert it into a VM
• Other methods I haven’t tried

I am using the Ghost -> vdisk -> P2V -> VM method, because at the moment, I don’t have a VMWare GSX server in my local production network. I only have one in the lab. At the end of this “virtualize TRIM” project, I’ll have a shiny new GSX server in production, but I need to recycle one of the existing physical machines used by TRIM right now fisrt.

I tried using an external USB hard drive we have as a destination disk for Symantec Ghost. You boot from a Ghost floppy, which loads SCSI drivers that talk to the USB disk, and then runs Ghost. Unfortunately, when you have HP ML370 servers like I have, they won’t boot from a floppy when you have an external USB hard drive attached.

My next try was using an external SCSI disk connected to the internal SCSI card. This worked for my first server, and I successfully built a VM from the image, but when I tried the second server, the Ghost image wasn’t readable in my staging GSX server. I tried three times, after checking and reformatting the external SCSI disk, but nothing worked. The Ghost image would be only partially readable each time.

I also tried booting the P2V boot CD on the real machine, and imaging the hard drive to a P2V helper machine running under VMWare Workstation 5. This was successful, but the resulting virtual disk was not bootable in VMWare GSX server; it only worked in VMWare Workstation.

Monday I’ll try using Ghost’s peer-to-peer imaging capabilities to image from the TRIM server directly to a file on another machine’s disk. Tomorrow I’m going to the races.

TRIM is our Records Management software. Although TRIM can do document management as well as records management, we use only the records management component. That makes TRIM very light-weight, because the records management component is a database application with a Windows client application. We run it as three servers: A database on Oracle, which is on a standalone server, the TRIM server, which is a set of small Windows services that manage all the data in the database, and a Windows terminal server, which runs the client. This arrangement works well, but the TRIM server services are so light-weight that it seems a shame to devote a whole server-grade box for them.

A few weeks ago I did a proof of concept to convert the TRIM server and the terminal server into VMWare virtual machines. This worked out well, and we discussed it in our most recent group meeting in the context of the other stuff we are doing right now. Virtualization of the TRIM server fits in well with our other initiatives, so now I am doing it for real.

The first step is to take images of our existing physical TRIM servers, and then I’ll be able to use VMWare’s Physical-to-Virtual assistant to convert the images into virtual machines.

It has been pointed out to me that I have been quite negative in my blog about TRIM, our records management system. The Manufacturer, Tower Software, is concerned that we are having difficulty implementing TRIM, and they have offered to help us resolve our problems, without our asking.

Before I continue, I just want to express something important. This weblog contains my own opinions, not those of my employer. The company does not necessarily share any of the opinions expressed on this site. I also write sarcastically sometimes which can be misinterpreted as extremely negative. I don’t wish to disparage Tower Software or TRIM in such a negative fashion, and my comments about TRIM should be interpreted as ribbing, not condemnation.

I feel the need to clarify a couple of things, because I don’t want anyone getting the wrong idea. First off, we are using TRIM for records management on our paper records, but we don’t use the document management functionality. For records management, we use TRIM to keep track of all our paper files. From a user perspective, TRIM is very effective at this function, and all our research into available products clearly indicated that TRIM was the best product for us to select. It serves a business need for our company, which is why we bought it. It works, it is stable, and the users like it. The goal of our IT group in this company is that we are here to help the business make more money by making our users jobs easier and more effective. We are not here to make our own jobs easier. If making the business more effective and profitable incurs some IT headaches, that’s a positive thing for the company.

I only deal with the back-end of TRIM. I am not a TRIM user. Our TRIM users don’t complain about TRIM much, if at all.

Our implementation of TRIM has been very positive for the company as a whole, and the pain and cost of our issues with TRIM have been small compared to the overall benefit from using it. Tower software’s support staff are also very knowledgeable and responsive, so in the instances where we’ve asked them about a problem they had encountered before, they were very helpful in getting it resolved.

Bart and Shawn, our programming dynamic duo, have been coding away on the new project initiation tool that will implement the filing and directory structure of our new Records Management system. Every time we start a new project, a directory structure gets created for it. For the past 8 years or so, we’ve used the time-tested projinit.bat batch file that Bart initially wrote way back. It has received a few updates by myself and Bart in the intervening time, but it is still essentially the same as the original. It just creates a new project directory and populates with a predetermined set of directories stored in a template. Then it applies access control rights to the newly created set of directories according to our security policy. This worked adequately for a long time.

Now we are moving to a more elaborate filing system that is based on activities, and as part of that system, the company wants to make the electronic directory structure for projects more similar to the paper filing system. We are revamping (that’s the royal we, meaning Bart and Shawn) the project initiation tool to facilitate this, and to automatically generate records for our records management software TRIM. TRIM provides an import mechanism to allow the creation of a large number of records at once from a CSV file.

Today I spent a few hours with Bart going over the XML template structure that he and Shawn designed for the tool, that represents our directory structure and the information needed by TRIM. We did a lot of consistency checking and validation of the data. Hopefully soon the system will be ready for rollout. I expect that by the end of next week it will have been tested for compatibility with TRIM and will be ready to roll out (if we can get the records management committee to stop making changes to the records structure).

We use TRIM from Tower Software for records management. I’ve blogged about Trim before several times, but this entry best describes it. It requires Windows domain authentication of the client, which means netbios has to be available between the client and the server. We don’t run Windows servers for anything else important, so we don’t let netbios travel over our WAN. Instead, to provide client-server access for TRIM, we run the client on one Windows server, the server on another one right next to it, and allow the users in using Terminal Services. Much more secure.

I have been having problems with the TRIM client, which is a win32 application. The Import / Export facility broke, and none of Tower Software’s technical support could help solve the problem, despite good effort. I built a vmware virtual machine, joined it to the domain, and installed the client on it, in order to try another approach with the Import / Export. This helped me solve the problem, but the solution left a sour taste in my mouth. When I tried the import on the vmware client, it failed like it did on the terminal server. However, it gave a different error message. It said it couldn’t create or access a file called TRIMport.mdb, which is an Access database file. It seems that some settings for the client are stored in this file, and the file lives in the application directory, which normal user logins don’t have write access to. It turns out that initially the file doesn’t exist, so you have to give a user write access to the shared installation directory of the TRIM client so that the file can be created there. Then, any user can go and bugger up TRIM. On top of that, you can’t even revoke write access to the TRIM client directory after the TRIMport.mdb file is created, because the Import and Export features seem to write some kind of temporary files there.

I think that’s a suprisingly bad design.

Bah. At least it’s working now, and I made a full copy of the TRIM directory before changing the rights.