Wrestling with ESXi Client Backups

Continued from Fun with ESXi and HP issues. Learning Lessons.

To recap, I had two new drives and a backplane en route and really wanted to get complete image backups of the VM clients running on this VMware ESXi server before I started swapping drives. I’d tried the vSphere built-in file copy utility, FTP and SCP but all had timeout issues on copying the large disk image (.vmdk) files.

While looking for more backup options I recalled reading about ghettoVCB in the past and this seemed like a great time to check it out. This is a very well written – free! – script that you can use to backup ESX(i) clients’ files to direct attached storage (which I’m avoiding for obvious reasons!) as well as NFS or iSCSI volumes. While I don’t have anything currently offering NFS or iSCSI drives I figured I could whip something together easily enough.

First off, I was looking at one of my Windows 2003 file servers. This machine has tons of unused storage and an unused network card so I figured it would be a good place to start. Unused network card is nice to keep all the network traffic for NFS/iSCSI off of the main network.

Hey, did you know that Windows Services for NFS is built-in to Windows 2003 and 2008 Server versions? I didn’t either until last week. Seemed worth a shot so I checked the box on the Windows Component Installer and gave it a shot.

Tip: When prompted for Windows media pay attention to the differences between x32 and x64 versions. I had no issues installing this component from x32 media – alas, the server is x64 and the service definitely didn’t start! Took an uninstall and a reboot to get back on track…

Once I sorted through some “user issues” I was ready to try connecting the ESXi server to the Windows exported NFS share. Sadly, no matter what I tried I couldn’t seem to hit the right combination of options to get this to work.

A few days later I noticed that checking “Microsoft Services for NFS” doesn’t give you all of the NFS service options… In particular, you don’t get the options around authentication and user mapping. This could very well explain why I couldn’t get it working. I will be revisiting this in the near future. I guess the other way to do this is via Windows Services for Unix, something I have very little experience with but might check into.

Time being of essence I set aside the Windows option and moved on to try with my OpenFiler server (first mentioned back in December of ‘09). Enabling the secondary network card and then configuring NFS on it took about 5 minutes. Connecting the ESXi server to it took about another minute. Easy!

I configured ghettoVCB and gave it a try. The first few machines backed up nicely! Alas, the next one had issues – a few file copy restarts and then it finally gave up after a timeout error.

Side note: My OpenFiler server is running an older pair of Hyper-Threaded Xeon 3.0 Ghz processors. When the backup to NFS is running it is seriously bogged down. That caught me by surprise.

After farting around with backups for (literally) over 12 hours I finally opted to accept the fact that I wasn’t going to get them all backed up and went ahead and did the hardware swaps. I figured perhaps the drives being in a degraded state were causing my issues.

Nope.

All the hardware has been replaced. All drives happy, RAID arrays rebuilt and everything is “green” and good to go on the server. I still can’t get reliable ghettoVCB backups and, perhaps related, I still can’t get full copies of the images using vSphere, FTP or SCP either.

At the moment I’m thinking this may be more of an ESXi issue since I can’t get good copies with or without NFS in the mix. Or maybe I need to try iSCSI from the OpenFiler server? I’ll need to spend more time on this one as I think ghettoVCB backups would be invaluable – once I can get all my stuff working.

Possibly Related posts: