Office Backups: The Journey, Part 1
Posted on November 17, 2007
8 Comments
I’ve had quite an education these past few months while trying to keep office backups chugging along. It’s been interesting, but I have to admit that I’m ready to be done learning for a bit. I need to get to a place where backups “just happen.”
I’ve now experienced MozyPro and SOS Online Backup. I’ve spoken with folks from several other vendors or solutions, including Iron Mountain, HP and IBM. What follows are my experiences and final solution (hint: none of the above).
This all started when I took over Technical Operations for my employer. I determined we needed better backups in place and I had no (none. zip. zero. nada) desire to do tape backups. Yucky. Thus, I entered the world of online backups…
MozyPro
The MozyPro solution is the one I have the most experience with. I started with them back in early July and have just started to phase them out this month. It’s a solid solution and I highly recommend them up until you cross some sort of threshold (either very large number of files or size, not sure which). Then things, in my experience, start to get sketchy.
Right after I started with MozyPro I had a major RAID array crash that caused all sorts of issues. Not long after that I was inspired to write Things I Wish I’d Thought of Regarding Online Backups. You know, the common sense stuff: be prepared for long initial backups, make sure you understand restore time frames, etc.
But really, after the initial debacle, things were smooth for a few months. What few small issues I did have were easily handled by the support folks. Speaking of their support folks — I found them to be quite helpful and very responsive until just recently. I suspect their recent acquisition news has caused a large growth in business? Great bunch of folks though.
Around the time of their acquisition, I started to have some problems with backups not completing on my servers that have larger numbers of files and data (20GB and 200k to 300+k files). Either the client would time out or there would be a variety of network/server interruptions. They shot me a few different beta versions but things didn’t improve enough. Ultimately I had to move on. I was spending hours a week manually restarting backups just trying to get one solid backup each week. That was making me tense.
To be clear though, on my other servers with lower file counts/backup size, MozyPro has continued to be rock solid. I still use the non-Pro version of Mozy at home and continue to love it as well.
SOS Online Backup
After spending some time researching and exchanging email and phone calls with many companies and their sales teams, I settled on SOS Online Backup as the next service to try.
I was assured that my backup sizes wouldn’t be a problem, nor would the large amounts of files that some of my servers have. The reviews I found were quite positive. I signed up for the 30 day free trial.
I suspect the claims are true as I didn’t see any evidence off issues with my backup sizes. However, I had other issues. On three different machines I managed to rack up a variety of assorted .NET framework errors when running their client. What was odd was that it would blow up on some specific action, yet if I went right back into the client it would be fine.
However, the deal breaker came when I realized that while it did seem to be handling the backup sizes, it was slow. For instance, after a few days of initial backups, I stopped the client (during business hours). I then restarted it that evening and monitored things a bit. It took it over 3 hours to inspect the 100,000 files that had been backed up.
Now, there are over 300 thousand files on that server (MozyPro would usually do it in 1-2 hours before the timeout issues started). Extrapolating that out tells me that I could be looking at a 10+ hour backup window each night. Perhaps not a huge deal, but while running this client also tends to peg a CPU. Sure, the server in question is a dual dual-core, but still… I decided that was just too extreme (and slow) and terminated the trial.
Iron Mountain
Just a quick note on Iron Mountain — I haven’t tried ‘em yet. They are, by far, the most expensive of the services that I’ve considered. They also appear, at least on paper, to be the most polished, complete and perhaps even mature(?).
Turns out some of the bigger companies re-brand and sell from these guys. For instance, the price I got from HP would’ve have been using the Iron Mountain solution, but at a several hundred dollars a year premium.
They were next in the queue, but…
Now What?
While pondering what to try next (and how much I wanted to spend), and continuing to vow to avoid tape, I was starting to get even more tense and concerned. Fortunately, while working on a different project I happened across a great solution.
More on that in Part 2!
Tags: backup, Iron-Mountain, mozy, mozyPro, operations, SOS-Online-Backups
Possibly Related Posts
Comments
8 Responses to “Office Backups: The Journey, Part 1”
Leave a Reply
Had the exact same experience with Mozy Pro. Started out great, backing up a server with 10GB of data and about 40,000 files. When the data got up to 30GB and 800,000 files, it started developing connection errors. Tech Support was friendly, and they were quick to admit it was “back end” errors on their side, I have lost faith in them for server backups. Like you, I use them for smaller home related backups and have never had a problem. I also think they will eventually fix their back end problems since they have the money from the EMC buyout.
Bob
Ok, I don’t know what the love for online backups is…I really *really* don’t like them, all I see is the cons…slow times (no matter how good their service is, your maximum speed is still just as fast as your internet pipe is…and nobody has gigabit internet…at least none of my clients); then you are putting your data in the hands of a complete unknown…your important business data…ugh, don’t get me started on that. And if you talk about encryption…think about even more time.
For me, backups are done one way and one way only…amanda+removable HDs+gigabit ethernet. Amanda rules the world when it comes to backups, it backs up linux or windows with ease, fast as all hell, and the restore options are simple, because it uses, at least in linux, standard tools to do the actual backup archives.
@bob - I’m actually glad to hear that as it helps me realize I’m not the only one with the issues. I kept asking, “Am I really the only one with these issues?” but never got a square answer…
@vox - good grief man, of course encryption is in the mix! Files are encrypted before they ever leave the machine and are sent over an encrypted link.
Until tonight, the only Amanda I’ve ever heard of is my niece living 3 states away. I run a (mostly) windows shop. Also, I don’t have the luxury of running gigabit between the office servers and our two remote data centers… I probably won’t be installing samba or cygwin on my production servers to explore this option (and can a windows server even be the repository?!?)
That being said, I think you’ll find part 2 of this a bit more palatable, at least from an architectural standpoint.
Also, thanks for mentioning it. Definitely a product that’ll be good to be aware of for the future in other environments!
The thing with encryption is that it slows down things ever more than the throttled bandwidth of your ISP…and when you have a 30,000 files backup, it’s gonna be slooooow…10hr for a backup? only for a complete network, not for a single server.
As for amanda…I don’t know if it runs on a windows server as backup repository, but…you don’t need to install samba or cygwin on your windows backup clients…samba = SMB = CIFS…it’s how windows shares files with windows…so, no…you’d only have to install, at worse, a small linux box (amanda is not a resource monster) and the amanda clients on your boxes, at best (if it runs as backup server on windows), just the amanda clients.
As for remote data centers….it all depends on how they are connected to the main one…but I’d probably go the sneakernet way…local backup server with detachable HDs and shipping around once a month.
Amanda is a PITA to understand because it’s completely different from the usual backup paradigm (you don’t tell it when you want a full backup and incremental backups…it decides on its own depending on network load and other factors, you just tell it “I want a full backup once a week” or once a month or whatever, and it decides on its own when it happens), but once you got it working, nothing comes close to it, IMNSHO and experience.
Hell, I’m thinking that if your major problem is the upload time with online backups, I’d do an on-disk backup with amanda on each site and just backup the amanda server online…that way, you have the best of both worlds…a local fast backup and an offsite backup for when the building burns down.
How well does Amanda handle NTFS permissions? I started using Unison with Cygwin and ran into several issues, especially on Vista. Unison uses Rsync technology to perform copies which means it’s able to do “diff” file updates as opposed to entire file copies every time it changes (read: less bandwidth.) But I’m finding that *nix archive tools on Windows don’t work so well because of permissions and file locking issues, or even long file names. So now I’m playing around with SyncToy and possibly Robocopy, both Windows oriented. I back up to a remote (parent’s house) NAS device (NSLU2) running Linux firmware that allows me to tunnel SMB over SSH, ensuring the traffic is encrypted. I can also enable SSH compression to cut down on bandwidth and utilize its fast blowfish (or even faster, RC4) encryption cypher. Seems to work pretty well and I don’t have to worry about loading some company’s proprietary software on my computer or worry about them snooping through my files–or worry about having to pay them!
When restoring, Mozy Pro Can take 24 hours (or more, as I sit waiting at hour 30) on a weekend (weekdays would take longer) to simply GATHER the files that you need to download before you can even start to download them. (50k files or a VERY SMALL server) This makes it worthless for Dr.s lawyers, dentist, hotels,… or anyone who needs their data restored rapidly.
Further, I have been on the phone waiting simply for someone to answer the phone for over an hour so far with no response.