| Saturday, September 23, 2006 |
17:40 - P4WNED
|
(top)  |
Some of you might recall what happened back in May, when my hosting company (Managed.com) got bought by another company, WebHostPlus. This entailed a move from the data center in San Jose to a "new" one in New Jersey, just across the GW Bridge from Manhattan. The transition was something of a debacle, as you can read at the link if you don't remember it. But I thought, hey—Managed.com's hardware had always been crashy and brittle, and WHP might actually turn out to be an improvement. Who knows?
Well, the hardware at WHP was actually better. It didn't randomly crash every few days. But it did have one or two incidences of mysterious service outages that lasted for several days at a stretch, which were resolved without so much as an explanatory bulletin from the company or even a helpful message on the support page. Just the ever-popular "Live Tech Support Chat" message button that rotated between variants of "Unavailable" or "Offline" or "Back Shortly".
Sometime around September 7th, one of my two servers at WHP abruptly went offline, with on explanation. (This was my other server, not the one hosting this blog and many other user services.) Then, a few hours later, a friend's server also hosted at WHP went down. Then mine came back up. Then this server, my other machine, went down. Chris's server came back up a day or so later, but my server remained offline for ... well, 13 days and counting.
There was never any adequate explanation given for this. WHP received hundreds of trouble tickets from its colocated and managed customers within those few days (judging by the incrementing ticket IDs), and they replied to all of them with this canned message:
Hello and Good evening. At this time unfortunately we are having network issues that should be resolved shortly. We apologize sincerely for the downtime and will be issueing credits based on a case by case basis. Please do not reopen multiple tickets, if you are still experiencing issues please just reopen 1 ticket so the admins can pass correct information that can be effeciently processed.
-whp staff-
I opened another ticket (as requested); today, nine days later, it's still open, assigned to some tech whose name suggests he might be in India, and has never been updated in that time.
Chris patrolled a number of hosting discussion forums, where he found lots of kindred spirits who had been marooned with WHP servers that had mysteriously gone offline, never to be seen again. From them, and from certain people who claimed to be "on the inside", he learned that apparently WebHostPlus had gone bankrupt—possibly from eating too many other hosting companies too quickly—and had to sell a whole bunch of their servers to pay the rent.
This may or may not be true. But considering that WHP tried to smooth over the inevitable challenge to their "guaranteed uptime" SLA by offering him a credit of 40 whole dollars, I'm not too disinclined to believe it.
Fortunately, I had backups. Unfortunately, my backup was in my garage, behind a dynamic IP, and with limited upload bandwidth. My restore process works best (or, indeed, at all) only if the source of the data is on a static IP. So I bought a hard drive at Fry's, copied the backup data onto it, and FedExed it to another friend in New York who has access to a high-speed network with a static IP.
Meanwhile, I set up with yet another hosting company: The Planet. I'd heard good things about them. And while their prices weren't the lowest, minutes after signing up I realized that this would be a good move, because their remote management system (orbit.theplanet.com—ya get it? Hyuck, hyuck) not only exists, something that no other company I've been with can boast, it's actually very sophisticated and functional. It has bandwidth usage monitoring with actual useful statistical projections, remote reboot and serial console access, a place where you can (voluntarily) register your root passwords so the techs can get into the machine for maintenance, hardware details, and thirty or forty other tools for serving customers' common needs. They even have top-notch FreeBSD support, including a local CVSup mirror for synchronizing your sources at LAN speeds rather than from the central repository. It's like being at some sort of geek spa.
Restoring the data began on Wednesday, and it completed late last night. There remain some issues to work out (having to do largely with the 6.1 kernel and the em driver not playing nice with IPFW), but I think it's stable now, or at least enough so as to allow me to get the blog back online at its new IP and post this update.
The old server is still down, and awaiting service; or, more accurately, it's probably at some pawn shop being mined for personal information. And yet WHP charged me for another month's service... while it was still down, and had been for over a week.
Now I have leisure with which to compose my "Screw you guys, I'm goin' home" letter. This ought to be cathartic.
|
|
Brian Tiemann
