Yesterday's outage
You may have noticed we had an outage that lasted all day yesterday. I just wanted to put up a quick note to fill you in on some of what's happened.
Yesterday our web host, Westhost, had a hardware failure in their data center. Apparently their back-up is several days old, which is why you see some missing posts on the front page of the site currently. I'm in touch with support to see if they have a newer version of the database.
I had performed a back-up of all the files on the site the night before, but hadn't had a chance to set up an automated database back-up yet. We only moved our hosting here a week-and-a-half ago. In fact, ironically enough, I found out about the outage when I was trying to log in through a secure shell to do a back-up of the database.
This is obviously bad. There is no circumstance where a 24-hour outage and data loss is acceptable, to me or to you. I am personally embarrassed about this, and am exploring options for hosting this site elsewhere as I try to piece together the missing data.
If you have any suggestions for hosting vendors, please leave them in the comments or email them to me at rusty.tanton@gmail.com. I hope you'll stick with us through these hiccups.
[Update 9:07 a.m.] Episodes, blog posts and comments have been restored. We're still waiting to hear from Westhost about having a newer database back-up. Here is what is still missing for now that we know of:
- User accounts and updates to user accounts created after May 16
- Statistics and tags for episodes submitted after May 16
- Podcasts created after May 16 (1 network program, 3 or 4 affiliates)
We still hope to be able to restore some or all of that information, but it may not be available for a little while. Thank you for patience.
[Update 11:39 p.m.] User accounts have been restored (passwords needed to be reset though, so you'll need to use the site's "email me a password" option next time you try to log in if you signed up after May 16 and before today). Affiliate podcasts have been added back to the directory.
The rest, I'm sad to say, appears to be gone. It's not much, but it's all important to us. A thousand apologies. I'll be in touch privately with the person who had started a network program that we couldn't retrieve information for.