Database Work This Weekend

Normally we just try to pretend that we never have any server problems…

but today, we had a kernel fall apart in our MySQL database server…that cascaded into a corrupt database that needed to be repaired. We were able to get the webserver back up, but without the database server, there was nothing to serve up correctly…so we went dark instead of serving up garbage.

We were down for about 2.5 hour today (feels like an eternity) and later tonight, we’re going down for 2 hours for a deep repair of our database.

The search engine on the top of our site has been removed as it’s overloading our servers.

A 5th high-end dedicated server is in the process of being acquired and this will be dedicated to the search function.

So far, no data has been lost and we have multiple redundant off-premise secure backups of our systems.

I/we apologize for the inconvenience this has caused and know that we are working on re-stabilizing our systems this weekend.

Posted by on October 21, 2005 at 8:13 pm | 1,932 views

4 Comments »


1
Ken Little writes:

Greatly appreciate the update Chris.
Hope the reapair work goes smoothly
Warm Regards
Ken Little

Comment provided October 21, 2005 at 8:34 PM

[Reply]

2
Jean Tracy writes:

I guess all this overload proves you are the greatest and most interactive. Hope you don’t get a headache over this.

Comment provided October 21, 2005 at 11:58 PM

[Reply]

3
Matthew C. Keegan writes:

As always, we appreciate the feedback.

I was a bit concerned yesterday when I pulled up my stats and got nothing but zeroes!

Best wishes with the work…a real *p-a-i-n* that must be done!

Comment provided October 22, 2005 at 10:04 AM

[Reply]

4
Ken Little writes:

Your efforts to get Jeff to share his secrets on the Teleclinic are greatly appreciated.
Thanks Chris

Comment provided October 23, 2005 at 6:56 PM

[Reply]

RSS feed for comments on this post.

Leave a comment

Please read our comment policy before commenting.