More upstream problems today

April 1, 2009

Engine Yard suffered an outage today from their datacenter. Here’s what they had to say:

We have been in constant contact with our Herakles data center throughout this service interruption. Unfortunately, root cause determination at Herakles for the ISP outage is unknown at this time. Herakles started troubleshooting at the core switching, then to the routers coming into the facility, and are now focused on the distribution switches inside the data center. At this time, Herakles data does not have an ETA for resolution of this problem. We will continue to keep you updated as we hear information from the Herakles data center operations team.

Unfortunately, after the issue was resolved, DevjaVu was still down. Since then I’ve been in contact with them and they’re on it. There’s no ETA, but my guess is within the next hour. Although I’m sure they have a number of follow up issues with their clients after this massive outage.

Anyway, sorry about this outage. If only DevjaVu had an offline mode, no?

Update @ 4/1 10am: No April Fools here. Downtime was extended overnight due to a billing glitch. It’s been resolved and Engine Yard is currently restoring our slices, which should be up momentarily.

Update @ 4/1 1pm: We’re back! After much unnecessary delay from problem after problem after problem. *sigh* I feel like I could have done better. Feel free to yell at me next time you see me.

-Jeff

7 Responses to “More upstream problems today”

  1. Chao Says:

    Yes, we need you guys!


  2. Thanks for the update.

  3. Chao Says:

    Any ETA on this? It’s really affecting us now.

  4. Kevin Lewis Says:

    Any further updates? Much appreciated.

  5. Jeff Lindsay Says:

    Update added to the post. Things are really on their way back up now.

  6. Ted Paff Says:

    update? when its live, can you please post it here?

  7. AD Says:

    Devjavu is down again for about an hour now. Could be signs of a recurring issue?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: