Grid down

Huh, The OSgrid is down. Only twitter mentions some downtime, but not what the reason is, nor how long it will take to get things back up and running. The message on twitter states :

ALERT : Grid is offline possibly for the next few days or maybe even longer, we are sorry about this inconvenience but it is unavoidable!

I assume / hope it will be back up before friday, as OSG6B is scheduled to start on that day.  I see some forum topics here and here  that could indicate something went FUBAR on the grid servers. Last outage was in august 2012  just after OSG5B. It was resolved within some hours back than.

I assume the admins will keep us posted when there is news to share. If it remains silent with updates here, you can always check OSG's Forum, Twitter, Newsfeed or scroll down to the bottom of this page, and join the IRC channels to try and get a hold of any admins. ( mind they might be to busy to chat when there trying to fix stuff ). Mind your sumilators will not start, or produce a gazzilion of errors if still "up". best shut them down, and wait for the heads up.

Update : Well that didn't take long. Looks like all is online again. Found this on Nebadons G+
UPDATE : Grid is back online we were able to do a quick band-aid to get things moving again, may be more outages later this week! Stay Tuned

Red Dragon moved & Grid outage

Last week, after installing the latest version of the OSG software, we put the Red Dragon back to "old" map positions. Also the previous build that used to lag insanely, is now back up and running properly and as it used to.  When writing this the grid is Offline for over an hour already, although were still partying hard on D&J's friday party at close encounter sim.

The quote of the day in the category : "Things you don't want your sysadmin to say" :
<nebadon> I accidently whacked the inventoryitems database
<nebadon> gonna have to restore from backup
No stress Neb, we all know the feeling of computers not doing what we would like them to. You'll fix it.
 (put's reminder in to read "excuses.txt" in next version of OSG ;-).

It is a thing we rarely see. OSgrid being down. As a backup is being restored we expect this should not have much impact, other than some time. You can keep track here when the services are back up and running, or be patient and keep an eye on the twitters, or IRC ( scroll to the bottom of this page for that ) to keep updated on the status / news. In the meantime, we party on !

Update: Grid already up again. Downtime was limited to less than 6 hours, and only inventory of the last 12 hours prior to that can be missing.
Chapeau for OSG. We have suffered 0 impact from this whole database issue. Now that's disaster recovery management that deserves 2 thumbs up.

If you also like the OSgrid, and the awesome service they offer you, you might want to consider sending them a Donation . Bang for your bucks, and money well spent.