Crushbone down? - Page 4

Naeowan · 11-15-2007, 01:59 AM

/cries

danmuntz · 11-15-2007, 01:59 AM

Thanks for the update Gnobrin!

Ridiculous · 11-15-2007, 02:02 AM

Gnob please explain to me the interiors of what hardware? memory hardware of world? anything drastic? consistent glitches that can be fixed in an hour or two? I appreciate the info, just trying to get as much as I can. I'll stop drinking my coffee now if I know I'm only gonna get an hour of gameplay before ANOTHER patch occurs....

Gnobrin · 11-15-2007, 02:03 AM

Yes Anap, and this issue was not intentionally done to spite you. Unless you're one of the folks working in the server room at the exact time, patience is the only thing we can have in hopes of progressing the process.

This was an unforseen issue, one that is being worked on by the techs at this time. Believe me, they want the server up and going as much as you do, and are working hard to get it that way.

~Gnobrin.

Quinntus · 11-15-2007, 02:03 AM

As Anap pointed out, shouldn't there be redundant hardware systems to handle such outages?

Dzien · 11-15-2007, 02:03 AM

So what kind of reassurance will we get that this won't happen again? There seems to be a common theme involving crushbone and hardware problems. No other server experiences the same difficulties we do. It may be a minor problem for many, but it will have lasting effects on the progression of this server vs others. We are now at a good disadvantage for WW progression in terms of first kills etc.

andigro · 11-15-2007, 02:04 AM

Yeah, but we (Crushbone) will be 13+ hours behind the rest of the world as far as discoveries and such SMILEY<img src= " width="15" height="15" /> " width="15" height="15" />

Yeah, can you reset gamewide discoveries until all servers are up and running? That does kind of suck I agree.

danmuntz · 11-15-2007, 02:05 AM

Hah. You'll never know what hardware broke. I'd be kind of embarassing for them to say that it was a power supply, or a stick of memory, or maybe a caster on one of the wheels fell off. But, then again, the entire server may have collapsed in upon itself and melted into a pile of goo... then we would feel like crap for giving them a hard time when it was a real "oh-crap" issue. SMILEY

Kal · 11-15-2007, 02:05 AM

Well I have good news. I just saved a ton of Platinum by switching to Geico.:p

Gutterfly · 11-15-2007, 02:05 AM

Since Test and Test PVP are both showing down now in addition to Crashbone, I'm going to speculate they are robbing from Peter to pay Paul.

Quinntus · 11-15-2007, 02:06 AM

At least we know that somebody is there after hours doing something.

Morgue · 11-15-2007, 02:07 AM

Anyone that played EQ around the time of Kunark.. will remember the late delivery on over half the orders. We did all get one month free time the servers where to say the least unplayable for 2 days after the launch.. It only strike me as ironic that Rise of Kunark would once again cause EQ even its distant kin EQ2 as much trouble as before. But at the time of Kunark for EQLive, I dont think sony had bought Everquest yet atleast not completely.

Just a thought...

-raz

danmuntz · 11-15-2007, 02:07 AM

Gnobrin wrote:

Yes Anap, and this issue was not intentionally done to spite you. Unless you're one of the folks working in the server room at the exact time, patience is the only thing we can have in hopes of progressing the process.
This was an unforseen issue, one that is being worked on by the techs at this time. Believe me, they want the server up and going as much as you do, and are working hard to get it that way.
~Gnobrin.

Ack, what? I'm not popular enough to be single-spited yet by SoE? Darn. SMILEY

Thanks Gnobrin for the updates. Tell the guys in the server room that I will send them cookies whilst they work!

Magar · 11-15-2007, 02:07 AM

I don't chase discoveries myself, but I do feel badly for those who are after that piece of the pie. One would only hope that some sort of compensation would be forth coming.....

Meskin · 11-15-2007, 02:07 AM

My previous post stands! I am on AB, Meskin, look me up, we can start our own guild, CB Outcasts or something and still enjoy the game.

Meskin · 11-15-2007, 02:09 AM

Kaldu@Crushbone wrote:

Well I have good news. I just saved a ton of Platinum by switching to Geico.:p

LOL! You and the dude from Beverly Hillbillies!

Ridiculous · 11-15-2007, 02:09 AM

I was thinking the exact same thing....

andigro · 11-15-2007, 02:11 AM

I think I figured out how it broke:

zukem · 11-15-2007, 02:12 AM

Quinntus wrote:

At least we know that somebody is there after hours doing something.

Hopefully, it's not the Janitor on the phone with a dev. Getting instructions on what to do. (Don't laugh, I worked for a fortune 500 company. And our idiot General Manager asked why one of our guys wasn't in. We told him he had to come in to fix a problems with a server last night. His answer was, couldn't you have told one of the telemarketers, who was in the building at the time, and walked them through it.)

Fatkiddown · 11-15-2007, 02:12 AM

Quinntus wrote:

3 hours for a single server to be down is tad odd. Power supply go bad?

You may know this but just in case for those that don't know, Crushbone "server" is most likely alot more complicated than just 1 physical server. Most likely we're talking about at least several 44U racks full of servers/storage/fibre channel backplanes working as a cluster. There is no magical quick fix switch that they can flip to get the server back up. Blizzard npsoft whoever you want to pick would be in the same boat if faced with the same hardware issue (if thats the case).

Yea it is nice to have play by play of the status but I know from the hardware end I really could care less what the users think if I'm working on fixing something I get it done as soon as possible end of story and when it is fixed it is fixed. Wasting my time giving touchy feely updates to some PR guy is only prolonging the time it takes to fix the issue.

So all the crushboners that are "like totally omg whats taking so long" take this time to clean the cheetos from your keyboard and let the tech's do their job because there is no doubt that they will fix the issue as soon as possible.

Borga007 · 11-15-2007, 02:12 AM

ohhhh! I think we're gettin closer i hope.. server status screen says crushbone (locked) now istead of down!

DeadGopher · 11-15-2007, 02:13 AM

Gnorbin,

Why isn't there redundancy built into the hardware?

I work in a datacenter that supports over 3000 servers. We would never tell a client that their production application that pays their bills is down because of a hardware problem. That's unacceptable. Hardware does fail, but it should never cause an extended outage to production.

Even if a server fails, that's what clustering is for. The other nodes in the cluster share the extra work until the broken server node can be brought back into the cluster.

Fatkiddown · 11-15-2007, 02:13 AM

andigro wrote:

I think I figured out how it broke:

They run Unix servers, nice try though

Quinntus · 11-15-2007, 02:13 AM

Hmm, look at thisAll servers are up except for Crushbone (is locked) Test PVP Testthe "is locked" is new (within the last 30 min)

Ridiculous · 11-15-2007, 02:13 AM

come onnnnn come onnnnn I've been up for the past three days of my "Vacation" playing EQ consistently, this is tearing me apart from valuable time. extremely valuable time... and to think if I ever spend my Paid Vacation in Norrath AGAIN!

Quinntus · 11-15-2007, 02:14 AM

Its up

Fatkiddown · 11-15-2007, 02:14 AM

Anap@Crushbone wrote:

Yeah, but we (Crushbone) will be 13+ hours behind the rest of the world as far as discoveries and such " /> " />

So does that make you cry inside?

Ridiculous · 11-15-2007, 02:15 AM

kaching

danmuntz · 11-15-2007, 02:15 AM

I shed a small tear, yes SMILEY

zukem · 11-15-2007, 02:15 AM

Crushbone is up!!!! Or maybe not. Stuck on entering game. /sigh

11-15-2007, 01:59 AM	#92
danmuntz Loremaster Join Date: Dec 2005 Posts: 95	Thanks for the update Gnobrin! __________________

11-15-2007, 02:02 AM	#93
Ridiculous Server: Crushbone Loremaster Join Date: Mar 2005 Location: United States Posts: 13	Gnob please explain to me the interiors of what hardware? memory hardware of world? anything drastic? consistent glitches that can be fixed in an hour or two? I appreciate the info, just trying to get as much as I can. I'll stop drinking my coffee now if I know I'm only gonna get an hour of gameplay before ANOTHER patch occurs.... __________________ Seeks the eye and finds to die.

11-15-2007, 02:04 AM	#97
andigro Champion Join Date: Nov 2007 Posts: 4	Yeah, but we (Crushbone) will be 13+ hours behind the rest of the world as far as discoveries and such " width="15" height="15" /> " width="15" height="15" /> Yeah, can you reset gamewide discoveries until all servers are up and running? That does kind of suck I agree.

11-15-2007, 02:05 AM	#98
danmuntz Loremaster Join Date: Dec 2005 Posts: 95	Hah. You'll never know what hardware broke. I'd be kind of embarassing for them to say that it was a power supply, or a stick of memory, or maybe a caster on one of the wheels fell off. But, then again, the entire server may have collapsed in upon itself and melted into a pile of goo... then we would feel like crap for giving them a hard time when it was a real "oh-crap" issue. __________________

11-15-2007, 02:05 AM	#100
Gutterfly Loremaster Join Date: Mar 2005 Posts: 20	Since Test and Test PVP are both showing down now in addition to Crashbone, I'm going to speculate they are robbing from Peter to pay Paul. __________________ ...singing in the rain

11-15-2007, 01:59 AM	#91
Naeowan Loremaster Join Date: Jan 2005 Posts: 6	/cries

11-15-2007, 02:03 AM	#94
Gnobrin Steam-monkey Join Date: Jul 2007 Location: Ak'Anon Posts: 1,007	Yes Anap, and this issue was not intentionally done to spite you. Unless you're one of the folks working in the server room at the exact time, patience is the only thing we can have in hopes of progressing the process. This was an unforseen issue, one that is being worked on by the techs at this time. Believe me, they want the server up and going as much as you do, and are working hard to get it that way. ~Gnobrin.

11-15-2007, 02:03 AM	#95
Quinntus Loremaster Join Date: Nov 2004 Posts: 24	As Anap pointed out, shouldn't there be redundant hardware systems to handle such outages?

11-15-2007, 02:03 AM	#96
Dzien Server: Crushbone Guild: Legacy Rank: Guild Leader Master Join Date: Aug 2007 Posts: 1	So what kind of reassurance will we get that this won't happen again? There seems to be a common theme involving crushbone and hardware problems. No other server experiences the same difficulties we do. It may be a minor problem for many, but it will have lasting effects on the progression of this server vs others. We are now at a good disadvantage for WW progression in terms of first kills etc.

11-15-2007, 02:05 AM	#99
Kal Server: Crushbone General Join Date: Aug 2006 Posts: 1	Well I have good news. I just saved a ton of Platinum by switching to Geico.:p

11-15-2007, 02:06 AM	#101
Quinntus Loremaster Join Date: Nov 2004 Posts: 24	At least we know that somebody is there after hours doing something.

11-15-2007, 02:07 AM	#102
Morgue Server: Crushbone Guild: After Dark Rank: Shadow Overlord Alts General Join Date: Nov 2004 Posts: 47	Anyone that played EQ around the time of Kunark.. will remember the late delivery on over half the orders. We did all get one month free time the servers where to say the least unplayable for 2 days after the launch.. It only strike me as ironic that Rise of Kunark would once again cause EQ even its distant kin EQ2 as much trouble as before. But at the time of Kunark for EQLive, I dont think sony had bought Everquest yet atleast not completely. Just a thought... -raz

11-15-2007, 02:07 AM	#103
danmuntz Loremaster Join Date: Dec 2005 Posts: 95	Gnobrin wrote: Yes Anap, and this issue was not intentionally done to spite you. Unless you're one of the folks working in the server room at the exact time, patience is the only thing we can have in hopes of progressing the process. This was an unforseen issue, one that is being worked on by the techs at this time. Believe me, they want the server up and going as much as you do, and are working hard to get it that way. ~Gnobrin. Ack, what? I'm not popular enough to be single-spited yet by SoE? Darn. Thanks Gnobrin for the updates. Tell the guys in the server room that I will send them cookies whilst they work! __________________

11-15-2007, 02:07 AM	#104
Magar Loremaster Join Date: Dec 2004 Posts: 10	I don't chase discoveries myself, but I do feel badly for those who are after that piece of the pie. One would only hope that some sort of compensation would be forth coming.....

11-15-2007, 02:07 AM	#105
Meskin Loremaster Join Date: Jul 2006 Posts: 35	My previous post stands! I am on AB, Meskin, look me up, we can start our own guild, CB Outcasts or something and still enjoy the game.

11-15-2007, 02:09 AM	#106
Meskin Loremaster Join Date: Jul 2006 Posts: 35	Kaldu@Crushbone wrote: Well I have good news. I just saved a ton of Platinum by switching to Geico.:p LOL! You and the dude from Beverly Hillbillies!

11-15-2007, 02:09 AM	#107
Ridiculous Server: Crushbone Loremaster Join Date: Mar 2005 Location: United States Posts: 13	I was thinking the exact same thing.... __________________ Seeks the eye and finds to die.

11-15-2007, 02:11 AM	#108
andigro Champion Join Date: Nov 2007 Posts: 4	I think I figured out how it broke:

11-15-2007, 02:12 AM	#109
zukem Loremaster Join Date: Aug 2006 Posts: 18	Quinntus wrote: At least we know that somebody is there after hours doing something. Hopefully, it's not the Janitor on the phone with a dev. Getting instructions on what to do. (Don't laugh, I worked for a fortune 500 company. And our idiot General Manager asked why one of our guys wasn't in. We told him he had to come in to fix a problems with a server last night. His answer was, couldn't you have told one of the telemarketers, who was in the building at the time, and walked them through it.)

11-15-2007, 02:12 AM	#110
Fatkiddown Loremaster Join Date: Jan 2005 Posts: 95	Quinntus wrote: 3 hours for a single server to be down is tad odd. Power supply go bad? You may know this but just in case for those that don't know, Crushbone "server" is most likely alot more complicated than just 1 physical server. Most likely we're talking about at least several 44U racks full of servers/storage/fibre channel backplanes working as a cluster. There is no magical quick fix switch that they can flip to get the server back up. Blizzard npsoft whoever you want to pick would be in the same boat if faced with the same hardware issue (if thats the case). Yea it is nice to have play by play of the status but I know from the hardware end I really could care less what the users think if I'm working on fixing something I get it done as soon as possible end of story and when it is fixed it is fixed. Wasting my time giving touchy feely updates to some PR guy is only prolonging the time it takes to fix the issue. So all the crushboners that are "like totally omg whats taking so long" take this time to clean the cheetos from your keyboard and let the tech's do their job because there is no doubt that they will fix the issue as soon as possible.

11-15-2007, 02:12 AM	#111
Borga007 Loremaster Join Date: Jan 2007 Posts: 6	ohhhh! I think we're gettin closer i hope.. server status screen says crushbone (locked) now istead of down!

11-15-2007, 02:13 AM	#112
DeadGopher Loremaster Join Date: Nov 2004 Location: Minneapolis, MN Posts: 63	Gnorbin, Why isn't there redundancy built into the hardware? I work in a datacenter that supports over 3000 servers. We would never tell a client that their production application that pays their bills is down because of a hardware problem. That's unacceptable. Hardware does fail, but it should never cause an extended outage to production. Even if a server fails, that's what clustering is for. The other nodes in the cluster share the extra work until the broken server node can be brought back into the cluster.

11-15-2007, 02:13 AM	#113
Fatkiddown Loremaster Join Date: Jan 2005 Posts: 95	andigro wrote: I think I figured out how it broke: They run Unix servers, nice try though

11-15-2007, 02:13 AM	#114
Quinntus Loremaster Join Date: Nov 2004 Posts: 24	Hmm, look at thisAll servers are up except for Crushbone (is locked) Test PVP Testthe "is locked" is new (within the last 30 min)

11-15-2007, 02:13 AM	#115
Ridiculous Server: Crushbone Loremaster Join Date: Mar 2005 Location: United States Posts: 13	come onnnnn come onnnnn I've been up for the past three days of my "Vacation" playing EQ consistently, this is tearing me apart from valuable time. extremely valuable time... and to think if I ever spend my Paid Vacation in Norrath AGAIN! __________________ Seeks the eye and finds to die.

11-15-2007, 02:14 AM	#116
Quinntus Loremaster Join Date: Nov 2004 Posts: 24	Its up

11-15-2007, 02:14 AM	#117
Fatkiddown Loremaster Join Date: Jan 2005 Posts: 95	Anap@Crushbone wrote: Yeah, but we (Crushbone) will be 13+ hours behind the rest of the world as far as discoveries and such " /> " /> So does that make you cry inside?

11-15-2007, 02:15 AM	#118
Ridiculous Server: Crushbone Loremaster Join Date: Mar 2005 Location: United States Posts: 13	kaching __________________ Seeks the eye and finds to die.

11-15-2007, 02:15 AM	#119
danmuntz Loremaster Join Date: Dec 2005 Posts: 95	I shed a small tear, yes __________________

11-15-2007, 02:15 AM	#120
zukem Loremaster Join Date: Aug 2006 Posts: 18	Crushbone is up!!!! Or maybe not. Stuck on entering game. /sigh