Updates for test first?

Sv_Lor · 09-28-2005, 11:58 PM

I don't know if Devs read this or not, but I have to at least say it; why in the world is live getting updates before test? Kinda defeats the purpose of having a test server, am I wrong?

AsheBail · 09-29-2005, 12:09 AM

Are you starting fires? Patch #14a hasn't hit live yet. Or are you referring to the little touchups we get after the servers come down daily? I suposse you wish a week of testing of those before they go live. So if we are lucky we may get these little small touch ups a few weeks later. Thanks lots for your post.

Sv_Lor · 09-29-2005, 12:19 AM

Talking about these, which do show up as updates; *** Desert of Flames *** - Those who have completed the three "Thief in the House of..." quests can now progress in "Welcome to the Peacock Club." *** The Splitpaw Saga *** - The ghosts in Anvilpaw's Grotto should no longer flee out of range. *** Gameplay *** - Adventure experience rewards have been increased for groups of 3 to 6 players. Solo/duo experience is unaffected. *** Spells and Combat Arts *** Ranger changes: - Arrow Rip will now summon Indium Arrows. Warrior changes: - Hold the Line can now be canceled via the Maintained window.

Ryan "Blackguard" Shwayder

EtoilePirate · 09-29-2005, 02:38 AM

The little hotfixes -- pataches with just a couple of items in them, a mob tweaked, or an item readjusted, or a quest-giver fixed, that sort of thing -- have always gone live nearly daily, without stopping on Test first (or hitting Test at the same time). It's been like that since before I started playing, and that was back in February. 14A is still Test-only, and since this is Wednesday I wouldn't be surprised if 14B is arriving in a matter of hours. (Either that or Friday; those are the post popular Test-patching days.) Usually no-one notices the hotfix patches going straight to Live unless they bring big bugs.

AsheBail · 09-29-2005, 07:46 AM

Sv_Lorax wrote:
Talking about these, which do show up as updates;

*** Desert of Flames ***

- Those who have completed the three "Thief in the House of..." quests can now progress in "Welcome to the Peacock Club."

*** The Splitpaw Saga ***

- The ghosts in Anvilpaw's Grotto should no longer flee out of range.

*** Gameplay ***

- Adventure experience rewards have been increased for groups of 3 to 6 players. Solo/duo experience is unaffected.

*** Spells and Combat Arts ***

Ranger changes:
- Arrow Rip will now summon Indium Arrows.

Warrior changes:
- Hold the Line can now be canceled via the Maintained window.

Ryan "Blackguard" Shwayder

I apologize for being a little annoyed but can you see how testing these for a length of time may be a little silly? Did you read them? Take 'Arrow Rip' for an example, it will either summon 'Indium Arrows' or not. Both of which isn't a biggie. Should we wait for Patch #14 for these to be put in game?

Daish · 09-29-2005, 08:45 AM

How about these... these seem trivial to you?

*** Combat and Spell Changes ***

- The difficulty of orange and red con creatures has been increased.
- Experience gained from epic targets has been reduced.

*** Epic Encounters ****

- The experienced gained from killing epic targets is now comparable to that of killing heroics.

*** Gameplay ***

- Players now mitigate much less of the damage done by encounters that con orange and red. It is recommended that low orange encounters be attempted only by very well-equipped players, and even then there is no guarantee of success. High-orange and red encounters are meant to be deadly, so it is wise to avoid fighting them.

*** Gameplay ***

- Defeating single opponents with one or two up arrows will now provide an experience bonus similar to that of defeating encounters of grouped down-arrow targets.

- High yellows, oranges, and red con mobs are now slightly harder to hit with spells and melee attacks.

All of that was done directly to the LIVE SERVERS with never a minute seen on any test server. This was after 4 MONTHS of work on the combat revamp. These have all happened in the weeks since the combat revamp went live.

I think the OP's point was the fact that whats the point of having a test server if you dont use it? Every single change presents an opportunity for a bug, a crashing issue, or some major change in gameplay. If there are changes that do not warrent time on the test server, then that specifically means that they believe that the paying customers will 'enjoy' testing it for them.

The 6 or so different changes to harclaves is another example of why one should test changes before you put them on live.

Planning to finish tweaking the combat revamp directly on the LIVE SERVERS should be unacceptable.

AsheBail · 09-29-2005, 11:57 AM

You are a little bit off. Its a wee bit more subtle. Those were changes made to update #13 because they didn't test patch #13 enough before its release.

They looked at the numbers after the big patch and decided to adjust some things after the live servers actually did the further testing of #13. But now I manage to support what you said because these fixes were implemented too early due to not enough testing of Patch #13 before release.

What I say to rebutt this is, have you ever went on the test server? One can generally do a /who all and find only 50 players most under lvl 20 playing. People generally don't wish to play on the test server. Most likely, there could be testing of a patch done for a full year before its put to Live and there will still be issues uncovered. Simply because there are not enough testers testing on the test server.

I redirect the question back to you. How can a player make judgements to testing and release of updates, if that player feels absolutely no responsibility to login to the test server and find these issues before they hit live? SOE doesn't have slave elves doing intricate testing for everyone else. Mostly player like us.

How can something bugged hit the live server? Maybe one can answer, that because YOU did not find it personally before it was put to live. Why do they not test the updates to patches before being pushed? Because after the live patch, YOU are now the testers, your server has become test server, and you end up testing the little tweaks put in because there wasn't enough people to do it before.

Oh, to answer your question why that update is trivial for me; since update 13, I cannot take on any high yellow, oranges, or reds. They could drop Fabergé Eggs or give 5 lvls of XP. I would never know cuz they hand me my [expletive ninja'd by Faarbot] in a few seconds.

Laute · 09-29-2005, 01:21 PM

Actually, there are quite a few dedicated testers, and a majority of testers do find the bugs. It's just that a lot of times, nothing's done about them. Back when LU13a first came out, someone made a thread about shirtless barbarians, as well as /bugged it. THey were shirtless in creation, but not in game. Lo and behold, it was largely ignored. Blackguard replied to a thread about this topic in the past couple of days, saying he'd pass this along.So yeah. Test is good, and a lot of bugs *are* caught, but many times it seems like way too many bugs are simply outright ignored. It's not really fair to blame test for the problems that exist. SOE also has an in-house testing server which gets updates before they go to test or live.Beyond this, keep in mind that we're talking not only about an MMO, but also about patching code to a server. Even if there were no issues with the code, adding a patch cold completely screw up any server, regardless for not. They're different servers, with different populations. Even if extensive testing has been done, there could be a version conflict which makes the game crash everytime someone logs on. SMILEY

And for the record, no, indium arrows aren't simply summoned or not. There's a thread going around detailing issues with them. Long story short, arrow rip is a level 58 ability. It summons Indium arrows, which according to naming conventions, should be tier 6. However, the arrows themselves are tier 5, as they have been since release. This is a bit of a problem.I do agree with the OP to an extent, but primarily in the extent that test seems largley ignored. Even patching to test doesn't matter if the only reported errors they fix are the ones that cause crashes.

EtoilePirate · 09-29-2005, 07:30 PM

AsheBailey wrote:
What I say to rebutt this is, have you ever went on the test server? One can generally do a /who all and find only 50 players most under lvl 20 playing. People generally don't wish to play on the test server. Most likely, there could be testing of a patch done for a full year before its put to Live and there will still be issues uncovered. Simply because there are not enough testers testing on the test server.

Um, sorry, but what Test server are you on? There are at least 45 different players I can think of that have one or more toons over level 45 -- many up to 52 or 53 by now -- and those are just the ones I know of from a certain board. It's true that the population's been ridiculously low for the past five days or so, but that's becahse 3/4 of the time the server's just back up from a crash and everyone hasn't logged back in yet. (Supposedly something in last night's patch was going to help fix our stability problem -- crashing several times a day is Not Fun. If you'd run a /who all yesterday between 1 and 4 EST there's a good chance you'd have gotten a /who all 6, as I once did, because of how many times the server crashed.) The extraordinarily low numbers and the crashes, however, are NOT the norm. As for new blood and the lower levels, every week a couple of players who are relatively new to the server join the guild I'm in. I presume it's the same rate for the other guilds. (And I say relatively new because I don't know of a single guild on the server that will accept brand-new players; most wait until they've leveled a little -- past 10, at least, often closer to 20 -- just to make sure the player wants to stay on the server.) That may not be an influx of hundreds of new people a week, but it does indeed add up. That's not all to say we couldn't always use more people, because we could, but please don't make up "facts" about what we have. There are people worldwide on Test and at any time of day or night you'll find them there, seeing what breaks and how.

Chog · 09-30-2005, 02:19 AM

Daishar wrote:

How about these... these seem trivial to you?

*** Combat and Spell Changes ***

- The difficulty of orange and red con creatures has been increased.

Players where mowing through Oranges like a hot knife through butter. That was not the intent of the new combat changes. The difficulty was increased to prevent this from happening. Oranges can still be taken down, just not as easily. Needed to be changed as soon as possible so players did not get used to orages being the color to hunt for exp.

- Experience gained from epic targets has been reduced.

Some guilds where using certain Epic zones that respawned Epic encounters very quickly to gain levels faster then a level 10 could imagine. This was not the intent of Epic encounters and was fixed. Needed a hotfix.

*** Epic Encounters ****

- The experienced gained from killing epic targets is now comparable to that of killing heroics.

See above.

*** Gameplay ***

- Players now mitigate much less of the damage done by encounters that con orange and red. It is recommended that low orange encounters be attempted only by very well-equipped players, and even then there is no guarantee of success. High-orange and red encounters are meant to be deadly, so it is wise to avoid fighting them.

This is how SOE made the orange+ mobs more difficult. See above.

*** Gameplay ***

- Defeating single opponents with one or two up arrows will now provide an experience bonus similar to that of defeating encounters of grouped down-arrow targets.

Because adding the bonus experience flag to an encounter requires 3 weeks of testing to make sure the flag was properly check marked...

- High yellows, oranges, and red con mobs are now slightly harder to hit with spells and melee attacks.

This one could go either way. SOE has the formulas and numbers to figure out what % they want resisted. Testing would be a good thing to ensure the coding / numbers was correct.

AsheBail · 09-30-2005, 11:31 AM

EtoilePB wrote: Um, sorry, but what Test server are you on? There are at least 45 different players I can think of that have one or more toons over level 45 -- many up to 52 or 53 by now -- and those are just the ones I know of from a certain board. It's true that the population's been ridiculously low for the past five days or so, but that's becahse 3/4 of the time the server's just back up from a crash and everyone hasn't logged back in yet. (Supposedly something in last night's patch was going to help fix our stability problem -- crashing several times a day is Not Fun. If you'd run a /who all yesterday between 1 and 4 EST there's a good chance you'd have gotten a /who all 6, as I once did, because of how many times the server crashed.) The extraordinarily low numbers and the crashes, however, are NOT the norm. As for new blood and the lower levels, every week a couple of players who are relatively new to the server join the guild I'm in. I presume it's the same rate for the other guilds. (And I say relatively new because I don't know of a single guild on the server that will accept brand-new players; most wait until they've leveled a little -- past 10, at least, often closer to 20 -- just to make sure the player wants to stay on the server.) That may not be an influx of hundreds of new people a week, but it does indeed add up. That's not all to say we couldn't always use more people, because we could, but please don't make up "facts" about what we have. There are people worldwide on Test and at any time of day or night you'll find them there, seeing what breaks and how.

I am one of the people about level 20 who only really goes on when my live server is down daily. Eveytime I do a /who all, there is about only 50 players; maybe 10 of which are around 50 and half are under 20. It does not mean that maybe its just the time of day. Also, it does seem bugs found are generally ignored when reported. Though that's a different issue. Though this somewhat off topic. Do you think the little daily tweaks should be put on test server before going to live or most of them are trivial in a need to be tested further?

Daish · 10-06-2005, 09:16 AM

kinda confused here, Chogar...

I never questioned WHY they made the changes.... the purpose of this thread was to discuss why SOE doesnt actualy USE the test server to TEST ... BEFORE they release to live...

All of those things would have been easily found and fixed BEFORE they hit live if they didnt smash in a crap load of changes in the weekend and few HOURS before they released.

No testing ='s Stupid bugs. Not using the test server means we all are paying to beta test it until they get it right.

Chog · 10-07-2005, 04:52 AM

Daishar wrote:

kinda confused here, Chogar...

I never questioned WHY they made the changes.... the purpose of this thread was to discuss why SOE doesnt actualy USE the test server to TEST ... BEFORE they release to live...

All of those things would have been easily found and fixed BEFORE they hit live if they didnt smash in a crap load of changes in the weekend and few HOURS before they released.

No testing ='s Stupid bugs. Not using the test server means we all are paying to beta test it until they get it right.

The test server is used to test code out before being released to the live servers. More bugs will be found on live then on test because of the population and different play styles. Most hot fixes will not go to test first becasue they need to be put on live as soon as possible to fix something that is not working as intended. You can argue all you want that test is not used to test. However, the update notes say differently.

Daish · 10-07-2005, 09:36 AM

umm.. ok, Chogar...

Did you read any of this thread?

So, again, the point of the thread was that they were making changes directly to the LIVE servers WITHOUT trying them out on the TEST servsers. Also, that the big combat revamp had so many changes put in the last weekend before release there was NO time to test ANY of it, with any degree of certainty.

This was, obviously, proved to be a bad idea, since they needed to do so much direct tweaking to all major components of the combat system almost daily for the next few weeks.

And, as I listed, all of those resulting 'tweaks' for the combat revamp were NOT done on the TEST server, they were done on the LIVE servers....

Again, the issue never was that there would be bugs found on the live servers... the issue was that SOE is NOT using the TEST server to even TRY to find bugs BEFORE the code is released to the paying customer.

If you honestly feel that SOE puts ALL code on the test server and spends weeks leaving it there to test, tweak, and finalize BEFORE they hit the live servers, then you apparently are not actually reading the update notes, and are just trying to believe in the happy world of SOE when all the facts stand before you.

Aszuth · 10-07-2005, 05:37 PM

Daishar -

Did you ever read the earlier parts of this thread? Or any of the other dozens of threads on this topic you've started/added your vomitous spew to?

Fact #1 - The primary method SOE uses and has stated as such in the past for catching bugs is their INTERNAL testing team, Test is there to catch large bugs in LARGE patches (like LU's) over an extended timeframe.

Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos.

You honestly are someone that will complain about this topic until the cows come home - it doesn't matter if there's one minute, one day, one week, one month or even an entire year testing something - you'll still find reason to complain about it, it's quite obvious from your posture on the topic. I'd recommend moving on to a topic you actually have a clue about in your attempts to complain needlessly, maybe you'll actually find one worth arguing for.

Daish · 10-08-2005, 01:55 AM

sigh.. you again...

apparently, you missed the fact that the REASON they had to do these tweaks directly to LIVE is that they NEVER tested them on the TEST SERVER.

My point, that you convienently miss, is that the reason these changes are happening on live and that the PAYING customers get the 'joy' of testing, is BECAUSE SOE decides not to try the changes out on test.

The combat revamp was 4 some months in planning, design, implementation.... 4 MONTHS, but, they are still making tweaks to it directly to the LIVE servers. How long have they been working on DoF? The lvl 50+ content is just now being put into the game...

If you honestly believe that SOE is giving an honest and appropriate amount of time testing all necessary code changes, trying hard to assure the bulk of the testing does NOT fall on the paying customers, and that they have been honest with us about the state of the game, the major changes, and the expansion content.... then, why are we having these problems? Why are they still tweaking the combat revamp, con system, and still have a bulk of the 50+ content to be put in?

Let me guess, God just doesnt want SOE to make good software, right? Or, is it space aliens?

Part of the original dev/design team was canned after the 'failure' of the intial release, the team responsible for the need for the combat revamp among other things, and now, imho, SOE will probably go through other round of restaffing due to the problems associated with the combat revamp and DoF.

You want to lay 'blame' about the whole situation on me, because I continue to have a problem with how SOE operates as a software company, fine. But, maybe you should take you head out of the SOE fishbowl, and maybe read a few other threads that show many other paying customers agree with me.

In the end, the truth of the situation is staring you in the face, but you continue to believe that this is the BEST that any software gaming company can do. That there is NO POSSIBILITY that things could be done better, when there are many other games out there that are built with a much higher level of quality and customer respect. Go ask an SWG player why most of the players left that game.... it will look strangely similar to what is being done to EQ2.

Gwide · 10-08-2005, 02:07 AM

Aszuthan wrote:

Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos.

I work in the brokerage industry on software, and I don't know who your bank or brokerage is, but I would consider changing. We are quite cautious on implementing things that could cost people 100s of thousands of dollars. We do make changes to test and user acceptace regions before changing them to on live. When changes are wide spread we engage the clients to perform script testing of functions to test day to day business.

Aszuth · 10-09-2005, 12:08 AM

Gwideon wrote:

Aszuthan wrote:

Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos.

I work in the brokerage industry on software, and I don't know who your bank or brokerage is, but I would consider changing. We are quite cautious on implementing things that could cost people 100s of thousands of dollars. We do make changes to test and user acceptace regions before changing them to on live. When changes are wide spread we engage the clients to perform script testing of functions to test day to day business.

Really? You take and test your software on an entire secondary set of mainframes that is redundant, only there for the purpose of testing over an extended timeframe?

If you were being honest, what you'd likely answer now is "No, we test on a similar, but smaller scale system with a much smaller load over a short period of time compared to the uptime we expect from the primary system the update will be pushed to" - which equates out to a rather insignificant amount of testing compared to the value of the information that could get screwed up in a failure - by comparison the amount of testing effort on practically identical software in comparable load situations is much more significant, for data that is much less valuable.

The amount of testing any firm I'm aware of is rather insignificant overall, in fact if a firm is running an entire secondary redundant system just for testing (not backups/etc) I'd really doubt they're spending their cut of the money they take from me wisely enough to trust their judgment on assets. Capeche?

Aszuth · 10-09-2005, 12:13 AM

Daishar wrote:

sigh.. you again...

apparently, you missed the fact that the REASON they had to do these tweaks directly to LIVE is that they NEVER tested them on the TEST SERVER.

My point, that you convienently miss, is that the reason these changes are happening on live and that the PAYING customers get the 'joy' of testing, is BECAUSE SOE decides not to try the changes out on test.

The combat revamp was 4 some months in planning, design, implementation.... 4 MONTHS, but, they are still making tweaks to it directly to the LIVE servers. How long have they been working on DoF? The lvl 50+ content is just now being put into the game...

If you honestly believe that SOE is giving an honest and appropriate amount of time testing all necessary code changes, trying hard to assure the bulk of the testing does NOT fall on the paying customers, and that they have been honest with us about the state of the game, the major changes, and the expansion content.... then, why are we having these problems? Why are they still tweaking the combat revamp, con system, and still have a bulk of the 50+ content to be put in?

Let me guess, God just doesnt want SOE to make good software, right? Or, is it space aliens?

Part of the original dev/design team was canned after the 'failure' of the intial release, the team responsible for the need for the combat revamp among other things, and now, imho, SOE will probably go through other round of restaffing due to the problems associated with the combat revamp and DoF.

You want to lay 'blame' about the whole situation on me, because I continue to have a problem with how SOE operates as a software company, fine. But, maybe you should take you head out of the SOE fishbowl, and maybe read a few other threads that show many other paying customers agree with me.

In the end, the truth of the situation is staring you in the face, but you continue to believe that this is the BEST that any software gaming company can do. That there is NO POSSIBILITY that things could be done better, when there are many other games out there that are built with a much higher level of quality and customer respect. Go ask an SWG player why most of the players left that game.... it will look strangely similar to what is being done to EQ2.

You're a complete [expletive haxx0red by Raijinn], you do understand that right?

Why should there be things tested for an extended timeframe that are OBVIOUS in their need to be fixed ASAP? Is sending up a warning flag saying "Hey, we're fixing this exploit!" on Test so that people can exploit the hell out of it in the meantime a good thing? (Any non-idiot would say that's a pretty foolish move)

Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works!

Until you understand and ACCEPT the purpose of Test, which is PRIMARILY TO TEST THE LIVE UPDATES (AKA large patches) you have nothing worthwhile to add to a debate on this topic.

Small lists of changes can be checked by a small team in a timely fashion and have less chance of unforeseen interactions - Large lists of changes on the other hand need a larger environment to test appropriately by an exponential degree of time and manpower.

Gwide · 10-10-2005, 03:53 AM

Aszuthan wrote:

Really? You take and test your software on an entire secondary set of mainframes that is redundant, only there for the purpose of testing over an extended timeframe?

If you were being honest, what you'd likely answer now is "No, we test on a similar, but smaller scale system with a much smaller load over a short period of time compared to the uptime we expect from the primary system the update will be pushed to" - which equates out to a rather insignificant amount of testing compared to the value of the information that could get screwed up in a failure - by comparison the amount of testing effort on practically identical software in comparable load situations is much more significant, for data that is much less valuable.

The amount of testing any firm I'm aware of is rather insignificant overall, in fact if a firm is running an entire secondary redundant system just for testing (not backups/etc) I'd really doubt they're spending their cut of the money they take from me wisely enough to trust their judgment on assets. Capeche?

We have a QA region that turns a batch every night in which we have "scrubed" data from production (actual accounts) to remove confidential information. We have multiple firm specific regions which also turn a batch nightly in which contain a subsets of accounts that the users have established and use to test not only our changes but thier own. The subsets of accounts are not a handfull, in many cases they are 10s of thousands of accounts. And, we have also recently run scripted test in which we have run for specific dates with the entire account base for the client, not once but twice with the same days so that they could see the effects of a change. In fact, the sum of our testing batches' CPU and data set usage, dwarfs the actual production usage of some of our smaller clients.

The sum of our clients spend millions (yes millions) of dollars anually on redundancy, testing, disaster recovery, and power back up to avoid errors and downtime that could cost them millions due to errors and down time. It is not insignificant. Errors can cost customer confidence and in some cases real money. I won't say an error never gets through to production, but they are few comparitively. Thier name is on the line as is ours.

Now recently, you said you doubt they run every account through every batch cycle twice every day, which we don't for the entire system. There are however many checks and balances which are put into the system which do run for every account, every day. For some subsystems it actually not only recalculates the current day, but every day in the current month's cycle.

In any case, it is a far shot from your original statement.

Daish · 10-11-2005, 02:19 AM

Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works!

umm... again, you are not reading what I am typing...

they are releasing BUGS into the LIVE servers with almost EVERY update... especially updates that never hit the TEST server....

If SOE NEVER released high priority, exploit capable, game breaking bugs directly to the LIVE servers... then they wouldnt need to do high priority, exploit fixing, game un-breaking changes directly to the live servers.

God did not curse SOE when they put code on the live servers, the CAUSE of the problems was a lack of testing, the CAUSE of the issue was a lack of attention to detail, the CAUSE of the need for a fix was due to a lack of due diligence by SOE to at least do even a small amount of checking to make sure the changes they put directly into the live server code does not negatively affect the consumer or the game.

The fact that you can argue that SOE is doing a wonderful, capable job in releasing code, and that they have no bugs, no exploits, and no game breaking issues while they are doing it... the fact you actually feel like arguing that what SOE is doing is 'good', and that they really only need to test the 'large' patches (which ALSO do NOT ENTIRELY GET PUT ON TEST).. just says more and more about you and has less and less about SOE.

Aszuth · 10-11-2005, 05:14 PM

Daishar wrote:

Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works!

umm... again, you are not reading what I am typing...

they are releasing BUGS into the LIVE servers with almost EVERY update... especially updates that never hit the TEST server....

If SOE NEVER released high priority, exploit capable, game breaking bugs directly to the LIVE servers... then they wouldnt need to do high priority, exploit fixing, game un-breaking changes directly to the live servers.

God did not curse SOE when they put code on the live servers, the CAUSE of the problems was a lack of testing, the CAUSE of the issue was a lack of attention to detail, the CAUSE of the need for a fix was due to a lack of due diligence by SOE to at least do even a small amount of checking to make sure the changes they put directly into the live server code does not negatively affect the consumer or the game.

The fact that you can argue that SOE is doing a wonderful, capable job in releasing code, and that they have no bugs, no exploits, and no game breaking issues while they are doing it... the fact you actually feel like arguing that what SOE is doing is 'good', and that they really only need to test the 'large' patches (which ALSO do NOT ENTIRELY GET PUT ON TEST).. just says more and more about you and has less and less about SOE.

Since you pretend to be a guru on MMO's - name ONE that has met your criteria of never releasing bugs to the public in most if not all of their patches.

Good is relative - MMO's deal with a phenomenal amount of code and server load to run every minute of every day (or nearly) - when a server is running millions of points of data every "tick" of uptime, you're going to have a few parts fall through the cracks due to the mammoth size of it.

I could draw analogies to a million different topics about how a larger code base compares similarly to a larger orphanage, or whatever else, but I really doubt you'd care about the reality of what you "expect" - you just want to moan and complain every word you type here.

09-29-2005, 12:19 AM	#3
Sv_Lor Join Date: Dec 2004 Posts: 13	Talking about these, which do show up as updates; * Desert of Flames * - Those who have completed the three "Thief in the House of..." quests can now progress in "Welcome to the Peacock Club." * The Splitpaw Saga * - The ghosts in Anvilpaw's Grotto should no longer flee out of range. * Gameplay * - Adventure experience rewards have been increased for groups of 3 to 6 players. Solo/duo experience is unaffected. * Spells and Combat Arts * Ranger changes: - Arrow Rip will now summon Indium Arrows. Warrior changes: - Hold the Line can now be canceled via the Maintained window. Ryan "Blackguard" Shwayder __________________ Lorynx Ranger of Vox Lordz Exile to be Summoner of Vox

09-29-2005, 02:38 AM	#4
EtoilePirate Tester Join Date: Feb 2005 Location: Washington, DC Posts: 1,258	The little hotfixes -- pataches with just a couple of items in them, a mob tweaked, or an item readjusted, or a quest-giver fixed, that sort of thing -- have always gone live nearly daily, without stopping on Test first (or hitting Test at the same time). It's been like that since before I started playing, and that was back in February. 14A is still Test-only, and since this is Wednesday I wouldn't be surprised if 14B is arriving in a matter of hours. (Either that or Friday; those are the post popular Test-patching days.) Usually no-one notices the hotfix patches going straight to Live unless they bring big bugs. __________________ Kella The Mighty Pirate, Assassin & Tailor Nimari, Fury-at-Large Test Server

09-29-2005, 07:46 AM	#5
AsheBail Join Date: Nov 2004 Posts: 61	Sv_Lorax wrote: Talking about these, which do show up as updates; * Desert of Flames * - Those who have completed the three "Thief in the House of..." quests can now progress in "Welcome to the Peacock Club." * The Splitpaw Saga * - The ghosts in Anvilpaw's Grotto should no longer flee out of range. * Gameplay * - Adventure experience rewards have been increased for groups of 3 to 6 players. Solo/duo experience is unaffected. * Spells and Combat Arts * Ranger changes: - Arrow Rip will now summon Indium Arrows. Warrior changes: - Hold the Line can now be canceled via the Maintained window. Ryan "Blackguard" Shwayder I apologize for being a little annoyed but can you see how testing these for a length of time may be a little silly? Did you read them? Take 'Arrow Rip' for an example, it will either summon 'Indium Arrows' or not. Both of which isn't a biggie. Should we wait for Patch #14 for these to be put in game? __________________ --------------------------------------- &nbsp&nbsp&nbspSenu Spellfury Dark Elf Wizard &nbsp&nbsp&nbspEryn Fistfury&nbsp Wood Elf Buiser &nbsp&nbsp&nbspRook Tuneshot&nbsp Half Elf Troubador &nbsp&nbsp&nbspAiela Dominae&nbsp High Elf Warden ---------------------------------------

09-29-2005, 08:45 AM	#6
Daish Join Date: Nov 2004 Posts: 72	How about these... these seem trivial to you? * Combat and Spell Changes * - The difficulty of orange and red con creatures has been increased. - Experience gained from epic targets has been reduced. * Epic Encounters - The experienced gained from killing epic targets is now comparable to that of killing heroics. * Gameplay * - Players now mitigate much less of the damage done by encounters that con orange and red. It is recommended that low orange encounters be attempted only by very well-equipped players, and even then there is no guarantee of success. High-orange and red encounters are meant to be deadly, so it is wise to avoid fighting them. * Gameplay *** - Defeating single opponents with one or two up arrows will now provide an experience bonus similar to that of defeating encounters of grouped down-arrow targets. - High yellows, oranges, and red con mobs are now slightly harder to hit with spells and melee attacks. All of that was done directly to the LIVE SERVERS with never a minute seen on any test server. This was after 4 MONTHS of work on the combat revamp. These have all happened in the weeks since the combat revamp went live. I think the OP's point was the fact that whats the point of having a test server if you dont use it? Every single change presents an opportunity for a bug, a crashing issue, or some major change in gameplay. If there are changes that do not warrent time on the test server, then that specifically means that they believe that the paying customers will 'enjoy' testing it for them. The 6 or so different changes to harclaves is another example of why one should test changes before you put them on live. Planning to finish tweaking the combat revamp directly on the LIVE SERVERS should be unacceptable.

09-29-2005, 11:57 AM	#7
AsheBail Join Date: Nov 2004 Posts: 61	You are a little bit off. Its a wee bit more subtle. Those were changes made to update #13 because they didn't test patch #13 enough before its release. They looked at the numbers after the big patch and decided to adjust some things after the live servers actually did the further testing of #13. But now I manage to support what you said because these fixes were implemented too early due to not enough testing of Patch #13 before release. What I say to rebutt this is, have you ever went on the test server? One can generally do a /who all and find only 50 players most under lvl 20 playing. People generally don't wish to play on the test server. Most likely, there could be testing of a patch done for a full year before its put to Live and there will still be issues uncovered. Simply because there are not enough testers testing on the test server. I redirect the question back to you. How can a player make judgements to testing and release of updates, if that player feels absolutely no responsibility to login to the test server and find these issues before they hit live? SOE doesn't have slave elves doing intricate testing for everyone else. Mostly player like us. How can something bugged hit the live server? Maybe one can answer, that because YOU did not find it personally before it was put to live. Why do they not test the updates to patches before being pushed? Because after the live patch, YOU are now the testers, your server has become test server, and you end up testing the little tweaks put in because there wasn't enough people to do it before. Oh, to answer your question why that update is trivial for me; since update 13, I cannot take on any high yellow, oranges, or reds. They could drop Fabergé Eggs or give 5 lvls of XP. I would never know cuz they hand me my [expletive ninja'd by Faarbot] in a few seconds. __________________ --------------------------------------- &nbsp&nbsp&nbspSenu Spellfury Dark Elf Wizard &nbsp&nbsp&nbspEryn Fistfury&nbsp Wood Elf Buiser &nbsp&nbsp&nbspRook Tuneshot&nbsp Half Elf Troubador &nbsp&nbsp&nbspAiela Dominae&nbsp High Elf Warden ---------------------------------------

09-28-2005, 11:58 PM	#1
Sv_Lor Join Date: Dec 2004 Posts: 13	I don't know if Devs read this or not, but I have to at least say it; why in the world is live getting updates before test? Kinda defeats the purpose of having a test server, am I wrong? __________________ Lorynx Ranger of Vox Lordz Exile to be Summoner of Vox

09-29-2005, 12:09 AM	#2
AsheBail Join Date: Nov 2004 Posts: 61	Are you starting fires? Patch #14a hasn't hit live yet. Or are you referring to the little touchups we get after the servers come down daily? I suposse you wish a week of testing of those before they go live. So if we are lucky we may get these little small touch ups a few weeks later. Thanks lots for your post. __________________ --------------------------------------- &nbsp&nbsp&nbspSenu Spellfury Dark Elf Wizard &nbsp&nbsp&nbspEryn Fistfury&nbsp Wood Elf Buiser &nbsp&nbsp&nbspRook Tuneshot&nbsp Half Elf Troubador &nbsp&nbsp&nbspAiela Dominae&nbsp High Elf Warden ---------------------------------------

09-29-2005, 01:21 PM	#8
Laute Loremaster Join Date: Mar 2005 Posts: 8	Actually, there are quite a few dedicated testers, and a majority of testers do find the bugs. It's just that a lot of times, nothing's done about them. Back when LU13a first came out, someone made a thread about shirtless barbarians, as well as /bugged it. THey were shirtless in creation, but not in game. Lo and behold, it was largely ignored. Blackguard replied to a thread about this topic in the past couple of days, saying he'd pass this along.So yeah. Test is good, and a lot of bugs are caught, but many times it seems like way too many bugs are simply outright ignored. It's not really fair to blame test for the problems that exist. SOE also has an in-house testing server which gets updates before they go to test or live.Beyond this, keep in mind that we're talking not only about an MMO, but also about patching code to a server. Even if there were no issues with the code, adding a patch cold completely screw up any server, regardless for not. They're different servers, with different populations. Even if extensive testing has been done, there could be a version conflict which makes the game crash everytime someone logs on. And for the record, no, indium arrows aren't simply summoned or not. There's a thread going around detailing issues with them. Long story short, arrow rip is a level 58 ability. It summons Indium arrows, which according to naming conventions, should be tier 6. However, the arrows themselves are tier 5, as they have been since release. This is a bit of a problem.I do agree with the OP to an extent, but primarily in the extent that test seems largley ignored. Even patching to test doesn't matter if the only reported errors they fix are the ones that cause crashes.

09-29-2005, 07:30 PM	#9
EtoilePirate Tester Join Date: Feb 2005 Location: Washington, DC Posts: 1,258	AsheBailey wrote: What I say to rebutt this is, have you ever went on the test server? One can generally do a /who all and find only 50 players most under lvl 20 playing. People generally don't wish to play on the test server. Most likely, there could be testing of a patch done for a full year before its put to Live and there will still be issues uncovered. Simply because there are not enough testers testing on the test server. Um, sorry, but what Test server are you on? There are at least 45 different players I can think of that have one or more toons over level 45 -- many up to 52 or 53 by now -- and those are just the ones I know of from a certain board. It's true that the population's been ridiculously low for the past five days or so, but that's becahse 3/4 of the time the server's just back up from a crash and everyone hasn't logged back in yet. (Supposedly something in last night's patch was going to help fix our stability problem -- crashing several times a day is Not Fun. If you'd run a /who all yesterday between 1 and 4 EST there's a good chance you'd have gotten a /who all 6, as I once did, because of how many times the server crashed.) The extraordinarily low numbers and the crashes, however, are NOT the norm. As for new blood and the lower levels, every week a couple of players who are relatively new to the server join the guild I'm in. I presume it's the same rate for the other guilds. (And I say relatively new because I don't know of a single guild on the server that will accept brand-new players; most wait until they've leveled a little -- past 10, at least, often closer to 20 -- just to make sure the player wants to stay on the server.) That may not be an influx of hundreds of new people a week, but it does indeed add up. That's not all to say we couldn't always use more people, because we could, but please don't make up "facts" about what we have. There are people worldwide on Test and at any time of day or night you'll find them there, seeing what breaks and how. __________________ Kella The Mighty Pirate, Assassin & Tailor Nimari, Fury-at-Large Test Server

09-30-2005, 02:19 AM	#10
Chog General Join Date: Nov 2004 Posts: 475	Daishar wrote: How about these... these seem trivial to you? * Combat and Spell Changes * - The difficulty of orange and red con creatures has been increased. Players where mowing through Oranges like a hot knife through butter. That was not the intent of the new combat changes. The difficulty was increased to prevent this from happening. Oranges can still be taken down, just not as easily. Needed to be changed as soon as possible so players did not get used to orages being the color to hunt for exp. - Experience gained from epic targets has been reduced. Some guilds where using certain Epic zones that respawned Epic encounters very quickly to gain levels faster then a level 10 could imagine. This was not the intent of Epic encounters and was fixed. Needed a hotfix. * Epic Encounters - The experienced gained from killing epic targets is now comparable to that of killing heroics. See above. * Gameplay * - Players now mitigate much less of the damage done by encounters that con orange and red. It is recommended that low orange encounters be attempted only by very well-equipped players, and even then there is no guarantee of success. High-orange and red encounters are meant to be deadly, so it is wise to avoid fighting them. This is how SOE made the orange+ mobs more difficult. See above. * Gameplay *** - Defeating single opponents with one or two up arrows will now provide an experience bonus similar to that of defeating encounters of grouped down-arrow targets. Because adding the bonus experience flag to an encounter requires 3 weeks of testing to make sure the flag was properly check marked... - High yellows, oranges, and red con mobs are now slightly harder to hit with spells and melee attacks. This one could go either way. SOE has the formulas and numbers to figure out what % they want resisted. Testing would be a good thing to ensure the coding / numbers was correct. __________________ ----------------------------------------------------------------------- "Your food stamps will be stopped effective March 1992 because we received notice that you passed away. May God bless you. You may reapply if there is a change in your circumstances."--Department of Social Services, Greenville, South Carolina

09-30-2005, 11:31 AM	#11
AsheBail Join Date: Nov 2004 Posts: 61	EtoilePB wrote: Um, sorry, but what Test server are you on? There are at least 45 different players I can think of that have one or more toons over level 45 -- many up to 52 or 53 by now -- and those are just the ones I know of from a certain board. It's true that the population's been ridiculously low for the past five days or so, but that's becahse 3/4 of the time the server's just back up from a crash and everyone hasn't logged back in yet. (Supposedly something in last night's patch was going to help fix our stability problem -- crashing several times a day is Not Fun. If you'd run a /who all yesterday between 1 and 4 EST there's a good chance you'd have gotten a /who all 6, as I once did, because of how many times the server crashed.) The extraordinarily low numbers and the crashes, however, are NOT the norm. As for new blood and the lower levels, every week a couple of players who are relatively new to the server join the guild I'm in. I presume it's the same rate for the other guilds. (And I say relatively new because I don't know of a single guild on the server that will accept brand-new players; most wait until they've leveled a little -- past 10, at least, often closer to 20 -- just to make sure the player wants to stay on the server.) That may not be an influx of hundreds of new people a week, but it does indeed add up. That's not all to say we couldn't always use more people, because we could, but please don't make up "facts" about what we have. There are people worldwide on Test and at any time of day or night you'll find them there, seeing what breaks and how. I am one of the people about level 20 who only really goes on when my live server is down daily. Eveytime I do a /who all, there is about only 50 players; maybe 10 of which are around 50 and half are under 20. It does not mean that maybe its just the time of day. Also, it does seem bugs found are generally ignored when reported. Though that's a different issue. Though this somewhat off topic. Do you think the little daily tweaks should be put on test server before going to live or most of them are trivial in a need to be tested further? __________________ --------------------------------------- &nbsp&nbsp&nbspSenu Spellfury Dark Elf Wizard &nbsp&nbsp&nbspEryn Fistfury&nbsp Wood Elf Buiser &nbsp&nbsp&nbspRook Tuneshot&nbsp Half Elf Troubador &nbsp&nbsp&nbspAiela Dominae&nbsp High Elf Warden ---------------------------------------

10-06-2005, 09:16 AM	#12
Daish Join Date: Nov 2004 Posts: 72	kinda confused here, Chogar... I never questioned WHY they made the changes.... the purpose of this thread was to discuss why SOE doesnt actualy USE the test server to TEST ... BEFORE they release to live... All of those things would have been easily found and fixed BEFORE they hit live if they didnt smash in a crap load of changes in the weekend and few HOURS before they released. No testing ='s Stupid bugs. Not using the test server means we all are paying to beta test it until they get it right.

10-07-2005, 04:52 AM	#13
Chog General Join Date: Nov 2004 Posts: 475	Daishar wrote: kinda confused here, Chogar... I never questioned WHY they made the changes.... the purpose of this thread was to discuss why SOE doesnt actualy USE the test server to TEST ... BEFORE they release to live... All of those things would have been easily found and fixed BEFORE they hit live if they didnt smash in a crap load of changes in the weekend and few HOURS before they released. No testing ='s Stupid bugs. Not using the test server means we all are paying to beta test it until they get it right. The test server is used to test code out before being released to the live servers. More bugs will be found on live then on test because of the population and different play styles. Most hot fixes will not go to test first becasue they need to be put on live as soon as possible to fix something that is not working as intended. You can argue all you want that test is not used to test. However, the update notes say differently. __________________ ----------------------------------------------------------------------- "Your food stamps will be stopped effective March 1992 because we received notice that you passed away. May God bless you. You may reapply if there is a change in your circumstances."--Department of Social Services, Greenville, South Carolina

10-07-2005, 09:36 AM	#14
Daish Join Date: Nov 2004 Posts: 72	umm.. ok, Chogar... Did you read any of this thread? So, again, the point of the thread was that they were making changes directly to the LIVE servers WITHOUT trying them out on the TEST servsers. Also, that the big combat revamp had so many changes put in the last weekend before release there was NO time to test ANY of it, with any degree of certainty. This was, obviously, proved to be a bad idea, since they needed to do so much direct tweaking to all major components of the combat system almost daily for the next few weeks. And, as I listed, all of those resulting 'tweaks' for the combat revamp were NOT done on the TEST server, they were done on the LIVE servers.... Again, the issue never was that there would be bugs found on the live servers... the issue was that SOE is NOT using the TEST server to even TRY to find bugs BEFORE the code is released to the paying customer. If you honestly feel that SOE puts ALL code on the test server and spends weeks leaving it there to test, tweak, and finalize BEFORE they hit the live servers, then you apparently are not actually reading the update notes, and are just trying to believe in the happy world of SOE when all the facts stand before you.

10-07-2005, 05:37 PM	#15
Aszuth General Join Date: Nov 2004 Posts: 99	Daishar - Did you ever read the earlier parts of this thread? Or any of the other dozens of threads on this topic you've started/added your vomitous spew to? Fact #1 - The primary method SOE uses and has stated as such in the past for catching bugs is their INTERNAL testing team, Test is there to catch large bugs in LARGE patches (like LU's) over an extended timeframe. Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos. You honestly are someone that will complain about this topic until the cows come home - it doesn't matter if there's one minute, one day, one week, one month or even an entire year testing something - you'll still find reason to complain about it, it's quite obvious from your posture on the topic. I'd recommend moving on to a topic you actually have a clue about in your attempts to complain needlessly, maybe you'll actually find one worth arguing for.

10-08-2005, 01:55 AM	#16
Daish Join Date: Nov 2004 Posts: 72	sigh.. you again... apparently, you missed the fact that the REASON they had to do these tweaks directly to LIVE is that they NEVER tested them on the TEST SERVER. My point, that you convienently miss, is that the reason these changes are happening on live and that the PAYING customers get the 'joy' of testing, is BECAUSE SOE decides not to try the changes out on test. The combat revamp was 4 some months in planning, design, implementation.... 4 MONTHS, but, they are still making tweaks to it directly to the LIVE servers. How long have they been working on DoF? The lvl 50+ content is just now being put into the game... If you honestly believe that SOE is giving an honest and appropriate amount of time testing all necessary code changes, trying hard to assure the bulk of the testing does NOT fall on the paying customers, and that they have been honest with us about the state of the game, the major changes, and the expansion content.... then, why are we having these problems? Why are they still tweaking the combat revamp, con system, and still have a bulk of the 50+ content to be put in? Let me guess, God just doesnt want SOE to make good software, right? Or, is it space aliens? Part of the original dev/design team was canned after the 'failure' of the intial release, the team responsible for the need for the combat revamp among other things, and now, imho, SOE will probably go through other round of restaffing due to the problems associated with the combat revamp and DoF. You want to lay 'blame' about the whole situation on me, because I continue to have a problem with how SOE operates as a software company, fine. But, maybe you should take you head out of the SOE fishbowl, and maybe read a few other threads that show many other paying customers agree with me. In the end, the truth of the situation is staring you in the face, but you continue to believe that this is the BEST that any software gaming company can do. That there is NO POSSIBILITY that things could be done better, when there are many other games out there that are built with a much higher level of quality and customer respect. Go ask an SWG player why most of the players left that game.... it will look strangely similar to what is being done to EQ2.

10-08-2005, 02:07 AM	#17
Gwide Join Date: Mar 2005 Posts: 86	Aszuthan wrote: Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos. I work in the brokerage industry on software, and I don't know who your bank or brokerage is, but I would consider changing. We are quite cautious on implementing things that could cost people 100s of thousands of dollars. We do make changes to test and user acceptace regions before changing them to on live. When changes are wide spread we engage the clients to perform script testing of functions to test day to day business.

10-09-2005, 12:08 AM	#18
Aszuth General Join Date: Nov 2004 Posts: 99	Gwideon wrote: Aszuthan wrote: Fact #2 - The patch you decided to quote was changing entries in a spreadsheet from top to bottom, if major banks and brokerage firms can update their version of the software to handle the spreadsheets and such that handle billions of dollars without significant testing, I think there's not too much to worry about a spreadsheet change tossing the system into chaos. I work in the brokerage industry on software, and I don't know who your bank or brokerage is, but I would consider changing. We are quite cautious on implementing things that could cost people 100s of thousands of dollars. We do make changes to test and user acceptace regions before changing them to on live. When changes are wide spread we engage the clients to perform script testing of functions to test day to day business. Really? You take and test your software on an entire secondary set of mainframes that is redundant, only there for the purpose of testing over an extended timeframe? If you were being honest, what you'd likely answer now is "No, we test on a similar, but smaller scale system with a much smaller load over a short period of time compared to the uptime we expect from the primary system the update will be pushed to" - which equates out to a rather insignificant amount of testing compared to the value of the information that could get screwed up in a failure - by comparison the amount of testing effort on practically identical software in comparable load situations is much more significant, for data that is much less valuable. The amount of testing any firm I'm aware of is rather insignificant overall, in fact if a firm is running an entire secondary redundant system just for testing (not backups/etc) I'd really doubt they're spending their cut of the money they take from me wisely enough to trust their judgment on assets. Capeche?

10-09-2005, 12:13 AM	#19
Aszuth General Join Date: Nov 2004 Posts: 99	Daishar wrote: sigh.. you again... apparently, you missed the fact that the REASON they had to do these tweaks directly to LIVE is that they NEVER tested them on the TEST SERVER. My point, that you convienently miss, is that the reason these changes are happening on live and that the PAYING customers get the 'joy' of testing, is BECAUSE SOE decides not to try the changes out on test. The combat revamp was 4 some months in planning, design, implementation.... 4 MONTHS, but, they are still making tweaks to it directly to the LIVE servers. How long have they been working on DoF? The lvl 50+ content is just now being put into the game... If you honestly believe that SOE is giving an honest and appropriate amount of time testing all necessary code changes, trying hard to assure the bulk of the testing does NOT fall on the paying customers, and that they have been honest with us about the state of the game, the major changes, and the expansion content.... then, why are we having these problems? Why are they still tweaking the combat revamp, con system, and still have a bulk of the 50+ content to be put in? Let me guess, God just doesnt want SOE to make good software, right? Or, is it space aliens? Part of the original dev/design team was canned after the 'failure' of the intial release, the team responsible for the need for the combat revamp among other things, and now, imho, SOE will probably go through other round of restaffing due to the problems associated with the combat revamp and DoF. You want to lay 'blame' about the whole situation on me, because I continue to have a problem with how SOE operates as a software company, fine. But, maybe you should take you head out of the SOE fishbowl, and maybe read a few other threads that show many other paying customers agree with me. In the end, the truth of the situation is staring you in the face, but you continue to believe that this is the BEST that any software gaming company can do. That there is NO POSSIBILITY that things could be done better, when there are many other games out there that are built with a much higher level of quality and customer respect. Go ask an SWG player why most of the players left that game.... it will look strangely similar to what is being done to EQ2. You're a complete [expletive haxx0red by Raijinn], you do understand that right? Why should there be things tested for an extended timeframe that are OBVIOUS in their need to be fixed ASAP? Is sending up a warning flag saying "Hey, we're fixing this exploit!" on Test so that people can exploit the hell out of it in the meantime a good thing? (Any non-idiot would say that's a pretty foolish move) Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works! Until you understand and ACCEPT the purpose of Test, which is PRIMARILY TO TEST THE LIVE UPDATES (AKA large patches) you have nothing worthwhile to add to a debate on this topic. Small lists of changes can be checked by a small team in a timely fashion and have less chance of unforeseen interactions - Large lists of changes on the other hand need a larger environment to test appropriately by an exponential degree of time and manpower.

10-10-2005, 03:53 AM	#20
Gwide Join Date: Mar 2005 Posts: 86	Aszuthan wrote: Really? You take and test your software on an entire secondary set of mainframes that is redundant, only there for the purpose of testing over an extended timeframe? If you were being honest, what you'd likely answer now is "No, we test on a similar, but smaller scale system with a much smaller load over a short period of time compared to the uptime we expect from the primary system the update will be pushed to" - which equates out to a rather insignificant amount of testing compared to the value of the information that could get screwed up in a failure - by comparison the amount of testing effort on practically identical software in comparable load situations is much more significant, for data that is much less valuable. The amount of testing any firm I'm aware of is rather insignificant overall, in fact if a firm is running an entire secondary redundant system just for testing (not backups/etc) I'd really doubt they're spending their cut of the money they take from me wisely enough to trust their judgment on assets. Capeche? We have a QA region that turns a batch every night in which we have "scrubed" data from production (actual accounts) to remove confidential information. We have multiple firm specific regions which also turn a batch nightly in which contain a subsets of accounts that the users have established and use to test not only our changes but thier own. The subsets of accounts are not a handfull, in many cases they are 10s of thousands of accounts. And, we have also recently run scripted test in which we have run for specific dates with the entire account base for the client, not once but twice with the same days so that they could see the effects of a change. In fact, the sum of our testing batches' CPU and data set usage, dwarfs the actual production usage of some of our smaller clients. The sum of our clients spend millions (yes millions) of dollars anually on redundancy, testing, disaster recovery, and power back up to avoid errors and downtime that could cost them millions due to errors and down time. It is not insignificant. Errors can cost customer confidence and in some cases real money. I won't say an error never gets through to production, but they are few comparitively. Thier name is on the line as is ours. Now recently, you said you doubt they run every account through every batch cycle twice every day, which we don't for the entire system. There are however many checks and balances which are put into the system which do run for every account, every day. For some subsystems it actually not only recalculates the current day, but every day in the current month's cycle. In any case, it is a far shot from your original statement.

10-11-2005, 02:19 AM	#21
Daish Join Date: Nov 2004 Posts: 72	Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works! umm... again, you are not reading what I am typing... they are releasing BUGS into the LIVE servers with almost EVERY update... especially updates that never hit the TEST server.... If SOE NEVER released high priority, exploit capable, game breaking bugs directly to the LIVE servers... then they wouldnt need to do high priority, exploit fixing, game un-breaking changes directly to the live servers. God did not curse SOE when they put code on the live servers, the CAUSE of the problems was a lack of testing, the CAUSE of the issue was a lack of attention to detail, the CAUSE of the need for a fix was due to a lack of due diligence by SOE to at least do even a small amount of checking to make sure the changes they put directly into the live server code does not negatively affect the consumer or the game. The fact that you can argue that SOE is doing a wonderful, capable job in releasing code, and that they have no bugs, no exploits, and no game breaking issues while they are doing it... the fact you actually feel like arguing that what SOE is doing is 'good', and that they really only need to test the 'large' patches (which ALSO do NOT ENTIRELY GET PUT ON TEST).. just says more and more about you and has less and less about SOE.

10-11-2005, 05:14 PM	#22
Aszuth General Join Date: Nov 2004 Posts: 99	Daishar wrote: Why should there be in-depth testing of things that are TRIVIAL changes, change a 1 to a 0 in a line of code to make an ability work instead of it failing every use - let's test it for a week to make sure it works! umm... again, you are not reading what I am typing... they are releasing BUGS into the LIVE servers with almost EVERY update... especially updates that never hit the TEST server.... If SOE NEVER released high priority, exploit capable, game breaking bugs directly to the LIVE servers... then they wouldnt need to do high priority, exploit fixing, game un-breaking changes directly to the live servers. God did not curse SOE when they put code on the live servers, the CAUSE of the problems was a lack of testing, the CAUSE of the issue was a lack of attention to detail, the CAUSE of the need for a fix was due to a lack of due diligence by SOE to at least do even a small amount of checking to make sure the changes they put directly into the live server code does not negatively affect the consumer or the game. The fact that you can argue that SOE is doing a wonderful, capable job in releasing code, and that they have no bugs, no exploits, and no game breaking issues while they are doing it... the fact you actually feel like arguing that what SOE is doing is 'good', and that they really only need to test the 'large' patches (which ALSO do NOT ENTIRELY GET PUT ON TEST).. just says more and more about you and has less and less about SOE. Since you pretend to be a guru on MMO's - name ONE that has met your criteria of never releasing bugs to the public in most if not all of their patches. Good is relative - MMO's deal with a phenomenal amount of code and server load to run every minute of every day (or nearly) - when a server is running millions of points of data every "tick" of uptime, you're going to have a few parts fall through the cracks due to the mammoth size of it. I could draw analogies to a million different topics about how a larger code base compares similarly to a larger orphanage, or whatever else, but I really doubt you'd care about the reality of what you "expect" - you just want to moan and complain every word you type here.