Runnyeye Server Stability?

[tomsky]

So what is up with the RE server atm, LD's happening at random to lots of people around europe, all on different ISP's, all in different zones, happened twice in the last hour. Its not my pc, its only been happening since LU and its happening to most of my guild and lots of the rest of the server?! wtb fix asap

[Flee65]

Can't stay connected to the server for more than 30mins and im currently in toronto Canada. Has been going on for quite some time now and about 85% of my guild can't even play due to this lag. Can we at least have some conformation that you guys at SOE even recognise this problem?

[tomsky]

And another crash, within 20 mins of last one, was at 22:40 UK time,Wtb response asap!

[TSR-DanielH]

<p>While I can't comment on server stability, and wouldn't have too much info on a server I don't play on, I would be happy to help look for any kind of connection problem on your end.  I would simply need to collect some of the connection related info requested in the sticky thread at the top of the forums.</p><p>Thanks</p>

[tomsky]

This may seem a bit rude but there is no way its a problem with my end of the connection if half the server is having the same problem. I live in UK, use a BT connection, and like EQ2, but tonight something has been disrupting connection, this may be a sony fault (dead hampster on wheel) or it may be a fault of who ever deals with your connections.Server chat showed that it is happening around the world, Canada, UK, Denmark, Greece, so its not an ISP problem, and its not a computer problem, its something wrong with SOE connection on your end <img src="/smilies/3b63d1616c5dfcf29f8a7a031aaa7cad.gif" border="0" alt="SMILEY" />

[TSR-DanielH]

<cite>tomsky wrote:</cite><blockquote>This may seem a bit rude but there is no way its a problem with my end of the connection if half the server is having the same problem. I live in UK, use a BT connection, and like EQ2, but tonight something has been disrupting connection, this may be a sony fault (dead hampster on wheel) or it may be a fault of who ever deals with your connections.Server chat showed that it is happening around the world, Canada, UK, Denmark, Greece, so its not an ISP problem, and its not a computer problem, its something wrong with SOE connection on your end <img src="/eq2/images/smilies/3b63d1616c5dfcf29f8a7a031aaa7cad.gif" border="0" alt="SMILEY<img src="/smilies/3b63d1616c5dfcf29f8a7a031aaa7cad.gif" border="0" alt="SMILEY" />" width="15" height="15" /></blockquote><p>I suppose its possible, but I haven't received word of any problems there.  I'll go ahead and send a ticket over to the platform support team to see if they have any information for me, though it may be a little while before I get a response.</p><p>Just to let you know, there was a problem with one of the networks leading into this area the other day(monday, I think).  Some of the network hardware used by the level 3 network went down and they had to switch to backups.  This was not at SOE, but at one of the providers that the ISPs use.  Its possible that there's some kind of residual issue there thats not due to either our side or your own.  </p><p>I'll see if I can run some connection tests to runnyeye to verify the problems you're reporting.  It wont be me that fixes it, but I should be able to pass the info on to the relevant people.</p>

[Krakato]

<p>I can confirm everyone in my guild has problems at peak times in our guild since the last patch. It does not occur early AM nearly as bad.</p><p>LD can happen literally every 2 mins to 10 mins.</p><p>ISP's vary BT here sky is one I know of. </p><p>Again this only started Thursday night and I hear it server wide</p>

[Oakleafe]

And I had thought it was just me!  My ISP is Virgin Media (ex-Blueyonder).  I don't have 100% confidence in this ISP (am tied to it as there's no alternative cable) so had been assuming it was a local issue for me, so am glad to have confirmation that it was not.I'll echo what Krakatous said, that it seems to be a peak time issue.  I played sporadically through Thursday and Friday during the (UK) day and only had the usual "played for a long time without rebooting" lag, UNTIL 5pm BST (GMT+1) when things got a little worse and then between 18:30-19:30 BST (GMT+1) when I crashed so many times (both days) I just gave up and logged off for the night.I had been doing the new live event relay!  So, my comparison of what was occurring should be fairly good.  I had the, now usual, lag whenever I hit a place where players were concentrated (Qeynos Exchange and the Concordium) in the evenings, where FPS would drop from 25-30 down to 4-11 but would pick up again when I've cleared the locale.  The packet loss was not restricted to any specific area (i.e. it happened in NQ, SQ and Antonica).I have the little performance window up permanently on my characters, specifically on the packet loss stats and can confirm that during the LD-moments the packet loss is showing continual loss; the values range, but was up to 30 constant at one point.During the Friday night packet loss I fired off a few pings to places like <a rel="nofollow" href="http://www.bbc.co.uk" target="_blank">www.bbc.co.uk</a> (as it's a good indicator of ISP created packet loss) and had fairly good response times even though the EQ2 packet loss was still firing in EQ2 (there was no packet loss to the sites I continually pinged).Is there anything a Techie can advise that would allow us to collect EQ2 specific information when it happens?  In preparation for what I'm assuming will be a bad play day I've set up a trace route (as explained in the sticky) and have included the runnyeye server.  Is there more we can do that is Runnyeye (and associated server(s)) specific?

[Mezza]

Yeah I had this start the day after the update and have been seeing it pretty much ever since. Well until I had to endure yet another minor download in client this morning and now the graphics are completely shot, but I'll leave that for another post which I'm gonna have to start /sigh. Oooh and on a weekend hurrah <img src="/smilies/9d71f0541cff0a302a0309c5079e8dee.gif" border="0" alt="SMILEY" /> Oh yeah and the original fault I reported in game 2 days ago and have heard zilch ever since. Thanks alot...

[Peysel]

It can't be a peak time issue, it's 08:30am now and I just logged in to be greeted with several crashes to character select.  Channel chat is full of people talking about lag spikes.  I'm on BT broadband.

[Krakato]

I too had the problem again yesterday morning early on then it just vanished. Combat was fine, but what I noticed was it was any time I tried to interact with brokers/bank/shared vault/house vault, even a door once. It was fine by around 9-10 an, before though ping just went through the roof and I DC'd to character select.

[Oakleafe]

I had the same problem occur again today.  I was crafting and hitting the reaction arts did nothing, then I noticed the mountain range of red lines in the Performance Window showing constant packet loss, peaking around 30 packets.  Eventually the game crashed and told me the Runnyeye server was not up.I opened Firefox and jumped to <a href="http://eq2players.station.sony.com/network_status.vm," target="_blank" rel="nofollow">http://eq2players.station.sony.com/...work_status.vm,</a> to confirm that I could still connect to the outside world and to check for any planned outages.  I did connect and there were no planned outages listed for today.Oh, btw, I connect via a modem (no router and I'm not wireless).I opened a cmd window and ran a pre-prepared batch file to fire off a number of trace routes (tracert) plus a pathping specifically on the runnyeye server.  The first attempt failed with the following messages:<span style="color: #ffcc00;">Unable to resolve target system name guk.everquest2.com.Unable to resolve target system name befallen.everquest2.com.Unable to resolve target system name valor.everquest2.com.Unable to resolve target system name runnyeye.everquest2.com.Unable to resolve target system name runnyeye.everquest2.com.</span>I've managed to run this batch file in the past with no issues, so I tried a "ping runnyeye.everquest2.com".  It worked, so I ran the trace route batch file again and got the following (slightly edited to remove my details):<span style="color: #ffcc00;">Unable to resolve target system name guk.everquest2.com.Unable to resolve target system name befallen.everquest2.com.Unable to resolve target system name valor.everquest2.com.Tracing route to ableq2-04-01.everquest2.com [199.108.202.129]over a maximum of 30 hops:  4     *        *       90 ms  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]   5    14 ms    12 ms    15 ms  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]   6    14 ms    12 ms    12 ms  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]   7    11 ms    19 ms    23 ms  tge8-1.fr3.lon.llnw.net [87.248.208.129]   8    84 ms    87 ms    85 ms  tge7-2.fr3.lga.llnw.net [69.28.171.125]   9   101 ms    95 ms   100 ms  tge2-4.fr3.iad.llnw.net [69.28.171.157]  10   186 ms   208 ms   203 ms  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]  11    92 ms    90 ms    92 ms  vl46.ashaens-1.sonyonline.net [64.37.144.177]  12    88 ms    89 ms    88 ms  ableq2-04-01.everquest2.com [199.108.202.129] Trace complete.Tracing route to ableq2-04-01.everquest2.com [199.108.202.129]over a maximum of 30 hops:  3  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   4  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]   5  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]   6  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]   7  tge8-1.fr3.lon.llnw.net [87.248.208.129]   8  tge7-2.fr3.lga.llnw.net [69.28.171.125]   9  tge2-4.fr3.iad.llnw.net [69.28.171.157]  10  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]  11  vl46.ashaens-1.sonyonline.net [64.37.144.177]  12  ableq2-04-01.everquest2.com [199.108.202.129] Computing statistics for 300 seconds...            Source to Here   This Node/LinkHop  RTT    Lost/Sent = Pct  Lost/Sent = Pct  Address  3   10ms     0/ 100 =  0%     0/ 100 =  0%  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]                                 0/ 100 =  0%   |  4   11ms     0/ 100 =  0%     0/ 100 =  0%  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]                                 0/ 100 =  0%   |  5   12ms     0/ 100 =  0%     0/ 100 =  0%  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]                                 0/ 100 =  0%   |  6   14ms     0/ 100 =  0%     0/ 100 =  0%  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]                                 0/ 100 =  0%   |  7   13ms     0/ 100 =  0%     0/ 100 =  0%  tge8-1.fr3.lon.llnw.net [87.248.208.129]                                 0/ 100 =  0%   |  8   83ms     0/ 100 =  0%     0/ 100 =  0%  tge7-2.fr3.lga.llnw.net [69.28.171.125]                                 0/ 100 =  0%   |  9   90ms     0/ 100 =  0%     0/ 100 =  0%  tge2-4.fr3.iad.llnw.net [69.28.171.157]                                 0/ 100 =  0%   | 10   96ms     0/ 100 =  0%     0/ 100 =  0%  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]                                 0/ 100 =  0%   | 11  ---     100/ 100 =100%   100/ 100 =100%  vl46.ashaens-1.sonyonline.net [64.37.144.177]                                 0/ 100 =  0%   | 12   90ms     0/ 100 =  0%     0/ 100 =  0%  ableq2-04-01.everquest2.com [199.108.202.129] Trace complete.</span>The fact that I got "Unable to resolve target system name" messages and also "  4     *        *       90 ms  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17] " on the trace route suggests to me that there was some sort of issue with my ISP BUT "11  ---     100/ 100 =100%   100/ 100 =100%  vl46.ashaens-1.sonyonline.net [64.37.144.177] " makes it look like there's something amiss at the SoE end as well.I pinged the other Sony servers and they were recognised so I ran the trace route batch file again (roughly 10 minutes after the above results) and got:<span style="color: #ffcc00;">Tracing route to sdkeq2-01-01.everquest2.com [199.108.12.33]over a maximum of 30 hops:  3     9 ms     9 ms    15 ms  osr02croy-v99.inet.ntl.com [195.182.179.58]   4     7 ms     9 ms     8 ms  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   5    10 ms     8 ms    10 ms  pop-bb-a-ae0-0.inet.ntl.com [213.105.174.229]   6    10 ms     9 ms     8 ms  195.50.91.69   7    11 ms     9 ms    11 ms  ae-0-54.bbr2.London1.Level3.net [4.68.116.98]   8   182 ms   156 ms   160 ms  as-0-0.mp2.SanDiego1.Level3.net [4.68.128.149]   9   157 ms   156 ms   155 ms  so-10-0.hsa1.SanDiego1.Level3.net [4.68.113.38]  10   155 ms   156 ms   155 ms  vl861.sdtermswitch-1.sonyonline.net [63.215.251.130]  11   156 ms   155 ms   156 ms  vl831.sdkollsw-1.sonyonline.net [64.37.144.82]  12   169 ms   154 ms   156 ms  sdkeq2-01-01.everquest2.com [199.108.12.33] Trace complete.Tracing route to ableq2-01-01.everquest2.com [199.108.202.33]over a maximum of 30 hops:  3     7 ms    18 ms     8 ms  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   4    12 ms     9 ms    10 ms  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]   5     9 ms    10 ms     9 ms  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]   6    12 ms    14 ms    12 ms  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]   7    27 ms    13 ms    13 ms  tge8-1.fr3.lon.llnw.net [87.248.208.129]   8    83 ms    93 ms    84 ms  tge7-2.fr3.lga.llnw.net [69.28.171.125]   9    89 ms    94 ms   101 ms  tge2-4.fr3.iad.llnw.net [69.28.171.157]  10    90 ms    87 ms    88 ms  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]  11   104 ms    89 ms    89 ms  vl46.ashaens-1.sonyonline.net [64.37.144.177]  12    89 ms    92 ms    88 ms  ableq2-01-01.everquest2.com [199.108.202.33] Trace complete.Tracing route to amseq2-01-01.everquest2.com [195.33.135.161]over a maximum of 30 hops:  3     7 ms    10 ms     7 ms  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   4     8 ms     9 ms     8 ms  pop-bb-a-ae0-0.inet.ntl.com [213.105.174.229]   5    16 ms    18 ms    15 ms  amst-ic-1-as0-0.inet.ntl.com [213.105.175.6]   6    15 ms    15 ms    16 ms  nlamtr1103er1.am.nl.attglobal.net [195.69.144.194]   7    22 ms    25 ms    23 ms  vl50.amsixmsfc-1.sonyonline.net [195.33.129.76]   8    24 ms    22 ms    23 ms  amseq2-01-01.everquest2.com [195.33.135.161] Trace complete.Tracing route to ableq2-04-01.everquest2.com [199.108.202.129]over a maximum of 30 hops:  3     7 ms     8 ms     8 ms  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   4    10 ms    12 ms    10 ms  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]   5    11 ms     9 ms    10 ms  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]   6    13 ms    13 ms    11 ms  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]   7    18 ms    12 ms    13 ms  tge8-1.fr3.lon.llnw.net [87.248.208.129]   8    85 ms    83 ms    90 ms  tge7-2.fr3.lga.llnw.net [69.28.171.125]   9    91 ms    99 ms    99 ms  tge2-4.fr3.iad.llnw.net [69.28.171.157]  10    88 ms    88 ms    91 ms  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]  11    91 ms    89 ms    89 ms  vl46.ashaens-1.sonyonline.net [64.37.144.177]  12    88 ms    89 ms    90 ms  ableq2-04-01.everquest2.com [199.108.202.129] Trace complete.Tracing route to ableq2-04-01.everquest2.com [199.108.202.129]over a maximum of 30 hops:  3  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]   4  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]   5  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]   6  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]   7  tge8-1.fr3.lon.llnw.net [87.248.208.129]   8  tge7-2.fr3.lga.llnw.net [69.28.171.125]   9  tge2-4.fr3.iad.llnw.net [69.28.171.157]  10  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]  11  vl46.ashaens-1.sonyonline.net [64.37.144.177]  12  ableq2-04-01.everquest2.com [199.108.202.129] Computing statistics for 300 seconds...            Source to Here   This Node/LinkHop  RTT    Lost/Sent = Pct  Lost/Sent = Pct  Address  3    9ms     0/ 100 =  0%     0/ 100 =  0%  pop-bb-b-ge-210-0.inet.ntl.com [195.182.178.86]                                 0/ 100 =  0%   |  4   11ms     0/ 100 =  0%     0/ 100 =  0%  nth-bb-a-so-120-0.inet.ntl.com [213.105.64.17]                                 0/ 100 =  0%   |  5   11ms     0/ 100 =  0%     0/ 100 =  0%  nth-bb-b-ae0-0.inet.ntl.com [62.253.185.118]                                 0/ 100 =  0%   |  6   13ms     0/ 100 =  0%     0/ 100 =  0%  tele-ic-1-as0-0.inet.ntl.com [62.253.184.2]                                 0/ 100 =  0%   |  7   13ms     0/ 100 =  0%     0/ 100 =  0%  tge8-1.fr3.lon.llnw.net [87.248.208.129]                                 0/ 100 =  0%   |  8   86ms     0/ 100 =  0%     0/ 100 =  0%  tge7-2.fr3.lga.llnw.net [69.28.171.125]                                 0/ 100 =  0%   |  9   91ms     0/ 100 =  0%     0/ 100 =  0%  tge2-4.fr3.iad.llnw.net [69.28.171.157]                                 0/ 100 =  0%   | 10   97ms     0/ 100 =  0%     0/ 100 =  0%  sony.ge4-12.fr3.iad.llnw.net [68.142.111.254]                                 0/ 100 =  0%   | 11  ---     100/ 100 =100%   100/ 100 =100%  vl46.ashaens-1.sonyonline.net [64.37.144.177]                                 0/ 100 =  0%   | 12   90ms     0/ 100 =  0%     0/ 100 =  0%  ableq2-04-01.everquest2.com [199.108.202.129] Trace complete.</span>So, it does <i>appear</i> that there was/is some issue with vl46.ashaens-1.sonyonline.net.  I'm still getting the same results 50 minutes after the initial outage, but have logged back into the game so it cannot really be 100% packet loss can it?  I have to admit that I don't fully understand the results of trace routes and path pings, but it doesn't look right!If I ping vl46.ashaens-1.sonyonline.net I get a couldn't find the host error (I'm guess that is because my IP's servers have no reason to know that servers name), but if I ping the ip address 64.37.144.177 I get averages in the 147-218ms range.  I left a ping -t running for a while and got an average of 148ms, but no reported packet loss.  I don't truly know if 148ms is good, bad or just to be expected; I'm guessing that it's the jump from Europe to the States.Anyway, the problem re-occurred so I thought I share the results for analysis by someone who knows what they are doing! <img src="/eq2/images/smilies/283a16da79f3aa23fe1025c96295f04f.gif" border="0" alt="SMILEY" width="15" height="15" />

[Oakleafe]

BUMP!Any chance on an update on any of the above, including the following please?<cite>TSR-DanielH wrote:</cite><blockquote><cite>tomsky wrote:</cite><blockquote>This may seem a bit rude but there is no way its a problem with my end of the connection if half the server is having the same problem. I live in UK, use a BT connection, and like EQ2, but tonight something has been disrupting connection, this may be a sony fault (dead hampster on wheel) or it may be a fault of who ever deals with your connections.Server chat showed that <b>it is happening around the world</b>, Canada, UK, Denmark, Greece, so its not an ISP problem, and its not a computer problem, its something wrong with SOE connection on your end <img src="/eq2/images/smilies/3b63d1616c5dfcf29f8a7a031aaa7cad.gif" border="0" alt="SMILEY<img mce_tsrc=" />" width="15" height="15" /></blockquote><p>I suppose its possible, but I haven't received word of any problems there.  I'll go ahead and send a ticket over to the platform support team to see if they have any information for me, though it may be a little while before I get a response.</p><p>Just to let you know, there was a problem with one of the networks leading into this area the other day(monday, I think).  Some of the network hardware used by the level 3 network went down and they had to switch to backups.  This was not at SOE, but at one of the providers that the ISPs use. <b> Its possible that there's some kind of residual issue there thats not due to either our side or your own. </b> </p><p><b>I'll see if I can run some connection tests to runnyeye to verify the problems you're reporting.  It wont be me that fixes it, but I should be able to pass the info on to the relevant people.</b></p></blockquote>Thanks.

[TSR-TrevorG]

Thanks for the information on this, I have passed it along to the proper people.In the future though, please do not "bump" posts. <img src="/smilies/3b63d1616c5dfcf29f8a7a031aaa7cad.gif" border="0" alt="SMILEY" />