Update Regarding Connection Issues from August 3rd-5th

[DelverRootnose1]

...My friend Kazarach on the Khyber server I can't join them. They are waiting for me.

[sir_smacks_alot]

I would certainly tell kaz for you if I could get on . Sadly I can't get on Khyber either

[DelverRootnose1]

Update 8/8/2013 3:31 AM - There has been a resurgence of this issue, and we highly recommend players explore the workarounds described below...

We have performed a thorough investigation of all our services, hardware, and datacenter connections since Saturday afternoon. After reviewing traceroute data from players, we have identified a broader internet issue with data routing external to our servers and networks. This morning we received word that one of our affiliated connection providers is taking steps to re-route services around certain areas of the internet that are experiencing issues. We believe this to be the root cause of the reported issues.

Their work may take some time to propagate through the internet in general. If you are able to, take advantage of the temporary workarounds to use an alternate connection method (such as VPNs, tethered cellular, or acquiring a new IP address from your ISP) - any alternate internet connection that does not happen to route through the cached network paths from your existing provider. This may allow some of you to enter the game until such time as this rerouting and propagation completes.

We appreciate the efforts by those who discovered workarounds, other reports of similar issues elsewhere on the 'net, and all of you who submitted tickets detailing your issues and locations and other requested data. (Please also keep in mind that this issue is not the same as the issue that causes some users to need to log in twice before they can connect.)

While I LOVE the game I am not all that tech savy and don't plan on going through loops to play. And I bet I am not alone so as a VIP I would suggest if you want people to keep paying for your product it would be incumbent on you to fix the problem.

This is sort of like a truck being stalled in front of your business blocking your driveway. Not your fault but telling people just to part blocks away and walk will not have the effects you probably desire.

[sir_smacks_alot]

I tried the VPN fix. This approach helped on Monday, but is not working for me this time. I suppose I should do this strange thing people are always telling me about called sleep now. Hope we get some sort of fix or workaround soon.

[Drenzilla]

So i had my hopes up and got logged in. YAY. Then i got into the game quite smoothly (on my second try). YAY. only to find that i cant enter any quests without DCing.. T_T

Try to log back in only to find i cant log back in because my fps gets stuck at 53/54 right after character selection and just keeps jumping back and fort between the 2.

It seems that after 20 or so minutes i can try logging back in without 53/54 happening. but i cant do anything but pick my nose in game anyway.

[moomooprincess]

If I have to run a tracert every day to see if ALTER.NET is in my path, well, I can skip doing that.

Somehow, the third party vendor has to dump alter.net from everyone's paths. I highly doubt Verizon will do anything about it. After all, the profits they make from the one server is minimal. More than likely negligible.

Anyone know if the kids of the execs play?

Lowell McAdam
Francis Shammo
Randall Milch
John Stratton
Daniel Meade

[sarig99]

This is almost absolutely a DDO specific issue. For most of the night I was online and playing my primary toon (on Thelanis) without problems. Then I DC'd with that toon and spent about 20 min trying to get back in. I logged into an alternate account without problems, then back into that account and that toon and was able to get in without problems. I played for about 2 more hours. Then I DC'd again. This time I was completely unable to get back in using that toon, but different toons on the same (or different) account on the same or different servers were able to get logged in and play without problems. Just this one toon was having problems.
Now several hours and another trouble ticket later, I'm completely unable to connect using my primary account. If I resort to my daughter's account on the same system, using the same router and connecting to the same server, I have no problem.

IF the issue were truly routing related, then it would affect every toon on this system going to that same server. It wouldn't be specific to a DDO account since the routing path is stored at the ISP level (per previous posts from Devs). I'm a VIP and becoming VERY frustrated with this issue. Please get this resolved today.

Thanks,
Tired and Frustrated

[PrdPolack1307]

Have had more problems, usually associated with going into or out of the Marketplace.

The issues were not with all my characters on the server, I could still play normally on a couple, but not all of my characters. However, if I went into the Marketplace, I 'lost' the character to the 'void' and had to restart client to get back to character screen, to choose a different one to play. At no time was I 'totally' shut out of a character to play, there was always 1 or 2 that would play, therefore, It is as stated above, a DDO specific error, and might be connected witht something in the Marketplace instances (whether or not it is connected to the asinine Crystal Cove, I leave that to you to investigate).

[Steiner-Davion]

While I LOVE the game I am not all that tech savy and don't plan on going through loops to play. And I bet I am not alone so as a VIP I would suggest if you want people to keep paying for your product it would be incumbent on you to fix the problem.

This is sort of like a truck being stalled in front of your business blocking your driveway. Not your fault but telling people just to part blocks away and walk will not have the effects you probably desire.

Apparently, you cannot comprehend what she said. There is nothing wrong with things on Turbine's end. In plain simple terms "Part of the internet is broke" It is a akin to when a storm comes through and knocks down all the electric transmission lines and you loose power to your home and community. What Tolero has suggested is akin to you or any business having a emergency generator. IF you don't then you will just have to deal with it and wait.

Besides do you really think Tolero enjoyed getting up at 3:30 this morning? Remember it is just a game, sit back relax and keep calm.

[moomooprincess]

And the same problems?

[Cyndder]

This is almost absolutely a DDO specific issue. For most of the night I was online and playing my primary toon (on Thelanis) without problems. Then I DC'd with that toon and spent about 20 min trying to get back in. I logged into an alternate account without problems, then back into that account and that toon and was able to get in without problems. I played for about 2 more hours. Then I DC'd again. This time I was completely unable to get back in using that toon, but different toons on the same (or different) account on the same or different servers were able to get logged in and play without problems. Just this one toon was having problems.
Now several hours and another trouble ticket later, I'm completely unable to connect using my primary account. If I resort to my daughter's account on the same system, using the same router and connecting to the same server, I have no problem.

IF the issue were truly routing related, then it would affect every toon on this system going to that same server. It wouldn't be specific to a DDO account since the routing path is stored at the ISP level (per previous posts from Devs). I'm a VIP and becoming VERY frustrated with this issue. Please get this resolved today.

Thanks,
Tired and Frustrated

Most certainly is a routing issue. I had the same issues you described above occur to me today--last time I could not get in at all this time I had characters lock up on load screens that I could not log back into. I activated the Cyberghost VPN after spending 30 minutes trying to correct it; low and behold I'm back in on that toon because the VPN is taking me through cogent.com instead of altern.net.

I understand the frustration...trust me I am frustrated by this as well; unfortunately this is an Internet issue in the Boston area that needs to be corrected but not Turbine specific issue. Is Turbine looking into this again....you betcha...because we the customers are not happy about it.

Backing up with facts: Route that failed!

3 14 ms 13 ms 14 ms te-0-5-0-3-ur03.cosprings.co.denver.comcast.net [68.86.105.17]
4 19 ms 23 ms 23 ms te-0-5-0-1-ar02.denver.co.denver.comcast.net [68.85.89.33]
5 20 ms 19 ms 23 ms pos-0-1-0-0-ar02.aurora.co.denver.comcast.net [68.86.128.242]
6 20 ms 22 ms 19 ms he-3-4-0-0-cr01.denver.co.ibone.comcast.net [68.86.90.149]
7 33 ms 35 ms 35 ms he-2-11-0-0-cr01.dallas.tx.ibone.comcast.net [68.86.87.30]
8 34 ms 32 ms 35 ms be-15-pe02.1950stemmons.tx.ibone.comcast.net[68.86.83.114]
9 42 ms 35 ms 35 ms 66.208.229.170
10 33 ms 33 ms 32 ms 0.xe-3-0-6.XL4.DFW7.ALTER.NET [152.63.97.81]
11 * * * Request timed out.
12 112 ms 84 ms 70 ms 0.xe-11-0-0.GW15.BOS4.ALTER.NET [152.63.21.42]
13 71 ms 71 ms 72 ms internap-gw.customer.alter.net [152.179.134.214]
14 73 ms 71 ms 70 ms border1.te7-1-bbnet1.bsn003.pnap.net [63.251.128.43]
15 71 ms 71 ms 71 ms turbine-7.border1.bsn003.pnap.net [64.95.76.202]
16 71 ms 73 ms 72 ms 74.201.102.154
17 72 ms 71 ms 71 ms 74.201.103.121

Trace complete.

Route that worked via VPN:

2 59 ms 59 ms 59 ms ash-eqx-01gw.voxility.net [37.221.173.225]
3 93 ms 59 ms 58 ms ae52.edge2.Washington4.Level3.net [4.53.114.137]
4 58 ms 60 ms 62 ms ae-2-70.edge3.Washington4.Level3.net [4.69.149.82]
5 60 ms 89 ms 61 ms te0-7-0-21.mpd22.iad02.atlas.cogentco.com [154.54.13.69]
6 60 ms 59 ms 61 ms be2045.mpd22.dca01.atlas.cogentco.com [154.54.26.117]
7 67 ms 66 ms 67 ms te0-1-0-7.mpd22.jfk02.atlas.cogentco.com [154.54.1.142]
8 72 ms 72 ms 72 ms te0-6-0-5.ccr22.bos01.atlas.cogentco.com [154.54.44.26]
9 78 ms 79 ms 79 ms te3-2.ccr01.bos06.atlas.cogentco.com [154.54.46.130]
10 72 ms 71 ms 72 ms 38.104.252.70
11 * 72 ms 71 ms border1.te7-1-bbnet1.bsn003.pnap.net [63.251.128.43]
12 71 ms 71 ms 73 ms turbine-7.border1.bsn003.pnap.net [64.95.76.202]
13 72 ms 72 ms 74 ms 74.201.102.154
14 76 ms 72 ms 73 ms 74.201.103.121

Trace complete.

[Hendrik]

And the same problems?

With very little research the issues with ALTER.NET/Verizon and what we are currently experiencing go back years.

I have found the exact same issues dating back to 2010 with games like WoW, World of Tank, and League of Legends, just to name a few.

This is a reoccurring problem with Verizon at the source.

[slarden]

Ooookay so it appears that it's flaring up again. Our datacenter team is going to try something creative to try and help more players get around it from our end. If I can get some more tech savy volunteers who are having trouble to PM me updated traceroutes again, I can collect them together and forward them over so they can work their magic. Thank you so much for your patience everyone, I know this is a super confusing issue.

...incidentally I think this is the first time in the years that I've been here where the culprit of a problem was "teh interntez" (as opposed to software, hardware, lightening, or bears).

Wow thank you for sacrificing sleep to work on this. It must be frustrating to deal with an issue the team can't fix and have your frustrated customers taking it out on you at the same time. Good luck with getting this worked out.

[slarden]

Here is Verizon's take on the issue:

http://publicpolicy.verizon.com/blog...cogent-dispute

Now keep in mind Verizon is accusing cogent of not paying their fair share based on rules that Verizon and other large carriers effectively dictated out of self-interest. Depending on how you view it Cogent is being stubborn or they are standing up to the bullies.

Either way, Turbine and we are caught in the middle. The same thing is true for many other MMOs and Netflix, although it can be argued that Netflix is ultimately the cause of this entire problem - again depending on your viewpoint.

[danotmano1998]

...incidentally I think this is the first time in the years that I've been here where the culprit of a problem was "teh interntez" (as opposed to software, hardware, lightening, or bears).

First off, thanks for all your efforts on this. It seems fairly obvious it's nothing on your end, yet you are still trying to make things better by any means possible.

Second...

Bears? I'd love to hear THAT story sometime!

[Steiner-Davion]

First off, thanks for all your efforts on this. It seems fairly obvious it's nothing on your end, yet you are still trying to make things better by any means possible.

Second...

Bears? I'd love to hear THAT story sometime!

I'm from the Boston Area, and live 2 towns over from Turbine HQ, and can help explain this....

Right around the time we got Druids, there Bears being sited in a couple of communities outside of Boston. Specifically, there was a bear in the parking lots in the business park where Turbine is located, and a bear which was in a tree in the city of Brookline (which by the way they captured, brought to the western part of the state, and then made its way back to the same area).

[Cordovan]

Just to make sure everyone's up-to-date: Yes, we did have a recurrence of the issues that arose over the weekend, with the same cause. One of our affiliate connection providers engaged in the same solution as last time (again re-rerouting traffic around the problematic area), around mid-morning Eastern time, and it appears to have largely resolved the issue.

[Cyndder]

I dropped from game and shutdown the VPN to check to see if the routing issuse for me was looking better;

3 15 ms 15 ms 15 ms te-0-5-0-3-ur03.cosprings.co.denver.comcast.net [68.86.105.17]
4 20 ms 40 ms 18 ms te-0-5-0-1-ar02.denver.co.denver.comcast.net [68.85.89.33]
5 34 ms 23 ms 18 ms pos-0-1-0-0-ar02.aurora.co.denver.comcast.net [68.86.128.242]
6 22 ms 19 ms 24 ms he-3-4-0-0-cr01.denver.co.ibone.comcast.net [68.86.90.149]
7 21 ms 18 ms 19 ms te3-5.ccr01.den03.atlas.cogentco.com [154.54.10.33]
8 20 ms 22 ms 18 ms te0-1-0-3.ccr21.den01.atlas.cogentco.com [154.54.83.29]
9 36 ms 37 ms 54 ms te0-4-0-7.ccr21.mci01.atlas.cogentco.com [154.54.27.238]
10 42 ms 59 ms 42 ms te0-7-0-1.mpd21.ord01.atlas.cogentco.com [154.54.7.137]
11 67 ms 70 ms 67 ms te0-2-0-2.ccr21.bos01.atlas.cogentco.com [154.54.43.178]
12 78 ms 73 ms 70 ms te3-4.ccr01.bos06.atlas.cogentco.com [154.54.46.134]
13 67 ms 68 ms 68 ms 38.104.252.70
14 65 ms 82 ms 66 ms border1.te7-1-bbnet1.bsn003.pnap.net [63.251.128.43]
15 67 ms 69 ms 68 ms turbine-7.border1.bsn003.pnap.net [64.95.76.202]
16 67 ms 68 ms 67 ms 74.201.102.154
17 68 ms 68 ms 67 ms 74.201.103.121

Trace complete.

Back in game without VPN and that is one more route; not just for me but for anyone forced into that pipe--that has been fixed. Thanks for your efforts.

[Cyndder]

Just to make sure everyone's up-to-date: Yes, we did have a recurrence of the issues that arose over the weekend, with the same cause. One of our affiliate connection providers engaged in the same solution as last time (again re-rerouting traffic around the problematic area), around mid-morning Eastern time, and it appears to have largely resolved the issue.

So my question for you is...can they keep that in place? Given the history of your ISP and Verizon...in order to get out of the middle of the Netflix war...can they just leave those routes in place?

[Mahiya]

Thanks,
Tired and Frustrated

I understand your frustration but not being a computer expert with great understanding of routing servers etc, I put 2 and 2 together when the game works fine on an alternate router and not my primary. Also people who seem to practically understand binary verify this, not to mention the issue is not DDO, turbine, limited. Forums of atleast a few other non related game providers (namely MMORPGs - all I have looked at) have the same complaints. I am not trying to step on you good person. I would be highly upset also, unfortunately I am dealing with an uncessary resolution found here in the forums (a VPN) but its unecessary based on the router type of problems and not the game provider :[