Lag after data centre move

[count_spicoli]

So much for impressing any influx of new players that might have been drawn in by the 10th Anniversary activities. This game needs new blood and new players to go along with the stalwarts, but if dedicated vets are walking away, what's to keep a newbie here when they can't even function in the game due to issues with lag? This concerns me as well.

Ya i had seen alot of new people logged in for whatever reason but after this mess they probably bailed. Interesting timing for this and why not wait until the lotro mess was fixxed before shoving us over and putting everybody in the same disaster.

[Valniel]

Anyone noticed lag in less populated serv like Wayfinder?

Sarlona is playable in daylight (I'm in Europe) but not at night so it seems to be based on the amount of people logged, the size of the party etc...

[Assassination]

Just lag wiped in Fire on Thunder peak raid. 11 members. Most of us hung on entry to raid for about 30 seconds, then went along ok for a few minutes. Then 5 minutes of bad choppiness with no deaths, about 10 minutes in it just sort of freezes up and wipes us.... 11 unhappy players on ghallanda, many mentioning uninstalling the game.... I did not notice the "yellow" lag sign at top of screen when any of this was happening.

Hope this can be resolved, raiding is currently not possible. Most quests are laggy as well, big lag hits in quests where a lot of things are going on. Did a wizking run, we split into three groups to do towers, lag caused many deaths and we had a difficult time completing.

Game is not fun. That's a problem.

[Captain_Wizbang]

It is bad, real bad. Couldn't even play.

[Eryhn]

The thing that seems to render the theory that its the connection or quest.net as unlikely is the fact that when its occurring it seems very zone specific.

If it was just some people randomly experiencing packet loss due to the connection, it wouldnt hit everyone in an instance all at once, which is what it seems to do. If it was the full on turbine connection experiencing the issue, other people in other quests wouldnt be spared.

yes that is very true. just half hour ago i was in a pug for highroad, and between 3rd and 4th quest was the only one briefly recalling to estar to sell. I say briefly, cause that was the intention. I spent 3mins from the highroad gate to market stuttering along on autorun, in so far as you can call it *run*, then another 2 mins for the vendor to somewhat load and take my stuffs and another 3min back.

when i stepped back into the highroad slayer area, the lag was gone immediately. meanwhile, the other party members were unaffected by this spike (which i saw other people in estar public sharing as they stuttered along as well) and had cleared half the quest. averse to that, when we had in quest lag before, which was present in one form another on all 5 of the quests, we all shared it at the same time.

so yeah,there is some weird instance thing going on, too. I wonder if all the anti-lag bandaids they slapt on over the last half year or so are still in place...

regardless of that the routing doesnt look exactly flawless

[maddong]

The thing that seems to render the theory that its the connection or quest.net as unlikely is the fact that when its occurring it seems very zone specific.

If it was just some people randomly experiencing packet loss due to the connection, it wouldnt hit everyone in an instance all at once, which is what it seems to do. If it was the full on turbine connection experiencing the issue, other people in other quests wouldnt be spared.

It seems zone specific to me. I have had more lag soloing the game than before. If a quest starts with lag it is going to have lag the whole time. If you start ok then it usually goes ok the whole time.

I think?

[Zachski]

I have no doubt that this will be fixed eventually. Because if it doesn't, this game is going to end, and that's bad for business.

Don't confuse optimism for happiness, however. No one is happy with this situation.

And on a pessimistic note, I don't know when "eventually" will be. And I don't know how much it's going to cost Turbine - and how much of that is gonna be passed onto us.

I'm guessing that if Turbine can't get LOTRO and DDO working with the current servers, they'll just have to abandon them and flat-out buy new ones - more expensive ones - and hope that they work. This will probably mean lay-offs (which will further hurt future endeavors, as being under-staffed is a terrible, terrible thing and yet the accountants don't care), more expensive quest packs for lesser content, and various attempts at encouraging players to spend money on things, to make up for the deficit.

I don't place any of the blame on the players, either. No one is *obligated* to spend money on a product that isn't meeting their standards. I place the blame on Turbine's upper staff.

[Kompera_Oberon]

It seems to me that the more expensive the application, the more it needs system modifications to run correctly.

I have worked for some of the largest producers of network management software in the world, installing COTS software for their customers. Very little software that is sold into a large corporate or government environment requires any system modifications. Customers simply will not put up with that kind of BS these days since it places additional burden on their IT staff to learn the idiosyncrasies of each different piece of software. Install the software, install a DB if that is required, configure them to talk to each other, and you're done. Often within just a few hours.

Customization of the application itself is an entirely different matter, of course. That can take weeks or months (or longer, in some especially horrible cases) depending on the customer and their specific needs, on their ability to manage the scope of their feature requests, and especially on their central authority and the amount of hoops their various divisions might force a project to jump through prior to approval.

Actually, it's been shown conclusively in this thread that a lot of the lag is in the network route, not at endpoints. Wouldn't say anything about "majority" here myself since I don't get to see any worthwhile and verified statistics, but there's a lot in the network.

No, it has not.

Bigger pipe doesn't necessarily help when one of the dimensions it's bigger in is length... well, that may solve itself once the routing stabilizes again. (Yes, I do realize that this involves a bunch of network / routing folks doing manual adjustments too, but anyway.)

DNS updates typically take far less than a day to propagate. And any slowpokes are typically very small players like little Ma and Pa ISPs. My traces show that there was no issue, for me at least, crossing three major carrier networks, almost immediately after the servers came back up. (And "almost" only because I did not run the traces immediately) This new lag issue is not and was never DNS, and that whole "the speed of light is fast" thing eliminates any differences in physical distance, which will be a very, very trivial portion of any of their customer's routing paths in any event.

And as to your parenthetical at the end there, no, it does not require that.

[Kompera_Oberon]

Anyone noticed lag in less populated serv like Wayfinder?

Sarlona is playable in daylight (I'm in Europe) but not at night so it seems to be based on the amount of people logged, the size of the party etc...

No. On Wayfinder where we were blessed with very little lag now almost every raid suffers from a huge amount of lag.

DotJ unplayable on EH. A second attempt on EN was successful, but only because the damage being taken was less so more players could live through the multiple lag spikes. LN Shroud suffered a huge amount of lag but we still completed.

But it's not fun fighting lag instead of monsters.

[Nobnarb]

Lag made things extremely unplayable. Got stuck at loading screen trying to transition from public area to wilderness. Attempted to run Feast or Famine. Minor lag pockets during quest, complete lockup on quest completion. Entire group locked up at same point, so wasn't just my route to DDO being issue. Lag was so bad, that on releasing was still being treated as stuck in a Wall of Fire, and continued to instantly die every time I released in public area. After 4th death on airship, I just closed client.

Ran traceroute immediately after closing DDO client. Verified that I don't have any entries in HOST file, nor do I have any port forwarding setup for 9000-9010 previously.

Tracing route to gls.ddo.com [198.252.160.23]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 1 ms <1 ms <1 ms 10.1.10.1
3 27 ms 8 ms 10 ms 96.120.25.133
4 14 ms 11 ms 15 ms te-0-3-0-21-sur04.nchicago.il.chicago.comcast.net [162.151.33.49]
5 32 ms 21 ms 16 ms te-2-7-0-11-ar01.area4.il.chicago.comcast.net [68.87.232.69]
6 17 ms 15 ms 16 ms be-33491-cr02.350ecermak.il.ibone.comcast.net [68.86.91.165]
7 13 ms 17 ms 14 ms hu-0-15-0-0-pe04.350ecermak.il.ibone.comcast.net [68.86.85.210]
8 31 ms 12 ms 11 ms chp-edge-01.inet.qwest.net [216.207.8.189]
9 41 ms 35 ms 35 ms ewr-cntr-11.inet.qwest.net [205.171.17.2]
10 34 ms 32 ms 39 ms 206.103.215.50
11 33 ms 34 ms 36 ms 63.236.3.130
12 35 ms 32 ms 46 ms 10.192.216.4
13 33 ms 32 ms 32 ms 198.252.160.23
14 34 ms 31 ms 31 ms 198.252.160.23

Trace complete.

[FifthTime]

Tried 1 raid with guildies because I have a rubber arm.

Was horrendously frustrating.

I quit at the 7 minute mark of a terribly long lag freeze.

The rest of the guild quit soon after.

We then logged out.

[blerkington]

Hi,

The last couple of tracerts posted here are no longer showing a timeout just before hitting Turbine's servers. When I run one myself, mine still does.

Anyone got a theory about why this might be changing? Is anyone who got that timeout before not getting it now?

Thanks.

[Knobull]

Hi,

The last couple of tracerts posted here are no longer showing a timeout just before hitting Turbine's servers. When I run one myself, mine still does.

Anyone got a theory about why this might be changing? Is anyone who got that timeout before not getting it now?

Thanks.

That private IP address still times out for me. I have no idea how people are able to ping it. It is a private IP address.

My current theory is that if you are behind a router and on a private network yourself, then you cannot ping it. If you are going commando online with a public IP address on your computer's nic, then you may be able to ping it. (I still do not see how this is possible.)

Others have tried to explain that having a private IP address on the public IPv4 network is just fine. I still think it is internet breaking. That is just not allowed as I see it.

Try and ping this private IP address on my network: 192.168.5.55 .. you can't, because that's how private IP addresses work.

Any carrier that allows you to connect to a private IP address that is outside your own network is breaking the internet.

What happens if I have my own internal network setup on 10.192.x.x and I try to ping 10.192.216.4 (the ip address in question) - I will see the one on my internal network, and the one that is incorrectly on the public network should not be seen.

This is just not done.

The Internet Assigned Numbers Authority (IANA) has reserved the
following three blocks of the IP address space for private internets:

10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)

We will refer to the first block as "24-bit block", the second as
"20-bit block", and to the third as "16-bit" block. Note that (in
pre-CIDR notation) the first block is nothing but a single class A
network number, while the second block is a set of 16 contiguous
class B network numbers, and third block is a set of 256 contiguous
class C network numbers.

An enterprise that decides to use IP addresses out of the address
space defined in this document can do so without any coordination
with IANA or an Internet registry. The address space can thus be used
by many enterprises. Addresses within this private address space will
only be unique within the enterprise, or the set of enterprises which
choose to cooperate over this space so they may communicate with each
other in their own private internet.

As before, any enterprise that needs globally unique address space is
required to obtain such addresses from an Internet registry. An
enterprise that requests IP addresses for its external connectivity
will never be assigned addresses from the blocks defined above.

https://www.ietf.org/rfc/rfc1918.txt

[Matadorz]

Of all the years I have been playing ddo this is the first time I have experience something like this.

After a couple quest the lag gets so annoying and boring that you just want to log off the game.

I think as paying customers we deserve a more detailed explanation of what's going on and how long are we going to have to wait for the devs to fix it...

[redoubt]

Anyone noticed lag in less populated serv like Wayfinder?

Sarlona is playable in daylight (I'm in Europe) but not at night so it seems to be based on the amount of people logged, the size of the party etc...

I am on Sarlona.

Tried Fashion Madness on EE with 4 people. 3 lagged and only the 1700hp warlock survived the fashion show. Then we lag wiped in the end fight. It spawned and all 4 died without moving.

We did then run the Stone Palace and the first 4 of the Stormhorns. Many times we would see very bad freezes. We completed these quests, but by the 3rd quest in the Stormhorns I was just trying to make it through. By the 4th I was done and I logged out without trying WGU. It just was not fun and trying to outmaneuver the lag and succeed was just exhausting.

[Steen]

Just wanted to put my two cents in. At the moment end game is near impossible if this isn't fixed soon the group of guys I raid with will all leave. I was looking forward to less lag with the server move but I don't know what I was thinking....

[Nyata]

DNS updates typically take far less than a day to propagate. And any slowpokes are typically very small players like little Ma and Pa ISPs. My traces show that there was no issue, for me at least, crossing three major carrier networks, almost immediately after the servers came back up. (And "almost" only because I did not run the traces immediately) This new lag issue is not and was never DNS, and that whole "the speed of light is fast" thing eliminates any differences in physical distance, which will be a very, very trivial portion of any of their customer's routing paths in any event.

And as to your parenthetical at the end there, no, it does not require that.

then what is your explanation for number 7 to 13? I am getting bounced back to europe *twice* before I make it to my destination. if that does not create delays, I don't know what does.

#, Country, Town, Lat, Lon, IP, Hostname, Latency (ms), DNS Lookup (ms), Distance to previous node (km), Whois

2, Germany, Hattersheim, 50.066696, 8.466705, 88.69.56.1, dslb-088-069-056-001.088.069.pools.vodafone-ip.de, 20, 20, 86, 88.69.56.1
3, *, *, 50.066696, 8.466705, *, *, 0, 0, 0, *
4, Germany, (Unknown), 51.0, 9.0, 188.111.217.204, (None), 20, 20, 110, 188.111.217.204
5, Germany, (Unknown), 51.0, 9.0, 92.79.212.45, (None), 25, 20, 0, 92.79.212.45
6, Germany, (Unknown), 51.0, 9.0, 92.79.213.170, (None), 25, 15, 0, 92.79.213.170
7, United States, (Unknown), 38.0, -97.0, 166.63.204.209, ae11-100-xcr1.fri.cw.net, 30, 20, 7794, 166.63.204.209
8, Europe, (Unknown), 47.0, 8.0, 195.2.31.234, ae14-xcr1.ptl.cw.net, 123, 15, 8000, 195.2.31.234
9, Germany, (Unknown), 51.0, 9.0, 188.111.222.106, (None), 65, 20, 451, 188.111.222.106
10, United Kingdom, (Unknown), 51.5, -0.13000488, 195.2.25.69, ae13-xcr2.nyk.cw.net, 50, 20, 638, 195.2.25.69
11, United States, Chesterfield, 38.649994, -90.5334, 206.103.215.50, (None), 10, 4189, 6786, 206.103.215.50
12, Germany, (Unknown), 51.0, 9.0, 188.111.222.106, (None), 5, 25, 7366, 188.111.222.106
13, United States, (Unknown), 38.0, -97.0, 198.252.160.23, (None), 103, 118, 7794, 198.252.160.23

[Aletys]

Playing today was an exercise in frustration. We've stopped playing EE because of the lag issues. We see lag in public areas, in quests, & raids. We actually lag wiped in an EH Devil Assault, something that has never happened to us before.

I checked the traceroutes during the lag, & saw absolutely no difference from when it's not lagging. Latencies were good. Pinging Google & the Turbine address (gls.ddo.com) on a continuous basis gave exactly the same (good) results: Average of 15-16 ms with no packet losses. I'm beginning to doubt the routing theory, and think it must be something inside Turbine.

[dontmater]

it there any other ips we can trace to help? i

[-Avalon-]

it there any other ips we can trace to help? i

Well, thanks to DDO, the first thing I thought when reading this was, 'Improved Precise Shot' but then my networking education kicked back in