Lag: DETAILS

[Thanimal]

I hold the possibly generous belief that if fixing the lag were easy, Turbine would have already done so. I further hypothesize this means it must be a subtle, evil bug. So what I want to do here is to provide as much detail as I can about an experience that ended up sucking because of lag, in hopes that this is useful in ultimately tracking down and fixing the bug. Perhaps others can do the same either here or in their own threads. Naturally I can't stop mindless "fix the freakin' lag you lazy developers" posts here or elsewhere, but I'll just remind you that's been done and doesn't seem to have helped.

I was running as Vrol on Thelanis from roughly 9p ET to 10:40p ET last night (3/8/09), doing Wizard King (I hope that's the name -- I'm a n00b in higher level content because I constantly start new characters!). Group of 6, mostly from the guild Remember Honor. Although there was random ~1 sec lag here and there, the first hour and a half was at least playable. Then pretty suddenly we all began to experience massive lag.

[Comment: This seems to eliminate the "client side" hypothesis that once or twice I've seen thrown around. The probability of 6 geographically distributed players hitting msasive lag at the same time on their local clients is negligible.]

The lag quickly ramped up to completely unplayable -- taking a step once every several seconds, somewhere around 10:30p ET. It remained like this for about 10 minutes, after which there was a slight improvement: still unplayable, but now able to take several steps at a time before lagging out.

At roughly 10:45p ET, we gave up. We then all recalled out. Once out of the instance, the lag disappeared and we could run around normally in the Sands.

[Comment: This seems incredibly important from a diagnostic standpoint, so I'm especially curious if this has been the standard experience. It seems to prove that the issue isn't hardware related, because I can't see why we would have changed hardware when we recalled out. It seems to suggest to me that something went wrong in the instance -- i.e. some kind of software bug.]

Unfortunately, since we weren't running a controlled experiment, we didn't try going back in, so I can't say if the "saved" instance remained hosed until it reset. Perhaps others have done that and can provide that information.

Good luck Devs! Godspeed to find and correct this horrible lag!

[Thanimal]

Just remembered something else that might matter: During this entire experience, there was never any noticeable voice lag -- we continued to be able to converse about how wonderful the lag was.

In my experience, whenever I've experienced obvious client-side lag, my voice has also lagged.

[MrCow]

there was never any noticeable voice lag -- we continued to be able to converse about how wonderful the lag was.

Turbine goes through a completely separate company to handle the voice transactions.

[Deadz]

One players lag can lag the whole group, can it not?

Players toons do interact with each other

[Thanimal]

One players lag can lag the whole group, can it not?

It cannot. There is no need to synchronize transactions in that manner, and DDO clearly does not: I've had a party member lag out locally before and it has no impact on anybody else. If I am lagging, it means I am getting essentially ALL of my packets on a huge delay. If somebody else is lagging, I may get updates about THEM on a delay, but this will not effect my own communications. These transactions are asynchronous.

Too bad that's not the problem: There would be a zillion examples they could turn to for how to fix THAT! :)

[Thanimal]

Turbine goes through a completely separate company to handle the voice transactions.

Would have guessed as much. I was just bringing it up because it provides further evidence against the client-side lag hypothesis.

[negative]

It seems pretty clear what you experianced was server side lag. However, there are still lag issues talked about on the forums (or I've seen in-game) that seem to be client side. Granted, I don't think this was one of those issues.

At roughly 10:45p ET, we gave up. We then all recalled out. Once out of the instance, the lag disappeared and we could run around normally in the Sands.

[Comment: This seems incredibly important from a diagnostic standpoint, so I'm especially curious if this has been the standard experience. It seems to prove that the issue isn't hardware related, because I can't see why we would have changed hardware when we recalled out. It seems to suggest to me that something went wrong in the instance -- i.e. some kind of software bug.]
Depending on the server setup this may not be the case. Each "server" is not a single box. I believe it's quite possible that switching instances will switch the hardware you are connecting to.

[Mattee]

It seems pretty clear what you experianced was server side lag. However, there are still lag issues talked about on the forums (or I've seen in-game) that seem to be client side. Granted, I don't think this was one of those issues.

Depending on the server setup this may not be the case. Each "server" is not a single box. I believe it's quite possible that switching instances will switch the hardware you are connecting to.

So therefore, the lag is server-based.... But wasn't there a MASSIVE downtime earlier this year to "upgrade the server hardware"? In my experience a hardware upgrade usually improves things... not makes them much much worse (I sight personal lag times of up to 2500ms over the last two months).

Also, why don't the servers carry the same hardware? and for that matter, What Is the hardware? At times it seems like they've hooked up an Atari 2600 to run the common areas! (Bit of early-morning sarcasm. sorry. hehe)

[negative]

So therefore, the lag is server-based.... But wasn't there a MASSIVE downtime earlier this year to "upgrade the server hardware"? In my experience a hardware upgrade usually improves things... not makes them much much worse (I sight personal lag times of up to 2500ms over the last two months).

Also, why don't the servers carry the same hardware? and for that matter, What Is the hardware? At times it seems like they've hooked up an Atari 2600 to run the common areas! (Bit of early-morning sarcasm. sorry. hehe)
Someone can correct me here, but I keep hearing that Turbine outsourced the servers recently. I'm guessing that was the "upgrade", if you get my drift.

Previously I think they were on blade servers, I don't know what they are on now. Probably more blade servers.

I ussually run around 60-120 fps, but I still get the server-wide lag spikes occasionally. If you really want some insight in whether you lag is more likely one or the other, just hit control-f and watch your framerate. If you are lagging with a high framerate, it's probably network or server related. If you are lagging with a bad framerate, I'd say it's your computer. My guess is those that complain about disco balls will see a bad framerate around them, unless there really is some sort of bug in the server code (which may be true as I've rarely seen what I believed to be server lag when disco balls are cast in part 1 of shroud).

I'd say a high latency is also a good indication of network or server lag. But that could be something out of turbines control. Could be your isp in that case, unless DDO is the only app with bad latency.

Bottomline is lag isn't simple.

I'm just an amateur computer nerd though so I could be way off base.

[Missing_Minds]

In your running of wizard king, how many of the mobs were you killing on sight? Or were you just zerging past them?

I'm actually starting to believe that zergers are the cause of lots of lag. My bases for this is the house J quest. The spire with the lich. (I want to call it 'and the dead shall rise. but I"m pretty certain I'm wrong on the name.) But that quest would spawn so many active NPCs that it would literally lag the party badly. Why? To many calls back and forth between all of the NPCs talking. I actually proved this when I jumped into the middle of about 40-50 odd of the skeletons that spawned and started to take them out. At the time I figured I was just going to have fun being suicidel, but after about 20 of them were taken out, we started to actually visually see lag reduction happening. Once the floor was cleared of the undead, we were moving normally again.

Now, one thing I will note, we never did lose connection or were in risk of losing connection under this circumstance.

[Thanimal]

In your running of wizard king, how many of the mobs were you killing on sight? Or were you just zerging past them?

I'm actually starting to believe that zergers are the cause of lots of lag. My bases for this is the house J quest. The spire with the lich. (I want to call it 'and the dead shall rise. but I"m pretty certain I'm wrong on the name.) But that quest would spawn so many active NPCs that it would literally lag the party badly. Why? To many calls back and forth between all of the NPCs talking. I actually proved this when I jumped into the middle of about 40-50 odd of the skeletons that spawned and started to take them out. At the time I figured I was just going to have fun being suicidel, but after about 20 of them were taken out, we started to actually visually see lag reduction happening. Once the floor was cleared of the undead, we were moving normally again.

Now, one thing I will note, we never did lose connection or were in risk of losing connection under this circumstance.

GREAT question! The information could be important to debuggers. This was actually a slow run -- my guild was teaching me the quest and were stopping to kill everything. I don't think we left anything alive or trailed any monsters around.

Last night I had a case more like what you're describing. I was zerging Dryden's Tomb and mostly not bothering to kill anything and then when I tried to read the insciption, surrounded by random baddies, I started to experience massive lag. (Just barely got out alive!)

So that may be a contributing cause, but I'm quite certain there is more to it than that, because I recently was part of a Delera's zerg in which nearly nothing was killed, and lag never became an issue at all.

[juniorpfactors]

In your running of wizard king, how many of the mobs were you killing on sight? Or were you just zerging past them?

I'm actually starting to believe that zergers are the cause of lots of lag. My bases for this is the house J quest. The spire with the lich. (I want to call it 'and the dead shall rise. but I"m pretty certain I'm wrong on the name.) But that quest would spawn so many active NPCs that it would literally lag the party badly. Why? To many calls back and forth between all of the NPCs talking. I actually proved this when I jumped into the middle of about 40-50 odd of the skeletons that spawned and started to take them out. At the time I figured I was just going to have fun being suicidel, but after about 20 of them were taken out, we started to actually visually see lag reduction happening. Once the floor was cleared of the undead, we were moving normally again.

Now, one thing I will note, we never did lose connection or were in risk of losing connection under this circumstance.


correct, and you will see the same thing in running to VOD all the teleporting devils and orthons, cause major lag, all the dps calculations the system cant handle when fighting parts 4/5 of the shroud, when zerging coalesence chamber leaving all the scorpians and trash mobs active...and then you have noob fall or player recall for mana, MASSIVE lag with so much trash mobs left alive.....
DDO cant handle all the calculations and teleporting and AI it simply cant handle the mathmatics, and especially at peek playtime ...we always say, enjoy your 9:30 Eastern shroud....

jrp

[Missing_Minds]

correct, and you will see the same thing in running to VOD all the teleporting devils and orthons, cause major lag, all the dps calculations the system cant handle when fighting parts 4/5 of the shroud, when zerging coalesence chamber leaving all the scorpians and trash mobs active...and then you have noob fall or player recall for mana, MASSIVE lag with so much trash mobs left alive.....
DDO cant handle all the calculations and teleporting and AI it simply cant handle the mathmatics, and especially at peek playtime ...we always say, enjoy your 9:30 Eastern shroud....

jrp

Haven't the dev's already admitted that teleportation was causing lag? I can't remember.

However... I wonder if the dps calculation lag.... As a player I know what you are talking about and know it is real. Suddenly all PCs are sinked up swingign at the same target, doing damage at the same time. (and missing/not getting swings also if you really analize it closely) I wonder if that lag is caused by their random number generator (RNG) not able to keep up with the requests... I mean it makes sense really if the generator only runs for X amount of time a frame, it is being called on more than what it can actually handle so either over frames or drops requests. (my guess is drops requests.)
Or even if the RNG is only framed for so much time an instance it still works out for causes of lag...

I mean I never understood why there could be lag in shroud part one, when the only thing alive was a "portal", but this.. this makes sence.

It is either that or one entity is only allowed so much calculation time. Hence why a swarm of kobolds won't lag a low level party, but one portal can.

Another thing I'll note. My hop path to gls.ddo.com has changed. 3 months back (when lag was apparently less) I had an average hop number of about 20-22. It is now an average of 18 hops and with the little I've played recently, the lag has seemed worse.

[Thanimal]

correct, and you will see the same thing in running to VOD all the teleporting devils and orthons, cause major lag, all the dps calculations the system cant handle when fighting parts 4/5 of the shroud, when zerging coalesence chamber leaving all the scorpians and trash mobs active...and then you have noob fall or player recall for mana, MASSIVE lag with so much trash mobs left alive.....
DDO cant handle all the calculations and teleporting and AI it simply cant handle the mathmatics, and especially at peek playtime ...we always say, enjoy your 9:30 Eastern shroud....

jrp

Hmm, the only computation-intensive part I can think of in all of that is pathing. And two things make me suspect this isn't the problem:

1) Throwing more server hardware at this would fix it. I know people accuse Turbine of being cheap a lot, but NOBODY could think they'd make more money in the long-run by turning off 10s of thousands of players (i.e. 100s of thousands of dollars PER MONTH) instead of buying some beefier servers. If the fix were that easy, they'd have done it.

2) Even if they were SO near-sighted that they refused to buy any more hardware, it would be simple to code around this, but just having mobs that are farthest from the players stand in place any time the pathing resources got tight. Again, this would be an easy fix.

So I'm sticking with my hypothesis that the bug is far more subtle and evil. But if I'm wrong, then: Turbine, implement one of the above solutions and let's move on!! :)

[juniorpfactors]

Hmm, the only computation-intensive part I can think of in all of that is pathing. And two things make me suspect this isn't the problem:

1) Throwing more server hardware at this would fix it. I know people accuse Turbine of being cheap a lot, but NOBODY could think they'd make more money in the long-run by turning off 10s of thousands of players (i.e. 100s of thousands of dollars PER MONTH) instead of buying some beefier servers. If the fix were that easy, they'd have done it.

2) Even if they were SO near-sighted that they refused to buy any more hardware, it would be simple to code around this, but just having mobs that are farthest from the players stand in place any time the pathing resources got tight. Again, this would be an easy fix.

So I'm sticking with my hypothesis that the bug is far more subtle and evil. But if I'm wrong, then: Turbine, implement one of the above solutions and let's move on!! :)


I brought up the devils and orthons 12 months ago, and "forum boys" called me crazy then too.... and now we know it to be true as they admitted as much

so I go with my hypothesis

jrp

[Thanimal]

I brought up the devils and orthons 12 months ago, and "forum boys" called me crazy then too.... and now we know it to be true as they admitted as much

Can you point me to that? If they KNOW what the bug is and haven't fixed it, that casts this whole thing in a VERY different (VERY ugly) light. But I reserve judgement until I read more details.

[Stormanne]

I'm not entirely sure if this makes any difference or is of any importance.

Last night, we did a guild run on Sarlona, Hound of Xoriat. No lag as the party gathered in the market. As soon as we zoned into the Subterranean, we started getting hit with lag. It wasn't random either. We did a little experiment while we were buffing for the run. The lag was persistent on an eight second interval. Every eight seconds we would get a two to three second lag spike. And it wasn't localized to one or two people, but the entire group. And it's not like we are all neighbors, this group was spread out from Canada to California.

Being on so regular an interval makes it almost seem deliberate (I know that no company would deliberately have lag as part of their product), just seems that that would make the lag easier to localize for the company responsible (either Turbine or their service provider).

[smatt]

There's so many differing possible causes for lag, some of which likely are occurring simultaneously, and even members of the same party could be experiencing lag for differing reasons. I'm sure that there are issues within the game engine itself that cause lag as well as the location of the game servers. As well as client side lag, individual ISP lag, internet hub lag, on and on and on. Also weather in certain parts to he country seem to contribute to lag. Primarily though, the worst of the overall lag seems to be related to game engine performance, in the high end content, within raid groups, with a lot of actions happening. It is likely that the game engine simply can't handle the huge amount of computations. Also, likely is that very busy instances IE: Shroud prime time with multiple groups running it at the same time play a roll, as the game engine might possible have a resource allocation limit in place.

In other words there are a lot of reasons for various kids of lag, I think it's funny that it's always Turbines fault. Yes, I think it's part of the cause, but not likely nearly all of it....

For us as players the best we can do is, if we want to lessen our lag on our side, have good well running systems with far more than the minimum requirements for the game. Have a good clean down AND uplink connection speeds with our individual ISP's. Make sure personal system resources are clean, not running extensive software,t hat places a higher demand on available RAM, clean-out internet temp files to remove those pesky tracking cookies that cause problems with uplink speeds.

Other than that, allow e can hope is that Turbine is working on further problem solving with the lag that is directly related to the game engine, and hardware on their side.

[smatt]

Can you point me to that? If they KNOW what the bug is and haven't fixed it, that casts this whole thing in a VERY different (VERY ugly) light. But I reserve judgement until I read more details.


It had to do with the way the the devils and orthons were teleporting... .. I seem to remember them saying it was hwo the mobs were deciding where to teleport to and that it was causing the engine to bog down.

So what is the point of all of this. If it's to work on individuals lag, and how to offer up some advice as to how people can mitigate it on their side, that's great. If it's to figure out what's going on Turbines side, well you will NEVER get enough information, or data to come to any reasonable conclusion IMO. Not meaning to swing a bat at you at all. We all want less lag, but Turbine is a corporation that guards things rather carefully, and rightfully so. So for you to obtain the information required to even have any real idea of how to solve the problems if it is infact something that can be remedied, well it's not going to happen....

[ArkoHighStar]

It really is unclear whether the servers were outsourced, we know the following, Turbine bought new hardware for DDO last fall, they also signed a lease for new office space. They also moved the servers into a new data center, it has never been stated that this datacenter is off site, just that the servers were moved to a new location. It is entirely possible that the datacenter is a new one inside Turbine's offices. We also do not know whether they have changed network providers, all we know is recently their vendors have been doing maintainence on their systems and that it could have caused an outage. All the recent outages had db rollbacks, and were brought down quickly to prevent further corruption, which says the staibility problems are related to db or storage issues.
The lag is more difficult to determine, as it could be internal network configuration, server configuration, thier AT&T lines to the internet (main pipes), could be code isues causing information delays.
The fact that all the server activity moves on while we get a huge lag spike tells me its is more a network issue either at the NIC card out or somewhere in between that and our PC. Given that they can easily monitor AT&T line quality my guess is somewhere in the datacenter network infratstructure things are getting bogged down both ways, the pipe is either overloaded with data because it is not wide/fast enough, or something is artificially making a big enough pipe act much slower than it should.

[Thanimal]

So what is the point of all of this.

To provide Turbine with data. I am a professional software developer, and while I have no expertise in network communications, the one thing I know for sure is that debugging is a LOT easier when you have detailed information about the problem. And I consider it "obvious" that they haven't isolated the problem yet, or they would have fixed it. (The alternatives to this assertion are too horrible to contemplate :))

Client-side lag is totally different and easily identifiable, since it only effects you. Obviously they cannot fix that.

[Gratch]

Well, we do know:

1) Moria was about to release
2) We had a maintenance "day" that crippled the servers in all quests for everyone for a few months...
3) Moria released
4) You'll soon be getting a hardware upgrade announced
5) Hardware upgrade, devil teleport fixes in patch, lag somewhat less
6) More server crashes and network problems and login server problems and Sarlona's always broke...
7) but getting slightly "better" now... *fingers crossed* *Knock on wood* *Leprechauns in captivity* *Hamsters fed* though we're still getting some lag filled instances (network or load balancing needs some more tuning or there's a server not operating on all processors).

The guess at the time, which seems about right based on events is that step 2 above was moving the DDO services to the new datacenter servers... and one that was "lacking" in either server power or network load capability. Sounds like we did get a hardware upgrade at that datacenter... and they did somewhat delay devils telporting... but they're still getting all the routing, load balancing issues figured out at the new datacenter area.

[Gratch]

Dup. - deleted.

[Tarrant]

I hold the possibly generous belief that if fixing the lag were easy, Turbine would have already done so.

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

[DagazUlf]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

Whatever. "Deeds not words" is appropriate here.

[weyoun]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

If you would make a sticky about this and maybe give us an update occasionally it might calm the storm here, might. Its really bad. If we didn't like playing so much we wouldn't care, but we love the game and want to know its in good health. So please more communication.

[vtecfiend99]

Whatever. "Deeds not words" is appropriate here.

Agreed. There have benn so many times over the last year that this tripe has been spewed at us. We have lag fixes coming! This should eliminate a bit of the lag! All that has happened is the lag has gotten worse and worse and worse until at times some quests in the game are unplayable. And, like so many other issues, the complaints on the forums have to reach a near riot befroe we can even get a single PEEP from someone at turbine about it.




The issue here is not just the lag, the issue with this company is some of the worst customer service I have ever seen or heard of. Simply appallingly bad. Tolero included, we have a 40,000+ view thread of complete hate about respec mechanisms, not word one. But someone posts a thread about vin diesel and we get like, 7 responses in an hour. Sheeeeeeesh.......

[kingfisher]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

thanks for sacking up and acknowledging the problem, you've got guts. like weyoun said, a sticky and updates would go along way. i used to laugh about the lag and the crazy runs it created but its getting to the point that i am running out of jokes. if you are trying to fix it, it let us know, no matter how small the info, if you are trying people will give you a chance to succeed.

[kingfisher]

Whatever. "Deeds not words" is appropriate here.

no. if its a lengthy fix, then its better to have 'words' while we wait for 'deeds'

[Gratch]

If you would make a sticky about this and maybe give us an update occasionally it might calm the storm here, might.

Tarrant: A quick word by SIG with a few specifics might calm us a bit. I've noticed the lag off/lag on switch seems to get flipped by Monday maintenance days and less so by patch/client updates days. So I'm guessing this is either processor loading balancing issues, data storage access issues or network loading. Pretty please with very very slowly falling sprinkles on top for a few words from SIG?

[Missing_Minds]

Whatever. "Deeds not words" is appropriate here.

Asinine to insulting commentary that has been flowing freely will keep any such information from being flowed back to us.

So by all means... keep killing a flow of feed back.

[Strakeln]

I hold the possibly generous belief that if fixing the lag were easy, Turbine would have already done so.In this case, fixing the lag is easy.

Turbine needs to undo whatever they did a few months back during their "mystery update" that caused all of this to start. Most seem to suspect that they removed hardware from DDO. If this is the case, then the problem isn't that they can't fix the lag... it's that they refuse to, because some exec somewhere decided they needed that hardware for something else, and won't be changing his mind anytime soon(tm).

[Baron]

I suspect most of the "Lag" is really latency. Will be harder to fix.

Network Engineer by trade....

[vtecfiend99]

Asinine to insulting commentary that has been flowing freely will keep any such information from being flowed back to us.

So by all means... keep killing a flow of feed back.

:rolleyes: Yeah, just like all lag is a product of our sub-par computers, all lack of feedback is also our fault.

[kingfisher]

:rolleyes: Yeah, just like all lag is a product of our sub-par computers, all lack of feedback is also our fault.


********, no dev has ever said to upgrade your puter that i can recall, thos comments are made by other players who are lucky enough to not have horrible lag. the comments mm was talking about were made towards a dev who finally acknowledged the lag issue and asked for some patience to fix it. those types of comments are idiotic and do not encourage further commentary by the devs, so yeah in this case they hurt feedback

[Ryss]

Let me chime in here.

I'm playing from EU and I'm lagging just like everybody else. This is clearly a server sided issue because all players in the party lag at the same time. It has nothing to do with FPS. My comp is one of the beefiest of them all :).

[Tolero]

I can do without the unnecessary flaming of users who are experiencing legitimate lag issues. I'm sure the Cube has no problem with an early St Patty's feast.

[baylensman]

in the spirit of giving specific info to help isolate and identify the problem.

I have three accounts, mom, dad, son. Only two active right now, mom working early shift so hits the sack during our playtime. until recently had the game running on three very different systems, system one is a new highend three coore MAD bassed machine with 3 gb mem and a BF video card by n-videa with its own power supply and 2gb mem (boys computer as he is studing game development in college) my tower an older 2.3gh with 1 meg and 128mb pny video card and a lap top with 1.7 gh and 1 mb memory and od board graphics. at times all machines experienced the same lag even though in different instances. Could be local network access except that other players complained of lag spike then also. Current set up is the boys system and my new system running 3 core amd 3gs mem and 786mb nvidea graphcis. new router and cable modem have upgarded to bussness class service with cable CO. Lag is not on our side, boy can play WOW (Ithink thats another MMO) while im in game i lag he doesn't. we play together both have lag at same times when in the same instance. when we are not in the same instance sometimes the lag is only on one of us at other times on both. During most lag spikes others complain of lag issues also, I frequently however outrun some of the group because they are laggy while i'm not. But when I get lag spikes so does everyone in the group.

[Arianrhod]

OK, well...in the spirit of the OP (at least i hope it is), here's what my group has experienced lately:

In general, very little lag - occasional little spikes where everyone will freeze in place for a second or two, then resume normal play. One of the players in the group (in PA - we have 2 in PA, 2 in MD & 1 in CA) has frequent, noticeable lag that affects gameplay; the other player in the same house does not - presumably her computer and not their ISP is the culprit here. Once recently she also had severe VC lag (during a run of Tomb of the Shadow Knight). Notably, this same player until recently could not enter the Necropolis without having severe graphics issues that made the game completely unplayable. I think she either changed some settings or upgraded a video card to deal with that issue. All her lag issues have been since that change.

We don't zerg - we go slow and kill everything. We also don't use a lot of big flashy magical effects (the occasional web or firewall, haven't got cloudkill or acid fog yet, and never did use hypnotic pattern). [edit - oh, yeah, one other thing - we play on Khyber]

No idea whether any of this will be useful, but no harm in providing info.

[miceelf88]

I have to admit I am happliy surprised that the devs acknowledged the issue. I am with DagazUlf, but acknowledging the problem is a Good Thing.

We'll never know the real reason for the lag, I suspect, although I think the working theory is the best guess.

(sigh). Frackin suits.

[ahpook]

GREAT question! The information could be important to debuggers. This was actually a slow run -- my guild was teaching me the quest and were stopping to kill everything. I don't think we left anything alive or trailed any monsters around.


Perhaps though if the other parties that you are sharing your server with is doing a lot of non-killing, they are using up CPU cycles and causing your instance to suffer as well. It could be there load balancing doesn't do a good job accounting for the potential CPU overload of certain play styles?

of course, this is only 1 of about a million possibilities.

[Bartikus]

I suspect most of the "Lag" is really latency. Will be harder to fix.

Network Engineer by trade....

It might be latency, but not based on what the client meter is telling me (us). My network latency floats from 45-70 ms, and does not coorelate to the lag experienced in game. It could be latency between frontend and backend servers, but my DDO client is not reporting huge spikes (or loss) in latency.
Some of my worst lag comes when I have the lowest latency ~50ms... then other times its smooth at 70 ms.... even smooth at 150 ms.....

[Hafeal]

We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away.

Tarrant thanks for your time to respond.

I took your statement to mean that you have at least knowledge of "what" is causing lag for so many players. Is there anything there that you can share? Are there certain game behaviors, for example, which help contribute to lag?

[Beherit_Baphomar]

Just incase you dont know, Turbine, but it would prolly be a good idea to stop all work on MOD9 and fix The Lag.

If you drop MOD9 and ****'s still acting this way it will be one of the worst move's you've ever made....and y'all have made some pretty bad move's.

[Gol]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.
Ok, so it's not flip a switch. It's flip a switch (shut down), unplug, carry back to the older non-sucktastic data center, plug in, flip switch back on.

Ever since you switched data centers, prime-time raiding has been almost impossible.

[Jefro]

Some of the shroud lag is contribute to combat log, mostly with alot of dual attacking rangers in the group of 12. Could there be way for combat be handled more separately from other data loads?

[Baron]

It might be latency, but not based on what the client meter is telling me (us). My network latency floats from 45-70 ms, and does not coorelate to the lag experienced in game. It could be latency between frontend and backend servers, but my DDO client is not reporting huge spikes (or loss) in latency.
Some of my worst lag comes when I have the lowest latency ~50ms... then other times its smooth at 70 ms.... even smooth at 150 ms.....


I will make a diagram and post it. The latency issues I am concerned with are at the Turbine end. Real latency will not record on your NIC. ISP lag will.

A short swift example below and then I must head home. I will explain better tonight.

Lets assume all the players have just replaced thier machines and all the players are using the same type computer with all the same options, same operating system and the same programs loaded in memory. Wifes have been booted off the network...Next door neighbors have been evicted if you are on a cable connection....

Lets look at network connections. each - is up to 10 milliseconds

Player1--ISP-Major Backbone1----5th Backbone----TurbineISP (Fibre to home)
Player2--ISP---Major Backbone1---5th Backbone----TurbineISP (Work T1)
Player3--ISP-----Major Backbone2----5th Backbone----TurbineISP(Cable20mbs)
Player4--ISP---------Major Backbone2----5th Backbone----TurbineISP(DSL10mbs)
Player5--ISP------------Major Backbone3----6th Backbone----TurbineISP(ISDN)
Player6--ISP---------------Major Backbone4----6th Backbone----TurbineISP(Sat)

Above is where ISP lag comes from. Player 4 hits the turbine ISP 80 milliseconds later than player 1. Player 2 up to 40 ms later than player 1. This is not a big a deal as it looks. The turbine software does take this into account. Players 5 and 6 will have some problems but can still play. Notice that once you get to a major backbone things are pretty much the same. This is real network provider lag and it show when you hover over the icon on your screen. There are two other things which need to be explained which are not shown by hovering over the icon.

I will get to that later. The () at the end is what type of connection each player has to his home or workplace if playing from there, perish the thought.

[Yuhjn]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.


What hasnt been said that needs to be is that lag would be greatly reduced by an investment in hardware. There is a limit to what software can do, and thinking that you can just keep optimizing code forever to "erradicate" lag is absurd.

Lag will be "fixed" when Turbine gives us more hardware.

[ArkoHighStar]

What hasnt been said that needs to be is that lag would be greatly reduced by an investment in hardware. There is a limit to what software can do, and thinking that you can just keep optimizing code forever to "erradicate" lag is absurd.

Lag will be "fixed" when Turbine gives us more hardware.

The problem is we do not know where the lag is originating, you can have massive hardware, but have incorrectly cofigured NICS' or routers throughput can be attrocious given the right conditions

[Turial]

Just remembered something else that might matter: During this entire experience, there was never any noticeable voice lag -- we continued to be able to converse about how wonderful the lag was.

In my experience, whenever I've experienced obvious client-side lag, my voice has also lagged.

Correct. That is the big kicker to see if you have client side lag or if it is server side lag. The likely hood of two separate, both geologically and hardware wise, servers hitting the can at the same time is pretty low.

So if you lag and have voice chat lag then it is a client problem.

If you lag and there is no voice chat lag then it may be a server lag issue.

[stockwizard5]

I think this (copied from other thread) is an indication that the problem cannot be explained by latency alone ...

Monday 10:30 Thelanis - Visions - Just stop dead, frozen, no spells, lag mixed with delayed spells, studder lag (which is the best it got).

Interesting note - during the duration of the first bad freeze (1st 4 Orthons) we noticed normal sent data but the recv data was holding above 5000 Bps for the duration.

That can't be good :eek:

[Beherit_Baphomar]

I think this (copied from other thread) is an indication that the problem cannot be explained by latency alone ...

Monday 10:30 Thelanis - Visions - Just stop dead, frozen, no spells, lag mixed with delayed spells, studder lag (which is the best it got).

Interesting note - during the duration of the first bad freeze (1st 4 Orthons) we noticed normal sent data but the recv data was holding above 5000 Bps for the duration.

That can't be good :eek:

5000?? Hell, I'll get 14k Bps in Shroud with 94.5% loss....but latency never changes.

[Raithe]

I got the really bad lag for the first time yesterday, and after reading the OP, I'm inclined to think its a hardware problem.

The lag seems isolated to an instance. Once you recall out and create a new instance, the lag can completely disappear (and I would recommend doing just that anytime a significant amount of lag is encountered.) I was in a horrible instance of Rainbow in the Dark, we wiped, and when we went back in just 10 or so minutes later we completed without so much as a peep of lag as far as I could tell. Also, certain public instances on Thelanis are contaminated, and it becomes obvious running around which ones to avoid as much as possible (right now House Deneith is bad).

I don't know how instances get divided up on their hardware/processors, but I'm thinking that some of the hardware that runs Thelanis is not working efficiently.

[Yuhjn]

Correct. That is the big kicker to see if you have client side lag or if it is server side lag. The likely hood of two separate, both geologically and hardware wise, servers hitting the can at the same time is pretty low.

So if you lag and have voice chat lag then it is a client problem.

If you lag and there is no voice chat lag then it may be a server lag issue.


Incorrect since voice chat is a completely different server.

[Yuhjn]

I got the really bad lag for the first time yesterday, and after reading the OP, I'm inclined to think its a hardware problem.

The lag seems isolated to an instance. Once you recall out and create a new instance, the lag can completely disappear (and I would recommend doing just that anytime a significant amount of lag is encountered.) I was in a horrible instance of Rainbow in the Dark, we wiped, and when we went back in just 10 or so minutes later we completed without so much as a peep of lag as far as I could tell. Also, certain public instances on Thelanis are contaminated, and it becomes obvious running around which ones to avoid as much as possible (right now House Deneith is bad).

I don't know how instances get divided up on their hardware/processors, but I'm thinking that some of the hardware that runs Thelanis is not working efficiently.

If you are moved to the right server on the cluster you wont have lag. Wrong server and you will. Solution is more servers in the cluster so there are less instances per server.

[Mercules]

Perhaps though if the other parties that you are sharing your server with is doing a lot of non-killing, they are using up CPU cycles and causing your instance to suffer as well. It could be there load balancing doesn't do a good job accounting for the potential CPU overload of certain play styles?

of course, this is only 1 of about a million possibilities.

This was my immediate thought. I wonder if Instances are farmed out to particular servers. "Thelonis - Shroud instance goes to Server X, Water Works to server Y." Get a popular instance with a lot of people running it, get some of them saying, "Run past and we will Firewall at the end." and we suddenly get a lot of processes going on for a particular blade. Everyone with an instance on that blade gets lagged as it catches up.


I remember the days of duo-ing the Catacombs and running past mobs in Dryden's Tomb and ending up rerunning that quest because after about 100 spawns it lagged so badly we couldn't move, only die and recall.

[Quanefel]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.


Thanks for the clairity. Most of us know ya'll are doing what you can. Good luck.

[Yuhjn]

This was my immediate thought. I wonder if Instances are farmed out to particular servers. "Thelonis - Shroud instance goes to Server X, Water Works to server Y." Get a popular instance with a lot of people running it, get some of them saying, "Run past and we will Firewall at the end." and we suddenly get a lot of processes going on for a particular blade. Everyone with an instance on that blade gets lagged as it catches up.


I remember the days of duo-ing the Catacombs and running past mobs in Dryden's Tomb and ending up rerunning that quest because after about 100 spawns it lagged so badly we couldn't move, only die and recall.


No, the farming out of instances is based on current server load on each of the servers in your world's cluster. So your "server", ie, Thelonis is served by a cluster of physical servers. Then a new instance is created, there would be a (probably simple) algoritm that looks for a low load server on the cluster and sends the instance there.

How this load balancing is done is anyone's guess. It's possible that 12 people standing in an instance waiting for buffs creates low traffic and shows low load on the server, until they start banging portals, by which time it might be too late... or that could be completely wrong and there just are not enough server cycles in the cluster to go around. Only Turbine knows how their clusters are load balanced.

But I'm telling you, the fix is double the hardware. To which business probably just says "not in the budget, sorry, do what you can in software". To which the PR people tell you, the player, that "we're working on it but it's really hard". And they probably are, but not very hard because there isnt a whole lot they can do in software at this point. They've already dumbed down the AI to the point it can hardly find you anymore... more "optimizations" are just going to be bad for gameplay more than lag is. /shurg.

[DoctorWhofan]

OK, well...in the spirit of the OP (at least i hope it is), here's what my group has experienced lately:

In general, very little lag - occasional little spikes where everyone will freeze in place for a second or two, then resume normal play. One of the players in the group (in PA - we have 2 in PA, 2 in MD & 1 in CA) has frequent, noticeable lag that affects gameplay; the other player in the same house does not - presumably her computer and not their ISP is the culprit here. Once recently she also had severe VC lag (during a run of Tomb of the Shadow Knight). Notably, this same player until recently could not enter the Necropolis without having severe graphics issues that made the game completely unplayable. I think she either changed some settings or upgraded a video card to deal with that issue. All her lag issues have been since that change.

We don't zerg - we go slow and kill everything. We also don't use a lot of big flashy magical effects (the occasional web or firewall, haven't got cloudkill or acid fog yet, and never did use hypnotic pattern). [edit - oh, yeah, one other thing - we play on Khyber]

No idea whether any of this will be useful, but no harm in providing info.

I think there is some justification of this. I recently ran a quest with twice wit htwo different groups. One Zerged and the other did not. We had less and less lag (not totally gone, mind you) in the non-zerging group.

HOWEVER, the lag that everyone seems to put everyone in synch (ie Part 1 shroud) is still there.

Hope that helps.

[Oxvon]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

Bah, programming is easy (http://www.youtube.com/watch?v=spSGNMJhWV0)

[jakeelala]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

I'm sorry. Are these some sort of state secrets? Is this privileged information? Would it be embarrassing somehow to the overlords if it was revealed the nature of the problem and your proposed solution?

You know the world has borne evidence in the last 10 years to the power of the social network...Wikipedia, THE OPEN SOURCE MOVEMENT, twitter, etc. Over and above the fact that if you just gave your slavering players a few choice morsels on something as mundane and boring and as your network stats and broken back planes they would sit and wag their tails like dogs...there's an off chance that some of them, given enough actual information, may actually be able to offer helpful advice.

I understand if you can't release upcoming feature details. But persistent lag issues within the game (for what has been literally years) really should not be something you withhold details and specifics on. It's:

1. An easy way to keep your customers informed and engaged. That's worth more than all your new subscriptions in a month.

2. Trivial, non-competitive, non-game specific information that does nothing to your control information flow around features, enhancements, and development work

3. Potentially a FREE way to get some very good advice, or at least other opinions on what's wrong and ways to look at it.

I work in the professional services industries. I know a thing or two about keeping clients happy, covering my ass, and controlling messaging and expectations. You guys need to start thinking more like consultants with clients, and start aiming for low hanging fruit for big returns on little efforts.

Letting your most devoted and interested gamers into some of the technical aspects of what they're frustrated with in-game is 100% win-win. It's easy, it's free, and it will go an enormously long way in building trust and goodwill.

Please, I beg, I DARE you to come up with a reason why I'm not right.

THE CUSTOMER IS ALWAYS RIGHT. And I don't mean me, I mean the forum members who are begging for a glimpse into just what has stumped you guys for so long, and affects their game play so disproportionately.

-Jake

[DoctorWhofan]

Also, it took over an 1/2 hour to do the Kortho's Island mayor quest. Stuck on the ladder. Was cleaning it out on my own on normal. mimiual buffs, if any. I actually recorded some of the lag. THEN I ge get out to the slayer's part-no lag. Do the Cannith Cannery quest in the same instance, no problems except for some step lag at the beginning. THis was Friday.

[greystone306]

The issue here is not just the lag, the issue with this company is some of the worst customer service I have ever seen or heard of. Simply appallingly bad. Tolero included, we have a 40,000+ view thread of complete hate about respec mechanisms, not word one. But someone posts a thread about vin diesel and we get like, 7 responses in an hour. Sheeeeeeesh.......
How the in the Heck is the amount of responses to a thread on the Forums a Custumer Service issue is beyond me..There may be potential Customer Service issues, but forum responses is not a way to Gauge them IMHO.... Now if it was 7 responses by Turbine Employees in an hour to the vin diesel thread that might be a problem.. Maybe nobody cares about the respec mechanisms or replied in 1 of the thousand other threads going on about respec mechs and just viewed your favorite 1..
and to the person about Deeds not words.. IT Deeds take time usually.. except for the DEED of Turbine just saying **** ** and deciding to shut it down and turn off the DDO servers permanently.. Also, if you haven't noticed the Economy right now is not the greatest, and people are cutting staff/costcutting across the board...its possible the needed Suppliers of equipment are also having production issues..

[greystone306]

THE CUSTOMER IS ALWAYS RIGHT. And I don't mean me, I mean the forum members who are begging for a glimpse into just what has stumped you guys for so long, and affects their game play so disproportionately.

-Jake
www.notalwaysright.com
Enough Said

[Fubars_Snafu]

It's not latency.. I, on avg, have a latency of 20 ms with 0% loss of packets.. I am about 15 miloes as the crow flies from turbine's HQ... this is definitely a server issue .. one thing it may be is the server trying to adjust the gamestate so everyone is on equal footing ( a way to "equalize" latency for all players in an instance) or there are not enough servers in the cluster to handle the work load and the game is constantly playing catch up... that's my two coppers

[jakeelala]

www.notalwaysright.com
Enough Said

there's a difference between a customer being uninformed (your link) and a customer being right in asking for more information in response to inadequate level of service (turbine customers on this forum)

[Vordax]

THE CUSTOMER IS ALWAYS RIGHT.

I would have to disagree with this.. The customer MAY be right. Some businesses will then pretend that the customer is "ALWAYS" right. These businesses usually don't last long though.

Vordax

[Dark_Helmet]

In this case, fixing the lag is easy.

Turbine needs to undo whatever they did a few months back during their "mystery update" that caused all of this to start. Most seem to suspect that they removed hardware from DDO. If this is the case, then the problem isn't that they can't fix the lag... it's that they refuse to, because some exec somewhere decided they needed that hardware for something else, and won't be changing his mind anytime soon(tm).

Not to sound insulting, but I see advice like this from people who know nothing about computers or data centers.
The issues are complex and while I don't know their current configuration, I do know that it isn't as simple as "put it back":
A lot of commercial companies depend on COTS (Commercial off the shelf software)and have no control. The companies who make that software put out patches to fix bugs and security issues (mostly security patches are badly written and the crux of the problems as the quickest fix gets out to prevent major damage).
So, they have some sort of Server software (Such as Windows or Unix), Clustering software (sometimes part of the OS), Database (such as Oracle, MS-SQL), hardware drivers (network agregation, optimization, SAN, RAID, backups, etc). Each manufacture of those pieces of software released updates, patches, security fixes (those are typically installed during the maintenance windows that aren't improvements to DDO itself).
Doing "roll-backs" don't always fully uninstall, leaving artifacts that can still cause lag (some DLL refused to leave the system and you would have to track it down if you didn't do a file level dump - yes, I have experienced that).
Put on top of that that Turbine may have outsource the running of the Data center and their risk is potentially higher and hard if not impossible to fix (something that bean counters never factor into the "cost saving" of outsourcing, but I digress). The only way people try to fix these issues is to "throw hardware at it", but the original, crappy code is still there taking up precious cycles preventing the system from ever running at normal loads again.

Evidently, DDO doesn't have any real Q/A test environment (something like a full replication of deployed hardware and test tools such as LoadRunner with a stress test replay that replicates user behaviors that was recorded during one of the heavily occupied timeframes). If they did, they would know when some maintenance change to software would impact the live systems and be able to point directly to the company responsible.

Only way to fix it is to restart from scratch with a proper system... which we won't see (just hand me downs from LOTRO).

[Yaga_Nub]

Agreed. There have benn so many times over the last year that this tripe has been spewed at us. We have lag fixes coming! This should eliminate a bit of the lag! All that has happened is the lag has gotten worse and worse and worse until at times some quests in the game are unplayable. And, like so many other issues, the complaints on the forums have to reach a near riot befroe we can even get a single PEEP from someone at turbine about it.

Yeah and even then they probably shouldn't have responded because the complaint/"issue"/thread wasn't worth responding to.


The issue here is not just the lag, the issue with this company is some of the worst customer service I have ever seen or heard of. Simply appallingly bad. Tolero included, we have a 40,000+ view thread of complete hate about respec mechanisms, not word one. But someone posts a thread about vin diesel and we get like, 7 responses in an hour. Sheeeeeeesh.......

It's not about customer service. This forum allows for interaction but does not guarantee ANY level of interaction with someone from Turbine. There have been many times that I've had an issue, submitted a bug report and gotten more than just the automatic response. Try that if you want a response.

If you're really ****ed, go for a walk. Go shag your boyfriend/girlfriend. Get away from the computer for a while until it doesn't bother you any more. It's not worth getting upset over. Well, maybe the Vin Diesel thing but Tolero is a woman isn't she?

[skyking613]

Was in the shroud last night. Lag right from the start, as soon as we went after the trash at the start. Someone suggested to turn off all combat feedback. I ignored him. But the lag did seem to improve a bit a couple seconds later. On about the 5th-6th portal lag came back a bit not as bad as the start bad still bad. So i thought what the hell wont hurt to try right. Turned it all off (Boy did i feel nekkid). But geusse what it improved. Portals were going down at an insane speed. When done with part one the walk up the stairs where it always lags, I didnt even hicup. Part 3 also went without much lag. All week the shroud has been absolutly horible with lag. I havent seen a shroud go this smooth in weeks. Was it a coincidence? Or does the server lag because it needs to send so many of those combat logs to the client. I can think of no other quest where there is as much combat text as in the shroud. What if we all try to turn off the combat text this weekend and see if this improves the lag across the server this weekend. As someone who troubleshoots all day for his job I know sometimes the client has to get involved. So why dont we help them out and do this for the weekend. And also announce to the party to do this at the start of the shroud. Some people will just blow it off, but if we get at least half of them to do it we may see an improvement. If we dont see any improvement, Well as long as enough people tried then we will know what it isnt.

[Missing_Minds]

Please, I beg, I DARE you to come up with a reason why I'm not right.

Because you have NO "rights" here, you have "privileges". Learn the difference.

[Vardak]

The lag is rather obvious... someone needs to call Bob, owner and operator of "Bob's Server Farm" and politely ask him to quite hosting all those p0rn sites.

[kingfisher]

Was in the shroud last night. Lag right from the start, as soon as we went after the trash at the start. Someone suggested to turn off all combat feedback. I ignored him. But the lag did seem to improve a bit a couple seconds later. On about the 5th-6th portal lag came back a bit not as bad as the start bad still bad. So i thought what the hell wont hurt to try right. Turned it all off (Boy did i feel nekkid). But geusse what it improved. Portals were going down at an insane speed. When done with part one the walk up the stairs where it always lags, I didnt even hicup. Part 3 also went without much lag. All week the shroud has been absolutly horible with lag. I havent seen a shroud go this smooth in weeks. Was it a coincidence? Or does the server lag because it needs to send so many of those combat logs to the client. I can think of no other quest where there is as much combat text as in the shroud. What if we all try to turn off the combat text this weekend and see if this improves the lag across the server this weekend. As someone who troubleshoots all day for his job I know sometimes the client has to get involved. So why dont we help them out and do this for the weekend. And also announce to the party to do this at the start of the shroud. Some people will just blow it off, but if we get at least half of them to do it we may see an improvement. If we dont see any improvement, Well as long as enough people tried then we will know what it isnt.


interesting indeed. worth a try, i mean you already know you are kicking the portals ass dont you?

[Missing_Minds]

I spent the last few days poking a few devs to see how we, the players, could help with data gathering that they would find useful. This is what we came up with.

http://forums.ddo.com/showthread.php?t=175724

So if you want to help, give them data. Thanks all.

[Thanimal]

I spent the last few days poking a few devs to see how we, the players, could help with data gathering that they would find useful. This is what we came up with.

http://forums.ddo.com/showthread.php?t=175724

So if you want to help, give them data. Thanks all.

AWESOME! I'll try to gather that data the next time I experience ugly lag and post it on that new thread.

Also, anyone have ideas for how to get the word out on "No Combat Log Weekend"? :) Oh, and: Can someone remind me how to turn it off!?

[skyking613]

Im at work right now so can't remember which option it was under off hand. It was towards the bottom of one of them though and there were like 15+ to turn off.

[Missing_Minds]

AWESOME! I'll try to gather that data the next time I experience ugly lag and post it on that new thread.

Also, anyone have ideas for how to get the word out on "No Combat Log Weekend"? :) Oh, and: Can someone remind me how to turn it off!?

No combat log weekend? I don't think you can turn off the combat log as I'm pretty certain the client will always receive that data. Other wise the simple method is to just remove that chat tab so that it no longer exists.

[skyking613]

With the chat tab removed it may still be sending the data to the client.

[skyking613]

Now that i think about it if you have any combat window (not even active) it will update it and you can click on it to see it after the fact. But what if there is none to begin with i think when you add it, it will only show you from that point on. I still think it would be easier to turn it off. Just in case someone has it turned on one of the tabs but forgets about it. It will also make it simpler to turn back on when we want to.

[The_Phenx]

12 Man shroud.

approx 4-6 seccond delay between casting time and effect time. Last evening at 10-12 EST.

It seems to peak in several areas... portal beat downs... harry fights.

Tried ultra high and then low graphics... saw no change in frame rate. Internet connection speed at that time was 8.8 MBS 35 ms latency. (using a 3rd party speed tester) No other programs running on my comp... xp - 2 gig ram dx 9 amd 3000...256mb graphics card

Its like the server has issues processing all the twf blows, and has to que everything and it gets all jammed up.

[skyking613]

Also, anyone have ideas for how to get the word out on "No Combat Log Weekend"? :) Oh, and: Can someone remind me how to turn it off!?


Maybe the Moderators can help us out, a sticky or poll or something.

[skyking613]

The lag is rather obvious... someone needs to call Bob, owner and operator of "Bob's Server Farm" and politely ask him to quite hosting all those p0rn sites.

Then what would I do when not playing DDO :D

[Renegade66]

You hit the nail on the head.

Guys, we know lag makes the game frustrating to play. We take a lot of pride in this game, and want you all to have the ultimate DDO experience. We want the lag eradicated. We're already working on fixes for lag issues, but unfortunately it is not as simple as flipping a switch and making it go away. These things take time, and we're going to need a little more.

Hang in there, we're doing everything we can, and we appreciate your patience.

Is there anything the player base can do in the interim to reduce lag (e.g. turn off sound, reduce vid settings, not use aoe spells, etc.)?

[Baron]

Is there anything the player base can do in the interim to reduce lag (e.g. turn off sound, reduce vid settings, not use aoe spells, etc.)?


Sure, turn off the game, cancel the account, not sure Turbine will get the message :p

Just kidding:rolleyes:

[skyking613]

Is there anything the player base can do in the interim to reduce lag (e.g. turn off sound, reduce vid settings, not use aoe spells, etc.)?


Well, if you read back a few posts, I posted about the combat logs. And would like to see if we can somehow get as many people that we can to turn off all their combat text for the weekend to see if it improves.

[Renegade66]

Well, if you read back a few posts, I posted about the combat logs. And would like to see if we can somehow get as many people that we can to turn off all their combat text for the weekend to see if it improves.

I didn't read through all, but great idea. I'll do my part on New Xoriat.

[Missing_Minds]

With the chat tab removed it may still be sending the data to the client.

Same thing if you just deselect it in the UI. The client will ALWAYS be sent the data. The client just filters the data per your selections. These are settings stored at your PC, not server side.

Please note, however, I'm not saying that the combat log data isn't slowing the servers down. It very well may be adding to the issue, however, it isn't something that players are really going to be able to test in any meaningful way.

[jakeelala]

Because you have NO "rights" here, you have "privileges". Learn the difference.

"rights" and "right" are homonyms. That means they sound and look the same, but they mean different things contextually.

I never purported to have "rights" to the information. I purported to present superior logic for which Turbine cannot deny reasonably.

I'm sorry if you have trouble reading.

[skyking613]

OK, To turn off "combat feedback"

Options/UI Settings

It is a little more than halfway down. There is a bunch to check

[adamkatt]

I cant even stomach the game atm.. I will wait for official word on the lag being fixed on the forums, Or in the launch pad.

Steele /AFK!


PS

FIX THE LAG!@#$%^&*()

[Missing_Minds]

"rights" and "right" are homonyms. That means they sound and look the same, but they mean different things contextually.

I never purported to have "rights" to the information. I purported to present superior logic for which Turbine cannot deny reasonably.

I'm sorry if you have trouble reading.

Your purported "superior" logic is just an ego screaming wishing to be heard. Should Turbine actually wish for help they would tell us.

Fact: they have. They want us to continue to post ideas here, and details in my other thread. This is the help they desire from us. For them to post more could get them into legal issues and/or possibly open themselves up for espionage.

As much as you are sorry for my lack of reading comprehension, I pity you for feeling the need to act as you are. It only makes the devs and mods wish to correspond with us less. Have a nice day.

[Missing_Minds]

OK, To turn off "combat feedback"

Options/UI Settings

It is a little more than halfway down. There is a bunch to check

How well did it actually end up working in your groups?

[vtecfiend99]

How well did it actually end up working in your groups?

Well, for Argo lag? it works very well. This eliminates all kinds of fun lag, like the lag from casting masters touch on yourself, and portal dps lag.

However, having recently played on thelanis i can tell you that it doesnt even BEGIN to touch the lag there. Perhaps thelanis has its own unique set of problems

[skyking613]

Ran shroud just one time last night and the lag was not bad at all. Some friends ran it before that run and still said the lag was better. I wish there was a way of telling how many people followed that advice.

[Missing_Minds]

Ran shroud just one time last night and the lag was not bad at all. Some friends ran it before that run and still said the lag was better. I wish there was a way of telling how many people followed that advice.

Well... in your player profile, not the bio, but that other one that gives you a few characters to put stuff up for all to see (such as looking for group stuff), you could put in there to do that with your UI to help reduce lag in shroud and such. We may not be able to see such results posted on the forums, BUT, it will spread the word around to the population that does not come to the forums.

[Missing_Minds]

Sat a new thing happened to me. Lag killed me.. and near as I can tell, literally.

Korthos Island, explorer zone. I just got done talking with the first quest giver out there that suggest you jump into the water for the quest entrance. Only my wife is in the party with me, no hirelings at all. A few seconds after I get done talking to the guy, I jump for the water. I'm sort of goofing off, so I jump in backwards.

Well... at the apex of my height, lag hits, and I hand there. Next thing I know I'm incapped at -4 because I took 59 points of damage from falling. Note, I'm still stuck in the freaking air! A few moments later, I finally fall, and where do I land.. in water. I never hit "earth"/ground/land any where. Not even that sewer drain.

Yes, combat log said falling damage... even though I was in mid air.

[skyking613]

Sat a new thing happened to me. Lag killed me.. and near as I can tell, literally.

Korthos Island, explorer zone. I just got done talking with the first quest giver out there that suggest you jump into the water for the quest entrance. Only my wife is in the party with me, no hirelings at all. A few seconds after I get done talking to the guy, I jump for the water. I'm sort of goofing off, so I jump in backwards.

Well... at the apex of my height, lag hits, and I hand there. Next thing I know I'm incapped at -4 because I took 59 points of damage from falling. Note, I'm still stuck in the freaking air! A few moments later, I finally fall, and where do I land.. in water. I never hit "earth"/ground/land any where. Not even that sewer drain.

Yes, combat log said falling damage... even though I was in mid air.

I've had the same thing happen to me in the same exact way, in the same exact spot. Weird.

[Missing_Minds]

I've had the same thing happen to me in the same exact way, in the same exact spot. Weird.

*sigh* do you think they have an instant fall spot like they do in the PoP boss room? I've ran over that spot twice. Sudden instant death from falling for walking over one spot in particular. Never did find the exact loc of it.

[Missing_Minds]

Something I've been noticing with the lag.

When it hits (and i've been running lowbie stuff, so it hasn't been consistent. Just hits randomly then vanishes. Normally 2-3 seconds at a time.)

my received Bps goes from a normal 200-400 to 1500-3000.

[crschoen]

Just to add my 2 cents, as I've been struggling to figure out the lag myself. It seems like in my expience certain area effects that effect multiple mobs simultaneously can often cause the server lag: Otto's Sphere of Dancing, Blade Barrier, and Intimidation actions come to mind. I personally think it's tied to the pathing and aggro calculations, and the many permutations involved. The intimidation action causes an area effect re-pathing and re-aggro on surrounding mobs. Same with the blade barrier. For whatever reason, the game has trouble re-calculating those things simultaneously and everyone in the instance/quest gets hit with a lag. The problem is the more people in the instance/quest, the more re-calculation possibilities are required. If you think about it like this:

- Each mob has an aggro target, and a path to get to that target.
- When you are soloing a quest the calculation is easy there is only 1 such target and 1 such path.
- For each person you add to the quest that goes up 1, so for a 6 person quest there are 6 possible aggro targets and 6 paths for each mob. For a 12-person raid there are 12 possible aggro targets and therefore 12 possible paths to get to that target.
- You can see how adding more mobs to a 12 person quest causes these possible permutations to go up very fast. The game also has to show this re-pathing (graphically) to every player simultaneously.


My workaround lately has been to use spells that do not require this re-pathing re-aggro calculation by the game. For instance, the Web spell, Solid Fog, and Symbol of Stunning.

[Allasar]

Has anyone tried using the LagMeter application from EndLagNow.org? I haven't, but it might be worth a try. Note that I take no responsibility for the software or what it may do to your machine, etc, but it might provide some valuable information both to the community and to the Devs.

http://www.endlagnow.org/ELN/TakeAction_LagMeter.aspx

[miceelf88]

Okay, seriously, it's been two weeks since any dev said anything about the lag. Progress report? update? Still working on it? Given up? Will we be able to play mod 9 content when it comes out?