Crippling lag...

[steelblueskies]

The bags are one issue , just keep autocollect on for an ingredient bag and try rapidly buying 100 stacks of crafting dissolver .
Then when that finally catches up disable autocollect and do the same with the items going to inventory and note he huge time difference to complete.

CrAfting post hotfix has a new delay on recipes coming up as well when on first uses a machine.

Seems like server delays handling the actions.

I'd also like to note we seem to be talking at cross purposes.

There's client lags based on fps drops or spikiness. These seem tied to certain graphical effects, especially in activity concentrated encounters like raids.

Then there's server decides to forget to send you any traffic or responses leading to hard pauses or connection to chat server lost. This type may or may not be picked up by the games connection indicator claiming traffic going out but none coming back from server, though ingame browser and forums/ myddo/ store still work, you can't move or jump till the server catches up.

Third type is when the second is happening in a spike burst fashion.
You can run, but the server may send traffic disagreeing a fraction of a second later. Ie rubberbanding.
Or you may cast, the client will handle the sp loss, graphics and cooldown, but the server won't acknowledge the action. Then the clients locked you out of recasting til the cooldown is up.
May do this repeatedly, though it won't chew up your framerate, and won't give any clear indication of a problem aside from you knowing you are missing the result from your action.

Fourth type is server mob handling traffic lag.
Velah dead 30 seconds pass and everyone falls down Dead from a fire breath. The traffic indicating something happened either finally got to the clients or the server handled events out of order.
Things like multiple abbot inferno's at once, or a case of arraetrikos standing like a statue while people wail away, then everyone falling over dead from 20+ seconds of damage coming in at once.
Or dead mobs hitting you . Or wandering for five minutes through a quest with a trap by the entry like the graverobber In house j, only to fall down dead due to a force trap and have it come up in the log as killed by "something".

A further instance I'm not sure where to classify involves parties zoning into a quest and getting stuck as a group on the load screen, fully able to voice chat about the problem for 5-15 minutes while stuck.

[altrocks]

The bags are one issue , just keep autocollect on for an ingredient bag and try rapidly buying 100 stacks of crafting dissolver .
Then when that finally catches up disable autocollect and do the same with the items going to inventory and note he huge time difference to complete.

CrAfting post hotfix has a new delay on recipes coming up as well when on first uses a machine.

Seems like server delays handling the actions.

I'd also like to note we seem to be talking at cross purposes.

There's client lags based on fps drops or spikiness. These seem tied to certain graphical effects, especially in activity concentrated encounters like raids.

Then there's server decides to forget to send you any traffic or responses leading to hard pauses or connection to chat server lost. This type may or may not be picked up by the games connection indicator claiming traffic going out but none coming back from server, though ingame browser and forums/ myddo/ store still work, you can't move or jump till the server catches up.

Third type is when the second is happening in a spike burst fashion.
You can run, but the server may send traffic disagreeing a fraction of a second later. Ie rubberbanding.
Or you may cast, the client will handle the sp loss, graphics and cooldown, but the server won't acknowledge the action. Then the clients locked you out of recasting til the cooldown is up.
May do this repeatedly, though it won't chew up your framerate, and won't give any clear indication of a problem aside from you knowing you are missing the result from your action.

Fourth type is server mob handling traffic lag.
Velah dead 30 seconds pass and everyone falls down Dead from a fire breath. The traffic indicating something happened either finally got to the clients or the server handled events out of order.
Things like multiple abbot inferno's at once, or a case of arraetrikos standing like a statue while people wail away, then everyone falling over dead from 20+ seconds of damage coming in at once.
Or dead mobs hitting you . Or wandering for five minutes through a quest with a trap by the entry like the graverobber In house j, only to fall down dead due to a force trap and have it come up in the log as killed by "something".

A further instance I'm not sure where to classify involves parties zoning into a quest and getting stuck as a group on the load screen, fully able to voice chat about the problem for 5-15 minutes while stuck.

The bags are an issue, to be sure. Another issue might be the MCL/CL calculations for savants. That's one big thing that was put in with U9 that I think is responsible for a lot of DB access/lag. Can't be sure, of course, but with the number of savants running around swapping ToD sets, clickies, casting spells with hard caps, semi-hard caps, no caps, and having modified MCL and CL to retrieve, compare and process before the spell effect even goes off, I can't imagine it's not putting a huge strain on things. Still, all speculation from this side since none of us have any first hand knowledge of the internal workings of the servers.

Still, try casting an Outiluke's at a decent sized group of mobs with a WS-III. Or, hit your Amrath belt for the clicky and set bonus and then load up your CL boost items/effects. It's like casting a few GH's on yourself. And then removing them when the timer runs out or you swap items again. Depending on how the info is being handled, it could cause pretty massive client and/or server side lag when you consider how many times things like these are happening across the server every minute, especially at peak times, or even in just one group in one raid instance.

[steelblueskies]

I like the vendor and bag autocollect interaction primarily because it is clearly a server handling issue and is easily meterable and measurable.

If average running around or questing generates x amount of traffic up/down and the traffic throughput for the action is clearly not the bottleneck. That rules out one issue for that particular culprit.
If there's no framerate hit, and no processor utilization change of merit with the action this also eliminates client performance as a culprit for this issue.

This allows clear metering of server performance at a task.

One could potentially repeat the test throughout the day with a macro and watch the time variance to get a clear picture of server load for whatever servers running the crafting hall instance. If anything else were shared with the crafting hall instances server that too would be suffering.

An alternate approach might be large volumes of items in inventory and using autogather and metering time to complete while in various zones, although the time to complete would be a significantly smaller margin for measurement/error range.

It seems like the umd recalcs are a client and server issue which makes them a bit messier to test with.

[Ruphus]

Okay, well I am glad I am not the only one experiencing these problems.

On friday, I ran a normal shroud, there was a few people I have ran with before in group, I knew the healers were good, and we had solid dps.

Part 1 - Little bit of lag and rubber banding, but np, might have taken us 2-3 minutes longer than it should, but we got through it.

Part 2 - Whoa... Gnoll, Cat, Kobold, Fire. Fire South, Cat NE, Kobold and Gnoll in between. it took 10 minutes to get the gnoll in position. We prepped everything, RL called for Gnoll and Kobold to go down, then Fire and Cat. Gnoll drops. Everything pauses, people run to Kobold, everything pauses. Kobold goes down. Everything pauses and people are rubber banding and complaining about it over voice (with no lag on voice, btw) Gnoll respawns, pause, cat dies, pause, kobold respawns, pause, fire dies, pause.

Annoying, but we dealt with it, I run over to pull the 4 bosses, I get their attention, I pause, die, someone else comes to pull, I watch them pause, but am able to see the 4 mini bosses beat him down, dead. Finally we sort our way through it, and we manage to complete, but my god was it every annoying. Part 3 people were dying left and right to blades, cause everyone keeps pausing while running and getting chewed up. Was obnoxious.

I took a break friday night and saturday because the lag is making the game annoying. It's not making it harder, it's just making it unplayable. I hate running about 30 feet, then pausing and waiting 20 seconds, then rubberbanding back 30 feet and repeating the cycle for 15 minutes just to cross the Orchard. When it takes 3-4 minutes to kill 2-3 mobs, and not because of DPS, but because my character keeps pausing mid combat, wildly swinging but not missing, not connecting or anything, but I can watch my hireling heal, attack, nuke etc and I can see the NPC/Monsters attacking, swinging, casting spells, debuffing, etc etc. It's getting old, REAL QUICK.

If I could cancel the 3 month membership I signed up for a couple weeks ago, and delay it until this gets fixed, I would.

[NaturalMystik]

Not to beat a dead horse, but I didn't want to miss out on all the fun...

I too have been noticing some of the strangeness that people have been reporting. What has been bugging me the most is the not sticking spells and clickies. I click on a spell and it seems like it gets cast, however it actually does not, for example the buff doesn't show up. Like someone else mentioned it feels like you failed a concentration check, but this happens when no concentration check is required, and I'm not wearing any armor so it shouldn't be arcane spell failure...

[Mister_Peace]

Thinking about it from a database perspective, there are 2 changes that were made that would seem to just be horrendous for performance.

1. Essence Deconstruction -> Deleting 1 record (or making it inactive) and then creating 5 records + the stack record. Not that bad except for the fact that people are having to crank through 40-50 greaters at a time.

2. Khyber/Syberis shards being increased from 1000 to 10,000. The lookup/update on that transaction is probably painful.

When you use a crafting machine, it looks up all possible recipes.

[Arwitch]

On Orien last week we failed a TOD run due to heals taking about 10 seconds to land... for all 3 healers. We rezzed out, went back in after about 30 mins (2 people in the group decided that jumping off the airship over and over was like doing a raindance to appease the lag monster :P ) and next run was a dream!

Today in crucible had a number of times when I'd hit mobs for about 30 seconds and my axe would swing right through them like they weren't even there.

I always get really bad lag in AOE's like ice storm and fogs but this is entirely different. I'm moving fine, things are just not working the way they should.

[Maxelcat]

had the starting of lag...

Mobs not where they should be, stuff dying before damage displayed... first time for me that it was that noticeable...


And I know its not on my end.

[jkm]

When you use a crafting machine, it looks up all possible recipes.

Given their normal architecture, that is probably a one time web service call when the window is opened and then cached in memory for the duration of the window being open. Because of the nature of web services, you can push those off the game servers and beef them up as needed.

In general, what really lags out client/server machines is database latency and that is normally formed when you update indexes on an insert call.

[SiliconShadow]

Well it might be client based because not everyone has the lag, I for one don't and I know a couple of others that don't either the ones that do lag at things like icestorms, t3 dots, any sfx that isn't always around or loaded the first time.

This makes me wonder next time you get lag like this check your client memory usage and compare it to your free ram, compared to when you first launch ddo, if there is a unexplicable memory loss then this could be the cause of the issue.

alternate theory client side:
Part of the game engine that slows down/impacts the fetching of game resources.

A bit akin to the UMD lag but more mainstream and hits the file fetching performance/graphics performance.

[steelblueskies]

Well it might be client based because not everyone has the lag, I for one don't and I know a couple of others that don't either the ones that do lag at things like icestorms, t3 dots, any sfx that isn't always around or loaded the first time.

This makes me wonder next time you get lag like this check your client memory usage and compare it to your free ram, compared to when you first launch ddo, if there is a unexplicable memory loss then this could be the cause of the issue.

alternate theory client side:
Part of the game engine that slows down/impacts the fetching of game resources.

A bit akin to the UMD lag but more mainstream and hits the file fetching performance/graphics performance.

There's a delay added to the recipe lists coming up on the machines post hotfix.
No associated increase in utilization during the delay so it seems to be an intended client delay added in.

As to the rest- well as stated above there's about 5 categories people are referring to when they say lag lately.

Memory/paging and disk seek would only be valid client issues for a small fraction of them.
Network traffic issues only account for another fragment of the list.

It's the perfect ping, normal traffic up/down normal CPU/ram/disk use, normal fps , but events not occuring in sync between client and server if they are fully occuring at all, ranging from simply moving around, to casting and having no effect, or taking damage from nonexistent mobs, or having mobs stand like statues, that's the big issue tied to original posting here.

That's the one that's existed before as rubberbanding, and has now been expanded to cover more related uh-ohs.

Fps issues or heavy disk access throwing off cast and effect or causing weirdness is to be expected and is a pure client issue. The real bug is the absolutely clear non client occurences.

[jkm]

Well it might be client based because not everyone has the lag, I for one don't and I know a couple of others that don't either the ones that do lag at things like icestorms, t3 dots, any sfx that isn't always around or loaded the first time.

This makes me wonder next time you get lag like this check your client memory usage and compare it to your free ram, compared to when you first launch ddo, if there is a unexplicable memory loss then this could be the cause of the issue.

alternate theory client side:
Part of the game engine that slows down/impacts the fetching of game resources.

A bit akin to the UMD lag but more mainstream and hits the file fetching performance/graphics performance.

We can go through the spells one by one and iterate the problems behind them. For example, ice storm has to do with its size and the sheer number of objects that get drawn in large areas like shroud part 4/5. To see just how many, have a caster cast it at the bottom of one of the coal shafts while you are standing on the top. There could be hundreds of ice objects in that chain.

As for your game engine theory, this certainly has some weight. The easiest example is ladder hitching.

[Mister_Peace]

Given their normal architecture, that is probably a one time web service call when the window is opened and then cached in memory for the duration of the window being open. Because of the nature of web services, you can push those off the game servers and beef them up as needed.

You really think that's how they do it? This is Turbine we are talking about.

[Chai]

Same types of symptoms here. All kinds of rubberbanding type lag.

There was even a point where no one on the server could sell or destroy an item for 90 minutes or so.

[blkcat1028]

I'm having an issue, though similar, hasn't been mentioned so far. The client will log in, the character selection page will display and I can choose character and load into the world (which can take up to 5 or 10 minutes) and then red connection -> "connection to the server has been lost".

From the command prompt I ping ddo.com with a latency of 253 (normal for NZ) and 0% loss. If I use the client network monitor thingy I get them same latency but with a loss of 40% or higher. A trace route doesn't reveal any issues. My ISO has tried to track it down. I've re-installed the client to a dedicated hard drive, defragged, and updated drivers. I can't think of anything else I can do on my part.

It appears to be worse from about 8PM to 1AM NZST. (My prime play time)

In the event that I can actually play the game I get lag, and I am not exaggerating, of up to 5 MINUTES. I completely missed part 5 in a Shroud run the other night. I never saw Harry. All I did was swing and hope I was contributing something to the party.

I have, of course, put in support tickets but Turbine has been unable to help so far.

Any ideas/ advice would be appreciated.

Thanks

[stille_nacht]

Call me crazy, but after nosing thru several threads like this, Im starting to sense a pattern. Seems that the harder the difficulty the more lag interferance...just guessing though as I've not done epics lately and not experiencing much lag.

Occasionally get that syrupy feel where I attack 6 times and strike 3 times, but even that doesn't happen too often for me.

the thing that causes this is more likely to happen at higher difs :[

[steelblueskies]

that is a thought. maybe some bad routine in the damage calcs tied to dungeon scaling? but then it shouldn't be happening on epic.

hmm. maybe an issue with variable type mismatches and the sizes of the values at play frequently.. no..

we all know full well about lag hour and vendors not responding even with a good ping and no client issues.

its like that, only it seems to escalate at times into the server just stops responding to a client at all( in those connection to chat server heavy packet loss ingame monitor moments.).

shrugs. if people had a clearer sense of the types of lag when reporting an issue to places like codeshapers post u9 lag thread they might have a ghost of a chance of redressing it. but since people seem to think a client fps drop, a connection issue, and things like the other weirdness noted here are exactly the same, they just say i lagged and post away.

the only kind they are really working on at present based on their statements seems to be the "lag hour" type occuring seemingly at random, which is pure server issue.

[stille_nacht]

that is a thought. maybe some bad routine in the damage calcs tied to dungeon scaling? but then it shouldn't be happening on epic.

nah, it has to do with monks

[anto_capone]

I first started noticing the lag with the last update, before the patch and the hotfix.

I seemed to notice it more on my TWF melee, I was swinging at things and the sounds of the impact (or lack of) weren't matching up. I would be swinging right at something, and suddenly it appeared 10 feet away and out of my range and all of what should have been hits, were misses.

I have had to restart my client more often than usual it seems after long sessions of play, I have been stuck at loading screens which had never happened before, and I have been experiencing rubber banding a thousand times more often.

It's getting worse...

[Bwahahaha]

Iv'e quit playing till it gets solved.

Man am I glad I'm ftp...

I'll be following this thread, so plz mention it when it gets fixed...

Thanks Bwahahaha