Lets talk about the DDO Client and its Memory Leak

[Junts]

I'm going to preface this by saying it doesn't just apply to the Lamannia client; I havent spent enough time on Lamannia to really empirically work out if its much different with that client, but maybe some others who play for longer periods there can verify this for me. I'm more concerned with how this works on the Live client, but if there are differences I suspect that would be very useful to the developers.



The DDO client has a persisent memleak connected to character switching. The more one switches characters without reopening the client, the worse it gets. This causes jerky animations, increasingly longer loadtimes, and difficulty with the client keeping up with large combats. This isn't even limited to raid combats: After 4 character swaps, I can't solo sins of attrition anymore because of the lag from all the monster attacks.

Re-starting the client will begin this process over again. The threshold at which this starts to affect people varies based on hardware; with a rather poor laptop, I tend to notice it by the 3rd or 4th. Others I play with notice by the 6th or 7th. It is pretty much always present and its definitely tied to character swapping. If i simply open the client and change characters 7 times, the game is unplayable within 10m. If I stay on one char for 3 hours, I don't develop this problem. A vast majority of people I play with have noticed this, and we all routinely restart the client on every other or every 3rd character swap.

I'd be interested in knowing A: if this is also present with the lamannia client, B: other people's experiences with it and C: whether this is a known issue.

[Gordo]

I absolutely experience it including not only a client crash but a full PC crash entering the desert earlier today.

This happens often. I am talking about on Argo. Have not had enough time on Lama. to opine.

[AyumiAmakusa]

Omg I never realised that this happens! That would explain my crashes! I seriously thank you for bringing my attention to this. Hopefully the devs read this too and propose a fix.

[kamimitsu]

Happens to me as well. Some guildies and I were actually talking about it recently. For me, it's noticeable after about 4 switches, game affecting after 6 or so, and unplayable after 8 or 9.

It makes inventory upkeep a bit of a pain.

[Hendrik]

I'm going to preface this by saying it doesn't just apply to the Lamannia client; I havent spent enough time on Lamannia to really empirically work out if its much different with that client, but maybe some others who play for longer periods there can verify this for me. I'm more concerned with how this works on the Live client, but if there are differences I suspect that would be very useful to the developers.



The DDO client has a persisent memleak connected to character switching. The more one switches characters without reopening the client, the worse it gets. This causes jerky animations, increasingly longer loadtimes, and difficulty with the client keeping up with large combats. This isn't even limited to raid combats: After 4 character swaps, I can't solo sins of attrition anymore because of the lag from all the monster attacks.

Re-starting the client will begin this process over again. The threshold at which this starts to affect people varies based on hardware; with a rather poor laptop, I tend to notice it by the 3rd or 4th. Others I play with notice by the 6th or 7th. It is pretty much always present and its definitely tied to character swapping. If i simply open the client and change characters 7 times, the game is unplayable within 10m. If I stay on one char for 3 hours, I don't develop this problem. A vast majority of people I play with have noticed this, and we all routinely restart the client on every other or every 3rd character swap.

I'd be interested in knowing A: if this is also present with the lamannia client, B: other people's experiences with it and C: whether this is a known issue.

Good post Junts.

One thing, when you start to lag, hitch, rubberband, etc., what is your data Rx rate?

Llama or Live, preferably Live.

I have personally witnessed and doc'd HUGE lag when Rx hits 12-13k and other players in same instance verifying the same.

I would definitely share your findings with Phax via PM.

[Noctus]

i have these symptoms, too.

[Junts]

Good post Junts.

One thing, when you start to lag, hitch, rubberband, etc., what is your data Rx rate?

Llama or Live, preferably Live.

I have personally witnessed and doc'd HUGE lag when Rx hits 12-13k and other players in same instance verifying the same.

I would definitely share your findings with Phax via PM.

While it is connected somewhat to data transfer to the client, turning off stuff like combat feedback doesn't help. Its pretty hard to get useful data when its happening, since its during raids and in the middle of combats and stuff. The more attacks going in and out, the worse it gets.

One thing worth noting is that this doesn't cause rubberbanding. Its specifically a client bogging issue that relates to large areas (instances and public) and large amounts of combat data. It will cause stutterstepping and inability to move, but more in the 'you are 60s behidn the party with data processing' way, not with actual rubberbanding or other 'my abilities dont fire' behaviors. Your abilities will fire normally, at least so far as they can when you are that far behind.

[dameron]

I'm going to preface this by saying it doesn't just apply to the Lamannia client; I havent spent enough time on Lamannia to really empirically work out if its much different with that client, but maybe some others who play for longer periods there can verify this for me. I'm more concerned with how this works on the Live client, but if there are differences I suspect that would be very useful to the developers.



The DDO client has a persisent memleak connected to character switching. The more one switches characters without reopening the client, the worse it gets. This causes jerky animations, increasingly longer loadtimes, and difficulty with the client keeping up with large combats. This isn't even limited to raid combats: After 4 character swaps, I can't solo sins of attrition anymore because of the lag from all the monster attacks.

Re-starting the client will begin this process over again. The threshold at which this starts to affect people varies based on hardware; with a rather poor laptop, I tend to notice it by the 3rd or 4th. Others I play with notice by the 6th or 7th. It is pretty much always present and its definitely tied to character swapping. If i simply open the client and change characters 7 times, the game is unplayable within 10m. If I stay on one char for 3 hours, I don't develop this problem. A vast majority of people I play with have noticed this, and we all routinely restart the client on every other or every 3rd character swap.

I'd be interested in knowing A: if this is also present with the lamannia client, B: other people's experiences with it and C: whether this is a known issue.

XP, Vista and Windows 7 all have excellent logging features. Perfmon.exe for Vista and Windows 7 and perfmon.msc for XP. They're build in and can monitor a process over time for memory usage, disk writes, basically any performance info you'd need, and you can create reports from the collected information very easily.

I've used them many time for "real" work but never really though to use them to debug the obvious memory leak that DDO has. And my observations exactly match yours, btw.

When I next have time to play for an extended period I'll collect the logs and performance reports and see what they tell me.

[Stanley_Nicholas]

This issue has been here for years - possibly since the original launch. And it has seemed to get more frequent each time there has been a graphical upgrade to the game.

It is absolutely the single most irritating long-time bug still in the game.

[Junts]

I don't have any idea how to use that. Please teach us.

[pSINNa]

This is very well known problem.

To the extent where guildies will say before a raid 'getting a fresh client', and everyone understands.

Try running through the markets after swapping between 4 characters, and you'll see it at it's worst.

If the devs didn't know about it, i'd be very suprised, i've always put it in the same basket as broken ladders and other various flaws we know about and just endure.

But in saying that, if it could be fixed, i'd be a lot happier (13 characters, level up one of them, then go looking for the twink gear for his new level can take a hour of swapping/re-starting clients etc, it gets painful).

Coit out~

[Stanley_Nicholas]

While it is connected somewhat to data transfer to the client, turning off stuff like combat feedback doesn't help. Its pretty hard to get useful data when its happening, since its during raids and in the middle of combats and stuff. The more attacks going in and out, the worse it gets.

One thing worth noting is that this doesn't cause rubberbanding. Its specifically a client bogging issue that relates to large areas (instances and public) and large amounts of combat data. It will cause stutterstepping and inability to move, but more in the 'you are 60s behidn the party with data processing' way, not with actual rubberbanding or other 'my abilities dont fire' behaviors. Your abilities will fire normally, at least so far as they can when you are that far behind.

Do you ever get actual crashes from combat-data-heavy situations? I've never seen lag at those times get nearly as bad as I've seen others describe, and I've never crashed during them. When I crash, it's always when loading into a new instance or loading a new character.

[FluffyCalico]

It appears that DDO treats characters as TSRs meaning once loaded they stay that way so the more you switch the more memory it uses.

[dameron]

Ok, here's my very quick summary of what I noticed.

First, this was on my laptop a Dell XPS m1530 with 2gb ram and a similar 2gb paging file.

I initially included many columns of data about the network throughput but it was uninteresting in the final result. I logged into a character on Thelanis then entered an additional instance, then logged off, switched characters and repeated until I was done with all 10 of my chars on Thelanis.

I didn't notice any issues this time, however this was only over about a 20 minutes run, but the graph points to disaster, as you will see.



The dotted green line is memory usage, and as you can see I killed chrome very early on and it varied not much at all.

CPU usage is the solid red line. You can actually see the CPU peak each time I log in a new character, producing 10 very identifiable peaks.

You can also see that with each new login there's a corresponding immediate increase in page file usage. (the solild green line) Every time the CPU spikes it indicates new login and the page file fills a bit. It does not empty in an acceptable manner.

Here's what Microsoft has to say about page file % (the solid green line in the chart):

Paging File, %pagefile in use: This is a measure of how much of the pagefile is actually being used. This is the counter you should use to determine whether the pagefile is an appropriate size. If this counter reaches 100, the pagefile is completely full and operations stop working. Depending on the volatility of your workload, you probably want to set the pagefile large enough so that no more than 50 to 75 percent of it is used. If a large part of the pagefile is in use, having more than one pagefile on different physical disks may improve performance.

The trend in the chart is not good wrt page file usage. When the page file fills all hell breaks loose (the symptoms Junts mentions).

So, either increase your page file size, or DDO should somehow clear its virtual memory usage on logout. Otherwise the nastiness trends toward inevitable.

[Modinator0]

I'd be interested in knowing A: if this is also present with the lamannia client

Not only is it present on lama land, it actually seems to bog things down a lot sooner than on live

[Stanley_Nicholas]

Yeah that makes sense. My page file is always under heavy use every time I crash, and it usually takes a while to clear after the crash before DDO fully closes - long enough that it's actually faster to open the task manager and kill the DDO process directly than allow it to fully close on its own via the crash.

It's interesting to see that loading characters has little or no effect on memory usage, and instead all the character data seems to be loaded directly to page file. I imagine that made more sense 4-5 years ago when many systems had <2gb of memory, but now with 2gb being the minimum standard on yesterday's machines and with many systems sporting 4+, it doesn't seem to make as much sense anymore to force that.

Although the obvious solution to the crashing problem is to release character info from the page file upon logout, perhaps the client could also do a check upon login and load character data into physical memory instead of page file if there are at least 2gb of memory available? The point is, why hog up all that page file and add extra wear and tear to the hard drive when all that physical memory is sitting there available to be used, and with much better access times to boot?

[Junts]

Alright, next explain how to increase the pagefile size.

[FluffyCalico]

Alright, next explain how to increase the pagefile size.

Well a bigger problem than the size is if you have it set to auto size...the default for windows and its on your main drive it will fragment terribly and cause horrible issues in ddo. Best is to partition off about 15G and call it drive S lable it swap or whatever will tell you to put nothing there. Then tell your swap file to use S. That way the swap file will never fragment and can grow up to 15G if needed.

[DrenglisEU]

Good, it good to know that the strange Win7 crashes I have experiences can be caused by the client, thanks mate

[bigolbear]

I too have experienced this issue - and its one that goes back a long time.

The GOOD news is that the extra memory (or page file) being used when you swap characters seems to be re allocated to the system when you close the client, so although its annoying its not a memory leak.

That being said i think there is a memory leak in the client but its only a few hundred k, and i cant be sure.. well i havent been sufficiently bothered to check.