Lets talk about the DDO Client and its Memory Leak

[barabel]

I agree there were a lot of variables at play in that last data sample.

Tonight I hope to try a more simplistic test with one character.

I plan on going to a low population area (the tavern in Atraxia's Haven) and repeat the Login/Logout test again.

If the numbers show there is still an increasing footprint then I'll look to expand the test to include multiple characters in the same location and see what the numbers look like.

From that we can see what additional tests may be necessary.

[jamkriz]

I had started a thread on this not too long ago after having similar issues with the live client. I have managed to minimize the effects though by doing the following:

1. If possible, reinstall the client to a different hard drive than the one where the pagefile is sitting.

2. If using an older operating system like Windows Vista or XP, manually set the min and max size of your pagefile to a multiple of your physical memory size (2x or 3x is fine). i.e. if you have 2GB or RAM then set pagefile to 4GB or 6GB. Best way is to look up exactly how much physical RAM your system (Right-Click My Computer->Properties) is showing as some of it may have been reserved for video.
Setting pagefile can be done as per http://www.theeldergeek.com/physical..._file_size.htm.
EXERCISE CAUTION WHEN MESSING WITH THESE SETTINGS!!!!!!

Windows 7 memory and pagefile management seems to be much better so this step not as much an issue there.

3. Update your DirectX (http://www.microsoft.com/downloads/d...a-9b6652cd92a3) and video drivers for your video card from the manufacturer site.

[EyeRekon]

Tonight I hope to try a more simplistic test with one character.

From that we can see what additional tests may be necessary.

Pre-thanks for trying to help the community.

[Irinis]

I always play on low graphics settings and have never had any zone-in problems even after switching characters several times. However I did increase the pagefile size shortly after getting this system as at the time I was doing some multimedia work. It's also win7 so less likely to have problems according to the above.

[barabel]

Test Setup.
To reduce as many factors as I could figure, I decided to start from scratch. I created a new veteran character and loaded him up on the "Heart of Wind" airship. There I had him delete his starter rags so he had no possessions.

I then restarted the client and recorded how much memory was consumed by simply logging in and logging out back to the character selection screen for this character.

Here the Working Set Windows was showing allocated to the DnDClient executable (measured in KB).

Test Run Memory Allocated (KB)
01 434560
02 470116
03 477060
04 493816
05 488232
06 507420
07 524112
08 559740
09 546200
10 571972

During the test I didn't move the character/camera and simply waited for the Chat Windows to load. Once that was done I logged out.

The numbers above show two important things to me.

1. As with the previous test there is a noticeable increase in the amount of memory being consumed by the client. Since nothing really changes from one session to the other that would seem to indicate there is a leak of some sort going on.
2. Memory was deallocated during two of the test runs (05, 09). I am not sure what was relinquishing the memory (Windows or the DnD Client internally), but some memory was returned back to the operating system at this time.

All in all, the footprint for the client increased by over 130 MB from start to finish. For those folks who are running with less available memory such switching may be a significant problem and this may help explain why restarts of the client is necessary.

Again, this test was done on Windows 7 Professional (x64) with 8 GB RAM and no configured page file. All memory numbers represent actual physical memory usage (no page file). Total System memory usage never went above 50% (i.e. At least 4 GB of RAM was available at all times during the test).

I don't think I'll have time to run any more tests tonight. If folks think of other things to try I might be able to try later this week.

Hope people find the data interesting.

Update: After doing some additional reading on Windows memory I thought I'd change the values from Committed Bytes to the recorded Values for the Working Set instead. This seems to be a more accurate picture of the memory usage on the system. ( Microsoft KB Article )

[rodarius]

If the Client is build as a lazy process it wouldnt release resources until another system part is requesting them. The whole .net Framework is built on this premise, so that also might be a reason for increasing memory consumption with freeing after some time.

[Dinglebarry]

This is not from the Lamannia client, but thought it relevant as it was talking about memory usage increase. This is related to Zoning crashes, but I noticed memory usage increasing just running from one Zone to another. Look here:


http://forums.ddo.com/showthread.php...20#post2813720

and a fix (for my zoning issues)
http://forums.ddo.com/showthread.php...71#post2814971

[Nospheratus]

To be honest i never noticed such a thing. I think that for such a test to be reliable the "conditions" of the test should be the same. What i mean with this is: place all your characters in the same place, preferably with few people or even none, like the guild airship. Then, restart the client and login with all characters. I believe the loading times will be very small for the second character onwards and the memory increase won't be that much.
Also, the page file is used by windows for several things and one of them is to allow computers with low memory (2gb is low these days if you want to play a game) can use more applications at the same time, but it's controlled by windows and you never know what's in there.

I don't think DDO has a memory leak, unless it's very subtle. Anyone that played dragon age shortly after launch, you know that it was a game with a memory leak and easily noticed! After the third time you had to change area, the game was already at 3gb of ram used...

It's only normal for DDO to load common things into memory, like armor skins, weapons, character skins, etc. That's why i suggested taking all the characters to the airship. It's nowhere near as crowded as any other place.

This is not from the Lamannia client, but thought it relevant as it was talking about memory usage increase. This is related to Zoning crashes, but I noticed memory usage increasing just running from one Zone to another. Look here:


http://forums.ddo.com/showthread.php...20#post2813720

and a fix (for my zoning issues)
http://forums.ddo.com/showthread.php...71#post2814971

The desert is known for crashing the game, even though i never had that. And it can crash the game even if you have just started the game and went there. I have seen it happen to guildies.

The increases in memory when zoning is easily explained by the game loading textures for the area, for whatever characters are in the area or NPCs for example...

The game has to load these things, otherwise you can't see them! And by loading them, there is only one place where they can go, which is the memory (including the page file).


If you have 2gb of ram, don't expect the client to be smooth after you loaded all it's content. If you notice, the game takes around 8gb or hard disk space and most of it are surely textures and animations! With 2gb of ram, 1gb or more will be taken by the operative system, anti-virus and any other applications that you or your machine start up.

[Hendrik]

I always wondered what it would look like to capture a video while experiencing the performance problems. Would the video show the same thing you experience? If so, I bet you could show the lag jitter/snap back/rubberbanding while your internet latency is fine.

Done that and shared with Phax.

[Schwarzie]

Also, the page file is used by windows for several things and one of them is to allow computers with low memory (2gb is low these days if you want to play a game) can use more applications at the same time, but it's controlled by windows and you never know what's in there.

[SNIP]

If you have 2gb of ram, don't expect the client to be smooth after you loaded all it's content. If you notice, the game takes around 8gb or hard disk space and most of it are surely textures and animations! With 2gb of ram, 1gb or more will be taken by the operative system, anti-virus and any other applications that you or your machine start up.

This is not entirely correct. Most (basically all games without a 64bit Mode which is the vast majority) can use a max of 2GB memory. Therefore if you have 3GB you have enough memory for every 32bit game. This is unrelated to your operating system! Run a 32bit game in a 64bit OS and it still will use only 2GB max. (There once was a nice bug with SupremeCommander where the 32bit client tried to use more then 2GB memory...)

For memory consumtption it is completely irrelevant how much space on your Harddisc is used by the game.

And if your system + some memory resistant programms take up 1+GB memory its either a huge filechache currently used (which will be freed once a programm needs more memory) or your system is horribly cluttered with garbadge.

[barabel]

If the Client is build as a lazy process it wouldnt release resources until another system part is requesting them. The whole .net Framework is built on this premise, so that also might be a reason for increasing memory consumption with freeing after some time.

This is very possible, though I would assume that if the Garbage Collection happened it would have freed up more resources than what I recorded. In theory, each time I loaded up the same character in the same small and isolated location, the overall memory footprint should be close to the same. I would have expected a GC clearing out unused references/objects to get much closer to the 1st test than it did.

Without knowing more about the technology used in building the client or what they are doing actually storing in memory, a lot is left to speculation.

[barabel]

This is not entirely correct. Most (basically all games without a 64bit Mode which is the vast majority) can use a max of 2GB memory. Therefore if you have 3GB you have enough memory for every 32bit game. This is unrelated to your operating system! Run a 32bit game in a 64bit OS and it still will use only 2GB max. (There once was a nice bug with SupremeCommander where the 32bit client tried to use more then 2GB memory...)

For memory consumtption it is completely irrelevant how much space on your Harddisc is used by the game.

And if your system + some memory resistant programms take up 1+GB memory its either a huge filechache currently used (which will be freed once a programm needs more memory) or your system is horribly cluttered with garbadge.

Well said (+1)

[Rasczak]

Yup I can add a confirmation to this thread.

If I log on and play one character through various raids I'll get only the usual latency, hirleing and server lag.

If I run slayer rare chests and swap between characters (i.e run a caster then switch to a FvS then switch to a cleric etc to limit ransacking), my HDD access goes up, my client battles with loading screens, I'll have staggered loading (I'll be loaded but it takes longer to be able to move or access bank etc), recalling takes forever and any sort of load (mob battles etc) put onto the client will cause massive jerking.

And it's not the pc setup, I have a strong machine....
It's way worse on live as well. I have to log on and switch chars to check auctions and mail (for example) and then leave the game and reload the client to be able to play effectively.

[Nospheratus]

This is not entirely correct. Most (basically all games without a 64bit Mode which is the vast majority) can use a max of 2GB memory. Therefore if you have 3GB you have enough memory for every 32bit game. This is unrelated to your operating system! Run a 32bit game in a 64bit OS and it still will use only 2GB max. (There once was a nice bug with SupremeCommander where the 32bit client tried to use more then 2GB memory...)

I don't see what's incorrect in that part. Even what you said doesn't counter what you quoted. But it's irrelevant for the matter at hand! The part you "[SNIP]"ed was what actually gave context to what you quoted...

For memory consumtption it is completely irrelevant how much space on your Harddisc is used by the game.

I just mentioned that so that people would have an idea of how much actual size the game *can* occupy.

And if your system + some memory resistant programms take up 1+GB memory its either a huge filechache currently used (which will be freed once a programm needs more memory) or your system is horribly cluttered with garbadge.

Not really, just use windows vista and you will see this happening. Boot the freshly installed system and take a look at the occupied memory... Install an anti-virus, DDO and there you are...
It may have changed at some point with a service pack, but i know my computer at work has more than 1gb occupied when i boot it up...


In any case, i think all these details are a bit irrelevant in the DDO context. I can't find in DDO a memory leak behavior, but that's just me in my machines. It is pointed out and the devs will surely consider the issue and possibly fix the crashing on some situations.

[barabel]

The installation size of a game isn't really an accurate depiction of how much system memory is required to actually run the game. It is unlikely that all the resources which are bundled will be needed all at the same time.

Also, with regards to 32-bit vs. 64-bit, On Windows platforms, all 32-bit processes are limited to a maximum memory allocation of 2 GB (with some limited exceptions - usually server-based products). 64-bit applications can get around this 2 GB process limitation and use as much memory is available on the machine.

So, since the DnDClient application is a 32-bit app, the most is could ever possibly use is around 2 GB.

As I stated before, I have never seen it go above 1.5 GB on my system, though YMMV.

[Montrose]

I'm going to preface this by saying it doesn't just apply to the Lamannia client; I havent spent enough time on Lamannia to really empirically work out if its much different with that client, but maybe some others who play for longer periods there can verify this for me. I'm more concerned with how this works on the Live client, but if there are differences I suspect that would be very useful to the developers.



The DDO client has a persisent memleak connected to character switching. The more one switches characters without reopening the client, the worse it gets. This causes jerky animations, increasingly longer loadtimes, and difficulty with the client keeping up with large combats. This isn't even limited to raid combats: After 4 character swaps, I can't solo sins of attrition anymore because of the lag from all the monster attacks.

Re-starting the client will begin this process over again. The threshold at which this starts to affect people varies based on hardware; with a rather poor laptop, I tend to notice it by the 3rd or 4th. Others I play with notice by the 6th or 7th. It is pretty much always present and its definitely tied to character swapping. If i simply open the client and change characters 7 times, the game is unplayable within 10m. If I stay on one char for 3 hours, I don't develop this problem. A vast majority of people I play with have noticed this, and we all routinely restart the client on every other or every 3rd character swap.

I'd be interested in knowing A: if this is also present with the lamannia client, B: other people's experiences with it and C: whether this is a known issue.

I verified this leak on multiple computers running multiple operating systems. I even posted perfmon graphs to show the (obvious) increase in memory usage as one logs characters in/out.

See here: http://forums.ddo.com/showthread.php...02#post2914702

Edit: This post tells you how to make your own graph: http://forums.ddo.com/showpost.php?p...1&postcount=12

[Montrose]

To be honest i never noticed such a thing. I think that for such a test to be reliable the "conditions" of the test should be the same. What i mean with this is: place all your characters in the same place, preferably with few people or even none, like the guild airship. Then, restart the client and login with all characters. I believe the loading times will be very small for the second character onwards and the memory increase won't be that much.

Luckily I did all this about 6 months ago. See previous post. The leak exists.

[barabel]

Great stuff Montrose!

It would be nice if we could get a "Read by a Dev" line in here as some sort of acknowledgement.

[EyeRekon]

Test Setup.
To reduce as many factors as I could figure, I decided to start from scratch. I created a new veteran character and loaded him up on the "Heart of Wind" airship. There I had him delete his starter rags so he had no possessions.

I then restarted the client and recorded how much memory was consumed by simply logging in and logging out back to the character selection screen for this character.

Here the Working Set Windows was showing allocated to the DnDClient executable (measured in KB).

Test Run Memory Allocated (KB)
01 434560
02 470116
03 477060
04 493816
05 488232
06 507420
07 524112
08 559740
09 546200
10 571972

During the test I didn't move the character/camera and simply waited for the Chat Windows to load. Once that was done I logged out.

The numbers above show two important things to me.

1. As with the previous test there is a noticeable increase in the amount of memory being consumed by the client. Since nothing really changes from one session to the other that would seem to indicate there is a leak of some sort going on.
2. Memory was deallocated during two of the test runs (05, 09). I am not sure what was relinquishing the memory (Windows or the DnD Client internally), but some memory was returned back to the operating system at this time.

All in all, the footprint for the client increased by over 130 MB from start to finish. For those folks who are running with less available memory such switching may be a significant problem and this may help explain why restarts of the client is necessary.

Again, this test was done on Windows 7 Professional (x64) with 8 GB RAM and no configured page file. All memory numbers represent actual physical memory usage (no page file). Total System memory usage never went above 50% (i.e. At least 4 GB of RAM was available at all times during the test).

I don't think I'll have time to run any more tests tonight. If folks think of other things to try I might be able to try later this week.

Hope people find the data interesting.

Update: After doing some additional reading on Windows memory I thought I'd change the values from Committed Bytes to the recorded Values for the Working Set instead. This seems to be a more accurate picture of the memory usage on the system. ( Microsoft KB Article )

Nice test. We cannot jump to conclusions though. As far as I remember the "Heart of Wind" airship isn't a private instance and other newly generated characters can show there. Maybe some showed up that you didn't notice and they chose their starter equipment and put it on, so your client had to load all the models and textures for that equipment. Then again maybe the client already had that stuff loaded as a precondition for the first time.

Even still, it does not seem to fit a 130MB increase. What else might be happening? Fetching and caching the who list? LFMs?

Thanks for testing!

[Rasczak]

Even still, it does not seem to fit a 130MB increase. What else might be happening? Fetching and caching the who list? LFMs?

Thanks for testing!

Not that possible. Maybe before U6 (or was it 5?) but the Who's list no longer automatically loads and the LFM panel can't take up that many resources on the times I play (Eubie so not US primetime)

The only differences I can see is that textures etc take longer to load each time. The instance itself loads slower as well as your character. If you switch instances I have seen an increase but not as bad as logging the character and logging back in.

The unequip bug helps with this, I can physically time how long it takes each time to load a decked out level 20. From the time the load screen vanishes to the time it takes for my avatar to become naked and re-equip, definitely increases each time I log. If I nuke the client and wait for all processes to terminate and then reload it, the times are decreased.

The HDD access is longer, the memory usage spikes and the game itself is slower even though my internet UL / DL stays the same. It's definitely a different kind of lag as if it's server-side lag my connection spikes when my client catches up. In these instances my connection stays the same.