Lets talk about the DDO Client and its Memory Leak

[zorander6]

I have started reading through this thread.

Sweet, I'll do some testing tonight if I have a chance. Probably won't switch characters much but will analyze memory usage as time permits.


Edit: System stats for future reference.

Intel core 2 centrino t5300 at 1.73Ghz
4G ram
Windows 7 32bit
3G paging file.
Toshiba Satellite laptop

[Junts]

I have started reading through this thread.

Thank you much.

I want to edit/confirm a few things from the thread that are consistent with my experiences:

1: what Draccus says about umd/skill swap gear. After 3-4 relogs, you have to click multiple times to get swapping to a skill item to register. Skill buffs aren't quite as bad, and UMD or all skills items are the worst because it makes the game re-check all your items for umd requirements (even if none have them). For example, at about the 4th relog my characters basically can't use vile blasphemy as a clicky without me stopping and dedicatnig several seconds to clicking onto it and activating it - and then clicking back. On a first relog, I can switch to it with a single hotkey, activate it with that same key, and then immediately switch back. When the pagefile is getting big, in-combat activation like this is suicidal, because I'll stop moving completely during both swaps and be unable to heal myself or cast any spells for 3 seconds on each side of the gear swap.

2: Its not a memory leak but rather the persistent pagefile/memory growth, in that it doesn't require a reboot to correct.

3: It doesnt correlate exactly with zone-into-area crashes, but they do come together: I know players who will routinely crash entering the desert if its after their first login, and may only do so 1/3 of the time on the first login.

4: it seems more consistently problematic for worse computers (no surprise), as you really need to do what Lithic describes in terms of char-swapping to get this problem consistently with a high-end machine.

5: Something that hasn't been emphasized enough is how much more prevalent this problem becomes for people who have played longer, because of the HD fragmentation issues that the pagefile use causes. If we were all better with defragging I bet this problem would go from ubiquitous to sporadic. It's still present, though, and I had it immediately after getting a completely new harddrive and installing the client again this January. I didn't recognize what caused it for some time, though.

[Sutekx]

I have started reading through this thread.

I started noticing it more when I keep swapping toons to transfer items to my bard to sell. At first I thought it was the onboard video card, but I had upgraded recently to a new video card and experienced the same issue. Client lag from the memory caching due to swapping characters constantly. As of right now, I do my swaps and sell, then exit the client completely, it will hang on exit due to the leak as windows is trying to recover the memory and clear it. I am surprised there isn't a way in the game to clear the caching that I have experienced in other MMOs.

Thank you for looking into this, and I hope you will find a solution to the problem.

[Sutekx]

5: Something that hasn't been emphasized enough is how much more prevalent this problem becomes for people who have played longer, because of the HD fragmentation issues that the pagefile use causes. If we were all better with defragging I bet this problem would go from ubiquitous to sporadic. It's still present, though, and I had it immediately after getting a completely new harddrive and installing the client again this January. I didn't recognize what caused it for some time, though.

I have to disagree with this point Junts, I always defrag (I hate seeing red lines to the point of obsessive compulsion) and scandisk my drives to keep optimal performance, not for this game, but other applications and easier file indexing as well. It still happens and gets very noticeable after 6 characters swaps, and exiting the client afterward takes a few minutes for the application to release the memory and close out properly.

[Montrose]

Thank you much.

I want to edit/confirm a few things from the thread that are consistent with my experiences:

2: Its not a memory leak but rather the persistent pagefile/memory growth, in that it doesn't require a reboot to correct.

That is not a correct definition of memory leak as it is commonly used in the sfotware industry. Due to the way that the OS insulates itself, it is actually quite difficult to design a program that will retain memory once the process has been terminated (where "memory" above does not refer permanent or semi-permanent storage such as hard drives, CF cards, USB sticks, etc).

A memory leak is simply an allocation of memory that you cannot (easily) get back.

This, for example, is a (trivial and short-lived) memory leak (assuming no garbage collector):

void LeakyLeak()
{
int *pnGoodByeCruelWorld = malloc(1);
pnGoodByeCruelWorld = 5; // At this point you will have a hard time freeing the original memory since the address is lost
}

[Aashrym]

1: what Draccus says about umd/skill swap gear. After 3-4 relogs, you have to click multiple times to get swapping to a skill item to register. Skill buffs aren't quite as bad, and UMD or all skills items are the worst because it makes the game re-check all your items for umd requirements (even if none have them). For example, at about the 4th relog my characters basically can't use vile blasphemy as a clicky without me stopping and dedicatnig several seconds to clicking onto it and activating it - and then clicking back. On a first relog, I can switch to it with a single hotkey, activate it with that same key, and then immediately switch back. When the pagefile is getting big, in-combat activation like this is suicidal, because I'll stop moving completely during both swaps and be unable to heal myself or cast any spells for 3 seconds on each side of the gear swap.

That's something to keep in mind. I didn't think of UMD as a cause myself but it makes sense and I have a lot of UMD equipment on several characters.

[Aashrym]

I have to disagree with this point Junts, I always defrag (I hate seeing red lines to the point of obsessive compulsion) and scandisk my drives to keep optimal performance, not for this game, but other applications and easier file indexing as well. It still happens and gets very noticeable after 6 characters swaps, and exiting the client afterward takes a few minutes for the application to release the memory and close out properly.

Long before this thread started I was noticing issues and ran a defrag. I gained 16GB memory back on my hard drive even with a regular defrag on a 2 week schedule. I changed to a 1 week scheduled defrag and that helped.

I tend to agree with Junts that fragmentation can be an issue. I also know that frequent character swaps will produce the issue regardless.

[Lorien_the_First_One]

5: Something that hasn't been emphasized enough is how much more prevalent this problem becomes for people who have played longer, because of the HD fragmentation issues that the pagefile use causes. If we were all better with defragging I bet this problem would go from ubiquitous to sporadic. It's still present, though, and I had it immediately after getting a completely new harddrive and installing the client again this January. I didn't recognize what caused it for some time, though.

On Monday I replaced the main HD for my computer. After copying back the image file, my new HD of course showed 0% fragmentation. My pagefile is located on a second physical drive and is of fixed size and can not therefore cause any file fragmentation.

The first thing I did was log into DDO, where I crashed after a series of character swaps.

[Chai]

I have seen more of this issue when on Windows 7 than on XP. Windows Vista and 7 seem to be handeling admin rights slightly differently for installs of 32 bit software. Basically I had to go into the registry and edit the permissions for all directories the ddo install writes into, so that they have full access, then make sure the subdirectories are inheriting permissions from the mains (to avoid hours of tedious editing, /barf). This way the program can release all memory it has used but is no longer using, which will open the bottleneck, so to speak.

I can now log in as many times as I want, within reason of course. I still get the delayed log on where it looks like my toon is in game but I cant move for 20 seconds or so, but once I am in, I_am_in_there!! No more character animation lag, button push lag, screen activation lag (when you open the inventory, your screen freezes, and then 2 seconds later it actually opens).

P.S. I dont recommend messing around with the registry unless you know what you are doing. I did this fully knowing that if I screwed my install up any worse I was just going to reinstall anyhow. Perhaps turbine technical support will provide a list of directories their install writes to. I also recommend backing the registry up before making edits.

[Coldin]

That is not a correct definition of memory leak as it is commonly used in the sfotware industry. Due to the way that the OS insulates itself, it is actually quite difficult to design a program that will retain memory once the process has been terminated (where "memory" above does not refer permanent or semi-permanent storage such as hard drives, CF cards, USB sticks, etc).

A memory leak is simply an allocation of memory that you cannot (easily) get back.

This, for example, is a (trivial and short-lived) memory leak (assuming no garbage collector):

void LeakyLeak()
{
int *pnGoodByeCruelWorld = malloc(1);
pnGoodByeCruelWorld = 5; // At this point you will have a hard time freeing the original memory since the address is lost
}

And then even if you have a garbage collector, it could be poorly implemented to not free all assets.

But yeah, once you close the client, all the memory it was using will be freed.

[Schwarzie]

I can now log in as many times as I want, within reason of course. I still get the delayed log on where it looks like my toon is in game but I cant move for 20 seconds or so, but once I am in, I_am_in_there!! No more character animation lag, button push lag, screen activation lag (when you open the inventory, your screen freezes, and then 2 seconds later it actually opens).

Even with excessive toonchange you should not have any Problems. Its just a Bug which needs correction Asap.

Perhaps turbine technical support will provide a list of directories their install writes to. I also recommend backing the registry up before making edits.

Perhaps Turbine will do it right an stop wrtiting in Directories outside the Installdirectory at all.

I will never ever understand why it should be necessary for a Programm to write garbadge into other directories or in the registry. From my Point of View this is just a stupid design failure. Programm should write in their own folder and maybe a Shared folder for all programms. But nowhere else. This would reduce the slow death of windows by a Huge margin. (IIRC are there OS outside the windows world which do that right)

But sadly the current heading is in another direction. Games for example now have 3 or more directorys where they put stuff to be compliant with the "Games for Windows" ********. Simply unbelievable.

[Chai]

Even with excessive toonchange you should not have any Problems. Its just a Bug which needs correction Asap.

Perhaps Turbine will do it right an stop wrtiting in Directories outside the Installdirectory at all.

I will never ever understand why it should be necessary for a Programm to write garbadge into other directories or in the registry. From my Point of View this is just a stupid design failure. Programm should write in their own folder and maybe a Shared folder for all programms. But nowhere else. This would reduce the slow death of windows by a Huge margin. (IIRC are there OS outside the windows world which do that right)

But sadly the current heading is in another direction. Games for example now have 3 or more directorys where they put stuff to be compliant with the "Games for Windows" ********. Simply unbelievable.

Yeap, I hear you. This hasnt really happened since the Windoze 95 / 98 days when you could just copy the program install folder from one hard drive to another and it was considered "installed" ....heh.

Also, the fact that more and more software installs are writing permissions to the registry as limited user access, even when installed to a full admin profile, is completely ********. Normal users arent going to open their registry and start editing directories any time soon. They would have to pay a tech to do that, and they will charge an hour of labor for 5 minutes of work.

[Sutekx]

Long before this thread started I was noticing issues and ran a defrag. I gained 16GB memory back on my hard drive even with a regular defrag on a 2 week schedule. I changed to a 1 week scheduled defrag and that helped.

I tend to agree with Junts that fragmentation can be an issue. I also know that frequent character swaps will produce the issue regardless.

It can be a major issue for those who don't know. But with mine even fully defragged, as you said and others, swapping characters still produce that issue. I don't crash at all, but client lag is extreme after swapping around a few times and hangs on exit due to the memory finally being released and that also happens on live servers.

[DME543]

Bump

[Battlehawke]

I have a hard time believing that the Devs do not know about this. It has been discussed on the forums many times. I believe its just one of those things that is just not a priority to them, because you can fix it by just restarting DDO. For me its about the 7th or 8th time that the game is just unplayable anymore. Has anyone seen any kind of respinse from any Devs on this?

[DME543]

I have a hard time believing that the Devs do not know about this. It has been discussed on the forums many times. I believe its just one of those things that is just not a priority to them, because you can fix it by just restarting DDO. For me its about the 7th or 8th time that the game is just unplayable anymore. Has anyone seen any kind of respinse from any Devs on this?

The only response in this thread was it was read by a dev.

[Postumus]

I wish I'd seen this thread three months ago. It explains a lot of things.

[SisAmethyst]

Long before this thread started I was noticing issues and ran a defrag. I gained 16GB memory back on my hard drive even with a regular defrag on a 2 week schedule. I changed to a 1 week scheduled defrag and that helped.

I tend to agree with Junts that fragmentation can be an issue. I also know that frequent character swaps will produce the issue regardless.

I guess fragmentation of the disk is only an issue if the swap file is on that disk. Also fragmentation does not necessarily gain you space on a hard-disk, it just reallocate the space that more consecutive memory is available. Also important is to know that the default Windows XP defrag is not capable of defragmenting the swap space. Anyway, I have a whole disk with several Gig for my swap and the maximum of 4Gig of RAM for Windows and still I am able to get the system to its knees if I swap through several of my characters and then enter one of the bigger areas like Sands. If I even swap more characters I am even able to crash on loading e.g. the Harbor.

I also noticed that if I reduce my graphics settings that it seems to take a bit longer before crashing. Probably the reason why regions like Sands crashing with the graphic settings set to a large distance drawing as this mean that the engine probably load all the environment and mobs to check if they are visible or not.

On Monday I replaced the main HD for my computer. After copying back the image file, my new HD of course showed 0% fragmentation. My pagefile is located on a second physical drive and is of fixed size and can not therefore cause any file fragmentation.

The first thing I did was log into DDO, where I crashed after a series of character swaps.

Well, internally a pagefile still can be fragmented, you just don't see it as the Windows just show it as one big block. Try to increase the size of the pagefile, which at least may put the point of a crash a little further away.

[Velexia]

I remember this. It started when MyDDO came out, and is definitely linked to MyDDO. Ask Smatt, he'll tell you all about it.

I stopped noticing it, but that's because I upgraded my RAM to 8 gigs... Which doesn't mean that everyone just needs more RAM. It's still a problem, and they REEEALLLY should have fixed it... a long time ago.

[Zenthalas]

I remember this. It started when MyDDO came out, and is definitely linked to MyDDO. Ask Smatt, he'll tell you all about it.

I stopped noticing it, but that's because I upgraded my RAM to 8 gigs... Which doesn't mean that everyone just needs more RAM. It's still a problem, and they REEEALLLY should have fixed it... a long time ago.

Your unneeded and unsolicited personal attacks on someone shows your character. Now to the problem at hand... yes it has been happening since MyDDO came out, however the RAM on ppls computers is not the issue, I run w/ 8 Gigs myself and will still lag after several character swaps.