The Existential Lag Problem: What can truly be done?

[DistantOceans]

The heart of the good and why people hold out hope: Ddo is a really good game, the gameplay mechanics are great, very fun, and highly playable and the community surrounding this game is excellent and highly committed. I’ve met so many great people playing DDO and I’m sure there are many more people who also love this game, and have put countless hours into maintaining and improving it. Whatever frustrations I feel are mediated by the fact that I love playing this game. As a newer player, who’s become invested in this game, I want to see it continue to grow and develop.

The single biggest issue with DDO relates to server performance (i.e. lags, freezing, zone’s not loading, etc.) which effect every single person who interacts with it. These issues have been explored extensively by others, and anyone who plays this game for any serious length of time is aware that they exist, and that there are multiple complex issues that cause them. I think because the way communication surrounding the game is structured, it’s difficult to perceive when there is a consensus around a single issue, but I would argue that addressing server/lag and freezing is the most universally agreed upon issue with the game among players and developers.

Most of these are not easily remedied or addressed, which has led to frustration from both the player base and developers. Over time, the chronic nature of these issues have put strain on the relationship between the company, player’s, and developers, and likely caused a breakdown of communication between groups. This has been further been exacerbated as conventional channels for communication between player, company, and developer have shifted. Player’s direct their frustrations through the forums, and the help ticket system, but because these systems are not designed to address these types of issues, the receive little attention, leading to a cycle of frustration on the part of players often directed towards the developers who are are also not in a position to address the issue of lag, outside of game design and attempts to approve the software. While optimization is likely to require ongoing efforts, what is also clear is that the network performance issues related to DDO are in part to hardware issues related to the servers and bandwidth allocated to maintaining the game. If this is true, that is a good thing, because they are issues that are imminently solvable by upgrading or purchasing better hardware/servers.

The interrelated core issues seems to be communication between all the parties involved, and their ability to effectively communicating with those who have agency. Furthermore, because the issue of server lag transverses multiple interrelated departments and spheres of control any effort to address the issue leads to a breakdown. Player’s direct their frustration towards developers, developers have to work within their domain’s and direct issues to their lead developer, the lead developers need to moderate between their teams tasks and the corporate elements of SSG. The corporate representatives of SSG need to communicate the needs of the developers to their supervisors, who again needs to communicate the issue to whomever can address hardware issues related to the game, which again, needs to work with what is happening with the issues the developers are trying to address. In order for anything to substantively change around this issue, each point of that chain needs to be functioning properly. Being as I don’t work for SSG, I have an imperfect understanding of how this is currently set up.

So in good faith and good hope, I ask the eternal question:

What can be done?
Who do we need to ask?
What do they need to make it happen?
What can we the players do to help make that a reality?

And if it’s something that can be solved just by throwing money at the problem, for the love of god just put of the kickstarter up and consider it funded.

[Dark_Lord_Mary]

The heart of the good and why people hold out hope: Ddo is a really good game, the gameplay mechanics are great, very fun, and highly playable and the community surrounding this game is excellent and highly committed. I’ve met so many great people playing DDO and I’m sure there are many more people who also love this game, and have put countless hours into maintaining and improving it. Whatever frustrations I feel are mediated by the fact that I love playing this game. As a newer player, who’s become invested in this game, I want to see it continue to grow and develop.

The single biggest issue with DDO relates to server performance (i.e. lags, freezing, zone’s not loading, etc.) which effect every single person who interacts with it. These issues have been explored extensively by others, and anyone who plays this game for any serious length of time is aware that they exist, and that there are multiple complex issues that cause them. I think because the way communication surrounding the game is structured, it’s difficult to perceive when there is a consensus around a single issue, but I would argue that addressing server/lag and freezing is the most universally agreed upon issue with the game among players and developers.

Most of these are not easily remedied or addressed, which has led to frustration from both the player base and developers. Over time, the chronic nature of these issues have put strain on the relationship between the company, player’s, and developers, and likely caused a breakdown of communication between groups. This has been further been exacerbated as conventional channels for communication between player, company, and developer have shifted. Player’s direct their frustrations through the forums, and the help ticket system, but because these systems are not designed to address these types of issues, the receive little attention, leading to a cycle of frustration on the part of players often directed towards the developers who are are also not in a position to address the issue of lag, outside of game design and attempts to approve the software. While optimization is likely to require ongoing efforts, what is also clear is that the network performance issues related to DDO are in part to hardware issues related to the servers and bandwidth allocated to maintaining the game. If this is true, that is a good thing, because they are issues that are imminently solvable by upgrading or purchasing better hardware/servers.

The interrelated core issues seems to be communication between all the parties involved, and their ability to effectively communicating with those who have agency. Furthermore, because the issue of server lag transverses multiple interrelated departments and spheres of control any effort to address the issue leads to a breakdown. Player’s direct their frustration towards developers, developers have to work within their domain’s and direct issues to their lead developer, the lead developers need to moderate between their teams tasks and the corporate elements of SSG. The corporate representatives of SSG need to communicate the needs of the developers to their supervisors, who again needs to communicate the issue to whomever can address hardware issues related to the game, which again, needs to work with what is happening with the issues the developers are trying to address. In order for anything to substantively change around this issue, each point of that chain needs to be functioning properly. Being as I don’t work for SSG, I have an imperfect understanding of how this is currently set up.

So in good faith and good hope, I ask the eternal question:

What can be done?
Who do we need to ask?
What do they need to make it happen?
What can we the players do to help make that a reality?

And if it’s something that can be solved just by throwing money at the problem, for the love of god just put of the kickstarter up and consider it funded.

something needs to be done - the lag on Hardcore League IV is killing very good players - its totally lame to watch my friends get frustrated and leave because of the game crashing, studdering, lagging, freezing, and rubberbanding in quests while the mobs just kill you and you can't do a thing

[Avocado]

DDO2 with different Engine, well thought out coding, non budget servers and hire professional people that actually know how to do these things. I'm not sure the financial sit for DDO, but they need to reinvest the money they make back into a better experience. It currently seems they are working with a skeleton crew. Bite the bullet and go negative in profit so you can get more profit later because your game isnt a buggy unprofessional mess.

[Justicesfury]

DDO2 with different Engine, well thought out coding, non budget servers and hire professional people that actually know how to do these things. I'm not sure the financial sit for DDO, but they need to reinvest the money they make back into a better experience. It currently seems they are working with a skeleton crew. Bite the bullet and go negative in profit so you can get more profit later because your game isnt a buggy unprofessional mess.

This is a 15 year old game. They have been pulling money out not putting money in, until the wheels fall off. We are the last 5000 or so still here. They talk big to keep people paying and dangle the carrot of hope on what they will be fixing. Let's face it, those of us still here have gone down with the ship.

[Knightrose]

It boggles the mind that lag has not been properly addressed in this game. This is years of suffered play and lost customers. Fixing the lag on ALL servers needs to be made priority. The constant BS excuses is getting old. I simply wont retain my VIP anymore. Countless friends and guildies have left DDO SOLELY due to the lag.

FIX.

IT.

[Telekinesis]

Simple fix is everyone not spend any money until they can resolve these issues.
Nothing, don't spend a cent. Force their hand, it obviously they will just keep taking or money and putting into rubbish expansions to make a quick buck or saying they have employed more people...it bs and always has been. On behalf of the community write a letter to the board members. If the game dies, the community can crowd fund and take over.

[lion_mouse]

I don't think it is hardware related - well not mostly as SSG has brought in old coders to try and answer where the lag is coming from - cause I don't think they actually know. However because they are such **** pore communicators that is the extent to which we actually know - the rest is all trying to work out where the problem from the outside without access to the coding or what they are actually running the servers on (could be a Micro B) for all we know.

So we are guessing where the problem is - and SSG has not updated us since their big announcement.

[Justicesfury]

I don't think it is hardware related - well not mostly as SSG has brought in old coders to try and answer where the lag is coming from - cause I don't think they actually know. However because they are such **** pore communicators that is the extent to which we actually know - the rest is all trying to work out where the problem from the outside without access to the coding or what they are actually running the servers on (could be a Micro B) for all we know.

So we are guessing where the problem is - and SSG has not updated us since their big announcement.

Remember the mega server announcement? Now it's not happening. We are beyond announcements. We need results.!

[Avocado]

This is a 15 year old game. They have been pulling money out not putting money in, until the wheels fall off. We are the last 5000 or so still here. They talk big to keep people paying and dangle the carrot of hope on what they will be fixing. Let's face it, those of us still here have gone down with the ship.

I play DDO because its sunk cost fallacy at this point, how sad.

[Inanout]

If someone has spent lots of money on the game, they can still play for free for a while and not feel guilty.
Over half the population it seems contributes little in the way of money already.
I am back to paying VIP monthly so I can vote with my wallet.

[Memnir]

What can be done?
Apply more resources to the problem. In this case more money for better servers.

Who do we need to ask?
Nobody who we haven't, sadly.

What do they need to make it happen?
Money, and lots.


What can we the players do to help make that a reality?
Not a bloody thing beyond continue to express our unhappiness with the current state and/or vote with your wallet.

And if it’s something that can be solved just by throwing money at the problem, for the love of god just put of the kickstarter up and consider it funded.
Sadly, while it could possibly maybe theoretically happen... it won't. For a host of reasons, most of them boring and legal. Also, I think the chances it would hit whatever funding goal necessary are depressingly slim.


As players, we're kinda a bunch of Stockholm Syndrome-y passengers on this train. We can get off the train at any time - they're not stopping us. But most of us who are still here are going to be along until the end of the line. The only real question is will that be a derailment, a head-on crash, or a surprising twist where the issues are fixed and it's a nice trip through pleasant countryside.

[AbyssalMage]

What makes you think SSG plans on fixing it?

As long as it exists developers can use the excuse "that it causes lag" to implement a nerf. They can use "lag" as a reason not to implement QoL improvements.

O'well, my 2cp's on why they will never actually address lag in a meaningful way.

[Dejvid]

I play DDO because its sunk cost fallacy at this point, how sad.

true that, same for me.

but i also wont invest any more of my money into it until i see some results. at some point you have to cut your losses and stop beeing a fanboy and defend every single mistake.

the lag issue is mindboggling, seeing how other games manage to play smoother with way more players doing things at the same place. especially those big pvp battles in say gw2 or eso. hundread of players spamming skills and effects and whatnot, yet less lag then doing a solo harbour dungeon...

but lag is only one issue, albeit the biggest one. i can tolerate bugs and technical issues, those happen in every game, but when the experience is hampered by lag, freezes, timeouts, desyncs (in my opinion the biggest issue, notable while jumping, or climbing ladders, its like the game does not care where you are!) its starts to exterminate the fun quickly.

and you can see the exact same issues with lotro as well. given its the same engine and all, its still different enough in terms of how many players are playing together, but still manages to have the identical lag problems.

[phillymiket]

DDO2 with different Engine, well thought out coding, non budget servers and hire professional people that actually know how to do these things. I'm not sure the financial sit for DDO, but they need to reinvest the money they make back into a better experience. It currently seems they are working with a skeleton crew. Bite the bullet and go negative in profit so you can get more profit later because your game isnt a buggy unprofessional mess.

Yeah I got driven out of the game by the lag tonight soloing off-hours. It's rough.
Worse than it was a few years ago for sure even though the population has to be much less.
I've maybe had more game freezes in the last week than all rest of my years playing put together.
I don't think that's an exaggeration. Every few hours there's a freeze while switching toons or zones.

DDO2 would be a dream. Could it be done? I doubt it.

Sure, plenty of other games milk their old versions while offering new versions. The Sims, Civilization, etc, etc.
But D&D isn't owned by SSG but rather by a company that guards their intellectual property like an elder Red Dragon.
I have to believe that the price for the licensing has gone through the roof or maybe isn't even available for a DDO2.
Why else wouldn't they have re-upped this thing years ago?

I suspect we'll never have a DDO2 and even if we did, let's face it, it won't be the same gameplay or community.
I also suspect they won't be doing too much upgrading or despaghettifying either.
DDO, even "now with less crippling lag" DDO, just isn't going to be a hot item on many people's wish list.

Our only hope is for some miraculous tech breakthrough that allows the servers to perform much better w/o costing too much.
Or maybe someone finding an errant "20 goto 10" that solves the issue.
If that happened, the die-hards could keep the lights on for a while. But, honestly, I wouldn't hold my breath.

Such a shame because there's nothing like it.
I was hoping NW would be the thing. Nope. I was hoping BG3 would be the one but I'm just not feeling it.
None of them are as good as old-as-dirt DDO (when working properly, that is).
Such a shame.

[Seph1roth5]

Maybe people that quote entire giant posts are contributing to the lag too, sure is lagging my reading of the forum.

[Cordovan]

We have been and will continue to work on game performance improvements, and this next week will be a particularly important one for both you and us. We are working to get a preview of some game performance improvement changes on Lamannia this week, which you can read more about here in the coming days (https://www.ddo.com/forums/showthread.php/523284-U49-Preview-2-4-06-4-08).



Changes focused on boosting game performance (A specific call to those that have a raid group and are able to spend some time putting these through their paces when this opens up. High end characters and Raid game performance, and as a dominoe effect general game performance, is expected to be smoother with these changes so as much real player load available to try it out will be good.)


For those of you who are willing to put in a bit of time to help, we would very much appreciate it if you'd check things out later this week on Lamannia.

[thunir]

Been playing this game for 12 years and thousands of dollars spent, I'm sure.

Its never been this bad. What is going on.

[Sarkastik]

Been playing this game for 12 years and thousands of dollars spent, I'm sure.

Its never been this bad. What is going on.

It's a 15-year old C++ codebase that has continued to incrementally grow far beyond what was ever originally imagined, that is what is going on. I've seen this type of beast many many times in my career. There's never an "easy" fix.

Neophytes often think that simply throwing hardware and more (ahem, "good") programmers at the problem is the way to fix it. It's not (https://en.wikipedia.org/wiki/The_Mythical_Man-Month).

The only reasonable course of action is for SSG to dedicate a core of 1-3 knowledgeable engineers to dig through the monster and very slowly and carefully re-engineer the core things that need addressing. With an aging codebase like this, there's a huge amount of risk involved in terms of unintentional side-effects, so there's a need for extreme care and patience from everyone involved. All we can do is hope that this is indeed what SSG intends to do. From the public statements I've seen, I believe they are trying.

[Hawkwier]

Lag is bad, no doubt, and I feel for those losing toons on HC (I don't play HC so can't comment directly, but sympathise, I would be furious were it me).

There have recently been a number of dev posts confirming they are looking at the lag, which is good to see, and I'm prepared to give them time to sort it.

I really enjoy the game but its been going into stop-frame animation during pretty much all combats for me recently and much as I love it, I can't continue to play a game I cant actually play.

Fingers crossed for success on this one. Would be a brilliant result if it could get sorted.

[myliftkk_v2]

It's a 15-year old C++ codebase that has continued to incrementally grow ... I've seen this type of beast...

Funny how Tiamat ended up in the game even through they didn't plan on it.

I would agree they need to hire about 2-3 top tier engineers and maybe the code Tiamat can be slain for glory and loot. That kind talent though, does not work cheap.

[DistantOceans]

I really do believe that the dev's try to do what they can, and was pleased to at least see it acknowledged in the new patch notes. Again, that aspect is going to take time, skilled programmers and be an ongoing thing.

There's alot of the same traps that I think seem to come up whenever this topic resurface which often run along the lines of:
- stop giving SSG money out of protest
- shout louder until the dev's do something about it
- give in to inevitable hopelessness and accept that things are in decline

I really do understand why people end up in that line of reasoning, it's really easy from the individual level to feel pretty powerless to do anything about it, but I also think that maybe because we tend to fall into these mental traps we're missing something in terms of how to mobilize a player base that has more to do with collective action than any real software/communication issue.

The best solution I could think of to get SSG to mobilize more resources for DDO to address the problem was to look into drafting a petition and gathering signatures of developers, guild leaders, VIP subscribers, and members of the player's council to try to send a clear message of what the player base wants.

I think it's really important how you ask for things too, if people feel like they're being forced to do something, they're gonna do it disingenuously and likely not actually address the issue, which also means that we need to communicate to SSG that addressing the problem is in fact in their interest (be that reputational or financial).

I also think that whatever message is sent to SSG it needs to be things that are clear and actionable, which mean we also need clear information from the Devs of what type of support is going to make a difference.

That's my working theory right now.

[phillymiket]

There's alot of the same traps that I think seem to come up whenever this topic resurface which often run along the lines of:
- stop giving SSG money out of protest
- shout louder until the dev's do something about it
- give in to inevitable hopelessness and accept that things are in decline

It's been literally over a decade of the forums unrelentingly trying trap #1 and #2 to no avail.
You could probably count on one hand the number of days out of the last 5000 without a page-one lag thread going.
So I've consigned myself to sit back and enjoy trap #3 like hireling basking in an acid jet spray.

The Arc of The Universe has been bending predictably in the same direction forever.
Maybe the lag fix will help, maybe it won't, but we're just passengers and won't effect the outcome.

I'm no program guy, but lag fixes - which always involve lessoning the calculations the game must make - seem to my layman eyes at odds with understandably needing to continually add new features to make money. So I figure even a best case scenario won't change the curve of the arc.
But hey, if it does, I'll be on here cheering.
Until then I'll continue to put my favorite game in the "it is what it is" category.

But I do feel strongly that if you're going to shake your fist, shake it at the sky, because it's not the Devs fault.
The issue has persisted from Dev team to Dev team, from parent company to parent company.
I'm sure they're bailing water and keeping this leaky dinghy afloat as much as possible given the circumstance.