LAG and Server POPULATION a rational view

[math92]

Lag. Yep. A lot of Lag, lately. Since, SSG took over my connection to the game server is a lot worse. And I'm not talking about recent problems. In general... My ping isn't stable as it used to be. Anyway...

DDO didn't move on with age as other games did. Which makes me very sad. Despite the success the game had.(And still has for me.)

I'm not an IT guy but let's see.

*The code. Most likely on same formula as in 2006 when the game was launched.
*The game engine. Most likely weren't touched for improvements either and is very outdated with a lot of holes.
*The network system. How the player interacting with the server. Most likely still running on the same formula as in 2006.

I have heard one of the Devs saying ,,We don't fully understand the game code'' That was years ago...
Maybe fixing a few things within it would be very pricey and very time consuming. So what would you choose, fix mobs spawn lag or new expansion? The company has to still make money. You can't sell patch.

Sometimes I wonder, does it even make a profit for them considering how many active players we have? I think is half/half. Half profit, half just for the community.
They can unplug the thing anytime. We should stop *****ing around and all pay the sub bills. I own almost every pack, class, race etc... But, still a subscriber, because they deserve it!
If you unhappy go outside. Don't like it? Don't play it. Play something different.

Happy holidays to everyone.

[Rexthor007]

Lag. Yep. A lot of Lag, lately. Since, SSG took over my connection to the game server is a lot worse. And I'm not talking about recent problems. In general... My ping isn't stable as it used to be. Anyway...

DDO didn't move on with age as other games did. Which makes me very sad. Despite the success the game had.(And still has for me.)

I'm not an IT guy but let's see.

*The code. Most likely on same formula as in 2006 when the game was launched.
*The game engine. Most likely weren't touched for improvements either and is very outdated with a lot of holes.
*The network system. How the player interacting with the server. Most likely still running on the same formula as in 2006.

I have heard one of the Devs saying ,,We don't fully understand the game code'' That was years ago...
Maybe fixing a few things within it would be very pricey and very time consuming. So what would you choose, fix mobs spawn lag or new expansion? The company has to still make money. You can't sell patch.

Sometimes I wonder, does it even make a profit for them considering how many active players we have? I think is half/half. Half profit, half just for the community.
They can unplug the thing anytime. We should stop *****ing around and all pay the sub bills. I own almost every pack, class, race etc... But, still a subscriber, because they deserve it!
If you unhappy go outside. Don't like it? Don't play it. Play something different.

Happy holidays to everyone.

I Had A great time reading all the posts very intelligent very diferent from normal good work everyone.

[Clemeit]

I was never particularly fond of using server logins as a predictor of server population for a number of reasons, so I decided to do my own investigatory work. If you wish, you may compare this data to DDOracle to extrapolate any number of conclusions.



Thelanis. PST. Nov 24 through Dec 25.

Also, try to keep the conversation cordial. We're all fans of this game; we're all on the same side, even if we have differing opinions on the root causes and potential fixes of the subject matter.

Happy Holidays!

[Deadlock]

Run What Goes Up then tell me you get no lag

[Goalt]

I think everybody can just think back to around 2009 or so and know that the population has dipped dramatically. You don't even need data, observation is enough (just watch older videos in public instances). For example, I remember when the LFM list actually took a while to scroll through. And it was easy to fill Tempest Spine raids. And people ran full parties of quests like Durk's got a Secret to farm Muckbane. Shroud raids, anybody?

A ghost town except for the guys with like 40 past lives. That's what DDO is now.

[Clemeit]

I think everybody can just think back to around 2009 or so and know that the population has dipped dramatically. You don't even need data, observation is enough (just watch older videos in public instances). For example, I remember when the LFM list actually took a while to scroll through. And it was easy to fill Tempest Spine raids. And people ran full parties of quests like Durk's got a Secret to farm Muckbane. Shroud raids, anybody?

A ghost town except for the guys with like 40 past lives. That's what DDO is now.

This is partly due to the expansion and diversification of game content and the increased level cap. It's also partly due to non-public, exclusive grouping that many older players tend to find themselves participating in. The gap between novice and veteran players is ever increasing, especially since the introduction of Reaper difficultly. You have those that run r10s with little trouble, and those that tend to stay out of reaper altogether. This has led to a significant rift opening up between the "casuals" and the "elitists" with very little room in the middle, and many would rather look inward to congregations of like-minded players such as guilds for grouping than to post for a public group and be paired with people that don't necessarily play the same way they do.

This game's content is still growing - although I fear that with the push for cosmetic features and mounts, its growth might begin to wane. That aside, a growing game with a dwindling population leads to desolate public areas, difficulty filling raids, and less public group postings. It's an unfortunate truth that many are unwilling to come to terms with. When observation isn't convincing enough, turn to data. But when data isn't convincing enough? I don't have an answer.

As for me: I'm holding out. I hope that one day this game will see a revival and that I'll be around to experience it. For now, however, I'll keep my credit card in my wallet. Until I see some serious changes that get me excited for this game like I used to be, I'll save my money for other titles - ones that don't feel like single-player games.

[Fenrisulven7]

Is your argument that as long as some customers are happy there can't be a real problem with a service? That's what you seem to be saying.

No, I think he means the lag is not global, so maybe the problem is on your end.

I've haven't experienced any serious lag in the past two months. I'm playing about 40 hours a week, 2 or 3 times a day the game will stall for about 3 secs. That's been for me so far.

So I don't think it's fair to blame SSG for all of it, or fair to assume that your lag is a result of SSG not "spending money on necessary server resources". And how would you know anyway?

If you are getting all this lag and others are not, perhaps the problem is between you and your ISP, not you and SSG? Else, explain why players like me aren't seeing lag?

[Brutuscass]

Run What Goes Up then tell me you get no lag

Ha ha, you have me on that one, it can get a little glitchy in areas, they need to smooth thing out there, it's a bit like that area in ghost of a chance after you kill the name WF and pull the lever the return is always laggy, or like the ship area used to be in The Back lock.

As I keep saying I am not putting my head in the sand and be game lag denier just offering suggestion that might help some have a better gaming experience, there are indeed some parts of the game that even with the best set up it can get bouncy on a bad day


Happy Hols

[liston33]

My experience with lag in this game is when i play in a group... The lag cripples all players in that group... I play with friends from all over the world and they experience the same lag at the same time.. So unless all the players in that group have the same equipment and internet connection i don't see the finger pointing in that direction.. I see it pointing at the server side...

[myliftkk_v2]

Thank you for taking the time to respond here, now I am not going to challenge you by questioning your programming knowledge, I have never made claim to being an expert, hell I wouldn't even know how to reconfigure the oracle account/pass combination via a Jenkins configuration interface.


However my post was simply made for the purpose of


1,addressing the on going question of population by simply providing access to the closest source that can be found out there atm, rather than the generalise subjective opinion that is inflames the forum now and again. It is not perfect and I make no claim to it being so.


2,To offer suggestions which may help reduces the effects of lag that some may be experiencing, (Am I wrong in trying to do so?)
Iam NOT stating that lag is never present in the game, though I am stating that I used to have a much greater issue with “Lag” than I do now, which is almost non existent now after I looked into possible contributing factors.


The two issues were not meant by me as being thought of as one causing the other. I intended them as two separate issues and at no point in my post suggested a causal link.


As you seem to be in the know when it comes to the field of development and coding in respect to servers and lag, may I trouble you with a question?


Is the cause of Lag always one source, if not can multiple causes of lag increase the overall effect and compound it?
And if that is the case then would it be best practise to try to reduce or remove any extra lag causing effects if possible, or in your professional opinion is it truly a case that the patient is dead and there is nothing to be done.


Are all the links I provided talking rubbish when it comes the subject of lag?
I am just asking here as a pint of beer is riding on this from my friend James.


One last question, what is the given value of X for you?
And I truly do not wish to upset anyone here but I play a lot, generally a few hours a day and though I have had to force a shut down due to zoning issues I find it reasonable rare.
Again I am not claiming to be an expert on anything just talking of my personal experience.

I'm not questioning the motives for posting, nor should a comment on the relevancy of a variable in data processing be taken that way. There are lots of variables when it comes to a client-server data processing loop, and we can deep dive into each one, but if we step back from the trees and look at the forest, there's only of handful of trees that are going to make up the colloquial lion share of the relevant data processing components most often where a potential delay lies.

There's not a right or wrong here. There's only a question of relevancy (although people often ascribe moral weight to it because of their personal investment in it, it's not actually related to morality at all).

Is the cause of lag only source? A delay is caused by whatever a delay is caused by. Whether or not it's a single factor, let's say a singularly inefficient database call or a multiplicity of calls that add up to a wildly inefficient processing loop only in particular combinations, or your isp throttling your connection because you're also download 10 movies and 3 games at the same time, that would answer the question, correct? We'd never know unless someone knew the technical details of the entire client-server loop as it happened and deigned to share them with us. False claims of universal certainly, aka "always" are how we end up arguing about saplings instead of looking at the towering redwoods. Thus why we should only look at relevancy.

Best practices. If you go to the doctor as an adult and complain of a headache, he could tell one to stop hitting onesself in the head with a hammer, but what's the likelihood that's what one is actually doing? It's plausible, and one certainly could be experiencing headaches from such an activity, but really, if that were the case, is it likely someone would have landed in the doctor office as an adult with that habit? So for a fleeting instant they consider the theory and then reject it out of hand as not relevant before they've even processed the image of one in front of one. My baseline assumption is most players who've managed to download, install, and execute DDO on a regular basis are at least savvy enough to not be downloading 10 movies and 3 video games at the same time they are trying to play. It's plausible that they paid for the cheapest internet connection, one that is actually gated far lower than their machine available data pass thru capacity. It's also plausible they just decided to kick up their graphical settings to the most computationally expensive options. But, all bundled together, what is the likelihood of any of that? Of course someone does those things, thus why we don't use terms like always, but it's also highly unlikely all players did those things, or any player does all of those things. For the purposes of winning a bar argument, yes one can say all of those things are possible.

Login info is useful. But data can tell whatever story we want to tell with it, so when we use it we should provide more context as to what we think its relevancy is. The login captured info is very, very high grain, and there are a quite limited set of relevant inferences we can draw from it, but there are a few.

1. One is that total logins patterns are quite stable on a chunk of servers at least for the quarter (which is a really short period of time, btw, as it should keep data for all time available).
2. Cannith is trending downwards
3. Some event precipitated a jump in Orien logins
4. Hardcore server trended downwards pretty sharply before the actual end of the server. This actually matches my anecdotal experience with the HC server in that once my toon died around like level ~10-12 I think in early Sept, I said f all that and never logged into it again. It's possible my experience in losing a toon when almost to the first favor gate was an outlier, but I wager it wasn't really. I'll clarify it wasn't my first toon lost on HC, but it was decidedly much, much more painful knowing I was close to the first favor gate.
5. Wayfinder is trending downwards.

The problem is nearly all of those trends are pretty amorphous without the following pieces of info. Who logged in, and for how long were they logged in. That would eliminate a large amount of the ambiguity around re-attempts, play length, etc. The logins on steamcharts are similarly stable, but of course, much different in that they represent single players so we do have an accurate picture on how many players come to the game from steam and it's a lot less than when the game was launched on steam.

All to say, unless one is very specifically on Orien, one should not be experiencing any more delays in processing due to server population fluctuations than would be a near term historical average. So not rubbish, so do collect your pint, but that data may not be saying what one thinks it says about delays.

X has become ~4-6 zones. I will say at a minimum X has become notable, where in the past, it was far, far, less prevalent. Any play session over maybe 1 hour is going to have at least 1 crash. Is that a problem with my setup? Not at all, considering I can run every other game with heavier graphical loads Nioh, Dark Souls III, Sekiro, etc, etc, with nary a crash ever. That this happens at all, from a development standpoint, is pretty egregious data processing behavior, considering code is either attempting to get outside Windows allowable bounds (and thus my platform is killing off a bad actor), or it's own internal platform cannot elegantly recover from what in all estimations is a memory leak.

Now, that in itself is an indicator that delays aren't likely to be the technological stack on which DDO executes, but instead DDO itself. If DDO crashes multiple times per play session, while I can leave Excel, Acrobat, an IDE, Chrome, and other apps up and running for days at a time without them ever crashing, then the odds that a delay problem exists in the technology stack are not nothing, but they are small. And beyond, if that is the behavior of the application that my client has, what are the odds the server code is qualitatively of different quality?

All of our rules of development thumb point to directly at server code being at issue here, except for the aforementioned client issue which exhibits signs of a good old fashioned memory leak, especially in light of the weakness in the client code with respect to similars.

[myliftkk_v2]

No, I think he means the lag is not global, so maybe the problem is on your end.

I've haven't experienced any serious lag in the past two months. I'm playing about 40 hours a week, 2 or 3 times a day the game will stall for about 3 secs. That's been for me so far.

So I don't think it's fair to blame SSG for all of it, or fair to assume that your lag is a result of SSG not "spending money on necessary server resources". And how would you know anyway?

If you are getting all this lag and others are not, perhaps the problem is between you and your ISP, not you and SSG? Else, explain why players like me aren't seeing lag?

Actually, SSG should be the one explaining why you're not seeing lag. If they had the toolset to do so, or the game was designed on a platform offering that level of insight, it would be easy to establish the variances between play sessions that result in lag, and those that don't.

A working assumption is most players are savvy enough to know whether their ISP is the problem or not. At a minimum, one can tab out of the game and pull up any number of websites and see if the delay is similar. In addition, most ISPs are competitive because they upgrade their equipment to keep their customers happy so the issue of ISP generated delays decreases in frequency over time (we'll see once net neutrality is completely done away with though).

[Fenrisulven7]

A working assumption is most players are -

You make make valid and bias free arguments. Thank you for that.

I'm curious what you think accounts for how the Cannith server almost mirrors the other servers (except Orion).

I would assume(?) a server that new players are routed to would comprise of mostly unique log ins - a new player on his 1st toon (1), not a vet swapping logs between 6 alts and 12 mules (18ish)

Yet, aside from the higher value, Cannith almost mirrors the peaks/valleys of the other ones. What am I missing here? Maybe Cannith has many more vets than assumed?

[myliftkk_v2]

I was never particularly fond of using server logins as a predictor of server population for a number of reasons, so I decided to do my own investigatory work. If you wish, you may compare this data to DDOracle to extrapolate any number of conclusions.



Thelanis. PST. Nov 24 through Dec 25.

Also, try to keep the conversation cordial. We're all fans of this game; we're all on the same side, even if we have differing opinions on the root causes and potential fixes of the subject matter.

Happy Holidays!

It would make sense to blend this data, and we could further narrow the gates on max players and see what that tells us in theory:

1. Your max average is 250 on one server, a server that has a stable login pattern similar to 4 other servers, so it might be reasonable to theorize that 1250 max concurrent players exist on average across 5 of the servers.
2. Cannith has a login pattern nearly double those four, while Wayfinder is nearly half. So add in another 500 and 125 players respectively, upping us to 1875 players.
3. Let's split the difference for Orien since the pattern itself splits, 375, and thus gives us 2250 players.
4. We can throw out the HC server since it went to zero, but doesn't seem to affect logins in other servers either during or after it's cycle (or maybe its population jumped to Orien).


Inference gives us a max 2250 concurrent players at a given moment (please note this is not total, nor does total player really matter, since player concurrency is how players define a healthy MMO) spread across 8 servers with ~400 quests available to play (a conservative quest count in 2018 was ~382 and it's come up since then). Combining all this gives us an interesting server average player per quest of .70. Suggesting the odds, irrespective of content appeal, of 1 player being inside a quest on a given server at a given time are ~70%.

Now, let's back up to 2012, when Steam had their average concurrent player count at 1368. Holding all things steady, let's add the difference of ~1300 - 500, or 800, to our concurrent max average (pretty safe to say they didn't move from Steam to the stand alone client) giving us 3050 max average (of course we've no idea the client population dip from that same period). Without altering anything having to do with quest count, adding in that population brings our server average player per quest up to .95., or in other words, we've experience ~27% drop unrelated to content growth, just on the side of steams contribution to concurrent players.

If we add in the delta from both the increase in quests that drop becomes even more precipitous and if we incorporate level settings into the mix (throwing out casual) we get

2019 - 2250 average concurrent players, 8 servers, 400 quests, 3 level setting + 10 Reaper = ~.055 or 5.5% chance of a concurrent server player per quest difficulty setting.

2012 - 3050 average concurrent players (only adding in Steams contribution), 8 servers, <300 quests, 3 level settings = ~.42 or 42% chance of a concurrent server player per quest difficulty setting.

Or, if we to turn back time and randomly log into any server and select any quest and difficulty there would be theoretically an ~87% less chance of there being a player there than there was 7 years ago.

The root problem is population growth is stagnant while world growth is not. DDO is a world where old systems or modules are never retired completely or cycled off and so dispersal of the existing stable population across a growing world leaving every player, whether their choice or not, more isolated than before (unless they themselves takes steps to counteract). Hence, the single-player game feel increasing while the social-player feel decreasing.

[FlavoredSoul]

OP has no clue what he is talking about.

Lag is almost NEVER the endusers fault.

In DDO we have many specific examples of things where lag is fully on SSGs servers.

Every single player gets crippling lag in specific content: thunderholme being the worst offender, get a full raid group in there and the game servers will cry and die. There are many possible reasons for this: large number of active mobs in general, content incorporeal/phasing mobs and to a lesser degree teleporting mobs both have particularly bad effects on game performance.

[myliftkk_v2]

You make make valid and bias free arguments. Thank you for that.

I'm curious what you think accounts for how the Cannith server almost mirrors the other servers (except Orion).

I would assume(?) a server that new players are routed to would comprise of mostly unique log ins - a new player on his 1st toon (1), not a vet swapping logs between 6 alts and 12 mules (18ish)

Yet, aside from the higher value, Cannith almost mirrors the peaks/valleys of the other ones. What am I missing here? Maybe Cannith has many more vets than assumed?

We'd likely need to look back at a much longer time period to gain clearer insight into the difference. Cannith could have been a default server for longer, or there could have been a wave of movement, or a number of things.

Whatever happened to push Cannith up, we can see a slowly sloping trend headed towards the other servers, so it would be prudent to try to go backwards to find the jump and see what event or confluence of events could have precipitated it (and ideally replicate the event(s)).

The most important insight however, could be that Cannith quite possibly feels twice as populated as 5 other severs, and 4 times more populated than Wayfinder. This is actually important because the social environment feeling is a key part of the MMO experience (I'd argue it's integral to DnD, but that's more of a discussion on the principles of DnD). While DDO might deliver the same technical game on all 8 servers, the experienced game would appear to not at all be the same.

[Letrii]

Actually, SSG should be the one explaining why you're not seeing lag. If they had the toolset to do so, or the game was designed on a platform offering that level of insight, it would be easy to establish the variances between play sessions that result in lag, and those that don't.

A working assumption is most players are savvy enough to know whether their ISP is the problem or not. At a minimum, one can tab out of the game and pull up any number of websites and see if the delay is similar. In addition, most ISPs are competitive because they upgrade their equipment to keep their customers happy so the issue of ISP generated delays decreases in frequency over time (we'll see once net neutrality is completely done away with though).

The problem can be somewhere in between server and player, meaning neither party is at fault. That's what it was in past when they had to get traffic rerouted around some chokepoints.

[nokowi]

If you are getting all this lag and others are not, perhaps the problem is between you and your ISP, not you and SSG? Else, explain why players like me aren't seeing lag?

The answer is throughout the thread, if you just read the responses.

While an individual may receive lag if there is an ISP issue, or windows update on their end, there have always been places in DDO where lag occurs, or situations where lag occurs, and these have always been repeatable. Examples include Shroud chest item spawning, blades in DOJ, shadows, red alert, and heavy use of the crafting hall.

People have described their entire group receiving lag at the same time, which indicates an issue at SSG's end.

Other people can accept your statement that you don't see much lag and attribute it to being a solo player, not running at peak lag times, or not running content that causes lag.

What we don't know is how you can ignores other peoples first hand experiences when they are easily explainable, and have been explained to you. SSG has listed player abilities as causes of lag over the years (EIN nerf, too many particles from ranged attacks, issues with aura's, etc), and to think they are not currently a significant cause of lag is just plain incorrect. They routinely list mob pathing as a big source of lag.

So given that SSG and other players have listed some of the key reasons, and correctly explained the symptoms people have expressed, how does your experience trump everybody else's, and also the direct statements from devs?

Saying not all lag is SSG's fault is pretty meaningless, as we want SSG to work on the portion of lag they control - and which we know to be significant from the context above.

[Brutuscass]

I was never particularly fond of using server logins as a predictor of server population for a number of reasons, so I decided to do my own investigatory work. If you wish, you may compare this data to DDOracle to extrapolate any number of conclusions.



Thelanis. PST. Nov 24 through Dec 25.

Also, try to keep the conversation cordial. We're all fans of this game; we're all on the same side, even if we have differing opinions on the root causes and potential fixes of the subject matter.

Happy Holidays!

FIRST OF ALL AND OF UP MOST IMPORTANCE HAPPY HOLS

Thank you for taking the time for this response great work and effort +1 to Rep.

Is there any chance that you could provide a few more details on the methods you used for data collection, the best I can come up with is doing snap shoots of the who page, which is a little time consuming.

For example I took these 3 at 21:00 GMT today, Boxing day Thurs 26th.

[IMG][/IMG]

[IMG][/IMG]

[IMG][/IMG]

I only took the 3, each one has 16 names on it, looking at the side bar I guess at about 17 and a bit pages, so a rough guess here gives me around 270 online who have not checked/ticked the Anonymous box.

ofc these numbers will change over time and shifts will swap, and many peps might be try to work off all that food by playing the game instead of taking the dog for a walk

[nokowi]

I was never particularly fond of using server logins as a predictor of server population for a number of reasons, so I decided to do my own investigatory work. If you wish, you may compare this data to DDOracle to extrapolate any number of conclusions.



Thelanis. PST. Nov 24 through Dec 25.

Also, try to keep the conversation cordial. We're all fans of this game; we're all on the same side, even if we have differing opinions on the root causes and potential fixes of the subject matter.

Happy Holidays!

Thank you for your efforts!

[Brutuscass]

Actually, SSG should be the one explaining why you're not seeing lag. If they had the toolset to do so, or the game was designed on a platform offering that level of insight, it would be easy to establish the variances between play sessions that result in lag, and those that don't.

A working assumption is most players are savvy enough to know whether their ISP is the problem or not. At a minimum, one can tab out of the game and pull up any number of websites and see if the delay is similar. In addition, most ISPs are competitive because they upgrade their equipment to keep their customers happy so the issue of ISP generated delays decreases in frequency over time (we'll see once net neutrality is completely done away with though).

And with any luck they will provide better responses concerning LAG to better debate, which I am happy to see is starting to develop here, and thank you again for your contribution.
It will be easier for them to add to a discussion which is logical rather than passionate.

And I am sorry that your gaming experience is lessen to a greater extent than mine, truly I play a hell of a lot, and am not seeing the nightmare which is being describe by others, yes there are areas in the game that do need to be improved and there can be the odd lag spike, but as for the crashes at the rate you are stating, truly if I were getting anywhere near that I would take some time off and start up a Total War campaign or two to fill my time, till it was sorted

I am a little confused though by your statement that "SSG should be the one explaining why you're not seeing lag" I imagine that you are saying that if they can, using a medical situation find what is keeping some healthy and use that to treat others.
I would prefer that they spend their time on continuing to improve the game for everyone than search for a panacea.

Anyway with any luck most of this will have become a lesser issue by February, though there will be a fair bit of moaning in January (as long as they remain on their time table) when they bring down the servers two at a time for up to 48 hours, something that any number of conclusions could be drawn with enough wild speculation.