Game Performance Improvements and Feedback

[Sam1313]

1. Server - Cannith
2. Approximate Time - 10:00pm EST to 5:30 AM EST. <<<-----This is in the AM NOT peak population time for Cannith.
3. Approximate location of your computer - Lafayette, Indiana (Midwest)
4. Quests/raids run - Siber Atoll chain & Reign of Madness chain
5. Difficulties run - Heroic Elite.
6. Your character class - Paladin/Rogue
7. Number of players in the group - 3
8. Anything else you think we should know - LAG! LAG! AND MORE LAG! Tried to run a guild member new to the siber atoll chain through it . Started with Prey on the Hunter, Made it to the end boss fight, Could not save the Dragon. Killed the giants trying to kill the dragon but we kill them all and 3 seconds later the game tells us we failed. EVEN THOUGH WE HAD KILLED EVERYTHING BUT THE exactly 3 whole seconds before the darn dragon drops dead.

[boltzman]

server - Thelanis
9:23-40pm PST
Temple of Deathwyrm EH
7 players in group - 2 clerics and 5 melee
multiple lag spikes from 3 to 30 seconds
both of the rubberbanding and frozen variety

[Shaelyn]

I just tried to short man a shadow dragon. We were 2 clerics, one halfling rogue, and 3 wolves. We had one lag we started timing after it lasted so long and at that point it went more than a minute! At least most of the trash lagged too, but eventually we party wiped. I had a jibbers so I was able to raise several people and we recovered before getting the dragon down enough to get shadows and get over when we again lagged and wiped. I am a VIP player since the second beta, and have a lot of friends in game and love this game and I'm seriously thinking of quitting. Most of my guild has quit playing and might not come back if you don't get this fixed. I can foresee the death of the game if it stays so unrunnable. I really don't want to see it go like this. PLEASE get this fixed!

[Sam1313]

Report for 4/9/16


1. Server - Cannith
2. Approximate Time - 2:00pm EST to 2:30 AM EST.
3. Approximate location of your computer - Lafayette, Indiana (Midwest)
4. Quests/raids run - Running anything I could with friends who I invited over to my house today for a day long gaming session.
5. Difficulties run - Heroic Elite./ Epic Elite
6. Your character class - Paladin/Rogue / Epic Ranger lvl 26
7. Number of players in the group - 5
8. Anything else you think we should know - LAG! Anywhere and Everywhere and at Any given time.

[Tormak]

Report for 4/9/16


1. Server - Cannith
2. Approximate Time - 2:00pm EST to 2:30 AM EST.
3. Approximate location of your computer - Lafayette, Indiana (Midwest)
4. Quests/raids run - Running anything I could with friends who I invited over to my house today for a day long gaming session.
5. Difficulties run - Heroic Elite./ Epic Elite
6. Your character class - Paladin/Rogue / Epic Ranger lvl 26
7. Number of players in the group - 5
8. Anything else you think we should know - LAG! Anywhere and Everywhere and at Any given time.

1. Server : Ghallanda
2. Approximate time: 10am pst -1am pst
3. Approximate Location of your PC ; Los Angeles, California ( Cali )
4. Quest/raids - Ran EE von 5 & 6, GH EE various quest....Amarath Chain...etc
5 Difficulty : Herioc Elite/ Epic Elite
6. Your Character class : Ran with my Barbarian 20/10 level 30.....Warlock 16/ 2 paladin 18 level
7. Number of players in the group ; 2-6
8. Anything else you think we should know - Lag Lag Lag.....all types of lag
a) Lag door/ lag quest giver....took about 10-15 sec to be able to enter a quest
b) Rubberbanding in quest with a small group
c) Lag spikes that last a min
d) Heals or spells don't cast but die because mobs are immune to lag
e) Get stuck on loading screen because of lag
9) Conclusion: Lag wipes and rage quits because of lag....sugguest hotfix ASAP
:mad::mad::mad::mad::mad::mad::mad::mad::mad::mad: :mad::mad::mad::mad:

[zaphear]

1 <1 ms <1 ms <1 ms 10.10.10.254
2 15 ms 10 ms 13 ms 142.254.153.89
3 28 ms 24 ms 33 ms ae61.milzwift02h.midwest.rr.com [24.164.241.197]

4 26 ms 15 ms 16 ms be39.gnfdwibb01r.midwest.rr.com [65.31.113.74]
5 15 ms 15 ms 16 ms bu-ether16.chcgildt87w-bcr00.tbone.rr.com [66.10
9.6.204]
6 29 ms 26 ms 40 ms 0.ae1.pr1.chi10.tbone.rr.com [107.14.17.194]
7 26 ms 27 ms 32 ms ix-ae-27-0.tcore2.CT8-Chicago.as6453.net [64.86.
79.97]
8 30 ms 33 ms 31 ms if-ae-22-2.tcore1.CT8-Chicago.as6453.net [64.86.
79.2]
9 33 ms 31 ms 29 ms chp-brdr-04.inet.qwest.net [63.146.26.13]
10 53 ms 57 ms 57 ms ewr-cntr-11.inet.qwest.net [205.171.17.2]
11 54 ms 51 ms 52 ms 206.103.215.50
12 68 ms 56 ms 50 ms 63.236.3.130
13 * * * Request timed out.
14 39 ms 38 ms 38 ms 198.252.160.23
15 39 ms 38 ms 38 ms 198.252.160.23

Trace complete.

[GregorianPL]

This is what have happened today... http://postimg.org/image/6v2tl14mz/
we had really massive lag raid boss just used AOE and we all wiped in massive lag still waiting in game for a gm response... looks like noone care as we are there for 2 hours

[Tormak]

Anyone in Ghallanda noticed after you finish a quest you still take damage from a firewall or Ice storm. After quest lag lol

[Sam1313]

Report for Sunday 4/10/16


1. Server - IT DOES NOT MATTER!
2. Approximate Time - IT DOES NOT MATTER!
3. Approximate location of your computer - IT DOES NOT MATTER!
4. Quests/raids run - IT DOES NOT MATTER!
5. Difficulties run - IT DOES NOT MATTER!
6. Your character class - IT DOES NOT MATTER!
7. Number of players in the group - IT DOES NOT MATTER!
8. Anything else you think we should know - YOUR GAME IS BROKEN, YOUR FORUMS IS BROKEN. That is all.

[bracelet]

Orien
Sunday noon 4/10/16
Atlanta, San Francisco, New York, London, Eastern Europe. Other places.
Defiler of the Just
Epic normal
Various classes
5, 6, 7 raiders
First run with 5 succeeded. Barely
Next five runs failed due to lagging, rubber banding, massive delays on player spells attacks.

Except for the first run it was a complete waste of time. There were some very sore heads at the end of it. I am certain that archon light would not have made the slightest difference and you should turn that back on now so we can enjoy it in these final weeks that are left.

[Silverleafeon]

Hello Sev,


I appreciate your putting most of your resources toward lag solutions.
Its pretty obvious that kensei and the anniversary are only taking a certain amount of resources.
But putting all that aside, because we both know that must be solved, and looking past that.


I really don't think that a legendary raid can:

Be challenging


Provide action for the caster & melee & specialist & tank


Be interesting and creative


not be laggish under weekend peak times


and allow a full 12 players inside the raid group



Case in point, look at all the raids designed specifically for epic play:
{disregarding current post data center move issues}


CitW ~ very prone to lag issues


FoT ~ can definitely lag although its fairly good non peak times


Thunderhelm ~ so much has been done to fix these raids, yet they are still problematic


MoD ~ again another raid that lives on the edge of lag, just waiting for a bad instance or a heavy weekend to fail


DoJ ~ this raid is still problematic even at the best of times


Converted Legendary Raids ~ all three have problems during peak gaming hours, but Shroud like DoJ is timed



You have been give a set of circumstances wherein, it would appear to me impossible to fulfill all of what would be expected of you.


I realize there are people who will claim that there was no lag before the data center move, but people tend to have very short memories. I have heard so many people complain about lag over the years, and I imagine you have too.


You could attempt to change the set of circumstances, which I believe DoJ tried and failed miserably at (not providing caster types with anything to do), yet this raid is failing more than any other due to lag.

The only circumstance available for you to comfortably change is raid party size, a circumstance inbedded inside the game of DDO since the beginning, not something that you had anything to do with creating.


Time will tell, and obviously the data center move problems need solving, but will you leave your fellow game designers circumstances that set them up to fail?



Sincerely,
Silver

[Silverleafeon]

I figured the weekend would be particularly bad, so I did not bother logging in.
Hoping a static IP address and some other fixes will help me.
I'll let you know when I find out.

[Faltout]

I really don't think that a legendary raid can:

Be challenging
Provide action for the caster & melee & specialist & tank
Be interesting and creative
not be laggish under weekend peak times
and allow a full 12 players inside the raid group
Those are statements out of the blue and in no way reflect what most raiders feel about raids. We have again and again praised creative raids where everyone of 12 or 16 people has a job. LoB, abbot, summoning chamber, hound, all have received commendations on the design.



CitW ~ very prone to lag issues
FoT ~ can definitely lag although its fairly good non peak times
Thunderhelm ~ so much has been done to fix these raids, yet they are still problematic
MoD ~ again another raid that lives on the edge of lag, just waiting for a bad instance or a heavy weekend to fail
DoJ ~ this raid is still problematic even at the best of times
Converted Legendary Raids ~ all three have problems during peak gaming hours, but Shroud like DoJ is timed


You have been give a set of circumstances wherein, it would appear to me impossible to fulfill all of what would be expected of you.

I realize there are people who will claim that there was no lag before the data center move, but people tend to have very short memories. I have heard so many people complain about lag over the years, and I imagine you have too.

You could attempt to change the set of circumstances, which I believe DoJ tried and failed miserably at (not providing caster types with anything to do), yet this raid is failing more than any other due to lag.

The only circumstance available for you to comfortably change is raid party size, a circumstance inbedded inside the game of DDO since the beginning, not something that you had anything to do with creating.
How about: Leave the raid party size as is because that is NOT the problem and fix the real problem which could be data packet sizes, server calculations, virtualization delays, network bottlenecks?

All your latest posts have told the devs to reduce raid party size. Yet, you have no evidence to suggest that it is a problem (even though there are evidence of runs with less than 6 people having lag problems) and when you posted a thread on the subject, the crushing majority said 12 or above. (https://www.ddo.com/forums/showthread.php/471708-How-small-could-a-raid-group-be-and-still-feel-like-a-raid?p=5778843&viewfull=1#post5778843 6 people under 12 with 2 saying that 12 should be left as is and 18 people 12 or more)

[zaphear]

Keep timing out on Hop 13, since Friday I've been running tracert occasionally just to check on my beloved game that is being crippled with lag.. but every time I time out on hop 13, so...there is one of your problems.

over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms *.*.*.*.*
2 10 ms 12 ms 12 ms 142.254.153.89
3 219 ms 32 ms 24 ms ae61.milzwift02h.midwest.rr.com [24.164.241.197]

4 14 ms 11 ms 16 ms be39.gnfdwibb01r.midwest.rr.com [65.31.113.74]
5 16 ms 16 ms 15 ms bu-ether16.chcgildt87w-bcr00.tbone.rr.com [66.10
9.6.204]
6 13 ms 12 ms 12 ms 0.ae1.pr1.chi10.tbone.rr.com [107.14.17.194]
7 15 ms 12 ms 12 ms ix-ae-27-0.tcore2.CT8-Chicago.as6453.net [64.86.
79.97]
8 14 ms 14 ms 15 ms if-ae-22-2.tcore1.CT8-Chicago.as6453.net [64.86.
79.2]
9 13 ms 15 ms 15 ms chp-brdr-04.inet.qwest.net [63.146.26.13]
10 38 ms 38 ms 38 ms ewr-cntr-11.inet.qwest.net [205.171.17.2]
11 37 ms 38 ms 38 ms 206.103.215.50
12 39 ms 38 ms 38 ms 63.236.3.130
13 * * * Request timed out.
14 49 ms 38 ms 38 ms 198.252.160.23
15 39 ms 38 ms 38 ms 198.252.160.23

Trace complete.



That datacenter you guys moved to can't handle this game and/or has their infrastructure FUBARED

[Maldrak]

Lag report for 4/3/2016 through 4/9/2016

1. Server - Cannith
2. Approximate Time - Various (pretty much covered every time throughout the week from "prime" times to "off hours"), Pacific Time Zone.
3. Approximate location of your computer - Sacramento, California (West Coast, USA)
4. Quests/raids run - Gianthold (Extensive Wilderness and associated non-raid quests), Vale of Twilight (Wilderness and the Quests), Orchard of the Macabre
5. Difficulties run - Heroic Elite. (Heroic Normal for Wilderness areas)
6. Your character class - Drow Paladin/Rogue/Fighter (Sacred Defender to Tier5/Vanguard to Tier4/KotC up to Tier3 Melee Evasion Tank/ with Disarm Traps/Open Lock) covered levels 18-20 during this time period
7. Number of players in the group - 1 to 3 (most often paired with an undead Archamage/Necro Wiz with Undead pet)
8. Anything else you think we should know -

Latency was a pretty consistent 100ms (ranged from about 94 to 112 - which is manageable, but way too high for a commercial online game really - most of the games I play range in the 20-30ms or lower on this machine with 15ms or even 12ms on a good day)

Packet Loss mostly hung out around 0.5% (varied from 0% to 10% during individual random samples, but was most often either 0.4% or 0.5%, and wasn't 0% or higher than 0.5% very frequently - really though, the consistent packet loss, though low, when coupled with the too high latency creates frequent, though minor, issues - still ANY packet loss, other than the freak rare moment, from a commercial online game is pretty bad in my experience, but continuous packet loss, even as minor as this, is not something I've ever experienced in an online game before.)

In short, I agree with one of the above posters, either the new data center is in China or there's ALOT of excess cabling/very indirect physical data route based on the latency. And/or it's got some serious infrastructure issues, possibly a bad/failing switch/router/hub/etc, based on the consistent packet loss. Or the virtual machine being run on the physical server is severely misconfigured or barely meeting system specs (when coupled with whatever other load may be on the machine)

The consistent latency also had a significant impact on my overall DPS, meaning it is meaningfully impacting game balance/mechanics, as I would frequently not be getting attack attempts on opponents that looked to be right in front of me, but were actually a few pixels out of weapon range (or behind me) resulting in frequent "whiffs" (not game mechanic misses/etc, but truly lost attack animations that didn't even register a valid target in their attack range - especially annoying with respect to cooldown abilities like stunning shield that would go off, trigger the CD, but then not actually connect with a target at all, or the times when you'd see your character swing 3 or 4 times but not see any interactions with your target, like swinging at empty space/air)

The latency and packet loss can make the T4 Vanguard Shield Charge pretty interesting as you get REALLY BIZARRE rubber-banding, overshooting target, lag spike mid charge can drop you off cliffs/etc, and my all time favorite circles around your target (nothing more disorienting than charging, spinning around your target in a circle and ending up facing a cliff wall - it was very difficult to maintain efficiency in those situations as I basically had to stop fighting and get my bearings back before I could re-engage my target - COMPLETELY nullifying the "surprise"/leap into combat benefit of the maneuver.


Also, since I'm writing stuff in the forum - on a side note:
The update 30 change: "Bags will no longer automatically gather items if they do not occupy a backpack slot. This prevents an issue where destroyed bags could gather ingredients."

Seems to be affecting quivers in the "equipped" quiver slot (ie, an equipped quiver won't automatically gather items), which is troublesome - if possible, allow the equipped quiver slot to auto-gather as well as the backpack slots.

And please talk to the developers about making the defender stances (Sacred Defender/Stalwart Defender) retain their toggle state when logging (they no longer have a negative side-effect, so it should be fine to let them stay "permanently" toggled on even through log offs like other combat stance toggles such as power attack/precision/etc)

[Lerincho]

1. Server - Argo
2. Approximate Time - Sunday afternoon central between 1 p.m. and 6 p.m.
3. Approximate location of your computer - Lubbock, Texas
4. Quests/raids run - Gianthold
5. Difficulties run - Epic elite
6. Your character class - Halfling Artificer
7. Number of players in the group - 1 other person, 2 hirelings and 1 pet
8. Anything else you think we should know - Very choppy in Madstone and Tor (stop action every 2 seconds.)

[Cordovan]

We are actively working to get a patch released this week that we expect will improve game performance. We will keep people updated.

[Bolo_Grubb]

I had been putting my lag reports here https://www.ddo.com/forums/showthread.php/474193-Argonnessen-Sarlona-Maintenance-Tues-April-5th-8-00-10-00-AM-Eastern-%28-4-GMT%29 but then I saw Cordovan stated this was the official thread for lag reports. So here are my last several:

1. Sarlona
2. 945pm eastern Thursday 4/7/16
3. Tucson, Az, You are at: r1 lx48 ly32 i2049 cInside ox-1.61 oy1.24 oz-126.79 h185.6
4. plane of night
5. Elite
6. Fighter/Barb/rogue
7. 12
8. Everyone freezing, multiple times, nearly unplayable

--------------------------------------------------------------------------------------------------------

1. Sarlona
2. 1015pm eastern Thursday 4/7/16
3. Tucson, Az,
4. Chronoscope
5. Elite
6. Fighter/Barb/rogue
7. 12
8. Everyone freezing, multiple times

-----------------------------------------------------------------------------------------------

1. Sarlona
2. 1106am eastern Friday 4/8/16
3. Tucson, Az,
4. 2 toed Tobias
5. E hard
6. Fighter/Barb/rogue
7. 6
8. Everyone freezing at end fight, took several minutes to loot the chest and then received "inventory error, placement failed" message. then all of us were unable to recall out for several minutes. For it took 4 minutes to get to a load screen, 2 minutes on the load screen, and then normal recall out of the slayer area.

----------------------------------------------------------------------------------------------------

1. Sarlona
2. 910pm to 930pm eastern Friday 4/8/16
3. Tucson, Az, You are at: r1 lx96 ly112 i256 cInside ox-8.75 oy-11.43 oz-176.32 h85.8
4. Mark Of Death
5. E hard
6. Fighter
7. 12
8. Everyone freezing repeatedly through out the entire time we were in the raid

--------------------------------------------------------------------------------------------------

1. Sarlona
2. 930pm to 1000pm eastern Friday 4/8/16
3. Tucson, Az,
4. Mark Of Death
5. E normal
6. rogue
7. 12
8. Everyone freezing repeatedly for the entire time we were in there

--------------------------------------------------------------------------------------------------

1. Sarlona
2. 130pm eastern Saturday 4/9/16
3. Tucson, Az,
4. Subversion
5. Heroic Elite
6. Warlock
7. 6
8. Everyone freezing repeatedly for duration of end fight, several deaths

----------------------------------------------------------------------------------------

1. Sarlona
2. 10pm eastern Saturday 4/9/16
3. Tucson, Az,
4. Legendary Shroud, High Lords Teaching Raid
5. Legendary Normal
6. Pure Rogue
7. 11
8. Everyone freezing, stuttering, rubber banding repeatedly for duration of entire raid

-------------------------------------------------------------------------------------------------

1. Sarlona
2. 815pm eastern Sunday 4/10/16
3. Tucson, Az,
4. Legendary Tempest Spine
5. Legendary Elite
6. Fighter/Barb/Rogue
7. 12
8. Everyone freezing, stuttering, rubber banding repeatedly for duration of entire raid

-----------------------------------------------------------------------------------------

1. Sarlona
2. 930pm eastern Sunday 4/10/16
3. Tucson, Az,
4. Legendary HOX
5. Legendary Hard
6. Fighter
7. 12
8. Everyone freezing, stuttering, rubber banding repeatedly for duration of entire raid,

Third planned raid of the night canceled due to frustration of lag

[JoseAntonio]

So,what caused the lag?and DDO will went down in months or no?
I think that's what players really want to know.Thx.

[Lerincho]

We are actively working to get a patch released this week that we expect will improve game performance. We will keep people updated.

Cordovan a lot of commonalities seems to be epic elite with many auras, spells, and hirelings, also where a lot of renown is being generated.

[Eryhn]

We are actively working to get a patch released this week that we expect will improve game performance. We will keep people updated.
have you ever even considered acknowledging that virtualization was a gross miscalculation and the best solution would be to buy actually proper servers? LotrO has had a lot of "small patches" which "hope" to "improve" and look what it got them.

I dont bother with typing reports here anymore, it is just tiring to spend yet more time for nothing after frustrating game experience. BUT, to sum up, over the last weeks, NONE of your tweaks and adjustments have made noticable improvements on Argo. Our static raid group still freezes and stutters through its cycle of EN LN raids every thursday and sunday night, and while with exception of xizzy which blows in ur face if bad lag, they can be completed, somehow, there is virtually NO improvement whatsoever on epic raids or EE 5 or 6man groups. While it is true that the frequency of solo questing and public instance lag has slightly decreased, as of just yesterday i still find myself hitching around midair in a lvl18 solo anniversary challenge, still lag out in various public places from necro4 to estar market to idk where on occasion etc pp.

you'll forgive me for not buying into *improvements* anymore untill i hear 14h downtime we are removing virtualization completely.

btw, i been wondering, have you done your tweak testing in pairs of servers always because now 2 servers share a physical machine on VMs?

[Taimasan]

Sarlona
Monk Pure
1830 EST
Triangle, NC 3 miles from HUB, 3 taps from amp 3rd in Cascade
Took two attempts to log in due to lag
Jitter, Rubberbanding, and Locking up for 10 min in the Portable Hole

Logged off in disgust after finally getting in.


EDIT: Decided to log in with lowbie and run a few quest's relatively lag free. Seems like my monk with his huge loadout.

Since monks are the weapon I am just one huge aura.

EDIT: Scratch that, same amount of lag, its annoying lag, random freezing. Rubber banding, almost makes me motion sick.

[Livmo]

Been bad off and on. Still had lag deaths this last weekend.

I had about a 10 min freeze on Saturday around 9:30pm-10:30pm PST. So I bad I had to stop playing during that time frame.

Sarlona

[Katanauser]

I and 3 others just totally lagged out in epic Storm Horns, we were going up the hill and had made it to the first fort. Suddenly while fighting, all were totally frozen, no move no shoot, no cast, no recall. We all just logged out from there. I had to knock down the client from Task Manager to get logged off. Another frustrating questing experience. And a very frequent one since build 30. We'd had lag all evening since 1700.

It's possible there are too many little data messages going back and forth to the servers carrying data Turbs think important. The game has been choked beyond fixing with **** craft+ collectables that each carry a unique id to prevent pirating, have no real use because it takes 10 million of them to craft / exchange for anything, therefore staying in the permanent item database for ever choking client response to nothing. Don't even get me started on **** loot like +8 dwarven axes with 2 spellcaster useful prefixes/ suffixes. When is the last time you played a dwarf wizard axe user???

Turbs FIX THIS GAME. And your gaming/ data paradigms. :rolleyes:

And no, you can't have my ****. The system is too borked to offload it to you. :rolleyes:

[Taimasan]

Stuck on getting in line to connect on loader, I can't play at all.

EDIT: Close out and relog fixed it. Used to hang on character choice, now I am hanging on loader.

2050 EST Sarlona

[Kantia]

1. Server - Sarlona
2. Approximate Time - 12 hours before this post
3. Approximate location of your computer - Melbourne, Australia
4. Quests/raids run - Gianthold Tor
5. Difficulties run - Heroic Elite.
6. Your character class - Ranger/Barbarian
7. Number of players in the group - 6
8. Anything else you think we should know - Every time we killed a dragon/giant pair in Gianthold Tor, the whole party reported freezing in place for about 5-10 seconds. Fighting the Heroic instance with Epic characters meant that the fights only took a few seconds, but the freeze was repeatable, noticeable and predictable (and bad if we were unknowingly standing in acid!)

[Sam1313]

Lag Report for Monday 4/11/16

1. Server - Cannith
2. Approximate Time - 4:00 pm EST to 10:00 PM EST.
3. Approximate location of your computer - Lafayette, Indiana (Midwest)
4. Quests/raids run - A Legend Revisited..
5. Difficulties run - Epic Elite X 43 runs. NO BLADE!
6. Your character class - Paladin/Rogue / Ranger
7. Number of players in the group - 3
8. Anything else you think we should know - I only ran that one quest the entire time I was logged in. Trying to farm for Jacks Blade. NO BLADE! after 43 passes.
I only had a small bit of lag inside the quest but experienced the most lag in the Harbor just running from the airship to 3bc.

[alancarp]

1. Server - Thelanis
2. Approximate Time - 10:20-25 EDT, Monday evening April 11
3. Approximate location of your computer - Alabama
4. Quests/raids run - In House K public area
5. Difficulties run - N/A
6. Your character class - Gnome/Rogue
7. Number of players in the group - me
8. Anything else you think we should know - Consistent rubber-banding and freezing while simply running around. Had been happening in most public areas and aboard the ship over the prior 2 hours. Quest (Cabal) and Gianthold slayer area was actually better. This was actually the worst I've seen it in a while (nightly runs), though it's been more noticeable over the past 3-4 days. This event was notable enough that I opted to make a report. Edit: after finishing this report, see,ed to have cleared up for the time being.


Tracing route to gls.ddo.com [198.252.160.23]
over a maximum of 30 hops:

1 <1 ms 1 ms <1 ms 192.168.0.1
2 43 ms 45 ms 43 ms adsl-98-81-165-1.hsv.bellsouth.net [98.81.165.1]
3 86 ms 90 ms 108 ms 70.159.223.41
4 117 ms 50 ms 56 ms 12.81.99.60
5 52 ms 53 ms 51 ms ixc01bhm-pos-5-0-0.bellsouth.net [65.83.239.116]
6 96 ms 107 ms 106 ms 12.81.36.0
7 176 ms 213 ms 219 ms 12.83.2.204
8 59 ms 54 ms 55 ms 12.81.16.57
9 52 ms 52 ms 52 ms 74.175.192.66
10 78 ms 77 ms 74 ms cr1.attga.ip.att.net [12.122.141.186]
11 75 ms 74 ms 75 ms 12.122.2.30
12 74 ms 76 ms 75 ms cr2.n54ny.ip.att.net [12.122.3.37]
13 73 ms 73 ms 72 ms gar14.n54ny.ip.att.net [12.122.105.81]
14 * * * Request timed out.
15 72 ms 76 ms 81 ms border1.pc2-bbnet2.nyj001.pnap.net [216.52.95.73]
16 73 ms 74 ms 72 ms turbine-14.border1.nyj001.pnap.net [70.42.39.234]
17 * * * Request timed out.
18 87 ms 79 ms 88 ms 198.252.160.23
19 77 ms 75 ms 73 ms 198.252.160.23

[lugoman]

Cordovan a lot of commonalities seems to be epic elite with many auras, spells, and hirelings, also where a lot of renown is being generated.

Renown gets the blame for the lag, but a lot of things happen at the same time. For me, the game stopping lag occurs when game objectives are met/updated. A portal is killed in shroud, harry is killed, book is killed, named mobs killed in tempest spine, dragon/giant pair killed in fot, etc

[Aletys]

Attempted a DoJ yesterda. Complete waste of time & effort ... we failed due to lag, it kicked us out. We tried 2x. Was able to do a von 5 earlier in the day, with little lag.

Today, tried an MoD, it took us 5 tries to complete, the lag was awful. We only succeeded the last time because there were several jibbers in the group.

[Silverleafeon]

1. Server - Khyber
2. Approximate Time - 10:20- Midnight CDT, Monday evening April 11
3. Approximate location of your computer - Alabama
4. Quests/raids run - Shroud
5. Difficulties run - LN
6. Your character class - Deep Gnome Fighter 16/Pally 4 in LD blitzing
7. Number of players in the group - 12 then 1 dropped immediately, 1 DCed towards end and was kicked
8. Anything else you think we should know - Annoyingly playable, lag every 10 seconds or so in part 1, autoattack seemed to work more than button pressing, an ungly 3x killing the bosses in part 2 due to lag from lieutenants dying especially the last one, an ugly 5 round harry due to getting renown from gnolls causing 5~9 seconds of lag, part 5 went fairly smoothly

Group stuck together as much as possible which seemed to help a lot.
Computer/internet kept up with downloading foes, chests.


All in all, a very sluggish 11 man LN Shroud, late Monday right after a "spring break" type weekend is over.
Logged out afterwards, between waiting to fill, and the raid, long time.


No news on static IP yet, went and got my router today.


Pretty clear like to me with the 3rd set of gnolls generating renown several rounds in a row, and suddenly 10 seconds of lag (while harry heals up), renown is definitely a big problem atm. We took harry down by getting lucky and not getting renown from gnolls on last round.

[eterna1_drag0n]

This message is directed directly to the Developers, Executive Personnel, and Systems Protocol personnel.

There has been a huge amount of data supplied by the clientele of this game, but little to no data supplied by the staff.

I propose the following: If you let your community help you, we will. Free of charge, no fees, service charges or addendum's; we will help you fix the issues.

DDO is supposed to be based on D&D, which is; "A game of the Imagination" (Dead Ale Wives) where the only limitation is a players own creativity and knowledge. Let us, your eager and active community, help you fix the problem that has been limiting ALL of our enjoyment. I have been an active player for the better part of the last seven years, and I know for a fact that there are quite a few networking specialists that play this game, I have conversed with several about the sometimes laughable protocol issues that have bogged down the servers. I understand that EVERYONE is doing their best to resolve this issue at your headquarters, but to put it simply, it isn't working.

It is time for the paradigm to shift again. DDO and Turbine has been the innovator for in-game, micro-transactions with the Free-to-Play format. Let this be your next innovation, Community Enabled Support, CES. Let your player base help! I assure you, we are pulling out our hair in frustration as much as you are. But our frustrations are based on our inability to give constructive feedback to on an issue that we have experience with. If you don't know exactly what the issue is, that's fine, lets start with the basics. Post some specs about your servers, volume, operating: how are they running? emulated/virtualized or hardware, average population, peaks highs and lows, how is your node integrated?...

If you let us help, we will...

[donblas]

Khyber

Approx 9pm BST (4pm EST?) Monday

2 in party: level 12 sorcerer, level 10/2 wizard rogue.

Marketplace

5 minutes or so of intense lag, rubberbanding, inability to interact with teleporter.

[JOTMON]

have you ever even considered acknowledging that virtualization was a gross miscalculation and the best solution would be to buy actually proper servers? LotrO has had a lot of "small patches" which "hope" to "improve" and look what it got them.

I dont bother with typing reports here anymore, it is just tiring to spend yet more time for nothing after frustrating game experience. BUT, to sum up, over the last weeks, NONE of your tweaks and adjustments have made noticable improvements on Argo. Our static raid group still freezes and stutters through its cycle of EN LN raids every thursday and sunday night, and while with exception of xizzy which blows in ur face if bad lag, they can be completed, somehow, there is virtually NO improvement whatsoever on epic raids or EE 5 or 6man groups. While it is true that the frequency of solo questing and public instance lag has slightly decreased, as of just yesterday i still find myself hitching around midair in a lvl18 solo anniversary challenge, still lag out in various public places from necro4 to estar market to idk where on occasion etc pp.

you'll forgive me for not buying into *improvements* anymore untill i hear 14h downtime we are removing virtualization completely.

btw, i been wondering, have you done your tweak testing in pairs of servers always because now 2 servers share a physical machine on VMs?


All that is happening with virtualization is the server is running in an instance on a server.
There are more benefits then detriments running in virtual.

Virtualization is the way of the future and is very stable when setup and allocated with proper resources.
Recovery from system issues, upgrades, system improvements all are much easier when running the servers virtualized.
Bigger hosters are running multiple servers as the platform to several virtual systems simultaneously, so losing/replacing a server does not take down the system.

There is tweaking time to ensure the virtual server load demands are adequately covered by real system resources.
This falls into the $ side of it since you have to buy/allocate resources from the live server host to the VM and find that sweet spot where you are not paying for more than you use.

Where the problems happen is when the server is not allocating enough resources to the virtual server, or is hosting too many virtual instances.
Farming out to cloud servers or other hosted servers leaves you at the mercy of the hosters..


DDO has more problems than this going on.. 10 year old custom code written by guys that are no longer around.
Layers of coding to patch issues and workarounds over the years with few (if any) clean rewrites of code.
Some known issues with effects that keep reapplying themselves behind the scenes causing some overstacks of effects.
After a while too many patches and temporary fixes catch up.. spaghetti code effect...
You can only patchwork fix for so long before the cost/time of patches outweigh the value.
with dwindling resources it becomes more difficult to keep up with the spot fires and never getting to the source fixes..

starts to layer like a Jenga game.

https://kudago.com/media/images/list/af/c4/afc47b6aefc9919bf636fa9018bece83.jpg

[Eryhn]

very stable when setup and allocated with proper resources.
it does not look like ressources being sufficient for peak usage. that asside, because of this:




DDO has more problems than this going on.. 10 year old custom code written by guys that are no longer around.
Layers of coding to patch issues and workarounds over the years with few (if any) clean rewrites of code.
After a while too many patches and temporary fixes catch up..
You can only patchwork fix for so long before the cost/time of patches outweigh the value.
Then its rebuild , replace, or drive it into the ground and walk away.



having been the case for years, this:

Where the problems happen is when the server is not allocating enough resources to the virtual server, or is hosting too many virtual instances.
is not the only issue.
rather, the problem is also happening when you put the spaghetti code u described on a VM in the first place. while in theory VM is stable, its when you have a bunch of clunky things put on it that the problems created greatly multiply in effect, ya get overhead like mad and dont even know from which of your age old little ****ups, and this would be an issue even if the initial bare metal allocation behind the VMs were benevolently calculated, which i bet you a shiny festival twig it of course wasn't.

the very premise to be able to do the needed tweaks on the fly over time on life servers on a minimal cost bare metal based VM is grossly irresponsible and a huge insult to customers. the amount of messup was predictable, with code like ddo its either have so much hardware behind the VM that you might just as well go bare metal from the getgo, or do far more excessive testing and configuration BEFORE going live if you insist on this ******** being implemented. Assuming the networking team is as strongly manned as the dev, which is to say it is grossly understaffed, how could anyone think this would go smooth in a even half way decent time frame, given they been through the same **** with the 2 preceding game server migrations?

and I bet you TWO shiny festival twigs turbine is so tightlipped because they knew EXACTLY this would happen, which is btw why all the rage is quite justified and all those, *oh hey, i STILL renewed my sub despite all this, cause, THINKPOSITIVE TM* folks have my deep sympathies on the astonishing depth of their illusionary world - it's the approximate equivalent of still paying your sub fees to turbine 2 years after the game servers were turned off cause *hey, back in the day it was good times*

[Cordovan]

We are currently working on a patch, and expect to announce downtime in the coming days. This patch includes a fix to several stacking enhancement issues along with some other optimizations. We'll keep you updated.

[Kompera_Oberon]

We are currently working on a patch, and expect to announce downtime in the coming days. This patch includes a fix to several stacking enhancement issues along with some other optimizations. We'll keep you updated.

Please keep pointing to this thread. Because of your horrible communications methods it is often *ahem* lost in the web.

[MeliCat]

We are currently working on a patch, and expect to announce downtime in the coming days. This patch includes a fix to several stacking enhancement issues along with some other optimizations. We'll keep you updated.

Does "fix" mean "remove"? As in simplify so that the new HW dimensioning can cope?

[Livmo]

Last night between 8pm-9pm PST. Legendary Shroud. My first one. Posted LH, but we lagged too bad and went with 10 on LN afters.

That was my first Shroud and not a good impression due to the lag on LH. Also it seems odd that the portals have so many hit points with such bad lag. Not a good combo. LN was laggy but not as bad. We completed on LN

We ran LH Hound of Xoriat after with mostly the same peeps with 10 in party. Lag was bad in Marketplace, but not too bad in the raid. We were able to run and complete that one on LH. That was a just after 9pm PST last night.

I have not run all the raids since data center move, because I've been busy lately, but 2 raids so far unplayable beyond normal, Legendary Shroud and Fire on Thnuderpeaks.

However, with Fire the lag was so bad we didn't even get a normal compeltion the last 2 weekends in a row.

In each instance the whole party seemed to lag at the same time.

I did enjoy both the new quests, although laggy, prior to LS.

[Cruxader]

We are currently working on a patch, and expect to announce downtime in the coming days. This patch includes a fix to several stacking enhancement issues along with some other optimizations. We'll keep you updated.

Cordovan and Dev and DC teams- First, thanks for keeping focus on this, the near and long term future of the game clearly depend on solving it completely, not partially as I'm sure you are well aware from all the shouting.
Second working in the Data Center Industry for 20+ years, I'd hope to offer some helpful guidance with a couple questions I hope bring value from my experiences fixing lag for both gaming and corporate systems for decades:

1. have the network and backbone fixes your referenced at the start of the thread been implemented yet? That seems the cause of some (but not all) of the recent lag issues.

2. regarding platform portability and code fixes - I'm sure you guys have the portability issues identified by now and hopefully that patch will help a bit here as well but...

3. Regarding the fundamental Resource constraints when processing 12 person parties in raids at peak hours (the biggest threat to this game)- The server virtualization... yes as some say it is the way of the future from a corporate cost savings and resource optimization perspective, though not often from a system performance perspective which unfortunately is exactly where the problems are here today with this game.

Has anyone there yet tried upping the resources allocated from each physical server to each virtual instance? Or tried running hosted back on bare metal instead of on someone else's uncontrollable Cloud?

I assume you were running cold iron before the recent data center move and the lag, though not perfect, was much better then.
If the game worlds are hosted on shared servers (virtualized) unless there is enough resources provided to accommodate peak max usage for each instance at peak times, unless they are allocated to handle that, or burst up to that (which is usually a very fragile solution anyway) then yes there can clearly be resource constraints coming from the under-resourced virtual instances there.

I again appreciate the patches and hope they help, and think they may a little bit, but the severity and type of lag we are seeing is indicative of either massively underpowered new Data Center network, or massively under resourced virtual server instances that need to be not tweaked 30% but perhaps tripled (200-300%) to really accommodate peak load (please test case Legendary Shroud with a 12 man party on LH on each server-world alongside with 12-man DOJ, to ensure that at minimum these can run these each per server without lag in production, on all servers *simultaneously during peak hours*- *THAT* is the *minimum* benchmark the tech and DC teams needs to plan and resource for if they want the game to continue, which all of us surely do because if the servers cant even handle that parallel load at peak hours then the game is effectively a single player game until it can.

Very best regards,

[Sam1313]

Report for Tuesday 4/12/16

1. Server - Cannith
2. Approximate Time - 8:00 pm EST to 12:30 AM EST.
3. Approximate location of your computer - Lafayette, Indiana (Midwest)
4. Quests/raids run - Devils Assault.
5. Difficulties run - Epic Normal / Helping a guild member farm for Tokens so she can TR.
6. Your character class - Ranger
7. Number of players in the group - 2
8. Anything else you think we should know - See video on my Youtube channel titled Lag Or click link here: https://www.youtube.com/watch?v=NhsqcgqMVyY

[GlassCannon]

If you are continuing to see poor in-game performance, then any information you can provide is helpful.

I've been experiencing barbarian rage and frenzy effects applying negative but not positive bonuses, taking damage from sources that appear to not actually be where they show/say they are, seeing Death effects persist until logout despite being very much alive, seeing expendable items/effects not actually do anything despite being expended and other various crippling incidents. I'm reinstalling DDO today to give it another shot.

If I experience the same "My client needs to catch up to the server" problems and the server simply ignoring what my client is saying and permanently destroying items that I rely on for their described benefit without applying said benefits, I'll just go back to Ashes of the Singularity or whatever it is I find to do.

The quests I experienced these problems in are: The entire Lords of Dust chain on Heroic Elite with a Barbarian, Temple of Elemental Evil on Epic Elite with a warlock, Eveningstar chain quests on Epic Elite, Druid's Deep and Stormhorns chain quests on Epic Elite (the most problematic was What goes Up after the Mythallar has wedged in the glacier and you fight huge spawns on the way up). I actually had to just quit WGU due to being forced to watch my character die within 2 minutes (10-30 seconds of actual game-time due to lag) of being raised, more than 6 times in a row. Consistently.

Please please please figure out what's causing these problems. I freaking love this game and want to help but I'm no technomage.

[Livmo]

Between 8-9pm PST last night we ran raids on Sarlona. Heroic Elite Shroud was not too bad, but when we ran Fall of Truth afters on EH it was bad (lag).

I've run FoT countless times on EH without lag. Last night I died after after completion, because could not move or heal and eventually the boom got me. At least it was after completion.

However, its frustration to be stuck in one spot helpless watching you health go down and then die. I could not even move after death for about a minute.

The freezing and rubber banding are horrible.

I also had some rubber banding just after 9PM PST while running the High Road chain on EE for sagas.

[Cordovan]

We are currently testing a patch. Once that's done and things look good, we'll formally announce downtime. That said, we're currently targeting a Thursday 8:00 AM to 11:00 AM Eastern downtime. This patch will include several optimizations to improve game performance, and correct an issue with barbarians that is contributing to poor game performance.

[Lerincho]

barbarians that is contributing to poor game performance.

Figures, the savages causing all the trouble. Can we kick them out of Eberron?

[Cordovan]

I wanted to make sure people were aware that I chatted with Sev for the first six minutes of today's livestream about the lag, and he provides some detailed information about what we are doing and what's been happening: https://www.twitch.tv/ddostream/v/60375811

[Silverleafeon]

1. Server - Khyber
2. Approximate Time - 6:30 ~ 7:30 CDT, Monday evening April 12
3. Approximate location of your computer - Alabama
4. Quests/raids run - Shroud
5. Difficulties run - LN
6. Your character class - Deep Gnome Fighter 16/Pally 4 in US, with mostly passive bonuses.
7. Number of players in the group - 9 (I got lead was going to hold at 10, but one player toon swapped an never returned.
8. Anything else you think we should know - 100% of lag for me from renown, red name dying, objectives being met, loot calcuations, etc...
Greatly improved by my static IP address and new open router system.
2 or 3 players complained about serious lag issues.
The renown lag was definitely less with 9 players compared to the 11 players the night before, despite being a more populated server time.
The initial spawn in part 1 moving toward the bridge did have some serious lag till we killed some of those foes, but the party was able to stay with the networking.

Group stuck together as much as possible which seemed to help a lot.
I think the trash killers suffered more from lag than the melee, not sure.
Computer/internet kept up with downloading foes, chests.


9 man LN Shroud, late Tuesday right after a "spring break" type weekend is over.
Long time filling, went quicker after I got the star.
Logged out afterwards.


News on static IP & Opened up router (along with tight security subscription):
Strong improvement on overall gaming so far.

[UurlockYgmeov]

I wanted to make sure people were aware that I chatted with Sev for the first six minutes of today's livestream about the lag, and he provides some detailed information about what we are doing and what's been happening: https://www.twitch.tv/ddostream/v/60375811

Might want to make the above a separate thread. :)

[Captain_Wizbang]

Interesting info. Thanks Cordo & Sev

[DagazUlf]

I wanted to make sure people were aware that I chatted with Sev for the first six minutes of today's livestream about the lag, and he provides some detailed information about what we are doing and what's been happening: https://www.twitch.tv/ddostream/v/60375811

That was really great information. Thanks Cordovan.

Any chance we could just see that as a few bullet points in a tweet or forum post?

[GoldyGopher]

1. Server - Khyber
2. Approximate Time - 1400 CDT
3. Approximate location of your computer - Minnesota
4. Quests/raids run - House J Quests at level
5. Difficulties run - Any
6. Your character class - Elf Pally 5/Rogue 3.
7. Number of players in the group - 2
8. Anything else you think we should know:

Over the period of 1 hour and 45 minutes today I ran a series of quests in House J with another player who was more AFK than present.

I saw so many types of LAG today that I am not sure I could describe them all. One detail I found interesting and it may be coincidental but is worth mentioning:

The more the quest used vertical space (as opposed to single plane) the more lag we saw. So in Haunted Library when we were on two vertical levels separated from one another the lag was unbearable and it took about 30 seconds or so to catch up.

[IronClan]

Thelanis raids are currently ultra laggy still, though we did manage to snap and freeze our way through a 12 man LH shroud the other day. I would call the gameplay experience technically possible to play but no ones idea of fun.

[Mithis]

I wanted to make sure people were aware that I chatted with Sev for the first six minutes of today's livestream about the lag, and he provides some detailed information about what we are doing and what's been happening: https://www.twitch.tv/ddostream/v/60375811

Is there any additional information that can be provided on the CenturyLink related networking issues that was mentioned by Sev at approximately 5:28?!?

[Cordovan]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

[Krelar]

Here's today's interview with Sev. It's the first six minutes of today's video:

That would be great if I were somewhere I could play sounds atm. (or didn't generally despise watching videos for information where plain text would be much easier.)

[Kza]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

Thanks for this! Really nice to hear you ppl and not just read.

[Eryhn]

I was rather negative of late so yeah, should point out that the more detailed information on what is going on from the stream is appreciated.

At the same time, allow me to point out that a number of people on here would probably not have been as negative if we had gotten something like this say 2 weeks ago.

I understand that the stacking enhancements on barb might only have been identified recently, but i would think the issue of the VMs actually not being able to properly load balance (if i read that bit on limited CPU switching correctly) is something you did not stumble across just 3 days ago. having info like this out a bit earlier would have helped players to better appraise the situation and thus react less itchy ...

that beeing said best of luck to us all with the fix lol

[Lerincho]

That would be great if I were somewhere I could play sounds atm. (or didn't generally despise watching videos for information where plain text would be much easier.)

Wow. Cordovan didn't have to do that, and Sev didn't have to answer, but this is the response from their efforts? Wow.

[Qhualor]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

Lol and people were blaming warlocks.

[Silverleafeon]

(Party:): [Party] _________: renown is an issue for sure

[Taimasan]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

Saw you around 20:00 min in you had to adjust the sound bar...I hate that!!

Is it not annoying to you too?? Ugh.

But thanks for putting this out there for the players.

[Livmo]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

Makes sense and I totally understand.

TYVM for the update!

[Phoenix-daBard]

I wanted to make sure people were aware that I chatted with Sev for the first six minutes of today's livestream about the lag, and he provides some detailed information about what we are doing and what's been happening: https://www.twitch.tv/ddostream/v/60375811

Any synopsis for us deaf players? Not complaining and I am asking from anyone who watched the video.

[Taurnish]

Thanks for the info Sev and Cord. I'm looking forward to the new Kensai changes and the anti-lag buff .

[Livmo]

Any synopsis for us deaf players? Not complaining and I am asking from anyone who watched the video.


New data center and new operating system need to be optimized regarding Microsoft.

Still having some issues w/3rd parties.

Barbs have some over performing stacking going on behind the scenes.


Fix will probably address 1 and 3. Feel bad for the people at Turbine that have to deal with 2. Aside from all this lag will continue to be worked on past these items.

[Gratch]

Any synopsis for us deaf players? Not complaining and I am asking from anyone who watched the video.

There's a server side update in test hopefully for a Thursday update. They're fixing a stacking-lag-isssue that affects Barbos and they've got a change to help with threads migrating across processors caused by swapping from direct Microsoft OS to their new Virtual Machine environment.

Also trying to cleanup network connection routing to the servers but its a long process due to lawyers and contracts being their own source of paperwork in triplicate lag.

There will be a few more updates coming on the current inDesign Fighter enhancements on the forums.
Player's Council is getting an early look now at U31 new loots.
Coming to Lama "Fairly Soon". (but not SOON).

[-D_Rock-]

Wow. Cordovan didn't have to do that, and Sev didn't have to answer, but this is the response from their efforts? Wow.

Which makes me wonder why this person could have ever made the players council. Or why a council even exists with people like this offering nothing but negativity.

[Aelonwy]

1. Server - Thelanis
2. Approximate Time - 7:50pm U.S. Central
3. Approximate location of your computer - Texas
4. Quests/raids run - nothing the rubber banding was so bad I couldn't ship buff because I kept trying to move forward only to be pushed away from the shrines, tried to use them several times, went through the motions five times before successfully getting one elemental resistance. Wanted to log off at that point and had to click log out several times before it actually went to the character screen.
5. Difficulties run - Lag difficulty : Champion Frustration
6. Your character class - Fighter5/Barb3
7. Number of players in the group - Solo
8. Anything else you think we should know: Earlier in the day I duo-ed with my hubby through Undermine and Seigebreaker on Elite, different alts, we had frequent stutters and sometimes spawns would be delayed and then suddenly everything was on top of us from out of no where. Also quite frequently we would be hitting/shooting the mobs and none of our damage was registering all though we were moving around fine then all of a sudden we'd be nearly dead and never saw them do such damage to us, also in these cases we'd still have to finish damaging/killing the mob because they simply were somehow lag-immune to our efforts.

So far I cannot tell on my end that anything you have done has improved game performance much since the data center move. I wish you luck but I'm getting awfully aggravated and disappointed.

[jaedom]

If you are continuing to see poor in-game performance, then any information you can provide is helpful.

As others have probably noticed renown acquired from mobs causes lag. We have tested it over our last two lives and it happens to us all no matter our location ie. east coast west coast or international. We all play on high performance gaming computers mine being only a few months old so age of computer is not an issue. Defiler seems to be one of the harder hit raids but as previously noted you receive renown for each mob killed with the amount of red named it freezes up unless short-manned and even then is nearly unplayable. Chest spawn lag is also an issue tho much less of a problem.

Another issue that has been causing problem and has been reported is perma walls. Ice storm,fire wall and acid fog that persist until relog. One does not merely get a single stack of aoe in many instances I have gotten up to three stacks of aoe.

We have been very patient and continue to hold out hope that the patch will help. It will take time but I am sadly hearing of more and more players leaving for more stable games. I know you guys are working hard for a fix you play the game too and if it is broken for us it is for you as well. Keep us posted. Thanks

[zehnvhex]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

At about 20 minutes they mention the Cannith Crafting deconstruct lag. I just want to mention I've been having this problem since early on. Sometimes I can mow through items no problem but there are other times where after one or two items it just gets 'stuck' and I have to close and re-open the interface.

I'd say in a typical 'deconstruct' session where I'm nuking 100 items or so it will get stuck about 13 or 14 times.

It's gotten to the point where I find it faster to just close the interface as soon as I hit deconstruct and re-open it. Even when it doesn't get stuck about another 30+ times it'll take a few seconds longer for it to register the deconstruct then if I force close the interface and just re-open it quickly.

I can tell you the following:

It has happened regardless of what size ingredient bag I have auto-collecting.
It has happened even with no ingredient bag auto-collecting.
It has happened with items of all sorts of levels.
It has happened with me deconstructing or stripping enchantments.
It has happened on my Windows 7, Windows 8 and now my Windows 10 computer.
It has happened since at least u24.
It has happened across 4 different ISP's and 3 different routers/modems.
It has happened across at least 5 different characters.

I've tried everything I can think of to eliminate it as a problem on my end though I obviously could be missing something. Given that I'm not the only one reporting the issue though makes me feel there's a commonality that extends beyond some setting I haven't switched.

As is I'm eagerly looking forward to a crafting revamp since right now there's about a half dozen twink items it's useful for and that's about it. If all it did was fixed the deconstruct lag that alone would keep me giddy for a month straight.

[Phoenix-daBard]

New data center and new operating system need to be optimized regarding Microsoft.

Still having some issues w/3rd parties.

Barbs have some over performing stacking going on behind the scenes.


Fix will probably address 1 and 3. Feel bad for the people at Turbine that have to deal with 2. Aside from all this lag will continue to be worked on past these items.

There's a server side update in test hopefully for a Thursday update. They're fixing a stacking-lag-isssue that affects Barbos and they've got a change to help with threads migrating across processors caused by swapping from direct Microsoft OS to their new Virtual Machine environment.

Also trying to cleanup network connection routing to the servers but its a long process due to lawyers and contracts being their own source of paperwork in triplicate lag.

There will be a few more updates coming on the current inDesign Fighter enhancements on the forums.
Player's Council is getting an early look now at U31 new loots.
Coming to Lama "Fairly Soon". (but not SOON).

Thank you both, it helps.

[Sam1313]

Here's today's interview with Sev. It's the first six minutes of today's video:


https://www.youtube.com/watch?v=h4nNLDdi5CY

Ah sadness. I didn't even get a chance to log into my home Cannith and play today as I was working from 5:00 AM EST to 7:30 PM EST. Maybe next time. And if you do happen to be on Cannith Hit up Samyel

[Kompera_Oberon]

Wow. Cordovan didn't have to do that, and Sev didn't have to answer, but this is the response from their efforts? Wow.
Wow. Cordovan didn't have to communicate their efforts to resolve the horrible LAG issues? Severlain didn't have to scrape a sparse 6 minutes in order to communicate with the customer base about those same issues?

Could you possibly be more condescending? Yes, this is the response to their efforts, because those communication efforts have been very poor for the past 5 weeks and 6 whole minutes at the start of watching Cordovan play the game is not any kind of solution. There are a large number of players who have been asking, begging, demanding answers, and to provide them in the form of a livestream with a very small viewership is just business as usual for Turbine: Scatter your communications efforts, as sparse as they already are, across a solid double handful of venues and expect the customers to manage to separate the chaff from the seed.

Did you see the post from the player who is deaf? That post alone should tell you that pointing to a livestream is a poor method of communicating with the customer base.

[MeliCat]

That was really great information. Thanks Cordovan.

Any chance we could just see that as a few bullet points in a tweet or forum post?

Yes please. Twitch.TV is slow as for me. Thank you Cordovan for this.

[sudzz]

So, the lag being experienced by those of us who do not have any barbs in the party is coming from where? Please don't scapegoat a class in this manner, not that I play any barbs or barb splashes myself atm, as it will set a bad precedent for folk to not want them in the party. Although I would like the option to switch off visibility of other's cosmetics like useless pets etc as surely they are adding to sever load without any positive contribution to a party in quests.

[Lerincho]

So, the lag being experienced by those of us who do not have any barbs in the party is coming from where? Please don't scapegoat a class in this manner, not that I play any barbs or barb splashes myself atm, as it will set a bad precedent for folk to not want them in the party. Although I would like the option to switch off visibility of other's cosmetics like useless pets etc as surely they are adding to sever load without any positive contribution to a party in quests.

They stated contributing, not causing. Additionally just because they are not in your party does not mean that it is not effecting the overall server. Each party may be their own instance in a dungeon but they are not their own server.

[FeartheBow]

Seem to be having problems hard targeting monsters. Once I do get them targeted it will automatically un-target them and I end up firing into the void.

[LeoLionxxx]

A lot of the time, it will seem that lag will (finally) stop only once every player in the instance/area is dead. Might this relate to how monsters are tracking the players and the hate levels they place upon each?

And of course, having a swarm of a dozen guys pop up at once is a really good way to incur that wonderful lag.

[VinceWhirlwind]

They're fixing a stacking-lag-isssue that affects Barbos and they've got a change to help with threads migrating across processors caused by swapping from direct Microsoft OS to their new Virtual Machine environment.

Also trying to cleanup network connection routing to the servers but its a long process due to lawyers and contracts being their own source of paperwork in triplicate lag.



The chances of the lag being caused by BOTH bad processor scheduling AND networking issues at the same time, is so small it's not funny.

When you're in a project meeting, and they are talking about two completely different issues in relation to the same problem, you know something has gone wrong with your incident management.

What it usually means is that the server guys have tried to divert attention away from the poorly-documented GUI-driven botch they are responsible for by mentioning invented "network problems" often enough to get these imaginary "network problems" onto the agenda. Then you have many hours over many weeks being spent by network guys and their project managers trying to find an imaginary issue and then trying to prove the imaginary issue has been fixed.

And how lawyers and contracts come into "network issues" is anybody's guess.

A genuine "network issue" is generally identified, isolated, and fixed within hours. If it drags out over a week, it is almost always going to be something happening in the application (that is, Layer4 and above - the TCP/IP stack, operating system, and other applications sitting above it).

"Network connection routing" in the datacentre is something that happens in 10s or 100s of microseconds. Delays in the 10s of milliseconds and above are not happening in the networking infrastructure.
Of course, the first thing that has been ruled out in the first few minutes of the incident is that there is no congestion issue - no dropped packets, no switch buffers overflowing, no collisions or FCS errors.
Shortly afterwards, you should be able to see whether traffic is spiking with a lot of TCP retries. In the absence of any lost packets, this tells you the application is at fault. If the TCP connections are up normally and traffic is dipping, the application is at fault. In almost every other case, the application is at fault.

[Cordovan]

Closing this thread to give things a reset and start fresh here. (https://www.ddo.com/forums/showthread.php/474557-The-latest-on-lag-and-game-performance-improvements)