End of the week with Major Malphunktion - Kobold Union Edition

[Cauthey]

That's why I thought static 20 or 30 minutes for the doors to open in Mabar or the Cove would have been a better alternative. The quest would be available at a reasonable rate in all timezones that didn't depend on the farming of ingredients. Additionally, it would end the speculation of when the doors will open. Sure, the drawback would be that there might not be enough people to do the Spectral Dragon each time, but perhaps a failed attempt is perferrable to no attempt at all.

That's the challenge, though. For Crystal Cove, you could go in there at any time and single man it.

The Spectral Dragon Chamber can not be single-manned. And often, cannot be short manned.

The only solution I could imagine would be to allow additional entries into the Spectral Dragon Chamber, but to gate them with a dynamically updating window (kind of like a quest entry window) that shows how many others are clicked on the window and waiting for the instance. Once you have a minimum number (12, maybe?), it sucks you into the instance. That way you can be protected from not getting a super empty, short-manned instance (which is very frustrating when it happens).

Something like this, perhaps?

[Seikojin]

So.. let's talk about my personal pet peeve. (Well, one of them anyway)

The ladder bug.

No, I'm not talking about the ladders that were accidently broken in one update a while back so that they don't connect in the middle anymore.

I'm talking about the reason that every experienced ddo player I know has to jump to get on a ladder. Any ladder, anyplace... Or sometimes they just rubberband in place at the bottom of the ladder.

Is this a limitation in the game engine? Is it something that only people over a certain amount of latency experience? It's been in the game since the very first day I started playing ddo, and it's been there consistently and reliably ever since, through five different computers on my end.

And yet I've never seen it mentioned anywhere.

It is a lag related bug from what I have seen. I dunno how DDO does its packet monitoring, but in some games (quake and many fps' since then) do some decision making for you. They take some assumptions based on your current path and assume you will continue doing said action. If you were running forward and made a hard right, you may see some stuttering. This is because the server assumes your path will maintain, so it starts sending your client updates along those lines. The sudden change of direction conflicts with the assumption and the game has to redraw everything base don your client decision, not the servers assumption.

That is if they do that.

Also, it could still be lag related if there are programmatical 'hooks' that have to link together to switch you from running, jumping, and climbing. If they try to do all three, there could be confusion and you could lose the linkup and get stuck in a little loop. Or there actually could be hook changes that lag out and time out and therefore leave you in a loop. The visual would be the snapping back to point X after climbing/descending a bit.

Programmatically, jump mechanics, running mechanics, and climbing may be fine. But you mix them in the way we players can, and you can create bugs that plague everyone. The fix however may be too deep and can butterfly some serious other mechanical issues.

[Jandric]

Because you can't pay non-employees for bug hunting. See: http://www.gamespy.com/articles/492/492177p1.html

Shakespeare had it right- "The first thing we do, let's kill all the lawyers."

Hey- Possible challenge idea!

[Cauthey]

Shakespeare had it right- "The first thing we do, let's kill all the lawyers."

Hey- Possible challenge idea!

New mob: Coin Lord Litigator (CR15)

Gotta hit him hard before he begins cross-examination.

[Ovrad]

New mob: Coin Lord Litigator (CR15)

Gotta hit him hard before he begins cross-examination.

Objection!

[Dark_Helmet]

Because you can't pay non-employees for bug hunting. See: http://www.gamespy.com/articles/492/492177p1.html

That is why the call them contests and typically reward the first person to report a replicatable bug. They are "one time" rewards and not on-going. Anytime you pay someone over time, you fall into that position where they may sue you.

And it would not be bug free. It is a statistical impossibility. Mostly free? Pretty Clean? Sure...bug free...never. There is a reason they are called bugs, squashing them is like trying to kill an cockroach infestation.

You have a large audience of people where some of us work with millions of lines of code who use vastely superior bug tracking products, so forgive us when we don't understand how some of these bugs can possibly reoccur without testing for them.

I guess your code is made to prevent automated scripting, but scripting that replicates actions that were bugged is bread and butter for some of our test departments. We also use loading products to replicate server loads based on replay of data (Examples: Shroud - have a script that bypasses all the portals and run a bunch of Clerics and with auras and their summoned creatures up and down the stairs near the chests in part I. Have an instance with 40 devils agro'd on a party of 12 PMs with lesser auras running in circles. Everyone has full buffs and massive HPs with regeneration going on and just start spawning duplicate instances.)

[Montrose]

You have a large audience of people where some of us work with millions of lines of code who use vastely superior bug tracking products, so forgive us when we don't understand how some of these bugs can possibly reoccur without testing for them.

I have found that there are certain major corporations who write millions of lines of code and who have very advanced testing procedures. Then there is the rest of the world that is basically in the stone ages.

I remember looking at a StarWest conference to see if I should send any of my people to attend. One topic was "Bug databases - Why they are good". I decided to save our training budget and spend it one something a little more useful...

So, yes, ideally they would have a ton of automation that ran through thousands of regression tests, but even that's not free. You still need people to analyze the failures. You still need people to maintain the test code. You still need people to run the lab automation, set up and hold machines, etc.

Test infrastructure for any sufficiently large program is not cheap. Hiring a horde of teenagers and giving them pizza to find bugs is. Guess which one the pointy-hairs go for?

[EarlofPain]

I still think of myself as a newb due to the fact there are a dozen plus quests I still haven't tried. All of those are due to the large amount of time it takes to either form a group/or just complete. The idea of challenge area that I can "goof" around in while waiting for a raid group to fill sounds exciting even if it's just tik tac toe. Also the concept of useless reward for each challenge to trade in for guild favor/crafting essences/ or even new cookies (fill that jar dude) would be a bonus. I think Mal is right about sometimes the fun is more important than the loot itself.

[Produktion_Malphunktion]

I have found that there are certain major corporations who write millions of lines of code and who have very advanced testing procedures. Then there is the rest of the world that is basically in the stone ages.

I remember looking at a StarWest conference to see if I should send any of my people to attend. One topic was "Bug databases - Why they are good". I decided to save our training budget and spend it one something a little more useful...

So, yes, ideally they would have a ton of automation that ran through thousands of regression tests, but even that's not free. You still need people to analyze the failures. You still need people to maintain the test code. You still need people to run the lab automation, set up and hold machines, etc.

Test infrastructure for any sufficiently large program is not cheap. Hiring a horde of teenagers and giving them pizza to find bugs is. Guess which one the pointy-hairs go for?

I've worked both in games and out. Nothing matches the unpredictability of game testing...except aerospace. Players do unpredictable things.Automation does not deal with that. Automation only deals with what you tell it to expect. I agree we can tighten up on some of the item a breaking and stuff like that, but eyeballs on monitors is the best way to test games.
Think about it, do people say, use a video editor program, and the first night they get it, spend hours trying to break it? Gamers do that. Gamers find bugs for FUN. It is a bit whacky if you think about it.

[MRH]

Man , here I was waiting for a Major message.... like..... yes in U12 we fixed the hirelings back to how they would only heal the master and would only attack if they were attacked.


/holds breath

[Galeria]

I've worked both in games and out. Nothing matches the unpredictability of game testing...except aerospace. Players do unpredictable things.Automation does not deal with that. Automation only deals with what you tell it to expect.

I am a website usability analyst. It still amazes me how many ways people find to do things that were never intended/never imagined even on simple straightforward websites and minor web applications. And they insist their way is the only logical way and you are ridiculous to suggest they might... read the instructions or look around a little.

It's not just gamers, although gamers might be the only ones that do it for fun.

I agree, nothing points out the issues like random live testers, no matter how thorough your staff/software testers.

[cdbd3rd]

Re: @ MissingMinds....
Apparently you don't spend enough time on the forums .....

If you'd've said that to me, I'd've sigged it for sure.

[quijenoth]

I don't get the hate for the ladder bug.

Sure it can be irritating at times, I've had a character dropped from the top to the bottom of a very long ladder on occasion.
My son was thrown onto the ladder in epic snitch just for turning round only to be dumped to the bottom of it as someone pulled the valve! (crispy)

If memory serves me though, DDO was the first MMO to offer vertical climbing and even Everquest II's attempt doesn't complete to how realistic it feels in DDO.


DDOs servers may be an issue - It certainly is the main contributor to what people perceive as unacceptable LAG. But lets get things into perspective. BTW when i say servers I'm referring to EVERYTHING that makes DDO work, cabling, hardware, software, etc.)

As far as I can gather, (and this is my hypothesis not an actual fact so please don't quote me on it as knowing exactly how DDO works.) zones are designed to hold 6 or 12 players maximum - each player interacts with each mob sending different information.
Area Spells create new objects that the servers must interact with, summoning spells create new mobs that interact with existing spells and players and other mobs. and so it goes on - each spell overlapping with another creates more work for the server.
Now with Mabar you still have the same size window of workers but you now increase the player count in that zone to 30 or 40! each casting area spells, summoning monsters and (if dungeon scaling is in place) creating more mobs. At some point the lag will kick in and the servers will have to play catchup to process the information.
To cope it will likely dump what it feels is unnecessary information (such as exact positioning of players and mobs) which results in freezing followed by teleporting to new locations.

The servers, like a business have an initial goal. Over the years that goal may change (as new content is added) but cost restrictions will prevent new servers being used so the old servers become stressed as there work load increases. You can only expand so far until the servers begin to struggle. Mabar Chamber is a perfect example of overworking the servers.

Lets look at it from a business point of view: You employ more staff to carry out the work but eventually your office space will need to be changed as you business grows (offering new products). At that point its a financial decision to invest in larger premises.

You want new stuff, great! but remember without a completely new game with fresh servers its just not cost effective to fix everything to work perfectly. At some point DDO 2 will come about if they feel its financially viable. but DDO 1 will remain as long as that old office still brings in some revenue.

[geoffhanna]

I've worked both in games and out. Nothing matches the unpredictability of game testing...except aerospace. Players do unpredictable things.Automation does not deal with that. Automation only deals with what you tell it to expect. I agree we can tighten up on some of the item a breaking and stuff like that, but eyeballs on monitors is the best way to test games.
Think about it, do people say, use a video editor program, and the first night they get it, spend hours trying to break it? Gamers do that. Gamers find bugs for FUN. It is a bit whacky if you think about it.

I would love to have a conversation about the use of Test Driven Development in large-scale systems if anyone is up for it.

But first, rule 1:
It will seem hard to the programmers and time-consuming to the managers. It is really neither, but it will seem that way, and you will need buy-in to even begin talking about it.

[BOgre]

Hey Major, not your department, but any word on when the Nov 9th date will hit the forums 'officially'? Or will we get scooped by every media site before getting the official announcement?

[maddmatt70]

Major, thanks for the posting. Although I appreciate the ambitious approach of DDO developers, it is unclear how succesful challenges will be??? We will see.

I want to comment on Mabar. I personally do not think that it should come back next year. This is not necessarily because of lag or any of the other difficulties associated with Mabar, but rather to keep things new and fresh and to try something new next time around. I trust in the Developers' creativity and feel along with the DDO anniversary Halloween is one of DDO's top holidays. Let the developers come up with something new and creative next Halloween to keep us excited...

[Hordo]

@ Folks not getting some hate for the bugs: It started out as a nuisance to players (stuff like the ladder bug) but when you keep getting told since Beta that X bug will be fixed and it isn't...it just really eats at some folks.

@ Maj: I wholeheartedly agree that "eyeballs on monitors" is the best way to find bugs and get them noted to be fixed. BUT it should be done before live...before Lama...on Mournlands. There are many of us who have been here since Beta and who have been consistently impressed by what we have and disappointed at the same time because we know what it SHOULD have been if the bugs were not present. No matter how large a group you may have on Mournlands, a gaggle of gibbering yes-men nodding like some geek-ed out bobble-head collection just to keep from being removed from what should be an important role in the games development is not going to do anything with those "eyeballs on monitors." They will nod their heads like the marionettes they are and bugs will go right on through to Lama and live. Get some folks who are willing to put up a fight to make sure that these bugs don't make it even to Lama. Open up Mournlands to some real, knowledgeable DDOers many of whom have been asking for that chance since Beta. That group knows what it will take to both fix the existing game and what to watch for to make sure stuff doesn't break in future releases. Use the resources you have at your fingertips and don't just put us on a shelf like exhibits in a menagerie.

[Gratch]

I've worked both in games and out. Nothing matches the unpredictability of game testing...except aerospace. Players do unpredictable things.Automation does not deal with that. Automation only deals with what you tell it to expect. I agree we can tighten up on some of the item a breaking and stuff like that, but eyeballs on monitors is the best way to test games.
Think about it, do people say, use a video editor program, and the first night they get it, spend hours trying to break it? Gamers do that. Gamers find bugs for FUN. It is a bit whacky if you think about it.

Usually you have devs do sandbox testing (good ones) and sometimes add that test to a build qualification (when possible). On the other end you have the full on testing of QA.

Then in the middle of the API's you probably also have client/server communication blackbox testing that's common for the LotRO/DDO engines.... then you have the actual 3d client itself which probably has its own testing hooks - maybe. Debug hooks running throughout everything (whitebox testing). Do you have a way of putting a larger blackbox test around the entire systems setup? Something that's basically a stats randoms, goes by all the systems rules to create various legal character setups, monster setups with legal equipment setups and then checks to output system stats layer to see what bonuses (attack, damage, speed increase, etc) would result? Or is this all too murky with the actual client render to have a stats-systems black box testing? Just wondering.

Does Eladrin have a black box?

I think the most impressive QA setup I've seen was at <LARGE GFX COMPANY> where the same render test, calc vectors and expected results ran on every platform conceivable starting with their flow design language to RTL design to gate design to FPGA implementation to actual device (using both the final frame buffer and then even an HDMI/DVI signal interpreter to check the expected outcome). Same tests from top to bottom. Of course they still get tons of bugs due to the changing landscape of OS'es, Capabilities, Standards, Graphics API's and the ways in which companies use the topside of the graphics API.

[BOgre]

I don't buy that answer. Whole cloth fabrication. We didn't TRY to break hirelings. We were told they were smarter and wouldn't draw agro when set to defensive. We summoned a hireling, set them to defensive (well, they summoned pre-set to D...) and off they went to try and get killed. The entire 'test' took about 5 seconds. Not to mention any of the other fixes listed in the notes that turned out to be bunk. C'mon 'fess up!

Try this: <feature a> was slated to be fixed and tested, but we dropped the ball. We're sending it back to the drawing board.

Instead of: <feature a> was tested thoroughly but you players pressed a bunch of buttons we didn't expect and broke it.

[BOgre]

Oh, and if you can SEE that certain players are spending considerable time and effort 'bug hunting' as you put it, and that these certain players often do FIND a great many bugs, and report them, and provide screenshots, and video, and detailed numbers, and stats, and on and on, then why oh why are you not hiring these individuals? Clearly you need them, clearly they enjoy doing the work. At the very least put them in Mournlands. Yeah, we'd know immediately that they'd been placed, since their usual flood of forum activity would suddenly drop to nearly nothing, but what difference does it make? The game needs better 'bug hunters' and they're out there... er, in here... whatever. Stop making excuses.