Honest (S)crap!

Well, well, well.  It seems that my buddy Ventris over at Alt Much? is trying to bribe my into posting with a shiny.  He gave me this:

Nice, huh?  He gave such a nice description of what it was and its history, I’m not going to try to one-up him.  Visit his site and learn all about why I’m about to do what I’m about to do.

My picks:

1. Breakfast at War. ’cause stunties tase like chikken.
2. Blame the Healer. I only recently started following this blog, but it’s always worth stopping by to see what is said.
3. Evan Sims‘ blog often has interesting insights from a technical standpoint.
4. Girl IRL. This girl posts more often than I eat.
5. Cry me a RvR. Miss Vindaloo tells it like it is, and pulls no punches.
6. WAAAGH. Syp’s blog is one of the best ones out there.
7. Last and not least, a pass-back to my buddy Ventris at Alt Much?. See ya on the battlefield, buddy!

It seems I’m also supposed to post 10 things about me, but that will have to wait for another time. “Real Life” rears its ugly head again 😦

WCPI Spotlight

Got a few more WCPI folks to pass around.  First, welcome back to Blame the Healer.  I love the post about the Chinese version of WAR, being a bit of a language whore, but do wonder where they are going to put all those Chinese players after the server consolidation… 🙂

Second, check out Evan Sims.  Evan is a game designer and the author of MinChat, an extremely handy addon as well as having useful tips like how to skip entering your password in the WAR launcher.

Patch Happy, Part Three

Well, it seems the patches from yesterday (was I the only one that got two?  Or did I just miss one from Monday?) have cleared up the crash to desktop at the selection screen, at least for me.  So, it’s high time to wrap up this little rant about what, why these things happen.

We’ve answered the what and the why, really, so the only question that I had revolves around my guildmate’s comment, “Eh, patches cause crashes”.  Why are we all so indifferent to this?  Why do we accept software that locks, crashes, eats our data and causes us to tear our hair out in frustration to be the norm?

This is not relegated only to games, not by any means.  I’m talking about everyone, all users of software.  Do we like crashy software?  Hell, no.  Do we get frustrated, call customer service, go round and round and finally delete the piece of crap, fuming all the while about it would have been so nice if it had just worked?  Oh, yes.  But are we ever surprised?  No.  Because software causes crashes.  And why do we accept that as Truth?

Let me tell you a little story.  Now, this is apocryphal because I did not actually see this happen as it was before I entered the workforce.  But, it rings so true even if it is not completely correct in detail it certainly captures the spirit of the issue.  Some years ago there was a program that was written to help do complicated and tedious mathmatics.  It started as a small idea and was added to and modified over the course of a couple of years.  Eventually, someone realized that people might pay money for this no-longer-little application, so they started selling it.  Eventually, someone did something with the program that had not even been done (or tested!) before and noted that the program supplied the wrong answer.  They also noted that if they had been using the program to do something important, like, say, designing load tolerance for bridge, this kind of error could lead to the collapse of the bridge.  While there were cars and such on it.  Ouch.

So they went back to the developer and told them about the problem.  Now, at some level, the developer or one of his office mates must have known something like this was likely to happen because when they sold them the application, there was a little bit of text saying that they (the selling/developing company) could not be held responsible in the event the program was wrong.

WTH?  Talk about the ultimate get out of jail free card!  So for years, we as a consumer base have been buying into the idea that of course, software ships with bugs.  You can’t get every one, after all.

Bull.  Remember the triangle?  That line is a way to wriggle out from the “quality” side.  You have a deadline, you’ve capped your budget, you need to get this puppy out the door!  What do you do?  Slash quality of course.  The user’s aren’t expectning a bug-free product anyhow.  Why bust the bank over this?

This is the prevailing attitude of almost everyone in the software business I have ever worked with, from maintenance programmers to VPs of development.  Release the product when it’s “good enough”, and the users will complain, but not leave.

Well, that’s wrong.  And we, all of us, are letting them get away with it.  As long as we shrug our shoulders and say that it’s OK for the developers to cut corners on quality and give us something less than what we purchased, they will continue.  It’s a winning model for them.  They get our dollars, users stay and are at least satisfied if not happy, and they get to keep their jobs.

I say enough.  I say it’s not right that Mythic obviously shorted their testing cycle and rushed 1.2 out the door.  My user experience was the worse for it, and many others as well.  Is it life threatening?  Of course not.  But the scary thing is, one day it will be.  Do you really want a medical device/automatic braking system/space shuttle designed by an industry that ships software when it’s “good enough”?  I sure don’t.

Shape up, Mythic.  We’re watching you.

WCPI spotlight

I’m going to take a break in my series of posts about the crashes due to the 1.2 patch to point everyone to this article at Breakfast at War.  Pancakez does a great job explaing the relationship between armor, toughness and wounds.  It really helps to understand this, and to understand which stat helps the most in PvE versus RvR so that when choosing gear the decision process is a little more sophiticated then “Higher numbers are better….hrrrr”.

Just for curiosity’s sake I’m going to plug my Witch Hunter’s stats into his formula once I get home and can log in.  I know it will just make me weep, but I’m curious to see if my “seconds of survival” is measured in decimals.

Patch Happy, Part Two

I’m baaaaack.

Last time I was talking about the recent patch to WAR, and the sudden CTDs I was getting as a result each time I tried to log out to the character selection screen.  My guildmates’ reaction to this was “Eh, patches cause crashes.”

There are two things I want to examine about this situation.  One, what might have caused a game to be released with a bug so severe that it shuts the client down, and two, why we as users think not just that this is OK, but that it is expected and accepted.

As I said before, I have been professionally developing commercial software for 20 years now.  Standard disclaimer applies: I have never programmed a game in my life, and I certainly have no knowledge of the working of the WAR program.  That said, there are some overarching concepts in software development the apply regardless of what you are developing.

In part one I brought up the concept of the “software triangle” and mentioned I did not think that it was responsible for the bugs in this latest patch.  While it’s certainly possible, the honest truth of the matter is that this patch has been very smooth for the most part.  The fact that not everyone is crashing to desktop (I and Jennifer at GirlIRL are the only two that I know of at this point) indicates that the software triangle probably was not the cause.  Problems with the triangle are usually more catestrophic.

So what did happen?  Most likely a simple truth is the cause of our CTD woes: Software is complicated.  All software, even the simplest “Hello, world” program is complicated.  Those out there who remember Charles Petzold and “Programming the OS/2 Presentation Manager” remember the eight pages of raw C init code you needed just to get your “Hello” to display know what I’m talking about.  Add multiple layers to that, interacting subsystems, OS, hardware and myriad other things and you start to get an idea what I mean by “complicated”.

So, let’s look at this CTD issue in WAR 1.2.  For those not experiencing it, when you log out of the game to go to the character selection screen, the game window disappears and shuts itself down, poof.  Colloquially known as a “crash to desktop”, developers refer to them as “abend” (that’s ab-end, as in “abnormal end”) and the most common reason for that to happen is the program tried to use some data that simply was not there, something known as a “null pointer reference”.  When the program could not find what it was looking for, and there was not protective code surrounding it (i.e., no error catching), the application simply shuts itself off.

So, that does not sound too bad, right?  We know that it happens something like 4 out of 5 times (I was able to log out successfully once) going to the character screen, we know we are looking for a “null pointer reference”, whatever that is, so it should not be too hard to find, right?

Wrong.  The great bugaboo with errors like this is inconsistency.  4 out of 5 times means that 20% of the time the code is not hitting the “null pointer reference”  Why?  It’s only happening on some people’s machines.  Why?

Now you start to see (I hope) why I said I did not think that the software triangle was responsible for this error.  If it were, this would have happened on everyone’s machine, all the time (OK, OK, most machines most of the time…happy?).  The point is, it probably would have been caught before it went out the door.  But, a crash that only happens on some machines some of the time is much less likely to be caught for the reason I stated above.  It’s complicated.  Lining up all the moving parts of a complex program like an MMO and stepping through every single iteration of every single possibility of everything that could go wrong…would be nearly impossible.  It makes cleaning the Augean stables look like child’s play.

So we’ve answered question one, above, why did this happen.  Because testing every single possibility in a complex system is not feasible (note I did not say ‘possible’) in a reasonable amount of time, so bugs will slip through.  I’ll save the answer to question two for the next installment.  See you then!

Patch Happy, Part One

Well, we got the new patch last night.  Overall I think the quality of the patch is pretty good, but I do have a couple of point that are worth examining.

First, would Mythic please take a clue from other MMOs like City of Heroes and kick off the “big patch” when the user logs off, instead of when they log on?  I sat down to play and was treated to a 171MB update, which was of course happening at the same time as everyone else was trying to get the same 171 megs.  To be fair, I have a slow (read: DSL) connection and even so the entire patch took only about an hour, but that was an hour I could have been snuffing Orcs and other tasty Destro minions.

OK, rant off.  Now to climb on my soapbox for real.  Since patching, I am getting a lot of CTD (crash to desktop, for the two of you who have not heard the term) whenever I log out to the character selection screen.  It’s not a huge big deal (at least it has not happened in the middle of a scenario or something!), but it does mean that I have to re-start the game and wait while it does all its leader screens yadda, yadda, yadda. 

More importantly, when an application crashes like that it’s very often the result of a misplaced pointer in the code (or as we used to say “the pointer is IN THE OPERATING SYSTEM”), which can have nasy side effects and lead to even less desireable results, like crashing the video card or the dreaded Blue Screen of Death.

So I log back in and apologize to the guildmate I was talking to when it happened for abruptly leaving the conversation.  His reaction was “Eh, patches cause crashes.”

WTH?

Now, I’m not taking umbrage with him, because of course this is a true statement.  My reaction is more to the fact that we all accept that “patches cause crashes”.  when what we should be demanding is a better quality product for our dollar.  Let me say up front that I develop software for a living, not games, but application software and this is not a problem that is exclusive to games.  This is endemic to all software development I have ever seen or worked on.  The question is, why do we keep letting the developers get away with writing crap and then patching over the crap with more crap until the system basically accidentally works and they call it done?

To avoid the hypocrisy inherent in my argument, I include myself in the list of developers that release crap.  Under the guise of anonymity, I will admit that I have released software knowing full well that there were bugs in it that could cause as serious an error as a CTD, and the decision was made to send it out and patch it later.  Notice that I switched to the passive voice, there.  While I will take responsibility for the release (I was running the engineering department, after all), the decision to release the application came from higher up.

Someone a long time ago told me that the answer to “Why?” was always “Money”, and that is certainly the case in every instance I have been involved in where poor quality software was released to a customer base.  It is deemed too expensive to develop most software to the quality level needed to ensure really reliable, robust product.  Is that true?  I contend that it is not, but that is an argument for a different post.

From a developer’s perspective, we want to write good software (yes, there are nuts out there who like to write viruses, but I’m talking about the majority of people, not the crazy minority).  We got into software because we love it, we want people to use our applications, play our games, and we want to be the solution to a problem.  We don’t want to write buggy crap, applications that don’t do what they should, or that are incomprehensible, or that crash every 15 minutes.

So why don’t we?  Well, there are a lot of reasons.  Let’s start with the one that I have seen be responsible for inflicting the most heinous releases in my career.  There is a little triangle we in the software biz use that has cost on one side, quality on another and time on the third.  What this means is that with any development effort, you have any two of these, but not all three.  You can be on time and spend little, but your quality will suffer.  You can have a quality product on schedule, but cost goes up.  Or you can have a quality product with low cost, but your timeline grows.

These are all relative to each other.  The more quality you try to pack in, the higher your costs and the longer your timeline.  Try to reduce costs and your deadline stretches over the horizon and you never finish.  Reining in the timeline means increased costs (often in the form of added bodies, which is another topic for a different post) and reduces the quality of the product.  Decreasing the cost (possibly by NOT adding more bodies) means that the end product will be so buggy as to be unusable.

Is this what I think happened with the latest patch for WAR?  Actually, it’s not.  But, I’m going to save my thoughts on that for another post, because I note that as I warm to my rhetoric, this post is getting awfully lengthy, so I’ll spare a few tidbits for later on.

Seven Favorites

My god, I’m lazy.  This is my first post in a month.  Thanks to Syp over at WAAAGH! for providing me with a topic so I don’t have to use my poor, strained brain.  Here are my favorite seven:

Favorite Zone

I’m going to have to go with High Pass for this one.  I love the snow and ice, and I really enjoyed the content there.  Troll Country is a close second, though.  I really started to get a feel for the game there.

Favorite Race

Empire, hands down.  I enjoy every class that Empire has (of course I have one of each!  The Alt-itis is strong in this one), and have never been able to get into playing stunties (sorry, Dwarf-lovers).  The high elves are OK, but there are too many guys in damn dresses for me to take seriously.  Destro side…I’m not sure.   It’s a toss-up between Greenskins and Dark Elves.  Decisions, decisions.

Favorite Career

I still luvs my main, Serica, who is a Witch Hunter.  I will admit to being frustrated occasionally trying to figure out how to play a MDPS class with, essentially, no armor, but the look and feel of the class still keeps me coming back.  I still let out an involuntary “heh” every time I set a DoK on fire.  Heh.

Oh, and the hat.  Love the hat.

 Favorite WAR Feature

RvR scenarios, definitely.  Without them I never would have explored PvP. 

Favorite Skill

Pfft.  I’m a Witch Hunter.  Absolution.  Was there another choice?

Favorite Scenario

Tor Anroc.  For some reason, even if Order gets it handed to them, it’s always fun.  Well, except for the time Serica accidentally picked up the glowie.  Dog pile on the Witch Hunter!  Ouch.

Favorite Live Event

Gotta go with my buddy Ventris for this one and say Heavy Metal.  All the quests were do-able solo (assassinate 10 Keep Lords?  Seriously?) without TOO much grinding, and the rewards were nice.

Ebil Marketeers

There are dings, and then there are DINGs!  While my primary focus is always on achieving the next level for a character, there is still a sense of accomplishment with getting a level of renown, getting a new influence reward or even getting that last crafting point that tips you into the next band.
 
My first real long-term MMO was City of Heroes which, at launch, did not have a crafting system.  This was actually by design, they were trying to make a loot-less MMO.  That failed, of course, and they have since retro-fitted in a crafting system.  But before WAR, I never really payed attention to crafting in any games I played, preferring just to buy what I needed and keep on leveling.

So I decided to give crafting a try, and it turns out that I enjoy advancing my characters through the various crafting levels as well.  With my main alt, Johcasta (21/16 WP) I decided to try cultivating and apothecary, reasoning that the herbs I produced with cultivating I would use in apothecary.  Clever, huh?  Well, there are two problems with that plan: One, cultivating won’t produce the water I need for apothecary, and you can only have one gathering skill, and two, I never have enough seeds in a band to get me to the next band.

Now, I understand what the design intent was here.  If there is no one gathering skill that will produce all the mats needed for any of the crafting skills (still only two?  C’mon, Mythic!) then that encourages you to trade with guildmates or teammates, or increases traffic at the auction house, all of which are good things.  But in practice what it leads to is me hoarding my meager supply of seeds, and having all my alts send me all their seeds (and water).  I then sit on them until I have enough to grind up to the next band.  Based on the prices of some of this stuff at the auction house, I don’t think I’m in the minority on this.  Mid-level waters go for a gold apiece, and seeds in the 25-50 range often cost that much (if you can even find any!).

However it’s happening, demand is far outstripping supply for these items.  Unless there are a bunch of “Ebil Marketeers” who are manipulating all the data, the vast majority of us could open our vaults and get buried by the deluge of mats raining down on us.  So, what to do?

Well, I’m only one player, but I’m gonna step up.  I’ll be putting my mats on the auction house (for REASONABLE prices, vendor price + posting price) in an attempt to get this under control, or at least a little less ridiculous.  If enough of us do this, maybe we can actually have common mats at reasonable prices.

Ding!

w00t!  Dinged 31 last night!  Goodbye T3, hello nurse!  I mean, T4 🙂

Still on my Soapbox

I know this is only the fourth post, and I know that I’m supposed to be talkng about leveling (seeing as how it is my favorite thing to do), but I’m not yet done ranting about PvP.  Hey – It’s my blog, if you don’t like it, take it elsewhere!

Anybody left?  Just a few?  Okay.  Actually, seeing how closely coupled PvP is in WAR, you really can’t avoid it and level with any kind of speed, so I am still talking about the ding.  As I have mentioned before, PvP in WAR is a great way to level up, which is why I do it.  It’s just frustratng when you get into a scenario and get trounced.  I don’t mind losing (much) as long as it’s not by a couple of hundred points.

I was on my main, Serica (Witch Hunter, 30 [w00t!  I dinged since the last post!]) and we were running the High Pass Cemetery scenario.  Now, I’m really bad about not reading load screens, so I will admit to ignorance as to exactly how the scenario worked.  I know, I know…gimme a sec.  It’s part of my point.  I generally just come out into scenario and follow along behind folks and shoot Destro when it shows up.  If we stop at a flag, I’ll stop, too.  I won’t carry the McGuffin, though, (if it’s that kind of scenario) unless it’s by accident.  I’m just too darn squishy. 

Anyhow, I’ve only run this scenario a couple of times before (due to the insane amount of love Tor Anroc gets) so dutifully followed the pack over to the nearest flag and parked for a sec.  When the majoity of them moved on, I made sure some folks were staying with our flag and moved on with them.  Woo hoo, killing time!  We were actually doing pretty well, targeting healers and taking them down and cleaning up the rest.

I looked up at the scores, though, and we were lagging pretty badly.  I was confused, so I asked about it in scenario chat.  Someone responded “Have to have both points.  Group stay at flag, the rest with me”.  So that’s what we did.

We WRECKED them.  Everyone sort of rotated in and out of flag guarding duty and destroying Destro duty.  We held both points for the majority of the match and only (temporarily) lost one or the other.  We wound up with something like 500 to 145, and I got 10K shiny XPs to my next ding!

So what, you ask?  If we were killing them that easily, we would have won anyhow?  Maybe, maybe not.  Like I said, we were lagging behind pretty badly.  The reason we won as well as we did was simple: Communication.  That one directive (and make no mistake, he was telling, not asking) told everyone in the scenario what they should be doing and to go do it.

I run too many scenarios where not a single word is said in scenario chat.  Well, except for expletives and whining that Destro always wins.  It doesn’t take much, just a quick note to let folks know what to do can make a huge difference.  Just letting them know what you are doing can make just as big a difference.

Don’t be afraid to speak up.  If someone knows better, or has a better plan, then do that.  But having any kind of plan, and letting folks in on it, is vastly better than twelve people running in random directions doing random things.