Sunday, December 25, 2011

Curse of the Performance Tester?

Seriously?!? After wrapping gifts until nearly 5am (I was behind by even my standards due mostly to work travel, client commitments & preparing to close the corporate books for 2011), and getting up before 8am to celebrate Christmas with my boys, I finally stole a few minutes when I noticed they'd both fallen asleep on the couch to play with *my* new toy (i.e. install Skyrim), only to be foiled by...



... poor performance!

You tell me, is this the curse of the performance tester? or is this just another reminder that there are still plenty of orgs out there that might want to seriously consider investing in some training and/or services from a "top-shelf" performance tester -- whether that's me ('cause it would be nice to make some $ during my unexpected availability due to Skyrim's unavailability), or someone else ('cause if Steam servers get less busy, it' be nice to have some play time before getting back on the road) isn't the point.

The point is, as consumers, we deserve - and should demand - better. And as technologists we should stop pretending that what we're doing is good-enough, and proactively make the appropriate investments in performance-related services, education, processes and attitudes before consumers rebel en mass (like they did when GoDaddy announced, then backed down, from strongly supporting SOPA earlier this week).

Oh yeah, I almost forgot - I wish you all a great holiday season and hope none of your holiday plans are foiled by the Curse of the Performance Tester.

Update: T=0 hrs 

Steam connected. Skyrim updates begin downloading @ 34Kbs (Tested my connection and data rate -- I'm well over 750Kbs)

Update: T+5hrs

Still connected, still downloading Skyrim updates. Down to 31Kbs. 40% complete. Dunno if it's Steam, Bethesda, or even if the two are related, but I wonder if it would still be downloading so slowly if I'd spent the last 5 hrs doing some perf testing instead of periodically looking at the number to see if I'll even get the game installed before my first con-call in the morning.

Update: T+5hrs 39min

Suddenly jumped to 1.8Mbs and am at 62%. Would love to know if someone fixed something, or if a bunch of folks just gave up around 11pm

Update: T+ 6hrs 8min

Updates downloaded... time to install... wonder if I'll even managed to stay awake long enough to get through character creation.


Update: 4 days later

Last night I installed the game on a new machine... went through the entire process and *averaged* 1.8Mbs for the download with a peak of 2.0Mbs. In theory, I should get up to 10Mbs for downloads through my service provider, so I'm curious whether there's a 2.0Mbs speed limit on the server side, on my router, due to wireless signal strength, etc, but I think I'd rather spend my time w/ my kids, doing some billable work, and/or actually *playing* the game than research that right now.

BTW, the game lives up to the hype, so in the end, I guess it's worth it -- making me just one more Netizen guilty of tolerating poor performance because "getting it done online eventually" is still better than the alternatives.

And that, my friends, is the entire point. Every one of us that quietly tolerates poor performance is giving companies tacit permission to continue delivering poor performance. We are the reason we are faced with rampant poor performance, not the service providers. Companies simply aren't going to put any more focus on improving performance than they have to in order to keep us at "quietly tolerating". If we want better, we need to be less quiet about it... myself included.
 
--
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
Director, Computer Measurement Group
About.me

Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contributing Author, Beautiful Testing, and How To Reduce the Cost of Testing

"If you can see it in your mind...
     you will find it in your life."

7 comments:

Shankar said...

There are serious number of organization's who still do not believe or understand the importance of investing in people.

Organization's should start training/educating developers and performance testers about the importance of performance, its implications and the ways it can be achieved, rather than just training them on tools (dev or pert testing tools).

error 1310 said...

Hello,

I think this was an informative experience for you.

ihodgetts said...

...but how long did you (finally) spend 'lost' in Skyrim once you'd entered? ;-)

Personally I've learned not to expect these big AAA titles to even work at peak times (like MMO launches). Having been part of various beta/stress tests in the past I know just what to expect (it's actually what got me interested in performance testing in the first place).

Skyrim is awesome though (and I try not to use that word unless I really mean it). Hope you eventually found some quality-time for yourself and your "new toy".

Unknown said...

At the cost of sleep I certainly could have benefited from, I've spent about 10 hrs in game -- and if you are into the combination "explore, solve puzzles, great scenery, multiple plot line, first person combat" type game, Skyrim is dead amazing!

My point on the curse was that this happens to me whenever I have something timely to accomplish... it is true that I should have probably expected as much for a well hyped game right after Christmas, but the same thing happens to me when I'm trying to buy airline tickets last minute, or trying to pay bills on their due date, or whatever. I guess I need to start blogging about those times too, huh? :)

Uniblue said...

Hello,

Great information in this post and I think this information will be helpful for us.

Error 1303 said...

Hi...

Nice post and really this post helps me a lot because i am searching this kind of information and i finally find in your post.
Thanks for sharing

affordable web design said...

Thanks for post. It’s really informative stuff.
I really like to read.Hope to learn a lot and have a nice experience here! my best regards guys!