Bring Me The Stats; Stat!

Posted on by
6

Disclaimer – There is likely nothing new in the following post, and I’ll likely rehash SEVERAL points that have been made hundreds of times… but I need to get this out of my system.

So I was recently informed of a manager who LOVES stats (shocking, I know).  I emphasise the LOVES, as I was told that there was a huge amount of passion that went along with this information.  In fact, the LOVE was so great it even spread to spreadsheets… so even the home of said stats was included in the LOVE.  Now that’s passion.  I’m not going to question the passion… that is important.  I am however going to question stats.

Now to give you some context, these are the common stats that get sought… namely number of test cases and percentage of those test cases that have been completed.  There are valuable stats out there in the wild, but these are not part of that group.

I want to look at a simple example…

We have three testers:

Han
Luke
Chewy

And the following stats:

Test cases outstanding 50
Test cases complete 50
Total test cases 100

What information can you gleam from the above stats?  I’ll let you in on a secret… the most you can gleam is that there are 50 test cases outstanding, 50 test cases completed, and there is a total of 100 test cases.  And even then, what does it all mean?  Who knows!

It would be a common mistake to say that as the stats show 50% complete, testing must be half way through.  If you were to ask someone to plot a point in a timeline as to where the testing is at they would likely plot that point dead centre.  So you’ll now have management telling even more management that testing has taken two weeks so far, and you’re halfway through, so there is two weeks left until you’re finished.

Let’s stick with the timeline theme and have a look as some of the things that happened in those first two weeks while 50 test cases we’re being completed:

  • Han was called away on a ‘cargo’ run and therefore was absent from the team for 3 days.
  • Luke did all that he could to make up for this by coming in on the weekends, but only managed 2 days instead of 3.
  • Chewy was there for all 10 working days, but spent most of the time moaning and groaning instead of working.
  • They all agreed that it would be a great idea to execute the quick and easy test cases first so that the manager would not only LOVE stats in general, but would also LOVE their stats.
  • To help with the above, and knowing that he had been absent for 3 days, Han decided to working a few extra hours each day – or just when he could be bothered.
  • Because Luke had worked so hard on the weekends he just couldn’t do a full day on the final working day; he was simply too exhausted.
  • They raised a bug at the beginning of week 2 which blocked 2 full features from being tested or 2 whole days.
  • Oh, and they also had a developer leave the team at the end of week 1 so had an extended lunch break to farewell her; Leia was her name.  Lovely lass that was affectionately known as ‘Princess’.
  • Etc, etc, etc.

This is hardly a minute by minute blow of ALL that happened during the first two weeks of testing, but you get the idea.

So should the manager still think that we are half way through testing?  It is reasonable to assume that all of the above will also happen in the final two weeks of testing?  It sounds as though the test cases may be more complex and time consuming, the loss of ‘Princess’ Leia will no doubt have an impact on bug turn around times, and Chewy may even jump ship if he’s so disgruntled!  There is a dev shop down the road that are looking for testers; they’re called Empire Development and he may be in with a shot given his extensive knowledge of his current employer, Rebellion.

If the manager has a crystal ball then they may be able to plot the testing against the timeline a little easier, but alas… there is no such thing.  So what do the stats tell you… still nothing.

There could be ways to make these stats valuable to somebody, but how much work would be required?  Tur manager could look at every test case and estimate how long each one is likely to take; however I dare say they would spend more time trying to build the value in the stats than it would take to just finish the testing.

Stats like these are dangerous and can back you into a very nasty corner.  Don’t hide behind these stats unless you know exactly what they mean, and can tell their story… in great detail.

Ahhh, now that feels better.  Thank you for allowing me to get that off my chest.  I know it’s a very simple example… so perhaps you can understand why it’s so frustrating that people still don’t get it!

Don’t Make Me Think…

Posted on by
2

Originally suggested to me by Darren McMillan in this topic on the Software Testing Club as a possible book to read in order for my wife to learn more about usability, I recently decided to read it myself…

Don’t Make Me Think: A Common Sense Approach to Web Usability (Second Edition), by Steve Krug.

First up, what an easy read!  Steve mentions in the book that it’s the type of read you can complete on a plane ride.  Well, not so much for me as I’m a terribly s-l-o-w reader, and my plane rides are short; however I did complete it quickly for my standards.  I think the ease of the read is due to Steve embedding usability into the book.  The text is a great size, there are larger blocks of white space at the bottom of most pages,  sections are broken up logically with distinct headings, and the use of pictures are a visual treat.  You never get the sense you a reading a ‘boring’ text book, which is a common thing when reading books of this nature.  

It’s been a great way to start my new learning journey.

Steve begins by explaining the second edition and how it differs from the first.  It’s actually an interesting couple of pages and sets the scene well for the remainder of the read.  Along with additions to existing chapters, Steve has added 3 (I think) new chapters.  So I would suggest it’s worth the read if you have only looked at the first edition.

Chapter 1 explains Steve’s number one rule of usability, ‘don’t make me think’.  This explanation is highlighted by some great visual representations of a user’s view of a web site.  It was very thought provoking and amazing to read about milliseconds of a user’s thought actually making a difference to their experience.  Even the label used on a particular button can make a huge difference in how long it takes most users to realise what the button does/can do for them.  A good example that Steve uses for different labels on the same button is – Jobs – Employment Opportunities – Job-o-Rama – I agree that I would have to stop and think before clicking on Job-o-Rama.  The different ways in which search functions can be executed was also a great way to highlight the potential thinking time of users.

Note – Steve doesn’t just highlight the issues, but also provides potential solutions for each throughout the book.

Chapter 2 takes brief, but powerful look at the way users use the web.  In this chapter I finally learned what satisfice means.  Even though I’m a regular visitor to James Bach’s web site I have never taken the time to learn what that actually is.  A combination of satisfy and suffice; in essence picking the first reasonable option (in the context of picking what we see on a web page) instead of searching for the ‘best’ option.  Think about it… when scanning a web page for a particular piece of information, if you see a link that ‘could’ be related to that information, you’ll likely click on it.  Steve goes on to explain how this impacts people’s decision making and also provides a great metaphor in relation to fire fighters.

Following on from this, chapter 3 talks about designing pages for scanning rather than reading.  Steve covers loads of different information in relation to how to set out your web pages.  Hierarchies, conventions, and breaking up your page into readable and usable sections are all written about neatly and concisely.  One particular ‘a-ha’ moment for me was how Steve related the layout conventions of a web page being similar to printed media, even newspapers which have been following such a convention for years.  Each newspaper follows the same convention, thus making it easier for readers to scan for the information that want, no matter what newspaper they are reading.  Such conventions are very useful on web sites.  It allows the user to scan quickly for what they’re after.

Chapter 4 is a very brief one that looks at why users like mindless choices.  Steve makes a good point when stating the amount of clicks is not as important as how hard each of the clicks are; i.e. How much do the users need to think before making the choice to click.

Chapter 5 tackles writing for the web and as I was reading this my mind quickly went back to the wonderful book Weinberg on Writing.  In this Gerry often mentions that unnecessary words should be removed from your writing, and Steve covers the same in this chapter.  One very useful section of this chapter is where Steve takes a blurb from a web site which is 103 words in length, and reduces it down to 41 words while covering all the same information.  Each logical section of the blurb is broken up while Steve comments on how each of them can be improved, finishing with his 41 word version.  These types of examples that are spread throughout the book are extremely useful and thought provoking.

In keeping with the ‘usability’ theme Steve provides a warning, so to speak, that chapters 6 and 7 are lengthy ones; so a packed lunch is advised.  Personally, I love this.  It suits my reading style, as I rarely commence reading a chapter unless I think I have time to finish it. Luckily I had just sat down on a plane so I was comfortable to get reading.

Chapter 6 covers navigation, and you can understand why this is one of the bigger chapters… as navigation is right up there when it comes to usability!  Using a metaphor of physical shopping Steve provides some very good examples of how navigation in a physical location directly relates to a users experience on the web.  There are some minor flaws in the process diagrams which Steve is aware of; however the impact of those diagrams is not lost because of them.  This chapter is jammed packed with useful information and I had several ‘a-ha’ moments throughout.  I think the majority of it is so obvious that we simply take it for granted.  When it is described for you it provokes thoughts that otherwise wouldn’t have been, and that was very useful for me.  One section that I do want to single out is the Trunk Test.  Look it up, it’s great.  I have actually used this on a project with some great information being gleamed.

The next large chapter is 7, and once again you can understand why as it’s all about the home page.  The first page that users see; the page where they decide if they are going to stay or go (i.e. the most important one).  Steve talks about the common battle amongst stakeholders for their slice of the screen real estate pie!  Having not worked on a web design team, this was new to me; however completely understandable.  The memorable moments from this chapter come from the many home pages that Steve includes in the book along with his comments on where they go wrong, and where they go right.  There are little exercises for the reader in which they need to determine the point of the site the home pages are representing.  Once you have a go then Steve walks you through the exercise as he sees it with loads of great information.

Chapters 6 and 7 are definitely the ‘meat and potatoes’ of this book, and cover enough valuable information to cover the cost of the book by themselves.

Chapter 8 takes a look at the web design team arguments and how they can be avoided.  The key take away for me was that each member of the design team will have their on take on what the ‘average’ user likes and dislikes; however this will be tainted by their own likes and dislikes.  The fact is, as Steve points out, there are no ‘average’ users.  All users are unique.  The answer for such arguments?  Testing.

This is where my eyes widened… testing.  Chapter 9 is all about usability testing, and more than that; about doing it inexpensively.  I loved this chapter, and I think that’s because I have lived through it on many occasions.  Steve provides a great method for keeping usability testing costs to a minimum, so that you can do more.  Many smaller rounds of testing are better than fewer big rounds of testing, and Steve tells you how to keep them small and valuable at the same time.  There are lots of great pointers in this chapter and after reading it I’ll definitely be grabbing Steve’s second book, Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems.  A whole book on the subject of usability testing.

Chapter 10 is actually quite an entertaining read.  Steve opens up with a story about one of his web site experiences and then goes on to talk about a user’s ‘reservoir of goodwill’; and how quickly it can be drained!  Steve covers both things that can diminish goodwill, and those that can increase it.  It’s a clever little chapter that reminds us of the small things that can be done to help the user’s experience be a positive one.

Many of us see usability and accessibility as two separate things; and I guess they are… but you can’t really have one without the other.  Chapter 11 covers accessibility along with cascading style sheets (CSS).  I think one of the more profound pieces of information for me was that to help accessibility of your web site, you need to get the basic usability in order first.  Making a site easier to use for a non-disabled person will almost always make it easier for a disabled person as well.  So logical, but something I had not really thought about before.  I would normally jump straight to the accessibility type issues (screen readers, etc.) and forget about all the basic usability issues while doing so.  So thank you Steve for that little gem.

The final chapter, number 12, is a great little chapter that talks about what you do when your boss wants you to make a change in the design that is actually a poor choice in relation to usability.  Steve actually offers to write your boss and email explaining why a certain decision is a bad one, and provides the email template that he would use.  It’s classic!

Despite that being the final chapter, there is more.  Steve provides a great list of recommended reading and I think I’ll be looking into most of the recommendations he gives.

I thoroughly enjoyed this book.  As it’s my first of it’s type, and the beginning of my self education in this particular area I think it did a great job of opening my eyes to the common issues faced when dealing with usability.  I actually feel that I’m better armed with useful information that I can apply straight away (and have done so).

Thanks Steve!

*Insert text Here*

Posted on by
5

Ever since my very first blog post, almost two years ago now, I have thoroughly enjoyed writing.  I’m not really sure what I enjoy most about it, I just like doing it.  I’ve never been a big reader, so maybe writing is my outlet (even if it took me thirty years to realise it).  It also helps to be writing about things that I’m passionate about, and having a personal blog allows me to say what I really want to at the same time.  I don’t think I’ve called anyone an idiot yet, but I could if I felt the need.  That’s comforting in a strange kind of way.

I’ve always been the one that aims to please… the nice guy that generally stays well and truly seated on the fence as not to upset either side of particular debates.  Over the past few years this has started to change, and writing has helped that (I believe).

Along with the personal blog posts, there have been articles of various shapes and sizes,  the corporate blog posts, and the commencement of my book.  The latter is still under wraps and likely a long way off but I’m enjoying every minute of it.  Weinberg on Writing has helped immensely with that.  The words seem to flow well for me most of the time.  Of course there are times where nothing bubbles to the surface, but the break away from it also helps… so it’s not big deal.

Both Matt Heusser and Peter Koevari have been great motivators for me.  Matt will regularly review and provide feedback on various articles and also help point me in the right direction when getting connected to people.  Peter is an amazing author and a friend. He shows me everyday that it can be done no matter how busy you may be.

Anyway, there is a point to the this post.  It may get a little quieter around here in the foreseeable future.  I’ll be concentrating more on my book, and will also be focused on writing for a few different websites.  It’s a great opportunity that I can’t stuff up, and therefore requires more of my attention.

I’ll be sure to post here how it all progresses and where various pieces of writing are… so that you can give me HONEST feedback!  No fence sitting.  Hmm, most of you that tend to read this blog are not fence sitters anyway, so it’s all good.  ;0)

2012 blog in review

Posted on by
2

The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.

Here’s an excerpt:

600 people reached the top of Mt. Everest in 2012. This blog got about 6,900 views in 2012. If every person who reached the top of Mt. Everest viewed this blog, it would have taken 12 years to get that many views.

Click here to see the complete report.

Hmm, not sure about the Everest stuff, but anyway…

Happy 2013 to all.  May it be a year of learning and success!

DG