A Newer Defect Severity Hierarchy…

Mike Talks wrote a brilliant post recently. I loved it so much it inspired my version…

Severity 1 – OMFG

Not much good being the ‘headlights’ of the project when the lights are out!


Severity 2 – WTF?

May stop you, if hard enough.


Severity 3 – Uh?

Will hurt a bit, but if your conditioned…


Severity 4 – Meh

Wotevs, I’ve got headgear!



If you happen to be a bug tracking tool developer, can I suggest you add these (and Mike’s) pictures. I’ll certainly use them!

The Evolution of Metrics Thinking

I had one of *those* moments recently… I was sitting at a dinner table with some great testers who were all part of OZWST. Somehow we got on to the subject of metrics in software testing (as is usually the case). I took a bit of a backseat for the discussion as I wanted to listen to the arguments being put forward by various people. While listening in, and trying to determine where each person was at in the conversation, it struck me…

The evolution of metrics thinking:

  • Metrics are good
  • Metrics are bad
  • Metrics are bad and here’s why
  • Metrics are good and bad, and here’s why

I’d like to clarify that this has been my evolution throughout the last 12 years of my software testing career. However, I do get the sense that many may also share this evolution.

Metrics are good:

This was my stance early on in my career. When I started in testing our teams used metrics (and pretty much only metrics) for all reporting. Of course they did; what other ways are there to report on progress, etc? At least that was how I used to think. While this frustrates me now, I can also relate with my former self and understand why I didn’t see anything wrong with it at the time. It was how it was always done, and it was how my *leaders* (or perhaps I should say managers) taught me to do it.

The innocence (and likely ignorance) of youth!

Metrics are bad:

These last three steps have happened relatively quickly when considering the first step lasted approximately 8 out of my 12 years!

Without delving too deep into how and why I started to align with the Context-Driven Testing (CDT) community, my evolution towards thinking metrics were bad began with the wonderful book; Lessons Learned in Software Testing. Certain lessons within this book helped me to question metrics in general, and the more I researched from that point on the more I convinced myself of this notion.

Metrics are bad and here’s why:

As any member of the CDT community would understand, you don’t get along too well if you make bold statements without backing them up. I was very happy to state to anyone that metrics were bad! I did this for a little while comfortable in the knowledge I had read it, and therefore it was true (to a certain extent). One day, someone asked me why metrics were bad… ah, well… I read it somewhere. BAM! A Batman right hook to the jaw.

Of course I need to understand why I was making such a bold statement.

More research, questioning, coaching, etc. They have all helped me to gain a better understanding of the dangers that are inherent in metrics.

Metrics are good and bad, and here’s why:

Not necessarily the final step, but the one that I’m currently at (or heading towards very quickly). After being involved in several conversations recently, and the wonderful metrics debunking series that we’re currently producing over at Hello Test World, I’m beginning to see *some* value in *certain* metrics in *particular* contexts. For me the biggest value comes from the identification of potential patterns. At the very least these can be used as indicators to prompt further investigation and/or analysis. That’s not to say the lack of patterns in metrics are an indication of anything good!

While this post is not in-depth, I do hope that it highlights an evolution of thinking that I have begun to notice more often in our industry (or perhaps more in the CDT community); one that is not limited to the subject of metrics. Sure, it could be stated that this evolution is simply the process of gaining knowledge, however I think it’s important for us stop and reflect upon, and therefore recognise, evolution’s such as these. By being able to explain such things I believe we stand a better chance of spreading the knowledge and helping others to relate, and hopefully learn.

Python is go (again)!

A while back I wrote about my Python course on Code Academy (who now appear to be CodeCademy) not responding

I let it go while I started the HTML & CSS course and then got busy with work, life, etc. After a recent piece of ‘I really should learn to code’ motivation, I decided to head back and see if the problem had been fixed.


So, I’m back at it and enjoying it just as much as before. There is one lesson that is obvious to me… you need to stick with it and therefore interruptions are a pain in the a**e!

I find that I get myself into an awesome rhythm, then I have to leave it for a day or so, then… I’ve lost it again! It’s really frustrating, and I can see why I’ve let it slip for so long.

Anyway, I’ll keep plugging away.



Let’s Test – Sweden

What an epic journey… approx. 30 hours from door to door (have I mentioned I don’t sleep while traveling?). So as I sit here with very little rest under my belt from last night (body still well and truly on Swedish time) I ask myself… was it work it?


To top off the long journey of no sleep, I arrived in Stockholm at lunch time on Saturday, it was sunny, and about 25 degrees! How could I possibly rest when there was a city waiting to be explored. I had already decided that I would walk through Gamla Stan (old town) and it was well worth it. A very beautiful part of Stockholm and obviously a tourist destination because there were cameras flashing all over the place (yes, mine included). I covered some reasonable distances by foot on Saturday before heading back to the hotel for some rest.






On Sunday we (Lee Hawkins, his wife, and I) had decided to meet later in the afternoon before heading out to Akersberga, and the Runo conference centre. So after changing my check out time I headed once more, this time in no particular direction. It was warm and sunny again and I think I even got a little sun burn. I got lost. Normally I would freak out at the notion of getting lost in a foreign city, but not Stockholm. I figured that if I didn’t cross any water I would still be within a reasonable distance of the hotel (in relation to getting back via a taxi if required). I later learned that I spend most of my time walking through Ostermalm. Wow, what a beautiful part of the city. Families, parks, dogs, cafes, shops… just my style.

After meeting Lee and his better half in the afternoon we started our not so epic (but still challenging) journey out to Akersberga. It was interesting to reflect on two testers struggling with the train ticket machine, even after changing the language to English… I claim bad usability over user error.  ;)

I had heard from several people that getting to Runo was a little difficult, and to add to this the train line was undergoing maintenance so we had to get off early and get on a bus for the remainder of the journey. The bus was sitting there waiting for us though, so there wasn’t too much ‘stressed traveler’ thinking required. We made it to Akersberga, and then took a short taxi ride to Runo…

runo sign


runo boats

runo thinking men


Our Greek taxi driver was a laugh a minute. Some of his words will not be repeated on my child friendly blog.  ;)

The welcome was warm and instant! Hugs, handshakes, pecks on the cheek… it really is like ‘coming home’. The biggest smile upon my arrival… Meike, and a top which said ‘I like hugs’ (or very similar to that anyway)! I had emailed Meike a few times via Miagi-Do, but it was great to be welcomed in such a way.

Check in done, bags unloaded in the room… beer time! Thus continued the Let’s Test experience over the next 3 days!

I met loads of cool people who I had seen on the interwebs, and loads of new people as well. Catching up with old friends was also a highlight. I won’t name name’s because there is simply too many… and for fear of missing some. That was just the Sunday night…

Monday kicked off in a way that every conference should… ACDC blasting through the speakers while watching a picture slideshow of previous Let’s Test conferences. My main man Henke then took to the stage to welcome everyone, followed by our Content Owner Rob Sab, and Lead Facilitator Ilari.

The program as I took it (all 3 days):

  • Tim Lister – Forty Years of Trying to Play Well with Others (opening keynote)
  • Jon Bach – Exploring Best Practices (full day workshop)
  • Steve Smith – Whose Ideas are In and Whose Ideas are Out (experimental keynote)
  • James Bach – Describing What You See: A Challenge (workshop)
  • Martin Hynie & Christin Wiedemann – Can Playing Games Actually Make you a Better Tester? (track session)
  • Ilari Henrik Aegerter – What if you Can’t See Each Other (workshop)
  • Dawn Haynes – Realism in Testing (track session)
  • Jon Bach – A Critical Look at Best Practices (closing keynote)

A couple that I’d like to call out…

Jon Bach – The theme for this years Let’s Test was ‘Raising the bar…’ and Jon did this by developing his closing keynote via the information gleamed from his workshop on day one. I was lucky enough to be sitting in on this workshop and even got the opportunity to facilitate when required. Jon had set out to run it like a peer conference and at certain stages that actually worked quite well. It was great to explore the notion of best practices as I have spent many years of my career in environments that preach them!

Steve Smith – A keynote with no seats… that’s just awesome. I’m sure there will be YouTube (or similar) videos that appear for this, but it was a great experience. A workshop that involved approx. 150 people. There were some great lessons to be learned from the exercise… and I’ll happily wear the label ‘Clock Blocker’ into the future (ask around about that one).

Ilari Henrik Aegerter – Another interactive workshop where paper planes were made within collocated teams, then again in distributed teams! There were planes flying across the room, Skype chats, pictures, filming, Dropbox, Facebook… there was also rule breaking (my favourite!). Some great lessons and considerations for distributed teams.

Every session I went to I would define as awesome, it’s just that the above stuck out for me.

The extra-curricular activities are also worthy of mention…

Disc Golf! I couldn’t believe this was actually a ‘thing’. Turns out Paul Holland is pretty damn good (just ask him). I played this 2 out of 3 mornings (I was a touch late to rise on the 2nd morning for some strange reason). It’s actually a really cool game, and with a bit of beginners luck I did OK (on the first morning – we won’t talk about the 3rd morning).


Testing games! Well of course, it’s Let’s Test. Erik Davis (very difficult to spot during the conference – with orange hair and a kilt) kindly hosted a game of Zendo which I took part in (thanks for letting me join late dude). Having never played it before it took me a while to catch up on the rules, but once I knew what was happening I thoroughly enjoyed it. I nice puzzle to solve to help us testers identify patterns. Then it was over to Paul Holland’s table for some mind reading and Art Show (a card game similar to the Dice Game). A late night, but a good night.

Music & dancing! I finally got the meet Duncs! That profile picture in Twitter is not point in time… that’s him ALL THE TIME! Duncs and I have become pretty good mates via the interwebs since both joining Miagi-Do at a similar time and taking a challenge together. So it was awesome to finally meet him. It was also a pleasure to help him and Eddy Bruin get the god damn audio working! Although, I still say that just looking at the mixer didn’t really help Duncs! Some great tunes, and some great dancers!


Scotch! Enough said…

The meals were also outstanding, and the venue was just what the doctor ordered for a ‘retreat’ style conference.

Some notable mentions (although please don’t see this as all of them – there are many, many more that I just can’t squeeze out of my exhausted and jet lagged head)…

Martin Hynie - Dead-set CLOWN! Can’t wait for Let’s Test Oz.

Scott Barber – Always fun to talk to. Scott keeps me entertained!  ;)

Dawn Haynes – We had a lovely long chat over dinner one evening and she is a great story teller.

Huib Schoots – Don’t believe the Huib (it’ll make sense if you pronounce his name correctly)!

Tim Coulter – Fellow rule breaker/partner in crime for Ilari’s workshop! Also my knowledge base when it comes to NYC.

Johan Lundstrom – Very awesome dude who has an awesome job! Thanks to Johan for showing me around his office in Stockholm!

Alessandra Moreira & Lee Hawkins – My fellow Aussies representing!

Paul Holland – Disc Golf is harder than it looks! Thanks for taking the time to teach me some moves…

So, in short… if you ever get the chance to attend a Let’s Tet conference, go! Speaking of which… Let’s test Oz is in September this year. I’m hoping my journey to Sweden will help inspire our event here in Oz so that it’s just as awesome!