The Invisible Gorilla…

I finally finished it… after a duration of more than a year! While I don’t entirely blame the book for such a long period of time being required (moving to NYC played a part) it was a difficult read. Not that it contained overly complex ideas, but I found it pretty tedious throughout. I would normally just stop reading a book like this, but I was very interested in the content it proposed, so I pushed myself.

Pretty sure most people reading this post would have heard of the short video titled ‘The Invisible Gorilla’. Count the passes of the basketball, get to the end, did you see the gorilla (sorry, spoiler)? It’s a sound experiment, and holds a great wealth of insight to EVERYBODY, not just software testers. It was that video that lead me to purchase the book. The video captures one illusion; the illusion of attention, and book further builds on that illusion including other common ones namely the illusions of memory, confidence, knowledge, cause, and potential.

Each chapter focuses on a different illusion in the above order. It’s a mix of example via real-life story and example via real-life experiment. I found the stories quite fascinating and likened them to those told within Freakonomics. It was fascinating to read about criminal witnesses mistaking what they sore or what they remembered, or how confidence can be the deciding factor in multiple situations, and so on. As with many elements of psychology, the explanations used to describe human behaviours in the book were broad, and you can never really be 100% certain of their accuracy, however it was enlightening for the most part.

When the examples moved into descriptions of experiments it all slowed down a bit for me. This is no doubt a personal preference on how a book can keep my attention, so don’t let be a deciding factor for you. Having the ‘data’ to back up the theories does help, albeit a little boring for me to read about. They weren’t all that way, but I think as you near the end of the book you can sense an element of repetition in the writing which causes things to drag on a bit.

For me personally the illusions of attention, memory, and cause were the most powerful.

The illusion of attention has been a big player in my software testing career and until I learned about inattentional blindness and James Bach’s focus/defocus heuristic (something that also helped me a lot with the Dice Game), I’m pretty sure I fell victim to it a lot more often. I have also witnessed many other testers move directly past obvious bugs because their focus was on another part of the product, or on something entirely different all together (having a bad day, etc.). I probably should clarify that the bugs were obvious to me, but using the word obvious generally is not entirely fair. Due to the illusion of attention the bugs were not at all obvious to the tester sitting in front of me, and they cannot be blamed for that… they’re human. Pair testing helps a great deal to counter this illusion. Even if you both try and focus on exactly the same thing, you won’t be. Each of us see things in a unique way, which helps us identify different bugs, even if only slightly. The trick is to be consciously aware that your attention is a limited resource. Don’t think that you’ll see it all, because you likely won’t (hell, it’s science baby!). Another thing that has helped me counter this is product tours with particular charters. If you spend a lot of time with the same product and you test for similar things, take a step back and chart a course for a usability tour, or a security tour, etc. Whatever type of tour you think may yield some important information that you may not have discovered otherwise.

Early failures with the Dice Game also alluded to the illusion of memory. I first played the Dice Game during my Rapid Software Testing training. I was lucky in that respect as we could work in groups attempting to find the pattern. During that game, and in some since, I very quickly forgot (well, actually misremembered) the previous patterns I had been working on. There were many occasions where I finally solved it thinking that I had already been down that path with no success. This continued until I began taking more significant notes throughout the game. With those notes I could constantly check on what theories I’d worked on previously and my problem of misremembering was gone. As a tester, how many times have you been asked why you didn’t identify a bug that was now present in production? How many of those times do you distinctly remember testing in that area of the product? How did you know? Did you just ‘remember’ testing it, or did you actually go back and look over your notes? The book does a fabulous job of showing the reader how human memory can deceive us, very easily. Reading about this illusion once again highlighted the importance of taking notes and gathering evidence while testing.

The illusion of cause has been an interest of mine since reading Freakonomics. Generally speaking the entire book is about this illusion and how people are quick to jump to a particular cause rather than seeing it merely as a correlation. Humans have evolved in a way that allows them to identify patterns, but not only this, they also seek them. Our understanding of time (always moving forward) drives many of the patterns we identify/seek. In a sequence of events it’s natural for humans to identify the first event as the cause of the remaining, when in reality they may not be related at all. In the software industry I think we can spend too much time seeking and then identifying the wrong cause for many of our failures. If the sequence of events lines up nicely we jump on the first event as the cause. This chapter was a reminder for me to look at events in a different way, and to question my own tendency to identify a particular pattern.

There are also many lessons to be found in relation to the illusions of confidence, knowledge (often a result of the former!), and potential. While it was a struggle at times, I would encourage people to read it.

The ultimate lesson I took away from this book? To live my life and do my work while being aware that these illusions exist. Sure I’ll forget from time to time (I’m only human after all), but hopefully I’ll be able to remind myself before I miss something really important.

Peace.

Seek, Identify, Gather… A model for talking about testing.

When I talk about testing I tend to talk about information. For me, testing is all about information. Information that helps decision makers make decisions, and information that informs risk. How we identify that information depends on the type of information we need, and to know that we can do several things… identify risk, ask questions, user research, analyse specifications, find out what’s important to those who matter; the list goes on.

Only recently have I actually thought about this as a model. So I thought I’d spend some time visualising it (something that helps me think deeper about a subject).

What testing means to me at a high-level:

If testing wasn’t all about information, then it would be all about risk (generally speaking). So first up I like to talk about risk and risk identification. Testers (for the most part) are good at that. We tend to come with varied backgrounds; business, tech, users, and so-on. So we can view a product with many different lenses which provides greater risk identification coverage.

Then I move to the information required in order to understand and potentially mitigate the risk. The information we seek depends on the risk we’re trying to understand, as does how we seek and identify it.

Let’s visualise that part:

I then like to talk about gathering and reporting the information. Test reporting can be very painful, or it can be a breeze. Which one it ends up being not only depends on your stakeholders, but also on how you gather the information while testing. If you don’t record your testing (screen captures, notes, etc.) then how are you going to gather the information you need to report? Pass and fail? Whatever… realistically that means nothing. You need to work out what your stakeholders need (or in the real world, what they want) to see, and then gather the information in a way that translates as easy as possible to it. You may have a project sponsor that totally gets what you’re doing and completely trusts you, and so doesn’t need to see anything… but lurking in the background is the internal audit team who need to see EVERYTHING! So I like to include that when I talk about testing as it can make your break your entire effort (rightly or wrongly).

Thoughts? Feedback is always welcome.

Peace…

Time to step back…

If you align yourself with the Context-Driven Testing (CDT) community, or you’ve been a part of it in the past, then you’ve likely seen the flame wars on Twitter recently. If you haven’t, lucky you… don’t bother looking. Instead, just worry about doing good testing… whatever that means to you and your organisation.

I’ve remained silent for a few reasons:

• In my opinion Twitter is absolutely awful for discussion/conversation, especially when there are more than two people involved.
• I think many of the Tweets carry hatred due to personal issues/personality conflicts.
• I think many of the Tweets are simply childish, like throwing the first punch and watching all your ‘mates’ deal with the aftermath.

I’ve seen awful words like:

• Bullies
• Asshats
• Cowards

While these were not directed at me personally, they were directed at members of the CDT community in a very general and broad way. I therefore have included myself within their general and broad target. Rightly or wrongly, this is what happens when you label a community in such a negative way… the self-identified members take it personally (shocker).

This is me now raising my voice because I’m sick of being labelled in such a generalised way.

There are members of both ‘sides’ (I feel dirty even writing that word in this context) that simply need to grow up. If I witnessed my 9 year old daughter partaking in such rubbish I would educate her on how to handle it in a far more mature way. In fact, I’d be surprised if my daughter was even that bad in the first place.

Drop the elementary/primary school stuff folks, you just look silly.

What follows is a very personal story/experience report. I would like it known up front that I may also use generalisations, and if I offend anyone while doing do, I apologise.

When I first started out in testing I worked for a large organisation for many years. If you were to label their approach to testing then you may call it ‘factory’, or perhaps ‘traditional’. Whatever you label it, it was pretty damn wasteful for the most part. I spent 8 years learning how to test with that organisation and toward the end I began discovering that the way they tested could have been so much better. The turning point for me was reading Lessons Learned in Software Testing. This was my introduction to the CDT approach to testing (not the community – an entirely different beast).

After reading the book I then began to research more. I found more books, blogs, Twitter, LinkedIn, etc. I was then beginning to get introduced to the CDT community. After 8 years of ‘factory’ testing and no interaction outside of my organisation it was very refreshing. I met people (most virtually) that were making the same discoveries I was, other people that had called themselves CDT for years, and others that were ‘kind of’ in the CDT community and ‘kind of’ not. At the time I couldn’t have cared less what the people called themselves or what community they thought they were in… and guess what? I still don’t. If someone is willing to interact with me and help me make my testing better, I’ll interact with them. It’s true that most of the people I could find at the time were in the CDT community. Perhaps this was because there was no ‘factory’ community, or perhaps it’s because of where I was looking. I don’t care… and yes, I still don’t.

I naturally moved towards the CDT community and eventually self-identified as a member. This is an important point… I self-identified. I wasn’t initiated, welcomed, tested, approved, etc. I simply referred to myself as a member of the CDT community. If James Bach (yes, I’m using his name… like grown-ups do) had told me that I wasn’t a member of his CDT community, then I would have been a member of my own CDT community, with other like-minded people. You don’t need James’ approval. Sure, you can attribute a lot of what people understand CDT to be to James, but there are also MANY others who constantly contribute to it and move it forward… whether James agrees or not (and he’s the first to welcome that). Before I move on from James (because this is bigger than that)… yes, he can be abrupt, loud, intimidating, rude, and so-on… he once left me hanging on a Skype chat with “I thought you were better than that.” Ouch. However, I cannot dispute his testing intellect, passion, drive, work-ethic, and at times what appears to be pure genius. He’s not for everyone (from an interaction perspective), but you’d be silly not to at least research his work… even if you have to do so from arm’s length.

In more recent years I’ve spoken and written about the need for labels (i.e. CDT), and how I would LOVE them to not be required. I think this is partly because I can see the tension within the different ‘schools’ of testing, and the absolute waste of energy is causes – for the most part. Perhaps gradually I’ve been moving away from identifying myself or the way I test with a particular label. I just want to do good testing, or help to get good testing done. An unfortunate part of testing is the need for labels. If I want to differentiate the testing I do to the testing that someone else does, how do I do that without showing it (preferred option which is not always possible), or having a label for it as a conversation starter? Perhaps ‘good’ can be my new label. Then I can talk about what ‘good’ looks like/is in a particular context.

I’ve never been an advocate for “Can’t we all just get along?” I don’t think that is helpful when trying to move our industry forward. We need debate, different approaches, and different personalities who focus on different aspects of testing. How else would we make our ideas better? Self-critique can work, but other’s critiques are often far more powerful (never proof read your own work, etc.). However, maybe I’ve been looking at “Can’t we all just get along?” in the wrong way. We can all get along while debating and disagreeing on different approaches to testing, can’t we? Then again, we’re human. If getting along was that easy when we disagreed on something… well, news headlines would be a lot different.

After spending 8 years as a ‘factory’ tester, and almost as many in CDT, I think the time is right to move on from labels. From now on I’m going to be a tester, no… a good tester (or test manager, etc.). I’m sick and tired of the ‘politics’. I know they will never go away, and it would be ignorant of me to think I can avoid them in the future, but for me personally I think I can do way better if I put my energy towards good testing instead.

For those of you that will jump at the chance to declare me another victim being ‘pushed’ or ‘forced’ out of the CDT community, it couldn’t be further from the truth. No-one has done that, and the only person you can ‘blame’ (because some of you have to it seems) is me. Besides, while I won’t label myself, I’ll continue to interact with members of the CDT community (because most of them are awesome), and I really hope that I also interact with those that aren’t as well (because some of them seem awesome too). 🙂

I’m going to focus on good work and not the bullshit that often surrounds it. I’ll deal with bullshit when I have to, but no more excess energy will be spent on it. I’d like to continue helping others like Katrina has written about. For example, I started OZWST after my crew from NZ introduced me to KSWT… and I’ll continue to do cool things like that, but I will not make it about labels.

Peace…

Wonderful Words from Jerry Weinberg

Today I received some wonderful words from a man who has inspired many parts of my software testing career. Those words were in relation to my book, Software Testing as a Martial Art…

“What a gift David Greenlees’ book is!

The day I received my copy, I spent all morning scanning, digging, smiling, nodding appreciatively. It’s a wise book, and I’m recommending it to all the testers (and others) I know.

But prepare yourself. This insightful book shows the reader that testing is like a martial art in a number of ways—ways essential for testers to understand. A caution though, about a way this book itself is like a martial art:

Beginners may simply not understand much of what Software Testing as a Martial Art has to say until they have practiced. Very few people begin their testing careers by applying principles they’ve read about, no more than I did when I began testing (or a martial art). So, though Software Testing as a Martial Art is a book testers should read at the beginning of their testing careers, it’s also a book they should re-read every year as their experience grows. I know I plan to re-read it every year on my birthday.

I also plan to continue my study of Aikido, my preferred martial art. Thanks to Software Testing as a Martial Art, I now understand that every Aikido practice will also improve my testing art.” – Jerry Weinberg

Thank you Jerry, I couldn’t be happier.

Tacit & Explicit Knowledge – Wha…?

Firstly I need to mention that I read this book extremely fast. Approx. 2 weeks is all it took, and for me that is a record. I think there are a couple of reason for this:

• It’s a relatively small book – 178 pages.
• I’ve been catching the train to work recently – reading time.
• I was very keen to see what I could learn from it.
• I would normally back track a lot while reading, going over paragraphs a few times to clear any misunderstanding. I didn’t do that as much with this book because not matter how many times I went over certain paragraphs I know I wouldn’t have understood it any better!

I recall telling some colleagues of mine that it was a ‘mind bender’ after reading only a few pages, and I still maintain that description. I don’t have an academic background so I’m not used to reading text such as this. It was a struggle to begin with, but about half way through certain elements began to make sense and I could start to relate them back to not only my testing, but also my martial arts training. The book prompted some tweets from me (some direct quotes):

• “What the mistaken claim that all knowledge is tacit does indicate is that, mostly, explicit knowledge is harder to understand…” H Collins
• “The modern world is thought of as driven by explicit knowledge – patterns.” H Collins
• “The explicit is taken as the norm rather than the tacit – and the contrast with what is not explicit is ever present.” H Collins
• “Education is more a matter of socialization into tacit ways of thinking and doing than transferring explicit information or instructions.” H Collins
• #testing standards are like lookup tables, being frozen history at best… and often not even that. 1/2
• –Little to no sensitivity to social constructs and changing contexts. #stopiso29119 2/2
• The /rules/ of #testing are like the /rules/ of society… constantly changing. Explicit standards don’t change with them. #stopiso29119
• So, just finished Tacit & Explicit Knowledge by Harry Collins. It’s going to take a while to digest, but some instant gems are evident.

Along with the tweets there were multiple thoughts that have found their way into my book (currently in progress); specifically related to knowledge transfer.

As soon as the book moved on from explaining the explicit, into explaining the tacit, my thoughts quickly moved to martial arts and how the many instructors I’ve work with have attempted to transfer knowledge on to me, their student. While Harry does well at explaining a very complex subject using working examples (riding a bike for example) I still found myself getting more from the book when linking it back to my own experiences. While this would seem common place, I want to make this point very clear because had it not been the case, I would have really struggled with this particular read. I found it to be a rather confusing style of writing; however with persistence I managed it.

Harry draws a very useful ‘map’ of tacit knowledge throughout the book, and this helps to analyse what can, should, cannot, and should not be explicit/explicable. My relatively naive understanding of both forms of knowledge, pre-read, appears to have been correct to some degree, but that understanding was nowhere near deep enough. The book has helped me to question further the extent of both forms of knowledge, and how they may impact the work we do, and could do, as testers.

Making something explicit can be as simple as writing it down on paper, but does that mean it will hold any meaning to the recipient? Will be useful to them? Will it afford them the understanding they need in order to do a good job? Are you making knowledge explicit when doing this, or simply writing words on paper? This book prompts a lot of great questions!

Harry, very neatly, breaks up tacit knowledge into three forms:

• Relational Tacit Knowledge (RTK – Weak) – Contingencies of social life.
• Somatic Tacit Knowledge (STK – Medium) – The nature of the human body and brain.
• Collective Tacit Knowledge (CTK – Strong) – The nature of human society.

He sets out to explain how each of these forms of tacit knowledge can, or could, be made explicit, and how elements of each cannot. CTK held the most meaning for me. This is the area of tacit knowledge that calls on our society, the knowledge that remains sensitive to the varying contexts of our lives, surroundings, interactions, etc. Learning to drive a car was a great example used for this. While the instructions for interacting with the car itself could be explicable, how would you go about making explicit the knowledge required to react to varying traffic conditions, driving rules, etc. Or simpler again… while you can tell someone (make explicit) how far away they must remain from another person while walking past them on the street, what would that person do if it were a busy walkway? What would they do if the person passing them was of the opposite sex? Can they break the distance rule and bump into them because it’s busy? What if they did this when it wasn’t busy? Each time they passed a different person, in different conditions, they would need to call on their CTK in order to navigate the situation. Thinking of, and making explicit, every conceivable scenario for passing someone on the street could be possible; however the limitations (we would die first, and where would we store all the data?) would be far too great for any of us to achieve it.

Now with all of the above in mind, think about your testing. How many possible scenarios do you think you could call upon in order to make the task explicit? Exactly… don’t bother! Testing is social. Our CTK (along with RTK and STK) plays a huge part in us doing a good job. This fly’s in the face of testing that solely calls upon explicit step by step procedural artifacts (test cases). While these may add value in some contexts, doing them alone will not allow for the CTK that is so greatly required in our work.

There is much more within the covers of this book. The above is merely one learning that fell from the pages for me; and while I ‘knew’ it already, I don’t think I knew it deeply enough.

I’d like to do an analysis of some testing activities and see where they fit on the map of tacit knowledge as it’s written in this book. That will take time.

I now need to read The Shape of Actions, one of Harry’s earlier books in collaboration with Martin Kusch. It’s mentioned a few times through this book, as are polimorphic and mimeomorphic actions. While I have a high level understanding of these, I would like to go deeper.

I recommend this book to testers out there. It may be tough going, but persist like I did. There is value to be gained.