I finally finished it… after a duration of more than a year! While I don’t entirely blame the book for such a long period of time being required (moving to NYC played a part) it was a difficult read. Not that it contained overly complex ideas, but I found it pretty tedious throughout. I would normally just stop reading a book like this, but I was very interested in the content it proposed, so I pushed myself.
Pretty sure most people reading this post would have heard of the short video titled ‘The Invisible Gorilla’. Count the passes of the basketball, get to the end, did you see the gorilla (sorry, spoiler)? It’s a sound experiment, and holds a great wealth of insight to EVERYBODY, not just software testers. It was that video that lead me to purchase the book. The video captures one illusion; the illusion of attention, and book further builds on that illusion including other common ones namely the illusions of memory, confidence, knowledge, cause, and potential.
Each chapter focuses on a different illusion in the above order. It’s a mix of example via real-life story and example via real-life experiment. I found the stories quite fascinating and likened them to those told within Freakonomics. It was fascinating to read about criminal witnesses mistaking what they sore or what they remembered, or how confidence can be the deciding factor in multiple situations, and so on. As with many elements of psychology, the explanations used to describe human behaviours in the book were broad, and you can never really be 100% certain of their accuracy, however it was enlightening for the most part.
When the examples moved into descriptions of experiments it all slowed down a bit for me. This is no doubt a personal preference on how a book can keep my attention, so don’t let be a deciding factor for you. Having the ‘data’ to back up the theories does help, albeit a little boring for me to read about. They weren’t all that way, but I think as you near the end of the book you can sense an element of repetition in the writing which causes things to drag on a bit.
For me personally the illusions of attention, memory, and cause were the most powerful.
The illusion of attention has been a big player in my software testing career and until I learned about inattentional blindness and James Bach’s focus/defocus heuristic (something that also helped me a lot with the Dice Game), I’m pretty sure I fell victim to it a lot more often. I have also witnessed many other testers move directly past obvious bugs because their focus was on another part of the product, or on something entirely different all together (having a bad day, etc.). I probably should clarify that the bugs were obvious to me, but using the word obvious generally is not entirely fair. Due to the illusion of attention the bugs were not at all obvious to the tester sitting in front of me, and they cannot be blamed for that… they’re human. Pair testing helps a great deal to counter this illusion. Even if you both try and focus on exactly the same thing, you won’t be. Each of us see things in a unique way, which helps us identify different bugs, even if only slightly. The trick is to be consciously aware that your attention is a limited resource. Don’t think that you’ll see it all, because you likely won’t (hell, it’s science baby!). Another thing that has helped me counter this is product tours with particular charters. If you spend a lot of time with the same product and you test for similar things, take a step back and chart a course for a usability tour, or a security tour, etc. Whatever type of tour you think may yield some important information that you may not have discovered otherwise.
Early failures with the Dice Game also alluded to the illusion of memory. I first played the Dice Game during my Rapid Software Testing training. I was lucky in that respect as we could work in groups attempting to find the pattern. During that game, and in some since, I very quickly forgot (well, actually misremembered) the previous patterns I had been working on. There were many occasions where I finally solved it thinking that I had already been down that path with no success. This continued until I began taking more significant notes throughout the game. With those notes I could constantly check on what theories I’d worked on previously and my problem of misremembering was gone. As a tester, how many times have you been asked why you didn’t identify a bug that was now present in production? How many of those times do you distinctly remember testing in that area of the product? How did you know? Did you just ‘remember’ testing it, or did you actually go back and look over your notes? The book does a fabulous job of showing the reader how human memory can deceive us, very easily. Reading about this illusion once again highlighted the importance of taking notes and gathering evidence while testing.
The illusion of cause has been an interest of mine since reading Freakonomics. Generally speaking the entire book is about this illusion and how people are quick to jump to a particular cause rather than seeing it merely as a correlation. Humans have evolved in a way that allows them to identify patterns, but not only this, they also seek them. Our understanding of time (always moving forward) drives many of the patterns we identify/seek. In a sequence of events it’s natural for humans to identify the first event as the cause of the remaining, when in reality they may not be related at all. In the software industry I think we can spend too much time seeking and then identifying the wrong cause for many of our failures. If the sequence of events lines up nicely we jump on the first event as the cause. This chapter was a reminder for me to look at events in a different way, and to question my own tendency to identify a particular pattern.
There are also many lessons to be found in relation to the illusions of confidence, knowledge (often a result of the former!), and potential. While it was a struggle at times, I would encourage people to read it.
The ultimate lesson I took away from this book? To live my life and do my work while being aware that these illusions exist. Sure I’ll forget from time to time (I’m only human after all), but hopefully I’ll be able to remind myself before I miss something really important.
Peace.
Martial Tester 