Rapid Software Testing – Class Notes

The following is a direct copy of my class notes (minus spoilers and exercises).  I’ll be exanding my thoughts on the class via another blog which I’ll link to once I’ve written it (now linked).

Warning – It’s long, but only in scroll length, not word count.

Some of them may not make sense as they are simply reminders for myself.  Each one could lead to an endless buccaneering session.  :0)  I hope there are some that provide value though.

• Don’t like the smell of confusion (smoke); don’t be a tester (fire-fighter)
• Reinvent software testing for yourself
• No best practices; only ‘favourite’ practices
• Broken Leg Heuristic
• Observation; don’t stop at the first thing you see
  • Someone else’s observation is not wrong, just different
• When searching for a ‘pattern’ don’t just search for what it is; also search for what it is not
• Creep & Leap Heuristic
• Chainsaw is a heuristic; can work as intended (cut wood) or not (cut off fingers, etc.)
  • It’s not a guarantee!
• Disprove theory = Tester
• Tester credibility is like a ‘health’ bar in a PC game; you can take a hit if required
• Argue to the point of understanding; not agreement
• Excel, Notepad++, Google, Perl – JB’s test tool favourites
  • GET GOOD AT TOOLS!
• Not just Requirements testing
• Live with uncertainty = Tester
• Be the sober one in the room
  • Rest of the team can be giddy and tipsy; tester remains sober
• When do we stop?
  • Diminishing value
  • Value of bugs found vs. cost of testing
• DEFOCUS!
  • Change how you are testing
• Take a nap!
  • This really can work
• Fake testing industry
  • Hired to not test
  • Hired to be a scapegoat
• Muddling through testing?  JB can help.
  • Ask how to stop
• Time = One thing we can measure
  • Count sessions; not tests
• All tests are different; don’t count
• Testing Testers – Socratic method
• “A student of the craft of testing” – JB
• “Get glasses” – JB
  • The thickest frame you can find
• Test activity vortex
  • Satisfice.com
• Testing Story!
  • Rapid Testing Framework = Story
• Testers light the way
• Quality is the job of others
  • Testers are the oil in the engine
• eVALUatE = VALUE
• Know your mission – Begin sympathetically – Then chase risk!
• “Why did you hire me?”
• Happy Path = Find bugs here and they are generally important
• (‘ OR ‘ ‘ + ‘)
  • Web login – use without brackets
• “You are probably not stupid” – JB heuristic
• Afraid of complexity
  • Have to battle that fear
• Don’t believe it’s simple
  • If it looks simple, it may not be
• Three types of answers/evaluations
  • Quick
  • Practical
  • Deep
• Cannot test without models
• Don’t know something?
  • Don’t say “I don’t know”
  • Say “Not by that name, tell me about it”
• Models can be your prison
• Draw pictures while testing
  • Helps to find visual gaps
• Anti-Random Testing
  • Visualise empty space
• Curious = Want to know; not need to know
• Phase Space = Total states a system can be in
• Complexity of controls for a product need to match the complexity of the product (as a minimum)
• Galumphing Heuristic
  • Inexpensive actions
  • Inert Actions
• Defocus – What have I not been doing?
• Can’t choose random! E.g. numbers
  • Brain cannot do it
  • There is always a reason why you choose it, even if you are unaware of it
• “Leaky Abstractions”
• Alt + Shift + Print Screen = High Contrast Mode (caution when trying this)
• Two type of bugs that are more expensive if found late (pre-implementation)
  • Requirements bug
  • Platform bug
• Six dimensions of coverage – Minimum
• Watch yourself test
  • Write about the patterns you see
• You can’t know everything
  • Seek answers
• Need to struggle with problems to learn
• 46 Dimensions of Testing
• Guide work heuristics
  • Words to guide your approach
• Shake & Bake
  • Test and fix at the same time
  • Remove the obvious/easy bugs quickly
• Black Flagging
• Mipping – Mention in Passing
• Run the right tests – Ask the right questions
• Mysterious Sphere
• “Carpenter” methodology
  • He/she makes what you want
• The Social Like of Information – Book to read
• Information that is available points you in one direction – Hidden information points you in the opposite
  • Critically think about the possible hidden information
• Thinking Fast and Slow – Book to read
• Huh?  Really?  So?
  • So = So what
• Two reasons why you don’t have a purpose to find bugs
  • Learning/information gathering
  • Under supervision
    • Someone else has responsibility
    • Being directed!
• Implicit – Noticed but not told (requirements)
• Move your eyes to a different part of the screen – Change focus
• Sapient – Done by humans
  • Testing is sapient
• Testing = Role of humans
• Checking = Role of tools
• Oracles are ALL the time; not just at the end
• Consistency over time
• Remain faithful to your mission
• Consistency relationships
• HICCUPPSEF
  • Consistent with the world?
• Consultant coming in to client
  • Can you understand documentation completely?
    • Yes = Bad
    • No = Good
      • Why no use of acronyms? Why nothing to reduce waste?
• You can never write it ALL; you have to draw the line somewhere
  • E.g. how often do you see an explanation of GUI?
• Draw that line somewhere else to effect change and reduction of waste
• Airline Safety Briefing – No-one listens
• Perfect repetition is not possible; close enough repetition is possible
• Rats detection landmine – Video
  • What about what is missed when the rate seeks the reward?
  • Even the rats must have skill
  • So why not Testers?
• Defined reason to repeat a test
  • Listed in private notes – no spoilers
• “Try it” – Use that when teaching something
• Exploratory Testing (ET)
  • Learn
  • Design
  • Execute
  • Evaluate
    • ALL at the same time
• ET is an approach; not a technique
• Mixing scripted with exploratory – Yes!
• Chef analogy – Creation of potato chips/crisps
• ET is not unstructured
• Good ET Tester can control and explain their structure
  • If not then it can be argues against
• You need to be able to explain what you did; otherwise you cannot own/be proud of it
• Testing Story
  • Your structure
  • It guides you
  • Get good at telling these stories
• Reports have three key elements
  • In private notes – no spoilers
• A ‘professional’ test report is one that satisfies the mission (in your given context)
• Practice reports!
• 7 Samurai – Movie to watch
  • JB says I am the 7^th Samurai
• Test framing – Linking your test to your framework/approach
• DICE GAME!
  • No spoilers
• Experiment design books – research these
  • Help with puzzle solutions
• Things That Make Us Smart – Book to read
• Describe a stone – Video
  • Satisfice.com
• Bill Anders
• Reading on Observation, Science, Thinking