Interviewing a Testing Expert - Jonathan Kohl from Kohl Concepts [Part-2]

Friends, I am back with the final part of the interview with Jonathan Kohl, co founder of Kohl Concepts and a world-renowned expert in Software Testing. In case, you missed out the first part, I would strongly suggest you to read it first before proceeding to read this one. So here we go:

Debasis: Tell me of any situation when you had wished you were NOT a tester!

Jonathan: Testing can be political. The wishes of the technical staff and executives sometimes don't map to the reality of what they are delivering. That means you can be the least popular person on the team, and it can be discouraging in the moment.

When you challenge someone's personal faith or political views, they get upset. If you challenge their favorite technology or methodology, they behave similarly. Confronting that requires a lot of courage, integrity, ethics and a thick skin. They respect you for it in the long run, but it’s tough to take in the short run. I've learned over the years that if I am tempted not to say something, that means I should say it, but with tact, diplomacy and empathy. It's served my career well, but it's tough in the moment to challenge something that is distracting us from providing value. Anytime someone has said: “don’t raise that important issue, it will be a career-limiting move” I have listened to my conscience, spoken up, faced initial resistance, but over the long term, they were always career-catapulting moves.

Debasis: Has the profession (testing) ever affected your personal life? If yes, how?

Jonathan: I think about testing everything. Sometimes it's hard to take time off and not analyze all sorts of systems and look for weak points. Now that we can work from home, or work on side projects and things like that much more easily with the Internet, it's hard to turn work off and relax sometimes. The '55 Pontiac and car club are good outlets, and when I have the time for it, so is performing music.

Sometimes my friends don’t want a thorough analysis on their ideas, they just want me to empathize, so once in a while they ask me to turn off my tester brain.

Debasis: What do you think as the most essential skills that make a great tester?

Jonathan: Mostly, skills related to investigation and communication. You can develop those just about anywhere. Most testers have something in their background that they learned some basic investigative skills from, such as scientific experiments at school, or reading detective novels. Sometimes making that connection to your testing work can be a real eye opener. Good investigators see things that others miss, and are able to find problems and report them clearly. There are many different skills testers draw on to do that, and that depends on their background and training. It doesn’t hurt to learn some technical skills so you can help track down issues more quickly and communicate with programmers and other technical people in terms they understand.

Good investigators have solid general knowledge. They know what new technologies are up and coming, and they have a deeper knowledge of the systems they are working with. What are the inherent problems with the technology? They have a catalog of problem areas, and remember things from their own experience to draw on as they look for potential problem areas. They also have a deep knowledge of the users of the software. Who are our users? What problems are they trying to solve with our software? What are their goals with our software? They also have a good understanding of the team, and the equipment available for testing. They also have an understanding of a lot of different testing techniques, and how to identify risks or opportunities, and recommend techniques to look into those areas more. They have at least a basic technical understanding of the software and the environment in which it operates. All of that helps them strategize, plan and generate a lot of testing ideas that leads to important information about the product they are testing.

It's not a skill, but having ethics and integrity is also important as a tester. Stakeholders need to be able to trust us to tell them the truth. When we violate that trust (even by giving in to their pressure in the short term) we lose our effectiveness.

You also need an interest in discovering problems. Sometimes testing can be tedious, and if you are creative, you can turn any situation into something interesting for you. Without that interest in finding and reporting problems, it’s going to be difficult to enjoy testing and want to develop skills.

Debasis: Tell me about the most fascinating bug that you have encountered in your entire testing career.

Jonathan: Several intermittent bugs have been fascinating. One of the most difficult to track down was due to a social issue. Every release, one of our biggest clients would file a bug report. We would try to repeat it, and just couldn't get a repeatable case. Our software could dial home with a stack trace on error, so we asked the clients to send it in. The programmers looked at the code and tried to make it as robust as possible, then we'd release it, still get the same bug report from the client. The team sent me to spend the day with one of the client's users to get more information, and I was shocked at what I learned. It turned out that the software would crash on startup, but the user blamed herself for that error because they had learned a workaround (and hadn't told us about it) and she forgot to go through those workaround steps. After she went through the workaround, a few minutes later the software crashed and she showed me the familiar stack trace. I got her to give us the information from the first crash, which is really where the bug was located. The stack trace after that was a red herring because the system was in an unstable state, and could fail any number of ways. I learned to pay a lot more attention to the users and the social context they use our software in, and to incorporate that into my testing. That led me to look into usability testing and user scenarios instead of just looking at the requirements and functional aspects of the application.

Debasis: What single thing would you want to tell every newbie who is struggling in the early stage of building software testing career?

Jonathan: Concentrate on developing your skills as a software tester. There are a lot of distractions from skill development, whether it is to pass some multiple choice certification exam, learning about the latest fad software development methodology. We also talk a lot about analysis and planning more than test execution. (There is value to all of those activities, but not at the expense of test execution skill development.)

Certifications, tools, methodologies and practices change over time, but demand for good testing skills remains constant. For example, if you are testing on an Extreme Programming team, the novelty of the unique environment and context wears off, and you need to deliver skilled testing. Learning the methods, values, shared language and rituals only gets you so far. Your team is going to expect you to add value as a tester. It’s kind of like extreme ironing. Once they get over the novelty of you ironing in an unusual context, your customers will expect nicely pressed shirts. If you focus on improving your test execution and reporting skills, you will find more interesting information, and will become more valuable to the organizations you serve. Good testing and communication skills transcend methodologies and tools, and good testers can provide value anywhere if they have the skills.

Some things to consider: Develop your investigative and reporting skills. Be a software investigator, and practice your test idea generation. Use your background to leverage your investigative side. If you have a programming background, use that. If you don’t look at other areas you developed skills as an investigator. I’ve worked with wonderful testers with backgrounds from accounting, writing, studying history, programming, systems administration, investigative journalism and others. Most people have some sort of investigative experience in their background. Find yours, and apply it to software testing. Failing that, read detective stories and watch investigative programs on TV and try to apply some of the techniques to testing.

Here is how to practice being a software investigator: Look at simple systems around you, pinpoint their weaknesses and imagine how they might fail. Apply that to software testing. Bounce the ideas off of colleagues, and take their criticism of the ideas seriously. Be self-critical – are you finding important problems? Is your feedback helping the team prevent defects? Do the problems you log get fixed? If not, why? Also, work on your bug and status reports. Ask for help from programmers on how to write better reports, and ask them what kinds of bugs they appreciate you finding. Work with them to learn more diagnostic and investigative reporting skills.

I thank Jonathan for taking time to answer these questions and for sharing his thoughts and insights with my readers and me. It was great to know him in a better way via these questions. I hope that you (my readers) enjoyed this interview as much as I did. Let me know how you felt about the interview, so that I will plan more such interviews on Software Testing Zone in future.

Happy Testing…

Share on Google Plus

About Debasis Pradhan

Debasis has over a decade worth of exclusive experience in the field of Software Quality Assurance, Software Development and Testing. He writes here to share some of his interesting experiences with fellow testers.