Risk-Based Testing - Meet Mr. Hacker Tester!

While “Functional Testing” ensures that a piece of software does what it is supposed to do under normal circumstances, “Risk-Based Testing” on the other hand, considers what a bad guy (could be a hacker, a malicious user, somebody from a competitor etc) might do based on risks identified by a security analysis. Risk-based testing is a fundamentally creative form of testing and is as strong as the risk generation that it relies on.

Do you consider the risk associated with the component while testing it? When you notice some anomaly while testing, do you ask yourself “What can go wrong here”? Do you try to imagine a worst possible scenario after breaking a feature while testing? Do you include the possible risk associated with a bug in the bug report?

If any of your answer to the above questions is YES, then you already do risk-based testing! For that matter, while testing most of the testers do some kind of risk analysis and risk-based testing either knowingly or unknowingly! Doing risk-based testing has some additional benefits too in addition to the obvious benefits! I would try to prove my point by a real life testing experience of mine.

Once I was testing a Login page of an online auction site. The Login Page had 3 major fields - User Type (a drop-down list box), User Id (a text box) and Password (again a text box), which accepted critical user inputs. While testing, I found that the User Type drop-down list box was accepting text input just like a combo-box! In other words, I was able to type text directly into the User Type drop-down and it was accepting it as legitimate inputs! How would you have rated such a behavior? I am not sure if any tester would consider calling this anomaly as a bug at all! At least, I was not ready to log this issue into the bug tracker yet, as I was not comfortable with the severity (very low!) of this bug. So I wanted to investigate more on this bug before logging it.

1. Since I was able to type inputs into the field, I tried to enter some invalid User Types into the field instead of selecting it from the drop-down menu. [FYI, there were 3 valid user types – Admin, Power Bidder and Lite Bidder]. I entered a valid User Id and Password and tried to Log In. Thankfully, the application was robust enough (!) to pass this test and did not allow me to log in.

2. This time I entered lot of characters into the drop down text and kept testing. [All this time, I was using valid inputs in the User Id and Password field]. However, it did not allow me to log in. But I kept increasing the number of characters in the input text in the User Type field. The size of my input text was more than 35,000 characters now and bang! I got a “HTTP Error 500 - Internal server error” and the application crashed. Investigating further revealed that when the input text size went passed 43,679 characters (with spaces), the application invariably crashed with an internal server error!

Now it was a good enough bug to be logged into the bug tracker. But there were at least a couple of important things (from a tester’s point of view) to note.

a) Although the User Type field was not supposed to accept any text input, still it had got an apparently critical upper boundary, crossing which was resulting in a crash due to internal server error.

b) Talking about risk, this might look like an innocent server side error. But as this crash was resulting from a possible buffer overflow/overrun, the risk of a breach of system security was high. A malicious user could have injected inputs specifically designed to execute malicious code or to make the program operate in an unintended way. The risk was great considering the fact that the involved field was meant for accepting “User Types”! Just think of a scenario when a hacker would have exploited the vulnerability by logging in the system, imitating an “Admin” user! Once successfully logged in as the Admin user, anybody could have taken control of the entire site! Still worst, being an online auction site, the financial aspect of the security vulnerability was too high to be compromised with.

Look how a seemingly less important bug (editable drop-down list) was in fact a serious security threat as far as risk assessment was concerned. No need to mention, it was considered a top priority issue and was fixed soon. I am wondering if this issue would have been considered as seriously, if I had posted it without the associated risk factor!

This certainly might not be the best example of a case where risk analysis had lot to do with the fate of the bug, but I am sure some of you also might have faced similar situation. Please feel free to share your own story regarding risk-based testing by leaving behind a comment.

Happy Testing…

Share on Google Plus

About Debasis Pradhan

Debasis has over a decade worth of exclusive experience in the field of Software Quality Assurance, Software Development and Testing. He writes here to share some of his interesting experiences with fellow testers.