Whiteboard Wednesday: The ROI of Testing

7 Responses to “Whiteboard Wednesday: The ROI of Testing”

1. ZebZiggle on January 9th, 2008 5:59 am

   What if you only do UI testing? That would catch all the CRUD and BL tests as well. Is that a better “Bang for the Buck?” Selenium is great for this and it less fragile.

   The con’s are:

   1. Slow to run.
   2. “Don’t touch” while running
   3. Expensive and repetitive set-up and tear-down.
   4. You need to orchestrate your build to support UI testing.

   At the very least automate your UI smoke test. Follow the 80/20 rule of application usage and you’ll be in good shape.

   Nothing beats eating your own dog-food during development … but that is only possible for certain applications.

   Nice presentation as it touches on many of the issues, but I would have like to see more numbers around actually computing ROI.

2. Max Pool on January 9th, 2008 8:49 am

   @ZebZiggle –

   1. Only doing UI automation testing would be the equivalent of doing an airplanes first test as a test flight. Technically you are testing all the pieces, but you never assured that each individual moving part was working in isolation. I rarely do UI automation test until I have exhausted myself doing unit tests.

   2. Actual ROI numbers are completely subjective to your team and project. As long as you are aware of the different types of testing and each of their cost/benefit, you may be able to start quantifying numbers and finding the “sweet spot”.

3. ZebZiggle on January 10th, 2008 1:23 pm

   Agreed. I was just being facetious (read: pita).

   That said, I find you can’t get the same degree of comfort from QA like you do from UI-based testing (automated or manual).

   I’m not sure about the statement about ROI. I suspect there’s a quick equation for “Cost To Write Test” vs. “Stability of Product” that could be applied to each of stages of testing and generate an ROI.

   Something else for me to push on the “think about” stack.

   Keep up the great site!

   -Z

4. What About Thad? on January 10th, 2008 11:44 pm

   Great demonstration–good motivation to improve both kinds of testing. I have to be cheeky though and ask, “…an average of excellency”?

   Keep up the good blog 🙂

5. Max Pool on January 11th, 2008 7:40 am

   @Thad –

   Yes! An “average” of excellency! To clarify, whether you get a 94% or a 110% on a college exam, you have still receive an A. Never quit shooting for 110%, but if you had to pay $1 for each %, you might consider paying only the lowest amount thus maximizing your ROI. I meant what I said! 😉

6. What About Thad? on January 11th, 2008 8:59 am

   Of course 🙂

   The meaning was actually never unclear. I just enjoyed the humorous sound of the phrase itself–not unlike Garrison Keillor saying “all the children are above average”. In this case, however, we really can make excellent the average!

7. Huvitavat vaatamist maailmast « Beautiful code on February 18th, 2008 5:48 am

   […] The ROI of Testing […]