The document discusses automation in software testing. It begins by outlining common claims made about the benefits of automation, such as saving time and improving quality, but argues that these claims often don't hold true. Automation does not inherently save time, guarantee quality, or reduce resources needed. It also does not always save money when development, maintenance, and infrastructure costs are considered. The document provides a formula for determining when automation is worthwhile based on how many times a test case would need to be rerun manually. It concludes by acknowledging that, despite these drawbacks, organizations will still automate testing because it is exciting, managers demand it, and it benefits careers.

1. Why You Shouldn’t
Automate...But You Will
Anyway
By Kevin Pyles

2. About Me

3. #ProQuality

4. My Feelings on Automation

5. Conversation with an Executive

6. Testing > Automation

7. Why You Shouldn’t Automate
“You are going to
present on What?”

9. Automation Marketing 101
Save Time
Better Quality
Save Money
Fewer Resources
Easy to Use

10. What you are sold

11. What you receive

12. Automation Saves Time
You can’t save time
You can’t manage time
You can’t get more time

13. Automation equals better Quality
Automation doesn’t make quality
Automation doesn’t fix bugs
Automation doesn’t determine quality

14. Automation saves money
Money can be saved
Automation costs money
Tool, people, servers
Devices, time, training

15. “Automation is not free”

16. How much money will we
spend before we start
“saving” money?
-Executive

17. Automation means fewer resources
Automation is a software project
Automation & Manual Teams
What are you automating?

18. Who tests the software
that tests the software?
-Paranoid Tester

19. Automation is Easy to Use
Hmm...might actually be true
Easy to use = powerless
Selenium IDE vs. WebDriver

20. MSPaint vs. Adobe Photoshop

21. Automation wasn’t working for us
Agile means our product is always changing
Automation was slowing down our testing
Automation was costing money
Frameworks change

22. Time Spent Automating vs. Bugs Found

23. Why You Shouldn’t Automate…
Automation does not save time
Automation does not guarantee quality
Automation does not mean fewer
resources
Automation does not save money
Good automation is not easy to write

24. What value is your
automation providing?
Should you stop automating?

25. Why You Will Still Automate
It is exciting
Your boss will ask for it
It looks good on the resume

26. When to Automate:
The Formula
Examples

27. The Formula
T(A) <= T(M) x 2.5

28. The Formula
T(A) <= T(M) x 2.5
T(A) = Time to Write Automation
T(M) = Time to Run Manual Test

29. Example 1
10 minutes to run test manually
25 minutes to write the automation
If you run the automation more than 3 times
it was worth your time
25 < 10 * 3
Yes! Worth automating

30. Example 2
30 minutes to run manually
3 hours to automated (180 minutes)
Run test 10 times
180 < 30 * 10
Yes! Worth automating

31. Example 3
5 minutes to run manually
45 minutes to automate
Run 5 times
45 > 5 * 5
Not Worth Automating

32. More Examples
Link Checker
Load Testing
Data Generation
Smoke Tests

33. Signs Your Automation Project is Failing
Vague responses to the status
Automation doesn’t run and report
regularly
Failed tests with apathy
No coverage metrics

34. I believe…now what?
Questions to ask yourself
How to approach this with your
manager

35. Questions to ask yourself or team

36. Discussions with your manager
Seek first to understand, then to be
understood (Stephen Covey)
Set a clear picture of Reality
Provide a solution, not a complaint
Agree upon a realistic expectation
Ask for a time to return and report

37. Takeaways
Don’t Automate
The Formula
Signs Your Project is Failing
Still Automating

38. Questions?
@pyleskevin
kevinpyles@gmail.com