Notes on “Which Tests Should We Automate by Angie Jones”

Notes from Which Tests Should We Automate - Angie Jones – Sr. Automation Engineer, Twitter.

Automate All the Things?

• automation requires maintenance
• lots of automation requires lots of time
• redundant automation is noisy

Automate the Right Things

Should we automate this:

1. What is my gut feeling?

2. What is the risk?

   x-axis multiplied by y-axis probability (frequency of use by customers) vs impact (if broken, what’s the impact to customers?)

3. Value

   distinctness (does this test provide new info?) vs induction to action (how quickly would this failure be fixed?)

4. Cost-Efficiency

   quickness (how quickly can this be scripted) vs ease (how easy will it be to script this)

5. History

   similar to weak areas (volume of historical failures in related areas) vs frequency of breaks (volume of historical failures for this test)

Tally up the scores into 3 categories:

• Automate: 67 - 100
• Possibly automate: 34 - 66
• Don’t automate 0 - 33

Links

• Which Tests Should We Automate - Angie Jones – Sr. Automation Engineer, Twitter
• Slides from the talk