Notes from Which Tests Should We Automate - Angie Jones – Sr. Automation Engineer, Twitter.

Which Tests Should We Automate - Angie Jones – Sr. Automation Engineer, Twitter

Automate All the Things?

  • automation requires maintenance
  • lots of automation requires lots of time
  • redundant automation is noisy

Automate the Right Things

Should we automate this:

  1. What is my gut feeling?

  2. What is the risk?

    x-axis multiplied by y-axis probability (frequency of use by customers) vs impact (if broken, what’s the impact to customers?)

  3. Value

    distinctness (does this test provide new info?) vs induction to action (how quickly would this failure be fixed?)

  4. Cost-Efficiency

    quickness (how quickly can this be scripted) vs ease (how easy will it be to script this)

  5. History

    similar to weak areas (volume of historical failures in related areas) vs frequency of breaks (volume of historical failures for this test)

Tally up the scores into 3 categories:

  • Automate: 67 - 100
  • Possibly automate: 34 - 66
  • Don’t automate 0 - 33