Expoqa17 - Cheesecake: The evolution of our automatic test suite

2. CHEESECAKE
The evolution of our automatic test suite

3. BIO

4. INTRODUCTION
• We had to create an automated test suite for a
web application
• Evolution of our automated test suite

5. OUR TOOLS
• Open Source
• Easy to learn
• Continuous Integration
• Document app behavior
• Flexibility

6. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 1
Scenario: Log in with valid credentials
Given I am on the login page www.myapp.com/login
When I fill the user name with jo@mail.es
And I fill the password with iLoveBdd
And I click the login button
Then I should be redirected to my dashboard

7. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 2
Scenario: Log in with valid credentials
Given I am on www.myapp.com/login
When I log in with the following data:
| username | password |
| jo@mail.es | iLoveBdd |
Then I should be redirected to my dashboard

8. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 3
Scenario: Log in with valid credentials
Given I am on the login page
When I log in with valid credentials
Then I should be redirected to my dashboard

9. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 3
Scenario: Log in with valid credentials
Given I am on the login page
When I log in with valid credentials [email:jo@mail.es]
Then I should be redirected to my dashboard

10. #2 BROWSER INITIALIZATION
Initial State: Browser started for each scenario
Before do
@browser = Watir::Browser.new :firefox
end
After do
@browser.quit
end
hooks.rb

11. #2 BROWSER INITIALIZATION
Problem: Slow
Solution:
• Open/close the browser when starting/ending the
suite
• Clean cookies between scenarios

12. #2 BROWSER INITIALIZATION
AfterConfiguration do
$browser = Watir::Browser.new :firefox
end
Before do
@browser = $browser
@browser.cookies.clear
end
at_exit do
$browser.quit
end
hooks.rb

13. #3 THE NIGHTLY BUILD
Initial State: Tests were executed on demand
Problem: Slow feedback loop
Solution: Nightly build + Rerun report

14. #3 THE NIGHTLY BUILD

15. #4 SPEEDING UP
Initial State: Long test execution
Problem: Slow feedback loop & flickering scenarios
Solution: Speed up the automation

16. #4 SPEEDING UP
How?
• Backend testing
• Micro-services testing
• Bypass login
• Parallelization

17. #4.1 BACKEND TESTING
Initial State: We were only doing UI testing
Problem: Slow feedback and flickering scenarios
Solution: Backend testing
• Full coverage in backend & re-implement some
tests in frontend

18. #4.1 BACKEND TESTING
• Reliable
• Fast – Quick feedback
• Easy to maintain

19. #4.2 COMPONENT TESTING
State: We were testing a monolith
Problem: Slow feedback
Solution: Component testing
• 1 service / project <> 1 testing suite

20. #4.2 COMPONENT TESTING

21. #4.3 BYPASS LOGIN
Initial State: All the UI tests go through the login
page
Problem: Interact with the UI is slow
Solution: Bypass the login page

22. #4.4 PARALLELIZATION
Initial State: Tests executed sequentially
Problem: Slow feedback loop
Solution: Parallelization

23. #4.4 PARALLELIZATION
Difficulties:
• Headless mode blocked other threads
– Solution: Managing the threads

24. #4.4 PARALLELIZATION
Difficulties:
• The initial state for the tests was not the expected
one
– Solution: Analyze each test and fix it

25. #4.4 PARALLELIZATION
Difficulties:
• One report was generated per thread
– Solution: Merge all the reports in one

26. #5 WHAT’S NEXT?
• Have code and tests under the same repository
• Dockerize your solution
• Make your test create data on the fly
• Execute tests in local environment
• Execute tests on each PR

27. #5 WHAT’S NEXT?
• Create your own “qa” classes to identify web
elements
• Talk with developers
– In which level to automate
– How to make testing easier

28. TAKE AWAYS
• Do not be afraid of changes
• Do not try to fix the big problem at once
• Do not think small improvements do not count
• Do not stop improving

29. THANK YOU
@bugbustersbcn
#bugbustersbcn
linkedin.com/in/gloriahornero
linkedin.com/in/aidamanna