Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (13)
Similar a A recipe for succeeding with test automation
Similar a A recipe for succeeding with test automation (20)
Último
Último (20)
A recipe for succeeding with test automation
- 1. A RECIPE FOR SUCCEEDING WITH TEST AUTOMATION
- 3. AGENDA - Intro - Step 1: Think - Step 2: Define - Step 3: Gherkin - Step 4: Code - Step 5: Review - The recipe
- 4. INTRO “Don’t Gherkin too early” - Aslak Hellesøy, CukeUp#16
- 6. THINK Do not Gherkin too early! Analyze the feature & solve all your questions
- 7. THINK EPIC: Thirsty plants Twitter for water
- 8. Hands on
- 9. THINK Asking the right questions Step 1: Give Context Step 2: Ask questions - Open questions - Closed questions
- 10. DEFINE Do not Gherkin too early! Define test scenarios with mind maps
- 11. DEFINE What is a mind map? A mind map is a diagram used to represent ideas and concepts in a graphical way.
- 12. DEFINE STORY 1: Tweet moisture status of the plant
- 13. Hands on
- 15. DEFINE
- 17. WRITE Write your Gherkin scenarios Do not Gherkin alone! Define what each step does
- 19. WRITE Given sets the preconditions of your test. When is the action you are performing, the behavior. Then is the expected result, the consequence of the action.
- 20. WRITE What are the benefits? Test scenarios can be read and understood by anyone on your team. Help to improve communication. Help to build domain language. Helps to understand the scenario when you read it in the future.
- 21. WRITE Scenario 1 Scenario: Plant Given I have a plant And I haven’t watered the plant When the plant goes moist Then the Twitter event should be “watering needed” When the plant goes dry Then the Twitter event is “urgent watering needed”
- 22. Hands on
- 23. WRITE Scenario 2 Scenario: Watering a plant that doesn’t require it Given the plant is ok and I have watered the plant When I measure the moisture level And I see it goes above 600 Then the Twitter event is “too much watering”
- 24. Hands on
- 26. WRITE Gherkin tips (I) Each scenario tests one behavior Scenarios are written in a declarative way Avoid incidental details inside the scenario Omit the obvious Avoid conjunctive steps Keep your scenarios short
- 27. WRITE Gherkin tips (II) Don’t have to many scenarios in the same feature Use descriptive scenario names Write Given steps in past tense Have only one When step Use the “should” in Then steps
- 28. Hands on
- 29. CODE Scenario by scenario & step by step Do not forget to make tests fail!
- 30. CODE “Automating is coding” - Dorothy Graham & Seretta Gamba, expoQA #14
- 31. Hands on
- 32. CODE Good practices (I) Follow design patterns. Page Objects pattern for UI testing Use same programming language developers use in the project. Use code analyzer tools (aka linter).
- 33. CODE Good practices (II) Mock dependencies. Each test scenario creates the data it needs. Integrate testing code with development code. Run your tests on every pull request.
- 34. REVIEW Make others review your scenario and code And then… work on more scenarios
- 35. REVIEW “Two heads are better than one, not because either is infallible, but because they are unlikely to go wrong in the same direction.” - C.S. Lewis
- 36. Hands on
- 37. REVIEW Offering feedback (I) Think before you react. Talk to arrive to an agreement. Don't make demands. Ask for clarification. Be humble, don’t use hyperbole or sarcasm.
- 38. REVIEW Offering feedback (II) Don’t assume that what your are reviewing is wrong. Don’t just say improve it, explain why something should be changed. Use concrete examples. https://github.com/thoughtbot/guides/tree/master/code-review
- 39. THE RECIPE THINK DEFINE WRITE CODE REVIEW
Notas del editor
- [Who we are] We are Aida and Gloria We come from Barcelona, Spain We are both QAs and passionate about testing and agile methodologies We run a meetup group called @bugbusters were we talk about testing and automation mainly. We work at Typeform [gif???] For those who don’t know our company We have a web application tool that allows you to do conversational data collection. We are super happy to have the opportunity to be here at this conference sharing how we do quality at Typeform.
- [Agenda] This is the agenda for today We will start by explaining how did we came up with our recipe Our recipe has 5 steps and we will discover together each one of steps little by little At the end of the workshop we will have a recipe that, we hope, will inspire you to succeed in your automation processes. After the “Step 2” we will do a break.
- Does everybody understand what means “Gherkin”? Last year we attended a conference, cukeUp, because in Typeform we use Cucumber to automate. That conference opened our eyes. I particularly remember one of the sessions Aslak did. Aslak is one of the Cucumber cofunders. He did a good Gherkin workshop, and I remember he was constantly saying: “Don’t Gherkin too early”. Gloria and I look to each other and said: “That’s exactly the problem we have, we Gherkin too early!” We sometimes had a lot of problems when doing the Gherkin, it was hard and we couldn’t came up with the sentences. After this workshop we realized it was because we used to jump to Gherkin directly without properly thinking on what we wanted to test. When we came back to Barcelona we started thinking how could we solve that problem. We started thinking on a light process we could implement. And that’s is how we came up with our 5 steps recipe we are going to share today with you. Now we will do a little game to make teams. This are going to be the teams for the whole workshop and there is going to be a facilitator on each team. [Present Raquel, Santi and Javi] On each step we will do exercises to practice. Whenever you see a blue slide with a clock that means that is time to practice. We are going to present a topic and this is the topic we are going to use for all the workshops.
- [Make teams using the animals technique] You have cards with animals In order to find your team mates imitate your animal. You can do the sound of your animal but keep in mind the sounds are different in different countries, so be open minded. [They have envelopes with the workshops material]
- [THINK] The first step of our recipe is: THINK. As we said before, before writing our scenarios we need to do some work. Does it happen to you that you read a story and it’s fully clear to you? And you have lot’s of questions? So the first thing we have to do is solving those questions. You cannot create your tests if it’s not clear to you what the story is about.
- [Explain the Epic] In order to start this workshop we are going to present an story we’ve created for you. You don’t need to take notes, you have this info in the material. The epic is: Thirsty plants Twitter for water. “We have come up with the idea of creating an application that allows plants to communicate their owners their watering level via Twitter. The product is made of soil-moisture sensors, a circuit board and a Raspberry PI. Our goal is to write the software for the Raspberry PI. The moisture level of the plant is measured using the soil-moisture sensor. This sensor is connected to a circuit board to convert the information the sensor measures into bits using a range between 0 and 1023. This information will be communicated to the Raspberry PI. The application has to determine whether moisture levels are too low, or too high. Every time there is a meaningful change in the moisture level a message should be posted on Twitter.”
- [Workshop: Asking the right questions] [Share: Inside the team share how did it go] Workshop: A part from the epic, which gives you an overall intro of the applications, you have two user stories. Take a few minutes to read the stories. So if you think you have all the information you need to start writing your tests you can go for a coffee, but if you think you don’t, you have 15 minutes to ask questions to your PO’s to gather all the information you need. The PO’s are your facilitators, and they have all the info about the application. The problem is that sometimes they don’t express themselves very well, so make good questions. Share: Now we would like you to take some minutes to think about how did it go and whether you got all the information you needed. Share that info with your colleagues.
- Step 1: Give context It is important to give some context before asking the question so that the other person knows what are you talking about. This is especially important if you are introducing a new topic. Example: What are the Tweet messages that the plant is going to post? Here we don’t know if we are talking about the first story or the second story. The messages that is going to post after a watering event? Step 2: Ask questions There are two kind of questions we can ask: open questions and closed questions. Open questions The objective is to become better informed. They invite the reflexion and thinking. Answers use to be long, at least the questioner seeks for that. You can make them starting with: what, how… After the question is answered it is a good exercise to summarize what the respondent said to make sure you understood it and you both are in the same page. Example: How do I know the moisture status of a plant after watering it? Closed questions Make them when you seek for a clear answer. They give you facts. They help to keep control of the conversation. You can use them for testing understanding, for example after an open question. Answers use to be a single word (ex. yes/no) or a short phrase. Example: Do we send urgent watering event when the plant moisture level is below 350? - So now, do you think you are ready to start writing your tests?
- [DEFINE] We think you are not! / We agree Something that we also forget when working on agile environments is doing a test plan. In the past, when we were working with Waterfall like methodologies, we used to spend countless hours doing a test plan and now we just skip it. In this second step we want to introduce a technique to create light test plans using mind maps.
- What is a mind map? A mind map is a diagram used to represent ideas and concepts in a graphical way. It is a visual thinking tool that helps structuring information. Its power lies on its simplicity. It provides an easy way to brainstorm thoughts without worrying about order and structure too much. It is often created around a single concept, to which we associate ideas. Major ideas are connected directly to the central concept, and other ideas branch out from those. Why do we use them? We use them to do light and quick test plans that will provide visibility on testing. We can even use mind maps to plan in which level are we performing tests. - Don’t worry we will see an example latter.
- [Mindmap for story 1] We’ve prepared a mind map for the first story, let’s analyze it together. [Explain the mind map]
- [Workshop: Create the mind map for story 2] [Share: Share 1-2 mind maps all together] Exercise: Now that you know how to create test plans using a mind map, let’s do it for the second story. You have a sheet with the mind map of the first story, you can add your part there as it is related. Share: - Does anyone what to share their mind map? Use a board they can write in.
- [Mindmap for story 1 and 2] This is our mind map for both stories Let’s zoom to the second part.
- [Zoom to the mindmap to show story 2]
- [Mindmap for story 1 and 2 with tags] Now we have a very visual representation of how do we want to structure our tests. And it is very easy to share it with developers and discuss at which level do you want to test each scenario. Then you can simply add a tag to represent it. Ideally you would create this mind maps together with your team, but as a first step you can create them and discuss it later. [Explain the tags] - Now, are we ready for writing our scenarios?
- [WRITE] Now that we have solved all our doubts And we have a clear understanding about what things we want to test We are ready to write the Gherkin scenarios for our tests By the way, do not Gherkin alone, do it with your team in order to start to create your domain language
- How many of you know what Gherkin is? Gherkin is a Business Readable, Domain Specific Language that lets you describe software´s behavior without detailing how that behavior is implemented. It is used in the tools like Cucumber.
- Tests are structured in 3 different parts. Set the testing data you need for your test Perform the action you want to test Verify the action has been correctly performed Gherkin provides keywords to follow this structure: [Explain the diapo] Given sets the preconditions of your test. When is the action you are performing, the behavior. Then is the expected result, the consequence of the action. If you want to add more info to the Given, When or Then you and use the And keyword.
- [Benefits of defining tests using Gherkin]
- [Scenario 1 wrong] This is an scenario that we’ve prepared to test one of the scenarios.
- [Workshop: Find what is wrong in story 1] [Share: Share the ideas all together] It is very important to spend time writing your scenarios. If you don’t do it well, others will not find it useful, it will be hard to code, with time it can be painful to add more scenarios… In teams, discuss the problems you find in this scenario. After x minutes we will share your thoughts. Problems: Use descriptive scenario names The scenario name is not telling us anything about the test that is going to be carried. Each scenario tests one behavior We are testing two behaviors in the same test: going moist and going dry. If there is more than one When that probably means something is wrong. Omit the obvious It is obvious that there is a plan involved, we don’t need to add that information. Avoid incidental details inside the scenario We can assume the default status is that there is no watering and avoid adding this information. Use the “should” in Then steps This is a recommendation to align on the way of writing scenarios
- [Scenario 2 wrong] - You have another scenario to practice a bit more.
- [Workshop: Find what is wrong in story 2] [Share: Share the ideas all together] Problems: Avoid conjunctive steps When you have conjunctive steps this probably means you can break the step into two steps. Scenarios are written in a declarative way & Have only one When step There are two When because we are describing the behavior in an imperative way. Having more that one When is an smell and we should take care when we see so. Use the “should” in Then steps This is a recommendation to align on the way of writing scenarios
- [Introduction to our tips] - Now we will like to share our tips. We’ve talked about most of them, let’s do a quick review. Some of them are very important but others are just recommendations. You should talk with your team to find out your tips.
- [Gherkin tips]
- [Gherkin tips]
- [Workshop: Rewrite story 2 following Gherkin tips] [Don’t share anything] - Now that you know best practices to write good Gherkin, re-write the second story following those tips.
- [CODE] Once we know the scenarios we want to test and we have prepared the gherkin, we are ready to code it. Remember when you are coding your tests that a good practice is to do it scenario by scenario and step by step In that way you are only going to code the things you really need for it. And the most important part, do not forget to make your tests fail in order to know your tests are actually testing something. We have seen UI tests that were green when the UI was broken.
- [Intro: Explain that in automation we should use the same good practices we use when coding an application] Remember that we are also generating code. This is the reason why we need to follow good practices as Developers do when they structure and develop their code. In this workshop we are going to see some of them but we are going to be more focused in automation good practices.
- [Workshop: Choose good practices for automating] In the envelope you will find cards with different “practices”, some of them are good and others not so good You will have some minutes to discuss them and choose the ones you like [Share: Share chosen cards and why they were chosen] Know is time to share with the other teams what are your “good practices” and why. [We will ask directly to each team to explain one of their good practices]
- [Good practices] Follow design patterns. From the cards given, this is the subset of good practices. Follow design patterns in order to have a clean and maintainable code Page Objects pattern for UI testing An example of a design pattern in testing is the well Page Objects pattern for UI testing. Everybody is familiar with this Pattern? What we are doing is map each webpage with a class. Inside this class we declare the elements of the web page we want to interact with. The class also includes all interactions we do with the webpage. Use same programming language developers use in the project. If we use the same programming language developers are more likely to help because they feel more confident with the language. We can reuse code produced by the devs to build our tests. [This is a thing we are not doing right now, but we would like to. Sometimes QAs in the same company tends to use the same programming language, in order to make it easy teams changes.] Use code analyzer tools (aka linter). Everybody knows what a linter is? Linters are tools that performs static analysis of the code in order to check that it follows some guidelines.
- [Good practices] Mock dependencies. Everybody knows what a mock is? A mock is a simulated service that mimic the behavior of the real one. It is not good to rely in third party services. If you do it and the service is down all the tests that rely on that service are going to fail, but your application may be working fine. If we mock we are going to remove that dependency and the tests are not going to fail even the service is down. A part from that, we are going to save time in our executions as usually third party services are slow. Remember that if you do that, you need to add a few integration tests in order to check that we can properly communicate with the service. Each test scenario creates the data it needs. If we create the data we need on each scenario, these scenarios are going to be independent between them. We can also execute our tests in different environments as we don’t rely on data being set up in the environments. Integrate testing code with development code If we place testing code in the same repository development code is developers are more likely to help us as they don’t have to change project to contribute to testing. Moreover build/deploy pipeline is going to be simplified because you are working only with one project. Moreover you can execute your test inside the build job. This way, if the build job fails you don’t know if it is because the build couldn’t be done or because tests were failing. In that way you cannot point if the responsible were devs or QAs. Run your tests on every pull request If you run your tests in every pull request you are getting early feedback and you will not merge code that is not passing all the tests. If new changes in the code makes our tests fail because there is a change on an existing functionality, the team can fix the tests in the same pull request.
- [REVIEW] - Once you have code your scenario it is a good idea that someone else reviews it
- [Intro: Explain the importance of working together] - When you are focused on a problem you may loose perspective. Having another person looking at your code can help you as two heads are always going to be better than one.
- [Workshop: Give feedback to other team’s story from Write workshop] Now you are going to review the gherkin done by the team in your right side. Follow the discussed Gherkin tips to leave that feedback. We are going to use post it to give that feedback Each post it is going to represent one idea and you have to place it close to the area to improve [do an example] [Share: Share how do you feel about the feedback given to your story inside the team] Go back to your stories and read the feedback the other team gave Discuss inside your team if the feedback is useful., if it helps you to improve, how you feel about it…
- As you may have realized offer feedback is not easy. You need to be careful not to hurt others feelings. Offering feedback: - If you disagree strongly, consider giving it a few minutes before responding; think before you react. If there is growing confusion or debate, talk about it and arrive to an agreement. - Ask questions or propose changes; don't make demands. ("What do you think about…?"). - Ask for clarification. ("I didn't understand. Can you clarify?"). - Be humble ("I'm not sure - let's look it up."), don't use hyperbole ("always", "never", "endlessly", "nothing") or sarcasm.
- Offering feedback: - Don't assume that what you're reviewing is wrong. Offer alternatives, but assume the author already considered them. ("What do you think about using a custom validator here?"). - Explain your reasons why something should be changed, instead of just asking for it to be changed. ("Not in line with the style guide? A personal preference?"). Offer ways to simplify or improve code while still solving the problem. Don't just say improve it. - Avoid speaking in general terms and providing the person with hints in hopes they get the bigger picture. Use concrete examples and make references to specific events. This some of our agreements. It is nice to meet with your team to agree on your own guidelines. Thoughtbot has a guideline that inspired us, you can check it following this link.
- [THE RECIPE] We have arrived to the end and we have discovered the whole recipe Remember that for succeed you need to… We hope you enjoyed the workshop
- [Thank you] Thank you very much Do you have any question about it?