The document outlines an usability testing workshop. It discusses why usability testing is important, when it should be conducted, and different methods for testing. Testing can range from minimal testing with one tester to maximal testing with many testers. The document provides guidance on developing test scenarios, recruiting participants, moderating test sessions, and analyzing results to improve the design. Both in-person and remote testing methods are described. The goal is to get feedback from real users to identify usability issues and verify the design is intuitive.

1. Usability Testing Workshop

2. snímek |datum |dokument | 29. 1. 2015 2
Introduction
• Adam Fendrych
» www.dobryweb.cz
» www.sighi.cz

3. snímek |datum |dokument | 3
Usability Testing
• Veryfying design with real users
• Qualitative research method
29. 1. 2015

4. snímek |datum |dokument | 4
Why
• Nobody can make a perfect design from
scratch
• Your user behavior guess is 75% wrong
» http://www.useit.com/alertbox/guesses-data.html
29. 1. 2015

5. snímek |datum |dokument | 5
Why
• A usable website performs better
• 60% shopping carts are left behind
• „The users will get used to it“ – only if they
do not have another option
29. 1. 2015

6. snímek |datum |dokument | 6
When
• The earlier the better!
• Testing early means better RIO
• Testing late means less problems fixed
29. 1. 2015

7. snímek |datum |dokument | 7
How
• The good thing - scalability
• Any kind of testing gives better data than
no testing
• Anybody can do it
29. 1. 2015

8. snímek |datum |dokument | 8
Minimal Testing
• With myself :-)
» http://www.sme-union.cz
29. 1. 2015
• 1 tester – collague,
relative, neighbour
• 1 test task

9. snímek |datum |dokument | 929. 1. 2015

10. snímek |datum |dokument | 10
Maximal Testing
• Preparation
» Target group analysis
» Writing scenarios
» Screener
» Recruiting 8 testers
• Testing
» Audio and video output
for observers
• Results
» Test summary
» Recommendations
» Presentation
» Video highlights
29. 1. 2015

11. snímek |datum |dokument | 11
Choosing Methodology
• What results do I need
• What questions I want answered
• How much money do I have
• And time, too
• Phase of project development
29. 1. 2015

12. snímek |datum |dokument | 12
TiVO Case Study
• Website development
» 12 test sessions in 12 weeks
» Each Wednesday with 4 users
» The whole development team observing
» http://www.useit.com/alertbox/weekly-usability-tests.html
» http://www.tivo.com/
29. 1. 2015

13. snímek |datum |dokument | 13
Link Recognition Test
• Can the users recognize all links?
• Quick test, immediate results
• How
» www.pixlr.com + wasd.cz/m.jpg
• Result
» http://klient.dobryweb.cz/skoleni/ut/link-test-01.jpg
» http://klient.dobryweb.cz/skoleni/ut/link-test-02.jpg
29. 1. 2015

14. snímek |datum |dokument | 14
Usability testing schedule
1. Analysis
2. Target groups
3. Recruitment
4. Test scenarios
5. Test set up
6. Test sessions
7. Result analysis
8. Fixing problems
29. 1. 2015

15. snímek |datum |dokument | 15
Analysis – What Do I Want?
• Qualitative vs. Quantitative data
• Partial vs. Overall usability
• Testing does
» Reveal problems
» Verify your thesis
• Testing does not provide
» Problem solution
» Success rates
29. 1. 2015

16. snímek |datum |dokument | 16
Target Groups
• Demographics
• Relation to the website
• Domain knowledge
• Computer skills
29. 1. 2015

17. snímek |datum |dokument | 17
Exercise
» Age
» Sex
» Social background, wealth
» Education
» Computer skills
» Domain knowledge
» Basic needs
29. 1. 2015
• What are your target groups?

18. snímek |datum |dokument | 18
Recruitment
• Friends
» Snow ball effect
• Client database
• Tester database
• Colleagues
• Online – website, Facebook
29. 1. 2015

19. snímek |datum |dokument | 19
Recruitment tips
• Start early
• Use phone, not e-mail
• Send confirmation e-mails
• Call again the day before
• Give basic info – time, incentives,
requirements
29. 1. 2015

20. snímek |datum |dokument | 20
How Many Testers?
• Consider your goals, finances…
• One-round testing 5—7
» Why: http://www.useit.com/alertbox/20000319.html
• Repetitive testing 3—4
• Two times 3 is better that 6
• Recruit 1 extra
29. 1. 2015

21. snímek |datum |dokument | 21
Test Scenario
• Typical user tasks
• Determine the test focus
and results
29. 1. 2015

22. snímek |datum |dokument | 22
Tasks
• Less specific
• Not too difficult
• With side tasks
29. 1. 2015

23. snímek |datum |dokument | 23
Tasks
• Make the first task easier
• Consider all possibilites
• Ask about each tester‘s needs
» Ask about their needs
» Ask about expectations
» Ask if the website met them
29. 1. 2015

24. snímek |datum |dokument |
Pilot Test
• Grab a colleague or friend
• Go through the scenario
• Count with more time
29. 1. 2015 24

25. snímek |datum |dokument | 25
Test Set Up
• A quiet room
• A computer
• Recording
» Camera
» Screen recorder (HyperCam, Morae)
• Observation room
• Snack
29. 1. 2015

26. snímek |datum |dokument | 26
Test Set Up
Make sure
everything works
(I bet it doesn‘t)
29. 1. 2015

27. snímek |datum |dokument | 2729. 1. 2015

28. snímek |datum |dokument | 28
Test Session
• Moderator talks to the
participant
• 1 hour session
maximum
• Opening and closing
interview
29. 1. 2015

29. snímek |datum |dokument | 29
Opening Interview
• Make the participant feel relaxed
• Tell them:
» We test the website, not you  you cannot do wrong
» Recording
» Session length
» No advice during the test
» Think aloud. Really aloud!
» Say „I am lost“ when you are
» Critisize! I didn‘t make this website
29. 1. 2015

30. snímek |datum |dokument | 30
Opening Interview
• Participant information
» Basic info recap
» Relation to the website
» Their habits
» What information they need
29. 1. 2015

31. snímek |datum |dokument | 31
Test Session
• No advice!
• Lead out of dead ends
• Ask when they hesitate!
• Do not listen, observe
» Users „lie“
• Make notes
» Create 5-second waiting breaks
• Stick to the time
29. 1. 2015

32. snímek |datum |dokument | 32
Closing Interview
• Overall impression
• Biggest problems
• Suggestions
• Post test satisfaction form
29. 1. 2015

33. snímek |datum |dokument | 33
After the Test
• Analyze as soon as possible!
• Fix the problems
• Optional
» Written summary
» Video highlights
» Presentation
• Implement new solution
• Test again
29. 1. 2015

34. snímek |datum |dokument | 34
What Can You Test
• Architecture
• Paper prototypes
• Wireframes
• Interactive wireframes
• Whole website
• Graphics
29. 1. 2015

35. snímek |datum |dokument | 29. 1. 2015 3535
Architecture
• Where would users look for „X“
• What do they expect under „Y“?
• Card sorting
• Online tools – www.websort.net
29. 1. 2015

36. snímek |datum |dokument | 36

37. snímek |datum |dokument | 37

38. snímek |datum |dokument | 38
Testing a Whole Website
• Natural user behavior
• Give task and observe
• Fixing problems is difficult
• Cannot test everything
29. 1. 2015

39. snímek |datum |dokument | 39
Wireframes
• Very easy iterations
» After the test
» Inbetween sessions
• No tasks
• Testing understanding
» „What happens when you click here“
» „What is this good for?“
• Wireframes: paper, MS Word, Inkscape,
Photoshop…
29. 1. 2015

40. snímek |datum |dokument | 40

41. snímek |datum |dokument | 41
Interactive Wireframes
• Simple tasks
• Easy iterations
• Note
» Not a complete website
» No graphics
» Lots of content missing
• How: Axure, Justinmind, Visio…
» http://www.youtube.com/watch?v=wM2k31MnDUc
29. 1. 2015

42. snímek |datum |dokument | 42
Paper Prototypes
• Rough sketches on paper
• Fast, efficient
» http://uk.youtube.com/watch?v=GrV2SZuRPv0
» http://uk.youtube.com/watch?v=c4-A-9hGn0U
» http://www.paperprototyping.com/
29. 1. 2015

43. snímek |datum |dokument | 43
Graphics
• Use eye tracking
» Testing visual priority
» Careful
• Results depend on how you ask
• Different results in different time
29. 1. 2015

44. snímek |datum |dokument | 44

45. snímek |datum |dokument | 45
Remote testing
• Moderated
» Direct communication
• Non-moderated
» Indirect communication
29. 1. 2015

46. snímek |datum |dokument | 46
Moderated
• Reach distant participants
• Same time cost as non-remote
• How
» Skype/Adobe ConnectNow + Morae/Hyper Cam
» Webex/GoToMeeting/UserVue
• Recruit online (Ethnio)
29. 1. 2015

47. snímek |datum |dokument | 29. 1. 2015 47

48. snímek |datum |dokument | 48

49. snímek |datum |dokument | 49
Non-Moderated
• No direct communication but written tasks
• Mass recruitment
• Incentives not necessary
• Very short test sessions
• No observation, just test results
• Tools
» OpenHallway, Usabilla, Webnographer
29. 1. 2015

50. snímek |datum |dokument | 50
Other Methods
• Five second test
» http://fivesecondtest.com/
• Navflow
» http://navflow.com/
29. 1. 2015

51. snímek |datum |dokument | 51
Let‘s Do Some Work
• Scenarios
• Moderation
• Analysis
29. 1. 2015

52. snímek |datum |dokument | 52
Test Session
1. Write 1 test task (10 minutes)
2. Moderate using your task
3. Analyze the session
4. Stay on stage as a participant
29. 1. 2015

53. snímek |datum |dokument | 53
Why Is This Not Relevant
• Participant is not your target group
• Tasks do not reflect participant‘s needs
• Too much disturbance
• No opening interview
29. 1. 2015

54. snímek |datum |dokument | 54
Test Session
1. Write 1 test task (10 minutes)
2. Moderate using your task
3. Analyze the session
4. Stay on stage as a participant
29. 1. 2015

55. snímek |datum |dokument | 55
Questions
29. 1. 2015
www.dobryweb.cz
adam.fendrych@dobryweb.cz