MILESTONe 6
Testing
Problem Statement
To perform usability tests on the hi-fi prototype, which we added micro-interactions to in Milestone 5.
Significance of Testing
Methodology
Modality
Our usability testing plan was expedited, due to the timeline of Milestone 6. While we would ideally have conducted more tests on our lo-fi sketches and mid-fi wireframes, we made the circumstances work to test our hi-fi prototype. We tested 4 people face-to-face and 8 people online. For 1 of the 4 in-person tests, we had 1 notetaker and 1 moderator. Due to time constraints, each of the other 3 in-person tests just had one person acting as both a moderator and note-taker. However, we recorded these sessions to make it easier to review footage and take notes later, if needed.
On the UserTesting.com, each of our 4 teammates administered an online test with 3 participants each, for a total of 12 participants. Aside from half of the tests not including the SUS questions (described below in the section System Usability Scale), the tests were all the same so that we could compile aggregate data. The only reason we administered 4 different tests is because UserTesting.com’s licensing capped each of us at 3 participants per test. That said, each of us had 1 tester that seemed to have irrelevant feedback (e.g. due to their screen being frozen, which is outside of our control), so for the rest of this report, we only examine responses from 8 of the 12 online testers.
Recruitment
The study included both moderated and unmoderated testing sessions, and participants were recruited based on the features of their user personas. The research was divided into two phases: pilot testing and main testing. The study team conducted both moderated and unmoderated sessions with genuine individuals during the primary testing. To examine the data collected from both types of testing sessions, a mixed-method technique was applied. The study does have certain drawbacks, such as a small sample size for the moderated testing sessions, self-reported data, and a narrow focus on UX UI design. The study's findings can help comprehend the website's user experience and recommend areas for UX UI design improvement. Future study might try to expand the sample size and include more aspects that influence the user experience.
Testing Plan
We utilized a testing document that includes a moderator script (for in-person testing), task scenarios and descriptions (for both in-person and online), and follow-up questions (for both). This document can be found in the Appendix of the report.
Tools Used
Key Takeaways
Conduct more testing, earlier.
Testing a hi-fi prototype is not ideal due to the time and cost of redesigning core functionality if users find an
issue during testing.
Ask different questions to evaluate whether the app is successful in cultivating a feeling of community.
We focused on usability metrics, but did not receive much feedback on the app's feel, which was important for distinguishing our product.
Expand testing in terms of quantity and demographics.
We wanted to conduct more in-person tests to focus on seller-side tasks and experience, but were unable to recruit any Black or African participants. We would expand our recruiting efforts in the future to target this market.