I have written here, my thought process and proposal for the research and design assignment.
Upon reading the design assignment prompt, my high-level approach to tackling the research problem was:
- Understand the problem and identify the main research goals.
- Outline a user’s mental model based on the 4 step user workflow.
- Propose a user research plan.
For the design portion of this assignment, I will attempt to create a high-level concept of the 4 step user workflow. I anticipate that undergoing this brief design exercise will generate even more questions, of which research may be in a position to help shed some light in the future.
1. Understand the problem and identify the main research goals.
According to the prompt, the two main problems to be solved are:
- Users are unable to find the answers they want easily or at all.
- Users are getting wrong or non-accurate answers.
While working with the team, to start I would most likely interview key members of the team for some background information. Having this information not only helps with scoping and targeting the research appropriately, but also helps me understand how the research can make the biggest impact for the team.
- How do we know users are unable to find the answers or getting wrong answers? Was this problem reflected through user reports? Or certain behavioral metrics?
- Are there specific types of users who frequently having trouble finding answers?
- Does the team have a hypotheses to this problem?
- What is their main goal for solving this problem? How would the team measure success?
- What does the team already know about its users (and their goals and needs)? Where are there still gaps in understanding the users?
Here, I am going to continue this assignment under the assumption that the team does not know what types of users frequently have trouble finding answers, nor do they have a working hypothesis on the problem. And their main goal is to uncover the usability hurdles to improve the search experience.
So how can research help to solve these problems, and improve the design? Translating the problems into more specific research questions:
- Where in the process of searching for answers are usability hurdles occurring?
- What are the usability hurdles and/or system bugs that is affecting users from finding or obtaining accurate answers?
- Are there any unmet needs or gaps between users’ expectations and what the product provides during search?
2. Outline a user’s mental model based on the 4 step user workflow.
Instead of jumping straight into creating a research plan, first I wanted to understand the “product” I am researching and designing for, and think like a user who may using this product. The 4 step user workflow that was provided in the design prompt are essentially four immediate tasks or goals a user would want to achieve on the website.
Next, I wanted to think about what thought process and mental checkpoints users go through while doing each task, essentially a mental model of sorts. This allows me to understand the problem space more and come up with a few hypotheses. The problem prompt mainly focuses on the first two steps of the user workflow, and that is where I focused my attention:
The usability hurdles could be occurring at any step of the mental checkpoints. At this point, if I had the website I could perform a cognitive walkthrough, I could come up with a hypothesis.
3. Propose a user research plan.
Here is a reiteration of the research goals:
- Identify where in the process of searching for answers are usability hurdles are occurring.
- Uncover usability hurdles and/or system bugs.
- Uncover any major gaps between users’ expectations and what the product provides during search.
Here is the process I propose to put in place to begin to understand the problem:
- Intercept survey (which includes an invitation and screener for either remote or in-person usability testing)
- Intercept usability testing
Site Intercept Survey
The goal of implementing the intercept survey on the site is to identify who is having trouble achieving the goal of finding an answer to their question, catch users in the context of them not being able to achieve their goal on the site, and invite them to explain where / how the product has failed to help them achieve their goal. Tracking the frequency of users clicking on the survey invite over time may even be a possible measurement of success.
One way to implement a site intercept survey is to include a survey invitation on the answer page after a user has submitted a question. I would work with an engineer to include the survey invite on the site.
Example: Answer page
Clicking on the survey invitation will bring them to a brief survey that fulfills two main purposes: 1) Ask users where/why they think they’re having trouble finding an answer, 2) Invite them to participate in a live-intercept usability testing.
Here is a sample of what the first draft survey + screener could look like:
INTRO: We’re sorry you couldn’t find the answer you’re looking for, and we’re here to learn how we can serve you better.
Which of the following describes your situation?
- The questions I submit are not returning any results.
- The answers I receive don’t match what I’m looking for.
- I don’t know how to ask a question.
- Other (Please Specify)
(OPTIONAL) Please give us some more detail: E.g. What were you searching for? What answer were you hoping to get? What answer did you get instead?
SCREENER INTRO: To help us learn more about how we can make your experience better, we would like to invite you to participate in a 30 min research study with us online through your computer right now. You’ll be compensated $50 for your time.
SCREENER QUESTIONS: Are you willing to participate in a 30 min research study with us? Yes / No
(…additional screener questions to follow)
Intercept Usability Testing
- 10 – 12 usability testing sessions per round (multiple rounds can be conducted as needed)
- 30 min sessions
- main task: searching for answers
While the survey will allow us to quickly get feedback from a large number of respondents, the main shortcoming is there’s often a difference between what the user says and what they do. Therefore, the goal of implementing additional usability testing following the survey allows us to observe where / why users are having trouble finding answers on the site. Although there is increased difficulty in managing live recruiting, the possibility of intercepting users live helps compensate for the shortcoming of traditional usability testing where users are doing tasks out-of-context.
What kinds of findings I hope to uncover:
- Users’ mental model, expectation, attitudes, and gaps in understanding that affect how they use AskData’s search product.
- Usability pain points during the search process that prevent users from getting answers or accurate answers to their questions.
- Possible system bugs that need to be fixed.
- High-level insights on users’ unmet needs during the search process.
I used the Balsamiq wireframing tool to quickly mock up a high-level concept showing how users will interact with the AskData web app from step 1 to 4. Click here (ThoughSpot Design Assignment) for an interactive version of the wireframe concept.
1. He asks a question on the data (“What is the revenue last year?”).
I’ve created a vertical navigation layout, with user’s main tasks making up the menu items. Since asking a question is the main goal users want to complete, I made the search bar the only focus on the landing page. However, if we find from research that users don’t know what questions can be asked or don’t know how to construct a question query, it may beneficial to include visual cues that guide users on how to search in the null state of the landing page.
On this answer page, the key features are a sub horizontal navigation tab that allows users to choose which type of data (graph, list, or raw data table) to view. This is based on the assumption that users will want to see different presentations of the data depending on their goal for seeking the data.
At the bottom of the page, there is a floating action bar that allows users to save their answer, download it to their local drive, or share the answer with others.
3. He can save his answer as a document in order to be able to get back to it easily later on.
If the user wants to save his answer as a document for easy access later, he can click on the “Save answer” button on the floating action bar. A feedback message will flash briefly, indicating the answer has been successfully saved.
4. He can browse through his saved documents.
The user can navigate to “Saved Documents” on the main navigation tab to access his saved answers. I have designed the layout of the Saved Documents page to resemble that of a document finder with a preview window for an easy way to view the answer data. A download button in the preview window allows the user to download the answer to his local drive.