UX Strategy and Planning » customer interviews

Retail Ethnography: Millennials and Multichannel

Paul Bryan — Wed, 10 Nov 2010 13:31:54 +0000

Usography conducted ethnographic research with millennials using an iPad. The mobile and multichannel pain points are obvious. Target’s app functioned very well, but there were some key points where it fell short. The main shortcoming was that the participant didn’t see any way to check inventory in her store, and then went to the store only to find that the item she wanted was sold out.

[If embedded video fails, use link: http://www.youtube.com/watch?v=HFdo1jKphSQ]

Other perceived shortcomings were:
• Channel Preference: She only wanted to view items that were available in the store, because she was doing pre-shopping for a store visit
• View Products in Context: She wanted to see the products in a room setting similar to her own, to virtually try them on in her space
• Sharing Options: She wanted to share the product via Facebook with her roommates, because they don’t read emails
• Quick Save Bin: Once in the store, she had a hard time pulling up the items that she had decided on. She had to search through her email, and then did a search to finally locate the product. She wanted a quick save bin on the iPad that didn’t require any kind of registration.

On the positive side:
• Visual Impact: She responded very well to the visual aspects of the product catalog on the iPad
• Product Reviews: The product reviews were very helpful and impactful on her decision process
• Aisle Locator: Once she found the products on the iPad that she had come to purchase, the display had an aisle number so she found the location fairly easily

The Walmart app was more functional in terms of checking store inventory, but she didn’t see any way to share the products with her roommates.

This participant, as with many of her peers, found the iPad shopping experience generally positive, with some required fixes. Ultimately her shopping trip was a failure because of availability. The participant said she would like to have a combination of the visual impact and convenience of the app, but the more robust availability functionality of the web site, with a range of sharing options. There is no doubt that she will adopt multichannel shopping with an iPad when the customer experience reaches the next level of maturity.

About the author
Paul Bryan is Director of User Research and Design Strategy at Usography Corporation.

Email: Paul [at] usography [dot] com

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Customer Interviews: Conducting the Interview

Paul Bryan — Thu, 31 Dec 2009 19:31:08 +0000

Once participants arrive, the interviews are mainly a matter of ensuring that the data you need to capture as outlined in the interview protocol is captured. After the first couple of interviews, the research lead will develop a sense of topics that are most likely to result in data that will be useful to innovative design work in the future. The research team should debrief after each interview to discuss whether the research protocol should be modified for any reason. Some questions or exercises don’t work as well as planned in terms of data capture and participation, and need to be amended or omitted. In my experience, the most changes are made either after the first session, the first day, or the first round of interviews.

As time goes by, the interview moderator may modify the interview for each participant to get the most value out of the interview. Some people feel that this is problematic because modifications to the interview precludes comparisons of data across participants for a given set of variables. My feeling is that assigning such weight and rigor to variables that are being measured qualitatively is to misunderstand the nature of the data being collected in customer interviews. I feel that it is best to get the most value out of each interview and to characterize each participant as clearly as possible, and then to compare variables across participants to the degree that it makes sense for the type of data being collected. If a participant is leading me down an interesting trail of discovery toward a potential opportunity for design innovation, then I will follow that trail in lieu of other sections of the interview that I feel will not be as fruitful. My sense of where to take the interview has developed over many years of interviewing customers for the purpose of creating innovative web sites.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Location Prep

Paul Bryan — Fri, 18 Dec 2009 16:38:14 +0000

In addition to the materials required to conduct the research exercises and to capture data, you will need to organize food and drink according to the standard practices of the organization sponsoring the research, and as allowed by the project budget. Sometimes the food and drink offering is elaborate, with a full lunch or dinner provided as a perk to the participants, researchers, and/or project sponsors. In other cases, it is very simple: “The snack machines are over there.” At a minimum, bottled water should be provided for the people who will be talking the most: the participant and the moderator. I prefer a middle of the road set up, because lots of food tends to cause unwanted distractions, and participants fumble with plates and trying to eat while being observed. I prefer a selection of soft drinks and juices, with a tray of assorted snacks like trail mix, power bars, or other easy to eat items.
Research days are often 10 to 12 hours long, and so a lunch tray for the research team is always welcome. I prefer to order lunch and have it delivered to the area where the team is located, rather than going out to lunch. Walking to lunch spots with the project sponsors and team is a good opportunity to bond, but in some cases it leads to many opportunities for getting off-track for the afternoon sessions. Project sponsors or wayward team members start window shopping or stopping for little errands, and then we’re late for the first afternoon session. Being cooped up in the same small room for 12 hours is no joy, but it tends to keep people talking over impressions, which can be very fruitful.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Daily Schedule

Paul Bryan — Tue, 15 Dec 2009 12:03:10 +0000

The research protocol should include a detailed schedule for each day. The following is a typical daily schedule for Usography research projects:

- Prepare participant testing station and observation room

- Make copies of script if necessary

- Set up video equipment when research assistant not present

- Replenish materials for exercises if necessary

- Prepare water and/or other refreshments

- Conduct Interview 1

- Debrief

- Replenish materials and prepare rooms, as needed

- Conduct Interview 2

- Debrief

- Replenish materials and prepare rooms, as needed

- Conduct Interview 3

- Debrief

- Replenish materials and prepare rooms, as needed

- Conduct Interview 4

- Debrief

- Assemble and label materials completed during the day

- Burn data to DVD

Research sessions are being scheduled at 2 hour intervals. The interviews usually last 75 to 90 minutes. There is a break between sessions of 30 to 45 minutes, when the team talks about what was observed in the previous session and the changes, if any that should be made to the research protocol. If the team takes a lunch break, it usually lasts 30 minutes and is taken on-site.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interview: Evaluate existing design

Paul Bryan — Fri, 27 Nov 2009 13:41:51 +0000

If a site design already exists at the time the research is conducted, then in most cases you will ask customers about that design in the customer interview. The design may exist as either current release of the site that customers are using, or it may be a non-released design in the form of page comps or a prototypeprototype. The case for omitting an interview module that evaluates this design can be made when you are conducting research for a completely new design that will not rely on previous design work. I’ll discuss the case of the customer interview that focuses solely on design evaluation in a more detailed post later.

Objectives of the design evalution module include:

Understand the gaps between the desired design and the actual design
Obtain specific feedback as to which design elements need to be increased or decreased in prominence
Determine components that participants would like to eliminate from the design
Determine components that participants consider missing from the design

Linked In: http://www.linkedin.com/in/uxexperts

In the next post, I’ll include an example protocol and script for a design evaluation exercise.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Participatory Design Exercise

Paul Bryan — Mon, 23 Nov 2009 13:23:46 +0000

The last post discussed a participatory design module for in-depth customer interviews. This post gives an example exercise and script.

To determine participant’s concept of the ideal Kiosk system for on-site booking of reservations and events.

1) Introduce list of 50 tasks used in card sort, to provide more raw material for participatory design exercise.

2) Provide blank wireframe with placeholders for header, navigation, portlets

3) Ask participant to design their own Home Page using their personal task list created above, and the list of 50 tasks provided

4) Probe regarding design elements

5) Offer possibilities based on previous comments if participant is stuck

Data to capture:

Wireframe sketch

Notes regarding optimal kiosk design

Script:

Introduce topic

In this exercise, I’d like to ask you to create the ideal Guest Kiosk Home Page. I’m not so much concerned about what everybody else would want on the Home Page. I’m concerned right now with what the ideal would be for you personally.

Before we start this exercise, I’d like to remind you of the tasks and content you’ve already mentioned. I’d also like to present you with a list of tasks have come up before, so that you can have plenty of raw material for doing this exercise. But again, we’re interested in what’s most useful and interesting for you.

Please take a moment to read through this list quickly, to jog your memory about things you would like to see represented at the very top of the kiosk Home Page.

Give participant time to review list and mark desired tasks.

This is a blank page, on which I’d like to ask you to draw your ideal kiosk home page. If you need more room, we can just make a notation on the front of the page, and continue that element or list on the back of the page.

First of all, what do you think the main sections you would like to see here at the top, or perhaps along the side?

What kinds of news or other content would you like to see spelled out here on the Home Page with a paragraph and perhaps a picture?

What kinds of links would you like to see?

What kinds of spotlights or highlights of content would you like to see here?

Would you want those to be just links, or part of the story as well?

What else would you like to place on the Home Page? Where would you place the search box? Do you want to be able to print this page? What about save this information for later viewing.

What kinds of functionality would you like to see on this page? How do you think it should work? Do you expect to have to sign in to use it?

Allow participant to fill in page. Probe as necessary using participant’s lists of tasks and content, as well as the pre-determined list of tasks.

Design review

Let’s talk about the page you’ve created, to make sure I understand what you’ve represented.

Review design, and ask for explanations as needed.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Participatory Design

Paul Bryan — Sat, 21 Nov 2009 16:52:38 +0000

Users are not designers, but design without user input frequently misses the mark and suffers from poor usability and lack of adoption by users. Therefore, designers and engineers should give users a role in designing new or improved systems. One way to do that is to ask participants in research sessions to draw their concept of the ideal web page or application screen. You shouldn’t feed them answers that they then capture and write down, because that defeats the purpose. You can, however, ask them how they expect to do certain tasks they’ve already said they want to do on the site. In this way, you are prompting their recall without giving them biased or predetermined answers.

Conducting participatory design exercises is much easier than getting stakeholders to agree to the sessions in the first place. In participatory design exercises I’ve conducted, results showed a surprising consistency. Using the results of other research exercises I’ve described previously in this blog, I was able to pull the participants’ concepts together into a coherent screen design that emphasized the consensus elements. This doesn’t mean that we took the participants’ design consensus as our final solution, but it was compelling to the project sponsors, and our resulting design had substantial anchors to customer mental models that made it difficult for non-substantiated design “arguments” to pull us off track.

In the next post, I’ll give an example script for a participatory design exercise for a resort user experience.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Capturing the Ideal Experience, part 2

Paul Bryan — Fri, 13 Nov 2009 11:09:34 +0000

The last post talked about capturing a customer’s ideal experience. This post gives a brief example of how to do that.

Objective: Explore the ideal guest information system experience for resort visitors with young children who spend an average of $200 per day at the resort beyond room fees. Determine gaps between the actual and ideal. Prioritize the gaps based on cost to implement and business value.

Script:
“I would like to ask you about the perfect mobile guest information system. You mentioned earlier that some of the features you would like to see incorporated into our guest information system include…”

“But now I’d like to take it a step further and discuss the ideal system. Don’t be concerned about the impossibility of it, or the cost. Let me worry about that. For now I’d like to ask you to think with me about the best system you can imagine. Think back upon your stay so far at the resort. When would you have liked to pull out a mobile device and use it to get information or order something, regardless of your location.”

Allow participant to describe scenarios. If participant needs help, probe gently first with locations, such as, “What about when you were at the pool? What about when you were in your room?…” Then probe with features that are currently under consideration: “Have you ever considered what it would be like if you could request dinner recommendations and make reservations no matter where you were at the resort?”

This is different from blue sky thinking or brainstorming. This exercise elicits the normal task process, but without constraints of what currently exists. There may be a simple solution for providing what the customer is asking for that doesn’t break the scope bank.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Capturing the Ideal Experience

Paul Bryan — Thu, 12 Nov 2009 00:21:16 +0000

For design projects that focus more on innovation than improvement, one helpful interview technique is to ask customers about the ideal experience. This exercise is much more specific than simply brainstorming a solution with customers. The ideal experience interview module helps uncover different facets of the participant’s mental model of the activity apart from the experiences they’ve had to date. The ideal should be constrained to tasks within the activity domain for which you are conducting the research. Ask participants to go into some detail about their ideal process interactions, step by step. It’s unlikely that you will be able to accommodate the blue sky thinking of your participants, but you may be able to get closer than you think using available technology.

In cases where there is consensus among participants on ideal interactions and experiences, you should create an interaction model that represents the compass and an overarching design goal to shoot for. Steve Jobs presented some very specific goals to his engineering team that designed the Ipod. It seemed impossible to some on the team that they could meet the goal, but after a long and arduous effort, they did reach it: Your music collection in your pocket. This vision was not contiguous with existing solutions. For innovative solutions, companies can’t rely on incremental improvements that result from assessment-focused research methods such as A/B testing and web analytics.

Objectives of the Ideal Experience module:

Understand each participant’s view of the ideal web site in your field
Obtain a “best case” experience that is generalized from all the participant interviews
Understand what customers ultimately want, and the intermediate steps that could be taken to ultimately reach that point

Participatory exercises can be used to determine a participant’s ideal experience (the ideal design is in the next module). For example, participants can be asked to describe their ideal concept of a web shopping experience or application, or the ideal store section. A single customer’s view of the ideal experience will not necessarily be significant on its own, unless a customer happens to come up with a brilliant idea for a future design. However, when combined with the results of other similar exercises with other participants, a model may emerges of a future innovative design concept that would meet the needs of many users.

If the participatory exercises result in scenarios or descriptions of experiences that are fundamentally different, then the research team needs to ponder what this result means for the future design. It could mean that a personalized experience is required, since everyone’s ideal solution was different. Or it could mean that the solution is not obvious to users, and must be driven by the design team’s experience and skills rather than customer-initiated design ideas.

Components found within the ideal experience or design should be tabulated across participants to understand trends, although quantitative methods are necessary before the results can be generalized to a population of users.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Customer Interviews: Task Analysis & Mode of Interaction

Paul Bryan — Tue, 10 Nov 2009 13:51:05 +0000

Customers access business web sites to achieve a goal or specific purpose. They are rarely there just to look around, unless they are looking for a job and want to understand the company better. Customer goals can often be grouped into distinct modes, such as reading content, finding products or documents, purchasing products or services, learning new processes or procedures, etc. Understanding these modes is important when designing the interactive space, to enable customers to easily enter a mode that clearly facilitates the activity and presents options related to that mode in a consistent way.

For example, in a resort web site or kiosk, guests may be looking for leisure activities, finding products they forgot, reading about the history of the area, etc. Each of these modes should be simple and straightforward to find in terms of access points, make it easy to achieve the most common and most valuable goals, and represent the task using design components that appear especially suited to the activity (i.e. high affordance). The modes should not be cluttered with lots of options that are unrelated to the mode customers have indicated they want to work in. Marketers often want to surround and interject this experience with lots of selling options, but many times this is interpreted by users as visual noise that damages the perception of the experience and isn’t effective. Looking at the analytics for such off-task design elements, I’ve nearly always found clicktrhroughs to be near zero. If they have to be there, make sure they are not obstructing progress in the primary activity. Billboards are okay, but on the side of the road, not in the middle of the road. And spaced apart so that the visual signal to noise ratio is at a reasonable level.

Linked In: http://www.linkedin.com/in/uxexperts