UX Strategy and Planning » Depth interviews

In-Depth Customer Interviews: Conducting the Interview

Paul Bryan — Thu, 31 Dec 2009 19:31:08 +0000

Once participants arrive, the interviews are mainly a matter of ensuring that the data you need to capture as outlined in the interview protocol is captured. After the first couple of interviews, the research lead will develop a sense of topics that are most likely to result in data that will be useful to innovative design work in the future. The research team should debrief after each interview to discuss whether the research protocol should be modified for any reason. Some questions or exercises don’t work as well as planned in terms of data capture and participation, and need to be amended or omitted. In my experience, the most changes are made either after the first session, the first day, or the first round of interviews.

As time goes by, the interview moderator may modify the interview for each participant to get the most value out of the interview. Some people feel that this is problematic because modifications to the interview precludes comparisons of data across participants for a given set of variables. My feeling is that assigning such weight and rigor to variables that are being measured qualitatively is to misunderstand the nature of the data being collected in customer interviews. I feel that it is best to get the most value out of each interview and to characterize each participant as clearly as possible, and then to compare variables across participants to the degree that it makes sense for the type of data being collected. If a participant is leading me down an interesting trail of discovery toward a potential opportunity for design innovation, then I will follow that trail in lieu of other sections of the interview that I feel will not be as fruitful. My sense of where to take the interview has developed over many years of interviewing customers for the purpose of creating innovative web sites.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Location Prep

Paul Bryan — Fri, 18 Dec 2009 16:38:14 +0000

In addition to the materials required to conduct the research exercises and to capture data, you will need to organize food and drink according to the standard practices of the organization sponsoring the research, and as allowed by the project budget. Sometimes the food and drink offering is elaborate, with a full lunch or dinner provided as a perk to the participants, researchers, and/or project sponsors. In other cases, it is very simple: “The snack machines are over there.” At a minimum, bottled water should be provided for the people who will be talking the most: the participant and the moderator. I prefer a middle of the road set up, because lots of food tends to cause unwanted distractions, and participants fumble with plates and trying to eat while being observed. I prefer a selection of soft drinks and juices, with a tray of assorted snacks like trail mix, power bars, or other easy to eat items.
Research days are often 10 to 12 hours long, and so a lunch tray for the research team is always welcome. I prefer to order lunch and have it delivered to the area where the team is located, rather than going out to lunch. Walking to lunch spots with the project sponsors and team is a good opportunity to bond, but in some cases it leads to many opportunities for getting off-track for the afternoon sessions. Project sponsors or wayward team members start window shopping or stopping for little errands, and then we’re late for the first afternoon session. Being cooped up in the same small room for 12 hours is no joy, but it tends to keep people talking over impressions, which can be very fruitful.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Preparing the Research Location

Paul Bryan — Fri, 11 Dec 2009 14:06:58 +0000

The research team should arrive at the interview location at least an hour before the first interview will take place. This gives you time to prepare the setting and agree on any mechanics of the interviews that are still open. If interviews will take place at different locations throughout the day, then my team usually meets for breakfast to run through the day’s activities and to make sure all materials are ready for use.

If interviews will take place in a central location, then a list of the current day’s interview participants should be printed out and posted in the area where they will take place. This list should also be emailed to team members and stakeholders.

Check with security prior to the day of the sessions to figure out what kind of documentation will be required to bring participants into the research area. Checking ahead with security is relevant in all kinds of research settings, from office buildings to retail locations, to special facilities like airports. This is often overlooked, and can lead to big headaches. At a minimum, security should be given a list of each day’s participants. They usually have some kind of guest badges or passes that can be issued at the check-in desk.

The research protocol you wrote identified the materials that will be required for your customer interviews. The materials identified in the research protocol should be refreshed at the beginning of each day. Copy machines for some reason seem to break more often the day that interviews take place, so we usually make lots of copies prior to the day we will need them. We are especially careful to have extra copies of any paper forms that participants will write on directly, because if we are lacking those, then we will be lacking data we need. Recording devices, media for data and copies, markers, pads, etc., should all be checked at the beginning and end of each research day to avoid unpleasant surprises.

As a point of reference, the following are materials that Usography typically requires for customer interviews:

Copies of research protocol
Release form (permission to use likeness and responses)
Payment received form
Cards with content types, features, concepts
Photos for brand impression analysis, photo elicitation
Grid with topic relevance on one axis, perceived value on the other
Digital camera, recorder, or video camera
Map of physical location where activities or interactions take place
Markers
Blank wireframes for screen sketches

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interview: Evaluate existing design

Paul Bryan — Fri, 27 Nov 2009 13:41:51 +0000

If a site design already exists at the time the research is conducted, then in most cases you will ask customers about that design in the customer interview. The design may exist as either current release of the site that customers are using, or it may be a non-released design in the form of page comps or a prototypeprototype. The case for omitting an interview module that evaluates this design can be made when you are conducting research for a completely new design that will not rely on previous design work. I’ll discuss the case of the customer interview that focuses solely on design evaluation in a more detailed post later.

Objectives of the design evalution module include:

Understand the gaps between the desired design and the actual design
Obtain specific feedback as to which design elements need to be increased or decreased in prominence
Determine components that participants would like to eliminate from the design
Determine components that participants consider missing from the design

Linked In: http://www.linkedin.com/in/uxexperts

In the next post, I’ll include an example protocol and script for a design evaluation exercise.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Customer Interviews: Task Analysis Exercise

Paul Bryan — Mon, 09 Nov 2009 12:06:55 +0000

In previous posts, I discussed a customer interview module for task analysis. It started with capturing the customer’s general list of activities associated with the task. After capturing a customer’s lists of activities related to the task, present to the customer a more comprehensive list of tasks that map to current or future features and content. Creating this list should be part of your preparation for the session, based on a cognitive walkthrough of site paths, and prioritized on the basis of business objectives. For example, one business objective at SeaSide Resorts Unlimited is that people will check the kiosk for local happenings before they stand in line at the concierge desk, because the kiosk has more exhaustive information that can be printed. So, in the task list that will be presented to users, the task of finding local happenings on the kiosk should be included.

Add any tasks that participants mentioned when listing activities that were not present in the task list. Then discuss each task individually, asking participants to list every step they are aware of taking when they complete that task, whether online or offline. Participants may not be willing to think at this detailed level without some prodding and probing. One thing that I’ve found that helps them engage at a more detailed level in the task analysis module is to draw out a process flow or map, and allowing the participant to view it, ponder it, and add extra details or move steps around. When you finish drawing the task flow, ask the participant to read through it one more time carefully to make any final corrections or additions.

The result of this exercise should be a comprehensive, prioritized set of task flows that can be used as an input to user experience design, and as an assessment tool to evaluate design.

Linked In: http://www.linkedin.com/in/uxexperts

Customer Interviews: Task Analysis Exercise 1

Paul Bryan — Sun, 08 Nov 2009 10:35:22 +0000

In the previous post I introduced the task analysis module for in-depth customer interviews. One way to start the task analysis part of the interview is to have the customer create a task list. A task list is a list of activities that participants recall when asked about a given subject matter. For example, in research for a resort web site, the interviewer asks what types of content and functionality the participant currently uses on travel web sites. Although analytics give a more accurate picture of the prevalence of different activities in the system in question, an activity list or index helps characterize the individual being interviewed in terms of usage patterns, preferences, breadth of activities etc. The activity index is useful later in the interview when the research team asks participants to discuss a common activity in detail, noting barriers and opportunities for each.

Some activities in the list may be related to the system being researched, but may not specifically be conducted within the system. For example, participants may be asked to list resorts they have visited, or which areas within a resort they typically use, both of which are broader than asking what tasks they typically undertake using a guest information system. The purpose for capturing these broader activity indexes is to characterize the person being interviewed. The data can help the research team formulate an experience model.

When constructing an activity list, it is helpful to ask participants how they expect to go about the activity using a web site or information device. Their expected task path can be used to support the mental model and current or proposed interaction model.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: The Retailer Selection Set

Paul Bryan — Sun, 25 Oct 2009 12:13:33 +0000

A major step in any purchase process Is the concious or unconscious determination of sources from which to make the purchase. These potential sources are called the consideration set, because they are the business entitites under consideration to fulfill the need, whether it be for shoes or for car service. In some cases, consumers may make a decision regarding brands or sources that is final and they don’t need to form a consideration set. However, even at that point, other providers are trying to find a way to compete with that source and therefore, the concept of consideration set is still a valid construct.

The objectives of the competitive set module are:

Understand how the participant forms a set of possible sources
Understand the criteria used to prioritize and select the source
Understand the degree of loyalty to each source and what led up to it

If your web site or company is not in the consideration set, it is unlikely to win the business. Therefore it is key to understand how customers form the consideration set of competitors, one of which will be selected to provide the product or service. If your interview sample exists solely of customers of your web site, then you will naturally be in the consideration set, unless some event has pushed your site or company out of consideration, which is important to understand. It may be helpful for this module to talk with people who are customers of your competitors but not yet your customers.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Context of Use, part I

Paul Bryan — Wed, 14 Oct 2009 21:40:07 +0000

The context of use module probes all aspects of the situations that lead customers to use a web site for the purpose being researched. The context of use module looks at all factors that influence online behavior, whether online or offline.

Objectives of the context of use module:

Understand the activities that precede, coincide, and follow the use of the web site
Understand role of information in key processes and activities related to subject matter
Understand perceived value of different types of information
Understand the role of information in decision making
Understand what participant considers reasonable or expected in terms of online offering
Determine typical online shopping processes, pain points

Tasks that people complete using a web site do not take place in a vacuum. There are a whole range of factors in the life situation of customers that surround the actual web site interaction. It’s important to understand the context in which a related set of activities occurs, because it gives many clues about the mental model that a user brings to the site, their expectations of how it will be organized and how they should be able to interact with it. Customers find web sites difficult to use when they come to the site with an expectation or mental model of how a it should be organized and it is quite different from their expectation.

Ignoring this real life context of a web site interaction leads to the “bucket of features” approach to web design, where design is simply a matter of taking list of features that someone else has decided are in scope for a particular release and arranging them on the page. Such web sites may be feature rich, but they are not intelligently structured to speak to customers’ needs in a way that engages them, reflecting the nuances of real life situations. They are, in effect, a one way conversation. This would be unacceptable behavior in any retail store on the planet, but happens all the time in web site design.

The context of use module helps researchers understand the context of the decisions that lead to transactions or other conversion events. As a result of this module, you will probably notice behavioral patterns that are directly related to the context. The needs and behaviors that are not supported by the current design represent gaps, or opportunities, for new content and features. Those behaviors need to be supported in the design system in order to provide a satisfying and successful user experience.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Using an agency to recruit participants

Paul Bryan — Tue, 15 Sep 2009 22:48:12 +0000

It is unlikely that a company conducting customer interviews is going to spend time cold-calling the general public to recruit participants. Typically they contact a market research recruiting company. These companies have a database of people in the general public who are willing to participate in research studies, such as a customer interview.

If an outside agency is doing the recruiting, then I usually write the screener in the form of a script. The recruiter calls potential participants and walks them through a series of questions. People are dismissed if they don’t match the criteria. If they do meet the criteria, then they are tallied so that the minimum and maximum number of people specified for each criteria is adhered to.

I have experienced a couple of problems when interviewing participants recruited by outside agencies. The reason that these problems arise is straightforward, but the solution is not. Recruiting companies have a vested interest in finding participants who match the screening questionnaire, while at the same time spending the least amount of time and money on the search. That means that their database of known participants is the first stop for your research sample. This database includes people who they’ve used before for similar projects, who in some cases “do testing” for extra cash. Such people are usually determined to take on whatever persona the researcher seems to want them to have, so that in the end, the data from these sessions is worse than useless, because it is misleading.

The second problem I’ve experienced is related to the first. The research recruiter’s database may include people who are unemployed and who desperately need the incentive money. They give very brief answers, waiting for the session to end so they can collect the money or gift card and disappear.

Of course, there are recruiting companies who are very adept at what they do. They try hard to eliminate the two types of problem participants mentioned above. I don’t hesitate to call them when I’m not able to recruit more directly and precisely from a customer database. But I add a fudge factor to the number of participants, anticipating that at least 10% of the participants will not contribute useful information to the study, and so adjust the total number of recruited participants accordingly.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Using a customer database to find participants

Paul Bryan — Mon, 14 Sep 2009 10:24:31 +0000

One way to ensure a relevant sample of participants is to select them from your customer database. You can establish precise criteria and then query the database for matches. This is my preferred method for recruiting participants for customer interviews, because the sessions tend to be very productive. This isn’t always possible or desirable. In some companies, privacy restrictions make it difficult to contact customers based on purchase history or other interactive behaviors. These companies don’t want their customers to feel that they are being observed while they shop on the web site. Of course, it is questionable whether there are still many customers out there who do not realize their every move on web sites is being tallied and reported.

One case where the customer database is only a partial solution is when you want to understand both customers and non-customers. In this case you can use the customer database to recruit customers, but must rely on other means to obtain the non-customers. One resource for the non-customer participants is discussed in the next section.

Linked In: http://www.linkedin.com/in/uxexperts