WHAT WE DO
Dial testing incorporates the use of subtle technological instruments to measure participants’ perceptional reaction to a stimulus. Dial testing affords real-time insight into how a person or group is feeling about a specific stimulus, with up-to-the-second specificity. Through the use of wired or wireless hand-held devises, researchers can monitor emotional responses to a stimulus in its entirety, as well as to focused elements of the stimulus. This is an ideal way to understand the perceptional response to media, as well as messaging campaigns.
“Nonverbal” and “semi-anonymous” are terms that are often used to describe the means by which participants give their candid, honest, and uninhibited feedback to stimuli. Participants give their feedback by turning a nob that gauges their response on a particular scale. This is the primary means to monitor their positive or negative reaction or response to what they are seeing or hearing. During this process and line graph is immediately generated, demonstrating the average of the responses of the participants within the group to display the mean or average to the stimulus. This happens in real time and is pinpointed against the stimulus, so the exact cause of perceptional spikes, whether positive or negative, can be tracked.
Focus groups are probably the best known, and certainly one of the oldest, qualitative research methodologies. Focus groups involve bringing a carefully-recruited group of targeted participants who represent key market segments together for a discussion. Focus groups are always led by a trained moderator who employs a variety of techniques to manage the group dynamic, ensure active participation from all members, and effectively probe to meet research objectives. These groups are typically comprised of anywhere from 4 to 12 participants depending on the topic of discussion and key participant demographics, and can be conducted in person or online.
Focus groups are ideal for understanding the behaviors and motivations of key market segments, concept testing and ideation, understanding brand perception, and testing marketing messages and advertising. In-person focus groups afford a skilled moderator more opportunity to foster meaningful discussion and to use their qualitative research expertise to read and analyze non-verbal communication. Online focus groups allow for participation without constraints of geography, both for respondents and clients. In online focus groups, while there is less opportunity for understanding non-verbal cues and fostering discussion if it lags, there is also less of a chance of “group-think” or the discussion being taken over by strong personalities. In addition, since respondents participate from the comfort and security of their own homes, they are often less inhibited in their responses, especially regarding sensitive topics.
Focus groups are excellent ways to understand the motivations and perceptions of your consumers. However, they are not appropriate for observing true consumer behaviors. In other words, focus groups can help you understand what your users think that they want, but not what they actually do. IDIs, usability testing, ethnography, immersive/longitudinal research and other methodologies can and should be used in conjunction with focus groups to understand true user behaviors. If a research question arises because of unexpectedly low utilization of web or multimedia technology, low customer conversion, or other questions involving the optimization of existing products and services, consider other methodologies instead of, or in addition to, focus groups.
As the number of active Internet users has continued to grow over the last 10 years, it’s become increasingly important to find ways to utilize the Internet as a means of efficiently contacting and engaging potential research participants and consumers. Some major backbone providers estimate that over the last two years alone the average growth in unique IP addresses used is roughly 5% per quarter. And while the United States and China account for nearly 40% of the observed IP addresses reported, users in Japan, Germany, France, the UK, South Korea, Canada, Spain, and Brazil also make up large sectors of the online world. While the highest overall numbers of IP addresses come from the US and China – Norway, Finland and Sweden top the charts as far as per capita Internet penetration is concerned and have done so for several years running.
This continually expanding connectivity gives researchers in the US an ideal (and cost-effective) method of reaching across vast distances to gain consumer insights in multiple countries and cultures. Below are just some of the types of research that have become highly translatable to online venues.
Online panels are groups of users who have a vested interest in your product or company and who want to have their opinions heard by the company. Users in these panels typically get involved out of brand loyalty first and foremost. Panels can be used for regular directional questions, opinion surveys, and other “quick answer” needs where a full study would not be feasible. Panels can be kept engaged for long periods of time and only “activated” as needed through email, through a message board system, and for various quick turn-around study pieces such as online focus groups and surveys. Panels are cost-effective in the long term and often have the added benefit of making users feel appreciated and that their opinions are valued.
Online focus groups
Like traditional focus groups, an online focus group gathers together a group of consumers in one place where a moderator walks them through a discussion of a product, a site, or a concept. Online focus groups, however, have the added benefit of not being tied to a geographical location when choosing participants. Not only does this remove the need for facility costs, but participants will often accept a lower incentive because they are not inconvenienced in any way with travel time, traffic, or the potential discomfort of talking in a group setting. Online focus groups allow both the moderator and the clients to observe and participate in the process from wherever they happen to be. The groups function very similarly to a chat room setting and allow for the same level of conversational exchange that happens in a physical focus group.
Intercept recruiting basically means displaying an invitation on the user’s screen as they visit a given site. This is an extremely cost-effective way to recruit current users of the site and has a high success rate. For more general recruiting needs or for extremely quick recruiting turnarounds, intercept recruiting can often be ideal. One drawback of intercept recruiting is a potentially low signal-to-noise ratio if the recruiting needs are very narrow or specific. Recruiting directly off a website is ideal, however, for a “snapshot” of actual users using a site or product at any given time.
A Live IDI uses the intercept recruiting model to pull users from a live site and immediately interview them about the site they were accessing, their opinions, and what they were there on the site to do. The advantage to doing a live IDI is to capture user information “in the moment”, just like you would get from field research – but with the immediacy of a one on one session.
Live IDIs are extremely useful when a company is trying to better understand the primary reason users are accessing their site in order to help define themselves and the major user paths on their site. This can take the form of real-time chat, a phone call, or even live web conferencing.
Online surveys have been around for a long time but continue to get more and more sophisticated. Innovations such as including surveys in intercepts and expansive survey pieces with rich reporting output have continued to make online surveying a means of producing in-depth quantitative feedback. Online surveys are easily combined with Intercept Recruiting, Online Focus Groups, and Field Studies to develop rich, multi-disciplinary studies that address concerns and questions from both a quantitative and qualitative angle.
Field studies See: Field Research
Consumer research has come a long way from the days when the only options were focus groups and sterile, time-on-task style usability testing. Widespread access to high-speed internet and more powerful computers, as well as social media, sophisticated mobile devices, and new tools and technologies have all contributed to a research world unconstrained by geography, time zone, or even the walls of a facility. Not only have the ways we can talk to consumers broadened, but so has what we can find out from them. All of these advances herald an era for research, where those with the skills and experience a virtually limitless in the actionable insights they can provide.
The impact on usability research has been particularly dramatic. New technologies allow researchers to conduct usability sessions remotely, coupling an in-depth qualitative conversation with visibility of the participants’ screen. In other words, we’ve moved from having to sit behind participants in an unnatural lab environment to being able to talk to them where they live, regardless of the place and time, and literally see exactly what they are seeing and doing on their screen. While the ability to capture behavior in the natural environment and without the constraints of time and travel is priceless, it also allows more of the research budget and research team’s time to be allocated to talking to consumers rather than traveling.
Focus groups also have benefited from the introduction of new tools and technologies. Online focus groups can be moderated in real time, allowing participants to be recruited from across the country, or even across the globe, and affording the researcher the opportunity to have a conversation with each and every participant unfettered by group dynamics and pressures. In addition, focus groups have expanded to include a moderated dialog that can occur on the participants’ schedule and at their convenience. Whether these conversations occur in real time or through the use of multimedia technologies that capture everything from words to facial expressions, the richness of the feedback and insights is invaluable.
Web analytics tools have long been employed as a measure of website success, but in the hands of a skilled consumer researcher they can be manipulated to provide amazing insight into the “why” of web traffic, not just the “what.” From the early stages of managing the implementation and tagging strategy, to the more sophisticated application of session-level analytics and the incorporation of analytics technology into field and longitudinal research, CIG is unique in our ability to get a thorough, actionable understanding of consumer behavior from analytics tools.
The proliferation of social media also presents a host of new platforms for research. Research communities can run the gamut from simple, cost-conscious platforms to customized, feature-rich applications, but regardless of their implementation, in the hands of an experienced consumer researcher whose skills have been developed through in-person interactions, they can become a salutary tool for keeping the proverbial “finger-on-the-pulse” of consumers, as well as a time-and-cost effective forum for conducting ad-hoc research.
In Depth Interviews
IDIs, or In-Depth Interviews, are one-on-one conversations with participants meant to glean insight not only into their perceptional preferences, but also into what truly drives their consumer behavior. The benefit of a one-on-one IDI is that a skilled moderator can adapt their style of questioning to get the most input from each individual consumer, unfettered by biases such as “group think” that must be managed in group conversations like focus groups and mini-groups.
IDIs are most often incorporated into usability/user experience research, yielding information about what users say they do and want and what their true behavior is, and often exposing discrepancies between the two. Like in focus groups, in an IDI, the moderator will draw out information such as preferences, likelihood to use an online or offline product, interest in proposed services, features, and functionality, etc. However, in an IDI, this information is coupled with observation of true interaction with a stimulus, whether an existing product, website, or interactive technology, an early version of a concept or wireframe, or competitive product.
The goal is to gather insights on optimal introduction of key information, value drivers, opportunities and unmet needs, overall likes and dislikes and general perceptional feedback in addition to a tactical user experience overview.
IN DEPTH INTERVIEWS
Eye tracking is the process of recording a user’s eye movements as they view your testing stimulus, be it a web page, a video presentation, or a real-life 3D environment. The eye tracking hardware typically consists of a monitor or other standalone device containing one or more cameras that watch the user’s pupils and general eye movements. Software then records this information, and can later process and display it for you in various ways. This information can be invaluable in helping determine what users actually saw during the test vs. what they think they saw.
Data from an eye tracking test is typically represented in one of several ways: heat maps, opacity (or “focus”) maps, and gaze plots. A heat map places colored spots over the areas of the stimulus (web page, video, etc.) that received the most attention from the users during testing. The end result is similar to a weather map displaying a storm, with the central and most concentrated areas in red, transitioning to yellow for medium amounts of attention, and finally fading to green and/or blue around the edges where there was less activity. This helps identify areas that were heavily used or were of the most interest to users.
An opacity map is almost the reverse of a heat map, where the entire screen is blacked out, and only the areas that got attention from users are shown. This can be extremely helpful in learning not only what the users saw, but more importantly, what they didn’t see.
A gaze plot is exactly what it sounds like – a visualization of the path a user’s eyes followed from one point to another. A line is drawn across the stimulus to represent the path, and circles of varying sizes appear as “fixations” – places where the user stopped and looked for a significant amount of time, with larger circles representing longer fixations. Isolating a typical user’s gaze plot for a specific task can help to show how easy or difficult the task was, as well as point out any stimulus elements that were distracting for the user along the way.
Depending on the study structure and type of stimulus, additional data can be gathered using areas of interest (AOIs). For example, a banner ad on a website could be defined as an AOI, and the eye tracking software can later calculate things like the amount of time users spent looking at the AOI, or how much attention it got relative to other parts of the page.
One important thing to keep in mind is that, while eye tracking can help you learn things on its own, it’s much more powerful combined with another methodology, such as a post-stimulus interview to gauge the users’ retention of what they saw during the test. This way you can compare what the users physically saw and what they actually absorbed, and adjust your designs accordingly.
Longitudinal research can take on many forms, but in general it means touching base with users over the lifespan of a product or site’s development at key milestones. The goal of longitudinal research is to show what impact improvements, progress, and changes have on the end user’s experience.
A longitudinal research approach can be taken with any number of research types – usability research, focus groups, IDIs, and field studies to name a few – but the concept remains the same across all types of research. Knowing how users react to and interact with a product or site early on in development is key in understanding how to best meet consumer needs. Once a baseline is set and development continues, additional touch points are made during the course of development to keep changes on track and to ensure that the product or site is achieving its goals before it is completely built and can still be easily altered.
Knowing during development if the product or site is meeting consumer needs and desires can help prevent costly redesign initiatives. In addition, the information gleaned from one development cycle can sometimes be applied to upcoming projects, improving your own processes going forward.
Having multiple check points with users through any kind of usability or perceptional research method helps keep an eye on ongoing usability concerns and on potential changes in user perceptions, including shifts in brand perception and engagement levels.
Longitudinal research can be crucial for brands reinventing themselves as well as for long-established brands that need to reaffirm their path. This type of research may be conducted over the course of several months or even several years and incorporate any number of research methodologies.
Field Studies are in-home or location based studies that show how consumers adapt to a product over time in the context of “real world” variables. Your site or product might work flawlessly on your lab computers, but what about in a real environment on a home computer that doesn’t come with support staff or guaranteed maintenance? Real environment use helps expose a wide array of information about a product or site that cannot be tested out in a simulated environment like a lab or focus group facility.
In addition to testing out alpha and beta products in real environments, the true power of the field study is gaining insights into consumer perceptions, attitudes, beliefs, and behaviors, in addition to uncovering basic usability issues. Because these types of studies allow for usage of a site or product over a period of time they are ideally suited to help gauge learning curve and identify features that are “sticky” or drive value and high satisfaction.
Field studies are very similar in some ways to online panels (see: Online Research). Panels, however, tend to be simpler polls of user’s impressions and a way to reach out to a consumer base and make their opinions heard. While field studies allow us to understand not just how users feel about a site but how it works or doesn’t work for them, how it can be better, and what motivates users to dig deeper into key functionality offered through the site or product.
Field studies are typically conducted for at least 10 days, but can last for several months. We typically recommend groups of at least 20 users, but multiple segments, multiple user types, and hundreds of users can be tested at one time. One of the most valuable aspects of field studies is their extreme flexibility, mixing survey work, journaling exercises, traditional tasking, and “show and tell” exercises for straightforward opinion gathering to gain deep understanding of consumer beliefs and motivations.
Developing products for mobile devices comes with its own unique set of challenges. Screens are small, and text entry can be cumbersome, be it via a phone keypad, a full or partial set of slide-out keys, or an on-screen “soft” keyboard. Each device may come with different hardware, software, capabilities, and form factors. Mobile devices are also used by a wide array of people, many of whom have average or even poor technical skills, making it all the more important that your mobile website or application (or “app”) be as intuitive as possible.
As a result, you may need to develop a different version of your product for each intended device or platform – one size rarely fits all. Also, if your target device supports it, an app is nearly always preferable to a website, and will allow for an interface that is completely tailored to your product’s purpose rather than one that is forced into a browser model. This last point is especially true for users of such app-driven platforms as iPhone/iPad OS and Android, who are not only used to running apps instead of websites, but strongly prefer this method as well.
All of these considerations make user testing for mobile products absolutely crucial. Not only does everyone have a different phone (MP3 player, netbook, etc.), but people use their devices in their own way based on their needs and preferences. Collecting feedback from actual users can provide tremendous insight into their behaviors and interests, and will help guarantee that your mobile product meets with their wants, needs, and expectations. By utilizing this information, you can also feel secure in knowing that your product functions ideally for the target device and its users.
The form was submitted successfully.