High quality research needs to employ carefully phrased and expressed questions that give participants the opportunity to say what they really think, without bias or misinterpretation.  Minimising bias in research is essential to get results that are as accurate, objective and actionable as possible.  Biases can take many forms and can be introduced at any stage of the process from initial design, through recruitment, right down to reporting the findings.

Implementing innovative research techniques or carrying out ‘agile’ research does not mean that potential bias becomes less important.  Quite the opposite in fact.  The biggest bias risk of all is creating a research paradigm that leads participants to the answers you (and/or the client) want(s) to hear.

Take for example an actual user experience research exercise in which a co-design workshop was used to explore user views and attitudes towards driverless cars.  Participants were invited to attend the three-hour workshop at the Coventry Transport Museum.  Whilst this might be a prefect venue from the perspective of facilities, the subject of the research was related to that of the Museum.  Inevitably, this could introduce a degree of participant conditioning and affect their attitudes and behaviour.  A neutral value reduces distractions and locational biases.

A more significant bias was introduced when participants were asked to place a cut-out of themselves on an ungraded physical positive/negative scale under the question; “Do you think driverless cars are a good idea?”

Whilst the idea of asking participant to engage with a physical scale may be appropriate, the problem is that the question is skewed to the presumption that driverless cars are a positive development and pushes respondents to think about the question framed in those terms.

Not only was the question leading, it was open to wide interpretation also.  What exactly does “a good idea” mean?  A conceptual “idea” as opposed to real world reality – the “idea” is good, but it’ll never work?  “Good” relative to what?  And so on.  Participants each have their own norms and values and careful questioning is essential to help to align these to provide meaningful results.

This question was asked twice, once when respondents joined the workshop and again at the end of the workshop, after they had had the ‘advantages’ of driverless cars explained to them, to see to what extent there was change to placement on the scale.  So, a risk of acquiescence bias was introduced during the workshop.

Even the photograph depicting the question results showed only the “A very good idea” end of the scale.  So, additional reporting bias added distortion to the cumulative effects already built-in.  A classic example of confirmational bias.

If you layer this with the client’s identity being fairly obvious to participants, thereby introducing bias around attitudes towards the sponsoring organisation, the problems just keep multiplying.

You may be wondering why this matters?  It matters because there are so many types of potential bias that can mislead in all sorts of directions.  In just this one example the more research participants were conditioned, the more ways a question could be interpreted, the more difficult it could become to analyse and interpret the results objectively.  When research findings become a matter of subjective interpretation of biased data, they become little more than a lamp-post used by someone who has had one-too-many on a night out; a form of support, rather than of illumination.  Worse still, the findings can end up making no sense.

No research methodology can ever be 100% bias-free.  But to achieve high-quality results means asking the right participants, the right questions, at the right time, in the right place, in the right way, whilst identifying potential and actual sources of bias at every stage and eliminating, reducing or accounting for these in every way possible.  And that includes the researcher’s own biases.

Online communities are a popular way for organisations to gather information and insights and to better understand users’ views of their existing and potential products and services.

These usually by-invitation-only ‘Voice of the Consumer’ moderated communities also enable users share opinions, connecting with other users, exchange advice and exchange with the brand. Naturally, promoting participation is vitally important to organisations that wish to engage with users if such a community is to thrive and provide valuable feedback.

This begs the question as to why consumers would want to become part of such an online community if invited so to do and what resulting biases might arise?  Similar attitudes can derive from different motivations.  Which might cause users to join and actively participate?

Psychological ‘Social Identity Theory’ can help us here.  Online communities play a social function for their members with like-minded or similarly interested individuals.  It is a silo of sorts.  They provide participants with a degree of enhanced self-worth, because they perceived they are being listened to and have some degree influence over organisation’s decisions – otherwise they wouldn’t have been chosen to join.

So, why join?  It might be to;

• Find out, or learn more about, or in some way assist the organisation, its products or services
• Gain financial or other incentives in return for participation
• Provide a virtual soap box for the individual to express their views
• Support their self-identity as invested in the brand, products and/or services

The online community itself can provide participants with a form of social self-identification by differentiation between members (us) with non-members (them).  This emotional attachment is akin to being in a tribe.  The strength of motivations for the individual can influence the degree of engagement in such communities.

This is also a two-way street as, for example, the organisation can communicate privileged information to the members of the community (us) that only they know and not to others (them).  A real conversation is a vital element of inclusion.

Research suggests that social and self-identity-supporting user motivations are particularly important in driving participation on online communities.  Great.  But there’s a problem.  Membership becomes part of who users are and consequently they become more subjective.  They may identify too closely with the brand, and over criticise competitors to support their self-perceived status in the community.

Also, if participants have significant time and/or resources invested in the brand, they may defend their purchase decision at all costs, especially in the face of objective criticism.  This may be overlain with the tendency to post-rationalise positive aspects of our purchase to justify the user’s choice to themselves and others engaging in the online community.  Also, the participant may express views online in a way they would not if face-to-face or over the telephone.

These biases are not inevitable, nor do they necessary apply in all circumstances or to all participants.  Indeed, they can become inverted. But it is as well be cognisant of the possibility of bias and to attempt to design-out opportunities to express such biases as far as possible – and to recognise them when they do occur.

Good research avoids supporting the pre-judged view but illuminates the scene instead.

The realm of CX and UX is a constantly evolving paradigm with components morphing and interchanging as exponents and experts rapidly iterate what each actually is.  UX has expanded and evolved from being a digital extension for HCI and industrial design into the broader world of CX.

It was supposed that UX was a subset of CX, with the latter more of the physical world.  But is it that simple anymore?

We know that an organisation can no longer simply offering a product or service and expect to compete or cut through.  Convenience is already king.  In order to compete with an unique offering, a transformative experience must be on offer.  In that way you can truly differential your brand, de-commoditise your offering, add value and increase market share, etc.  No longer do you risk being a ‘me-too’ purveyor of goods and services.  Instead, the experience you offer your customers becomes an unique standard bearer for your brand.

UX is now being taught with a major Service Design component, itself moving away from a purely digital domain.  Video prototyping is becoming mainstream.

Customers expect an integrated offering that cuts across delivery channels and devices, with a streamlined, seamless experience that is both fast and provides the ultimate in convenience.

So, an overarching vision is required to encompass the whole experience paradigm.  This is about more than just touchpoints.  It needs to bring customers/users and employees together in an holistic experience with an overarching brand/organisational strategy that delivers solutions perceived by individuals as personal, whilst delivered practically in a way that matches the norms and values of all those involved.  In this environment is it meaningful to differential between customer and users in the context of experience?

Whilst this may be the current direction of travel, one thing that we can be sure of is that this will shift and evolve in ways not yet clear.

Howsoever this develops, user research will still be at its core, not just strategically, but also feeding back to the CX/UX vision, iteratively.   Customers and users neither differentiate between each other, nor between experiences in silos.  For them CX = UX = the seamless whole.  Well designed, high-quality user research can bridge any conceptual gaps and deliver synergistic insights to unify the delivery of solutions.

Ethnography remains something of a catch-all description for a wide range of qualitative research techniques that aims to get closer to understanding the user in context of product/service use – activities, actions, norms and routines.

There remains a perception amongst many that this makes ethnographic research akin to nailing a fruit jelly to a plank of wood.  All you end up with is a mess of something…

But to get the most out of ethnography it needs to be carefully designed and moderated specifically for the issue being investigated – not left to chance or random actions – considering the type of product or service, the sector and the relevant level of user engagement.  The researcher needs to understand not only the extent of the issue to be investigated, but also why the client needs to know about the issue in order to enable them to get closer to their customers, which, after all, is the whole point of ethnographic research.  But it is qualitative, and this is another reason for its perceived shortcomings.

Organisations are increasingly run by numbers.  Data is big.  Accountants call the shots.  Of course, quantitative data is important, but it is so often seen as the only definitive measurement because it produces numbers and numbers = facts…don’t they?  Well, yes and no.  Certainly, what is happening can be described by quantitative data, but if these data provide an analysis of the direction of travel (usually a far from linear progression anyway), how do you know why this is happening?  Without a good grasp of “why?”, how can a solution be created?  This requires deeper qualitative understanding.

Like any qualitative data, ethnographic data needs to be analysed and interpreted in a methodical, structured, objective way.  The framework of analysis needs to be determined in close collaboration with client stakeholders taking account of the context, relevant and salient artefacts, and the environment of use.  Key insights need to be prioritised and ranked in importance and impact.

And as with any form of research, it’s worth doing only if actionable insights can be derived from the findings.  If well designed in collaboration with key stakeholders, ethnographic research can shine a Super Trooper on behaviour, attitudes and motivations that may not have been recognised or properly understood though other methodologies.  It can add vital details and perspective from users as they experience the product or service in the context of actual use.

That can make all the difference when revising, enhancing or developing new products and services, focussing resources where they will have the greatest impact, rather than just picking low-hanging fruit.  The results of this targeted effort will show in the quantitative data and that’ll make everyone happy.

Including customers in the process of (re)designing (new) products and services is central to user experience design and what better way than to harness customer voices and perceptions to drive product and service improvement and development.

But how best to achieve this?  It’s all about using the right listening techniques in the right context.

A whole range of widely-practiced research techniques can capture customer voices, both active and passive.  Capturing satisfaction feedback and scores and utilising panels of many colours is the stock in trade of the VoTC.  But it is the passive voice that is often overlooked.  The greatest advocates and detractors typically have the loudest voices, not least on social media.  Voices must not be left to be self-selecting – softer voices need to be paid attention to also.

By engaging with users collaboratively in panels, workshops and through observational techniques very valuable, meaningful insights can be drawn out, analysed and interpreted and then communicated most appropriately to achieve actionable outcomes.

It’s vital to avoid findings overload, and therefore, essential to prioritise and rank actionable insights that illuminate tactical and strategic imperatives that support the corporate, product and service vision.

Feedback should be tested iteratively, not least to ensure that the volume of feedback is not inadvertently up-scaled to be representative of “everyone”, in a variety of contexts over time and across places.

Empathetic user engagement, feedback and improving customer satisfaction can assume an iteratively virtuous circle of success for any organisation.  Good quality research is the foundation of this success.

Great researchers know that it’s not enough to simply understand user’s current behaviour – what they are doing.  You need to understand how the user’s current experience is shaped by the context of use, their task and experience goals, their frustrations and their emotional pain-points.

To achieve this a researcher must be able to empathise with the participant – to understand not only what participants are doing, but also what they are thinking and feeling…objectivity and impartially.

Prejudices, preconceptions, norm and values must be discarded.  To properly understand their motivations and attitudes, researchers must see the world from the user’s perspective; to climb out of their own and their client’s echo chamber and ask the right questions of the right people in the right way – appropriate to the context.

This is not about gathering information on requirements, it’s about solving user problems by placing yourself in their reality and seeing the world from their perspective.  These users may not have the faintest idea what solutions are possible, so don’t ask them what they want or ask how they want to get there.  Instead understand where they want to get to.  It’s the job of the researcher to use creative research techniques to extract insights and lay out the path of opportunities that can lead to prototype solutions.

If there’s one thing critical to achieve this, it’s empathy.

This holds wherever creativity is required.  As Susan Sarandon observed; “When you start to develop your powers of empathy and imagination, the whole world opens up to you.”

It has become almost a cliché to recall that Steve Jobs was totally against market research and never needed to do any.  His quotes on the subject have turned into a view widely held by many that no consumer research is necessary.

In fact, Jobs’ observations were about not asking customers what they want because they don’t know until you show it to them.  And this is the nub of the issue.  Good research is about asking the right questions of the right people in the right way to understand their behaviours, attitudes, habits, values, norms, beliefs and routines.  In other words, to understand their lives and what makes them tick to identify their frustrations, pain-points and un-met needs.

Consider the recent launches of very expensive, full-frame, mirrorless cameras by both Canon and Nikon that featured just one card slot.  This caused both consternation from those who feel two card slots are a minimum requirement in case of card failure (especially at the price points concerned) and ridicule of that consternation by others insisting they have never had a card fail and therefore it never happens…to anyone.

This argument in the photographic community has blown up into a huge issue with its own momentum.  There has even been a meme of someone taping two film canisters to the outside of a film camera trying to imply it was not a problem in the days of film.  Well, it was, which was why in the days of film pros always used multiple film camera bodies at events, in case of problems.

Interestingly, differing perspectives on the slots issue seems to be driven by the modern phenomenon of the ‘fanboi’ or ‘fangirl’; the allegiance-driven psychology of defending the ecosystem that’s been bought in to (often literally) and dismissing any objective analysis or counter position that might conflict with the fan’s subjective stance.  Some companies, in some industries, may depend on such unyielding loyalty to ameliorate shortcomings in their products and services, but it is likely to lead to a long-term decline in fortunes.

So, what could UX research have done to eliminate such user frustrations and pain-points?  Fundamentally, it comes down to posing the right questions of the right people in the right way.

Who ultimately decided that one card slot was “good enough” and how was their decision informed?  How deeply were users engaged in the design process?  Critically, what iterative testing processes and prototyping was undertaken and with whom?

Was the research unbiased or was designed to lead to a pre-determined result or, worse still, designed to validate a position already taken? Or was any research undertaken at all during product development?  Too often research stops at the un-met need.  Good user research takes the opportunities derived from insights and prototypes and tests, tests and tests again…with users.

It’s a myth that Apple doesn’t do research.  It does and always has.  It succeeds by focussing on understanding the user in the round, where they suffer frustrations, inconveniences, pain-points in their lives and uses these insights to create and inform opportunities that could appeal to archetypal users exploiting new products and services that are tested and improved through iterative evaluation and prototyping.

Research is not a bolt-on extra, it needs to be integral to the design process.  Well-designed research delivers well-designed products and services.