{"id":148,"date":"2018-12-04T19:26:36","date_gmt":"2018-12-04T19:26:36","guid":{"rendered":"http:\/\/www.veritay.co.uk\/?p=148"},"modified":"2018-12-04T19:26:36","modified_gmt":"2018-12-04T19:26:36","slug":"hall-of-mirrors-research-a-distortion-or-seeing-what-you-want-to-see","status":"publish","type":"post","link":"https:\/\/www.veritay.co.uk\/?p=148","title":{"rendered":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?"},"content":{"rendered":"

High quality research needs to employ carefully phrased and expressed questions that give participants the opportunity to say what they really think, without bias or misinterpretation.\u00a0 Minimising bias in research is essential to get results that are as accurate, objective and actionable as possible.\u00a0 Biases can take many forms and can be introduced at any stage of the process from initial design, through recruitment, right down to reporting the findings.<\/p>\n

Implementing innovative research techniques or carrying out \u2018agile\u2019 research does not mean that potential bias becomes less important.\u00a0 Quite the opposite in fact.\u00a0 The biggest bias risk of all is creating a research paradigm that leads participants to the answers you (and\/or the client) want(s) to hear.<\/p>\n

Take for example an actual user experience research exercise in which a co-design workshop was used to explore user views and attitudes towards driverless cars.\u00a0 Participants were invited to attend the three-hour workshop at the Coventry Transport Museum.\u00a0 Whilst this might be a prefect venue from the perspective of facilities, the subject of the research was related to that of the Museum.\u00a0 Inevitably, this could introduce a degree of participant conditioning and affect their attitudes and behaviour.\u00a0 A neutral value reduces distractions and locational biases.<\/p>\n

A more significant bias was introduced when participants were asked to place a cut-out of themselves on an ungraded physical positive\/negative scale under the question; \u201cDo you think driverless cars are a good idea<\/em>?\u201d<\/p>\n

\"\"<\/a><\/p>\n

Whilst the idea of asking participant to engage with a physical scale may be appropriate, the problem is that the question is skewed to the presumption that driverless cars are a positive development and pushes respondents to think about the question framed in those terms.<\/p>\n

Not only was the question leading, it was open to wide interpretation also.\u00a0 What exactly does \u201ca good idea<\/em>\u201d mean?\u00a0 A conceptual \u201cidea\u201d as opposed to real world reality \u2013 the \u201cidea\u201d is good, but it\u2019ll never work?\u00a0 \u201cGood\u201d relative to what?\u00a0 And so on.\u00a0 Participants each have their own norms and values and careful questioning is essential to help to align these to provide meaningful results.<\/p>\n

This question was asked twice, once when respondents joined the workshop and again at the end of the workshop, after they had had the \u2018advantages\u2019 of driverless cars explained to them, to see to what extent there was change to placement on the scale.\u00a0 So, a risk of acquiescence bias was introduced during the workshop.<\/p>\n

Even the photograph depicting the question results showed only the \u201cA very good idea<\/em>\u201d end of the scale.\u00a0 So, additional reporting bias added distortion to the cumulative effects already built-in.\u00a0 A classic example of confirmational bias.<\/p>\n

If you layer this with the client\u2019s identity being fairly obvious to participants, thereby introducing bias around attitudes towards the sponsoring organisation, the problems just keep multiplying.<\/p>\n

You may be wondering why this matters?\u00a0 It matters because there are so many types of potential bias that can mislead in all sorts of directions.\u00a0 In just this one example the more research participants were conditioned, the more ways a question could be interpreted, the more difficult it could become to analyse and interpret the results objectively.\u00a0 When research findings become a matter of subjective interpretation of biased data, they become little more than a lamp-post used by someone who has had one-too-many on a night out; a form of support, rather than of illumination.\u00a0 Worse still, the findings can end up making no sense.<\/p>\n

No research methodology can ever be 100% bias-free.\u00a0 But to achieve high-quality results means asking the right participants, the right questions, at the right time, in the right place, in the right way, whilst identifying potential and actual sources of bias at every stage and eliminating, reducing or accounting for these in every way possible.\u00a0 And that includes the researcher\u2019s own biases.<\/p>\n","protected":false},"excerpt":{"rendered":"

High quality research needs to employ carefully phrased and expressed questions that give participants the opportunity to say what they really think, without bias or misinterpretation.\u00a0 Minimising bias in research is essential to get results that are as accurate, objective and actionable as possible.\u00a0 Biases can take many forms and can be introduced at any … Continue reading Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-148","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\nHall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.veritay.co.uk\/?p=148\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay\" \/>\n<meta property=\"og:description\" content=\"High quality research needs to employ carefully phrased and expressed questions that give participants the opportunity to say what they really think, without bias or misinterpretation.\u00a0 Minimising bias in research is essential to get results that are as accurate, objective and actionable as possible.\u00a0 Biases can take many forms and can be introduced at any … Continue reading Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? →\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.veritay.co.uk\/?p=148\" \/>\n<meta property=\"og:site_name\" content=\"Veritay\" \/>\n<meta property=\"article:published_time\" content=\"2018-12-04T19:26:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"657\" \/>\n\t<meta property=\"og:image:height\" content=\"423\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Veritay Webmaster\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Veritay Webmaster\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148\"},\"author\":{\"name\":\"Veritay Webmaster\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/#\\\/schema\\\/person\\\/3194484920f6c197eb7cb0aa53cdf23c\"},\"headline\":\"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?\",\"datePublished\":\"2018-12-04T19:26:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148\"},\"wordCount\":651,\"image\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/www.veritay.co.uk\\\/wp-content\\\/uploads\\\/2018\\\/12\\\/Bias-1-300x193.jpg\",\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148\",\"url\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148\",\"name\":\"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/www.veritay.co.uk\\\/wp-content\\\/uploads\\\/2018\\\/12\\\/Bias-1-300x193.jpg\",\"datePublished\":\"2018-12-04T19:26:36+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/#\\\/schema\\\/person\\\/3194484920f6c197eb7cb0aa53cdf23c\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#primaryimage\",\"url\":\"https:\\\/\\\/www.veritay.co.uk\\\/wp-content\\\/uploads\\\/2018\\\/12\\\/Bias-1.jpg\",\"contentUrl\":\"https:\\\/\\\/www.veritay.co.uk\\\/wp-content\\\/uploads\\\/2018\\\/12\\\/Bias-1.jpg\",\"width\":657,\"height\":423},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/?p=148#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.veritay.co.uk\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/#website\",\"url\":\"https:\\\/\\\/www.veritay.co.uk\\\/\",\"name\":\"Veritay\",\"description\":\"Bringing sense to a complex world\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.veritay.co.uk\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.veritay.co.uk\\\/#\\\/schema\\\/person\\\/3194484920f6c197eb7cb0aa53cdf23c\",\"name\":\"Veritay Webmaster\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g\",\"caption\":\"Veritay Webmaster\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.veritay.co.uk\/?p=148","og_locale":"en_US","og_type":"article","og_title":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay","og_description":"High quality research needs to employ carefully phrased and expressed questions that give participants the opportunity to say what they really think, without bias or misinterpretation.\u00a0 Minimising bias in research is essential to get results that are as accurate, objective and actionable as possible.\u00a0 Biases can take many forms and can be introduced at any … Continue reading Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? →","og_url":"https:\/\/www.veritay.co.uk\/?p=148","og_site_name":"Veritay","article_published_time":"2018-12-04T19:26:36+00:00","og_image":[{"width":657,"height":423,"url":"https:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1.jpg","type":"image\/jpeg"}],"author":"Veritay Webmaster","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Veritay Webmaster","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.veritay.co.uk\/?p=148#article","isPartOf":{"@id":"https:\/\/www.veritay.co.uk\/?p=148"},"author":{"name":"Veritay Webmaster","@id":"https:\/\/www.veritay.co.uk\/#\/schema\/person\/3194484920f6c197eb7cb0aa53cdf23c"},"headline":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?","datePublished":"2018-12-04T19:26:36+00:00","mainEntityOfPage":{"@id":"https:\/\/www.veritay.co.uk\/?p=148"},"wordCount":651,"image":{"@id":"https:\/\/www.veritay.co.uk\/?p=148#primaryimage"},"thumbnailUrl":"http:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1-300x193.jpg","inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.veritay.co.uk\/?p=148","url":"https:\/\/www.veritay.co.uk\/?p=148","name":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See? - Veritay","isPartOf":{"@id":"https:\/\/www.veritay.co.uk\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.veritay.co.uk\/?p=148#primaryimage"},"image":{"@id":"https:\/\/www.veritay.co.uk\/?p=148#primaryimage"},"thumbnailUrl":"http:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1-300x193.jpg","datePublished":"2018-12-04T19:26:36+00:00","author":{"@id":"https:\/\/www.veritay.co.uk\/#\/schema\/person\/3194484920f6c197eb7cb0aa53cdf23c"},"breadcrumb":{"@id":"https:\/\/www.veritay.co.uk\/?p=148#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.veritay.co.uk\/?p=148"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.veritay.co.uk\/?p=148#primaryimage","url":"https:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1.jpg","contentUrl":"https:\/\/www.veritay.co.uk\/wp-content\/uploads\/2018\/12\/Bias-1.jpg","width":657,"height":423},{"@type":"BreadcrumbList","@id":"https:\/\/www.veritay.co.uk\/?p=148#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.veritay.co.uk\/"},{"@type":"ListItem","position":2,"name":"Hall of Mirrors Research \u2013 A Distortion or Seeing What You Want to See?"}]},{"@type":"WebSite","@id":"https:\/\/www.veritay.co.uk\/#website","url":"https:\/\/www.veritay.co.uk\/","name":"Veritay","description":"Bringing sense to a complex world","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.veritay.co.uk\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.veritay.co.uk\/#\/schema\/person\/3194484920f6c197eb7cb0aa53cdf23c","name":"Veritay Webmaster","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/addf434bdccd54ed7eddb01e44dfab0264dd735c689677012ec0a807e5cb7f0a?s=96&d=mm&r=g","caption":"Veritay Webmaster"}}]}},"_links":{"self":[{"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/148","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=148"}],"version-history":[{"count":1,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/148\/revisions"}],"predecessor-version":[{"id":150,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/148\/revisions\/150"}],"wp:attachment":[{"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=148"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=148"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.veritay.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=148"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}