The Measurement of User Experience (part 3 of 3)

In the third strand of my research on Positive User Experiences I am looking at the strategies and tools companies are currently using to conduct usability testing and measure User Experiences.

Some initial observations from the literature and the web:

EVALUATION:  different usability experts find different problems with the same product

DialogDesign,  a Danish Design agency have carried out several research studies over the last number of years on Comparative Usability Evaluation (CUE) investigating how different usability experts review the same product. They found on many occasions, that different usability teams report different usability problems when analysing the same product.

DATA ANALYIS:  informal testing and quick turnaround of results Vs formal testing matrices

Another recent paper looking at Analysis in Usability Evaluations talked to 11 usability professionals about how they conduct testing.   For the  most part, the experts’ analysis did not follow a formal process or structure but rather problem definition and recommendations were based on the expertise of the evaluator. This paper concluded  with a recommendation to develop a research-based analysis resources.

The debate for on-the-fly quick turn-around expert reviews vs more structured (and valid?) analysis continues

MEASUREMENT:  heuristics and guidelines such as those promoted by Neilsen and Krug are popular

In the study above, some of the experts did report the use of established heuristics, such as Nielsen’s.  Another popular expert often mentioned on the web is  Steve Krug and with his usability tips and his ‘common sense’ approach to usability.

METHODS:  common UX/usability tools and methodologies

Some common methodologies mentioned are observation, contextual enquiry, cognitive task analysis, Talk Aloud Protocol, RTA, eye-tracking, scenarios, personas,  interviews, A/B testing, comments, guerilla marketing?, card sorting, story boarding.    Remote testing and testing on multiple devices.

MEASUREMENT:  capturing the bigger picture

In addition to testing for usability errors, many agencies are developing methods and instruments to evaluate UX qualities such as trust, presence, satisfaction or fun. The intention is to better understand user attitudes and how they feel about the overall experience.  To make the experience a positive user experience

REPORTING:  connecting UX to competitive advantage

How to feedback findings to customers?  Reviews, customer journeys, flow charts, wireframes, mock-ups, prototypes.   The aim is to show how  an effective UX strategy can achieve company goals and enable business strategies.

**Got any comments or know of any resources related to this post?   Please leave me a comment!


Understanding the User Experience

We don’t see things as they are, we see them as we are…


I would like to know more about the User Experience – what is it? How do you describe it? How do you measure it? 

During my time as Manager of UX lab in National College of Ireland, I was involved in over 25 UX projects ranging from usability testing of on-line maths and accountancy support programmes through to analysing the search behaviour of Google users.

One development that I particularly found very interesting during that time was how important the emotional side of the engagement was to users.
For example, we would test two applications that performed similarly in terms of task completion and success, ease of use etc.  yet users would much prefer experience of one the applications over the other.    Why?

We see that as technology is becoming more accessible and users more technologically sophisticated, that users are increasingly looking past the mere functionality of an application and seeking to engage in interactions that are not only usable but also offer a more meaningful positive experience.   But how do we describe it? and begin to analyse it?

Of course we know users’ experiences vary on context, the product and user themselves.   Each user will bring with them their prior experiences, expectations and their values and beliefs; they have their own story.

As someone once said  ‘ we don’t see things as they are, we see things we are’,

And while we may not have a complete understanding  yet of what that story is,  the literature suggests that much of this engagement is psychological in nature and dependant on users’ attitudes, emotions  and feelings.   I believe psychology and scientific psychological research methods have a lot to offer us in better understanding this part of the users’ journey.,, and this is what this blog will be about 🙂