Having Empathy For Your Users
My partner Albert has a great post up today on the limitations of data and A/B testing in managing a product. He strongly advises that product teams do two things that many don’t do
1) do in-person product testing sessions to see users interacting with the product and develop an understanding of why users struggle with aspects of the product
2) use the company’s product (really all employees should do this)
I want to echo Albert’s point and suggestions. I feel like the companies we meet with and work with generally do a good job of instrumenting their products and collecting on data on what is working and what is not working. But they often don’t have good answers for why the behavior they are seeing is happening. It’s hard to fix something you know is broken unless you understand why it is broken.
Comments (Archived):
If you are not out touching your customers you are not doing your job is my general rule of thumb.In execution everything gets more complex-and more interesting.
Knowing WHAT is happening and WHY it is happening are two different things. Albert’s suggestions are great for the later and can validate the data which proves the former.Are there tools to figure out the WHY systematically beyond surveys and focus groups? Can you figure out the WHY using “data collection?”
Eat your own dog food. But don’t get high on your own supply.I’m surprised by the lack of QA testing in many agile development operations. I don’t think agile means ‘don’t test’.
wow, am surprised that this is the only comment about making sure the dogs (employees) eat the dog food (company product). if you have the right culture, this is the fastest and best way to find out what is wrong/broken. and maybe more important, if they don’t eat you should be talking about whether or not you should be making it in the first place.
{head nod}
It really does depend who your target audience is, though. If your customers are low-tech or 40yr+ then no amount of eating your own dog food will help. The knowledge gap between the product team and customers is substantial, and understanding what confuses low-tech users is anything but intuitive.
Fair point
Actually I see too much dogfooding these days. Employees assume that if they like/use the product then it is great. But they know too much about how it is supposed to work. You need fresh people to have an “out of box” experience. Back when I was a UI designer here’s how I would do it:0. Recruit strangers (CL works well, of course you can pay much more for targeted search)1. Have someone run the test who is not the designer of the product. Absolutely don’t have an engineer run the test. We had a rule at Tellme that you were not allowed to usability-test your own product.2. Get them to sign a release and videotape them. Can’t emphasize how important this is. You want engineers to watch the video. If you are b2b you want your clients to watch the video. I once had Citibank as a client and they were wholly obstinate about controlling the design until we showed them video of their own customers getting completely stuck.3. Start off the test by just letting the play around. Don’t (yet) give them directed tests; just let them fiddle about for 5-10 minutes. DON’T answer any questions; just write them down. Maybe give them some general overview of what the product is about.4. After that, give them directed tasks to do. Obviously, don’t tell them how to do the tasks. If they get stuck, DON’T help them and don’t answer questions.There are much better UI designers out there than I am, and I’m sure they’ll have comments on the above.
first, i’m not suggesting you don’t do in-person testing, albert’s first point, as i believe it is hugely valuable when done properly. i was surprised by the fact that not until jim’s comment had anyone mentioned point 2, which i believe is as important as point 1. and i don’t believe you can eat enough of your own dog food if your company and customer base is growing. there’s a difference between eating the dog food and drinking the kool-aid. when you eat the dog food you should do so critically.
Qa needs a process
You need someone with the right training, experience and disposition to do this. This is the skill set and mindset of an experience designer.Everyone thinks because they experience products that they know how to craft an experience. That’s the chief fallacy here — 99.9% of people don’t know the first thing about how to properly do in-person user testing, what to look out for, ask, not ask, etc. There’s a science to it, and it needs to be executed by someone who knows better.In the same way a non-technical person shouldn’t be the point person for code, a non-experience designer shouldn’t be the point person for how do the research needed to craft the optimal experience.
Not sure I agree completely. I get that parsing the information collected and designing the experience are areas where a person should be experienced but, to understand what you see a user struggling with, what you yourself would hate doing or love doing are things I believe anyone who has used an application can do. I do appreciate the fact that designers play a key part in the UX design to ensure those issues don’t arise but, sometimes an outsider who doesn’t have experience may be able to notice something that even a seasoned designer may not have.
That’s why the UX person should conduct interviews, to get the perspective of outsiders. That is, in effect, a very large part of the job of a UX designer.
HELLO!
Brandon – Are there any good resources on this you’ve seen? I’d be curious to see how user experience testing requirements are different than general customer interview requirements.
Resources on testing? I haven’t seen anything all that great. The best resource I’ve seen is experience working under smart people who know what they’re doing.Personally, I think the first big thing to remember when in-person testing is that people act differently when they’re being tested. They are not going to do in front of you what they’d do in front of a computer alone. There are a whole host of tactics you can use to combat this, but the biggest problem I see is that folks don’t even know that this is a phenomenon that needs to be countered.Next important, I’d say, is don’t lead the witness — even if the witness asks to be lead (and he most definitely will). When the user looks up to you and asks questions in the middle of a testing session, which always happens, don’t answer them. Ignore them, and simply watch what they do.And, lastly, don’t ask them why they did what they did. Well, actually you should ask, as it can lead to good discussion about a lot of important things — it just usually won’t lead to the real reason why the user did what she did. That user is being influenced by conscious and unconscious behaviors, fonts, colors, words, hierarchy, placement, past experiences, future wishes — so many things go into why someone clicks where, and there’s no way a regular person can parse through all those things and go “I did X because of Y.” They have no clue, and good UX person probably doesn’t exactly know either (but she should have a hunch if she’s experienced and has seen similar behavior before). That’s why it’s important to observe big enough of a sample size, and use trends between users to infer what’s really going on. Tweak things, with controls in mind, and run the test and observe again. And again. Eventually, answers will reveal themselves.
Thank you Joe, Susan, and Brandon.
guerilla testing: 5 people 5 times is infinitely more valuable than 25 people 1 time, http://www.nngroup.com/arti…
This is an old book, but still extremely useful. It nails the basics and is a quick easy read – https://www.sensible.com/dm…
Some good answers (and other resources) on what you’re asking here: http://alistapart.com/artic…
A good anthropologist also could do the same. Ethnographic techniques came from them
Brand Manager does this in his or her sleep as well, but tech had never figured out (even B2C tech) that this discipline even exists.
Tech hasn’t figured out a lot of things that, ironically, have already been figured out by others.Tech likes to pretend that the world didn’t function and no one in business knew what they were doing before startups arrived, and so now they need to reinvent the wheel. By far the biggest, most damaging and permeating fallacy of the tech world.
Agree, and in my experience it’s a fallacy most frequently on display in marketing and user design
Yep. And those are my worlds, so it pains me quite a bit.
having a conversation lately about how creatives invented agile / kanban a long long time ago — posting drafts, versions for markup on the walls.
it pains me when i think about how the creative industries (design, advertising, media, film. etc.) are so separated from tech.each side needs to incorporate talent from the other because each has totally different sensibilities — the creatives are more empathetic, the techies are more analytical — but its those same differences in sensibilities that keep them apart.i know several creatives who jumped ship to tech companies (google, facebook, tumblr and twitter seem to be the most popular for the switch) but then they flee, running. “not really into the culture,” they say. and then they go consult for $200+ per hour — because folks will pay a lot for hybrid skill sets — taking themselves out of the debate altogether, and nothing changes.
yes to all. see also, recruiters looking for ux people to code; the equivalent of looking for foot doctors to work on hands, or plumbers to do electrical work. design is not just what it looks like and feels like. design is how it works.
preach!
Yep couldn’t agree more. My product started out as a personal need and has evolved significantly because of my gap in knowledge on how teachers and therapists operate. Not sure if I could have done any of that without sitting down with the users. Thanks for the post, re-affirms a lot of what we are doing at Kidhoo
yes yes yes yeshttps://speakerdeck.com/lau…
The lack of classic marketing techniques w/ many start-up companies today is startling. Most rely on quantitative testing in one form or another, but few utilize qualitative techniques to better understand attitudes and perceptions of their products/services among both current and prospective users. Behavioral info that can add perspective to quant data, help identifiy salient attributes/benefits, fine tune product positioning, etc.
Agree. Great technology often hides very poor marketing. Want to find best practice in market research? Go to industries where there is virtually no technology. Think for a second about the technical differentiation between Malt-o-meal Tootie Fruities and Kellogg’s Froot Loops. Only way Froot Loops wins is by superior marketing, and that marketing plan begins with deep consumer empathy.
Deep consumer empathy. Great phrase.
I agree, and think part of it is based on a misconception people have about digital information. There are as many subtle signals and possibilities for the meaning of a click-thru as there are for the meaning of someone picking up a physical product in-store.
totally agreed. the pendulum needs to swing back. too much digital/content/growth marketing is overshadowing their attention to basic marketing, especially its strategic aspects.yes, buyer habits and preferences are changing due to online, but that change must be taken along the perspective of the bigger fundamentals.
Ding! Most also don’t get that many classic techniques are well verified by math as well
+understanding why something is broken is a prerequisite to knowing if it actually needs to be fixed.Nothing worse than hammering away trying to fix something which shouldn’t even be there in the first place.
“It’s hard to fix something you know is broken” – But that assumes you “know” what is broken. Data doesn’t always tell you what is broken.Whether it’s empathy or intuition, one needs to go beyond data. As I sometimes tell entrepreneurs, “stop looking at what under your nose, and start looking out the window”. A/B testing is mostly for tactical testing (it’s like doing micro focus groups), and will not lead you to strategic aha moments, unless you add a dose of empathy, intuition, insights and experience.
+ 1. And a lot of times you have to practice listening before asking anything of end users. Big difference in responses / information when you ask “tell me about how you use our product” vs. “tell me about the world you live in (that relates to our product).”
This is what UX means: You have to drill down on what the user experience is really like, hence the empathy you are talking about. It necessitates a product manager who is obsessed about nailing it just right — easier said than done.
Here’s a great story that illuminates this point. I work with several public transit agencies here in CT. We were getting a lot of complaints that there was no way to download schedules onto smartphones. Yet, the schedules were all available for download. Users just had to click on a PDF icon that was in a toolbar on each page. We decided to conduct a focus group because we couldn’t figure out where the disconnect was. We found out the answer in minutes. It turns out that most everyday people using the bus have no idea what a PDF icon is. The people also did not understand the download button that is now becoming more ubiquitous. The people wanted a clickable hyperlink that said “Download Schedule.” No data was going to show anybody that.
Great example. If you want everyone to use your website, you gotta design for everyone, not just computer savvy people. I believe that almost every icon should have text along with it. Some people are icon oriented and some are text oriented. If you only have an icon to represent an action, you will have some users that do know what it represents, but each time they see it they have to translate in their head from their icon image sense to their language sense.This is annoying and fatiguing. I know, I am one of those people. Decoding icons are work for me, even when I know what they are. And I have found that most designers are prejudiced toward using icons. They just like them better from a design sense, even though they are not that great from a user perspective.
What is the download icon? An arrow pointing down? That seems to be what i’m seeing more often.
Yes, that’s the download icon.
big improvement on the hard disk icon for save / pdf
Ab tests are often just a midpoint in creating good products. If you can’t think about such basic questions such as how your users are, you have other issues
A/B builds a good UI.Mapping how live users interact with the UI builds a great UX.Great example.
I 100% agree and on the off chance someone reads this comment and has a solution.HOW DO YOU STORE ALL THIS USER DATA?I started an eyewear company (check us out on Shark Tank tomorrow night ) and we have a constant stream of user data from the website analytics, product data from people at coffee shops, our own team (we all wear Rx glasses), in-person user tests, video recorded screens, Google surveys (this is an amazing tool), customer satisfaction surveys, our vendors, emails from our angel investors, other sites… The list goes on.Does anyone have a good method for this?
Depends on the type of data. Usually I try to keep data as close to the source as possible and do backups to Amazon S3 using CRON and shell scripts. Google usually does a pretty good job of keeping data including e-mails stored already.
Interesting question…and probably suggests another, slightly more complex one: how do you store all of it in a way that is consistent and usable over time?The sources you mentioned (and hundreds of others) vastly divergent by company. Most of the time the problem isn’t just collection and storage, it’s understanding where it comes from, if it matters, and what to do if it does (prediction and modeling, yay).I’m not a highly technical person but in product, ops, and marketing roles I’ve dealt with the middle ground a lot. To some extent the answer is, get insanely good at cleaning up data in excel / using sql, and find / build your own model for standardizing your data streams (likely to be different in each company). OpenRefine (an ex-google project) is a pretty good place to start ( https://github.com/OpenRefi… ). If you can clean up data and define your approach to categorizing it, there are all sorts of handy ways to toss it into excel or google docs online and run APIs or dashboards / apps off of it. Also it’s a good idea to run trello / basecamp / prism.io or some sort of project mgmt / dev tracker to profile & update data sources. No one ever does that and when the source for a particular stream changes even in a small way it’s usually a while before anyone notices if it has consequences.That was a little longer than intended. Anyway, maybe it’ll be helpful. Good luck in the shark tank!
Also, if you’ve got the dough, Domo is pretty serious stuff. Saw under the hood a little ways back and they know what they are doing, it’s just expensive.
As a UX design lead and (as of 9 days ago) former politician, empathy needs to be practiced company wide. Test and validate your product, problem definitions, and prototypes, but make room for blue-sky empathy for everyone in your org.
I have to ask….how on earth did you manage to survive being in politics while also being someone who cares about user design and empathy? Must have been frustrating…
You’re right; It was one of the most frustrating things I’ve ever done. But it was worth it for the bit of good I think I did, and for the extra insight it to how horribly bad our systems of government are. We’re bound by 18th century org structures that we don’t even realize. Talk about technical debt. I bet we could double world gdp with new governing structures.But I think my professional work in empathy did help uncover more root causes of public problems.
Too true. If ever there was an area that needed ux and design thinking / human centered design, it’d be gov’t. One of the reasons I appreciate and pay attention to civic tech, the results are really tangible…even a small thing to create transparency and access goes a long way.
CC: Every DMV office in the nation and also Comcast customer care.
The question is if they don’t care what will happen?With both of those example nothing will happen. Consequently they certainly have less of a reason to care. Caring is not going to get them more revenue. Unless they figure out it will for some reason. The best caring can do is make sure they don’t piss off someone important that they need for some reason (like someone at the FTC for comcast). Or maybe help with employees or moral or whatever (if that matters). Most likely the cost for caring for everyone has been carefully weight vs. the benefit that may in theory be possible. It’s really a simple calculation. Most likely they have no problem hiring who they need even with their reputation.When I was a kid there was only one place to eat at turnpike rest stops. My dad used to say “Doesn’t matter how the food tastes or how they serve you because there are no other choices. That’s why it can be bad” (or something like that).
Hmm….well with Comcast hypothetically at some point they’ll be so out of sync with their customers that their core products won’t be wanted / used and they won’t be getting good enough information back to build innovative products that keep them on top.Will have to go back and read up, but the automotive industry does come to mind..
A topic after my own heart! User testing is the only way to figure out the three or four different ways that your users will conceptually approach using your application. Then you have to design for each of these groups of users, which is quite challenging.You need to get into their heads and know why they did what they did. Once you understand where they are coming from, the entire app needs to be designed for each one of these approaches. So, you gotta design for three or four different approaches at the same time and put it all together into as seamless an experience you can. Then user test the crap out of it, so you can fine-tune everything. This is a lot of work, but actually quite fun and rewarding.
1) do in-person product testing sessions to see users interacting with the product and develop an understanding of why users struggle with aspects of the product2) use the company’s product (really all employees should do this)Well I am onboard with this thinking obviouslyIn person testing is what I used to call “find the stupid person and see if they understand”. We named it after the simplest employee that we had. That was back in the day when you could do things like that and not have any legal issues.Anyway I am obviously on board with this type of thinking. However keep in mind that there are plenty of situations where a company knows something is broken (or has a hint of it) and still does nothing about the problem. I will now pick on disqus as the obvious example of such a company as I and others have, from time to time, pointed out at least a few annoyances with that product.The question then becomes “why?”. There are really only a few possible answers. One is “they don’t care”. The other is “they don’t have the resources”. The last is “they don’t think it’s a problem”. Or perhaps a combo of all of the preceding. Maybe even “if we fix that then something else will break”.It’s hard to fix something you know is broken unless you understand why it is broken.Monitoring is also important. Just this AM I pulled up AVC and got a NGINX error which came from cloudflare. I didn’t tell anyone after all who would I tell (and why). The question is does the site owner (you) know? Almost certainly not nobody is going to email you. Does cloudflare know? (Good question). This has happened multiple times to me when pulling up AVC.
Having Empathy For Your UsersLack of empathy is really an artifact of the new world order where boatloads of money keep companies afloat and there are enough new users to replace old pissed off users. Way back you couldn’t not have empathy (generally) because you would quickly go out of business. Or loose your shirt.Have you ever noticed a lack of empathy when dining in a restaurant in a hot competitive market like NYC? Probably not, right? They have to give a shit about everything and each and everyday there is a feedback system in place whereby they know exactly what people like and do not like.
This is spot on. Of course that’s a common dynamic of quality vs. scaled growth …. just because you can scale the way in which information transfers and people communicate doesn’t mean you can skip the individual bits of data or interactions.
.In the old days, we used to call this a focus group.Still do.This generation didn’t invent sex or focus groups, apparently.This is so basic as to be laughable. Hope that sounds mean and condescending. Sorry.JLMwww.themusingsofthebigredca…
A focus group is a bit different at least to me.The idea (the way I see it) is more informal simply figuring out ways to observe people’s behavior in the wild.Paco Underhill has written books on the shopping experience. He essentially walks arounds and observes people and asks them questions casually.http://www.amazon.com/Why-W…I do a version of this with some research that I do (for various reasons). I will just strike up a conversation without tipping off the reason (leading the witness?) as to my true intent and purpose. In the end I get the info that I want without I believe as much bias. It’s a bit of social engineering.
Per #1, probably nothing sends message home harder than watching user X struggle with your product, as you shout/flail with frustration from behind one-way glass or while watching recorded session remotely (“No! Turn it left and press the button, it’s right there, damnit!”).Per #2, great in theory but not always in practice. E.g. a team member in company Y making tool Z used primarily by sugar sellers can certainly take steps to learn about their clients’ pain points but since he/she ultimately isn’t a sugar seller, there is a limit to his/her empathy and use of product beyond testing/QA. Alternatively, in applicable consumer product contexts, there is the real possibility that the product ain’t so hot but management will not change course or at least query the jarring fact that staff won’t voluntarily use it.
Was talking about some of this w/cofounder last week…in startups there is a lot of “we’re changing the world!” that actually translates to “I’m changing my world!” …and there is a difference.One of the things that happens as an extension of any product (and person, really) is the “tell me about you as it relates to me” problem. I think most people know this but the practice of listening / understanding someone before you even think about what they are to you and your product, we need so much more of that.
True. Listening and understanding effectively does require much practice, not least because it involves more than just the ears. If your eyes, ears, brain, +sometimes body (language) aren’t in sync during the convo, then much is lost.
A lot of the RideShare companies independent contractors have a problem with number 2. They have never taken a ride as a passenger and have no idea if/where to tip, put in destination and the purchase payment cycle works
“By Watching How Customers Shop in the Store, Crate & Barrel Lifts Web Sales 44%”Or, a lesson on how to achieve empathy:http://www.adweek.com/news/…
All “Big Data” and A/B testing systems are about probability and how people use a product.Affection for and affects of a product are about PERCEPTIONS and why people love and use it.This is why I invented a perceptions system.Data coherency = Perceptions x Probability in a weighted matrix.The other thing is that there’s a mistaken assumption that as long as a platform measures the frequency count of #users, #happy, # 5-star ratings that this is also a signal for managing a product.Frequency is a quantity which isn’t the same as quality.
Gif Constable calls it “getting out of the building”. Great book… and free!http://www.talkingtohumans….
Because sometimes a story makes the advice stick better: http://steveblank.com/2015/…
Good post; I’d add a number 3. Everybody on the team should field support calls for at least 1/2 day per week. And I mean everybody. From the CEO to developers to sales guys to designers to you name it.
There’s a certain CMS platform we have in alpha at work that I’m constantly using as a guinea pig and I wish I could get a screen capture movie working to capture all the frustrating interactions I have with that particular piece of software so that the developers can see it. I send them narratives and screenshots — sometimes they are shocked. Yes, it has to be inperson, and for long periods of time, or figure out how to install videos of the user using the product.On an off-topic question — after getting press accreditation for three years in a row at TechCrunch Disrupt as a blogger at my blog Wired State, when I applied this year I was ignored and didn’t even hear back. Can you help me get a press pass?
Great point. I remember talking about this with the Meetup team when I visited their offices in New York. A lot of tech companies have forgotten not only this, but also how to treat their users. I think Twitter lost their manners completely.
Great post. Big Data is King but small data is the power behind the throne. If one breaks things down to the smallest common denominator, clarity is sure to follow. Loved your comment @susanrubinsky:disqus. It is so often this way! When things are strange, there’s always a piece we don’t know. Finding it is the best outcome of the data treasure hunt. BE the consumer.