I came to this presentation from Google on their Quality Score measure because someone referred to it by saying “Quality Score is a measure of user experience”. It obviously peaked my interest because it is precisely the qualitative characteristic of user experience that makes it hard to measure.

When you get to slide 4 you realize that Google knows better and defines Quality Score as “an automated measure of how relevant each of your keywords is to your ad text and to a user’s search query.”

It has nothing to do with measuring users’ experiences with anything whatsoever. I realize it sounds naive to be cranky about attention-grabbing people but it baffles me that people do this: misuse the notion of user experience to mean anything at all that they want. It is such a coward move. Be bold, say what you want to say!

More than that, I worry that people just have no clue what they are talking about. Because it that is the case, it is even more worrisome. If people engaged at this level of discussion (i.e.: what measures to use) don’t understand a basic thing such as what user experience means (at its most basic what PEOPLE experience when they INTERACT with something), then we’re all very far from being able to have progress in advancing the conversation about measuring success in the context of user experience.

If we agree that quantifiable data, specifically the ever popular web analytics, provide you with rich detail to tell you WHAT is happening, it is comforting to realize that it is the type of data gathering that we already do – design research – that provides the qualitative color to answer WHY said things are happening.

What I am finding, however, is that it is more valuable to START with the quantitative work and get to the WHATs and ask WHYs based on those findings, rather than trying to figure out WHYs in exploratory mode (even if the WHAT’s are going to emerge at one point or another in this quest).

My point is that it’s not sustainable as an approach. It’s inneficient to start digging deeper to answer the WHY questions if you don’t have a baseline of WHATs identified.

The problem is that it is not intuitive for designers to start where they are uncomfortable. We are super comfortable with qualitative approaches – they are our go-to tools because that’s what makes sense for design research. However, quantitative research instruments really help narrow stuff down, but they do require you to understand those pesky numbers in order to a) dig in and get to concrete answers and b) understand what it’s saying so you can ask “why”.

In short, WHATs before WHYs are more efficient than WHYs before WHATs, but that requires designers to start with unfamiliar tools to then apply familiar tools. If it was the other way around I think it would be much easier for designers to bridge both approaches and come out the other end with more useful insights.

In other words, since we don’t particularly feel an attraction to numbers (to put it lightly), why would we start there? It’s such a leap from how we think about problems that it is counter intuitive. I don’t believe designers reject the notion of starting with Quant approaches (WHATS) to expand with Qual approaches (WHYs), but it’s inherently counter-intuitive to think that way.

How can I help designers do this when it goes against their nature? That’s what I’m working on right now. More on this later.

I was excited to go but apprehensive because I was meant to present on a topic that is new to me and I had not had an opportunity to have other conversations about it across the community. Also, after seeing the first two days of excellent content one is bound to feel nervous about their own stuff! It ended up being great – I talked about key performance indicators and measuring success in the context of user experience (slides forthcoming – I’m writing notes because they are not good enough on their own as they were only triggers for my talking points).

While preparing for this talk I expected to have few people show up, precisely because of the reason why I am investigating this topic in the first place: designers don’t like numbers. I didn’t think the topic would be attractive at all (thus my “out there” title and description). The feedback I received and the types of questions asked during the event were really interesting and helped validate some suspicions about how our community sees metrics and numbers.

My main goal was to put something out there about KPIs and measures of success for UX so that we could start a conversation and really explore this topic. I have grown tired of how this topic ALWAYS ends up going into a “what is the ROI” conversation and never advances our ability to express what success means to us. Measuring success to show our value to others is a secondary goal, measuring success for ourselves seems far more valuable to me, which is why I am going to continue to explore this and try to focus the conversation on that goal.

The main theme I saw emerge from the feedback I received is that people felt validated; that I brought up the questions they all have but had not seen articulated in the community — which is precisely what I felt when I started looking into this 4 months ago. We can’t really learn and expand our understanding as a community if we don’t figure out what questions we’re trying to answer. And that is why, in my opinion, we always fall back on the pointless ROI calculation discussions.

I am very grateful for all who came, participated and found me later to discuss the topic. I am very excited about seeing what’s next.

The bigger problem is that if I am not providing updates to other people than it is likely I am doing a bad job keeping track for myself. That really should be the central reason for doing it in the first place but without external accountability I’m just a lazy ass.

Today I had 5 minutes so I decided to write my boss an email just to give him a glimpse into where I am with things. I used this model:

I am working on 7 projects at the same time right now at different stages of development so I wanted to give him just a taste of what is going on where.

Project name and a one liner about the last thing I accomplished was the bare minimum I thought was necessary. Two bullets indicating what is going to happen next and what risks may be incurred seemed to be the additional two most relevant pieces of information.

Finally, the red/yellow/green flags are really just to make the one page scan-able so he can see that I have 2 projects on green, 1 on yellow and 4 on red and without my whining – but knowing what the issues highlighted are – see that there are blockers or resource problems making that happen.

How do you keep people up to date about what you’re working on on a regular basis? How do you provide project updates to your peers and bosses?

After the presentation I actually started asking myself why the hell is it that I so rarely have to design for search behaviors. The reality is that oftentimes I’m designing for existing services where search is an existing capability and iterating it is never in scope.

One of the problems with that, which became more apparent to me after the presentation, is that treating search as a separate behavior from browse is really misguided. I thought about this problem before but could not quite articulate it very well until today.

Historically I had been taught and understood search and browse as distinct elements – which they are visually and from a UI elements standpoint – but from a behavioral perspective, they really are not, rather, they are part of a continuum. A spectrum of discovery behaviors if you will.

If we think, for example, about how faceted classification emerges in search interfaces and in browsing interfaces it becomes really clear how intertwined they are.

One of my questions to Peter during the presentation (which unfortunately did not get addressed but hopefully will be part of the UIE follow-up podcast) was if he had identified patterns of use of faceted search and if there were any emergent patterns that could help answer if faceted search is more appropriate for a particular kind of content or context — and when it might not be appropriate.

Faceted browse/search is a hot topic at work and I feel like it’s been historically a random requirement that ends up on a project brief because of process inertia. Someone saw it somewhere and thought it was cool so decided that it should be applied to the kind of content we are surfacing for our audience.

I have no good evidence to substantiate my hypothesis at this point (unless lack of examples in the wild is enough), but I suspect that for our content – namely video content, generally in the entertainment realm, frequently movies, series and other TV programs – having faceted search as a primary tool for discovery is really inappropriate.

I have definitely seen and appreciated the application in e-commerce and feel like there is a prevalent pattern there for its use. But on the content I design for, I just don’t know. If I am to rely on what I know from user behavior learned observing people try and get to the video content they want (across different platforms in a number of distinct scenarios of use) the attributes they need to make decisions are frequently few. The variation in behavior is little in terms of user motivation, and greater in content type (i.e.: people look for movies differently from how they look for series).

How can I make a compelling argument that this particular pattern is not the right fit when I am not sure what is? I’ve seen it fail in usability tests but that only makes people try to fix it and improve it, not to try a completely alternate solution that might be appropriate. Any ideas out there?

Also, I’m not on a crusade against faceted search, I am just looking for ways to 1) articulate that there might be a problem picking this particular pattern 2) explore other ways to do it (both in the context of use and content I described). Any ideas are welcome.

Regardless, I think it will help me in the future to frame the scope of what I need to design for when dealing with content discovery behaviors by thinking about them in the browse-search spectrum. At least I expect that to give me a better argument to combat feature requirements void of context.

Per the project’s page:

Project52 is a personal challenge geared toward getting fresh content on your website. The goal is to write at least 1 new article per week for 1 year. Because we all know what it‘s like to procrastinate on our content. A website is not just a fresh design that can be uploaded to the web and forgotten about!

Let’s see how I do this thing!

Every year I write a blog post on my birthday as a way to pause and take stock of the past year of my life and think about the future. I am not one to make resolutions, I’d rather just set some direction to help shape the path (for the design geeks in the audience, I’m more about identifying principles than setting explicit goals).

This past year was specially important in become more self-aware. Things I’ve learned about myself this year:

I love surprising people – this often means exceeding people’s expectations, at work, in my personal life and other contexts. What I learned in the past year is that I will go to so many lengths to try and do this that it often puts more strain on me than necessary. Sometimes just doing what is expected is the best for myself and whoever benefits from it. Exceeding expectations is a wonderful goal, but if there is no good payoff or return of effort, it just burns you out. It gives me great joy when I do things and people go “wow”, but the harder I have to try, bigger my expectation of the “wow” reaction. And like an addiction, looking for more “highs” can really put you over the edge.

I don’t know how to fail – I haven’t figured out how to articulate this well enough yet, but in blunt terms, I have been extremely successful in pretty much anything I have tried in my life thus far. I am not apologetic about this – I have worked hard to make all these successes happen – but I am very aware of what I don’t learn from not failing frequently enough. As a designer, I understand the value of failure – but I only understand it on a rational level, not as an internalized practice. I think I would be a better at what I do if I knew how to fail more. And “knowing how to fail” may sounds odd, but only if you think of failure as an external factor that is not your own doing. I think of myself as someone who is willing to take risks and good at taking calculated risks, but I can’t avoid thinking that my infrequent failure is not because I am particularly gifted, but because I focus too much “getting it right” soon. This is extremely well received in the working world, but my gut tells me allowing myself more “failure opportunities” could help me achieve better outcomes – professionally and personally.

I never put myself first – This one just plain sucks. I was raised to think I could do anything. I was also raised with the ethos that gloating and selfishness were negative things. I somehow internalized these as “do everything wonderfully for everyone else, then yourself” and consciously or not, I always try and make people feel appreciated, deserving and important. That combined with my love for surprising people, generally means I put myself last. It’s a shame because these are not mutually exclusive things, but in circumstances when they are, I don’t even hesitate and will relegate my needs to second-thought at best. I haven’t really figured out why this happens or how, I’ve only just become aware of it enough to see just how frequently I do it, but it is really tiring for me. Since I was never conscious of this as an underlying attitude, it was not something I could address. Now I am more aware of it so I can make better choices and I understand it’s about balance, neither putting others nor myself first, but nurturing myself and others sufficiently, which probably requires better judgment when deciding.

So in the future, I will continue to try and surprise people because I really get great pleasure from it, but not to my detriment. And rather than trying to get to a positive outcome by taking calculated risks, I’ll invest some extra time in exploring more risky opportunities. Perhaps I’ll fail more and hopeful I’ll learn more too.

Now let’s talk about age. 30. Whether I care about age or not, you can’t really say “a kid of 30″, so this is effectively an old enough age. This is only helpful or relevant to me because for the majority of my professional life, I tried to stay the hell away from any conversations about age (which was annoying because unlike a LOT of people in my generation who fuss about what age they are or are not, I REALLY don’t care). The reason for that is I have ALWAYS been the youngest person in the room. Whether I was the junior person, the established practitioner or the boss. Whether I was a consultant, a contractor, a partner or a corporate in-house team member. ALWAYS. Every single context and circumstances. And I don’t just mean professionally. In the sports team, in the guitar class, in the volunteer group. Always the youngest. And because of how most people in our current culture do make assumptions and are influenced in their behaviors by perceptions of ability, capacity and maturity that they derive from someone’s age, my age always had the potential to work against me.

Because of that I took the necessary precautions to mitigate that risk. I have made wardrobe choices, hair cut choices, speech and tone choices, language and writing style choices, and many other choices to deal with the problem. At times, it was almost like acting, playing this older version of me. I would not and have never lied about my age – if someone asked me I was absolutely honest about exactly what age I was – but I never volunteered this information. At the times when I experimented with volunteering this info (in safe settings), I was proven right and people’s behaviors towards me were transformed, always in negative ways. Telling people I am gay did not provoke nearly as much transformation in behavior as real “young” age did (maybe a comparison for another time).

As with any other irrational reaction to a natural attribute, like age, or color or sexual orientation, the negative response is based on fear. Ageism comes from both older and younger people. There is no win; but it’s all fear based – specially fear of obsolescence by those who are older (how can they do/know/be ____ so early when it took me this long?) and fear of failure by those who are the same age/younger (how can they do/know/be _____ already when I’m not there yet?). I sincerely hope this ageism is a generational trait and one that is overcome or better dealt with by the generations that follow. Either way, now I’m old enough for most circumstances and don’t have to worry about this nearly as much. I think this is the best gift I could get turning 30. Happy birthday to me.

Dave made an important point, to say we are all generalists and specialists in different circumstances. I like the visualization but I feel like it doesn’t tell me what effect the different approaches produce. I don’t mean the outcome, but in how they approach it differently, what else is different other than breadth and depth?

I believe Generalists and Specialists approach defining goals, solving problems and designing solutions similarly. The difference is in what lenses they apply in the middle. In our quest to go from where we are to where we need to be we first diverge to seek options then we converge to find solutions. The Generalist goes for BREADTH when seeking options while the Specialist goes for DEPTH. The lens applied regulates how much they need to diverge and how soon they can converge to get to a solution.

In this very simplified white-board sketch I fail to convey the variability, but you could see how a generalist would stretch and go as wide as possible for options before converging into a direction to solve a problems, defining goals or designing solutions. The specialist, on the other hand, would likely not stretch as much but lengthen the process in his quest for depth.

Just a thought.

On somewhat but not entirely related topic, I really like Jared’s take on Specialist versus Generalist distinction in UX teams.

All donations to this event will benefit the Information Architecture Institute, the organization behind the IDEA conference.