New user experience person working in (digital) libraries

A couple of days ago I found another person who is doing work pretty similar to me, only she blogs about her practical work a lot more than I do.  Lorraine is posting a series of usability analyses on digital library sites and software, and her insights are very interesting.  You can read her blog at lorrainepaterson.wordpress.com, and you can follow her on Twitter (lorraine_p).  I’ve also added her to my blogroll.

Why users like federated search (even though they shouldn’t)

‘Federated Search’ is a library term, it refers to search engines that search a variety of library databases (things that contain journal articles, conference papers and the like) and combine the results in some way to be presented to the user.

Federated searching is a somewhat fraught topic in libraries; many librarians don’t like federated searching and are hestitant to recommend it to library users.  This reluctance is not without good reason–federated search is inferior in many ways to using native database search interfaces, including problems with relevance ranking, the false appearance of comprehensiveness, and the inadequate de-duplication that many offer.  On the other side of this, federated search offers the holy grail of library searching: a single search box (well, almost–federated search usually doesn’t include the local catalogue, though sometimes it does, as in this example at UNSW).  The single search box is seen as being “like Google” in offering users a lot of different content from one search–and even has a slight edge over Google scholar in that search results will usually reflect more closely than Google Scholar which results a searcher can actually access.

Federated search has some issues that would normally be pretty big rpoblems from a user perspective too:

• The relevance ranking doesn’t really work. Because federated search is pulling in material from a range of sources, each of which use different approaches to relevance ranking and different metrics to express a rank.  Any combination of these results is likely to produces flawed relevance ranking.  This means that often, the most relevant results will not be in the magical first couple of pages.
• Federated search is very, very slow.  Again, because federated search is searching a number of remote databases and then applying some metric to combine results before these are presented to the user, federated search is very slow. Typically users are unhappy with slow response times, so this should be a real problem for users.

So, we know librarians are often hesitant about recommending federated search, and that users have every reason not to like it…and yet study after study shows that users do like and use federated search.  So why is federated search so popular?

• One stop shopping: Federated search offers users a one-stop shop, and even though they know it isn’t as good, they will often use it anyway.
• Time saving: Despite the long load time for search results, users know they will save themselves time (and likely frustration) by visiting only a single site.
• Search syntax: Search syntax varies slightly from site to site, and federated search allows users to forgo learning the variationson syntax required by individual databases.  Given that we know boolean searching is hard (sorry, paywall), it is easy to surmise that learning less about it is considered a good thing by users.
• Low user expectations: Users expect library systems to be slow and clunky, so their expectations of federated search are lower than they would be for other web-based services.

Users’ willingness to use a system we don’t expect them to like is an object lesson in how usability principles are not entirely universal: Occasionally users will tolerate unusable systems over more-usable ones because the end result is still a faster and easier user experience.

So, does users’ willingness to put up with the limitations of federated search mean we should stop striving for anything better? I don’t think so.  I think that as web technology improves, users will have less tolerance for slow and clunky systems.  We’ve already seen this at Swinburne with the library catalogue–while it hasn’t changed our users surveys show increasing levels of dissatisfaction as a result of user expectations that have been raised by their interactions with other systems.  I don’t believe that users are going to be willing to individually visit library databases in the future any more than they are now; even Google is meshing different kinds of data in its search results.  I believe there is real benefit to be had for librarians and library users alike in making headway in one-stop searching, I’m very much looking forward to seeing Primo Central and Summon (the next generation of federated search, where metadata is locally indexed making search faster and relevance ranking better) in action.  In the meantime though?  Users still like federated search, even though it is slow and awkward.

Connex: a great example of systemic failure to care

Tell me, if you wanted your automated train ticket machines cleared, would you choose 823 AM on Monday as a good time to do it?  Even if ticket clearing takes 13 minutes? Even if, during those 13 minutes, four trains stop that station according to your schedule, and in fact 6 actually stop because two are running late?  Even if it was a station where not many trains stop, because it is not a primary station? Even if there is no other way for your passengers to buy tickets, unless they have enough coins (tickets start at $3.70)? Even if you regularly ended up with 10+ people waiting behind you?

No, neither would I, and yet between them, this is what Connex and Armorguard think is a good idea.  There is actually plenty of scope to clear the ticket machines at that station where not one single train stops there, even during weekdays.  I’ve written about Connex before because of their poor approach to user experience, and while they have often given me cause to do it again, I didn’t want this to be the “I hate Connex” blog, so I’ve left it alone. This particular example, though, was a  perfect demonstration of how little Connex cares about its’ customers’ user experience, particularly when you factor in the Connex guards who were present to prevent the “fare evasion” not being physically able to buy a ticket during peak hour might normally cause.

It comes as no surprise that Connex have lost their contract for the Melbourne metro train system, and while it is likely true that the State government needs to come to the party if services are genuinely to be improved, I won’t miss the callous disregard Connex shows for its customers, nor their pre-recorded message apologising “for any inconvenience caused”. There are things the new operator can do, even without government support, that will show that they are interested in their users’ experience of their system, and this more than anything will make a difference to that experience.

Apologising: Google is doing it right

As some of you will know, gmail went down for 100 minutes early thismorning.  I did notice it, but assumed it was my internet connection acting weird again–and I didn’t really need to read email at 7AM anyway.  For people elsewhere, however (for example in the US where this was anything from midday to close of business) and even people in New Zealand where the workday was just beginning this could have been a real problem, especially for those using gmail for business porposes.

Given how reliable Google usually is, this sudden and lengthy failure will understandably shake confidence in the service, and may even make people more righteously angry than service failures by unreliable companies (consider my eyerolling acceptance above, when I thought the problem was my ISP).

Generally speaking, users can think one of three ways when things go wrong (and lets face it, things do go wrong sometimes with any product or service):

• That the product or service is unreliable and therefore they have lost faith in the product or service and the parent company
• That something went wrong, but that the company did what they could about it and the solution was acceptable so they will continue to use the product or service
• That the resolution to the problem was not satisfactory, but that they have no option but to use the company next time anyway (for example when the company has a monopoly–if this is the case though, as soon as the company no longer has a monoply they can expect customers to jump ship).

Google probably has a lot of people in the second category after today, because they did two things right: They updated people, and they wrote a fabulous and public apology.  The apology was probably even more effective than one normally would be because a large company apologised for an outage in a free service, but there are a few other things Google did right:

• They apologised unreservedly, and with an understanding of their users.  There was no “we’re really sorry but it wasn’t our fault” or “we’re really sorry but you shouldn’t be so mad”–they understood why people might be annoyed, and they said sorry.
• They explained the cause of the problem.  Not everyone is going to care about this, but it is good practice to explain for those who do, when writing for a public audience
• They described what they are doing to make sure it doesn’t happen again.
• They subtly reminded users why they chose gmail in the first place, not by saying “we are the most reliable”, but “we’re trying to keep failures rare”.
• The apology was public (right up there on Google’s gmail blog), but not forced on those who didn’t notice the failure.

This is probably the work of Google’s PR people, but dealing with the failures that inevitably happen in life is a really important part of good user experience, and (I swear I don’t work for Google) this is one that Google have done really well.

Google search isn’t just search anymore

I know I’m a bit lot late to the table with this, but Google search isn’t restricted to just searching anymore!  They’ve introduced some browsing tools as well (see the video below for more):

Now, it’s easy to figure out that I am very pro-browsing, and therefore I think it’s great that Google has included these things into their search experience, but I’d like to unpack just why I think browsing is such a good thing (and make a couple of suggestions for extensions of what Google is doing) along the way.

Google has been very pro-search as an information organisation and finding strategy for a long time, their search-don’t-sort appraoch to gmail being one obvious example of this.  It’s completely understandable that this has been Google’s whole approach for so long, after all, search is what they do (and they do it very well).

Search isn’t always the answer though (and if you watch this video of a Google user experience researcher talking about the search options, it is evident that Google knows that).  For one things, humans employ more than just search in their information seeking strategies: the research (PDF) shows that information seeking is generally an interative process that includes searching, browsing, and refinement.  Not only is search not the only approach we use for finding information, but sometimes search isn’t enough on its own: with all the information on the web, it can be hard to know when someone types ‘Placebo’ into a search box whether they want to know about the psychological effects of sugar pills, or whether they’re interested in the British based rock band (this ambiguity applies to any number of terms). Similarly, information seekers may want a particular type of information (for example reviews, or places where a product can be bought), or information from a particular geographic location, time or author, or general subject field.  Also, even with known-item searches (those where the searcher knows exactly what they are looking for, and that it exists somewhere, because they have found a pointer to it or seen it) if the searcher doesn’t remember the exact words that occur in the document, they might not find what they are looking for.

Google’s ‘more search options’ are beginning to deal with this problem.  They allow people to find three specific types of content (reviews, forums and video), they provide suggested search terms, they allow the user to look at results from a specific time, and also see how the search terms popularity has changed over time.  I’m not entirely sure what value the ‘wonder wheel (see below)’ adds, given that the related search terms provide all the wonder wheel terms and more, but  I suppose some people may find the visual presentation useful.

It certainly is heartening, for someone as vested in browsing as I am, to see Google incorporating browsing into their search.  All I want now is to see it expanded:  I want to filter news by topic and country (and standard search results for that matter); when I use Scholar, I want to be able to browse by author or year.  What Google has provided is an excellent start, and I look forward to seeing where this goes in the future.

The new Facebook: Not yet unfriended by users, but close

Facebook recently made a change to their interface that was the subject of outrage for many of their users, inspiring more than 1.7 million to sign a petition to reject it.  Facebook has made some changes to accomodate some of the things users said were problems, but many of the changes (including the slower-to-render rounded corners on pictures) appear to be here to stay.

Initially I was mildly irritated by the new interface, but I put it down to my change aversion (users near-universally hate change, which is why if you’re making major changes, they better help users out substantially).  However, as time has gone on, I have become more irritated with the new interface, not less.  As I see it, there are a few problems with the new interface:

• The proliferation of nonsense in my news feed, without an option to show status updates only.  Yes, I can turn the rubbish from every application off, if I want to, but this requires effort on my part, and will happen every time a new crop of applications becomes popular.  It’s also fairly irritating that I had to go to a help guide to even find out how to do this much, because the mechanism for operating these options is hidden unless you happen to look in the right place at the right time.
• Another side of the same coin: having to edit applications not to publish my life story immediately upon adding them.  I don’t particularly want to bombard my friends with nonsense every time I play a turn in Lexulous.  This means I have to be particularly pro-active in editing the settings for my applications so that they don’t bombard people, and the function for editing this is reasonably difficult to find
• The lack of automatic updating.  I know the old interface didn’t have it, but the trade off for change was supposed to be that we got automatic updating. This change has had no benefit for me, so I resent the fact that the one useful thing that was supposed to happen didn’t.

Do I think no interface should ever change their look and feel?  Absolutely not.  Do I think that Facebook should have done some usability testing before lanching this design?  For sure.  Do I think they did?  Dubious at best.  The Facebook approach, which is one that will always generate negative publicity, is to test their designs on real live users.

According to this blog post, the best way to plan change requires four steps: knowing your customers, listening to them, communicating with them, and responding to them. I think that sounds pretty good–pretty much like doing good user experience, in fact.  And Facebook didn’t do too badly, on a points system–they did warn users (albeit not in a way that most users would notice), and they did respond to some of the complaints users had (albeit not in a way that is really that satisfying).  Unfortunately, you can’t pick and choose which things you want out of that list–good user experience requires all of them.

Nonetheless, I think many (if not most) Facebook users will suck up the changes, even though they don’t like them, because for now, Facebook offers them more than the changes have taken away.  Having said that, though, like I said in my earlier post about Facebook and MySpace, people have personal purposes for using social networking tools.  If Facebook continues to change in a way that breaks that purpose (as the first iteration of these changes did), they will find that users (and thus their advertising dollars) drift away.

What product or service have you used that has slowly worn away at your loyalty until you couldn’t stand it any more?

‘Giving back something broken’ undoes all your good work and then some.

In Stephen Donaldson’s Second Chronicle of Thomas Covenant, the protagonist Covenant points out that ‘there’s only one way to hurt a man who’s lost everything: Give him back something broken’.  While that is certainly melodramatic for the tone of this post, it is something than rings true in user experience.  If your system does something good for users and then takes that something good away without good reason it will make the user angry.

Let me give you an example:  Long time readers of this blog will remember how pleased I was with the pre-pay option available at my local supermarket.  Recently the chain involved in that post has brought out a loyalty card which allows you to collect those four cent petrol vouchers on the card, rather than stuffed into your wallet.  This works particularly well for my partner and I, since we don’t own a car and therefore only rarely purchase petrol; it means when we do purchase petrol we can just hand over the card rather than having to remember to save the dockets and present them at the right time.

So far it all sounds pretty good, right?  The problem is, this card breaks the pre-pay option on the supermarket tills.  You can put all the information in, but when you hand the cashier your loaylty card, it cancels the transaction you have begun and forces the cahsier to start all over.

This problem takes a system that works in favour of the customer, and turns it on its head: the customer does the work of entering their information, only to have that work completely wasted.  It would be less annoying not to have the option to pre-pay in the first place. The supermarket needs to fix this problem, or remove the pre-pay option before they get into more loyalty schemes (as is happening in the next couple of months).

What experiences do you have of systems that were working really well only to turn on you at the last minute?

Human meaning in machine encoding? Thoughts on the semantic web

Tim Berners-Lee, the inventor of the world wide web, outlines his goals for the semantic web in the book he wrote about the development of the web.  I love his dream, that one day we would be able to ask “find out where a baseball game was played today and it was also 22C”.  I just don’t believe it is very likely to happen, for two reasons:

• Effort
• Natural language

The effort question is a really interesting one.  Somewhere along the line, someone has to expend the effort to make human semantic concepts in some way machine encoded, or, alternatively to answer their own questions.  For some, a certain level of machine encoding of the semantics they personally attach to an object (usually in the form of tags) is useful, either for some purpose of their own (information retrieval, for example), or for some social-capital reason (see a more detailed explanation of this here).  However, when a person has only a small amount of information to organise they are considerably less likely to add semantic information to it.

If there is no human being willing to expend the effort to add semantic information, there may be a human being willing to write computer programs to extract such information.  This will be more or less successful dependent on the kind of information to be extracted, and what it is to be extracted from, for example:

• Artist and title information encoded in mp3s are relatively easy to extract, where song lyrics are much harder (unless you combine the artist and title information with a lyric search on the internet).
• It is possible to (with a relatively high degree of success) automatically extract historical date information from free text (full disclosure: that is my work from a previous lifetime).
• Extracting information from a digital picture abut what it is of is quite difficult  (and an open research question), but if the pictures are born-digital photos information about the camera, the date and all sorts of other things is often encoded in them and easy to extract.

This is lesser effort than tagging, because it can be done once and used multiple times, but it is still effort that someone has to expend.

One further approach is, as in this paper (sorry, paywall), leveraging human-created tags to allow machines to do things that look like they understand the semantic web–so in the paper, for example, the author wrote a program that used the way people had combined tags on flickr to unsdersdtand what concrete things (for example tulips) were associated with abstract concepts (for example spring).

In any of the three cases human effort is required to generate the information needed for machines to do the kind of processing Berners-Lee suggests the semantic web ought to be able to do for us.  To actually get people to expend this effort requires them to have a special interest in it, either at a personal level (as with tagging) or a research interest (as with automatic extraction programs.  I think this effort is a major impediment to more widespread “semantic web” applications and uses.

The natural language question is also a barrier, and a much more usability centred barrier.  Even if we could get evertyhing tagged up, either by human hands or automatically, how people would then ask this semantic web to answer their questions is an open question.  glenn, an acquaintance of mine who works in the field (and like his name spelt wiht a lower case ‘g’) thinks that we need query languages, and I am inclined to agree.  If natural language searching on the free-text internet fails (paywall again, sorry), it will surely fail in any kind of structured environment.  Unfortunately, users are known to do poorly with Boolean search, and it is reasonable to expect that other query languages would porduce similarly bad results, so even if the web was tagged up, it may still be fairly difficult for the average user to ask the question Berners-Lee posed in his book.

I think tagging is great, because it imbues objects with personal meaning, and allows people to find things more easily.  I have yet to see evidence of a truly workable (and by implication usable) semantic web, though, and as such I don’t believe people will be able to answer questions about baseball games at 22C for some time to come. I also believe that even when it is possible to answer these soorts of questions, it will be not because of advanced tagging of web-pages, but more form advanced text processing by search engines–and that isn’t the semantic web, it’s search engine companies prioritising user experience.

Culture, gender, and why Kartoo’s interface isn’t inclusive.

I’m not going to write about Kartoo’s interface in general in this post, beyond saying that the clustering is poor, the seaqrch results are uninspiring, the visual cues are unhelpful at best (and an accessibility problem at worst–those little moving stars could trigger seizures in someone with a seizure disorder).  Basically, Kartoo isn’t a very good search engine, ad it doesn’t have a very good interface.  Many of my colleagues have said much the same thing, and I don’t need to re-hash it here.

Since the 23 Things has started, however, the interface has changed.  Many weeks ago, when I looked at Kartoo, there was a graphic of a windsurfing genie, which I found to be uncomfortable at best: It had no relation to anything else to do with the site, and played on cultural stereotypes, which potentially alienates large groups of users either by offending them, or by playing on a metaphor they do not understand and cannot engage with (in this case I think the metaphor was supposed to mean that this magical being could help you surf the web).

With the change in interface, however, the genie has been moved off his windsurf board and into the corner of the interface, and a new character has been introduced:

Yes, that’s right, an exoticised image of a woman with a figure designed to be appealing to the male gaze, and wearing very little clothing.  What you can’t see from this still image is that the light behind her torso pulsates as you wait for your search results to load.  This is insensitive at best, and sexist and racist at worst.  It is likely to offend a wide range of users, from feminists to those who see the female body as sacred and something that should be covered modestly (as is the case in many religions). I’m sure it is supposed to be ‘fun’, but in fact a large number of users (including yours truly) will see it as a sign that Kartoo was not designed to appeal to them, and has little to offer them.  Given that it does not add anything helpful to the user experience (for example the pulsating light does not pulsate faster to tell you your results are nearly ready), this can be seen as a serious misstep by Kartoo in terms of the user experience (unless they only want to appeal to a certain demographic).

This example really highlights the risks involved in using metaphors, particularly culturally loaded ones.  Many cultures understand metaphors quite differently than one would expect, for example the Maori (minority indigenous gorup in New Zealand) understanding of a ‘library’ is quite different to the New Zealand European understanding, and acts as a barrier to Maori accessing useful, relevant information ind a digital library (as reported in Duncker, 2002).  Metaphor can be very useful if used carefully, for example the desktop metaphor was one of the driving factors behind usable personal computing.  However, if ill-used, metaphor and cultural artefacts can confuse, offend, and actively drive away users.  Have you ever been offended or confused by a metaphor that didn’t fit your understanding or cultural values?

VuFind: An interesting case of open source usability

We all know that library users are consistently frustrated with library systems, and cannot find what they want, particularly since the advent of Google (PDF). Some academics berate and despair of their students’ information seeking practices, and claim that Google is ruining young minds. In my opinion, as I have stated before, berating students (and Google) is going after the wrong target. It is human nature to maximise benefits while minimising effort, and for many students the time they will spendf searching a number of interfaces for relevant resources–particularly when the interfaces are confusing, archaic, and unhelpful–is simply better spent reading the resources they find on Google, and writing their assignments. The only way to change this “satisificng” approach and reveal the vast range of library resources available to our students is to make them findable through interfaces that do not confuse or humiliate users, and do not require a librarian to operate. While libraries can’t expect to compete with Google while they are buying information from a multitude of vendors that do not have standardised search results or formats, library search interfaces can offer some additional features (such as metadata-based faceting and primary browsing) that Google doesn’t offer–and if the information is better, or gets better results (like higher grades) that will also prove an incentive to use library interfaces.

Typicall I expect library catalogues to be ugly and cantankerous, I see that as the price I pay for finding the books I want(and don’t even get me started on finding journal articles–usually I start with Google Scholar). This is why, when I looked at VuFind on the National Library web site, I was so impressed with it: it is clean, attractive, and very usable:

• It searches more than one type of holding; my search results included books, online resources, and microfilm. This is much closer to the “one stop shop” expectations that users have than any library system I have used in the past.
• I can choose between my search results based on metadata facets–that is, I can choose books, or works by a certain author, or items from a specific subject. This means that single term searches are much more likely to be successful, as I can easily disambiguate my search and bring the results that are most relevant to me to the top
• Results are relevance ranked (don’t laugh, some library systems don’t do this). This feature is the one that has given Google search engine market dominance; their excellent relevance ranking meant that people found what they were looking for in the one to two pages of results they typically view.

These are just a few of the features that make VuFind feel like a breath of fresh air. Another thing that is unusual about VuFind, though, and one that makes it especially exciting to me, is th fact that it is open source. This basically means that you can get the software for free (though if you want support you will generally pay for it), and that if you want to change something about it, all you need is a willing programmer.

Open source software provides large scope for improving usability of software locally, because unusable features can be altered, however generally speaking open source software is not as usable as its “closed source” or commercial counterparts (a problem that is recognised, but not well handled, in the open source community). Dave Nichols and Mike Twidale, colleagues of mine, have long been interested in usability in open source software (and indeed how to open source usability bug reporting). In a 2003 paper they published (which anyone interested in open source or usability should read), they suggested several reasons why open source software might have usability problems:

• Open source communities, famous for comments like “RTFM” (read the **&%@& manual), are not generally welcoming to experts from other backgrounds, as usability experts often are
• Design for usability generally has to start before design for coding
• Open source communities are populated by programmers, who generally cannot see the problems that users with a lesser understanding of computers might have
• Open source software programming is often done to meet a need of the programmer, and as mentioned above, programmers have very different user interface needs to other users
• Design by committee and software bloat are not usually good for usability, and open source software is prone to both

In another paper on open source usability, Dave and Mike noted that it can be hard to report usability bugs in the same way as technical bugs, and that open source interfaces may be prevented from innovating by playing “catch up” with their commercial counterparts.

So VuFind is positively fascinating for its usability, both among library systems (though some of the newer commercial systems look interesting), and among open source projects (Koha is similarly fascinatingly usable and open source). Why is it that VuFind is such an exception to the rules?

• It was created by a library, under one umbrella, and not in a typical open source community. Being under a single umbrella demonstrably helps open source projects’ usability (Dave and Mike again, there), largely by ameliorating design by committee and imposing some order on the process. This will also have meant that the community was different — VuFind’s website comments that it was developed “by libraries“, and thus not just programmers, meaning that feedback from other disciplines was likely welcome
• Typical library system websites (though again, I can’t speak for some of the newer ones) are not effective for users, so VuFind didn’t have to play interface “catch up”
• VuFind was developed “for libraries” not “for programmers”
• It looks suspiciously (to me) like VuFind might have had a formal usability process, though I can’t find any evidence for this one way or another

In the end, whatever the specific differences are, VuFind is not just exciting in terms of its user experience, but fascinating, and an exemplar of how to do usability in an open source project. I don’t know if it is the way we will go with our discovery layer (and not having seen many of the other possibilities, I can’t comment on whether it is the way we should go either), but it certainly is a fascinating project, and I will be watching it further.