Why users like federated search (even though they shouldn’t)

‘Federated Search’ is a library term, it refers to search engines that search a variety of library databases (things that contain journal articles, conference papers and the like) and combine the results in some way to be presented to the user.

Federated searching is a somewhat fraught topic in libraries; many librarians don’t like federated searching and are hestitant to recommend it to library users.  This reluctance is not without good reason–federated search is inferior in many ways to using native database search interfaces, including problems with relevance ranking, the false appearance of comprehensiveness, and the inadequate de-duplication that many offer.  On the other side of this, federated search offers the holy grail of library searching: a single search box (well, almost–federated search usually doesn’t include the local catalogue, though sometimes it does, as in this example at UNSW).  The single search box is seen as being “like Google” in offering users a lot of different content from one search–and even has a slight edge over Google scholar in that search results will usually reflect more closely than Google Scholar which results a searcher can actually access.

Federated search has some issues that would normally be pretty big rpoblems from a user perspective too:

• The relevance ranking doesn’t really work. Because federated search is pulling in material from a range of sources, each of which use different approaches to relevance ranking and different metrics to express a rank.  Any combination of these results is likely to produces flawed relevance ranking.  This means that often, the most relevant results will not be in the magical first couple of pages.
• Federated search is very, very slow.  Again, because federated search is searching a number of remote databases and then applying some metric to combine results before these are presented to the user, federated search is very slow. Typically users are unhappy with slow response times, so this should be a real problem for users.

So, we know librarians are often hesitant about recommending federated search, and that users have every reason not to like it…and yet study after study shows that users do like and use federated search.  So why is federated search so popular?

• One stop shopping: Federated search offers users a one-stop shop, and even though they know it isn’t as good, they will often use it anyway.
• Time saving: Despite the long load time for search results, users know they will save themselves time (and likely frustration) by visiting only a single site.
• Search syntax: Search syntax varies slightly from site to site, and federated search allows users to forgo learning the variationson syntax required by individual databases.  Given that we know boolean searching is hard (sorry, paywall), it is easy to surmise that learning less about it is considered a good thing by users.
• Low user expectations: Users expect library systems to be slow and clunky, so their expectations of federated search are lower than they would be for other web-based services.

Users’ willingness to use a system we don’t expect them to like is an object lesson in how usability principles are not entirely universal: Occasionally users will tolerate unusable systems over more-usable ones because the end result is still a faster and easier user experience.

So, does users’ willingness to put up with the limitations of federated search mean we should stop striving for anything better? I don’t think so.  I think that as web technology improves, users will have less tolerance for slow and clunky systems.  We’ve already seen this at Swinburne with the library catalogue–while it hasn’t changed our users surveys show increasing levels of dissatisfaction as a result of user expectations that have been raised by their interactions with other systems.  I don’t believe that users are going to be willing to individually visit library databases in the future any more than they are now; even Google is meshing different kinds of data in its search results.  I believe there is real benefit to be had for librarians and library users alike in making headway in one-stop searching, I’m very much looking forward to seeing Primo Central and Summon (the next generation of federated search, where metadata is locally indexed making search faster and relevance ranking better) in action.  In the meantime though?  Users still like federated search, even though it is slow and awkward.

How to deal with ‘too much information’: where should we put search refinement facets?

Swinburne Library is in the process of making some changes; we’re replacing our library system with a fancy new one, and as the user-experience-person-in-situ it is up to me to make suggestions for the search and discovery interface our users will see.  Some of those decisions I will blog about here, and search facet placement is one of them.

Search facets are one of the search tools that I think will be most instrumental in making stuff easier to find (and the OCLC report (PDF) on user expectations vs. librarian expectations suggests library users feel the same way). Facets are the little categories you see on search interfaces that let you narrow down your search results to things that are more relevant to you; they started out in tools with well-defined metadata (like eBay and Amazon, and even some of the newer library systems) and they are slowly working their way into searches with less-well-defined metadata, like Google.

With anything new like this, though, you have to figure out where in the search interface to put it.  So far I have seen facets placed to the left of search results:

to the right of search results:

and below search results:

At Swinburne, we talked a bit about facet placement, and in all likelihood ours will be on the left.

So, what are the arguments for and against each position?

• Facets below search results: When facets are below search results, they don’t distract the user when they are viewing search results, which is a good thing.  However, given that the vast majority of users don’t scroll all the way down, and only look at the first couple of pages of search results (and they look more at the first results on these pages), placing facets below search result is pretty likely to mean that users don’t see them or use them.  This likelihood is reinforced by the fact that this is a significantly uncommon location for facets, so users won’t think to look for them here.
• Facets to the right of search results: From a user-centred-design purist standpoint, in my opinion the right-hand position for facets is probably the best in an interface where the language is read from left to right.  This position means that user see search result first, and then facets if the search results don’t contain anything immediately useful.  Given the number of commonly used interfaces that put facets on the left, however, this could be a risky proposal.
• Facets to the left of search results: This is what Google have gone with (possibly because their advertising is on the right).  It is also common in other commonly-used information seeking interfaces, such as eBay, Dymocks (in Australia) and Amazon (for the US and the UK). Use of these interfaces will train users to look to the left for facets; and it would seem that at least a small sample of users have already developed this preference for left-hand search facets.

Swinburne has a real opportunity with this project to provide a search interface for our users that is not “slow motion search, typical library“; however, to do this we must pay as much attention as we can to our users.  Putting the search facets to the left is just one of the decisions we will make with the users in mind, and I hope to blog about more in the future.

Google search isn’t just search anymore

I know I’m a bit lot late to the table with this, but Google search isn’t restricted to just searching anymore!  They’ve introduced some browsing tools as well (see the video below for more):

Now, it’s easy to figure out that I am very pro-browsing, and therefore I think it’s great that Google has included these things into their search experience, but I’d like to unpack just why I think browsing is such a good thing (and make a couple of suggestions for extensions of what Google is doing) along the way.

Google has been very pro-search as an information organisation and finding strategy for a long time, their search-don’t-sort appraoch to gmail being one obvious example of this.  It’s completely understandable that this has been Google’s whole approach for so long, after all, search is what they do (and they do it very well).

Search isn’t always the answer though (and if you watch this video of a Google user experience researcher talking about the search options, it is evident that Google knows that).  For one things, humans employ more than just search in their information seeking strategies: the research (PDF) shows that information seeking is generally an interative process that includes searching, browsing, and refinement.  Not only is search not the only approach we use for finding information, but sometimes search isn’t enough on its own: with all the information on the web, it can be hard to know when someone types ‘Placebo’ into a search box whether they want to know about the psychological effects of sugar pills, or whether they’re interested in the British based rock band (this ambiguity applies to any number of terms). Similarly, information seekers may want a particular type of information (for example reviews, or places where a product can be bought), or information from a particular geographic location, time or author, or general subject field.  Also, even with known-item searches (those where the searcher knows exactly what they are looking for, and that it exists somewhere, because they have found a pointer to it or seen it) if the searcher doesn’t remember the exact words that occur in the document, they might not find what they are looking for.

Google’s ‘more search options’ are beginning to deal with this problem.  They allow people to find three specific types of content (reviews, forums and video), they provide suggested search terms, they allow the user to look at results from a specific time, and also see how the search terms popularity has changed over time.  I’m not entirely sure what value the ‘wonder wheel (see below)’ adds, given that the related search terms provide all the wonder wheel terms and more, but  I suppose some people may find the visual presentation useful.

It certainly is heartening, for someone as vested in browsing as I am, to see Google incorporating browsing into their search.  All I want now is to see it expanded:  I want to filter news by topic and country (and standard search results for that matter); when I use Scholar, I want to be able to browse by author or year.  What Google has provided is an excellent start, and I look forward to seeing where this goes in the future.

Boolean search made easy to understand?

Recently a colleague sent me a link to a little tool called ‘Boolify‘. Boolify is a visual representation of Boolean search and is meant to be an educational tool to help people learn how boolean search terms work together.

Boolean logic (and one of its common applications, Boolean search) is a common ground field between computer scientists and librarians, particularly those who try to teach it. Boolean search is hard to understand for most people, but if users could understand it it would be one of the most powerful tools in their search toolbox (particularly when they are using a search term that has more than one meaning). Good search engine interfaces try and get around it by offering a more ‘natural language’ advanced search interface that lets users do some Boolean searching (see for example the Google advanced search interface below), but the true power and flexibility of real Boolean searching is somewhat lost even in these interfaces.

Google’s advanced search interface

Now it’s debatable whether the power users lose in an interface like this is even that much of a problem, given that the vast majority of users enter short queries and don’t use advanced search. Arguably there is a chicken-and-egg problem here; users may not use advanced search because (in many cases) it is too hard, meaning they never learn to use it, meaning it is too hard…Whatever came first, I think the answer is that users can usually (at least with Google) find the information they want (or something close enough) without having to get into Boolean logic, and therefore they don’t learn it.

There are some people, though, for who a good grasp of boolean logic is essential. Computer programmers are in this group, as are the librarians to whom people turn when their own searches fail. Potentially students and researchers should have some understanding of Boolean search as well, though I would generally argue for better search interfaces for these groups. So, given the difficulty of teaching Boolean logic (and having taught first year computer science I have first hand experience of this difficulty), anything that would help make it easier would be great.

At first glance, Boolify looks like just such a tool. It would be more awesome if it had a predefined results pool to play with (as well as providing an internet search), so students could see how query reformulation affects what is included in the results pool, but that is pie in the sky, and this thing looks like a good first step. It has puzzle pieces that show how you can interlock different bits of a boolean search, and the horizontal and the vertical can act like brackets in a real search term–the thing at the top is in the innermost brackets, and so on down.

Sadly, though, the promises Boolify makes are not delivered upon. Despite the puzzle pieces providing every promise of being interlockable, queries can only be developed in a linear fashion, and so the pciture below is what happens when you try to develop the query ((libraries OR museums) NOT gallerys) AND ( user experience OR usability). It simply isn’t possible to do, despite the puzzle pieces having every visual and logical affordance needed–you can on develop queries in a line, which means this tool does not represent the full power of Boolean logic.

Trying to develop a complex Boolean query using Boolify

Usually I would not be so hard on a tool designed to teach something difficult in a novel way, but this tool is a poor teaching aid. It doesn’t allow users to do what it looks like they can do by clicking bits of puzzle in anywhere–thus not showing the true power of Boolean queries, and having a significant usability problem. It also (and I think this is more important) doesn’t let users reformulate queries by moving puzzle pieces around, thus denying a real opportunity to learn how Boolean operators work together by reformulating queries and seeing what happens to the results. I can honestly say that until this tool fulfilled more of its promise, I wouldn’t use it teaching–it is bright and colourful, but it doesn’t bring much more than that to the table.

So, what does everyone else think? Should we leave Boolean logic to the geeks and logicians? Who needs to know Boolean logc, and how should we teach it?

The ‘Google effect’: A trend toward mediocrity, or away from it?

Today, there is a special section of the Guardian on digital academic libraries. It covers a wide range of perspectives, and is probably worth a read if you’re interested in academic libraries, digitization, digital preservation, or student habits.

I have to take issue, though, with ‘Academia’s big guns fight the ‘Google Effect”’. The definition of ‘Google effect’ given in this article, and apparently coined by one Tara Brabazon, is ‘a tendency towards mediocrity’. The article goes on to accuse students of information illiteracy, and point out that they like to use Google for everything, which gives them less-than-academic results. Attempts to provide good academic-resource search engines are touched upon, as is Google Scholar (which is ‘acceptable’, but ‘too broad’ according to Professor Brabazon).

There is actually an excellent study (see ‘British library and JISC’ on this page) about information literacy skills of the current generation of university students which is the basis for much of another article in the series. That study found that undergraduates are not necessarily as information literate as they are perceived to be, and that they use ’shallow’ searching and don’t really read online (but neither, necessarily, do their older counterparts).

I’m not arguing with the results of that study — it seems pretty sound to me. I suspect, however, that the thing that has changed with the ‘Google generation’, though, is not actually their information literacy, but their ability access information without strong information literacy skills and/or the help of a librarian. Google, having a very simple user interface, and great results ranking, has made it easy for the average person to find answers to their questions on the internet. It has also shown users that it isn’t necessary to jump through hoops, understand boolean search, or wade through pages of results to find information.

The mediocrity Professor Brabazon has termed ‘the Google effect’ arguably does not apply so much to her students, who I suspect are much the same as always, but to the information interfaces they are forced to use to locate scholarly materials. It is understandable, I think, that students prefer to spend time on their assignments reading and writing, and now they have tools which to them appear to let them bypass the cumbersome, splintered interfaces of academic journals. There is an information literacy problem here, but it is far from “whippersnappers these days not knowing how to use our journal databases”; it is the twofold problem of the proliferation of self-published non-authoritative easily accessible material that is the internet, and the vastly superior search technologies available to sift through that material.

If Professor Brabazon and her colleagues want to encourage young people to use scholarly resources the answer is not to lambast them for being mediorce (when likely they are no different to those who have come before them), nor to throw up their hands in disgust; the answer is to improve search interfaces and online access to academic materials so they can compete with Google, or (in my opinion the more likely solution) encourage widespread use of Google Scholar.

The ‘Google effect’ as I see it is not ‘a tendency toward mediocrity’ in students, it is an exposure of the dire mediocrity of the interfaces and search process for academic material. Google has democratized information searching, and made it possible for the average untrained adult to find information — academic publishers and other information providers need to catch up by providing seamless, well-ranked searches (again most likely through Google Scholar), and at least for those who are subscribers to their resources (either individually or through their institution)* make the results available with a single click. The alternative to this will not be improved information literacy skills, people are not going to learn something more difficult if they believe the tools they have will do an adequate job. I hope the end result of the Google effect will be a trend away from mediocrity–the mediocrity of academic information interfaces–and toward usable information search interfaces for all kinds of materials.

*Agruably, these results should be more widely available than that, but this post is not about the merits of open access, and academic publishers are not likely to change their access model so radically any time soon.

Podcasts: an alternative, not a replacement

This post is another of the reasons why I have been ignoring this blog: I struggled to get into podcasts at all. According to this “learning styles” test (which may or may not mean anything), I should not be averse to receiving my information in an auditory/verbal format — I fall right in the middle of the verbal/visual scale (and indeed, I often listen to the TV while I am surfing the net, cook while talking on the phone, and listen to music at work). And yet, somehow, podcasts feel cumbersome and inconvenient. Nonetheless, I managed to find this podcast describing usability testing methods and when to use them (mp3, time unknown — less than 10 minutes, size unknown), and this one about folksonomies, taxonomies and metadata (an interview with Karen Loasby from the BBC — mp3, 18 minutes and 20 seconds, 8.6MB), both from the User Experience Podcast.

So why did I find podcasts so hard? For me, I think the problem is affordances (the properties of an object that dictate what you might do with it). Podcasts are hard to search for, and it is almost impossible to tell before you listen to a podcast from an unknown creator whether you will find it interesting or not (partly because the blurbs written about podcasts are near-universally unhelpful). Of course, if I were using podcasts “correctly” (i.e. finding ones I liked and subscribing to them) this would not be such a problem but I don’t have time to listen to podcasts — and this is another affordance problem: I can read very, very quickly, and I would absorb most of the information in a podcast much faster from reading it than listening to it. What’s more, if I am listening while I sit in front of a computer, I am inclined to attempt to do other work while listening, and then I lose the thread of the podcast.

Away from the computer, though, I could believe that podcasts might come into their own; in theory I could download interesting radio shows or documentary podcasts onto my shuffle and listen to them at a time of my choosing — even away from the internet (for me, the example that springs to mind is while travelling in a car — I get very carsick if I read as a passenger). Podcasts are also useful where they contain information that users already know exists — podcasts of lectures would make a great alternative to lecture notes for visually impaired students, or those with reading difficulties, or those who simply learn better from auditory material. They could also be a lightweight way for me to catch up on the things I necessarily miss at conferences because something I want is going on in another room — these “known item” uses sidestep the search problem.

So in terms of user experience, when are podcasts a good idea?

• When your users generally have a high speed internet connection, because podcasts are much larger than text files and users hate waiting for content to download
• Only if you are prepared to label each with a good blurb, length, and file size
• When the content is something that natively appears in an auditory format, such as lectures, radio broadcasts, conference presentations, concerts, etc.
• When you know you have users who find text hard to access, and you want to offer an alternative to a screen reader
• When your users already know your content exists so they don’t have to search for it using a non-google search interface (while iTunes’ podcast search is fairly effective it also means downloading and installing 3rd party software, which your users may not be able to do at work, or in the lab, for example).
• When you are doing it purely for your own enjoyment, and (like so many bloggers) don’t really mind what audience you have, if any.

Used well, podcasts can almost certainly make for better user experiences for students, conference-goers, radio-listeners and the like, however for vital information, they should be an alternative way to get information, not the only way. And me? I think I prefer to read my blogs, thanks anyway.