Dana’s user experience blog

Today, there is a special section of the Guardian on digital academic libraries. It covers a wide range of perspectives, and is probably worth a read if you’re interested in academic libraries, digitization, digital preservation, or student habits.

I have to take issue, though, with ‘Academia’s big guns fight the ‘Google Effect”. The definition of ‘Google effect’ given in this article, and apparently coined by one Tara Brabazon, is ‘a tendency towards mediocrity’. The article goes on to accuse students of information illiteracy, and point out that they like to use Google for everything, which gives them less-than-academic results. Attempts to provide good academic-resource search engines are touched upon, as is Google Scholar (which is ‘acceptable’, but ‘too broad’ according to Professor Brabazon.

There is actually an excellent study (see ‘British library and JISC’ on this page) about information literacy skills of the current generation of university students which is the basis for much of another article in the series. That study found that undergraduates are not necessarily as information literate as they are perceived to be, and that they use “shallow” searching and don’t really read online (but neither, necessarily, do their older counterparts).

I’m not arguing with the results of that study — it seems pretty sound to me. I suspect, however, that the thing that has changed with the ‘Google generation’, though, is not actually their information literacy, but their ability access information without strong information literacy skills and/or the help of a librarian. Google, having a very simple user interface, and great results ranking, has made it easy for the average person to find answers to their questions on the internet. It has also shown users that it isn’t necessary to jump through hoops, understand boolean search, or wade through pages of results to find information.

The mediocrity Professor Brabazon has termed the Google effect arguably does not apply so much to her students, who I suspect are much the same as always, but to the information interfaces they are forced to use to locate scholarly materials. It is understandable, I think, that students prefer to spend time on their assignments reading and writing, and now they have tools which to them appear to let them bypass the cumbersome, splintered interfaces of academic journals. There is an information literacy problem here, but it is far from “whippersnappers these days not knowing how to use our journal databases”; it is the twofold problem of the proliferation of self-published non-authoritative easily accessible material that is the internet, and the vastly superior search technologies available to sift through that material.

If Professor Brabazon and her colleagues want to encourage young people to use scholarly resources the answer is not to lambast them for being mediorce (when likely they are no different to those who have come before them), nor to throw up their hands in disgust; the answer is to improve search interfaces and online access to academic materials so they can compete with Google, or (in my opinion the more likely solution) encourage widespread use of Google Scholar.

The ‘Google effect’ as I see it is not ‘a tendency toward mediocrity’ in students, it is an exposure of the dire mediocrity of the interfaces and search process for academic material. Google has democratized information searching, and made it possible for the average untrained adult to find information — academic publishers and other information providers need to catch up by providing seamless, well-ranked searches (again most likely through Google Scholar), and at least for those who are subscribers to their resources (either individually or through their institution)* make the results available with a single click. The alternative to this will not be improved information literacy skills, people are not going to learn something more difficult if they believe the tools they have will do an adequate job. I hope the end result of the Google effect will be a trend away from mediocrity–the mediocrity of academic information interfaces–and toward usable information search interfaces for all kinds of materials.

*Agruably, these results should be more widely available than that, but this post is not about the merits of open access, and academic publishers are not likely to change their access model so radically any time soon.

I was going to write about the vagaries of public transportation, and in particular air travel, today, but I am planning at least two further round trips to New Zealand in the near-ish future, and so I shall wait to confirm my newly formed opinions (and hopefully simmer down some) before launching myself on the poor user experiences involved in that particular endeavour. Instead, I want to talk about something underpinning good usability (and to a certain extent, user experience): Usability assessment.

So far on this blog, I have talked endlessly about user experiences (and to a certain extent, usability) with little reference to how we know the things we know about users of any given system. The way we know anything about users is assessment, either previous assessment that has contributed to a body of knowledge that allows us to make generalisations about “the user” (indeed “the user of any system”), or new assessments that answer specific questions about specific user groups and systems.

There are numerous ways of assessing usability (combined with that background knowledge about “the user” mentioned above, knowing these methods and being able to apply them appropriately is what makes a usability professional), but to discuss each type is well beyond the scope of this post. What I want to talk about here is good usability assessment — and because a lot of the work I have done recently has been with surveys, I’m going to use those as a reference point.

Given that usability assessment informs design and development, our understanding of our users, and (sometimes) the body of general knowledge about users, it’s a good idea to get assessment results as right as possible. This imperative is compounded by the fact that usability needs to do more than just make users happier, it also needs to be cost effective (though to be fair, the barrier for this can be quite low — a representative of a large firm I once did some consulting for told me that every time users ‘phoned that company’s helpline, it cost the company a minimum of $10 — at that rate it doesn’t take many users who don’t call to pay off a few hundred dollars worth of usability consulting). There are basically three steps to making sure usability assessment results are useful:

• Doing the right tests: This seems obvious, but it is worth mentioning all the same. Just like a chest x-ray can’t tell you if you have a cracked kneecap, a lab-based usability study can’t tell you how software (or any system) gets used in the real world (similarly, usage studies can’t tell you why people do the things they do, and observational studies can’t tell you whether you should make that button blue or purple). Which test is right depends on what you’re trying to find out, how much money and time you have, what stage of development you’re at, and who your users are. The other part of doing the right test is knowing what things to investigate; it’s all very well to assess the usability of your homepage (for example), but if 90% of your customers access your service via the telephone it is the usability of your phone system you should be testing.
• Testing the right users: This is more subtle than it seems. Testing on members of the development team is clearly not going to be effective, but there is more to it than that. Let’s examine how survey participants are chosen:
  • Where you advertise will affect the makeup of your participant population; for example if you advertise a library survey only in the physical library, only those who come to the library in person are likely to see the ad.
  • Participants of a public survey are, to a certain extent, self-selecting. Those who feel they have something to say on a topic will be more likely to start a survey, and more likely to complete it. These effects can be ameliorated to a certain extent by offering rewards, and using broadly inclusive language in the advertising and survey wording can help, but it is important to still recognise this bias.
  • Survey timing is important. Running a survey during exam time at a university may attract a disproportionate number of procrastinators for example, while running it during summer term can only give reliable information about summer school attendants, and not the population at large.
  • How you collect your surveys is important. Paper-based surveys have a much lower response rate than online surveys (and skew the results toward highly motivated participants — usually those who hold strong opinions). Collecting results online in a population which includes less tech-savvy participants (as older adults often are), however, will skew the results toward more technically able users. Decisions have to be made with your whole user population in mind.

  While it is almost certainly impossible to test all users for any given system, and in any heterogeneous population it is difficult to even get a truly representative sample, it is important to try to minimise sample bias (and understand and acknowledge it, where it happens).

• Making the test usable: This one is where it is easy to make mistakes, especially with surveys. I recently saw a survey where the participant was given a list of statements and asked first how important the item was, and then how well they felt the system met their needs. Given that the goal of a survey participant is usually to give their opinion first and foremost, I bet a lot of participants will fill this out wrong. Using language users of your system don’t understand will also reduce the reliability of your results — instead of asking how happy users are with their ISP, it might be better advised to ask how happy they are with their internet service. One final (and insidious) example of poor (in this instance) survey usability is bias — letting the phrasing of a question influence the answer (I’ve made this mistake recently myself, asking what users would call a service they used to make contact with the library, and repeating the word contact in one of the options given).

Usability assessment is a tool that can help make your users happier, and possibly reduce your costs. Like anything, though, it only works if you get it right.

Gordon gave me the idea for this post, while venting his frustrations about eBooks (someone needs to tell me whether that capitalization ought to be there — I never really know). His specific irritation was that he could not print more than four pages, thus meaning that the e-version of a real book one of his lecturers has set as required reading does not do the same job as a physical copy would (and the physical copy is on back order). What, asks Gordon, is the point of these things?

To me, it seems that eBooks are a bit like Wikipedia (only more authoritative): They’re good for getting short sharp bursts of information while you’re already online. My library’s subscription to Safari Techbooks saved me no small amount of time during the tail end of my masters; instead of having a recall war with someone over the only book our library had on the (then relatively new) DHTML, I was able to read about it, with code examples, online and just in time. EBooks are probably good for all sorts of things like that, from physics equations to Shakespeare’s 116th sonnet. If you want to read ‘King Lear’ or ‘A Brief History of Time’, though, forget it. Buy the book, if you can;t get it from your library.

The (apparent) reason why eBooks are so awful seems to be to me a triumph of copyright over common sense. Copyright concerns seem to be the reason why eBooks are neither fish nor fowl, neither electronic nor book. eBooks, at least the ones I have seen at Swinburne, are printed in a PDF-like format, making for worse on-screen reading, longer load times, and a distinct lack of the rich hyperlinking that adds value to online reference content. I can only think of 3 reasons why PDFs are being used here instead of natively online formats:

• Because you can lock a PDF and prevent someone copying the text
• Because the books are natively created in PDF-like form and the publisher sees no need to convert them
• To present the Greek symbols so often found in mathematics textbooks.

The only reason of those, that is really good enough, is the last one, and we can only hope that text-presentation technologies catch up with need soon enough that we won’t be dependent on preformatted text for too much longer (yes, theoretically unicode can handle it, but too often web browsers fail to adequately interpret unicode, resulting in either garbled nonsense, or that little square box thing). Not only do eBooks fail at being electronic, though, they fail at being books. They can’t be read without a web connection, the amount you can print is dictated by an online publisher and embedded in the technology, rather than reflecting copyright law, and all the wonderful affordances of a regular book — annotating, falling open at a frequently used page, coming back to where you left off and prolonged comfortable reading — are not available.

Despite the poor usability and poor readability of online book, though, I think it is important that we continue to have the available. The web statistics show that eBooks are quite heavily used, and a recent survey of our students has demonstrated that they like and expect to be able to access their textbooks online. Are our eBooks popular because the next generation is different? Possibly. My guess, though, is that most students are using eBooks for reference, to avoid purchasing (or carrying about) hard copy textbooks. As for me? I’ll read more eBooks when the usability of the electronic interface, and the complete unwillingness on the part of publishers to publish in an online readable format changes.

Addendum 18-2-2008: My colleague Tony has made some excellent points in the comments on this post that need raising here: eBooks have a significant advantage over traditional books in storage space and price, and are a hugely valuable resource for distance students.  Not only that, but our eBook provider (EBL) is very generous in terms of the printing users are allowed — 20%, considerably more than copyright in Australia; it seems Swinburne users have been facing some technical hitches at our end in this regard.  Tony’s most important point, though, was one I missed because I am used to libraries (and I should have caught this): It is not necessarily easy to find a book on the shelves of a library, or find the right information in that book, so eBooks may have the advantage in this regard.  All these points are reasons to continue to purchase eBooks, but also to manage expectations about what they are, so their users get the most out of them and not the least.

Computers, unlike any tool in history, seem to be the butt of a number of jokes. Sometimes it is about the stupidity of they way a computer talks to us:

Sometimes it is a common irritation with a feature designed to be helpful:

In a certain type of siege mentality, even the images of someone finally losing it with their computer are humourous:

Sadly, though, the frustrations our computers create for us are not always funny. Sometimes, as in this 2003 case, they are dangerous — a man shot his computer four times in a fit of rage that he could not do what he wanted to do. Anyone who has done a course on human computer interaction will almost certainly have seen the beer taps used to replace levers in a nuclear power plant to make it easier for workers to distinguish “the big red button” and not press it.

While the cost of bad user interfaces is not usually counted in danger, it is incredibly pervasive. A 2001 survey of 6000 computer users showed an average of 5.1 hours wasted per week grappling with computer problems, and frankly, I don’t think much has changed since then. Think of the last time you called a call centre, or asked a retailer if they could order something they didn’t have in store. How many times did they apologise for having difficulty with the computer, and taking so long? How much time did you spend standing there, or hanging on the line? Were you, as I was yesterday in a shoe store, sympathetic, and slightly embarrassed?

We create jokes like those above because computers make us feel bad — they make us angry, they make us feel stupid, they waste our time, and the things designed to help us (like Clippy) are often insulting. User interfaces designed with users in mind, like iPods and Nokia cellphones engender tremendous loyalty because they don’t create those negative feelings. While I am sure that their competitors may be technically just as brilliant, I have never used them because I like my iPod and my Nokia, and because user experience is at least as much who I am as what I do, I take poor user interfaces very personally.

There are two things everyone who uses computers can do to break out of the siege of wasted time, danger, and outright rage that computers impose:

• Buy the products that irritate you the least. Look for user reviews and see whether the company has a user focus (right now there is a radio ad for navman that mentions usability specifically). When a product frustrates you, if possible, complain to the company that made it. If there is a customer service line, call it — this costs the company money (as poor usability should).
• If you are designing a product or service, once you have made sure that that product or service is useful, make sure it is also usable. Being useful means you haven’t wasted money on a product no-one will use, but usability will save everyone time and blood pressure.

Computer humour is fun, but it’s black humour. I’d rather not need to resort to humour to cope with my daily interactions with, what are (when all is said and done) tools.

This post is another of the reasons why I have been ignoring this blog: I struggled to get into podcasts at all. According to this “learning styles” test (which may or may not mean anything), I should not be averse to receiving my information in an auditory/verbal format — I fall right in the middle of the verbal/visual scale (and indeed, I often listen to the TV while I am surfing the net, cook while talking on the phone, and listen to music at work). And yet, somehow, podcasts feel cumbersome and inconvenient. Nonetheless, I managed to find this podcast describing usability testing methods and when to use them (mp3, time unknown — less than 10 minutes, size unknown), and this one about folksonomies, taxonomies and metadata (an interview with Karen Loasby from the BBC — mp3, 18 minutes and 20 seconds, 8.6MB), both from the User Experience Podcast.

So why did I find podcasts so hard? For me, I think the problem is affordances (the properties of an object that dictate what you might do with it). Podcasts are hard to search for, and it is almost impossible to tell before you listen to a podcast from an unknown creator whether you will find it interesting or not (partly because the blurbs written about podcasts are near-universally unhelpful). Of course, if I were using podcasts “correctly” (i.e. finding ones I liked and subscribing to them) this would not be such a problem but I don’t have time to listen to podcasts — and this is another affordance problem: I can read very, very quickly, and I would absorb most of the information in a podcast much faster from reading it than listening to it. What’s more, if I am listening while I sit in front of a computer, I am inclined to attempt to do other work while listening, and then I lose the thread of the podcast.

Away from the computer, though, I could believe that podcasts might come into their own; in theory I could download interesting radio shows or documentary podcasts onto my shuffle and listen to them at a time of my choosing — even away from the internet (for me, the example that springs to mind is while travelling in a car — I get very carsick if I read as a passenger). Podcasts are also useful where they contain information that users already know exists — podcasts of lectures would make a great alternative to lecture notes for visually impaired students, or those with reading difficulties, or those who simply learn better from auditory material. They could also be a lightweight way for me to catch up on the things I necessarily miss at conferences because something I want is going on in another room — these “known item” uses sidestep the search problem.

So in terms of user experience, when are podcasts a good idea?

• When your users generally have a high speed internet connection, because podcasts are much larger than text files and users hate waiting for content to download
• Only if you are prepared to label each with a good blurb, length, and file size
• When the content is something that natively appears in an auditory format, such as lectures, radio broadcasts, conference presentations, concerts, etc.
• When you know you have users who find text hard to access, and you want to offer an alternative to a screen reader
• When your users already know your content exists so they don’t have to search for it using a non-google search interface (while iTunes’ podcast search is fairly effective it also means downloading and installing 3rd party software, which your users may not be able to do at work, or in the lab, for example).
• When you are doing it purely for your own enjoyment, and (like so many bloggers) don’t really mind what audience you have, if any.

Used well, podcasts can almost certainly make for better user experiences for students, conference-goers, radio-listeners and the like, however for vital information, they should be an alternative way to get information, not the only way. And me? I think I prefer to read my blogs, thanks anyway.

Recently I had the displeasure of doing something that should have been very pleasant — booking my flights back to New Zealand for my wedding. A friend of my gave me the heads up that there was an especially cheap flight with Emirates around the time I wanted to go, so I duly went ahead and attempted to book the flight.

The process of booking this flight — a simple flight, only a single leg — took over an hour and a half. Partly this was because the Emirates website was painfully slow, but partly it was user interface issues. When I made an error entering the dates I wanted to travel (and I discovered this near the end of the process) I had to go back and completely start the process over — there was no way to change the travel date. When I made an error with the seats I selected I was kicked back to the personal information screen, where some of the information had reverted to the default. And when I made an error entering my credit card details, the booking failed irrevocably (while saving the incorrect details to a customer profile that could not be accessed because the server was down). When I tried to start the booking afresh, I was trapped with the bad credit card details, and was offered the choice of collecting the tickets from an Emirates office, or having them mailed to me — no e-ticket for me. In the end, we logged in with my partner’s account and booked the tickets that way, but as I said, at this point it was 90 minutes later.

I tell you this story not to deride Emirates, who are, after all, flying me to NZ for an absolute pittance, but because whenever I book with an airline, the user experience is near-universally bad. I haven’t yet found an Australian website that searches as many airlines as House of Travel in New Zealand, but when I do, I won’t be booking directly with the airlines again — travel agents know their websites have to actually offer a service if they want people to come back, and so the user experience is much, much better.