Yesterday morning (a Monday no less) I had one of those “oh, $%#@*” moments while packing my work bag.  I was ready to pack my iPod shuffle (a second generation, for anyone considering trying this at home), when I realised it was not with my phone and my wallet, and I knew exactly where it was: In the pocket of a polar fleece jacket that had been laundered the night before.  Having heard a few stories of miraculous technology survival, and knowing that the shuffle is essentially just flash memory, I thought I would see whether it was still working.  I pressed the play button, and though no sound came from the headphones, the light came on.  It turned out that both the iPod and the headphones survived, but that the headphones took a day longer to dry out and begin working again. I’m not sure if this is a designed part of the iPod shuffle second generation user experience or not, but I sure am glad of it.

While I wouldn’t recommend laundering your shuffle deliberately, I have seen more than one example of people with a similar story to tell.  Given that (as one of the posters here says) ’shuffles are designed for pockets’, designing them to be as laundry-resistant as possible (and not advertising the fact that they are) is an excellent user experience strategy:

• It takes into account the likelihood of error — it is probably not unlikely that these devices that fit easily in a pocket and are light enough to go unnoticed will get laundered (or dropped into a lake, or exposed to the rain or–in one case–run over).  Not only is this error taken into account, users do not (always) suffer catastrophically for it.
• The user’s expectation (that water will bust their iPod) is surpassed, rather than not met.  If Apple had labelled their devices water resistant, in cases where this failed (and after all, who knows if I would be enjoying music right now if it had been a hot water cycle) people would be disappointed.  Instead, people get something they don’t expect and are delighted (this is not to say that it is necessarily a good idea to set expectations lower than what will always be delivered–an unusual pleasant surprise will be remembered, a common pleasant surprise will eventually become an expected experience).

Like I say, I don’t know if this is a design feature or a happy accident, but either way, right now I am seriously impressed with my iPod, and that is always a user experience win.

WordPress has just added a feature to include related links to the end of your posts — which is a good feature, and one that users wanted — but done it in such a way that it has really annoyed their users. Tony alerted me to this, and Lavratus Pradeo has a good, brief summary of what is wrong with this feature as implemented (creates links to content that may be unsavoury by the blog owner’s definition, while implying that the links are endorsed by the blog owner), and more importantly, how to turn it off.

There is a user experience lesson in this. WordPress took a feature that their users wanted, and tried to make it significantly better for WordPress, but in so doing made it significantly worse for their users. The end result of this is that WordPress is getting criticized heavily on the blogs they host, and that many people (probably every non-spammer that hears about this and reads the instructions on how to turn it off) will not use the feature. This situation is lose-lose — WordPress (especially by rolling out the feature without telling anyone) appears to its users to have acted in bad faith, and users still don’t get the feature they have been requesting. Had WordPress executed this mopre sensibly, their users, delighted with the extra feature, would be singing their praises right now instead of condemning them. The moral of the story is that it is best to provide users with what they actually want, rather than what it is felt they ought to want.

Update 29 April 2008: I saw some of these today, and the words ‘Automatically generated’ have been added to ‘Related posts’.  This addresses the potential reader misconception that the posts are recommended by the blog author, but not the other problems listed in the links above.  This is a step in the right direction, but not a complete solution.

Today, there is a special section of the Guardian on digital academic libraries. It covers a wide range of perspectives, and is probably worth a read if you’re interested in academic libraries, digitization, digital preservation, or student habits.

I have to take issue, though, with ‘Academia’s big guns fight the ‘Google Effect”. The definition of ‘Google effect’ given in this article, and apparently coined by one Tara Brabazon, is ‘a tendency towards mediocrity’. The article goes on to accuse students of information illiteracy, and point out that they like to use Google for everything, which gives them less-than-academic results. Attempts to provide good academic-resource search engines are touched upon, as is Google Scholar (which is ‘acceptable’, but ‘too broad’ according to Professor Brabazon.

There is actually an excellent study (see ‘British library and JISC’ on this page) about information literacy skills of the current generation of university students which is the basis for much of another article in the series. That study found that undergraduates are not necessarily as information literate as they are perceived to be, and that they use “shallow” searching and don’t really read online (but neither, necessarily, do their older counterparts).

I’m not arguing with the results of that study — it seems pretty sound to me. I suspect, however, that the thing that has changed with the ‘Google generation’, though, is not actually their information literacy, but their ability access information without strong information literacy skills and/or the help of a librarian. Google, having a very simple user interface, and great results ranking, has made it easy for the average person to find answers to their questions on the internet. It has also shown users that it isn’t necessary to jump through hoops, understand boolean search, or wade through pages of results to find information.

The mediocrity Professor Brabazon has termed the Google effect arguably does not apply so much to her students, who I suspect are much the same as always, but to the information interfaces they are forced to use to locate scholarly materials. It is understandable, I think, that students prefer to spend time on their assignments reading and writing, and now they have tools which to them appear to let them bypass the cumbersome, splintered interfaces of academic journals. There is an information literacy problem here, but it is far from “whippersnappers these days not knowing how to use our journal databases”; it is the twofold problem of the proliferation of self-published non-authoritative easily accessible material that is the internet, and the vastly superior search technologies available to sift through that material.

If Professor Brabazon and her colleagues want to encourage young people to use scholarly resources the answer is not to lambast them for being mediorce (when likely they are no different to those who have come before them), nor to throw up their hands in disgust; the answer is to improve search interfaces and online access to academic materials so they can compete with Google, or (in my opinion the more likely solution) encourage widespread use of Google Scholar.

The ‘Google effect’ as I see it is not ‘a tendency toward mediocrity’ in students, it is an exposure of the dire mediocrity of the interfaces and search process for academic material. Google has democratized information searching, and made it possible for the average untrained adult to find information — academic publishers and other information providers need to catch up by providing seamless, well-ranked searches (again most likely through Google Scholar), and at least for those who are subscribers to their resources (either individually or through their institution)* make the results available with a single click. The alternative to this will not be improved information literacy skills, people are not going to learn something more difficult if they believe the tools they have will do an adequate job. I hope the end result of the Google effect will be a trend away from mediocrity–the mediocrity of academic information interfaces–and toward usable information search interfaces for all kinds of materials.

*Agruably, these results should be more widely available than that, but this post is not about the merits of open access, and academic publishers are not likely to change their access model so radically any time soon.

I’ve been thinking about why airlines have been on this blog so often of late, and I have come to the conclusion that it must be because I travel more often than average, and small things that might not be annoying if they only affected me once a year have been affecting me roughly once a month for the past four months.

This time it is an airline booking website that has frustrated me, and (worse) wasted my time (which is, after all, the only thing in life that is completely irreplaceable, once spent).  I tried to book a domestic flight on Air New Zealand, and thus went to the local New Zealand website.  I searched for a flight, found an appropriate flight time and price, and tried to book the flight using Airpoints dollars.  After being redirected through a log-in page, I was shown the following error message:

When I clicked the continue button, it took me to the Australian site, but it had not passed on the search or selections I had made on the New Zealand site, so I had to perform that search over again (and then when I did, the prices presented were quoted in New Zealand dollars and the Australian price did not show until I had selected a flight).  There is no way I could have known this in advance, because there is no standard for which regional variant of an airline website users should use (Qantas insists you use the website of the country where your flight will originate, Air New Zealand likes you to use the site where you live, for example), and nowhere on the Air New Zealand website does it actually say which variant to use.

There are two problems with this scenario:

1. I am not Australian, and there is no reason for my Airpoints membership to think I am.  The membership was created in New Zealand, and it has me registered as a New Zealand passport holder.  Now, I am not patriotic, and I don’t particularly care about a website calling me Australian, but the text is misleading and could actively confuse some users (or seriously annoy users more patriotic than me). It should read “Airpoints members resident in Australia…” (because the sole reason it thinks I am an Australian is my address.
2. The website did not (though this is a technically easy feat) pass on what I was trying to do — I landed on a search screen on the Australian web-site and had to begin the booking process again from the start.  At best this is annoying and a waste of my time, at worst it could have meant I missed out on fast-selling sale fares.

Nowhere on any of the Air New Zealand websites does it tell you that you must book through your local version if you want to use your Airpoints membership to provide your information, accumulate points, or spend your accumulated points, nor does it use the IP address of your computer (the number your computer identifies by on the internet) to redirect you before you begin searching.  This is an easy error to make, and the time cost in recovering from it is relatively high (the two minutes it might take to make a booking basically doubles, given that the user has to start over).  Air New Zealand has ample opportunities to prevent this “error” (I find it hard to call reasonable user behaviour an error), and also to make it easier for users to recover from the error without costing them a lot of time.

Errors are something that should be considered in the design of any interactive system — both how to make it harder for user to make them, and how to make it easier for user to recover when they do make them — and Air New Zealand has failed in this.  Are there any systems you make mistakes in all the time?  It might not be your fault.

After the number of shows Mike and I saw last year (at last count, it was nine, ten if you count the second time we saw Priscilla), someone very thoughtfully gave us Ticketek vouchers of a significant denomination for Christmas. Sadly, this year there has been less we have wanted to see, and as a result we decided we wanted to spend the vouchers on going to see the closing night of Priscilla (that would be time number four). You see (and this is part of the user experience problems with the vouchers) we have to use the vouchers within six months, or they become worthless (never mind that rock tours last literally years, and shows run for months, nor that Ticketek has the money for the vouchers whether we use them or not, we have to find something we want to see within six months). This limitation means that we have not been able to save the vouchers for something we really wanted to see, and thus considerably reduced their value to us as a product.

The situation got worse, however, when we tried to spend the vouchers. I read the terms and conditions and discovered to my dismay that rather than book online (as we did for each and every one of the events we went to lsat year), we had to use the vouchers in a Ticketek agency (most of which are open during working hours). There should be no technological reason for this, as the vouchers appear to have unique numbers on them (and other vendors use online vouchers all the time), but nonetheless this is the way they must be used. Fortunately, there is an agency in a music store not far from where Mike works, so he went at lunch time to try and buy the VIP class tickets we wanted for the closing night of Priscilla. The agency worker told him that the VIP class tickets were sold out, so he phoned me asking what I wanted to do. Being somewhat sceptical, I looked up the VIP class tickets on the Ticketek website and found I could purchase them there. After a little to-ing and fro-ing where the Ticketek agent suggested I try and purchase the tickets online “because I wouldn’t be able to” and me saying I was only going to risk that if the agent would pay for the tickets if they went through, the agent phoned Ticketek and found out that in fact there were still VIP tickets available, and we might be able to use the vouchers to buy them through the agency at the theatre, but that he did not have access to them from his system. It turns out (after a significant amount of running around on Mike’s part) that you can use vouchers to purchase VIP tickets from the venue, but by the time Mike got this far there were only single seats left. Maybe we’ll see Priscilla again in New Zealand.

There are a number of usability problems with this scenario, affecting different people in the equation:

• The six month timeframe can significantly limit the use of the vouchers to recipients due to long touring seasons and short-ish pre-season availability of tickets
• Though it seems to me that there is no technological reason why the vouchers should not be used online, they can only be used in person, making it much more difficult in today’s “always on” world to actually purchase tickets with them.
• The agencies where the tickets are sold do not necessarily have access to all kinds of tickets, so the vouchers can not necessarily be used to purchase the tickets you would like.
• The information screen that the agencies have does not appear to indicate that they don’t have access to tickets (as opposed to tickets being unavailable) clearly enough — customers can be misled into believing that the show of their choice is sold out, where all they really needed to do was go to the venue (though that could be difficult if the venue is in another city).

So, while the Ticketek vouchers are a lovely gift, they have proven significantly difficult to actually use to buy tickets, which makes the process of using them less like fun and more like hard work. Ticketek could significantly improve the experience of using their vouchers by extending the period for which they are valid, and making them available to use online. In the meantime, does anyone have any suggestions for Ticketek-sold shows that are coming up? Mike and I have three months left to spend our vouchers.

Over the weekend, I went to a grocery store local to my home. Normally grocery shopping is not something I consider a good experience, user or otherwise. I walk faster than the average person, and I don’t like crowds. I’m irritated by not being able to find anything, and the minute I can find everything the store seems to get rearranged (apparently this is to entice me to buy more when I see new and interesting products in the space where the stuff I was looking for was last week, but it doesn’t work — it often means I leave the store without things I had intended to buy because I couldn’t find them).

Over the weekend, though, I left the store with a bounce in my step, because they had introduced something that made my life easier, and got me out of the store faster (and that small improvement was enough to change the whole tone of the visit — being a user experience geek, improved user experience — and therefore things I can blog about — really make me happy).

The usual scenario at checkout is one of three, at most stores:

1. Stand in a feeder line for one of several express checkouts, where people have usually jammed far more than 15 things into a basket (rather than getting a trolley) to delude themselves that they are entitled to use the express checkout. Then they pay with cash, only it never occurred to them to get their wallets out at any point prior to actually having to pay, so keep waiting while they find their wallet in a large purse or backpack, or in one of their 50 pockets. Not usually so express.
2. Stand in line for a self checkout machine, and pack your groceries yourself (I spent a summer as a packer, and I can pack into my own backpack, so I actually like packing my own groceries). If you’re lucky the machine behaves for those in front of you (and you) so that you don’t have to wait for a shop assistant to come and make it scan items correctly or deliver the right change (this is risky, so I don’t usually use this line).
3. Find a non-express line where the person in front of you is nearly done, and check out through there. Even if the person still has 30 or so items, there is only one transaction (and thus one chance for a lost wallet), and it is only the same number of items as two express checkout customers. This is my preferred option at larger supermarkets.

Clearly a large part of my irritation with supermarkets is the time I waste standing in line (I know, I could read the magazines, but I don’t like the ones they have on display, and besides…it feels a bit wrong to read a magazine someone else will take home). So recently at my local Safeway, having taken option 3 and being ready to pay for my groceries with my credit card I looked down at the credit card terminal while I was waiting and noticed something new: I could pre-swipe my card. The screen read “Paying by card? swipe now”, and so I did. I then selected my account, and had my transaction pre-approved — all this while the cashier was still scanning my groceries. This saves a significant amount of time once the groceries are packed (especially, if like me, you can never figure out which way to swipe your card) — all the check out operator had to do was ask me if I wanted cash out, and then, unusually in Australia, the machine accepted the PIN I have on my credit card (instead of forcing me to sign).

This pre-swipe thing saves time in three ways:

1. Encouraging shoppers to have their cards out ahead of time
2. The check out operator not having to ask how you want to pay for your groceries (if you have already swiped your card — if you haven’t, they will still have to ask)
3. The time taken to swipe the card and select the account (not insignificant if the card is being temperamental and/or like me you can’t figure out which way to swipe it.

There is also a fourth advantage, in that it gives shoppers something to do in that awkward time where talking to the checkout operator might annoy them or slow them down, but not talking to them feels rude. The time saving is, admittedly, in the process of grocery shopping relatively small, however, it probably represents a large saving in the time-per-transaction for the cashier (and therefore a labour cost savings for the store). In situations of long lines where people pre-swipe, though, or for those who loathe grocery shopping, this small time saving (and awkwardness aversion) can make a big difference to their experience. This is a win-win user experience improvement — it will save the grocery store money, and may have an impact on consumer impression of the store — I know it has improved my perception, and will make me more likely to use the Safeway that has implemented the system than others that have not.

Remember back in 2001, when in October, white headphones appeared everywhere seemingly overnight, and all of a sudden anything that wanted to be trendy and fresh was an i-Something? Since it was first unveiled, the iPod has captured the attention and devotion of users around the globe — initially just music lovers, but later users of all kinds of media.

So why is it that the iPod was as much a revolution (if not more) than the walkman? I would guess there were four major factors (and Leander Kahney and other commentators would agree with me):

• The iTunes music store tie-in. In the past few years the iTunes music store has been heavily criticised for selling “DRM infected” m4ps that only play on Apple players, and to my mind this is a valid criticism (though one that is being eroded as music retailers come on board DRM free and competition opens up). However, in 2001, the music store was a music revolution — you could buy a whole bunch of stuff (some that was difficult to get any other way) on a per-song basis, legally and for a reasonable price. And it was all integrated with a device that you could cart all of it around on and play it on.
• Meeting a need. The iPod was the first small device with long battery life and storage for a significant amount of music. And unlike travelling CD players, iPods almost never skipped. Way back when I bought my first iPod (I have a third generation 15 GB and a second generation 1GB Shuffle), in 2003, it weighed about as much as two CDs in their cases, took up much less space, and could hold about 3,500 songs. It still isn’t full, and my music goes with me when I travel.



• Design and the cool factor. Apparently they white headphones were a happy accident, but they became iconic, and the iPod became a must have. The advertising campaign helped with this — the primary colours with silhouettes rocking out to their music grabbed people’s attention so much that people started making their own takes on them (as above), and services to iPod your own photos professionally popped up on the web.
• Usability. Not only did the iPod do something that users wanted, it was easy to do it. The device, the music store, and the software are easy to install and use — and this didn’t happen by accident, it was a designed in feature. Some commentators go so far as to claim that the usability of iPods is the reason why people love them so much.

I don’t actually think usability alone can account for the emotion — I think it is the whole user experience of the right thing that is not only easy to use, but sexy as well.

So, imagine my disappointment to discover a fairly serious oversight in my shuffle. The second generation shuffle is designed in the shape of a clip (see below) . The clip is great, it clips onto clothing or backpacks readily and effectively. But it is designed for men, or more specifically, people who wear men’s shirts. This makes me feel just a little bit like my shuffle wasn’t designed for me, and if it weren’t for the shuffle being otherwise excellent, could affect how I feel about it.

For historical reasons, men’s and women’s shirts button in opposite directions — the buttons on men’s shirts are on the right, and women’s are on the left. Originally this was a usability consideration, men dressed themselves, and women were dressed by maids, so the buttons are closest to the right hand of the dresser — unfortunately, though, we have never moved past this even though women no longer have maids. The second generation iPod shuffle’s interface is up the right way (with the headphones going into the top) when it is clipped to a menswear shirt, but upside down (with all the functions going backwards and the headphone cord looping down and then pugging up into the iPod) when clipped to a womenswear shirt. This is particularly unfortunate, given that menswear is significantly more likely to have a pocket to clip the iPod to, and therefore an alternative where the interface is rotated 90 degrees rather than 180.

Now, this may seem like nitpicking, and it probably is — but for a company and a product that has such an excellent user experience track record, small disappointments like this (particularly when they affect 50% of the potential user population, though maybe slightly less of the actual user population) are surprising. What should Apple have done about it? Well, ideally clothing would all be changed so it was more usable in modern times, but given that this is wildly unlikely (because this is a standard, and they are notoriously hard to change), I would have suggested one of things:

• An “equal-opportunities” user interface where the clip was vertical instead of horizontal
• Selling left and right clipping iPod shuffles
• Having a reversible clip.

Apple have a lot of it right, and I am not about to throw my shuffle out just because I have to work a little bit harder to find somewhere to clip it, but I do think this is an excellent example of how small things matter to a user experience, and that standardization isn’t always a great idea. Still, though…I’ve seen the iPod touch. And I want one.

Again I have left this blog too long without a post, and again it is because I have been travelling. As much as I used to love to travel, I am now throughly sick of it — sick of waiting in airport lounges and lines, sick of the poor design of systems that require me to fill out a card when there could be a machine that scanned my passport and boarding card (if you really need my signature, how about I just sign my boarding card?), sick of trying to decide if kohl pencil counts as a liquid (it doesn’t) and sick of answering the question “have you got any liquids aerosols or gels in your carry on?” with the cumbersome “yes, and it is packed in a clear 1l plastic bag”. Many of these irritations could be dramatically reduced by better systems, and in many cases, I could fly business class to reduce my aggravation.

Business class/first class on airlines really are the ultimate sale of a user experience, as opposed to a user necessity, for the vast majority of passengers; the extra movies, better food, and real cutlery are all nice, but they don’t help you to do business any better at the other end. I can’t find any statistics on how many passengers who are flying business class are actually on business, but my guess would be less than half. And lets face it, with business class travellers making up only 10% of travellers, but 35% of revenue (at least in the US), it makes sense to keep the business class passengers happy.

One of the privileges business class passengers get is the exclusive use of a set of toilets for their class. Generally speaking, given that they pay twice what a economy class passenger has paid, this is only appropriate. However, I recently flew back to Australia from New Zealand on the Air New Zealand Airbus A320,and the layout of the plane meant that not only did some economy passengers use the business class toilets, those who didn’t were involved in serious interruptions to service, and a general health hazard.

The A320 has 8 business class seats, and 144 economy class seats. There is one business class toilet right at the front of the plane, and two economy class toilets, both in the tail section (yes, this means that there is one toilet per 72 economy class passengers — this is less than is recommended for restaurants in the American Restroom Code (PDF), though more than is recommended for passenger terminals). There is a single centre aisle, with three seats on either side in economy class, and it’s pretty narrow. You can see a diagram of the plane layout here. Now, the flight I was on was completely full, and as always, the flight attendants began the economy class food and beverage services from the front of the plane — same with tray collection. You can see where this is all going to go wrong: given that the cart was blocking the aisle, and passengers were not allowed to go forward (though some of them did anyway, including one elderly and disabled lady), access to the toilets was severely limited for the majority of passengers for the majority of the flight. Passengers who did need to go could access the back of the plane by having the flight attendants wheel the cart back to the galley, squeezing past it, and going thus seriously interrupting service and coming very close to the food that was being served to other passengers. Once the aisle was finally cleared, there was a significant line of people waiting, all out of their seats and definitely not wearing seatbelts, which puts those passengers (and the people seated around them) at greater risk of injury (and it was nearly made illegal in the wake of 9/11). What’s more, with the prevalence of moderate urinary incontinence at a minimum of 3% among men and women, this had a high probability of causing someone discomfort and/or embarrassment.

It’s all very well to sell an excellent user experience to your business class passengers, who pay more. It’s not appropriate, however, to create barriers to accessing necessary facilities to uphold the exclusivity of this experience. Air New Zealand, and indeed any airline using this aircraft in this configuration needs to consider either offering economy class passengers access to the business class toilets or designing the cabin space so there is alternative access to the back of the cabin or a toilet at the front. Selling a great user experience at a premium is a good idea, but not if it compromises the health, safety, and basic comfort of your other users.

I was going to write about the vagaries of public transportation, and in particular air travel, today, but I am planning at least two further round trips to New Zealand in the near-ish future, and so I shall wait to confirm my newly formed opinions (and hopefully simmer down some) before launching myself on the poor user experiences involved in that particular endeavour. Instead, I want to talk about something underpinning good usability (and to a certain extent, user experience): Usability assessment.

So far on this blog, I have talked endlessly about user experiences (and to a certain extent, usability) with little reference to how we know the things we know about users of any given system. The way we know anything about users is assessment, either previous assessment that has contributed to a body of knowledge that allows us to make generalisations about “the user” (indeed “the user of any system”), or new assessments that answer specific questions about specific user groups and systems.

There are numerous ways of assessing usability (combined with that background knowledge about “the user” mentioned above, knowing these methods and being able to apply them appropriately is what makes a usability professional), but to discuss each type is well beyond the scope of this post. What I want to talk about here is good usability assessment — and because a lot of the work I have done recently has been with surveys, I’m going to use those as a reference point.

Given that usability assessment informs design and development, our understanding of our users, and (sometimes) the body of general knowledge about users, it’s a good idea to get assessment results as right as possible. This imperative is compounded by the fact that usability needs to do more than just make users happier, it also needs to be cost effective (though to be fair, the barrier for this can be quite low — a representative of a large firm I once did some consulting for told me that every time users ‘phoned that company’s helpline, it cost the company a minimum of $10 — at that rate it doesn’t take many users who don’t call to pay off a few hundred dollars worth of usability consulting). There are basically three steps to making sure usability assessment results are useful:

• Doing the right tests: This seems obvious, but it is worth mentioning all the same. Just like a chest x-ray can’t tell you if you have a cracked kneecap, a lab-based usability study can’t tell you how software (or any system) gets used in the real world (similarly, usage studies can’t tell you why people do the things they do, and observational studies can’t tell you whether you should make that button blue or purple). Which test is right depends on what you’re trying to find out, how much money and time you have, what stage of development you’re at, and who your users are. The other part of doing the right test is knowing what things to investigate; it’s all very well to assess the usability of your homepage (for example), but if 90% of your customers access your service via the telephone it is the usability of your phone system you should be testing.
• Testing the right users: This is more subtle than it seems. Testing on members of the development team is clearly not going to be effective, but there is more to it than that. Let’s examine how survey participants are chosen:
  • Where you advertise will affect the makeup of your participant population; for example if you advertise a library survey only in the physical library, only those who come to the library in person are likely to see the ad.
  • Participants of a public survey are, to a certain extent, self-selecting. Those who feel they have something to say on a topic will be more likely to start a survey, and more likely to complete it. These effects can be ameliorated to a certain extent by offering rewards, and using broadly inclusive language in the advertising and survey wording can help, but it is important to still recognise this bias.
  • Survey timing is important. Running a survey during exam time at a university may attract a disproportionate number of procrastinators for example, while running it during summer term can only give reliable information about summer school attendants, and not the population at large.
  • How you collect your surveys is important. Paper-based surveys have a much lower response rate than online surveys (and skew the results toward highly motivated participants — usually those who hold strong opinions). Collecting results online in a population which includes less tech-savvy participants (as older adults often are), however, will skew the results toward more technically able users. Decisions have to be made with your whole user population in mind.

  While it is almost certainly impossible to test all users for any given system, and in any heterogeneous population it is difficult to even get a truly representative sample, it is important to try to minimise sample bias (and understand and acknowledge it, where it happens).

• Making the test usable: This one is where it is easy to make mistakes, especially with surveys. I recently saw a survey where the participant was given a list of statements and asked first how important the item was, and then how well they felt the system met their needs. Given that the goal of a survey participant is usually to give their opinion first and foremost, I bet a lot of participants will fill this out wrong. Using language users of your system don’t understand will also reduce the reliability of your results — instead of asking how happy users are with their ISP, it might be better advised to ask how happy they are with their internet service. One final (and insidious) example of poor (in this instance) survey usability is bias — letting the phrasing of a question influence the answer (I’ve made this mistake recently myself, asking what users would call a service they used to make contact with the library, and repeating the word contact in one of the options given).

Usability assessment is a tool that can help make your users happier, and possibly reduce your costs. Like anything, though, it only works if you get it right.

Gordon gave me the idea for this post, while venting his frustrations about eBooks (someone needs to tell me whether that capitalization ought to be there — I never really know). His specific irritation was that he could not print more than four pages, thus meaning that the e-version of a real book one of his lecturers has set as required reading does not do the same job as a physical copy would (and the physical copy is on back order). What, asks Gordon, is the point of these things?

To me, it seems that eBooks are a bit like Wikipedia (only more authoritative): They’re good for getting short sharp bursts of information while you’re already online. My library’s subscription to Safari Techbooks saved me no small amount of time during the tail end of my masters; instead of having a recall war with someone over the only book our library had on the (then relatively new) DHTML, I was able to read about it, with code examples, online and just in time. EBooks are probably good for all sorts of things like that, from physics equations to Shakespeare’s 116th sonnet. If you want to read ‘King Lear’ or ‘A Brief History of Time’, though, forget it. Buy the book, if you can;t get it from your library.

The (apparent) reason why eBooks are so awful seems to be to me a triumph of copyright over common sense. Copyright concerns seem to be the reason why eBooks are neither fish nor fowl, neither electronic nor book. eBooks, at least the ones I have seen at Swinburne, are printed in a PDF-like format, making for worse on-screen reading, longer load times, and a distinct lack of the rich hyperlinking that adds value to online reference content. I can only think of 3 reasons why PDFs are being used here instead of natively online formats:

• Because you can lock a PDF and prevent someone copying the text
• Because the books are natively created in PDF-like form and the publisher sees no need to convert them
• To present the Greek symbols so often found in mathematics textbooks.

The only reason of those, that is really good enough, is the last one, and we can only hope that text-presentation technologies catch up with need soon enough that we won’t be dependent on preformatted text for too much longer (yes, theoretically unicode can handle it, but too often web browsers fail to adequately interpret unicode, resulting in either garbled nonsense, or that little square box thing). Not only do eBooks fail at being electronic, though, they fail at being books. They can’t be read without a web connection, the amount you can print is dictated by an online publisher and embedded in the technology, rather than reflecting copyright law, and all the wonderful affordances of a regular book — annotating, falling open at a frequently used page, coming back to where you left off and prolonged comfortable reading — are not available.

Despite the poor usability and poor readability of online book, though, I think it is important that we continue to have the available. The web statistics show that eBooks are quite heavily used, and a recent survey of our students has demonstrated that they like and expect to be able to access their textbooks online. Are our eBooks popular because the next generation is different? Possibly. My guess, though, is that most students are using eBooks for reference, to avoid purchasing (or carrying about) hard copy textbooks. As for me? I’ll read more eBooks when the usability of the electronic interface, and the complete unwillingness on the part of publishers to publish in an online readable format changes.

Addendum 18-2-2008: My colleague Tony has made some excellent points in the comments on this post that need raising here: eBooks have a significant advantage over traditional books in storage space and price, and are a hugely valuable resource for distance students.  Not only that, but our eBook provider (EBL) is very generous in terms of the printing users are allowed — 20%, considerably more than copyright in Australia; it seems Swinburne users have been facing some technical hitches at our end in this regard.  Tony’s most important point, though, was one I missed because I am used to libraries (and I should have caught this): It is not necessarily easy to find a book on the shelves of a library, or find the right information in that book, so eBooks may have the advantage in this regard.  All these points are reasons to continue to purchase eBooks, but also to manage expectations about what they are, so their users get the most out of them and not the least.