New user experience person working in (digital) libraries

A couple of days ago I found another person who is doing work pretty similar to me, only she blogs about her practical work a lot more than I do.  Lorraine is posting a series of usability analyses on digital library sites and software, and her insights are very interesting.  You can read her blog at lorrainepaterson.wordpress.com, and you can follow her on Twitter (lorraine_p).  I’ve also added her to my blogroll.

How to deal with ‘too much information’: where should we put search refinement facets?

Swinburne Library is in the process of making some changes; we’re replacing our library system with a fancy new one, and as the user-experience-person-in-situ it is up to me to make suggestions for the search and discovery interface our users will see.  Some of those decisions I will blog about here, and search facet placement is one of them.

Search facets are one of the search tools that I think will be most instrumental in making stuff easier to find (and the OCLC report (PDF) on user expectations vs. librarian expectations suggests library users feel the same way). Facets are the little categories you see on search interfaces that let you narrow down your search results to things that are more relevant to you; they started out in tools with well-defined metadata (like eBay and Amazon, and even some of the newer library systems) and they are slowly working their way into searches with less-well-defined metadata, like Google.

With anything new like this, though, you have to figure out where in the search interface to put it.  So far I have seen facets placed to the left of search results:

to the right of search results:

and below search results:

At Swinburne, we talked a bit about facet placement, and in all likelihood ours will be on the left.

So, what are the arguments for and against each position?

• Facets below search results: When facets are below search results, they don’t distract the user when they are viewing search results, which is a good thing.  However, given that the vast majority of users don’t scroll all the way down, and only look at the first couple of pages of search results (and they look more at the first results on these pages), placing facets below search result is pretty likely to mean that users don’t see them or use them.  This likelihood is reinforced by the fact that this is a significantly uncommon location for facets, so users won’t think to look for them here.
• Facets to the right of search results: From a user-centred-design purist standpoint, in my opinion the right-hand position for facets is probably the best in an interface where the language is read from left to right.  This position means that user see search result first, and then facets if the search results don’t contain anything immediately useful.  Given the number of commonly used interfaces that put facets on the left, however, this could be a risky proposal.
• Facets to the left of search results: This is what Google have gone with (possibly because their advertising is on the right).  It is also common in other commonly-used information seeking interfaces, such as eBay, Dymocks (in Australia) and Amazon (for the US and the UK). Use of these interfaces will train users to look to the left for facets; and it would seem that at least a small sample of users have already developed this preference for left-hand search facets.

Swinburne has a real opportunity with this project to provide a search interface for our users that is not “slow motion search, typical library“; however, to do this we must pay as much attention as we can to our users.  Putting the search facets to the left is just one of the decisions we will make with the users in mind, and I hope to blog about more in the future.

Global corporate challenge not that global: Where on my dress do I put this pedometer?

Today is the first day of the global corporate challenge (or GCC), a challenge where you team up with six of your closest workmates and try to walk 10,000 steps (or more) per day each.  The theory is that by increasing workers’ average number of steps from 3,500 (the stated pre-challenge average on the GCC web site) to 10,000, those who participate in the challenge will see an increase in health and wellbeing from their increased activity levels.

Making people feel better is an admirable goal, and despite the wider issues with the GCC (for example prioritisation of walking over all other forms of activity, as evidenced by the ridiculously stingy cycle to walk conversion, the “speeding ticktets” issued for those who do too much exercise, and the relatively rigid defninition of an athlete) both testimonials and research show that it is helping at least some of its participants to feel better, and that’s a good thing.

In the interest of full disclosure, I will say that I am regular exerciser (on average 6 days per week) who does a variety of types of exercise (cycling, walking, aerobics, weight training, swimming, yoga…), who is female, and who is participating in the GCC.  As a participant, and as a usability consultant I have one major problem with how things work within the parameters of the GCC:  The pedometers we have been issued.

The rules of the GCC state that steps can only be entered from the official GCC pedometer (each participant gets two pedometers at the beginning of the challenge).  Given that one could reasonably expect that approximately 50% of participants are likely to be women (or maybe slightly more, if we take into account that cross culturally, women appear to walk more than men (PDF)) the choice of pedometer design for the challenge seems less than ideal.

The pedometer is the type designed to be worn on a waistband, completely upright, at one’s hip.  Moreover it does not have the type of clip that opens and closes, but rather it slides down over the top of a waistband.  This makes it considerably difficult to wear with a wide variety of women’s clothing:

• Women’s trouser styles are much more likely to have trousers stop at the waist (or above the hip) than men
• Skirts are often held up by women’s hips, meaning they too sit higher than the ideal for pedometer placement
• Dresses leave nowhere to clip the pedometer at all. Given that this is a coroporate challenge, and women  are in some corporations required to wear a skirt (and that even where it is not required, in some places it is recommended), the pedometer not really working with a dress seems a considerable oversight)
• Belts and sashes make the pedometer difficult to clip on because of the thickness of the material
• The style of clip means the pedometer is much more likely than an open-close clip to come off when trousers are pulled down–arguably something women are likely to do more often than men.

There are alternative styles of pedometer (including those that can be worn around the neck or placed in a bag, and watch-style pedometers), so I assume that the pedometer chosen by the GCC was based on some combination of accuracy and price. In my opinion, neither accuracy nor price can justify the difficulty presented to women by this model of pedometer (when alternatives are available.  Clip-style pedometers are only accurate when worn at all (impossible with some women’s clothing), and worn in the right place, so many women’s readings will not be accurate.  The entry fee for the GCC was nearly $100 AUD per person, and for this it would seem considerably more sensible to supply participants with pedometers that actually count all their steps accurately, rather than providing backpacks, hats, water bottles and extra pedometers.

Like the clip-style ipod shuffle, it feels like the organisers of the GCC just didn’t think about the whole population when they were making design decisions, and as a result of this women participants are disadvantaged (at least in terms of their step count, if not in terms of their actual gained benefit).  To let the organisers know for next year, I will be emailing a link to this blog post to their follow up email, included on the pedometer box, and I encourage all other participants to do the same.

Nonetheless, I’m looking forward to participating in the challenge, and perhaps learning something about my daily habits (I’m well over the 3,500 average workers make without having done any actual targeted steps, so it is nice to know that I am not as sedentary as the average worker, for example).  If only I could figure out where to clip the thing for those two weddings I have to go to…

The new Facebook: Not yet unfriended by users, but close

Facebook recently made a change to their interface that was the subject of outrage for many of their users, inspiring more than 1.7 million to sign a petition to reject it.  Facebook has made some changes to accomodate some of the things users said were problems, but many of the changes (including the slower-to-render rounded corners on pictures) appear to be here to stay.

Initially I was mildly irritated by the new interface, but I put it down to my change aversion (users near-universally hate change, which is why if you’re making major changes, they better help users out substantially).  However, as time has gone on, I have become more irritated with the new interface, not less.  As I see it, there are a few problems with the new interface:

• The proliferation of nonsense in my news feed, without an option to show status updates only.  Yes, I can turn the rubbish from every application off, if I want to, but this requires effort on my part, and will happen every time a new crop of applications becomes popular.  It’s also fairly irritating that I had to go to a help guide to even find out how to do this much, because the mechanism for operating these options is hidden unless you happen to look in the right place at the right time.
• Another side of the same coin: having to edit applications not to publish my life story immediately upon adding them.  I don’t particularly want to bombard my friends with nonsense every time I play a turn in Lexulous.  This means I have to be particularly pro-active in editing the settings for my applications so that they don’t bombard people, and the function for editing this is reasonably difficult to find
• The lack of automatic updating.  I know the old interface didn’t have it, but the trade off for change was supposed to be that we got automatic updating. This change has had no benefit for me, so I resent the fact that the one useful thing that was supposed to happen didn’t.

Do I think no interface should ever change their look and feel?  Absolutely not.  Do I think that Facebook should have done some usability testing before lanching this design?  For sure.  Do I think they did?  Dubious at best.  The Facebook approach, which is one that will always generate negative publicity, is to test their designs on real live users.

According to this blog post, the best way to plan change requires four steps: knowing your customers, listening to them, communicating with them, and responding to them. I think that sounds pretty good–pretty much like doing good user experience, in fact.  And Facebook didn’t do too badly, on a points system–they did warn users (albeit not in a way that most users would notice), and they did respond to some of the complaints users had (albeit not in a way that is really that satisfying).  Unfortunately, you can’t pick and choose which things you want out of that list–good user experience requires all of them.

Nonetheless, I think many (if not most) Facebook users will suck up the changes, even though they don’t like them, because for now, Facebook offers them more than the changes have taken away.  Having said that, though, like I said in my earlier post about Facebook and MySpace, people have personal purposes for using social networking tools.  If Facebook continues to change in a way that breaks that purpose (as the first iteration of these changes did), they will find that users (and thus their advertising dollars) drift away.

What product or service have you used that has slowly worn away at your loyalty until you couldn’t stand it any more?

On signs, and the need to carry my camera everywhere

Recently, as part of my work, I have commented on a signage policy for my workplace. Signs fascinate me.  It’s a basic usability premise that simple objects should not need signs, but so often I see a sign that either isn’t working, or which (with a little thought) doesn’t need to exist in the first place. I really do need to carry my camera with me more often so that when I see signs like this I can take a picture, but in the meantime I will describe a couple of examples to you:

There are two ways ofr a sign to fail: not provide the information that is needed, or provide information which may be actively misleading to some or all of the population.  A library I frequent has only a male cleaner; as such when he cleans the women’s toilets he must hang a sign to alert them to his presence in the toilets.  The sign he uses reads “not in use”, and I suspect that this is supposed to mean that it is not in “circulation” (to borrow a liubrary metaphor, but hung on a library toilet door, it reads more like “vacant”.

Signs that don’t work are interesting, but signs that shouldn’t need to exist raise my usability analyst ire something awful.  Recently in a public building I noticed a photocopier with not one, but two signs on it letting users of the space know that the copier wasn’t working.  Now, it’s excellent that the people responsible for the building let users know the copier isn’t working, but two signs seems like overkill, and given that the copier is on wheels, why didn’t they just move it out of the public space?

Are there signs in your work or recreation spaces that make no sense?  I’ll leave you with one I found on flickr, taken by brionv:

The bottom sign reads "door is alarmed"

Not that helpful

I was filling in an online form the other day and I got to the anti-spam device. This is what I saw:

If I literally “can’t read” then the way out isn’t going to help me much. On the other hand if I am (for example) red-green colourblind (as many people are) I may have trouble reading the text they presented here regardless of my reading abilities. I think it’s probably best to stick with the stock-standard “can’t read this?”, personally.

VuFind: An interesting case of open source usability

We all know that library users are consistently frustrated with library systems, and cannot find what they want, particularly since the advent of Google (PDF). Some academics berate and despair of their students’ information seeking practices, and claim that Google is ruining young minds. In my opinion, as I have stated before, berating students (and Google) is going after the wrong target. It is human nature to maximise benefits while minimising effort, and for many students the time they will spendf searching a number of interfaces for relevant resources–particularly when the interfaces are confusing, archaic, and unhelpful–is simply better spent reading the resources they find on Google, and writing their assignments. The only way to change this “satisificng” approach and reveal the vast range of library resources available to our students is to make them findable through interfaces that do not confuse or humiliate users, and do not require a librarian to operate. While libraries can’t expect to compete with Google while they are buying information from a multitude of vendors that do not have standardised search results or formats, library search interfaces can offer some additional features (such as metadata-based faceting and primary browsing) that Google doesn’t offer–and if the information is better, or gets better results (like higher grades) that will also prove an incentive to use library interfaces.

Typicall I expect library catalogues to be ugly and cantankerous, I see that as the price I pay for finding the books I want(and don’t even get me started on finding journal articles–usually I start with Google Scholar). This is why, when I looked at VuFind on the National Library web site, I was so impressed with it: it is clean, attractive, and very usable:

• It searches more than one type of holding; my search results included books, online resources, and microfilm. This is much closer to the “one stop shop” expectations that users have than any library system I have used in the past.
• I can choose between my search results based on metadata facets–that is, I can choose books, or works by a certain author, or items from a specific subject. This means that single term searches are much more likely to be successful, as I can easily disambiguate my search and bring the results that are most relevant to me to the top
• Results are relevance ranked (don’t laugh, some library systems don’t do this). This feature is the one that has given Google search engine market dominance; their excellent relevance ranking meant that people found what they were looking for in the one to two pages of results they typically view.

These are just a few of the features that make VuFind feel like a breath of fresh air. Another thing that is unusual about VuFind, though, and one that makes it especially exciting to me, is th fact that it is open source. This basically means that you can get the software for free (though if you want support you will generally pay for it), and that if you want to change something about it, all you need is a willing programmer.

Open source software provides large scope for improving usability of software locally, because unusable features can be altered, however generally speaking open source software is not as usable as its “closed source” or commercial counterparts (a problem that is recognised, but not well handled, in the open source community). Dave Nichols and Mike Twidale, colleagues of mine, have long been interested in usability in open source software (and indeed how to open source usability bug reporting). In a 2003 paper they published (which anyone interested in open source or usability should read), they suggested several reasons why open source software might have usability problems:

• Open source communities, famous for comments like “RTFM” (read the **&%@& manual), are not generally welcoming to experts from other backgrounds, as usability experts often are
• Design for usability generally has to start before design for coding
• Open source communities are populated by programmers, who generally cannot see the problems that users with a lesser understanding of computers might have
• Open source software programming is often done to meet a need of the programmer, and as mentioned above, programmers have very different user interface needs to other users
• Design by committee and software bloat are not usually good for usability, and open source software is prone to both

In another paper on open source usability, Dave and Mike noted that it can be hard to report usability bugs in the same way as technical bugs, and that open source interfaces may be prevented from innovating by playing “catch up” with their commercial counterparts.

So VuFind is positively fascinating for its usability, both among library systems (though some of the newer commercial systems look interesting), and among open source projects (Koha is similarly fascinatingly usable and open source). Why is it that VuFind is such an exception to the rules?

• It was created by a library, under one umbrella, and not in a typical open source community. Being under a single umbrella demonstrably helps open source projects’ usability (Dave and Mike again, there), largely by ameliorating design by committee and imposing some order on the process. This will also have meant that the community was different — VuFind’s website comments that it was developed “by libraries“, and thus not just programmers, meaning that feedback from other disciplines was likely welcome
• Typical library system websites (though again, I can’t speak for some of the newer ones) are not effective for users, so VuFind didn’t have to play interface “catch up”
• VuFind was developed “for libraries” not “for programmers”
• It looks suspiciously (to me) like VuFind might have had a formal usability process, though I can’t find any evidence for this one way or another

In the end, whatever the specific differences are, VuFind is not just exciting in terms of its user experience, but fascinating, and an exemplar of how to do usability in an open source project. I don’t know if it is the way we will go with our discovery layer (and not having seen many of the other possibilities, I can’t comment on whether it is the way we should go either), but it certainly is a fascinating project, and I will be watching it further.

Boolean search made easy to understand?

Recently a colleague sent me a link to a little tool called ‘Boolify‘. Boolify is a visual representation of Boolean search and is meant to be an educational tool to help people learn how boolean search terms work together.

Boolean logic (and one of its common applications, Boolean search) is a common ground field between computer scientists and librarians, particularly those who try to teach it. Boolean search is hard to understand for most people, but if users could understand it it would be one of the most powerful tools in their search toolbox (particularly when they are using a search term that has more than one meaning). Good search engine interfaces try and get around it by offering a more ‘natural language’ advanced search interface that lets users do some Boolean searching (see for example the Google advanced search interface below), but the true power and flexibility of real Boolean searching is somewhat lost even in these interfaces.

Google’s advanced search interface

Now it’s debatable whether the power users lose in an interface like this is even that much of a problem, given that the vast majority of users enter short queries and don’t use advanced search. Arguably there is a chicken-and-egg problem here; users may not use advanced search because (in many cases) it is too hard, meaning they never learn to use it, meaning it is too hard…Whatever came first, I think the answer is that users can usually (at least with Google) find the information they want (or something close enough) without having to get into Boolean logic, and therefore they don’t learn it.

There are some people, though, for who a good grasp of boolean logic is essential. Computer programmers are in this group, as are the librarians to whom people turn when their own searches fail. Potentially students and researchers should have some understanding of Boolean search as well, though I would generally argue for better search interfaces for these groups. So, given the difficulty of teaching Boolean logic (and having taught first year computer science I have first hand experience of this difficulty), anything that would help make it easier would be great.

At first glance, Boolify looks like just such a tool. It would be more awesome if it had a predefined results pool to play with (as well as providing an internet search), so students could see how query reformulation affects what is included in the results pool, but that is pie in the sky, and this thing looks like a good first step. It has puzzle pieces that show how you can interlock different bits of a boolean search, and the horizontal and the vertical can act like brackets in a real search term–the thing at the top is in the innermost brackets, and so on down.

Sadly, though, the promises Boolify makes are not delivered upon. Despite the puzzle pieces providing every promise of being interlockable, queries can only be developed in a linear fashion, and so the pciture below is what happens when you try to develop the query ((libraries OR museums) NOT gallerys) AND ( user experience OR usability). It simply isn’t possible to do, despite the puzzle pieces having every visual and logical affordance needed–you can on develop queries in a line, which means this tool does not represent the full power of Boolean logic.

Trying to develop a complex Boolean query using Boolify

Usually I would not be so hard on a tool designed to teach something difficult in a novel way, but this tool is a poor teaching aid. It doesn’t allow users to do what it looks like they can do by clicking bits of puzzle in anywhere–thus not showing the true power of Boolean queries, and having a significant usability problem. It also (and I think this is more important) doesn’t let users reformulate queries by moving puzzle pieces around, thus denying a real opportunity to learn how Boolean operators work together by reformulating queries and seeing what happens to the results. I can honestly say that until this tool fulfilled more of its promise, I wouldn’t use it teaching–it is bright and colourful, but it doesn’t bring much more than that to the table.

So, what does everyone else think? Should we leave Boolean logic to the geeks and logicians? Who needs to know Boolean logc, and how should we teach it?

The ‘Google effect’: A trend toward mediocrity, or away from it?

Today, there is a special section of the Guardian on digital academic libraries. It covers a wide range of perspectives, and is probably worth a read if you’re interested in academic libraries, digitization, digital preservation, or student habits.

I have to take issue, though, with ‘Academia’s big guns fight the ‘Google Effect”’. The definition of ‘Google effect’ given in this article, and apparently coined by one Tara Brabazon, is ‘a tendency towards mediocrity’. The article goes on to accuse students of information illiteracy, and point out that they like to use Google for everything, which gives them less-than-academic results. Attempts to provide good academic-resource search engines are touched upon, as is Google Scholar (which is ‘acceptable’, but ‘too broad’ according to Professor Brabazon).

There is actually an excellent study (see ‘British library and JISC’ on this page) about information literacy skills of the current generation of university students which is the basis for much of another article in the series. That study found that undergraduates are not necessarily as information literate as they are perceived to be, and that they use ’shallow’ searching and don’t really read online (but neither, necessarily, do their older counterparts).

I’m not arguing with the results of that study — it seems pretty sound to me. I suspect, however, that the thing that has changed with the ‘Google generation’, though, is not actually their information literacy, but their ability access information without strong information literacy skills and/or the help of a librarian. Google, having a very simple user interface, and great results ranking, has made it easy for the average person to find answers to their questions on the internet. It has also shown users that it isn’t necessary to jump through hoops, understand boolean search, or wade through pages of results to find information.

The mediocrity Professor Brabazon has termed ‘the Google effect’ arguably does not apply so much to her students, who I suspect are much the same as always, but to the information interfaces they are forced to use to locate scholarly materials. It is understandable, I think, that students prefer to spend time on their assignments reading and writing, and now they have tools which to them appear to let them bypass the cumbersome, splintered interfaces of academic journals. There is an information literacy problem here, but it is far from “whippersnappers these days not knowing how to use our journal databases”; it is the twofold problem of the proliferation of self-published non-authoritative easily accessible material that is the internet, and the vastly superior search technologies available to sift through that material.

If Professor Brabazon and her colleagues want to encourage young people to use scholarly resources the answer is not to lambast them for being mediorce (when likely they are no different to those who have come before them), nor to throw up their hands in disgust; the answer is to improve search interfaces and online access to academic materials so they can compete with Google, or (in my opinion the more likely solution) encourage widespread use of Google Scholar.

The ‘Google effect’ as I see it is not ‘a tendency toward mediocrity’ in students, it is an exposure of the dire mediocrity of the interfaces and search process for academic material. Google has democratized information searching, and made it possible for the average untrained adult to find information — academic publishers and other information providers need to catch up by providing seamless, well-ranked searches (again most likely through Google Scholar), and at least for those who are subscribers to their resources (either individually or through their institution)* make the results available with a single click. The alternative to this will not be improved information literacy skills, people are not going to learn something more difficult if they believe the tools they have will do an adequate job. I hope the end result of the Google effect will be a trend away from mediocrity–the mediocrity of academic information interfaces–and toward usable information search interfaces for all kinds of materials.

*Agruably, these results should be more widely available than that, but this post is not about the merits of open access, and academic publishers are not likely to change their access model so radically any time soon.

Usable usability assessment

I was going to write about the vagaries of public transportation, and in particular air travel, today, but I am planning at least two further round trips to New Zealand in the near-ish future, and so I shall wait to confirm my newly formed opinions (and hopefully simmer down some) before launching myself on the poor user experiences involved in that particular endeavour. Instead, I want to talk about something underpinning good usability (and to a certain extent, user experience): Usability assessment.

So far on this blog, I have talked endlessly about user experiences (and to a certain extent, usability) with little reference to how we know the things we know about users of any given system. The way we know anything about users is assessment, either previous assessment that has contributed to a body of knowledge that allows us to make generalisations about “the user” (indeed “the user of any system”), or new assessments that answer specific questions about specific user groups and systems.

There are numerous ways of assessing usability (combined with that background knowledge about “the user” mentioned above, knowing these methods and being able to apply them appropriately is what makes a usability professional), but to discuss each type is well beyond the scope of this post. What I want to talk about here is good usability assessment — and because a lot of the work I have done recently has been with surveys, I’m going to use those as a reference point.

Given that usability assessment informs design and development, our understanding of our users, and (sometimes) the body of general knowledge about users, it’s a good idea to get assessment results as right as possible. This imperative is compounded by the fact that usability needs to do more than just make users happier, it also needs to be cost effective (though to be fair, the barrier for this can be quite low — a representative of a large firm I once did some consulting for told me that every time users ‘phoned that company’s helpline, it cost the company a minimum of $10 — at that rate it doesn’t take many users who don’t call to pay off a few hundred dollars worth of usability consulting). There are basically three steps to making sure usability assessment results are useful:

• Doing the right tests: This seems obvious, but it is worth mentioning all the same. Just like a chest x-ray can’t tell you if you have a cracked kneecap, a lab-based usability study can’t tell you how software (or any system) gets used in the real world (similarly, usage studies can’t tell you why people do the things they do, and observational studies can’t tell you whether you should make that button blue or purple). Which test is right depends on what you’re trying to find out, how much money and time you have, what stage of development you’re at, and who your users are. The other part of doing the right test is knowing what things to investigate; it’s all very well to assess the usability of your homepage (for example), but if 90% of your customers access your service via the telephone it is the usability of your phone system you should be testing.
• Testing the right users: This is more subtle than it seems. Testing on members of the development team is clearly not going to be effective, but there is more to it than that. Let’s examine how survey participants are chosen:
  • Where you advertise will affect the makeup of your participant population; for example if you advertise a library survey only in the physical library, only those who come to the library in person are likely to see the ad.
  • Participants of a public survey are, to a certain extent, self-selecting. Those who feel they have something to say on a topic will be more likely to start a survey, and more likely to complete it. These effects can be ameliorated to a certain extent by offering rewards, and using broadly inclusive language in the advertising and survey wording can help, but it is important to still recognise this bias.
  • Survey timing is important. Running a survey during exam time at a university may attract a disproportionate number of procrastinators for example, while running it during summer term can only give reliable information about summer school attendants, and not the population at large.
  • How you collect your surveys is important. Paper-based surveys have a much lower response rate than online surveys (and skew the results toward highly motivated participants — usually those who hold strong opinions). Collecting results online in a population which includes less tech-savvy participants (as older adults often are), however, will skew the results toward more technically able users. Decisions have to be made with your whole user population in mind.

  While it is almost certainly impossible to test all users for any given system, and in any heterogeneous population it is difficult to even get a truly representative sample, it is important to try to minimise sample bias (and understand and acknowledge it, where it happens).

• Making the test usable: This one is where it is easy to make mistakes, especially with surveys. I recently saw a survey where the participant was given a list of statements and asked first how important the item was, and then how well they felt the system met their needs. Given that the goal of a survey participant is usually to give their opinion first and foremost, I bet a lot of participants will fill this out wrong. Using language users of your system don’t understand will also reduce the reliability of your results — instead of asking how happy users are with their ISP, it might be better advised to ask how happy they are with their internet service. One final (and insidious) example of poor (in this instance) survey usability is bias — letting the phrasing of a question influence the answer (I’ve made this mistake recently myself, asking what users would call a service they used to make contact with the library, and repeating the word contact in one of the options given).

Usability assessment is a tool that can help make your users happier, and possibly reduce your costs. Like anything, though, it only works if you get it right.