There are excellent automated tools on the web which are regularly used for web accessibility testing, for example, accessibility validators like Wave or web developer Firefox extensions or Luminosity Contrast Ratio Analyser or FireEyes.
As accessibility expert Glenda Sim explained at Webstock recently 27% of testing for accessibility can be done automatically. Such testing can establish whether the basics of accessibility have been achieved.
But she also said that what she called ‘hands on’ testing and we call real world testing, is the final arbiter of accessibility. Automated tools will not take the place of live human testers.
Downloading a screen reader and testing using a sighted person, even with the screen turned off, although that may be a salutary experience in itself. Is no substitution for the real thing. A person who can see well won’t have the same way of thinking or perceiving information as a blind person.
Of course real world testing is not about blindness alone. Other disabled people experience their own particular barriers to using web sites. They may have low vision, which is a very different experience from blindness, or they may have physical impairments that prevent them from using the mouse or the keyboard, For Deaf the written language is not their first language. Others may take medication that impairs their concentration. Yet others may have dyslexia or cognitive difficulties. Their experience or way of seeing, perceiving and processing information cannot be simulated either.
Disabled people using computers will be reasonably accustomed to impairment, and have a level of competence in using their technology a non-impaired person would not be expected to have. It is impossible, and rather insulting, to simulate the experience of another person in this context. In the real world no one is ever suddenly confronted with impairment and expected to function immediately and efficiently with or without different technology. Simulation is not therefore an option.
There is no escaping the imperative of listening to the voices and experience of disabled people. We are the best experts on our own experience.
The most important thing about real world testing is that it identifies accessibility problems not found in the rest of the accessibility testing process. When fixed all visitors to your site will have a better experience.
Usability testing is generally done with outside users. That is, people who are not part of the organisation which owns the site. They should also not be familiar with the content of the site being tested. The same should be true for accessibility. Yet I know some organisations will check with a handy blind employee and consider that as adequate accessibility user testing.
It is not practical or best practice to use disabled people from within your organisation to test a website. While they may be able to contribute useful feedback, their familiarity with the organisation and the content of the site will mean the experience will not be that of a ‘real’ external user.
They may also feel constrained by their position within the organisation from freely responding in a test.
Accessibility testing requires a range of potential disabled users with a variety of impairments and using different assistive technologies in their everyday situations. It is unlikely that most organisations include a full range of disabled people on staff anyway, so results from testing with a limited range of people with a limited range of impairments and technologies will not give the best outcomes.
Achieving accessibility on the Web requires organisational commitment, which means sound policy, training and accountability, an understanding of standards and best practice, good technical backup and an acceptance of universal design principles and the reasons for making a web site accessible in the first place. Real world testing is an essential part of the mix in achieving accessibility.
You’ve carefully read and tried to tick the boxes for most of those irritating government web accessibility standards, and the W3C accessibility Standards, and still the pesky ‘crips and blindies’ complain about your web site not being accessible to them. What to do?
Sadly all the box ticking in the world won’t guarantee an accessible web site. Accessibility is not all about a score on a list of standards, although of course standards provide the basis for best practice, and they are certainly a necessary foundation for accessibility. But adherence to standards alone will not guarantee the accessibility of your site.
Real world testing is an essential element of web accessibility. By that I mean testing by a variety of disabled users with their regular technology and in their regular everyday situations so the real problems they as users experience can be revealed, and solved.
In ten years experience we have discovered that there are some critical aspects of accessibility that will only be discovered by real disabled testers. Alternative text for images is an example. Only a human tester can tell if it adds real meaning to the information on the page for them.
Nothing about us without us!
Many people misunderstand vision loss. They assume that you are either blind or you can see reasonably well. The truth is quite different. This is my attempt at clarification.
- Low vision is very different from blindness, although they both exist on a vision continuum. If you have low or impaired vision, it will probably affect your; clarity of vision (visual acuity), ability to differentiate colours, and/or range of vision (visual fields).
- Low vision does not mean we are all the same. Low vision can affect each person differently. This has significant implications for information accessibility and real world testing of web sites.
- Making things big will not always help, although it might in some situations. We also need clarity and definition
- We don’t all need identical colour contrast although we will need good contrast
- While we may look at you while you direct us, we might not have a clue where you are pointing. You might need to describe more. It is not funny to make us try and guess who you are if we don’t immediately recognise you in the street.
- It you ask us how much we can see you may not get a sensible answer. You are asking for a comparison between what you and I can see.
- We may have other impairments which may present different issues in different situations
- We won’t all wear glasses and we don’t all use screen readers or Jaws, canes or guide dogs.
- There are a lot more of than you think. Over 80,000 New Zealanders are blind or live with a sight limitation that cannot be corrected with glasses or contact lenses. Of this number, only 11,500 are completely blind. Numbers will grow as our population ages.
- Like all disabled people, when we state our needs we are not being a nuisance or demanding – we need these things. We really need clear makings of the edges of steps for example. If you aren’t sure just ask, respectfully.