In-Depth Answer: How do I tie accessibility procurement language to integrated quality assurance testing?

What are the odds?

Procurement is a game of chance. If I buy this product, how likely is it that it will work for me? It’s the same sort of question we ask ourselves about any purchase. Will this microwave oven serve my cooking needs? Will this new enterprise computer system meet our organization’s needs?

Imagine not having any ability to predict the outcome of your choices. When a blind individual puts a dollar into a drink vending machine, historically and even today there’s no way to know what flavor drink they’ll get. That’s why many in the blind community refer to it as ‘vending machine roulette’. (What if there was a technology that could be employed to make the vending machine talk?)

In procurement, the aim is to increase your odds of winning by making a list of appropriate requirements, and then going through a methodical process to assure yourself that you have made the best choice. If the vendor offers guarantees, all the better.

For a long time, accessibility procurement language looked a bit like this:

Customer’s RFP: [Blah Blah Blah] … and the software must be accessible to people with disabilities… [Blah Blah Blah]

Vendor: Oh sure, yes, it will be accessible. Sure thing. Honest. Trust us.

In other words:

Customer: I’d like a cola. I need some caffeine.

Vending machine: Here’s a bottle of water. Enjoy!

Thankfully, times have changed…

ICT testing, standards, and procurment

There are a large number of providers of Information and Communications Technology (ICT) accessibility testing tools and processes. Each type of approach has its pros and cons (see our companion article comparing testing approaches in the ‘Related Sections of the Accessibility Switchboard’, below). There is an accepted (de facto) industry standard for the performance of ICT, the Web Content Accessibility Guidelines (WCAG) 2.0. However, there is no industry standard for testing ICT. The coverage of WCAG 2.0 by various available test tools and methods varies. Therefore, the claims of the developers of the test tools and methods must be evaluated individually by ICT customers and vendors involved in procurement. Even though there is no current standard for testing, the key thing to know is that it is now possible, in procurement documentation, to refer to an actual test process that covers WCAG 2.0. A proof of concept assertion is in marked contrast to the less effective (and spartan) historic language of “[it] must be accessible to people with disabilities”.

Note: In US Federal Government procurement circles, one of the main communication vehicles between vendors and customers has been the Voluntary Product Accessibility Template (VPAT). For over a decade, this document didn’t make any reference to how the product was tested (i.e., on what basis the underlying claims of accessibility were assessed). In March of 2018, the VPAT was updated to include—as a best practice suggestion—to include testing / evaluation methodologies employed. It now also uses the name ‘Accessibility Conformance Report’.

QA and QC testing: Shared understanding and shared language

Quality Control (QC) is what happens before you let a product out of the door. Quality Assurance (QA) is what happens while you are in the process of designing, developing and testing your product. (Good QA eases QC; and QC without QA is ineffective in practice.) Our question in this article concerns tying procurement language to actual QA tests that happen throughout development. In the case of buying a finished product, you are assessing the whether the product meets your business needs which will include accessibility requirements. The vendor may choose, via a VPAT, to share test results as a result of their final QC process. In the case buying development systems or platforms, and buying development services for creating customized ICT, it is about assuring that the appropriate QA will take place during development. These needs are expressed through procurement language, and it is important that the various stakeholders involved share a common language and a common understanding.

 

Having a shared understanding of what will be tested and how provides those involved with a higher degree of confidence that accessibility will be adequately addressed in the purchasing of (or development of) ICT systems. Referencing a specific test process in the RFP and supporting documentation is a way of demonstrating shared understanding between all who are affected.

What’s happening in QA now? What do you want to happen in the future?

Once the relevant team members have (a) a good grasp of accessibility requirements; (b) a testing tool and test method options; and (c) there is a shared language / shared understanding between stakeholders, there are nominally three additional steps needed to ‘tie accessibility procurement language to integrated quality assurance testing’:

1. Evaluate what is happening now in your systems for QA in ICT development, and whether they include accessibility testing;
2. Evaluate the requirements that are now possible to consider regarding integration of accessibility in ICT QA testing; and
3. Incorporate the desired requirements into RFPs, any IV&V (Independent Verification and Validation) activities, and subsequent purchasing and implementation decisions in the organization’s QA processes, including reference to specific accessibility QA test processes.