just one more geek in a sea of austin techies

January 12, 2018

WAVE Web Tool results vary by browser #WebDevGeek

One-line summary:  The WAVE tool from WebAIM is inconsistent across different web browsers.

My website work often depends on ensuring web accessibility requirements are met.  These were typically referred to as "508 requirements" but the US government has finally moved on from the old 508 rules and has now embraced the newer WCAG specifications.  Specifically, if you're earning money from the US government to provide any kind of web content, as of 1/18/2018 you're required to meet WCAG 2.0 "AA" web accessibility standards.

Recently I found an issue with one tool, WAVE, which is widely used by some federal government entities to scan websites for accessibility issues...

Whatever level of web compliance you may be interested in, you'll need a compliance scanning tool to help you find and highlight any items that aren't up to snuff.  A common free tool is the WAVE Web Accessibility Tool from  The WAVE tool got a bump in popularity due to a rough patch for the long-standing accessibility scanner CynthiaSays (which I blogged about back in 2012) whereby CynthiaSays was actually unavailable to use for a period of time.

The WAVE tool is offered as an online scanner -- just enter the web page URL to be scanned -- and also as a web browser plugin.  The WAVE tool has its up and downs and it isn't able to check every accessibility rule but it typically does a good job of scanning items that can be programmatically examined.

It's good... until it isn't
I recently ran into a case where a customer alerted me to a contrast issue flagged by WAVE for a website I'd just verified the day before as "WCAG compliant".  I rechecked the site using both the WAVE browser plugin and the WAVE online tool and found no problems.  The client then re-checked the site and turned up the same issue as before under the WAVE plugin and with the online tool.  I finally asked the client for a screenshot of the WAVE results showing the flagged item.

The problem:  The client was using Firefox while I had performed my scans under Chrome and IE.  Now, typically Chrome and Firefox have been relatively similar in web support -- if something worked in Firefox it generally worked in Chrome and vice versa.

In my case, though, a certain web page element (a dropdown selection box) scanned with WAVE via Firefox was incorrectly flagged as violating a web accessibility rule while that same page scanned with WAVE via Chrome or IE found no issue with the dropdown.  The incorrect result appeared with both the Firefox plugin and the online tool when the online tool was accessed via Firefox.

It appears that the current version of Firefox (v56.0) has some critical difference from other browsers when it comes to parsing page elements.  Here's why I think the issue is related to Firefox's JavaScript engine and/or DOM-parsing implementation:

  1. The WAVE browser plugins are self-contained.  Each plugin does its own parsing rather than send page source back to a web-based engine to be parsed.  This allows the plugin to be used on websites in environments that may not have a connection to the Internet (such as an internal-only test site).  The WAVE browser plugins are likely little more than local vessels for the same JavaScript library employed by the online tool (see #2).
  2. The WAVE online scanner works by loading and running a minimized JavaScript library ("wave.min.js") against rendered page source.  A test of Chrome and Firefox browsers revealed that the same JavaScript library is employed under both browsers when using the online tool.  There are not "browser-specific" variations of the JavaScript library between these two browsers.
  3. I compared page source as rendered by Chrome and by Firefox.  Except for a few differences in some dynamically-generated ID values (these changed with every new page load and so were expected to have variations) and an extra leading blank line under Chrome, page source was identical between the two browsers.  There were no rendered code variations at all within 100 lines of the incorrectly-flagged element.  The same JavaScript library was scanning essentially the same page code but under two different browser script engines.

In most cases WAVE returns the same scan results under all major browsers.  I've found it to be very consistent across browsers over the past several years.
That said, based on direct experience I am certain there is some difference in the current version of Firefox that may result in inaccurate WAVE tool results.  It's possible that similar hiccups might exist for WAVE under other browsers but, in my case, WAVE performance under Firefox was less reliable than either Chrome or IE.

If you have a deeper insight into how WAVE works and/or examples of inconsistent WAVE scan results, please share in the comments!

No comments:

Post a Comment