Cross-browser Testing: Crucial, but Commonly Untested

By Peter Koevari — Senior Consultant, Revolution IT

I would like to discuss a form of testing, which I have scarily seen "forgotten about" in many projects with web facing applications, or even with internal applications but with multiple browsers installed in their environments.

In my career in software testing, I’ve worked on many projects where they have either:

1. Only considered testing their application/website on their browser of choice
2. Thought about cross-browser testing, but decided that they would just choose an outdated version of IE and Firefox to test it on
3. Considered it, but don't have the "time" to do cross-browser testing
4. Tested in the latest versions of two browsers of choice
5. Testing in all popular browsers (extremely rare)

I did however work at one Agile organisation, on a key project where they were completely on the ball with cross-browser and mobile interface testing. But that would be one of the exceptions to the rule in the IT world.

So, let's say for an example that you work on a project to design a car tyre. Simple enough right?

Now, imagine if you only tested that car on standard bitumen. Would that be enough testing?

Because surely, nobody will ever drive on dirt, off road, or through water, right? Testing all of that would be too hard?

So, you release the tyres to production and you end up having a Toyota incident (brakes, anyone?). Imagine the fallout from that, imagine how much that would cost your company or the client you work for?

Now, if we take the same example to the IT world. Let's imagine that you work for Microsoft and you are on a massive team, designing the next version of Windows.

Do you think they will be testing every possibly combination of hardware that can run on its platform? Do you think they will be testing every app they can get their hands on for compatibility?

If they did all of that testing themselves, we would not see Vista and Windows 7 for a very, very long time.

As we know, exhaustive testing can take a lifetime.

Now you know why there are Release Candidates, but I digress.

Browsers generally use different rendering engines, as well as having differences in the way they present and handle code. It is for this reason, that unless you are in a severely locked down environment with one installed browser, and nobody in the world outside your organisation needs to use whatever it is that you are developing, then you need to perform cross-browser testing.

You do not need to go to the point of ridiculous and test every version of every possible browser (Microsoft example?)

But if you are developing something which will see the public eye, and you want to ensure that it does not break or contain high severity bugs that may actually cost you more money to fix than to test, then you need to test it on the latest versions of the most popular browsers.

Why the latest versions? Because in the real world (not in the locked down, corporate operating environment with a release of a browser update usually years behind the real world), people update their browsers automatically from the internet. This update is usually with minimal impact on their browsing experience.

And if you use automation, some tools will let you run the same scripts on multiple browsers. Utilise that as part of your shakedown/smoke tests to save time, but automation will be another topic to discuss entirely.

If you are lucky enough to be developing something that can be reached *from* the internet, you could also make use of Browsershots to give you visual feedback on a wide range of browsers. Be aware that if you are testing with sensitive data, never use that option.

The next time that you are on a project, be brave and ask the question if it is not already discussed... "Are we testing multiple browsers?"

And if you are really brave, also ask, "Are we testing mobile phones? Android? iPhone? Do we have an interface optimised for mobile 3G browsing?"

Sometimes, you will be amazed that it just has not been thought of.