For some time now I have been wondering why we test our source code so thoroughly but when it comes to CSS we just plainly stop caring about it.
Looking at the test pyramid, it is mentioned that tests through the UI are brittle, in fact you are testing the whole stack from top to bottom and anything anywhere can go wrong. However that doesn't mean that testing the UI needs to be similarly brittle. In fact you can mock out the underlying functionality that your process rendering the UI depends on.
A broken UI can break the user experience just like a faulty functionally (i.e. source code) does. Especially in a bigger project where several people are involved possibly across teams it is hard to keep the UI consistent and errors out.
In my current project a glitch in the UI can keep the product owner from pushing the next release candidate to production. And there are several teams that together deliver a single page to the user, meaning that bits of the page including the layout come from different sources. In the end we need to make sure that everything comes together just right.
On top of that there is this browser issue. Each browser renders a page quite differently. Consistently checking that changes don't break the layout in browser X can be a very tedious manual task.
I've heard from some people that Selenium is used to do a screenshot comparison, i.e. regression testing on reference images. One example is Needle. There have been undertakings to test actual values of properties on DOM elements, e.g. at Nokia Maps.
Why am I saying all that? Because I'm currently looking into developing yet another css testing tool that I want to share with you.
My take on this problem builds on the image comparison technic similar to the Selenium stuff. However my approach is to keep the stack simple and to make it dead simple to use: everything should be done from inside a browser window.
With the feedback from my colleagues at ThoughtWorks I've set up a small project on Github to implement an experimental solution with the goal of driving out a feasible solution.
The steps to verify a layout should be pretty straightforward: A new page (either from production or a mock page) that includes the CSS under test is registered with a "regression runner". That is a simple HTML page running the test suite (if you know jasmine and its SpecRunner.html you get the point). On the first run the page under test is rendered and can be saved as a future reference. In subsequent runs this image is used for the page to be compared against. Running the tests is as simple as opening the runner in a browser. If the layout changes, the regression test will fail. If the change was intentional a new reference needs to be created, if not you found your bug.
Technically this works by rendering the page under test to a HTML5 canvas element inside the browser and using a handy JS library for doing a diff between the canvas and the reference image.
Open points: So far works in Firefox only, and as browsers do not render pages across systems consistently, the solution is local only.
Do watch the screencast to see how it works: