Showing off CSS Critic with example applications

TL;DR I agree that setting up your first test with CSS Critic can be a hurdle that needs overcoming. That’s why I created csscritic-examples.

One benefit of working at ThoughtWorks is that you get to see many different projects. In my last two years I’ve seen three different implementations of a user facing application and more importantly three different ways of delivering an application’s UI.

In the phase of each project I’ve sat down and mulled over how to set up a first user interface tests.

While CSS Critic tries to be as simple as possible to set up, your choice of application framework heavily influences the integration of the tool. Unlike other testing solutions (e.g. Selenium-based) that fire up your application through a web server, CSS Critic wants to be a lightweight solution by picking up small components of your application for testing. This forces the developer to carefully wire it into the development setup.

More clearly it needs you, the developer, to think about components of your UI architecture. Once parts of the UI are split into different files and dependencies are made clear (say for example your HTML template needs an adjacent JavaScript view implementation) you can define small and independent tests. This is the basic formulation of how CSS Critic would like to work.

The latest application I have worked on was an AngularJS-based single-page application. Working with an opinionated framework that AngularJS is I realized how difficult it can be to tap into the inner bits of a framework. While designed to be easy to set-up and use for day-to-day work, accessing inner components of a framework (and for the purpose of UI testing that means the templating system and view binding) can be difficult.

To counter the difficulty of the initial set-up phase I now plan to collect examples of how to bring together CSS Critic with the web framework of your choice. I’ll start with a simple application based on AngularJS and one in plain JavaScript. Have a look at the repository over at GitHub. Cheers go to the TodoMVC project that both examples are based on.


Gathering an extensive set of examples can only be a community-based effort and so I welcome contributions. If you, dear user of CSS Critic, have a new set-up to share, please start a pull-request and have others benefit from your good work.


Testing CSS

For some time now I have been wondering why we test our source code so thoroughly but when it comes to CSS we just plainly stop caring about it. 

Maybe I'm wrong, I'm still relatively new to the TDD business, but looking at my colleagues, everybody is quite eager to have their Java or JavaScript code covered. But speaking of CSS, there isn't much help around for doing tests here.

Looking at the test pyramid, it is mentioned that tests through the UI are brittle, in fact you are testing the whole stack from top to bottom and anything anywhere can go wrong. However that doesn't mean that testing the UI needs to be similarly brittle. In fact you can mock out the underlying functionality that your process rendering the UI depends on.

A broken UI can break the user experience just like a faulty functionally (i.e. source code) does. Especially in a bigger project where several people are involved possibly across teams it is hard to keep the UI consistent and errors out.

In my current project a glitch in the UI can keep the product owner from pushing the next release candidate to production. And there are several teams that together deliver a single page to the user, meaning that bits of the page including the layout come from different sources. In the end we need to make sure that everything comes together just right.

On top of that there is this browser issue. Each browser renders a page quite differently. Consistently checking that changes don't break the layout in browser X can be a very tedious manual task.

I've heard from some people that Selenium is used to do a screenshot comparison, i.e. regression testing on reference images. One example is Needle. There have been undertakings to test actual values of properties on DOM elements, e.g. at Nokia Maps.

Why am I saying all that? Because I'm currently looking into developing yet another css testing tool that I want to share with you.

My take on this problem builds on the image comparison technic similar to the Selenium stuff. However my approach is to keep the stack simple and to make it dead simple to use: everything should be done from inside a browser window.

With the feedback from my colleagues at ThoughtWorks I've set up a small project on Github to implement an experimental solution with the goal of driving out a feasible solution. 

The steps to verify a layout should be pretty straightforward: A new page (either from production or a mock page) that includes the CSS under test is registered with a "regression runner". That is a simple HTML page running the test suite (if you know jasmine and its SpecRunner.html you get the point). On the first run the page under test is rendered and can be saved as a future reference. In subsequent runs this image is used for the page to be compared against. Running the tests is as simple as opening the runner in a browser. If the layout changes, the regression test will fail. If the change was intentional a new reference needs to be created, if not you found your bug. 

Technically this works by rendering the page under test to a HTML5 canvas element inside the browser and using a handy JS library for doing a diff between the canvas and the reference image.

Open points: So far works in Firefox only, and as browsers do not render pages across systems consistently, the solution is local only.

Do watch the screencast to see how it works: