The most contentious point between software engineering culture and visual design culture is the question of whether important things can be always seen in absolutes. The engineering approach values measurable, reproducible results which can be represented in a graph or a checklist. Unit tests and benchmarks illustrate progress.

[…]

Visual design is often the polar opposite of engineering: trading hard edges for subjective decisions based on gut feelings and personal experiences. It’s messy, unpredictable, and notoriously hard to measure. The apparently erratic behavior of artists drives engineers bananas. Their decisions seem arbitrary and risk everything with no guaranteed benefit.

— Scott Stevenson
Theocacao: Measuring the Design Process
(writing about Doug Bowman’s stopdesign : Goodbye Google)

Software design is not data-driven engineering. Unit tests and benchmarks are about testing an implementation and certainly what’s learned needs to feed back into and inform the design but strictly speaking these are not design activities. There is a myriad of decisions in a software design that are made on qualitative grounds, not quantitative data. Maybe Google tries to make even those decisions driven by data. I’d like to know if it’s so. Most places I’ve worked there was someone with a bigger paycheck and the word ‘architect’ in his/her job title who was expected to pass judgement on the software design. Generally this person did so based on her/his skills, knowledge, and experience. A visual designer who’s aware of that may be all the more vexed when asked to defend the choice of pixel width for a border with data. Where’s the supporting data for the choice to use MVC? How come the debate over WS-* versus REST hasn’t been agreeably resolved with a graph?

If something is a ‘religious debate’, like choice of text editor or choice of programming language or choice of OS, does that mean it’s a question that is not (currently) answerable with data? Or does it mean there are different values and needs and experiences, qualitative things, that lead to different answers?

There’s all kinds of inexact unpredictable squishy messy issues, questions, and decisions that go into software development even without considering the visual interface. Scott is right on when he says:

Data and measurements are essential in software, and can take you a long way on their own. But feelings and instincts are necessary too if you want to do something remarkable.