Response To My Post About Crappy Code in Crap4J - Agile Software Process Improvement
Jason Gorman has a thoughtful reply on his blog about using Crap4j.
He also brings up the potential for self-interest due to the fact that I work at Agitar.
“I think it’s worth highlighting the fact that Bob works at a company that sells a tool designed to help teams improve their test coverage, so I can’t help wondering if there isn’t a little bias here towards metrics that highlight lack of coverage. Which is fine, because Bob’s absolutely correct in saying that complex methods with low test assurance are pretty much universally crappy. It is a problem, and you should address it.”
I’d like to respond to that part first. Causality is always tricky. It turns out that I work at Agitar because I think these things are important. Agitar happens to be a great way to work on code quality and testing issues which are near to my heart. We invented the CRAP metric and Crap4j as ways to point out particular code quality problems. Naturally, as opposed to being cynically profit-driven, we do also offer tools that try to address these problems. Being an engineer, I don’t think I could bring myself to espouse any methodology or ideas just for my material gain. I have to believe in it. Of course, if anyone wants to offer me millions of dollars to espouse some helpful technology, I will be glad to contemplate the dilemma 
Now, on to his point.
“Getting developers started with quantitative code analysis - metrics - is one of the things I do for a living, and I’ve always found the opposite. You need to try to establish balance right from the start. Focusing in on just one area of quality, no matter how valuable by itself, can lead to people ignoring other important areas.”
It is a point we discussed when trying to identify the metric. It is a valid point, and I am glad there are people like Jason working to help teams get a balanced picture of their systems. I hope eventually that the CRAP metric, or at least the tool, will do sort of a triage of code quality issues. It will present the most egregious metric failures first, and then offer more refined levels of CRAPpiness for the developer to ponder as they clear away the more immediate issues. Imagine a video game where you have to beat the level 1 boss to move to level 2.
This raises an issue that I haven’t spent time analyzing yet, but intuitively I know will present itself. There isn’t a hierarchical order between all the problems that can exist in a project, so, at some point you have to give them multiple, equally crappy, problems in a report. Perhaps this is like being able to get to a certain point in a video game where you can take multiple paths, but ultimately you have to finish the breadth of those paths to move on to the next level?
This is in tension with our other goal of giving teams a measure that is easy to act to remedy. It should be obvious what problem is being presented, and what can be done to fix it. Otherwise, I imagine that there could be paralysis, and consequently, no action taken to improve quality.
