Pawel Brodzinski on Leadership in Technology

Whatever it takes to lead a team, build a product, and run a business

When Unit Testing Doesn’t Work

Written by

Pawel Brodzinski

in

Unit testing is such a nice idea. Developers create tests as they develop working software. Then they (tests, not developers) are executed during every build and it’s verified whether something hasn’t been broken over the way.

Unfortunately unit tests very often doesn’t work well even when team formally boast they employ this practice. Why?

1. Unit testing can’t be enforced. Yes, yes, I have heard about tools which measure code coverage. A problem is they tell you whether your code is covered but they don’t tell you with what. It can be covered with quality tests but odds are it is covered with a lot of crap generated just for the sake of achieving target code coverage. Remember you get what you measure.

2. Writing unit tests is easy – maintaining them is a hard part. Consider you have your code and your tests perfectly working. Now you change something which breaks several tests. Is that a flaw in the code? Or maybe it was done on purpose and now you have to rethink and rewrite all broken tests. Heck lot work to do.

3. Tests should be added all the time. When you find a bug which wasn’t caught by tests you already have you should add one which will do it in the future. And yes, that means more work on crappy maintenance tasks which is neither funny nor developing.

4. It takes so long to run all unit tests. Sometimes tests are switched off because build process takes too long and they don’t have to be run each time, right? Well, wrong. They have to.

5. Good unit test checks border condition while easy-to-write unit test exploits optimistic scenario. It’s pretty easy to write unit tests not using your mind at all. 2 plus 2 should be 4. Task completed. Case closed. How about using 0 or negative numbers or floats? Forget about them – too much work.

6. Writing unit tests is boring. That’s not amusing or challenging algorithmic problem. That’s not cool hacking trick which you can show off with in front of your geeky friends. That’s not a new technology which gets a lot of buzz. It’s boring. People don’t like boring things. People tend to skip them.

A key thing to have unit testing implemented well as a practice is right approach of developers. Unfortunately it can’t be trained. These lads and gals either believe unit testing helps them to create good software and they do their best to develop high quality unit tests or they just cover the app with some crap just to get proper code coverage result and pull the problem out of their ass.

If you’re going to use unit test better be sure your people are the former. Other way around it’s not worth the effort.

Comments

6 responses to “When Unit Testing Doesn’t Work”

  1. johnfmoore Avatar
    johnfmoore

    Good post Pawel. Unit tests are only useful if written correctly, executed with every build, and maintained. I was reading another post on unit testing, the excuses people use for not writing them:

    http://bit.ly/Acmd

    which is worth a review.

    Feel free to also check out my older post on the topic at:

    http://bit.ly/tgLYO

    John

  2. Radenko Zec Avatar
    Radenko Zec

    If one little change in your code is resulted in 10% tests to be crashed you are not working your software based on SOLID or Single responsibility principle?

  3. Pawel Brodzinski Avatar
    Pawel Brodzinski

    Radenko,

    Actually if you bring discussion between Jeff and Joel I linked to then they actually answered this question:

    “Like, you don’t want to have an Employee class, because it’s got his name which might get changed if he gets married, and it has his salary, which might get changed if he gets a raise. Those have to be two separate classes, because they get changed under different circumstances. And you wind up with millions of tiny little classes, like the EmployeeSalary class“

    My approach is pretty aligned. I don’t think single responsibility principle should be applied very conservatively. Of course we can discuss this issue but that’s not my point.

    My point is vast majority of software isn’t written in accordance to single responsibility principle. That doesn’t mean you shouldn’t use unit testing there. What more, there are people out there (including me) who wouldn’t follow SRP blindly but they’d use unit testing.

  4. Vukoje Avatar
    Vukoje

    Excellent reminder to keep focus tests quality. Unit tests introduce completely new challenges, that will introduce great problems if not considered in project early phases.

  5. Software Development Company Avatar
    Software Development Company

    Nice explanation… your points are correct to some extent, but still various companies prefer to have unit testing their development cycle…

  6. Cloud Computing Avatar
    Cloud Computing

    Nice to see a detailed article on Testing, now days many companies prefer to have tester in their own premises so that they can do the testing process along with development process. I have bookmarked your blogging site :)

Leave a Reply

Your email address will not be published. Required fields are marked *

More posts