thirdbIT

We’ve several times complained about testing in rails. It’s difficult. It’s confusing. It’s time-consuming. No one really shows you how to do it well. There are too many choices for methods. You feel inadequate. Still, we soldier on. Or at least, in theory we soldiered on. In practice, our test code languished.

Until. A major refactoring was required on an app. Tried to convince client that feature requiring refactoring was not really that important. Oh, yes it is, says client. Okay, says I, and go in with a sledgehammer.

Well duh, I broke everything. I managed to implement the feature, and broke practically everything else in the process. And I couldn’t run my tests, which I’d been avoiding since long before the sledgehammer got involved, so they were in no shape to tell me what I’d broken anyway. And so Max just kept sending me Trac notifications from the other room, and I fell into despair when I realized that I had turned the code base into a steaming pile of putrid excrement. I don’t deserve to call myself a programmer, I thought. If only I had been diligently testing, this would never have happened.

So that was my ‘hit-bottom’ moment, when I realized that I had no control over my code and that I was ready to give my code over to the higher power of test-driven-development. I just had to figure out how to do it.

A lot of things were holding me back from making the transition to TDD. The biggest one, I discovered, was perfectionism and overwhelm. So many different ways to test. So many different things to test. So many different rails luminaries talking about their favored testing methods. So many different testing plugins, add-ons, and approaches. Too many decisions. I would sit down to write tests and just be absolutely paralyzed.

Once I realized that my big problem was perfectionism, I thought immediately of FlyLady. FlyLady is the reason our household is no longer buried in laundry and our kitchen is no longer buried in dirty dishes. FlyLady has some really great hacks for getting housework done, and those hacks can be applied to writing test code too.

Here are the most important points that Flylady makes:

• The goal is “progress, not perfection”.
• Baby steps will get you there.
• You can do anything for 15 minutes.

FlyLady counsels us to start with the tools we have on hand, not to try to do everything all at once, to put one habit in place before we start worrying about others. Most importantly, she constantly reminds us that housework done badly still, in her words, “blesses” your household. In the same way, testing done badly still blesses your application. Any testing is better than no testing. Ugly, non-DRY, simple, non-idiomatic, inefficient tests are still tests, and they’re getting you closer to where you want to be, testing-wise.

With these principles in mind, I’ve finally been making progress on developing the TDD habit. Here’s my very own 12-step program for testing in rails. I sincerely hope it helps other fledgling Rails TDD programmers, and I’ll keep everyone posted on my progress.

1) Admit that without tests, you have no real idea of what your code does. You have no control over your code.

2) Forgive yourself for all the untested code that you’ve written in the past.

3) Open up any one of the generated test files in any one of your rails projects, and write a test. Just one. Do not write fixtures. Do not decide you really need to be using RSpec instead and get distracted downloading and playing around with it. Do not spend a lot of time reading other peoples’ clever testing strategies. Don’t even think about mocks or stubs. Just write a test. Start with unit tests, they’re easiest.

4) When you’re done writing the test, applaud yourself.

5) Now open up the terminal, cd to your project dir, and type “rake test”.

6) When the test is done running, applaud yourself.

7) Now make the test pass.

8) Again, applause.

9) Do the whole thing again. And again. And again. Don’t do it for more than 15 minutes at a time. Set a timer, write 15 minutes of awful, non-idiomatic, stinky, not DRY, trivial test code, and when the timer rings, ‘rake test’ and give yourself a cookie.

I’m only at step 9. I think steps 10, 11, and 12 will be:

10) After a long time, testing will start to feel more natural. You can take a few minutes to fiddle with fixtures, if you want. You can branch out into assertions you haven’t used before. Go crazy with assert_select. No, now is not the time to check out rspec, rbehave, or cruisecontrol. Just keep writing tests.

11) By now, you’ve started to write your tests before writing your code. You don’t have to look up assertions. You’ve developed opinions about how to organize your test code, and you know what really bugs you about test code, what you wish it could do differently, what trips you up and slows you down. Now, and only now, are you ready to look into the various testing plugins, tools, and alternate testing theories.

12) You are now a testing master. Spread the TDD gospel far and wide.

So the other day Max and I had a joint interview for a Rails job, and the hiring manager obviously felt that our knowledge of testing in rails left something to be desired. Too true, too true. But he also told us that he thought that learning to test in rails, and learning TDD in general, was basically straightforward and simple. As in, “you are idiots that you flubbed your answers to my questions about it.” I am sure we are idiots for some other reasons, but not completely grokking TDD should not be one of them. If TDD were easy to grok and easy to do, there wouldn’t be so much evangelizing around it. There wouldn’t be courses and lectures and demos and new articles every day about why you might want to bother doing it. Here’s Gregory Brown, just a couple weeks ago, still having to argue that rails testing is Not Just for the Paranoid.” :

The real issue most people have with testing is not that they think it’s a bad idea, but that it often means a whole lot more configuration, a whole lot more to learn, and lots of things that smell like extra work. Folks who have been in that crowd will be pleasantly surprised when working with Rails.

Thanks Gregory, but no, they really won’t be pleasantly surprised. Or at least, not immediately. We’ve been working with rails testing since March, and it still smells like more stuff to learn and more work to do. I’m sure it’ll start seeming easy in the future, but right now, not so much.

First off, there’s fixtures. Apparently everyone important agrees that rails fixtures suck. But mostly no one tells you this when you’re first learning to write tests. They reload before every damn method, which would be useful, except that they slow your testing down amazingly. I was feeling very proud of myself for using a Rails Recipe for erb in my fixture files to generate lots and lots of test data for very little effort, and wow, did it show in test slowness, not so much on my machine, but on Max’s, which is older and slower. Then there’s that whole weird transactional fixtures flag that Max wrote about, that no one tells you about but that can cause your db not to look anything like what you expect at the end of a test method. Lots of people refuse to use fixtures at all, or only use them to, hmm, load data directly into the test db, not for testing. Other people point out that fixtures are a pain to keep updated when you make changes to your db, so every time your model changes, all your tests may break because your fixtures no longer work properly. So then a model change involves: change your model. Make a db migration. Change your fixtures. Change your test code. Someone has written a nice little fixture migration task, but at this point, I think we’ll just change our testing strategy to not involve much in the way of fixtures.

Then there’s the TDD/BDD and third-party testing tools thing. Do we invest our efforts in TDD even though BDD is the new big thing, so should we just skip straight to RSpec? Do we learn Test::Unit really well before we switch over to ZenTest’s Test::Rails? What about MonkeyTest? And rcov?

Oh, and how about assert_select? I turned around the other day and suddenly I was knee-deep in the DOM. All I wanted to do was check to make sure that when my user was viewing an item that belonged to him, he also saw two buttons, ‘edit’ and ‘delete’. And that when he wasn’t, he didn’t. It wasn’t obvious how to do it, and it turns out that if I want to make it easier to use assert_select, I probably need to be putting way more ids in my html tags. And then, I added a new feature, and a new button, and my tests broke.

Of course, tests are always breaking. I understand that it is a feature, not a bug, to be forced to understand all the implications of new code. Still, it’s frustrating.

Finally, and this is, perhaps, the root of the problem, there’s all the additional decision-making overhead imposed by the very act of testing. Not all this overhead is ‘good’ overhead — that is, not all of it ultimately contributes to the quality of your code. Most of it is just one niggly decision after another. How do I organize my test code? What do I test? How many tests are enough? What should be its own test method, and what should just be a couple of lines in some other test method? Should I use setup or not? Fixtures or not? Which frameworks? There are fifteen different assert methods that will do what I need them to do, in slightly different ways: which one should I use? Oh wait, is that one deprecated? How come this assert method doesn’t work? Oh, because it’s a Test::Rails method, and I’m not actually using Test::Rails, I just saw it somewhere. All those decisions are exhausting.

Bitch. Bitch. Bitch. I realize I may now have destroyed all my credibility as a test-driven-developer by complaining about all the things I don’t get and don’t like about TDD in rails. But I think it’s annoying to be struggling with something and read nothing on the internet about it other than “it’s easy and awesome”. I have drunk the TDD kool-aid, obviously, because if I hadn’t I wouldn’t bother with it since it obviously causes me a fair amount of pain. And maybe in six months I too will say “hey, yeah, testing in rails is easy and fun!” But right now, it’s neither easy nor fun. But right now, today, I am saying it loud and saying it proud, to you, the internets, to you, other struggling Rails developers, to you, the world: TDD is good for you, like flossing, but it is much, much harder.

The moral of this story is that every night as you floss your teeth, you should thank your lucky stars that something so good for you is also so easy to do. I know I will.