Testing in Rails: Way More Trouble Than Flossing
So the other day Max and I had a joint interview for a Rails job, and the hiring manager obviously felt that our knowledge of testing in rails left something to be desired. Too true, too true. But he also told us that he thought that learning to test in rails, and learning TDD in general, was basically straightforward and simple. As in, “you are idiots that you flubbed your answers to my questions about it.” I am sure we are idiots for some other reasons, but not completely grokking TDD should not be one of them. If TDD were easy to grok and easy to do, there wouldn’t be so much evangelizing around it. There wouldn’t be courses and lectures and demos and new articles every day about why you might want to bother doing it. Here’s Gregory Brown, just a couple weeks ago, still having to argue that rails testing is Not Just for the Paranoid.” :
The real issue most people have with testing is not that they think it’s a bad idea, but that it often means a whole lot more configuration, a whole lot more to learn, and lots of things that smell like extra work. Folks who have been in that crowd will be pleasantly surprised when working with Rails.
Thanks Gregory, but no, they really won’t be pleasantly surprised. Or at least, not immediately. We’ve been working with rails testing since March, and it still smells like more stuff to learn and more work to do. I’m sure it’ll start seeming easy in the future, but right now, not so much.
First off, there’s fixtures. Apparently everyone important agrees that rails fixtures suck. But mostly no one tells you this when you’re first learning to write tests. They reload before every damn method, which would be useful, except that they slow your testing down amazingly. I was feeling very proud of myself for using a Rails Recipe for erb in my fixture files to generate lots and lots of test data for very little effort, and wow, did it show in test slowness, not so much on my machine, but on Max’s, which is older and slower. Then there’s that whole weird transactional fixtures flag that Max wrote about, that no one tells you about but that can cause your db not to look anything like what you expect at the end of a test method. Lots of people refuse to use fixtures at all, or only use them to, hmm, load data directly into the test db, not for testing. Other people point out that fixtures are a pain to keep updated when you make changes to your db, so every time your model changes, all your tests may break because your fixtures no longer work properly. So then a model change involves: change your model. Make a db migration. Change your fixtures. Change your test code. Someone has written a nice little fixture migration task, but at this point, I think we’ll just change our testing strategy to not involve much in the way of fixtures.
Then there’s the TDD/BDD and third-party testing tools thing. Do we invest our efforts in TDD even though BDD is the new big thing, so should we just skip straight to RSpec? Do we learn Test::Unit really well before we switch over to ZenTest‘s Test::Rails? What about MonkeyTest? And rcov?
Oh, and how about assert_select? I turned around the other day and suddenly I was knee-deep in the DOM. All I wanted to do was check to make sure that when my user was viewing an item that belonged to him, he also saw two buttons, ‘edit’ and ‘delete’. And that when he wasn’t, he didn’t. It wasn’t obvious how to do it, and it turns out that if I want to make it easier to use assert_select, I probably need to be putting way more ids in my html tags. And then, I added a new feature, and a new button, and my tests broke.
Of course, tests are always breaking. I understand that it is a feature, not a bug, to be forced to understand all the implications of new code. Still, it’s frustrating.
Finally, and this is, perhaps, the root of the problem, there’s all the additional decision-making overhead imposed by the very act of testing. Not all this overhead is ‘good’ overhead — that is, not all of it ultimately contributes to the quality of your code. Most of it is just one niggly decision after another. How do I organize my test code? What do I test? How many tests are enough? What should be its own test method, and what should just be a couple of lines in some other test method? Should I use setup or not? Fixtures or not? Which frameworks? There are fifteen different assert methods that will do what I need them to do, in slightly different ways: which one should I use? Oh wait, is that one deprecated? How come this assert method doesn’t work? Oh, because it’s a Test::Rails method, and I’m not actually using Test::Rails, I just saw it somewhere. All those decisions are exhausting.
Bitch. Bitch. Bitch. I realize I may now have destroyed all my credibility as a test-driven-developer by complaining about all the things I don’t get and don’t like about TDD in rails. But I think it’s annoying to be struggling with something and read nothing on the internet about it other than “it’s easy and awesome”. I have drunk the TDD kool-aid, obviously, because if I hadn’t I wouldn’t bother with it since it obviously causes me a fair amount of pain. And maybe in six months I too will say “hey, yeah, testing in rails is easy and fun!” But right now, it’s neither easy nor fun. But right now, today, I am saying it loud and saying it proud, to you, the internets, to you, other struggling Rails developers, to you, the world: TDD is good for you, like flossing, but it is much, much harder.
The moral of this story is that every night as you floss your teeth, you should thank your lucky stars that something so good for you is also so easy to do. I know I will.
Popularity: 13% [?]
Theoretically Related Posts
Use this link to trackback from your own site.