Share this:

Thoughts On TDD (A Case Study With Ruby And RSpec)

test_driven Oh yeah, we do TDD, after all we’re an agile team! That’s what we tell our peers and it is even true, it is just not true 100% of the time. But everyone kind of agrees not to dig too deep – after all they are in exactly the same boat – and we all get to feel good about our process and how we do things in our neck of the woods. Let’s face it, we all fall back into non-TDD practices every day, that doesn’t mean we don’t write tests it just means we don’t always write the tests first. For some reason people often feel like they need to cover this up, as if they loose some credibility by not being a TDD maniac and that’s patent nonsense. In the kind of work we do as developers, it is perfectly natural to not be doing TDD all the time, the breadth of technology we work with on a daily basis almost demands this. Let’s examine a typical TDD scenario (or at least typical for me) and perhaps things will get a little clearer.

A Typical TDD Scenario

We have a class and we feel the need for a new method, we write an empty method definition and we’re now almost ready to TDD.

class FileOperations
  def read_file_and_print_line
  end
end

Of course we don’t just start hacking away, we need to build a picture in our head of what we want our new method to do. In this case we have the following:

  • find the path of the file we want to open
  • open the new file and read it line by line
  • when we find the relevant line we want to print it out
  • the line is considered relevant if it matches a certain condition

We’re now ready to get testing:

describe "FileOperations" do
  before(:each) do
    @file_ops = FileOperations.new
  end
  describe "read_file_and_print_line" do
    it "should match a line in a file" do 

    end
    it "should not match a line in a file" do
    end
  end
end

The fact that we’re trying to match against a condition is an immediate flag that we can have a positive and negative outcome so we require 2 tests. If you haven’t got the second test you will easily be able to implement the method incorrectly without your test picking this up. Alright, lets fill in our tests and then pick them apart.

describe "FileOperations" do
  before(:each) do
    @file_ops = FileOperations.new
  end 

  describe "read_file_and_print_line" do
    it "should match a line in a file" do
      @file_ops.should_receive(:find_path_of_file_to_open).and_return("path to file") 

      mock_file = mock(File)
      File.should_receive(:open).with("path to file").and_yield(mock_file) 

      mock_file.should_receive(:each).with(no_args()).and_yield("string1").and_yield("string2").and_yield("string3") 

      @file_ops.should_receive(:line_matches_condition).with("string1").and_return(false)
      @file_ops.should_receive(:line_matches_condition).with("string2").and_return(true)
      @file_ops.should_receive(:line_matches_condition).with("string3").and_return(false)
      @file_ops.should_receive(:puts).once().with("string2") 

      @file_ops.read_file_and_print_line
    end 

    it "should not match a line in a file" do
      @file_ops.should_receive(:find_path_of_file_to_open).and_return("path to file") 

      mock_file = mock(File)
      File.should_receive(:open).with("path to file").and_yield(mock_file) 

      mock_file.should_receive(:each).with(no_args()).and_yield("string1").and_yield("string2").and_yield("string3") 

      @file_ops.should_receive(:line_matches_condition).with("string1").and_return(false)
      @file_ops.should_receive(:line_matches_condition).with("string2").and_return(false)
      @file_ops.should_receive(:line_matches_condition).with("string3").and_return(false)
      @file_ops.should_not_receive(:puts) 

      @file_ops.read_file_and_print_line
    end
  end
end

There are many things going on so we’ll start at the beginning.

  • We want to cover our method with a minimum number of tests, this allows us to keep our methods small and tight and makes it easier for everyone. This means that whenever there is an opportunity to push some functionality into a collaborator we need to take it. In the case of our tests we did this twice find_path_of_file_to_open and line_matches_condition. We don’t care how these collaborator methods work, we can figure it out later, for now we simply mock how we expect these methods to behave.
  • Because we know a little bit about Ruby file system access we know that when we open a file we can yield to a block so we need to represent this in our test (to enforce this behaviour on our method implementation). We also know that we can call each on our file which will allow us to yield each line of the file to a block. Enforcing things like this can potentially make the test brittle if we decide to change our mind about the internal implementation of our method, but it also protects us from implementing the method incorrectly and having our tests still pass (more on this later).
  • We’ve set our tests up is such a way that we know how many strings should match and so we know that one line should be printed out in the first test and no lines in the second, we need to be explicit about this, once again, this ensures that the actual implementation can only go one way.

Essentially our two tests ensure that the easiest way to implemented the method is also the correct way, which is the goal of TDD. If you find that your tests allow you a much easier path to implement your method incorrectly (and still have the tests pass) then you need to tweak your tests. Oh and here is our finished method, only one way to write it now:

class FileOperations
  def read_file_and_print_line
    currently_script_file = find_path_of_file_to_open
    File.open(currently_script_file) do |file|
      file.each do |line|
        puts line if line_matches_condition line
      end
    end
  end 

  private
  def find_path_of_file_to_open
  end 

  def line_matches_condition line
  end
end

The two collaborators are still empty, we can now TDD them and fill in their implementation.

What We Learned

Having gone through the above exercise a couple of things become abundantly clear:

  1. We form a more or less complete picture in our head of where we want our method to go before we start writing the tests or the implementation
  2. We need to have a reasonably good understanding of the tech we’re working with in order for us to write our tests first (i.e. how Ruby IO works, blocks, etc.)

What if you’re new to Ruby and have no idea how file IO works, what picture will you form in your head (a blank page) and how will you evolve the functionality through tests (with great difficulty). You’re much more likely to go off and do a little bit of research and try some stuff out in order to understand the tech you’re dealing with and of course as you’re trying stuff out, the solution to your current problem will naturally take shape and your method is more or less done. Once you’re fairly confident of what you’re doing you can delete and start over or you can just write the tests after the fact. In my opinion either way is correct. Of course I picked something simple to illustrate my example (Ruby file IO), but we run into this kind of situation all the time in our day to day work as developers. I need to write some code using a framework, API, language I am not really familiar with or am not an expert in, how do you fit TDD into that scenario – not easy?

And before we bring up the whole Spike argument, lets be clear. The research you do into tech you’re working on (but are not very familiar with) is not really a Spike, it is certainly Spike-like but it is not explicit. A spike has to be explicit (i.e. there is card for it). On the whole I consider this “research”, part of natural knowledge acquisition, so there is no need to throw your code out after you finished “doodling” (although you certainly can if you’re so inclined and are not pressed for time).

The amount of APIs, libraries, DSLs, formats, standards that a typical project deals with can be truly formidable, noone can be an expert on all of it. And by the time you start to get a handle on things you move on to a different project with a different tech stack and you’re “adrift” again. And when you come back to more familiar ground again it is not so familiar any more and so the cycle continues. Is it any wonder at this point that we tend to fall back into non-TDD from time to time?

Being Careless

The argument is that TDD lets you evolve your tests so that both your code and your tests are better as a result. I say being careful and thinking about what you’re doing is what allows you to have better code and better tests. Lets say we wrote the method above first and then decided to write the tests afterwards, we could potentially end up with something like this:

describe "bad tests read_file_and_print_line" do
  it "should forget to yield" do
    @file_ops.should_receive(:find_path_of_file_to_open).and_return("path to file") 

    mock_file = mock(File)
    File.should_receive(:open).with("path to file").and_return(mock_file) 

    @file_ops.read_file_and_print_line
  end 

  it "should forget to output" do
    @file_ops.should_receive(:find_path_of_file_to_open).and_return("path to file") 

    mock_file = mock(File)
    File.should_receive(:open).with("path to file").and_yield(mock_file) 

    mock_file.should_receive(:each).with(no_args()).and_yield("string1").and_yield("string2").and_yield("string3") 

    @file_ops.read_file_and_print_line
  end
end

Both of those tests pass, but are clearly nowhere near as good as our original pair. But being careful and conscientious developers we know this already, we wouldn’t leave them in this state. If we didn’t know our tech well enough and thought that these two tests were fine, no amount of TDD would have helped. It’s not about the TDD it’s about the knowledge, practice, attitude and experience.

The worst thing in my opinion is when you try to force the use of TDD where you would be better off without it. You don’t know the tech and yet you try to force the tests (which end up crap anyway), spend exorbitant amounts of time on them and get nowhere. In this situation TDD can seriously slow you down and when deadline is of the essence can you really afford that?

Look, TDD can be a great tool in your arsenal as a developer (especially if you can manage to TDD your acceptance or integration tests i.e. black box) but there is no need to be a purist about it and there is no need to feel guilty when you choose not to employ this particular tool.

For more tips and opinions on software development, process and people subscribe to skorks.com today.

Image by onkel_wart

  • Aaron

    This post is a nice example for new features being developed from the ground up by a developer.

    Where most developers get confused is whilst editing an existing piece of work. Its probably worth a whole blog entry itself, but in my experience I tend to go straight to the implementation and start hacking, providing there is suitable coverage of the code in the first place. Of course if there is low coverage on that existing piece the developer will have to write some tests to have any level of confidence to amend that code anyway.

    I also think “Test-First” often gets confused with “Test-Driven”, I think there is a distinction and Test-Driven does not necessarily mean write a test before you do anything.

    Nice post!

    • http://www.skorks.com Alan Skorkin

      It is actually an interesting point you make regarding Test first vs Test driven, I do believe you’re right that there is a distinction, but it is very fine and an argument could be made either way. I guess the question is, do we want to evolve our implementation through our tests or do we want to simply be conscious of making sure our code is covered, depending on how you look at it one can be a subset of the other. It is a point worth pondering.

  • Pingback: Dew Drop – February 1, 2010 | Alvin Ashcraft's Morning Dew()

  • Steve Conover

    Criticizing someone’s style is a bit like criticizing someone’s mom. Perhaps this is all a matter of taste and we can agree to disagree.

    I write code test-first every day and honestly if the mock-oriented style above were to take over I think I would elect to stop testing. I just don’t see the point, I don’t understand what it is that you think you’re testing…to me you’re just re-expressing the implementation one more time. The details of your implementation are strewn about and now you just have one more binding to that particular way of doing things.

    Couldn’t you just really make a file, really read it, maybe pass in a StringIO or something instead of stdout and examine what it prints? Then the assertion of postconditions can even come *after* the thing under test executes (Set up experiment. Run. Assert postconditions).

    Every time I get a test failure from a test that’s a mock/stub extravaganza I wonder, what’s the point of even having this “test”? Why do you care so much about the “how” – isn’t this just pouring a cast around my code rather than adding any value? Ideally (to me) tests read like experiments of a black box.

    But maybe I’m just a deeply committed state-based tester talking to a committed interaction tester.

    -Steve

    • http://www.skorks.com Alan Skorkin

      I actually really appreciate hearing this point of view, I tend to oscillate in my thinking about testing every few years, from “mocks are evil” to “mocks are great” (and in countless other ways as well). I believe your thinking about tests constantly evolves as you learn more and depending on the project you’re working in and the language you’re using and even your environment you may find yourself happily doing things with your tests that you never would have considered before.

      You’re right I could easily have used stringIO and not bothered with interaction testing. My current thinking is, if you’re doing unit tests then I do care about the interaction and how the method does it’s work, as soon as I go a level up to integration style tests, then black box is the way to go. But like I said, I do remember myself holding a different opinion at one point :).

      The ultimate goal is to produce better quality, more readable and maintainable code, if the testing style you use helps you do that, then that is a good thing.

  • Allan

    Hi, nice article, got me thinking.

    I don’t believe that the white box approach to TDD you describe is really a flexible approach.

    What if the developer decides that it’s more efficient to slurp the whole file into memory and iterate over that, emitting a character at a time? The end result is identical but the implementation is different and the tests fail. Now how does that fit into the whole refactoring scenario where your tests ensure you haven’t broken anything when making functionally neutral changes?

    I think a less invasive test that supplies input and expected output would be better, rather than prescribing the implementation also. As written above, if you change your implementation, you don’t know if it’s correct. At least not until you also change your test to match, which kind of defeats much of the purpose of tests…

    Allan

    • http://www.skorks.com Alan Skorkin

      That is definitely a good argument against interaction based unit tests, however if you employ that approach how do you drive your implementation through your tests? It is test driven development after all if all you supply is input and outputs, i can very easily dummy up the functionality of the method to get the test to pass, or implement incorrectly but still fir the tests that you have.

      You will need a great many tests to ensure correct implementation this way. With the interaction based approach, you can force your implementation where you want it to go with a reasonably number of tests (1 or 2). There are points for and against either approach and like I said, I do tend to change my mind from time to time regarding which is best.

      • Allan

        Being able to pass the tests given is the whole point. If the tests are not exhaustive (or sensible) then, yes, you can write an implementation that does nothing but pass the tests.

        I’d say that your interpretation of TDD is Test Defined Development no Test Driven Development :-)

        For me the, ability to refactor safely with more tests outweighs the brevity of the test suite.

        Allan

        • http://www.skorks.com Alan Skorkin

          I would definitely agree with that, write as many tests as you need to feel like you’ve adequately covered the code.

          My point is that with interaction based testing, you don’t have to write as many, since you cover the interactions there is only one or two ways the implementation can fall out.

          If you do intent based testing you’re forced to write many more tests simply to insure (i.e. drive) the correctness of the implementation.

      • Andrew

        One of the things I enjoy about TDD is that it really encourages me to think about what I want from a given method or object independently from how it would be implemented. I don’t care how it matches strings to determine which is the right one to select, I just want to make sure it follows my expectations for what it should and shouldn’t match.

        The process forces me to really think about what it is that I need first, and then that understanding drives implementation.

        The other benefit this has is that when someone else comes along and looks at the test, the test can communicate the intent, not the implementation. I feel like using mocks so heavily makes it harder to see that intent.

        • http://www.skorks.com Alan Skorkin

          You’ve picked up on my main point in that however you choose to write your tests as long as your way forces you to think about your implementation upfront and really try to understand what you’re doing your already a step ahead.

          You said it yourself ‘the understanding drives the implementation’, not the test. This is I think the difference between test driven and test first. My understanding is that strong TDD proponents argue that the tests can drive the implementation without you needed a deep understanding of what you’re trying to do. This is where it falls over for me.

  • http://twitter.com/petyosi Petyo Ivanov

    Nice article, bothered by the similar thoughts these days. Constantly re-reading the rspec book, and lurking David Chelimsky writings to figure stuff out.

    My two cents: There should be a clear distinction between white box and black box tests. I prefer to think of them as integration and and unit ones (would rather not argue over the names though).

    Integration tests set a goal. Passing an integration test means you have completed certain functionality. Unless you really have to, you are not supposed to mock there. They are the ones revealing intention, the ones describing the purpose of the task.

    Unit tests (or specs) are more like a design tool, supposed to help you doing it right. Feel free to mock and stub there, especially when you deal with interaction. You will probably cut yourself at first (at least I did, the first hundred times or so) doing it, it gets better over time.

    Of course, nothing can save you in case you don’t know what you’re trying to do (mocks and stubs may confuse you even more). Spikes and code tend to be thrown away can help here.

    Refactoring may break such kind of design oriented tests – but it is ok as long as the integration tests pass. By the way if you’re cool, you will do refactoring test-first.

    • http://www.skorks.com Alan Skorkin

      Hi Petyo,

      I think I am of a similar mindset at the moment, i.e. mock and stub all you like in unit tests, but integration tests are a different story. That is to say, unit tests are interaction tests while integration tests are functionality tests.

      And yeah definitely would be good to have more rspec stuff to read around the web. Hmm, maybe i’ll write some, I am no expert, but somebody gotta do it :).