I think I am of a similar mindset at the moment, i.e. mock and stub all you like in unit tests, but integration tests are a different story. That is to say, unit tests are interaction tests while integration tests are functionality tests.

And yeah definitely would be good to have more rspec stuff to read around the web. Hmm, maybe i’ll write some, I am no expert, but somebody gotta do it :).

My two cents: There should be a clear distinction between white box and black box tests. I prefer to think of them as integration and and unit ones (would rather not argue over the names though).

Integration tests set a goal. Passing an integration test means you have completed certain functionality. Unless you really have to, you are not supposed to mock there. They are the ones revealing intention, the ones describing the purpose of the task.

Unit tests (or specs) are more like a design tool, supposed to help you doing it right. Feel free to mock and stub there, especially when you deal with interaction. You will probably cut yourself at first (at least I did, the first hundred times or so) doing it, it gets better over time.

Of course, nothing can save you in case you don’t know what you’re trying to do (mocks and stubs may confuse you even more). Spikes and code tend to be thrown away can help here.

Refactoring may break such kind of design oriented tests – but it is ok as long as the integration tests pass. By the way if you’re cool, you will do refactoring test-first.

You said it yourself ‘the understanding drives the implementation’, not the test. This is I think the difference between test driven and test first. My understanding is that strong TDD proponents argue that the tests can drive the implementation without you needed a deep understanding of what you’re trying to do. This is where it falls over for me.

My point is that with interaction based testing, you don’t have to write as many, since you cover the interactions there is only one or two ways the implementation can fall out.

If you do intent based testing you’re forced to write many more tests simply to insure (i.e. drive) the correctness of the implementation.

The process forces me to really think about what it is that I need first, and then that understanding drives implementation.

The other benefit this has is that when someone else comes along and looks at the test, the test can communicate the intent, not the implementation. I feel like using mocks so heavily makes it harder to see that intent.

I’d say that your interpretation of TDD is Test Defined Development no Test Driven Development :-)

For me the, ability to refactor safely with more tests outweighs the brevity of the test suite.

Allan

You will need a great many tests to ensure correct implementation this way. With the interaction based approach, you can force your implementation where you want it to go with a reasonably number of tests (1 or 2). There are points for and against either approach and like I said, I do tend to change my mind from time to time regarding which is best.

I don’t believe that the white box approach to TDD you describe is really a flexible approach.

What if the developer decides that it’s more efficient to slurp the whole file into memory and iterate over that, emitting a character at a time? The end result is identical but the implementation is different and the tests fail. Now how does that fit into the whole refactoring scenario where your tests ensure you haven’t broken anything when making functionally neutral changes?

I think a less invasive test that supplies input and expected output would be better, rather than prescribing the implementation also. As written above, if you change your implementation, you don’t know if it’s correct. At least not until you also change your test to match, which kind of defeats much of the purpose of tests…

Allan

You’re right I could easily have used stringIO and not bothered with interaction testing. My current thinking is, if you’re doing unit tests then I do care about the interaction and how the method does it’s work, as soon as I go a level up to integration style tests, then black box is the way to go. But like I said, I do remember myself holding a different opinion at one point :).

The ultimate goal is to produce better quality, more readable and maintainable code, if the testing style you use helps you do that, then that is a good thing.

I write code test-first every day and honestly if the mock-oriented style above were to take over I think I would elect to stop testing. I just don’t see the point, I don’t understand what it is that you think you’re testing…to me you’re just re-expressing the implementation one more time. The details of your implementation are strewn about and now you just have one more binding to that particular way of doing things.

Couldn’t you just really make a file, really read it, maybe pass in a StringIO or something instead of stdout and examine what it prints? Then the assertion of postconditions can even come *after* the thing under test executes (Set up experiment. Run. Assert postconditions).

Every time I get a test failure from a test that’s a mock/stub extravaganza I wonder, what’s the point of even having this “test”? Why do you care so much about the “how” – isn’t this just pouring a cast around my code rather than adding any value? Ideally (to me) tests read like experiments of a black box.

But maybe I’m just a deeply committed state-based tester talking to a committed interaction tester.

-Steve