A Unit Testing Framework In 44 Lines Of Ruby

TestingLate last year I attended some workshops which were being run as part of the YOW Melbourne developer conference. Since the workshops were run by @coreyhaines and @jbrains, TDD was a prominent part. Normally this would not be an issue, but in a spectacular display of fail (considering it was a developer conference in 2010), the internet was hard to come by, which left me and my freshly installed Linux laptop without the ability to acquire Rspec. Luckily a few weeks before, I decided to write a unit testing framework of my very own (just because I could :)) and so I had a reasonably fresh copy of that code lying around – problem solved. But, it got me thinking, what is the minimum amount of code needed to make a viable unit testing framework?

A Minimum Viable Unit Test

I had some minimal code when I first toyed with writing a unit testing framework, but then I went and ruined it by adding a bunch of features :), fortunately it is pretty easy to recreate. What we really want is the ability to execute the following code:

describe "some test" do 
  it "should be true" do 
    true.should == true
  end
 
  it "should show that an expression can be true" do
    (5 == 5).should == true
  end
 
  it "should be failing deliberately" do
    5.should == 6
  end
end

As you can see it looks very much like a basic Rspec test, let's try and write some code to run this.

Building A Basic Framework

The first thing we're going to need is the ability to define a new test using "describe". Since we want to be able to put a “"describe"” block anywhere (e.g. its own file), we're going to have to augment Ruby a little bit. The "puts" method lives in the Kernel module and is therefore available anywhere (since the Object class includes Kernel and every object in Ruby inherits from Object), we will put describe inside Kernel to give it the same ability:

module Kernel
  def describe(description, &block)
    tests = Dsl.new.parse(description, block)
    tests.execute
  end
end

As you can see, "describe" takes a description string and a block that will contain the rest of our test code. At this point, we want to pull apart the block that we're passing to "describe" to separate it into individual examples (i.e. "it" blocks). For this we create a class called Dsl and pass our block to its parse method, this will produce an object that will allow us to execute all our test, but let's not get ahead of ourselves. Our Dsl class looks like this:

class Dsl 
  def initialize
    @tests = {}
  end
  def parse(description, block)
    self.instance_eval(&block)
    Executor.new(description, @tests)
  end
  def it(description, &block)
    @tests[description] = block
  end
end

What we do here is evaluate our block in the context of the current Dsl object:

self.instance_eval(&block)

Our Dsl object has an "it" method which also takes a description and a block and since that is exactly what our describe block contains everything works well (i.e. we're essentially making several method calls to the "it" method each time passing in a description and a block). We could define other methods on our Dsl object and those would become part of the "language" which will be available to us in the "describe" block.

Our "it" method will be called once for every "it" block in the describe block, every time that happens we simply take the block that was passed in and store it in hash keyed on the description. When we're done, we simply create a new Executor object which we will use to iterate over our test blocks, call them and produce some results. The executor looks like this:

class Executor
  def initialize(description, tests)
    @description = description
    @tests = tests
    @success_count = 0
    @failure_count = 0
  end
  def execute
    puts "#{@description}"
    @tests.each_pair do |name, block|
      print " - #{name}"
      result = self.instance_eval(&block)
      result ? @success_count += 1 : @failure_count += 1
      puts result ? " SUCCESS" : " FAILURE"
    end
    summary
  end
  def summary
    puts "\n#{@tests.keys.size} tests, #{@success_count} success, #{@failure_count} failure"
  end
end

Our executor code is reasonably simple. We print out the description of our "describe" block we then go through all the "it" blocks we have stored and evaluate them in the context of the executor object. In our case there is no special reason for this, but it does mean that the executor object can also be a container for some methods that can be used as a "language" inside "it" blocks (i.e. part of our dsl can be defined as method on the executor). For example, we could define the following method on our executor:

def should_be_five(x) 
  5 == x
end

This method would then be available for us to use inside our "it" blocks, but for our simple test it is not necessary.

So, we evaluate our "it" blocks and store the result, which is simply going to be the return value of the last statement in the "it" block (as per regular Ruby). In our case we want to make sure that the last statement always returns a boolean value (to indicate success or failure of the test), we can then use it to produce some meaningful output.

We are missing one piece though, the "should" method, we had code like:

true.should == true
5.should == 5

It seems that every object has the "should" method available to it and that's exactly how it works:

class Object
  def should
    self
  end
end

The method doesn't really DO anything (just returns the object); it simply acts as syntactic sugar to make our tests read a bit better.

At this stage, we just take the result of evaluating the test, turn it into a string to indicate success or failure and print that out. Along the way we keep track of the number of successes or failures so that we can produce a summary report in the end. That's all the code we need, if we put it all together, we get the following 44 lines:

module Kernel
  def describe(description, &block)
    tests = Dsl.new.parse(description, block)
    tests.execute
  end
end 
class Object
  def should
    self
  end
end
class Dsl 
  def initialize
    @tests = {}
  end
  def parse(description, block)
    self.instance_eval(&block)
    Executor.new(description, @tests)
  end
  def it(description, &block)
    @tests[description] = block
  end
end
class Executor
  def initialize(description, tests)
    @description = description
    @tests = tests
    @success_count = 0
    @failure_count = 0
  end
  def execute
    puts "#{@description}"
    @tests.each_pair do |name, block|
      print " - #{name}"
      result = self.instance_eval(&block)
      result ? @success_count += 1 : @failure_count += 1
      puts result ? " SUCCESS" : " FAILURE"
    end
    summary
  end
  def summary
    puts "\n#{@tests.keys.size} tests, #{@success_count} success, #{@failure_count} failure"
  end
end

If we "require" this code and execute our original sample test, we get the following output:

some test
 - should be true SUCCESS
 - should show that an expression can be true SUCCESS
 - should be failing deliberately FAILURE

3 tests, 2 success, 1 failure

Nifty! Now, if you're ever stuck without a unit testing framework and you don't want to go cowboy, just spend 5 minutes and you can whip up something reasonably viable to tide you over. I am, of course, exaggerating slightly; you will quickly start to miss all the extra expectations, better output, mocking and stubbing etc. However we can easily enhance our little framework with some of those features (e.g. adding extra DSL elements) – the development effort is surprisingly small. If you don't believe me, have a look at bacon it is only a couple of hundred lines and is a reasonably complete little Rspec clone. Attest – the framework that I wrote is another decent example (even if I do say so myself :P). Both of them do still miss any sort of built-in test double support, but adding that is a story for another time.

Image by jepoirrier

  • http://pdelgallego.com/ pedro

    Nice post, and perfect timing for me I was writing yesterday my own framework. it’s a little bit bigger (~90 loc) but includes pluggable matchers, before and afters, pending examples, coloring …

  • http://pdelgallego.com/ pedro

    ah, and the url to the code in case someone want to take a look and fork
    https://github.com/pedrodelgallego/ackee/blob/master/lib/ackee.rb

  • Pingback: Tweets that mention A Unit Testing Framework In 44 Lines Of Ruby -- Topsy.com

  • http://jmrtn.com/notes James Martin

    Nicely done, Alan. Did you write tests for it, first? :)

    • http://www.skorks.com Alan Skorkin

      Funny you should say that, I spent many an hour raking my brain over how to test a unit testing framework using the framework itself :).

  • http://lucapette.com lucapette

    Very nice. A good example of ruby features too. Thanks for sharing

  • http://www.hackinghat.com Steve Knight

    My 2c.

    This meta-issue has cropped up before for me too. Given the lack of acceptable framework (for any given scenario: lack of internet, lack of implementation, lack of whatever) how long would it actually take to knock something up.

    The answer, usually, is not very long because usually I only require a small subset of the total functionality of the framework.

    Sometimes the time spent on integrating a half-matching framework far outweighs the cost of just knocking something up.

    The problem I’ve found is when you start scaling up. But if you make your knocked-up version follow a similar interface to the de-facto standard in the area then you won’t lose out too badly if you need to switch.

    • http://www.skorks.com Alan Skorkin

      That’s exactly right, you can always switch if you need more power as long as your dsl is similar/the same.

  • http://ngauthier.com Nick Gauthier

    Also check out minitest/spec

    https://github.com/seattlerb/minitest

    MiniTest replaces Test::Unit in 1.9, and it has a minitest/spec “library” that gives the above functionality. It’s available in any ruby 1.9 installation already.

    • http://www.skorks.com Alan Skorkin

      That does look pretty cool, definitely need to have a play with it.

  • http://metaskills.net/ Ken Collins

    Yes, MiniTest::Spec is awesome. It currently has a soon to be fixed bug that blows away super class access in describe block scopes. I did an article about switching all of my Test::Unit code over to MiniTest::Spec by building a minimal shoulda syntax on top of it.

    http://metaskills.net/2011/01/25/from-test-unit-shoulda-to-minitest-spec-minishoulda/

    Once MiniTest::Spec fixes the describe block scope, building frameworks on top of it should be a breeze!

  • Guillermo

    Liked it, you can write a minimal mocking/stubbing framework for the next post :D

    • http://www.skorks.com Alan Skorkin

      Haha, I’ll see what I can do.

  • http://wetherubyists.com Justin Baker

    I also wrote my own testing framework the other day:

    https://github.com/justinbaker/huh

    not as much metaprogramming though

  • Pingback: A Smattering of Selenium #42 « Official Selenium Blog

  • Peter

    I’m not so sure about internet being hard to come by at a conference in 2010 being a spectacular bit of fail. I went to a couple last year and hotels just don’t have the capacity to handle a couple hundred people on wifi at once. It’s really sad but not surprising.

  • Pingback: Ruby Unit Testing Framework in 44 Lines of Code | ChurchCode

  • Pingback: Delicious Bookmarks for July 24th from 10:12 to 10:37 « Lâmôlabs

  • http://wowkhmer.com Samnang Chhun

    Nice post! I get one question, you verify the test by check the result from execute ‘it’ block, so how do you handle it if we get two asserts in ‘it’ block, eg:

    it ‘multi asserts’ do
    2.should == 3
    5.should == 5
    end

    In this case, the test still pass.

  • Pingback: 44行写就Ruby单元测试框架 - 博客 - 伯乐在线

  • Pingback: (Possibly) The World’s Smallest Ruby Unit Test Tool | Nat On Testing

  • http://www.natontesting.com Nat Ritmeyer