Playing with a Testing Library

• I'd like to be able to express my unit tests fairly naturally, using the conditional operators built into the language. So, for example, I'd want to write:

  
        expect(factorial(5)) == 120
        expect(factorial(10)) > 10000
      

• I'd like the error messages to show both the code that caused the error and the values that caused the error. So, for example, I'd want the following (incorrect) test

  
        expect(factorial(6)) == 600
      

  to output something like

        /Users/dave/tmp/tmc/blog_tests.rb:16
            the code was: expect(factorial(6)) == 600,
            but 720 != 600
      

  and

  
        expect(1) > 2
      

  should say

  
        /Users/dave/Play/tmc/blog_tests.rb:11
        	the code was: expect(1) > 2,
        	but 1 <= 2
      

  (Note how the expression showing the actual values negates the comparison operator to make it easier to read.)

• I annotate my code with comments, so I'd like to be able to annotate my tests the same way.

  
      expect(factorial(6)) == 600  # Deliberate bad test
    

  should produce something like

      /Users/dave/tmp/tmc/blog_tests.rb:17
         Deliberate bad test
         the code was: expect(factorial(6)) == 600,
         but 720 != 600  
    

  Sometimes I write longer comments.

  
     # The factorial of 6 is a special case,
     # because of the labor laws in Las Vegas
     expect(factorial(6)) == 600
     

  So the resulting errors are longer, too.

             
     /Users/dave/Play/tmc/blog_tests.rb:21
        The factorial of 6 is a special case, because of the labor laws in Las Vegas
        the code was: expect(factorial(6)) == 600,
        but 720 != 600
     

• I like to be able to group my tests.

        testing("positive factorials") do
          expect(factorial(1)) == 1
          expect(factorial(2)) == 2
          expect(factorial(5)) == 120
        end
  
        testing("factorial of zero") do
          expect(factorial(0)) == 1
        end
  
        testing("negative factorials") do
          expect(factorial(-1)) == 1
          expect(factorial(-5)) == 1
        end
      

• I like the description of the group to appear along with any individual test annotation if a test fails.

  
        testing("factorial of zero") do
          # this test is deliberately wrong
          expect(factorial(0)) == 0
        end
      

  will produce

        /Users/dave/Play/tmc/blog_tests.rb:31--while testing factorial of zero
            this test is deliberately wrong
            the code was: expect(factorial(0)) == 0,
            but 1 != 0
      

• I like to have the flexibility to set up the environment for a group of tests. I also like to have the idea of a global environment which doesn't get messed up by the running of tests (so that subsequent tests can run in that environment. I don't see why I should have to package things into methods with magic names to have that happen. Instead, why not just have transactional instance variables? That way, I can use regular methods to set up the state for a test.

        @order = Order.new("Dave Thomas", "Ruby Book")
  
        testing("normal case") do
          expect(@order.valid?) == true
        end
  
        testing("missing name in order") do
          @order.name = nil
          expect(@order.valid?) == false
          expect(@order.error)  == "missing name"
        end
  
        # Check that order is reset to valid state here
        expect(@order.valid?) == true
      

  So, in the preceding case, the second testing block changed the @order object. However, once the block terminated, the object was restored to its initial (valid) state.

So, while waiting for the last day of the Rails Studio to start, I hacked together a quick proof of concept. It's less than 100 lines of code. All the output shown here was generated by it. Is this worth developing into something usable?

Comments

[Dave Thomas]

Nicolás:

It does indeed support comparisons other than '==', and adding custom matchers would be trivial.

Dave

[Christian Romney]

Release early, release often.

[Dave Thomas]

Gregor:

Having expect{ 1 == 2} wouldn't give me access to the individual values when reporting errors--I can only change the meaning of == after the code being tested has run.

Dave

[Eric Broyles]

I love the idea of using code comments for your assertion messages. This is something I've always wanted as it makes for much cleaner code while providing useful output. I also like the grouping concept.

I'd love to know as soon as this is available.