April 28, 2008

Shoulda used this earlier

In many ways, testing software is like going out and getting exercise. You know you should do it, and you know it does you good, but it's also pretty easy to find an excuse to skip it (I'll make it up tomorrow).

So anything that makes testing easier is good, because it cuts down on the excuses not to do it.

One thing I've never really liked about the conventional xUnit-style testing frameworks was the setup and teardown structure. In these frameworks, a test case is a class, and setup and teardown are implemented by methods in that class. Each test is also a method, so the basic flow is

  for each test method in the class
    run setup
    run the test method
    run teardown
  end

Nice and simple. Each test method got the benefit of a standard environment created by the setup method, and the teardown method got the job of tidying up after.

Except… when I'm writing tests, I typically want to set up lots of different scenarios. I'll want A and B and C, then A and B but not C, then A and not B, then A and D, and so on. I had two choices—write lots of test case classes, using subclassing to inherit common setup behavior, or write per-test method setup code (often factored out into helpers). In the end, I almost always did the latter, And that was tedious, and it made it harder to see the tests for the setup code.

I flirted with RSpec. Its spec framework seemed to have what I wanted. But I just couldn't get myself to enjoy using it. (I think it's a cat people/dog people kind of thing)

Enter shoulda

Then, a couple of weeks back, Mike Clark and Chad Fowler introduced me to shoulda. Shoulda isn't a testing framework. Instead, it extends Ruby's existing Test::Unit framework with the idea of test contexts. A context is a section of your test case where all the test methods have something in common. At it simplest, a context could be simply used as an annotation device (and, yes, this is a silly example):

context "My factorial method" do
  should "return 1 when passed 0" do
    assert_equal 1, fact(0)
  end
  should "return 1 when passed 1" do
    assert_equal 1, fact(1)
  end
  should "return 6 when passed 3" do
    assert_equal 6, fact(3)
  end
end    

The stuff in a context can share common setup code—just write a setup block.

class CartTest < Test::Unit::TestCase

  context "An empty cart" do
    setup do
      @cart = orders(:wilmas_empty_cart)
    end

    should "have no line items" do
      assert_equal 0, @cart.line_items.size
    end

    should "have a zero price" do
      assert_equal 0, @cart.price
    end
  end

  context "Some other context..." ...
  end
end

So now, within a single test case I can set up multiple contexts, and each context can have its own environment.

But, take it back to my original problem. I often want to set up hierarchies of related environments for my tests. The shaoulda code handles this wonderfully, because it lets me nest contexts. For example, I'm adding a feature to our store that gives customers some additional information if, during checkout, their credit card transaction was initially rejected because the address was wrong, and was then accepted when they fixed the address. I wanted two tests, one without the prior address error, and one with.

To set up this environment, I needed to set up a shopping cart, create a dummy response from our payment gateway, and post that response to the application. In the case of the prior address error, I also wanted to inject an entry containing that error into the transactions associated with the order prior to generating the response.

With shoulda, I simply created some nested contexts. The top level context did the shared setup, and the inner contexts then set up appropriate environments for their tests. It looked like this:

  context "Checking out"  do
    setup do
      @cart = cart_named(:freds_full_cart)
      @cart.prepare_for_store_authorize!
      @params = approved_authnet_response(@cart)
    end                  
    
    context "with no AVS errors in CC transaction history" do
      setup do
        post :post_from_authnet_authorize, @params
      end

      should_redirect_to "{:action => :receipt}"
    end 
    
    context "with AVS errors in CC transaction history" do
      setup do
        avs_error = CcTransaction.new(:response_code => 2, :response_reason_code => 27)
        @cart.cc_transactions << avs_error
        post :post_from_authnet_authorize, @params
      end

      should_redirect_to "{:action => :explain_avs_mismatch}"
    end
  end 

The outer setup gets run before the execution of each of the inner contexts. And the setup in the inner contexts gets run when running that context. And shoulda keeps track of it all, so I get very natural error messages if an assertion fails. For example, if the test in the second context above fails, I'd get

Checking out with AVS errors in CC transaction history should 
redirect to "{:action => :explain_avs_mixsmatch}". 

So, now, I can finally set up my hierarchies of test environments in a natural way. It isn't revolutionary. It's just one less excuse for not testing…

April 11, 2008

Ruby 1.9 Standard Library Changes

Here's a top-level overview of some of the changes to date in the standard library that comes with Ruby 1.9. (These are the libraries that you get preinstalled with Ruby, but that you have to require into your code.)

  • The base64 library has been removed. Use Array#pack and String#unpack instead.
  • Much of the Complex and Rational libraries are now built in to the interpreter. However, requiring the external libraries adds additional functionally. In the case of Rational, this functionality is minimal.
  • The CMath library has been added.
  • The Enumerator library is now built in.
  • Added Fiber library (adds coroutine support to fibers).
  • Removed ftools (replaced by fileutils).
  • The Generator library has been removed (use Fibers).
  • Added notes on using irb from inside applications.
  • jcode is removed in favor of built-in encoding support.
  • The json library is added.
  • The matrix library no longer requires that you include mathn.
  • The mutex library is now built in.
  • parsedate has been removed. The Date class handles most of its functionality.
  • readbytes has been removed. IO now supports the method directly.
  • require_relative added.
  • Add description of Ripper.
  • Add description of SecureRandom.
  • I've omitted the shell library, as it seems more like a curiosity than something folks would use (and it's broken under 1.9).
  • The soap library is removed.
  • I've omitted the sync library. It is broken under 1.9, and the monitor library seems to be cleaner.
  • Win32API is now deprecated in favor of using the DL library.

It's interesting to me just how much is still changing in Ruby 1.9. But, as I use it more and more, it's also gratifying to see how some of the new idioms make coding just that little sweeter.

I just pushed a new beta of the PickAxe Third Edition with all the library changes.

April 09, 2008

BabyDoc

One of the fun things about updating the PickAxe is getting to come up with examples to show the various APIs in action. Here's a very silly example of using Ripper's event-based API to extract comments that are associated with basic class definitions. It clearly has holes (it doesn't handle class A::B::C, for instance) but it's fairly easy to see how to add a proper state machine and produce something that might be interesting to play with...

require 'ripper'

# This class handles parser events, extracting
# comments and attaching them to class definitions
class BabyRDoc < Ripper::Filter
  def initialize(*)
    super
    reset_state
  end

  def on_default(event, token, output)
    reset_state
    output
  end

  def on_sp(token, output) output end
  alias on_nil on_sp

  def on_comment(comment, output)
    @comment << comment.sub(/^\s*#\s*/, "    ")
    output
  end

  def on_kw(name, output)
    @expecting_class_name = (name == 'class')
    output
  end

  def on_const(name, output)
    if @expecting_class_name
      output << "#{name}:\n"
      output <<  @comment
    end
    reset_state
    output
  end

  private

  def reset_state
    @comment = ""
    @expecting_class_name = false
  end
end

BabyRDoc.new(File.read(__FILE__)).parse(STDOUT)

Run this with Ruby 1.9 (or, I guess, 1.8 with Ripper installed), and you'll see

BabyRDoc: 
    This class handles parser events, extracting 
    comments and attaching them to class definitions

April 08, 2008

Fun with Ruby 1.9 File Encodings

Ruby 1.9 allows you to specify the character encodings of I/O streams, strings, regexps, symbols, and so on. It also lets you specify the encoding of individual source files (and a complete application can be built from many files, each with different character encodings). Expect to start seeing a rash of obscure source code, at least until the initial excitement abates and cooler thinking prevails.

In the meantime, we can get away with


# encoding: utf-8
require 'mathn'
class Numeric
   def ℃
     (self - 32) * 5/9
   end
   def ℉
     self * 9/5 + 32
   end
end
 
puts 212.℃
puts 100.℉

Or, for those who'd like a peek at the start of a road that eventually leads to madness:


alias ✎ puts 
 
✎ 212.℃
✎ 100.℉

I'm betting this post displays badly on about 50% of the machines that are used to view it. Which is reason enough to tread very lightly down this path…

April 03, 2008

Importing RSS Feeds into Mail.app

As an experiment, I wanted to try using Mail.app instead of NetNewswire to read RSS feeds (I know, I know...). To get my existing feeds across, I did the following.

  1. In NNW, export your feeds as an OPML file (say to ~Desktop/MySubscriptions.opml)
  2. In the directory holding that file, do
  3. % egrep -o 'xmlUrl="(.*?)"' MySubscriptions.opml | egrep -o 'http.*[^"]' | pbcopy
    
  4. In Mail.App, select “Add RSS Feed”, and in the text area that pops up, paste in the clipboard.

March 24, 2008

I'd like to apologize

Just before the weekend, I did something stupid and hurtful, and I'd like to make it right.

For the last few weeks, I've been getting e-mails about a book under development over at O'Reilly called Software Craftsmanship—from Apprentice to Journeyman. People kept pointing out that the main title was the same as Pete McBreen's book (for which I wrote the foreword), and that the overall structure of the title was similar to that of The Pragmatic Programmer.

After a while, this started to get under my skin. I wasn't so much concerned about the “journeyman” bit, but the duplication of the title just seemed wrong to me—I really liked Pete's book, and I didn't want to see it getting eclipsed. I complained about this to a senior editor at O'Reilly, and he said he'd bring it up with the book's editor, who worked for him. I heard nothing back.

So, at the end of a tiring week, I wrote a blog post, complaining about the title.

That was wrong of me.

It was wrong for a number of reasons.

  • I could, and probably should, have bypassed etiquette and contacted the authors directly, even though they write for a rival publisher.
  • It really wasn't any of my business.
  • But, most importantly, it took something which was a kind of intellectual annoyance and turned it into something that made the authors of the book feel bad. And for that, I apologize.

This year, I've been the target of some cruel blog posts. Most readers of these posts viewed them as fine sport. But as the recipient of the criticism, I'm here to tell you that it hurts. It doesn't matter whether it is based on truth or whether it isn't. It doesn't matter whether the person writing them knows you or is a total stranger. It hurts. Public attacks like this are virtually impossible to defend against, and that is a cruel violation. It's cruel when it is done to you, and it's cruel when you do it to others.

So, I of all people should have known better. I should have had the common sense to realize that my comments, aimed at a book, were going to be hurtful to the authors. It's kind of obvious, really.

I wasn't thinking straight, and I messed up.

So, Dave and Ade, I'm sorry for any distress I caused.

Good luck with your book.

March 19, 2008

Ruby 1.9 Built-in Library--Finished First Pass

One of the scary things about revving the PickAxe for Ruby 1.9 is updating the reference section where I describe all the built-in classes and methods. It involves working through the interpreter source, looking for all the rb_define_method calls (and their friends) and then reading the C implementation of the corresponding methods. Many methods are unchanged from 1.8. But, at the same time, many have changed. Often they take an additional parameter, or return an Enumerator where previously they required a block. Then there are the new classes (something like 6 of them) and new methods. (It looks like there are over 200 new built-in methods in the current Ruby 1.9).

All in all, I count something like 300 [1.9] flags in the new library reference. Some flag stuff is as trivial as a change of a default return type, while others flag entire new classes.

It's incredibly time-consuming work, and I'm constantly grumbling while doing it. But I come out the other end knowing a whole bunch about the library, and with a deeper respect for the folks who maintain it.

March 18, 2008

Complex and Rational are now built-in to 1.9

Just when I thought I'd finished documenting the standard library for the new PickAxe, I did one last svn up of the Ruby interpreter source and discovered that the Complex and Rational classes are now builtins—no need to require the library to get the basic functionality. The change also affects a number of other built-in classes (you can now say nil.to_c, for example). I'm not 100% sure I agree with rolling in Complex, but the addition of rational numbers is a welcome change.

Unfortunately it's back to the drawing board on my plans to release a new beta today...

March 14, 2008

Source Code for that Testing Library

Ring the bells that still can ring
Forget your perfect offering
There is a crack in everything
That's how the light gets in.
  —Leonard Cohen

Yesterday, I posted on a trivial little testing library I hacked together. I've put the source online. Get the source through Rob's git repository (see below).

In the meantime, I discovered a problem with the idea of intercepting comparison operators, the technique used by the expect method. Ruby doesn't really have != and !~ methods. Instead, the parser maps (a != b) into !(a == b). This means that the ComparisonProxy cannot intercept calls to either of these. This is a problem because

   expect(1) != 1
actually passes, because it becomes !(expect(1) == 1), and the expect method is happy with that.

I'm betting there's a way around this...

Update: 14:26 CDT.

  • Rob Sanheim has set up a Git repository for the code. He says

    I've put this up on github to watch what forks or releases develop around it.

    
    git clone git://github.com/rsanheim/prag_dave_testing.git
    
  • Michael Neumann suggested a way around the negated == and =~ tests using source inspection:

    
     class ComparatorProxy
       def ==(obj)
         # try to get the source code position of the call
         # and see if it's a != or a ==
       end
     end
    

March 13, 2008

Playing with a Testing Library

  • I'd like to be able to express my unit tests fairly naturally, using the conditional operators built into the language. So, for example, I'd want to write:

    
          expect(factorial(5)) == 120
          expect(factorial(10)) > 10000
        
  • I'd like the error messages to show both the code that caused the error and the values that caused the error. So, for example, I'd want the following (incorrect) test

    
          expect(factorial(6)) == 600
        

    to output something like

          /Users/dave/tmp/tmc/blog_tests.rb:16
              the code was: expect(factorial(6)) == 600,
              but 720 != 600
        

    and

    
          expect(1) > 2
        

    should say

    
          /Users/dave/Play/tmc/blog_tests.rb:11
          	the code was: expect(1) > 2,
          	but 1 <= 2
        

    (Note how the expression showing the actual values negates the comparison operator to make it easier to read.)

  • I annotate my code with comments, so I'd like to be able to annotate my tests the same way.
    
        expect(factorial(6)) == 600  # Deliberate bad test
      

    should produce something like

        /Users/dave/tmp/tmc/blog_tests.rb:17
           Deliberate bad test
           the code was: expect(factorial(6)) == 600,
           but 720 != 600  
      

    Sometimes I write longer comments.

    
       # The factorial of 6 is a special case,
       # because of the labor laws in Las Vegas
       expect(factorial(6)) == 600
       

    So the resulting errors are longer, too.

               
       /Users/dave/Play/tmc/blog_tests.rb:21
          The factorial of 6 is a special case, because of the labor laws in Las Vegas
          the code was: expect(factorial(6)) == 600,
          but 720 != 600
       
  • I like to be able to group my tests.

          testing("positive factorials") do
            expect(factorial(1)) == 1
            expect(factorial(2)) == 2
            expect(factorial(5)) == 120
          end
    
          testing("factorial of zero") do
            expect(factorial(0)) == 1
          end
    
          testing("negative factorials") do
            expect(factorial(-1)) == 1
            expect(factorial(-5)) == 1
          end
        
  • I like the description of the group to appear along with any individual test annotation if a test fails.

    
          testing("factorial of zero") do
            # this test is deliberately wrong
            expect(factorial(0)) == 0
          end
        
    will produce
          /Users/dave/Play/tmc/blog_tests.rb:31--while testing factorial of zero
              this test is deliberately wrong
              the code was: expect(factorial(0)) == 0,
              but 1 != 0
        
  • I like to have the flexibility to set up the environment for a group of tests. I also like to have the idea of a global environment which doesn't get messed up by the running of tests (so that subsequent tests can run in that environment. I don't see why I should have to package things into methods with magic names to have that happen. Instead, why not just have transactional instance variables? That way, I can use regular methods to set up the state for a test.

          @order = Order.new("Dave Thomas", "Ruby Book")
    
          testing("normal case") do
            expect(@order.valid?) == true
          end
    
          testing("missing name in order") do
            @order.name = nil
            expect(@order.valid?) == false
            expect(@order.error)  == "missing name"
          end
    
          # Check that order is reset to valid state here
          expect(@order.valid?) == true
        

    So, in the preceding case, the second testing block changed the @order object. However, once the block terminated, the object was restored to its initial (valid) state.

So, while waiting for the last day of the Rails Studio to start, I hacked together a quick proof of concept. It's less than 100 lines of code. All the output shown here was generated by it. Is this worth developing into something usable?

Now in Beta

  • Programming Ruby, 3rd Edition
    Third Edition, Covering Ruby 1.9, now in beta
My Photo

Site Search

  • Google Search

    The web
    PragDave

Pragmatic Stuff

Photos

  • www.flickr.com
    This is a Flickr badge showing public photos from pragdave tagged with pragdave_badge. Make your own badge here.