Opinionated Programmer - Jo Liss's musings on enlightened software development.

Why I stopped using assert{ 2.0 }

Update Mar 31, 2011: Check out wrong for a newer assertion library inspired by Assert{ 2.0 }. (Personally, I’ve come to like RSpec’s simple should syntax by now – see the note at the bottom – but wrong should be a great library if you like Assert{ 2.0 }’s style.)

Assert{ 2.0 } lets you write assert { x == 42 } and gives you helpful error messages through introspection when the assertion fails. I used it in a project for a while, but then decided to move away from it. Here’s why.

1. First, why assert{ 2.0 }?

Assert{ 2.0 } allows you to stop writing

1
assert_equal 42, x

and start writing

1
assert { x == 42 }

without wrecking your test diagnostics. If the test fails, it generates good error messages through code introspection. (The author wrote a longer introduction a while back if you’d like more details.) Before you say “that’s just overly cute syntax,” consider this:

1
assert page.has_content?('foo')

It’s not possible to rephrase this with one of the (proliferated) assert* methods, and without a message string you get really unusable output if the test fails. So you essentially end up having to write a helper assert* method, or add an unDRY message string. Assert{ 2.0 } fixes this by letting you write

1
assert { page.has_content?('foo') }

and still get usable failure messages.

2. So why not assert{ 2.0 }?

There are three things that in the end made me abandon assert{ 2.0 } – let me elaborate one-by-one:

2.1. Spurious tests

The first issue I encountered was that, just by adding

1
gem 'assert2'  # even for Ruby 1.9, despite what the website says

to the Gemfile of your Rails 3 project, a spurious empty test suite show up:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ rake test
(in /home/jo/src/mercury)
Loaded suite /var/lib/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader
[... unit tests run ...]
Loaded suite /var/lib/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader
[... functional tests run ...]
Loaded suite /var/lib/gems/1.9.1/gems/rake-0.8.7/lib/rake/rake_test_loader
[... integration tests run ...]
Loaded suite /var/lib/gems/1.9.1/bin/rake
Started

Finished in 0.000900 seconds.

0 tests, 0 assertions, 0 failures, 0 errors, 0 skips

Test run options: --seed 16595

That’s really weird. And it gets stranger: It even happens when you run the rails command:

1
2
3
4
5
6
7
8
9
10
$ rails generate doesnotexist
Could not find generator doesnotexist.
Loaded suite script/rails
Started

Finished in 0.000863 seconds.

0 tests, 0 assertions, 0 failures, 0 errors, 0 skips

Test run options: --seed 63004

I have no idea what’s going on, but this is wrong. Perhaps it’s just a problem with Rails 3 compatibility, but then again, assert{ 2.0 } claims to be compatible.

(Update Jan 10, 2011: Apparently this happens via an exit handler that Ruby 1.9’s minitest installs. Just in case anyone wants to to dig deeper and cares for a starting point.)

2.2. Unreviewed code

Now digging into the intestines of the magic reflection code that picks up the tests is not my idea of a fun afternoon. But still I open the source, and here’s what it says at the top of assert2-0.5.5/lib/assert2.rb:

1
2
3
#  TODO  add :verbose => option to assert{}
#  TODO  pay for Staff Benda Bilili  ALBUM: Très Très Fort (Promo Sampler) !
#  TODO  evaluate parts[3]

Ahem. ;-) So apparently some automated to-do system wrote into the wrong file. To be sure, that’s funny (in a good way). But on second thought, it also indicates that Phlip didn’t review the diff before he checked in changes to his code. And that, I like to think, is bad practice.

So at this point, I’m starting to lose faith in the code. And all it takes for me is a final nail:

2.3. Overly bold code

I scroll around in the file and see this at bottom:

1
2
3
4
5
class File
  def self.write(filename, contents)
    open(filename, 'w'){|f|  f.write(contents)  }
  end
end

I’m not opposed to changing standard classes per se, but frankly, this is excessive. (And I don’t think it’s even used once.)

2.4. Moving away is difficult

At this point, you might say, “okay, so maybe assert{ 2.0 } isn’t the cleanest piece of library code, but why not use it, and if it becomes unmaintainable or breaks, just replace all instances of assert { stuff } with assert_block { stuff }?”

Sounds good in theory. But when one of your assert_block tests fails, you get error messages like this:

1
2
3
  1) Failure:
test_validations(FooTest) [test/unit/foo_test.rb:22]:
Expected block to return true value.

And now you’re off checking what’s in line 22 of foo_test.rb before you can fix your code. Not fun.

So moving away from assert{ 2.0 } really means having to rewrite all your test code to use assert_equal and friends.

3. Conclusion

I like to give my test code the same love that I give my production code. Relying for my testing on a library that I don’t trust just doesn’t work for me. So that’s why I eventually stopped using assert{ 2.0 }.

4. Postscripts

  • This is not to slam Phlip for publishing it. If we only ever published code once it’s in perfect shape, there wouldn’t be a lot of projects on github.

  • Alex Chaffee created a fork; however the README completely focuses on HTML testing and doesn’t even mention the assert { test code } syntax. Hence I’m not convinced that his version of assert{ 2.0 } would alleviate my concerns much.

  • On another note, if you want friendlier syntax than assert_equal and friends, but without using assert{ 2.0 }, try RSpec. It lets you write

    1
    2
    
    x.should == 42
    page.should have_content('foo')
    

    This is magically (though documentedly) equivalent to

    1
    2
    
    assert_equal 42, x
    assert page.has_content?('foo')
    

    but it generates beautiful descriptive error messages when it fails.

    RSpec code happens to read quite nicely in English, but even if you think of your tests as pure Ruby code, it’s very easy and transparent to use. You can mix RSpec-expectations with Test::Unit if you want to use just the should matchers and not the whole testing framework.