In praise of Shoulda macros
Shoulda contexts let you to share setup code between different tests. This is for me one of Shoulda’s most attractive features.
When you combine this with the technique of defining your own macros to encapsulate assertions or setups that come up often, you end up with seriously DRY and readable tests.
I see a few different kinds of Shoulda macros:
Assertion macros
Assertions macros often begin with should_. They encapsulate one or a few assertions.
For ActiveRecord models:
should_require_attributes :name, :phone_number
They may even accept a block and do assertions on its execution, like should_raise:
should_raise(LoadError, :message => /vespene/) do
require "more vespene gas"
end
To learn more about assertion macros, you can take a look at Shoulda macros allows you to embrace your inner slacker by Josh Nichols. Inner slacker? I’m right there!
Setup macros
This kind of macro encapsulates a setup that comes up often in your test suite. One inspired by Restful Authentication’s login_as helper method could be used like this:
logged_in_as :mat do
# Shoulda tests
end
These kinds of macros accept a block that defines more Shoulda tests, rather than a block of code testing your app per se.
Turnkey macros
Turnkey macros are beefed up assertion macros. The main difference is their extent. They contain many contexts and a lot of should blocks. They usually accept substantial options hashes or are configured with a setup block. Like should_be_restful in the following example, inspired by the Shoulda documentation:
logged_in_as :stranger do
should_be_restful do |resource|
resource.create.params = { :subject => "test", :body => "message" }
resource.denied.actions = [:edit, :update, :destroy]
resource.denied.redirect = "login_url"
resource.denied.flash = /only the owner can/i
end
end
Two Shoulda best practices around setup macros
This article is specifically about setup macros.
Here’s the implementation of a pretty generic Shoulda macro I could define in my test_helper*. This is an implementation of the macro I mentioned at the beginning:
# Sets the current person in the session from the person fixtures.
def self.logged_in_as(person, &block)
context "logged in as #{person}" do
setup do
@request.session[:person] = people(person).id
end
yield
end
end
Which can then be used like this in any controller test:
logged_in_as :mat do
# tests for users
end
logged_in_as :admin do
# tests for admin
end
Setup macros have a very subtle catch, however. Here’s a modified version of the first example above:
logged_in_as :mat do
setup do
@request.session[:last_login] = Time.now
end
# Some tests
end
The setup block you see here is never going to be executed. Why?
If we were to replace the logged_in_as macro by the actual code it contains, here’s what it would look like:
context "logged in as #{person}" do
setup do
@request.session[:person] = people(person).id
end
setup do
@request.session[:last_login] = Time.now
end
# Some tests
end
Does that make sense? Not so sure.
Shoulda doesn’t like to have multiple setup blocks for a given context. That part does make sense.
Best practice #1: Always describe the situation with a context.
You should always describe the situation in which your test takes place (what your setup is doing) with a context.
logged_in_as :mat do
context "with last login set to now" do
setup do
@request.session[:last_login] = Time.now
end
# Some tests
end
end
Fair enough. We blame it on the user of the macro :-)
Since we’re using Ruby, most of us are probably in agreement with Matz’ “Make the programmer happy” motto.
So can we also solve the problem from the other end? Create a setup macro that supports a direct inner setup block? Of course we can, this is Ruby, not VB.
Best practice #2: Create setup macros that support a second setup block
A setup is grafted to a context that describes it. As the creator of the macro, I don’t know what crazy setup blocks programmers will put inside their macro. So I simply create a mute context:
# Sets the current person in the session from the person fixtures.
def self.logged_in_as(person, &block)
context "logged in as #{person}" do
setup do
@request.session[:person] = people(person).id
end
context '' do
yield
end
end
end
Now my macro supports the following test without a hitch:
logged_in_as :mat do
setup do
@request.session[:last_login] = Time.now
end
# Some tests
end
And of course, programmers who stick to best practice #1 can still write a cleaner test without a problem. The awesomeness of contexts lies in the fact that they can be nested:
logged_in_as :mat do
context "with last login set to now" do
setup do
@request.session[:last_login] = Time.now
end
# Some tests
end
end
Conclusion
Best practice #1 is simple. A setup block should be described by its encompassing context. It’s a question of readability. Nesting a setup block immediately inside a Shoulda macro is a dubious practice.
Best practice #2 is a more pragmatic solution to the problem. Ok, nesting a block right inside a macro isn’t always the best idea.
But when you don’t have the macro right under your nose, it may take you a while before you think about looking at said macro. I don’t know about you, but I have a tendency to have a great deal of confidence in macros that work well across my test suite.
So after you’ve spent an hour questioning Shoulda (or your sanity, or whether you should have become a gardener instead of a software developer) because your setup block isn’t executing, best practice #2 starts to make sense.
It may or may not be necessary in all your setup macros. I find it’s especially useful for macros that are generic enough to be used across your test suite. Or most of all, in setup macros you will share with the world.
Best practice #2 makes setup macros bulletproof to the problem of multiple setups.
Now go refactor your setup macros!
To learn more about Shoulda, check out Thoughtbot’s comprehensive documentation.
* A note on where to define Shoulda macros. Shoulda 2 can now auto load macros that are in the right location. This will help you keep your test_helper cleaner. Read more about it succinctly in Shoulda can automatically load custom macros by Josh Nichols or in the Shoulda 2.0 release post.
If you use Shoulda and, like me, you hate Test::Unit’s assert_raise(), I may have something of interest for you.
Why the hate?
Well, assert_raise accepts an *args list of exception types.
If you don’t pass any, you get some nonsense because an empty array doesn’t jive with the exception raised by your block. Useful. So if you don’t care what exception is raised, assert_raise isn’t gonna help you.
Also, assert_raise doesn’t let you specify what kind of exception message you’re expecting. I actually don’t mind that a given assertion should verify exactly one thing. On the other hand I have to jump through hoops to capture the exception if I want to assert on the error message.
A shoulda macro to the rescue
As usual, Shoulda is there to help us keep our test code DRY and intuitive. I’ve concocted a useful macro called should_raise. Here’s the gist:
It must be called with the block you expect to raise an exception, of course. You can also specify two optional arguments, the exception type and the message.
:kind_of or :instance_of
If not specified, an assertion is made that an exception was raised, but with no restriction on the type of the exception.
If you specify :kind_of, the assertion will be that much more precise. It will check that the exception raised was of the type specified, or a descendant.
If you specify :instance_of, the assertion is now that the exception raised was exactly of the type specified.
A shorthand is also available, where the type of the exception is supplied directly, like should_raise(LoadError), in which case the assertion is the same as with :instance_of.
In all of these cases, exactly one assertion is generated, whether or not :type is specified.
:message
If :message is specified, a second assertion will be added in order to make sure the error message matches the parameter. This can either be a string or a regex. The assertion is simply an assert_match.
If not specified, no assertion is generated for the message.
Examples
should_raise do
require "more vespene gas"
end
# 1 assertion
should_raise(LoadError) do
require "more vespene gas"
end
# 1 more restrictive assertion
should_raise(:instance_of => LoadError) do
require "more vespene gas"
end
# 1 assertion, the same as should_raise(LoadError)
should_raise(:kind_of => ScriptError) do
require "more vespene gas"
end
# 1 assertion, slightly less strict than with :instance_of (note: LoadError < ScriptError)
should_raise(:message => "no such file to load") do
require "more vespene gas"
end
# 2 assertions
should_raise(:message => /vespene/) do
require "more vespene gas"
end
# 2 assertions
should_raise(LoadError, :message => "such file to load") do
require "more vespene gas"
end
# 2 assertions
should_raise(:kind_of => LoadError, :message => "file to load") do
require "more vespene gas"
end
# 2 assertions
should_raise(:instance_of => LoadError, :message => "to load") do
require "more vespene gas"
end
# 2 assertions
Conclusion
As you can tell, I’m eagerly awaiting Starcraft II.
No, I meant: check out the code on gist 20019. I’ve included a reasonable suite of unit tests in a comment at the end.
Feel free to use it in any way you like. Just make sure you don’t sue me.
git_remote_branch 0.2.6 is out!
I’ve just released a new and improved version of git_remote_branch. Code named 0.2.6!
Ok, I admit. I haven’t really begun using code names.
I’m promoting the project from a pre-alpha to an alpha release. There’s still a lot to do, but the stability and “testedness” have improved greatly. Following are both sides of the maturity story.
The project is maturing
- grb got its first contributor, Caio Chassot
- I added a lot of tests, both unit and functional
- there might even be interesting stuff to see in there for those who need to test command-line tools
- the gem can now be installed directly from RubyForge
- git_remote_branch now has a Google Group
The project is still immature
- it swears a lot
- no rubyforge page, despite the project being on rubyforge (at rubyforge.org/projects/grb)
- no real documentation other than running grb help
- very little in code documentation. On the other hand the code is spectacularly clean and readable, so that’s completely unnecessary. Just kidding.
What’s new in 0.2.6?
Three new actual features
- the ‘rename’ command, contributed by Caio Chassot
- the ‘publish’ command
- the −−silent option to completely mute grb output as well as every git command run by grb on your behalf
And other stuff
- the grb bin file now works when symlinked (also thanks to Caio Chassot)
- lots of unit and functional tests
- bug fixes
- more flexibility for running grb outside of a git repository (e.g. for ‘explain’ or ‘help’)
- now officially under the MIT license
- refactored a bunch of rake tasks
Git the new version
To install the newest version of the gem, simply run
sudo gem install git_remote_branch
If you really want to be on the bleeding edge you can also get it on GitHub. Note however that in ‘bleeding edge’ the word ‘bleeding’ is still the most important one at that point.
git clone git://github.com/webmat/git_remote_branch.git cd git_remote_branch rake install
The ‘install’ task will run the tests before installing so you’ll need Shoulda, mocha, redgreen and ruby-debug for that approach.
Not familiar with git_remote_branch?
What it is
The basic idea for git_remote_branch is to trivialize the interaction with remote branches. The first goal is to make the commands for the simple situations easy.
The secondary goal, is to help you learn the commands by seeing them displayed in a beautiful shade of red each time you use grb, along with git’s output.
git_remote_branch lets you
- create local-remote branche pairs, and tracks the remote branch automatically (for automatic merges when you git pull)
- publish a local branch as a remote branch, very similar to create
- delete local-remote branch pairs
- track a remote-only branch
- rename a local-remote branch pair
- explain by simply spitting out the necessary commands to do any of the above
How to use it
explain
If you simply want to use grb as a cheatsheet (and run nothing on your behalf), you can use the explain command:
$ grb explain create
git_remote_branch version 0.2.6
List of operations to do to create a new remote branch and track it locally:
git push origin current_branch:refs/heads/branch_to_create
git fetch origin
git branch −−track branch_to_create origin/branch_to_create
git checkout branch_to_create
or
$ grb explain create my_branch my_origin
git_remote_branch version 0.2.6
List of operations to do to create a new remote branch and track it locally:
git push my_origin current_branch:refs/heads/my_branch
git fetch my_origin
git branch −−track my_branch my_origin/my_branch
git checkout my_branch
Notice that you can specify any normally expected parameter you’d normally include and ‘explain’ will use them in the list of commands it suggests.
Even better, if you’re in your repository, the current branch is going to be taken into account:
(master) $ grb explain create my_branch my_origin
git_remote_branch version 0.2.6
List of operations to do to create a new remote branch and track it locally:
git push my_origin master:refs/heads/my_branch
git fetch my_origin
git branch −−track my_branch my_origin/my_branch
git checkout my_branch
Of course, ‘explain’ works for all commands: create, publish, delete, track and rename.
The main commands
I’m not going to painstakingly give an example for each command. I’ll only give two, to show how git’s responses are displayed when running grb without ‘explain’:
(master)$ grb create test_branch git_remote_branch version 0.2.6 git push origin master:refs/heads/test_branch Total 0 (delta 0), reused 0 (delta 0) To git@github.com:webmat/git_remote_branch.git * [new branch] master -> test_branch git fetch origin git branch −−track test_branch origin/test_branch git checkout test_branch Switched to branch "test_branch" (test_branch)$ grb delete test_branch git_remote_branch version 0.2.6 git push origin :refs/heads/test_branch To git@github.com:webmat/git_remote_branch.git - [deleted] test_branch git checkout master Switched to branch "master" git branch -d test_branch (master) $
Yes my friends, I have just boldly used grb on my real repository for your viewing pleasure.
But worry not, no repository was hurt during the writing of this article.
Feedback
For any feedback you’re of course welcome to
- comment on this article
- post in the google group
Thanks
- To Caio Chassot for the code contribution
- To the Thin team for a good inspiration on how to help manage gem creation and deployment with rake;
- Feedback from James Golick in day to day collaboration as well as all the people that commented on the initial announcement of git_remote_branch;
- To Chris Wanstrath (defunkt) from the GitHub team for pimping of the initial announcement of grb on the GitHub blog.
The context
Recently at GiraffeSoft we started a new project, based on another existing project we already had going. We could call this a long term fork. Let me give you a little bit more context on the situation.
- These projects will both keep being actively developed in the future;
- They will have some fundamental differences that will not be reconciled;
- They will however keep many similarities and we expect that they will benefit from the exchange of some specific patches, as development on both moves forward.
In days past, this problem could have been solved reasonably well by cloning the central repository and then exchanging patches and applying them manually.
As you’ve guessed already, we’ve decided to try using Git to help manage this long term relationship.
I must warn you however. It’s the first time we attempt keeping a long term fork like that. We’re not sure whether it’s going to be worth the trouble and whether the diverging of the two projects will eventually prevent us from efficiently benefiting from using Git to manage the exchange of patches. We’re not sure whether this is the best way to accomplish this either.
On one hand, all this trouble may or may not be worth the effort. On the other hand, all of this sounds like ultra elite Git-fu. Hence our decision to explore this approach.
Another less technical reasons was also at play in our decision. We wanted to have one wiki per project on GitHub :-)
What we’re going to do
So at first, we have a remote repository for the initial project. There are of course an arbitrary number of client side clones of the repository (what Subversion calls working directories). None of them will be affected in any way.
The first thing I will do is clone the initial project to a new client side repository. I’ll set this up in such a way that it won’t use the initial project as its default origin. On the other hand I’ll make sure it has one branch that interacts with the initial project for future patch exchanges.
Then from that client-side repository, I’ll initialize a new remote repository, which will serve as the default remote repository for the new project.
What we’ll end up with is 2 remote repositories which will have a lot in common but won’t be linked to one another in any way. It won’t be a GitHub fork, for instance.
There will be exactly one client side repository that knows about the two central repos and that can exchange commits between them. All other new client-side clones of the new project will be plain old regular Git clones.
It would be easy to set up more client side repositories to be aware of both repositories, but to me it doesn’t really seem necessary.
Article too long?
As with my previous articles about Git, I’ll provide detailed instructions for you to follow along on dummy repositories (if you’re interested). When you return to this article to attempt something similar, you may want to skip to the executive summary section, where I list strictly the important operations without all the rambling.
The instructions with the rambling
Setting up a dummy initial project
Find yourself a comfortable directory and run these few commands to initiate a dummy client-side repository and its corresponding dummy remote:
mkdir test; cd $_ mkdir initial_project initial_wd GIT_DIR=initial_project/ git init # Creating a dummy repo cd initial_wd git init echo foo > file.txt git add . git commit -a -m "initial commit" echo bar >> file.txt git commit -a -m "modification" # Setting up what's gonna be the central repo for our initial app git remote add origin ../initial_project/ git push origin master git config branch.master.remote origin git config branch.master.merge refs/heads/master
The last 2 config commands configure your master branch to automatically track remote master when issuing the command ‘git pull’. In other words, it’s equivalent to running the command ‘git branch −−track master origin/master’, with the only difference being that ‘branch −−track’ is primarily intended to create a new local branch, whereas here we already have our local branch.
We can now check for the expected behavior:
# mat@mm initial_wd (master)$ git pull # Already up-to-date.
Note also that here I simplify the instructions for following along by creating a dummy remote repository that’s in fact only in another local directory. If for example you wanted to use GitHub, your Git remote command would simply look like:
git remote add origin git@github.com:username/initial_project.git
The actual fork
# Create a directory to host the new remote repository cd .. mkdir project2 GIT_DIR=project2/ git init
Now we will use Git clone with the -o option. This lets us give the initial project another name than the default ‘origin’. We’ll want to use the name ‘origin’ for the new repository we’ll create to actually track the new project. I decided to name it the initial project’s origin ‘ip_origin’.
# Specify origin name, then the path to the shared repo # and finally a directory name for the local working directory. git clone -o ip_origin initial_project wd
Setting up the relationship with the initial repository
We first create a branch specifically to track the initial project’s master branch. Then we set it up to track the initial project.
cd wd git branch ip_master git config branch.ip_master.remote ip_origin git config branch.ip_master.merge refs/heads/master
Setting up the relationship with the new project’s repository
git remote add origin ../project2 git push origin master git config branch.master.remote origin git config branch.master.merge refs/heads/master
Let’s pretend some new development happened
echo 'shareable modification' > shareable_file.txt git add shareable_file.txt git commit -m "shareable modification" echo "specific to new project" > not_shareable.txt git add not_shareable.txt git commit -m "specific to new project"
So I have now begun working on the new project and I already have one commit that could benefit the initial project as well. The progress looks a little like this:
So I switch to the branch that manages the relationship with the initial project and I pick the commit before the last one in master.
git checkout ip_master git cherry-pick master^
Everything’s dandy so far, except for the subtle fact that I have brought us all to the edge of a cliff.
If I tried to push to the initial repository right now, I’d be in for a nasty surprise:
# mat@mm wd (ip_master)$ git push # Counting objects: 7, done. # Compressing objects: 100% (4/4), done. # Writing objects: 100% (6/6), 564 bytes, done. # Total 6 (delta 1), reused 0 (delta 0) # Unpacking objects: 100% (6/6), done. # To /Users/mat/blog/long-term-fork/test/initial_project # 7178a89..3ca0240 master -> master
Oops! By default, ‘git push’ syncs up all branches of the same name with the current branch’s origin. So that would push the new project’s master branch on our initial project’s shared master.
This is obviously not what we want. We only want ip_master to be pushed to the initial project’s master.
Trying to remember to always explicitly run ‘git push ip_origin ip_master:master’ wouldn’t do it for me. To keep the analogy, that would be akin to doing a cartwheel on the edge of said cliff: a lot of fun until you make a mistake. So obviously we’d like a simple ‘git push’ to do the right thing.
So here’s how we configure it:
git config remote.ip_origin.push refs/heads/ip_master:master
Now we can safely issue the ‘git push’ command from ip_master and have Git push ip_master to ip_origin/master.
git push # Counting objects: 4, done. # Compressing objects: 100% (2/2), done. # Writing objects: 100% (3/3), 310 bytes, done. # Total 3 (delta 0), reused 0 (delta 0) # Unpacking objects: 100% (3/3), done. # To /Users/mat/blog/long-term-fork/test/initial_project # 027a48f..9db6c3f ip_master -> master
There’s now only one remaining annoyance we’re not yet protected against. If new branches are created in the initial project’s central repository, a ‘git pull’ when standing in the ip_master branch would pull them all in our new project’s working directory. I consider this one only an annoyance, since I could just decide not to pay attention to them. On the other hand they will be distracting and may also clash with the branches I create for the development on my new project. So we want to avoid that behavior as well.
To better understand the current Git configuration, let’s have a look at .git/config:
...
[remote "ip_origin"]
url = /Users/mat/blog/long-term-fork/test/initial_project
fetch = +refs/heads/*:refs/remotes/ip_origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
[branch "ip_master"]
remote = ip_origin
merge = refs/heads/master
[remote "origin"]
url = ../project2
fetch = +refs/heads/*:refs/remotes/origin/*
So in both ‘remote’ sections I can see that fetch is configured to bring everything locally (the * wildcards).
So here’s how I limit what gets pulled when I pull from the initial project’s repository.
git config remote.ip_origin.fetch +refs/heads/master:refs/remotes/ip_origin/master
The part before the colon is the name of the interesting branch on the remote server. The part after the colon is your local Git repo’s internal branch, used to track the remote branch (not to be confused with our user branch ip_master).
Setting up remote branches on both projects, pulling, pushing and exchanging more commits is left as an exercise to the reader.
Conclusion
So that’s the gist of it, my friends. I am basically set up to work on my new project like I would in a more typical situation. I also have a special branch set up to interact with the initial project. With this branch I’ll be able to do two things:
- Pull new developments from the initial project and then cherry-pick only the shareable commits into the new project’s other branches.
- Cherry-pick in the other direction to bring certain commits from the new project into this branch and then push them up to the initial project.
For this approach to be useful however, we’ll have to make sure we create as concise commits as possible. Gone are the days of committing a whole afternoon in one meaningless commit containing 12 different modifications.
Of course coding sprees of a couple hours are not out of the question. Features like ‘git add −−patch’ are a great help when comes time to extract meaningful commits out of the result of a few hours of intense coding. For a good introduction to −−patch (and a few other powerful features), be sure to read Ryan Tomayko’s The Thing about Git.
The executive summary
So let’s reiterate strictly the interesting bits necessary to set up a long term fork when starting a new project from an existing one.
We already have:
- the initial project’s repository at url git@github.com:username/initial_project.git
- the new project’s empty repository, also created at git@github.com:username/new_project.git
# Create new local clone for the new project git clone -o ip_origin git@github.com:username/initial_project.git new_project cd new_project # Set up a standard track between initial master and local ip_master git config branch.ip_master.remote ip_origin git config branch.ip_master.merge refs/heads/master # Automatically push the right branch git config remote.ip_origin.push refs/heads/ip_master:master # Don't bring in the other shared branches from initial project git config remote.ip_origin.fetch +refs/heads/master:refs/remotes/ip_origin/master # Push new local repo it to new shared repo git remote add origin git@github.com:username/new_project.git git push origin master # Configure standard track of master with new local repo git config branch.master.remote origin git config branch.master.merge refs/heads/master
That’s it! We now only have to cherry-pick like there’s no tomorrow.
How to load gems only when your tests are not run from TextMate
Working with new people often influences the way you work. The influences can range from picking up simple tricks to seeing fundamental facts about your craft in a new light.This week when I began working with James I saw him run his tests directly from TextMate. Of course I knew it was possible to run Ruby from TM, including tests.For some inexplicable reason however I had never bothered to try it. This trick is very convenient for two reasons: TextMate cleans up the backtraces and resolves each level of the trace to a clickable link to your code file. So I decided to include this trick in my workflow.I encountered two problems with this however. Two gems I usually use in my tests don’t play well with running tests from TextMate.
redgreen
The redgreen gem highlights the . F and E (among other bits) in your test runs with green, red and yellow. When running my whole test suite it’s something I want to have. It’s not only visually pleasing, but it also lets me see at a glance whether any problems were encountered.When run from TextMate, tests using redgreen are displayed without having the console coloring stripped out, which gives something like this:
Riiiight.For the record, here’s the nice result when run at the console:
quietbacktrace
The other gem I can’t do without is James Golick and Dan Croak’s quietbacktrace. This gem lets you specify filters and silencers to clean up those huuuuuge Rails or Merb backtraces.Filters let you remove useless parts of a given line in the backtrace, such as the path leading to your gems. Silencers let you completely remove some lines from the backtrace, such as all the lines referring to what happened inside Rails, leading to your error. The most popular filters and silencers are already provided with quietbacktrace, as you’ll see below.So the result, of course is that messing with backtraces breaks TextMate’s ability to link a backtrace line with the corresponding file.
A simple snippet to fix this
Since I didn’t really want to do away with these tools, I included a bit of a hacky snippet in my test_helper.rb file to detect whether a test run was happening in TextMate or at the console.If you find yourself in the same fix as me, feel free to use the following.Among your requires:
IN_TM = !ENV['TM_DIRECTORY'].nil? unless IN_TM require 'redgreen' require 'quietbacktrace'end
And then:
class Test::Unit::TestCase
unless IN_TM
self.backtrace_silencers << :rails_vendor
self.backtrace_filters << :rails_root
end
#...
end
Now I can have my testing niceties when running my tests from the console and they don’t break the TextMate integration :-)
Trying this out for the first time?
If like me you just decided to try this out for the first time, the shortcuts are easy to remember. Command-R runs any Ruby file (in this case, the whole test file) and Command-Shift-R runs the single test the cursor is in.There’s a little hiccup in sight, however. If you’re using Rails 2+, you have to apply a very small fix to TextMate before you’ll be able to run your tests. Read more about the fix on Marc-André Cournoyer’s blog.