This is going to change, but it’s not all going to happen on Programblings.

It’s been a long time coming, but giraffesoft finally has a blog. We’re going to kickstart our blog with a week of open source releases.

At giraffesoft we like DRY code. We all know that creating Rails plugins is barely more work than actually implementing the functionality inside of a specific application. For that reason, we create plugins all the time when working on projects.

So this week, we’re going to polish up a few of them – big and small – and officially introduce them to the world.

For now, please let me direct you to the brand-spanking new giraffesoft blog.

If you’re too lazy to read the introductory post, here’s the punchlines:

• follow our RSS feed or
• follow us on Twitter.

As Brian Ford pointed out, Rubinius is a community project. And Rubinius is not going away. A lot of people can’t wait to have a Ruby written more in Ruby than in C or C++. Koichi Sasada (lead developer on Ruby 1.9) even recently projected that Rubinius would eventually be the Ruby implementation of choice.

Rubinius’ future is promising. I’d like to go from that positive note, and have YOU try Rubinius now. I promise that in 20 minutes, you’ll be running Rubinius and enjoying it. The longest parts of the process will be waiting (cloning, compiling), so it won’t even be difficult. Note: the article may look long, but it’s not. Half of it is code samples and the results returned.

So it’s been aeons since the last article of the RFTL series. But yes, this article is part of a series. You can look at the first 2 parts if you want. Some details¹ won’t be up to date anymore, but it’ll help you get the gist of Rubinius if you’re not familiar with the project yet.

• Rubinius for the Layman, Part 1: Rubies All the Way Down
• Rubinius for the Layman, Part 2: How Rubinius is Friendly
• Rubinius for the Layman, Part 3 – Try Rubinius in 20 minutes

The 20 minutes timer starts now, for those who went ahead and read the past articles.

Install Rubinius

Prerequisites

Prerequisites are probably already taken care of on most Ruby developers’ machines:

• ruby 1.8, rubygems, rake and ParseTree (sudo gem install rake ParseTree)
• git (no need to understand it, really, just having it installed)
• general C++ building tools

Dependencies for Mac users

Leopard users should have all they need when Xcode is installed. Insert your Leopard upgrade DVD and from your terminal:

open /Volumes/Mac\ OS\ X\ Upgrade\ DVD/Optional\ Installs/Xcode\ Tools/XcodeTools.mpkg 

Dependencies for Linux users

Linux users have to make sure the following packages are installed. Use your the equivalent command for your distro:

sudo apt-get install gcc bison make pkg-config libtool git

(where make == GNU make)

Get the Rubinius code

Right now the GitHub feature to download a tarball is disabled for big projects like Rubinius. They say the feature on the radar as one of the things to fix in the next few weeks. Which will be awesome.

For now we have have to clone the repo, which in this case takes a few minutes. Find yourself a comfortable directory and:

git clone git://github.com/evanphx/rubinius.git
# go make coffee
cd rubinius

Build Rubinius

Assuming you’ve installed all the dependencies, the following should work right off the bat. If it’s not the case, check out the notes on the subject.

rake build
# go get a http://www.brawndo.com/

Later, when you want to recompile from a clean slate, just run rake distclean.

If you have not gotten a BRAWNDO, please skip over the next paragraph.

RUBINIUS, like BRAWNDO, is one of the CRAZIEST ideas of the LAST DECADE! Can you realize you’re ABOUT to try the Ruby IMPLEMENTATION that’s got the BEST Ruby code ratio AMONG THEM ALL! Isn’t that FREAKING AWESOME? You can also do some of the CRAZIEST INTROSPECTION with MethodContext, StaticScope and other classes like THAT!

Run Rubinius

The Rubinius executable is rbx in the ‘bin’ subdirectory. It’s a bit different from MRI in that rbx starts an irb session if you don’t specify a file to run. So let’s do that:

bin/rbx
# in irb
puts "Don't you spring a hello world on me"
#=> Don't you spring a hello world on me

Ok then. Well, technically you’ve now tried Rubinius in less than 20 minutes. Now if you keep reading, you’ll really taste some true Rubinius awesomeness.

Kick-ass introspection

Let’s start slow by patching Object to help us quickly grok the new kinds of objects we may encounter:

class Object
  # Return only the methods not present on basic objects
  def interesting_methods
    (self.methods - Object.new.methods).sort
  end
end
#=> #<CompiledMethod interesting_methods file=(irb)>

And now let’s create a basic little class that will help us start our exploration of a MethodContext instance.

class C
  def initialize
    @inst = 42
  end
  def get_mc
    local_var = 'value'
    MethodContext.current
  end
end
#=> #<CompiledMethod get_mc file=(irb)>

So with the help of an instance of the class C, let’s start poking gently at a MethodContext.

c   = C.new
ctx = c.get_mc
#=> #<MethodContext:0xcc #<C:0xca>#get_mc (irb):7>

ctx.interesting_methods
#=> ["__add_method__", "_get_field", "_set_field", "activate",
 "active_path", "alias_method", "back_ref", "block", "class_variable_defined?",
 "const_defined?", "const_path_defined?", "context_from_proc", "context_stack",
 "copy", "current_scope", "describe", "disable_long_return!", "dynamic_locals",
 "file", "fp", "from_eval?", "get_eval_local", "ip", "ip=", "last_match",
 "last_match=", "line", "lines", "locals", "locals=", "location",
 "make_independent", "method=", "method_module", "method_scope",
 "method_scope=", "name", "normalized_name", "nth_ref", "position_info",
 "receiver", "receiver=", "reload_method", "script_object", "send_private?",
 "sender", "set_eval_local", "set_iseq", "sp", "stack_trace_starting_at"]

ctx.name
#=> :get_mc

ctx.describe
#=> "C\#get_mc"

ctx.method_module
#=> C

Interesting, that reminds me of the monkey-patching discussion that often comes up in the Ruby community. Let’s try something else:

module ModuleMC
  def module_mc
    MethodContext.current
  end
end
#=> #<CompiledMethod module_mc file=(irb)>

C.include ModuleMC
#=> [ModuleMC]

c.module_mc.method_module
#=> #<IncludedModule:0xd6>

c.module_mc.method_module.name
#=> "ModuleMC"

Wouldn’t it be nice if we had that kind of introspection, when comes time to debug some mixin magic?

Now let’s go back our MethodContext object. Or rather, its method accessor, which gives us a CompiledMethod instance:

m = ctx.method
#=> #<CompiledMethod get_mc file=(irb)>

m.interesting_methods
#=> ["__ivars__", "__ivars__=", "activate", "activate_as_script",
"as_script", "child_methods", "compile", "decode", "describe",
"exceptions", "exceptions=", "file", "file=", "first_ip_on_line",
"first_line", "from_string", "hints", "hints=", "inherit_scope",
"is_block?", "iseq", "iseq=", "line_from_ip", "lines", "lines=",
"literals", "literals=", "local_count", "local_count=", "local_names",
"local_names=", "locate_line", "min_stack_size", "name", "name=",
"primitive", "primitive=", "private?", "protected?", "public?",
"required_args", "required_args=", "scope", "scope=", "send_sites",
"serial", "serial=", "splat", "splat=", "stack_size", "stack_size=",
"total_args", "total_args="]

m.describe
#=> "method get_mc: 0 arg(s), 0 required"

m.local_names
#=> #<Tuple: :local_var>

m.literals
#=> #<Tuple: "value", :MethodContext, #<SendSite:0xda
#     name=current hits=0 misses=0>>

m.file
#=> :"(irb)"

The local_names method sounds extremely promising, but unfortunately for now, there’s no primitive for actually getting the local variable’s value, but it’s perfectly possible³. It’s just not been done yet.

By the way, what is that? #describe summarizes the arguments? Let’s try something more interesting with it:

def method_with_args(arg1, arg2='default', *args)
  MethodContext.current
end
#=> #<CompiledMethod method_with_args file=(irb)>

ctx2 = method_with_args(42, 'towel', "don't panic")
#=> #<MethodContext:0x16e main#method_with_args (irb):4>

m2 = ctx2.method
#=> #<CompiledMethod method_with_args file=(irb)>

m2.describe
#=> "method method_with_args: 2 arg(s), 1 required, splatted."

m2.local_names
#=> #<Tuple: :arg1, :arg2, :args>

m2.literals
#=> #<Tuple: "default", :MethodContext, #<SendSite:0x16c
#     name=current hits=1 misses=0>>

Nice! Ok, now let’s come back to our CompiledMethod instance and check out it’s scope accessor.

ss = m.scope
#=> #<StaticScope:0xea parent=#<StaticScope:0xe8 parent=nil
#     module=Object> module=C>

ss.interesting_methods
#=> ["initialize", "module", "parent", "script", "script="]

ss.parent
#=> #<StaticScope:0xe8 parent=nil module=Object>

So now we’ve essentially poked 2 levels deep: ctx.method.scope. Let’s rewind again and look at a our context’s receiver and sender accessors. To better understand both, let’s come back to our OO roots of 30 years back and start calling ‘method calls’ ‘messages’ instead.

sender (sends message ‘get_mc’) => receiver

sender is then the caller of the method, and receiver is, the object receiving the message. Also known as self, during the execution of the method.

Let’s see that in action:

r  = ctx.receiver
#=> #<C:0xca @inst=42>

r == c
#=> true

So ctx.receiver is a reference to the instance we’d put in the variable c. From there of course we can do Ruby’s regular meta-poking around:

r.instance_variable_get '@inst'
#=> 42

Now let’s look at the sender:

s = ctx.sender
#=> #<BlockContext:0xf0 main#irb_binding (irb):1>

s.class.ancestors
#=> [BlockContext, MethodContext, Object, PP::ObjectMixin, Kernel]

s.interesting_methods - ctx.interesting_methods
#=> ["env", "home"]

s.home
#=> #<MethodContext:0xf6 main#irb_binding
#     /Users/mat/dev/_rubies/rubinius/rubinius/lib/irb/workspace.rb:1>

s.env
#=> #<BlockEnvironment:0xf8 @initial_ip=0 @last_ip=268435456
#     @post_send=0 @bonus=#<Tuple: true>>

When calling a method from IRB, we’re in a BlockContext instead of a MethodContext, but it’s still in the family.

s = ctx.sender
#=> #<BlockContext:0xfe main#irb_binding (irb):1>

s.class.ancestors
#=> [BlockContext, MethodContext, Object, Kernel]

Anything new we need to know about?

s.interesting_methods - ctx.interesting_methods
#=> ["env", "home"]

s.home
#=> #<MethodContext:0x102 main#irb_binding
#     /Users/mat/dev/_rubies/rubinius/rubinius/lib/irb/workspace.rb:1>

s.env
#=> #<BlockEnvironment:0xf8 @initial_ip=0 @last_ip=268435456
#     @post_send=0 @bonus=#<Tuple: true>>

All of this is strangely reminiscent of a stack trace. Before you go collecting all senders to explore the execution stack, let me point you to the convenient context_stack:

ctx.context_stack.length
#=> 27

puts *ctx.context_stack
# Too noisy to output here

puts *ctx.context_stack.map{ |s| s.describe }

#=>
# C#get_mc
# Object#irb_binding {}
# Kernel(IRB::WorkSpace)#eval
# IRB::WorkSpace#evaluate
# IRB::Context#evaluate
# IRB::IrbRubinius#process_statements {}
# IRB::Irb(IRB::IrbRubinius)#signal_status
# IRB::IrbRubinius#process_statements {}
# RubyLex#each_top_level_statement {}
# Kernel(RubyLex)#catch {}
# ThrownValue.register
# Kernel(RubyLex)#catch
# RubyLex#each_top_level_statement
# IRB::IrbRubinius#process_statements
# IRB::Irb(IRB::IrbRubinius)#eval_input
# IRB.start {}
# Kernel(Module)#catch {}
# ThrownValue.register
# Kernel(Module)#catch
# IRB.start
# main.__script__
# CompiledMethod#activate_as_script
# CompiledMethod#as_script
# Compile.single_load
# Compile.unified_load
# Kernel(Object)#require
# Object#__script__
# #=> nil

I’m pretty sure there’s other areas specific to Rubinius that can be explored like that. Please share any insight in the comments.

S-Expressions

Rubinius groks s-expressions out of the box (similar to standard Ruby with ParseTree or ruby_parser. An example).

require 'pp'
pp sx = "
  class C
    def meth(arg)
      arg * 2
    end
  end".to_sexp

#=>
# s(:class,
#  :C,
#  nil,
#  s(:scope,
#   s(:defn,
#    :meth,
#    s(:args, :arg),
#    s(:scope,
#     s(:block, s(:call, s(:lvar, :arg), :*, s(:arglist, s(:fixnum, 2))))))))

sx[0]
#=> :class

sx[3][1][1]
#=> :meth

With something that reminiscent to Lisp, it’s probably better to explore recursively, though.

S-expressions are used by a lot of the Ruby code inspection tools to understand your ugly Ruby code.

Gems

I won’t touch trying out gems for today. There seems to be little issues as the moment. They do install, but I’ve been having problems running them. Please leave a comment if you’ve had success with specific gems.

If you’re curious and want to try playing with gems, Rubygems is already installed.

rbx gem install rails --no-rdoc --no-ri

Pro tip: always skip the documentation when playing with gems Rubinius. The doc takes unusually long to compile.

Run the famous test suite

The spec suite that’s been keeping all Ruby implementations honest was born from the Rubinius project. It’s been split into a separate project a while ago, since it’s now such an important and central piece of the Ruby ecosystem.

Update the specs

Since they are now in a different project, we first have to get the most recent version. Easy stuff:

rake rubyspec:update

rbx in your PATH

Before you run the specs, you need to do one little thing. One of the specs expects a shell call to rbx to start Rubinius (as in, the executable must be in your path).

The simplest way to do that for now is just to temporarily add the directory to your path (right in your console, not in your .bash_profile).

pwd
#=> /path/to/project/rubinius
export PATH=$PATH:/path/to/project/rubinius/bin
rbx -v

Run the specs

rake spec
# Time for another BRAWNDO!

Or rather, time to actually look at some of the specs you’re currently running. If you look in the spec directory, you’ll see that it’s pretty extensive, to say the least. To start with something familiar, navigate to spec/ruby/1.8, in subdirectories core or library. Open up a few of the specs in there and stare at them for a few minutes. Or better, improve a few of them and try them out on MRI, JRuby and of course, Rubinius.

Conclusion

Well, now I’ve tricked you into putting the Rubinius project on your hard drive. And you’re a Ruby developer. What are you waiting for?

• Contribute
• Documentation
• Tickets overview

I think Rubinius will be an awesome runtime for our Ruby programs. It probably won’t be only Ruby in the close future, but the kernel of Ruby (base classes & stuff) is mostly implemented in Ruby, and the compiler is also implemented in Ruby. This is awesome to help understand the workings of the language and to lower the barrier to contribution. Which is already pretty low.

Rubinius is here to stay and it’s gonna keep rocking.

Footnotes

1. Examples of details not up to date in the old articles are: the LOC numbers and the base language of the VM (used to be C, now C++).
2. If the build craps out with a message you understand, cool. Try to install the dependency through apt-get or macports/fink, or search your hard drive to see if the tool’s just not in your PATH. Otherwise, here are a few related pointers.
   • Getting Started
   • Installation
   • Common Problems
   • If you can’t find an answer in the doc, don’t hesitate and ask on IRC in #rubinius (freenode.org). The crew’s very friendly.
3. For a quick discussion about getting local variable’s values out of a CompiledMethod, check the [IRC logs around 19:20][9].

$ git config #tab
apply.whitespace               core.compression        ...
branch.                        core.fileMode
clean.requireForce             core.gitProxy
color.branch                   core.ignoreStat
color.branch.current           core.logAllRefUpdates
...

With sub-options too, on words ending with a dot:

$ git config remote.origin. #tab
remote.origin.fetch               remote.origin.receivepack      ...
remote.origin.push                remote.origin.skipDefaultUpdate

And as usual, calling git-config with a setting name without specifying a new value displays the current value.

$ git config remote.origin.url
git://github.com/evanphx/rubinius.git

Git’s getting easier by the day. Awesome!

1. A recent change from a previous position!

Editor’s note: I failed.

I recently decided to test my git_remote_branch gem with Ruby 1.9, for the heck of it. Well, I was making sure it ran on a bunch of platforms: Windows, Ruby 1.8.7 and with the most recent Git version (1.6.0.2, get it). So it seemed fitting to check it out under Ruby 1.9.

On Leopard, the only missing dependency to Ruby 1.9 is readline 5.2. This article will present the installation of both. And help heat up your apartment.

Linux too

These instructions will mostly work on Linux as well (tried on Ubuntu). There will be a few minor differences though.

• Make sure you download the original readline from the gnu site and patch it yourself;
• Uninstall older versions of Ruby1.9;
• Make sure you have the basic dev tools installed, like gcc and make;
• Skip the part about installing Xcode :-)

Prerequisites

You first need to have the Apple developer tools installed on your mac. They’re available on your installation CD. Put it in, run the installer. It’s pretty straightforward.

# in your terminal
open /Volumes/Mac\ OS\ X\ Upgrade\ DVD/Optional\ Installs/Xcode\ Tools/XcodeTools.mpkg 

If you don’t compile your own stuff often, you may have to set up your PATH variable in your ~/.bash_profile.

# file  ~/.bash_profile
export PATH="/usr/local/bin:$PATH"

Now, prepare a working directory to keep the source close to the corresponding executables.

# in your terminal
sudo mkdir -p /usr/local/src
sudo chgrp admin /usr/local/src
sudo chmod -R 775 /usr/local/src
cd /usr/local/src

Once you’ve set yourself up, if you don’t care about the details, you can skip to the end for the Cliff’s notes.

Installing readline

This one’s not as straightforward as it could have been. The gzipped readline library available on the gnu site is 12 patches behind. It so happens that the 12th patch fixes a problem with compilation under OS X. So I applied all 12 to the code and repackaged it. The example uses that file, compiled by me.

You can also do do the patching by yourself if you so wish. Here’s where you can download the readline code:

• The main file

• The patches

So let’s get on with the instructions to install readline from my patched package:

# in your terminal
curl -O http://s3.amazonaws.com/webmat-public/readline-5.2-patch012.tar.gz
md5 readline-5.2-patch012.tar.gz
# should be a9f37d2a22d181f8c23c6a320907917d
tar xzf readline-5.2-patch012.tar.gz
cd readline-5.2-patch012

./configure --prefix=/usr/local
make
sudo make install
cd ..

Build Ruby 1.9

Build options

Note that here you have a few options as to how you want to distinguish your 1.9 stack from your main 1.8 one.

In the following example, I build Ruby with the ‘1.9’ suffix, which means all executables will be suffixed with 1.9: ruby1.9, gem1.9, irb1.9, rake1.9 and so on. This approach is ideal for casual use of two versions side by side. If you don’t care about the details, skip right over the next paragraph.

The industrial approach would be to put ruby in a non standard directory and only add it to your path when you want to use that version (or use the explicit path to invoke executables). To go industrial, you can simply use the --prefix=/usr/local/ruby1.9 option and then drop the --program-suffix argument when you run configure for Ruby. This setup is ideal if you really want to have a bunch of versions living side by side (e.g. all 1.9 versions as well as 1.8.7 in addition to the current 1.8.6).

Actual installation of Ruby 1.9

Pick the most recent version or Ruby 1.9 on the ftp server. At the time of writing, 1.9.1-preview1 is the most recent.

So, still from /usr/local/src:

# in your terminal
curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-preview1.tar.gz
tar xzvf ruby-1.9.1-preview1.tar.gz
cd ruby-1.9.1-preview1
./configure --prefix=/usr/local --program-suffix=1.9 --enable-pthread --with-readline-dir=/usr/local --enable-shared
make
sudo make install

Now you’re about to see some of the funniest looking progress indicators around. Ruby’s about making the programmer happy, and it delivers even in the details!

Note: the recent source packages of Ruby1.9 automatically include the documentation, as the end of the make install attests.

Try Ruby 1.9

# in your terminal
ruby1.9 --version
gem1.9 --version
irb1.9

Once inside the Ruby interactive shell,

# in irb1.9
RUBY_VERSION
#=> "1.9.1"
stabby = ->(msg='inside the stabby lambda') { puts msg }
stabby.call
# => "inside the stabby lambda"
stabby.call 'hello world'
# => "hello world"

Yep, Ruby 1.9 introduces the very cool stabby lambda syntax. Ruby 1.8’s lambdas couldn’t have optional parameters (unless you fiddled with *args). 1.9’s stabby lambdas can, with a syntax as clean as a simple method definition, as you just experimented.

Now install the gems you use everyday (or kindly make available to your peers) and help make them 1.9 compatible.

For the sake of the stabby lambda!

That’s it! (Except for those who skipped to the Cliff’s Notes).

Cliff’s Notes

# Install patched readline
cd /usr/local/src
curl -O http://s3.amazonaws.com/webmat-public/readline-5.2-patch012.tar.gz
md5 readline-5.2-patch012.tar.gz
# should be a9f37d2a22d181f8c23c6a320907917d
tar xzf readline-5.2-patch012.tar.gz
cd readline-5.2-patch012
./configure --prefix=/usr/local
make
sudo make install
cd ..

# Install Ruby1.9
curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-preview1.tar.gz
tar xzvf ruby-1.9.1-preview1.tar.gz
cd ruby-1.9.1-preview1
./configure --prefix=/usr/local --program-suffix=1.9 --enable-pthread --with-readline-dir=/usr/local --enable-shared
make
sudo make install

# Try Ruby 1.9
ruby1.9 --version
gem1.9 --version
irb1.9

# in irb1.9
RUBY_VERSION
stabby = ->(msg='inside the stabby lambda') { puts msg }
stabby.call
stabby.call 'hello world'

Both gems being command-line tools that help you use Git in a friendlier manner, the question makes a lot of sense. It makes so much sense in fact, that I decided to blog about it. A post about it will scale much better to answer other users who may potentially ask the same question.

So here’s a slightly edited excerpt from the answer I gave him. And yes, I also ramble in email.

I’ve seen what Scott added to the github gem. Pretty cool stuff indeed. I think the idea of merging with the github gem has merit. I definitely can see a future where we start having too many distinct command-line tools that help deal with Git’s sometimes obscure or numerous commands.

However I’m not sure I’d like to merge grb into the gh gem, despite the additional awareness it would get. Here’s why.

In my mind, the github gem should be mostly features about GitHub itself, like managing pull requests (the way GitHub does them). It’s starting to accumulate features that probably aren’t GitHub-specific, which I don’t mind, of course. But grb’s features really are GitHub-agnostic. I’ve started working on it before I even started using GitHub, in fact :-)

Reinforcing the previous point, I wouldn’t want someone who doesn’t use GitHub to not realize the features of grb are available to him because they’re included in a gem called ‘github’ ;-)

Also, one of the goals of git_remote_branch is to explicitly teach the underlying git commands. I do this by always spewing out the underlying commands in red, and by having the explain command. I’m pretty sure an unsuspecting user would wonder what hit him if a few of the github commands started spewing out red text in his console ;-) Having explain for a few commands (ported from grb) and not for the rest of the github gem’s commands would be weird, too.

There’s also a few other architectural decisions of grb that may not fit with github’s, like having aliases. Once again, grb being a teaching tool, I want to offer a bunch of aliases for forgetful people like me. So far I don’t see anything like aliases in github and I’m not sure how the authors of github-gem would react to a pull request polluting it with a bunch of aliases ;-)

Last but not least, the github gem already has a track command, which conflicts with grb’s. github’s is used to track a new remote repo in your local repo while grb’s is to track another branch from your current remote repo.

So git_remote_branch is GitHub-agnostic. You can use it with any Git hosting solution: GitHub, your own Gitosis / gitweb / Gitorious installation, Gitorious.org hosting, Rubyforge or any other.

Better, grb supports working with all of them at the same time. All grb commands support an optional origin argument.

Learn more about git_remote_branch

git_remote_branch 0.3 has been released!

Previous releases were pretty much only usable by rubyists on OS X.

No more. This release is mostly focused on making sure git_remote_branch works on a broader range of platforms. A few actual features squeaked in, but nothing big like introducing new commands.

If you don’t care about the details just type the following at your command-line.

sudo gem install git_remote_branch

And check the help

grb --help

If you encounter installation problems, refer to the readme.

Platforms

git_remote_branch 0.3 has been tested with the following configurations:

• OS X Leopard / Ruby 1.8.6 / Git 1.5.4.3 and 1.6.0.2
• OS X Leopard / Ruby 1.9.1 / Git 1.5.4.3 and 1.6.0.2
• Ubuntu Intrepid Ibex / Ruby 1.8.7 / Git 1.5.6.3
• Windows XP / Ruby 1.8.6 / Git 1.6.0.2 (the msys version)

Features

Better track

Track now works even if you already have a local branch of the same name. It uses git config instead of branch —track in that case. The subtlety can be observed by running (from a git repository):

grb explain track master
grb explain track non_existent_branch

Force the use of a specific git executable

Set the environment variable GRB_GIT to point to it and grb will use this one for all its operations.

Documentation

I’ve also worked quite a bit on the actual documentation. I used to be ashamed at the quality and availability of the documentation of git_remote_branch. At last I’ll be able to sleep at night :-)

git_remote_branch in a nutshell

I’ve rewritten the intro of the readme to be (hopefully) a bit clearer.

git_remote_branch is a simple command-line tool that makes it very easy to manipulate branches published in shared repositories.

It achieves this goal by sticking to a few principles:

• keep grb’s commands extremely regular (they all look alike)
• support aliases for commands
• print all commands it runs on your behalf in red, so you eventually learn them

Another nice thing about git_remote_branch is that it can simply explain a command (print out all the corresponding git commands) instead of running them on your behalf.

Note: git_remote_branch assumes that the local and
remote branches have the same name. Multiple remote
repositories (or origins) are supported.

Documentation availability

The main readme is now available on the main grb page on rubyforge.

Documentation quality

I’ve added clearer information on getting grb to run in different kinds of situation, due to helpful feedback from Axelson and Glenn Rempe.

I’ve also added some information for playing with the code for git_remote_branch (test dependencies and so on). See the end of the readme.

Finally, I’ve updated the links section quite a bit:

Dare

I dare you to find a platform on which git_remote_branch doesn’t work :-)

If you do, please send me feedback through GitHub or via email. I’m using gmail and, as usual, I go by the handle of webmat.

Last note: the git_remote_branch gem on GitHub

Excerpt from the readme:

Note that the only stable version of the gem you should trust is the one from Rubyforge. The GitHub gem is a development gem. The GitHub gem WILL be rebuilt with the same version number, and other horrible things like that. If you use the GitHub version of git_remote_branch, children will die!

You’ve been warned.

export PATH=$PATH:/var/lib/gems/1.8/bin

Also worth noting is the fact that the default ruby interpreter on 8.10 is back to the 1.8 branch: it’s 1.8.7 (1.9 was the default on 8.04 iirc). 1.9  also be installed right besides 1.8.

$ ruby --version
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
$ ruby1.9 --version
ruby 1.9.0 (2008-06-20 revision 17482) [i486-linux]

Neither comes installed by default, however. You must install them explicitly.

sudo aptitude install ruby irb rubygems

While I’m at it, why not mention that rubygems 1.2.0 is installed by default. It doesn’t want to update to 1.3.0 with the usual “gem update –system” command. Since it’s not my main machine I didn’t investigate further, but the suggestion is to use apt-get or aptitude. The repos don’t seem to be up to date with 1.3.0, but rather with a version named something like 1.3.0really1.2.0.

Shoulda contexts let you to share setup code between different tests. This is for me one of Shoulda’s most attractive features.

When you combine this with the technique of defining your own macros to encapsulate assertions or setups that come up often, you end up with seriously DRY and readable tests.

I see a few different kinds of Shoulda macros:

Assertion macros

Assertions macros often begin with should_. They encapsulate one or a few assertions.

For ActiveRecord models:

should_require_attributes :name, :phone_number

They may even accept a block and do assertions on its execution, like should_raise:

should_raise(LoadError, :message => /vespene/) do
  require "more vespene gas"
end

To learn more about assertion macros, you can take a look at Shoulda macros allows you to embrace your inner slacker by Josh Nichols. Inner slacker? I’m right there!

Setup macros

This kind of macro encapsulates a setup that comes up often in your test suite. One inspired by Restful Authentication’s login_as helper method could be used like this:

logged_in_as :mat do
  # Shoulda tests
end

These kinds of macros accept a block that defines more Shoulda tests, rather than a block of code testing your app per se.

Turnkey macros

Turnkey macros are beefed up assertion macros. The main difference is their extent. They contain many contexts and a lot of should blocks. They usually accept substantial options hashes or are configured with a setup block. Like should_be_restful in the following example, inspired by the Shoulda documentation:

logged_in_as :stranger do
  should_be_restful do |resource|
    resource.create.params   = { :subject => "test", :body => "message" }
    resource.denied.actions  = [:edit, :update, :destroy]
    resource.denied.redirect = "login_url"
    resource.denied.flash    = /only the owner can/i
  end
end

Two Shoulda best practices around setup macros

This article is specifically about setup macros.

Here’s the implementation of a pretty generic Shoulda macro I could define in my test_helper*. This is an implementation of the macro I mentioned at the beginning:

# Sets the current person in the session from the person fixtures.
def self.logged_in_as(person, &block)
  context "logged in as #{person}" do
    setup do
      @request.session[:person] = people(person).id
    end

    yield
  end
end

Which can then be used like this in any controller test:

logged_in_as :mat do
  # tests for users
end

logged_in_as :admin do
  # tests for admin
end

Setup macros have a very subtle catch, however. Here’s a modified version of the first example above:

logged_in_as :mat do
  setup do
    @request.session[:last_login] = Time.now
  end
  # Some tests
end

The setup block you see here is never going to be executed. Why?

If we were to replace the logged_in_as macro by the actual code it contains, here’s what it would look like:

context "logged in as #{person}" do
  setup do
    @request.session[:person] = people(person).id
  end
  setup do
    @request.session[:last_login] = Time.now
  end
  # Some tests
end

Does that make sense? Not so sure.

Shoulda doesn’t like to have multiple setup blocks for a given context. That part does make sense.

Best practice #1: Always describe the situation with a context.

You should always describe the situation in which your test takes place (what your setup is doing) with a context.

logged_in_as :mat do
  context "with last login set to now" do
    setup do
      @request.session[:last_login] = Time.now
    end
    # Some tests
  end
end

Fair enough. We blame it on the user of the macro :-)

Since we’re using Ruby, most of us are probably in agreement with Matz’ “Make the programmer happy” motto.

So can we also solve the problem from the other end? Create a setup macro that supports a direct inner setup block? Of course we can, this is Ruby, not VB.

Best practice #2: Create setup macros that support a second setup block

A setup is grafted to a context that describes it. As the creator of the macro, I don’t know what crazy setup blocks programmers will put inside their macro. So I simply create a mute context:

# Sets the current person in the session from the person fixtures.
def self.logged_in_as(person, &block)
  context "logged in as #{person}" do
    setup do
      @request.session[:person] = people(person).id
    end

    context '' do
      yield
    end
  end
end

Now my macro supports the following test without a hitch:

logged_in_as :mat do
  setup do
    @request.session[:last_login] = Time.now
  end
  # Some tests
end

And of course, programmers who stick to best practice #1 can still write a cleaner test without a problem. The awesomeness of contexts lies in the fact that they can be nested:

logged_in_as :mat do
  context "with last login set to now" do
    setup do
      @request.session[:last_login] = Time.now
    end
    # Some tests
  end
end

Conclusion

Best practice #1 is simple. A setup block should be described by its encompassing context. It’s a question of readability. Nesting a setup block immediately inside a Shoulda macro is a dubious practice.

Best practice #2 is a more pragmatic solution to the problem. Ok, nesting a block right inside a macro isn’t always the best idea.

But when you don’t have the macro right under your nose, it may take you a while before you think about looking at said macro. I don’t know about you, but I have a tendency to have a great deal of confidence in macros that work well across my test suite.

So after you’ve spent an hour questioning Shoulda (or your sanity, or whether you should have become a gardener instead of a software developer) because your setup block isn’t executing, best practice #2 starts to make sense.

It may or may not be necessary in all your setup macros. I find it’s especially useful for macros that are generic enough to be used across your test suite. Or most of all, in setup macros you will share with the world.

Best practice #2 makes setup macros bulletproof to the problem of multiple setups.

Now go refactor your setup macros!

To learn more about Shoulda, check out Thoughtbot’s comprehensive documentation.

* A note on where to define Shoulda macros. Shoulda 2 can now auto load macros that are in the right location. This will help you keep your test_helper cleaner. Read more about it succinctly in Shoulda can automatically load custom macros by Josh Nichols or in the Shoulda 2.0 release post.

Why the hate?

Well, assert_raise accepts an *args list of exception types.

If you don’t pass any, you get some nonsense because an empty array doesn’t jive with the exception raised by your block. Useful. So if you don’t care what exception is raised, assert_raise isn’t gonna help you.

Also, assert_raise doesn’t let you specify what kind of exception message you’re expecting. I actually don’t mind that a given assertion should verify exactly one thing. On the other hand I have to jump through hoops to capture the exception if I want to assert on the error message.

A shoulda macro to the rescue

As usual, Shoulda is there to help us keep our test code DRY and intuitive. I’ve concocted a useful macro called should_raise. Here’s the gist:

It must be called with the block you expect to raise an exception, of course. You can also specify two optional arguments, the exception type and the message.

:kind_of or :instance_of

If not specified, an assertion is made that an exception was raised, but with no restriction on the type of the exception.

If you specify :kind_of, the assertion will be that much more precise. It will check that the exception raised was of the type specified, or a descendant.

If you specify :instance_of, the assertion is now that the exception raised was exactly of the type specified.

A shorthand is also available, where the type of the exception is supplied directly, like should_raise(LoadError), in which case the assertion is the same as with :instance_of.

In all of these cases, exactly one assertion is generated, whether or not :type is specified.

:message

If :message is specified, a second assertion will be added in order to make sure the error message matches the parameter. This can either be a string or a regex. The assertion is simply an assert_match.

If not specified, no assertion is generated for the message.

Examples

should_raise do
  require "more vespene gas"
end
# 1 assertion

should_raise(LoadError) do
  require "more vespene gas"
end
# 1 more restrictive assertion

should_raise(:instance_of => LoadError) do
  require "more vespene gas"
end
# 1 assertion, the same as should_raise(LoadError)

should_raise(:kind_of => ScriptError) do
  require "more vespene gas"
end
# 1 assertion, slightly less strict than with :instance_of (note: LoadError < ScriptError)

should_raise(:message => "no such file to load") do
  require "more vespene gas"
end
# 2 assertions

should_raise(:message => /vespene/) do
  require "more vespene gas"
end
# 2 assertions

should_raise(LoadError, :message => "such file to load") do
  require "more vespene gas"
end
# 2 assertions

should_raise(:kind_of => LoadError, :message => "file to load") do
  require "more vespene gas"
end
# 2 assertions

should_raise(:instance_of => LoadError, :message => "to load") do
  require "more vespene gas"
end
# 2 assertions

Conclusion

As you can tell, I’m eagerly awaiting Starcraft II.

No, I meant: check out the code on gist 20019. I’ve included a reasonable suite of unit tests in a comment at the end.

Feel free to use it in any way you like. Just make sure you don’t sue me.

For example, I’ve frequently used it in the past to keep track of the modifications I make to an article I work on for few days. God knows I write a lot of these. There’s a reason I called this blog Programblings ;-)

To be honest, I’m using git as I write even this short article.

I also use it for trivial one evening coding projects. As soon as I spend more than an hour on code, whatever it is, I’ll usually create a local git repo for it.

One of the annoying things I realized when creating repositories more and more often, is that I always ended up ignoring the same files. Over and over again. Boring.

Fortunately for me, git can be configured to take into consideration a global ignore file. Heck, I can even create a system-wide ignore file if I want (check out git config’s doc for more info).

Configure your personal ignore file

I like to stick to conventions so I call my file .gitignore, and I put it in my home directory. But that’s up to you, really.

git config --global core.excludesfile ~/.gitignore

Note that there’s one little gotcha to be aware of. If you prefer to edit the .gitconfig file directly (or if you use a weird shell), git expects an absolute path. In the example above, bash converted the ~ shorthand to my home directory.

Now I just add ignore globs to it like any other project level (directory level, really) git ignore file.

echo .DS_Store >> ~/.gitignore

Once I’ve ignored all my favorite useless files, I can get cracking and never worry about them again.

I still have to ignore files

When I create a new repository on which people may actually contribute, I’ll still create a proper ignore file, however. Otherwise I’d convey the message that I consider contributors as slaves who only deserve the boring work of creating ignore files. Since I’m pure of heart, that’s not how I roll.

Screenshot of the main qgit screen

Install qgit from source on Leopard with these instructions.

Warning: installing the Qt 4.3 prerequisite takes an eternity or two (instructions for that are included as well).

Also note that you should make sure to use the newest versions of the downloads: the instructions point to an old 2.0rc1 release of qgit.

I’ve just released a new and improved version of git_remote_branch. Code named 0.2.6!

Ok, I admit. I haven’t really begun using code names.

I’m promoting the project from a pre-alpha to an alpha release. There’s still a lot to do, but the stability and “testedness” have improved greatly. Following are both sides of the maturity story.

The project is maturing

• grb got its first contributor, Caio Chassot
• I added a lot of tests, both unit and functional
  • there might even be interesting stuff to see in there for those who need to test command-line tools
• the gem can now be installed directly from RubyForge
• git_remote_branch now has a Google Group

The project is still immature

• it swears a lot
• no rubyforge page, despite the project being on rubyforge (at rubyforge.org/projects/grb)
• no real documentation other than running grb help
• very little in code documentation. On the other hand the code is spectacularly clean and readable, so that’s completely unnecessary. Just kidding.

What’s new in 0.2.6?

Three new actual features

• the ‘rename’ command, contributed by Caio Chassot
• the ‘publish’ command
• the −−silent option to completely mute grb output as well as every git command run by grb on your behalf

And other stuff

• the grb bin file now works when symlinked (also thanks to Caio Chassot)
• lots of unit and functional tests
• bug fixes
• more flexibility for running grb outside of a git repository (e.g. for ‘explain’ or ‘help’)
• now officially under the MIT license
• refactored a bunch of rake tasks

Git the new version

To install the newest version of the gem, simply run

sudo gem install git_remote_branch

If you really want to be on the bleeding edge you can also get it on GitHub. Note however that in ‘bleeding edge’ the word ‘bleeding’ is still the most important one at that point.

git clone git://github.com/webmat/git_remote_branch.git
cd git_remote_branch
rake install

The ‘install’ task will run the tests before installing so you’ll need Shoulda, mocha, redgreen and ruby-debug for that approach.

Not familiar with git_remote_branch?

What it is

The basic idea for git_remote_branch is to trivialize the interaction with remote branches. The first goal is to make the commands for the simple situations easy.

The secondary goal, is to help you learn the commands by seeing them displayed in a beautiful shade of red each time you use grb, along with git’s output.

git_remote_branch lets you

• create local-remote branche pairs, and tracks the remote branch automatically (for automatic merges when you git pull)
• publish a local branch as a remote branch, very similar to create
• delete local-remote branch pairs
• track a remote-only branch
• rename a local-remote branch pair
• explain by simply spitting out the necessary commands to do any of the above

How to use it

explain

If you simply want to use grb as a cheatsheet (and run nothing on your behalf), you can use the explain command:

$ grb explain create
git_remote_branch version 0.2.6

List of operations to do to create a new remote branch and track it locally:

git push origin current_branch:refs/heads/branch_to_create
git fetch origin
git branch −−track branch_to_create origin/branch_to_create
git checkout branch_to_create

or

$ grb explain create my_branch my_origin
git_remote_branch version 0.2.6

List of operations to do to create a new remote branch and track it locally:

git push my_origin current_branch:refs/heads/my_branch
git fetch my_origin
git branch −−track my_branch my_origin/my_branch
git checkout my_branch

Notice that you can specify any normally expected parameter you’d normally include and ‘explain’ will use them in the list of commands it suggests.

Even better, if you’re in your repository, the current branch is going to be taken into account:

(master) $ grb explain create my_branch my_origin
git_remote_branch version 0.2.6

List of operations to do to create a new remote branch and track it locally:

git push my_origin master:refs/heads/my_branch
git fetch my_origin
git branch −−track my_branch my_origin/my_branch
git checkout my_branch

Of course, ‘explain’ works for all commands: create, publish, delete, track and rename.

The main commands

I’m not going to painstakingly give an example for each command. I’ll only give two, to show how git’s responses are displayed when running grb without ‘explain’:

(master)$ grb create test_branch
git_remote_branch version 0.2.6

git push origin master:refs/heads/test_branch
Total 0 (delta 0), reused 0 (delta 0)
To git@github.com:webmat/git_remote_branch.git
 * [new branch]      master -> test_branch

git fetch origin

git branch −−track test_branch origin/test_branch

git checkout test_branch
Switched to branch "test_branch"

(test_branch)$ grb delete test_branch
git_remote_branch version 0.2.6

git push origin :refs/heads/test_branch
To git@github.com:webmat/git_remote_branch.git
 - [deleted]         test_branch

git checkout master
Switched to branch "master"

git branch -d test_branch

(master) $ 

Yes my friends, I have just boldly used grb on my real repository for your viewing pleasure.

But worry not, no repository was hurt during the writing of this article.

Feedback

For any feedback you’re of course welcome to

• comment on this article
• post in the google group

Thanks

• To Caio Chassot for the code contribution
• To the Thin team for a good inspiration on how to help manage gem creation and deployment with rake;
• Feedback from James Golick in day to day collaboration as well as all the people that commented on the initial announcement of git_remote_branch;
• To Chris Wanstrath (defunkt) from the GitHub team for pimping of the initial announcement of grb on the GitHub blog.

Recently at GiraffeSoft we started a new project, based on another existing project we already had going. We could call this a long term fork. Let me give you a little bit more context on the situation.

• These projects will both keep being actively developed in the future;
• They will have some fundamental differences that will not be reconciled;
• They will however keep many similarities and we expect that they will benefit from the exchange of some specific patches, as development on both moves forward.

In days past, this problem could have been solved reasonably well by cloning the central repository and then exchanging patches and applying them manually.

As you’ve guessed already, we’ve decided to try using Git to help manage this long term relationship.

I must warn you however. It’s the first time we attempt keeping a long term fork like that. We’re not sure whether it’s going to be worth the trouble and whether the diverging of the two projects will eventually prevent us from efficiently benefiting from using Git to manage the exchange of patches. We’re not sure whether this is the best way to accomplish this either.

On one hand, all this trouble may or may not be worth the effort. On the other hand, all of this sounds like ultra elite Git-fu. Hence our decision to explore this approach.

Another less technical reasons was also at play in our decision. We wanted to have one wiki per project on GitHub :-)

What we’re going to do

So at first, we have a remote repository for the initial project. There are of course an arbitrary number of client side clones of the repository (what Subversion calls working directories). None of them will be affected in any way.

The first thing I will do is clone the initial project to a new client side repository. I’ll set this up in such a way that it won’t use the initial project as its default origin. On the other hand I’ll make sure it has one branch that interacts with the initial project for future patch exchanges.

Then from that client-side repository, I’ll initialize a new remote repository, which will serve as the default remote repository for the new project.

What we’ll end up with is 2 remote repositories which will have a lot in common but won’t be linked to one another in any way. It won’t be a GitHub fork, for instance.

There will be exactly one client side repository that knows about the two central repos and that can exchange commits between them. All other new client-side clones of the new project will be plain old regular Git clones.

It would be easy to set up more client side repositories to be aware of both repositories, but to me it doesn’t really seem necessary.

Article too long?

As with my previous articles about Git, I’ll provide detailed instructions for you to follow along on dummy repositories (if you’re interested). When you return to this article to attempt something similar, you may want to skip to the executive summary section, where I list strictly the important operations without all the rambling.

The instructions with the rambling

Setting up a dummy initial project

Find yourself a comfortable directory and run these few commands to initiate a dummy client-side repository and its corresponding dummy remote:

mkdir test; cd $_
mkdir initial_project initial_wd
GIT_DIR=initial_project/ git init

# Creating a dummy repo
cd initial_wd
git init
echo foo > file.txt
git add .
git commit -a -m "initial commit"
echo bar >> file.txt
git commit -a -m "modification"

# Setting up what's gonna be the central repo for our initial app
git remote add origin ../initial_project/
git push origin master

git config branch.master.remote origin
git config branch.master.merge refs/heads/master

The last 2 config commands configure your master branch to automatically track remote master when issuing the command ‘git pull’. In other words, it’s equivalent to running the command ‘git branch −−track master origin/master’, with the only difference being that ‘branch −−track’ is primarily intended to create a new local branch, whereas here we already have our local branch.

We can now check for the expected behavior:

#   mat@mm initial_wd (master)$ git pull
#   Already up-to-date.

Note also that here I simplify the instructions for following along by creating a dummy remote repository that’s in fact only in another local directory. If for example you wanted to use GitHub, your Git remote command would simply look like:

git remote add origin git@github.com:username/initial_project.git

The actual fork

# Create a directory to host the new remote repository
cd ..
mkdir project2
GIT_DIR=project2/ git init

Now we will use Git clone with the -o option. This lets us give the initial project another name than the default ‘origin’. We’ll want to use the name ‘origin’ for the new repository we’ll create to actually track the new project. I decided to name it the initial project’s origin ‘ip_origin’.

# Specify origin name, then the path to the shared repo
# and finally a directory name for the local working directory.
git clone -o ip_origin initial_project wd

Setting up the relationship with the initial repository

We first create a branch specifically to track the initial project’s master branch. Then we set it up to track the initial project.

cd wd
git branch ip_master
git config branch.ip_master.remote ip_origin
git config branch.ip_master.merge refs/heads/master

Setting up the relationship with the new project’s repository

git remote add origin ../project2
git push origin master

git config branch.master.remote origin
git config branch.master.merge refs/heads/master

Let’s pretend some new development happened

echo 'shareable modification' > shareable_file.txt
git add shareable_file.txt
git commit -m "shareable modification"

echo "specific to new project" > not_shareable.txt
git add not_shareable.txt
git commit -m "specific to new project"

So I have now begun working on the new project and I already have one commit that could benefit the initial project as well. The progress looks a little like this:

So I switch to the branch that manages the relationship with the initial project and I pick the commit before the last one in master.

git checkout ip_master
git cherry-pick master^

Everything’s dandy so far, except for the subtle fact that I have brought us all to the edge of a cliff.

If I tried to push to the initial repository right now, I’d be in for a nasty surprise:

#   mat@mm wd (ip_master)$ git push
#   Counting objects: 7, done.
#   Compressing objects: 100% (4/4), done.
#   Writing objects: 100% (6/6), 564 bytes, done.
#   Total 6 (delta 1), reused 0 (delta 0)
#   Unpacking objects: 100% (6/6), done.
#   To /Users/mat/blog/long-term-fork/test/initial_project
#      7178a89..3ca0240  master -> master

Oops! By default, ‘git push’ syncs up all branches of the same name with the current branch’s origin. So that would push the new project’s master branch on our initial project’s shared master.

This is obviously not what we want. We only want ip_master to be pushed to the initial project’s master.

Trying to remember to always explicitly run ‘git push ip_origin ip_master:master’ wouldn’t do it for me. To keep the analogy, that would be akin to doing a cartwheel on the edge of said cliff: a lot of fun until you make a mistake. So obviously we’d like a simple ‘git push’ to do the right thing.

So here’s how we configure it:

git config remote.ip_origin.push refs/heads/ip_master:master

Now we can safely issue the ‘git push’ command from ip_master and have Git push ip_master to ip_origin/master.

git push
#   Counting objects: 4, done.
#   Compressing objects: 100% (2/2), done.
#   Writing objects: 100% (3/3), 310 bytes, done.
#   Total 3 (delta 0), reused 0 (delta 0)
#   Unpacking objects: 100% (3/3), done.
#   To /Users/mat/blog/long-term-fork/test/initial_project
#      027a48f..9db6c3f  ip_master -> master

There’s now only one remaining annoyance we’re not yet protected against. If new branches are created in the initial project’s central repository, a ‘git pull’ when standing in the ip_master branch would pull them all in our new project’s working directory. I consider this one only an annoyance, since I could just decide not to pay attention to them. On the other hand they will be distracting and may also clash with the branches I create for the development on my new project. So we want to avoid that behavior as well.

To better understand the current Git configuration, let’s have a look at .git/config:

  ...
  [remote "ip_origin"]
    url = /Users/mat/blog/long-term-fork/test/initial_project
    fetch = +refs/heads/*:refs/remotes/ip_origin/*
  [branch "master"]
    remote = origin
    merge = refs/heads/master
  [branch "ip_master"]
    remote = ip_origin
    merge = refs/heads/master
  [remote "origin"]
    url = ../project2
    fetch = +refs/heads/*:refs/remotes/origin/*

So in both ‘remote’ sections I can see that fetch is configured to bring everything locally (the * wildcards).

So here’s how I limit what gets pulled when I pull from the initial project’s repository.

git config remote.ip_origin.fetch +refs/heads/master:refs/remotes/ip_origin/master

The part before the colon is the name of the interesting branch on the remote server. The part after the colon is your local Git repo’s internal branch, used to track the remote branch (not to be confused with our user branch ip_master).

Setting up remote branches on both projects, pulling, pushing and exchanging more commits is left as an exercise to the reader.

Conclusion

So that’s the gist of it, my friends. I am basically set up to work on my new project like I would in a more typical situation. I also have a special branch set up to interact with the initial project. With this branch I’ll be able to do two things:

• Pull new developments from the initial project and then cherry-pick only the shareable commits into the new project’s other branches.
• Cherry-pick in the other direction to bring certain commits from the new project into this branch and then push them up to the initial project.

For this approach to be useful however, we’ll have to make sure we create as concise commits as possible. Gone are the days of committing a whole afternoon in one meaningless commit containing 12 different modifications.

Of course coding sprees of a couple hours are not out of the question. Features like ‘git add −−patch’ are a great help when comes time to extract meaningful commits out of the result of a few hours of intense coding. For a good introduction to −−patch (and a few other powerful features), be sure to read Ryan Tomayko’s The Thing about Git.

The executive summary

So let’s reiterate strictly the interesting bits necessary to set up a long term fork when starting a new project from an existing one.

We already have:

• the initial project’s repository at url git@github.com:username/initial_project.git
• the new project’s empty repository, also created at git@github.com:username/new_project.git

# Create new local clone for the new project
git clone -o ip_origin git@github.com:username/initial_project.git new_project
cd new_project

# Set up a standard track between initial master and local ip_master
git config branch.ip_master.remote ip_origin
git config branch.ip_master.merge refs/heads/master

# Automatically push the right branch
git config remote.ip_origin.push refs/heads/ip_master:master
# Don't bring in the other shared branches from initial project
git config remote.ip_origin.fetch +refs/heads/master:refs/remotes/ip_origin/master

# Push new local repo it to new shared repo
git remote add origin git@github.com:username/new_project.git
git push origin master

# Configure standard track of master with new local repo
git config branch.master.remote origin
git config branch.master.merge refs/heads/master

That’s it! We now only have to cherry-pick like there’s no tomorrow.

redgreen

The redgreen gem highlights the . F and E (among other bits) in your test runs with green, red and yellow. When running my whole test suite it’s something I want to have. It’s not only visually pleasing, but it also lets me see at a glance whether any problems were encountered.When run from TextMate, tests using redgreen are displayed without having the console coloring stripped out, which gives something like this:

Riiiight.For the record, here’s the nice result when run at the console:

quietbacktrace

The other gem I can’t do without is James Golick and Dan Croak’s quietbacktrace. This gem lets you specify filters and silencers to clean up those huuuuuge Rails or Merb backtraces.Filters let you remove useless parts of a given line in the backtrace, such as the path leading to your gems. Silencers let you completely remove some lines from the backtrace, such as all the lines referring to what happened inside Rails, leading to your error. The most popular filters and silencers are already provided with quietbacktrace, as you’ll see below.So the result, of course is that messing with backtraces breaks TextMate’s ability to link a backtrace line with the corresponding file.

A simple snippet to fix this

Since I didn’t really want to do away with these tools, I included a bit of a hacky snippet in my test_helper.rb file to detect whether a test run was happening in TextMate or at the console.If you find yourself in the same fix as me, feel free to use the following.Among your requires:

IN_TM = !ENV['TM_DIRECTORY'].nil?
unless IN_TM
  require 'redgreen'
  require 'quietbacktrace'end

And then:

class Test::Unit::TestCase
  unless IN_TM
    self.backtrace_silencers << :rails_vendor
    self.backtrace_filters   << :rails_root
  end
  #...
end

Now I can have my testing niceties when running my tests from the console and they don’t break the TextMate integration :-)

Trying this out for the first time?

If like me you just decided to try this out for the first time, the shortcuts are easy to remember. Command-R runs any Ruby file (in this case, the whole test file) and Command-Shift-R runs the single test the cursor is in.There’s a little hiccup in sight, however. If you’re using Rails 2+, you have to apply a very small fix to TextMate before you’ll be able to run your tests. Read more about the fix on Marc-André Cournoyer’s blog.

Even though I feel I’m a pretty competent software developer, I’m very excited to join another developer for which I have a truly profound respect. I’ve only recently made the career move of working on the web full time, and this is a great opportunity for me to kickstart this phase of my career by working with a Rails / REST / jQuery guru.

Starting today, I’m working for GiraffeSoft, with James Golick. We’ve wanted to work together for a while, and the stars have finally aligned to make this possible.

Due to its roots, Git supports a wide array of usage scenarios for interacting with remote repositories, and we love it that way. It’s a big factor in the flexibility and power of the tool.

However in simple scenarios, there’s still a bunch of commands you must run to accomplish simple tasks. I believe the commands for the simple scenario can be simpler.

Last January, Carl Mercier created git-remote-branch. I’ve found this script very useful and, with his permission, I’ve decided to keep moving it forward and add a few features to it.

git_remote_branch

The first purpose of git_remote_branch is to encapsulate all the commands that need to be run to interact with remote branches in simple scenarios.

Its second purpose is to be a learning tool. git_remote_branch does two things to help you learn these commands:

• It clearly displays the commands it runs on your behalf, in a beautiful shade of red;
• It has an ‘explain’ meta-command that will simply display the list of commands instead of running them for you.

Examples

Help

Let’s start with the simplest of all:

$ grb

or

$ grb help
git_remote_branch version 0.2.2

  Usage:

  grb create branch_name [origin_server]

  grb delete branch_name [origin_server]

  grb track branch_name [origin_server]

  If origin_server is not specified, the name 'origin' is assumed (git's default)

  The explain meta-command: you can also prepend any command with the keyword 'explain'.
  Instead of executing the command, git_remote_branch will simply output the list of commands
  you need to run to accomplish that goal.
  Example:
    grb explain create
    grb explain create my_branch github

  All commands also have aliases:
  create: create, new
  delete: delete, destroy, kill, remove
  track: track, follow, grab, fetch

As you can see, the syntax for all commands is very regular: action, branch_name and optionally, origin_server.

To facilitate learning even more, aliases are also provided. So to take an example,

$ grb track his_branch

and

$ grb fetch his_branch

are perfectly equivalent.

create

Create lets you create a new branch both remotely and locally. Note that it’s not made to share an existing branch (that feature’s coming).

So what it does is to push your current branch as a new remote branch, then create it locally and track it, for easier pulling afterwards.

$ grb create some_branch
git_remote_branch version 0.2.2

git push origin master:refs/heads/some_branch
Total 0 (delta 0), reused 0 (delta 0)
To /path/to/repo/
* [new branch]      master -> some_branch

git fetch origin

git branch −−track some_branch origin/some_branch

git checkout some_branch
Switched to branch "some_branch"

delete

Presenting features with names that are too self-evident is boring. Let’s get to the point, already.

$ grb delete some_branch
git_remote_branch version 0.2.2

git push origin :refs/heads/some_branch
To /path/to/repo/
- [deleted]         some_branch

git checkout master
Switched to branch "master"

git branch -d some_branch

track

Track lets you easily track the changes that are made to an existing remote branch you’re not tracking already. Each time you pull from the remote repository, the local branch will be automatically merged with the remote branch.

$ grb track his_branch
git_remote_branch version 0.2.2

git fetch origin
From /path/to/repo/
* [new branch]      his_branch -> origin/his_branch

git branch −−track his_branch origin/his_branch

explain

Explain will spew out all commands necessary to accomplish one of the previous actions. There are two ways of using it. The simplest will give you dummy commands:

$ grb explain create
git_remote_branch version 0.2.2

List of operations to do to create a new remote branch and track it locally:

git push origin master:refs/heads/branch_to_create
git fetch origin
git branch −−track branch_to_create origin/branch_to_create
git checkout branch_to_create

Or you can have steps that are tailor-made for what you want to accomplish.

$ grb explain create my_branch github_origin
git_remote_branch version 0.2.2

List of operations to do to create a new remote branch and track it locally:

git push github_origin master:refs/heads/my_branch
git fetch github_origin
git branch −−track my_branch github_origin/my_branch
git checkout my_branch

get git_remote_branch

(Hey, this title has a nice ring to it)

sudo gem install webmat-git_remote_branch −−source=http://gems.github.com

Now with rubygems 1.2.0 out (don’t do it with a prior version), you can also add GitHub as a permanent source for your gems:

sudo gem sources -a http://gems.github.com

Now and ever after, you will be able to get anything from GitHub with a simple

sudo gem install webmat-git_remote_branch

If in your eagerness you’ve added github as a source before running

sudo gem update −−system

Please refer to What to do when gems.github.com breaks gems FOREVAR

Look ma, no tests!

Uhhhh, yeah, I know…

This tool is still extremely early in its life and – dare I say it – it’s only hand-tested for now. I’ve been using it personally for a while and it’s working very well for me, if that means anything :-)

So this is still a quick script, only now it has lipstick.

This software should be considered an early version of a pre-alpha. You’ve been warned!

If this makes you queasy, there’s always the ‘explain’ command that can act as your cheat sheet without actually having grb run the commands on your behalf.

For the extra queasy, well you can find the commands very easily without running grb. Just point your favorite editor to lib/git_remote_branch.rb. All commands are there, at the beginning of the file.

For the brave, try it out and tell me what you think. Improvements, bug reports and contributions are all welcome.

And remember, in case of an emergency, you can always refer to my illustrated guide to recovering lost commits with Git ;-)

Here’s what’s to come:

• lots of tests
• a ‘remotize’ functionality (for existing local branches)
• a much better resilience to use in faulty situations (e.g. deleting something that’s not present locally or remotely)
• the possibility to specify different branch names locally vs remotely
• slap an open source licence on it all
• and so much more!

You’ll never find a group who wears their ignorance of technology more proudly than the average business person. “I’m not a computer guy,” they’ll say with a big smile on their face. Well gee, the personal computer is only the most significant invention to come along in the past 100 years.

By Tales of an IT Director

Like with any such tool, as soon as you get to know it enough, you start pushing the boundaries. Git gives you a lot of control over your repository:

• trivial branching and merging (even with long lived branches);
• rebasing as a cleaner alternative to merging;
• stashing aside your changes for a quick fix elsewhere;
• extracting logical, distinct commits from a multi-hours coding spree;

The list goes on…

More traditional version control systems don’t give you as much power as Git by any stretch of the mind. They are like taking a walk in the woods with your parents, at age 14.

You’re probably gonna see and do neat stuff, but you sure ain’t gonna get lost or anything.

Using Git on the other hand is more akin to being handed a cool motocross to go play alone in the woods… Also at age 14.

We all know what’s bound to happen, right?

You’ll smash into a tree.

The source control equivalent to slamming into a tree is losing commits. Getting all of Git’s power and flexibility at once can be somewhat dangerous. You’ll find it so easy and helpful to branch and merge that you’ll start doing it way more often. On the other hand — especially in the beginning — you’ll misunderstand or plainly miss some important warnings, and make errors. Or you may just end up in weird merging situations you never thought of, and don’t necessarily understand. These situations can often result in losing commits or whole branches.

My goal with this article is to make sure you understand the situation you’re really in: you have temporarily lost commits or branches.

Disclaimer

This article assumes a basic knowledge of how git works, e.g. committing, branching and merging.

My first time

The first time I lost a commit was a good while ago. I can’t remember the details, but basically I got bit by the fact that under the covers, Git uses hard links liberally. Which means that copy / pasting your code directory as a recovery solution isn’t going to save your ass when you attempt a potentially damaging operation you don’t fully understand.

Note that compressing your code directory will, though.

So there I was, after attempting an operation I didn’t really understand. I knew I had failed what I attempted and I knew I had lost my last commit. Ironically, I still had Gitk open, displaying that very commit. As long as I didn’t refresh the Gitk view with F5 I could see the lost commit.

Here’s a fun fact: under OSX (not sure about Linux) you cannot select and copy text from Gitk’s interface, except for the SHA1 field [1]. I knew Git probably had a way to recover from that… But you know, I just wanted to get back to work and NOT search documentation and blog posts endlessly.

So I took screenshots, passed them real quick through GOCR, just to see how far it would get.

The result: GOCR doesn’t like the font Monaco :-)

How to (really) recover lost commits with Git

Recently I lost a commit again. This time however, Gitk was not up to date. I knew I’d just lost something I wouldn’t necessarily remember in its entirety. It was a commit an hour old, touching many files. And I have a crappy memory.

This time I had to do it the right way. I found out it’s really easy (once you figure it out), but I found no really clear explanation anywhere. So here goes.

Initial setup

If you wanna follow along — and I strongly recommend it — here’s the boring few steps to create a dummy repo and bring it up to speed with for the rest of this article. We’re going to beat the hell out of this repo and it’s going to be fun.

So just paste the following into a console:

mkdir recovery;cd recovery
git init
touch file
git add file
git commit -m "First commit"
echo "Hello World" > file
git add .
git commit -m "Greetings"git branch cool_branch
git checkout cool_branch
echo "What up world?" > cool_file
git add .
git commit -m "Now that was cool"
git checkout master
echo "What does that mean?" >> file

Ok, let’s look at where we’re at:

gitk ––all &

The ––all option lets you see all branches at the same time, as well as your stashes.

Click here to enlarge your picture!!1

We can see the cool_branch as well as some yet uncommitted changes over the master branch.

mathieu@ml recovery (master)$ ls -l
total 16
-rw-r--r--  1 mathieu  staff    15B  7 Jun 18:19 cool_file
-rw-r--r--  1 mathieu  staff    33B  7 Jun 18:19 file

Got my 2 files, I’m good to go.

Let’s make a mistake

Let’s say I decide I want to bring in these cool changes in master. I’ll do it with a rebase. I know there’s no big risk of conflicts so that’s a no-brainer.

mathieu@ml recovery (master)$ git rebase cool_branch
file: needs update

Now if you look carefully you’ll notice I wasn’t paying attention when Git gave me a feeble complaint about ‘file’.

Everything’s well, so I think “Ok, I don’t need cool_branch anymore”.

mathieu@ml recovery (master)$ git branch -d cool_branch
error: The branch 'cool_branch' is not an ancestor of your current HEAD.
If you are sure you want to delete it, run 'git branch -D cool_branch'.

Huh? Whatever you say, Linus. Let’s get on with it.

mathieu@ml recovery (master)$ git branch -D cool_branch
Deleted branch cool_branch.

Ahh, it feels good to be a Git ninja. Now let’s see where we’re at and refresh Gitk with F5.

Oops, my cool commit is gone! That thing can’t be right. Let’s panic:

mathieu@ml recovery (master)$ ls
file

mathieu@ml recovery (master)$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#    modified:   file
#
no changes added to commit (use "git add" and/or "git commit -a")

mathieu@ml recovery (master)$ git diff
diff --git a/file b/file
index 557db03..f2a8bf3 100644
--- a/file
+++ b/file
@@ -1 +1,2 @@
 Hello World
+What does that mean?

Oh sh!t

So the ‘file: needs update’ message back there meant that the rebase didn’t happen, because I had pending changes.

Helpful.

Recovering a lost commit

Since I don’t think my uncommitted work is complete, I’ll just stash it instead of committing it. Then I’ll hunt down my lost work.

mathieu@ml recovery (master)$ git stash save "Questioning the universe"
Saved working directory and index state "On master: Questioning the universe" HEAD is now at 6da726f... Greetings

In the name of paranoïa, let’s make sure this got in right:

Ok, let’s get on with our rescue mission:

mathieu@ml recovery (master)$ git fsck −−lost-found
dangling commit 93b0c51cfea8c731aa385109b8e99d19b38a55be

That sounds right, exactly one commit in the lost and found.

Let’s just make sure:

mathieu@ml recovery (master)$ git show 93b0c51cfea8c731aa385109b8e99d19b38a55be | mate

Bingo!

Different ways to recover the commit

There are a few different ways to recover that commit. Obviously we can just copy and paste that snippet, but in the case of a bigger commit, that approach will just amount to a lot of error-prone busywork.

I’ll reclaim my Git ninja status and try it a few different ways.

Recover it with rebase

Let’s just replay this change on top of master:

mathieu@ml recovery (master)$ git rebase 93b0c51cfea8c731aa385109b8e99d19b38a55be
First, rewinding head to replay your work on top of it...
HEAD is now at 93b0c51... Now that was cool
Fast-forwarded master to 93b0c51cfea8c731aa385109b8e99d19b38a55be.

Neat! Now I feel like a ninja worthy of the title again.

So let’s rewind one commit and try it another way.

mathieu@ml recovery (master)$ git reset --hard head^
HEAD is now at 6da726f... Greetings

Ok, the commit’s gone.

(Don’t tell anyone but my inner ninja is feeling queasy again.)

Recover it with merge

There are cases where rebase is not powerful enough. For example when you expect to face a lot of conflicts. In this case merge is a better solution:

mathieu@ml recovery (master)$ git merge 93b0c51cfea8c731aa385109b8e99d19b38a55be
Updating 6da726f..93b0c51
Fast forward
 cool_file |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 cool_file

Too easy… Rewind!

mathieu@ml recovery (master)$ git reset --hard head^
HEAD is now at 6da726f... Greetings

Recover it with cherry-pick

If instead you had a few commits one after another but you just want to pick the last one, rebase and merge won’t do. They would bring the whole branch back in master. That’s a situation for cherry-pick.

mathieu@ml recovery (master)$ git cherry-pick 93b0c51cfea8c731aa385109b8e99d19b38a55be
Finished one cherry-pick.
Created commit f443703: Now that was cool
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 cool_file

Insane!

This only leaves one open question: WHO’S YOUR DADDY NOW, GIT?

Now that we’ve established the answer to that question, let’s get back to work!

Let’s make a second mistake

mathieu@ml recovery (master)$ git stash clear

Or was it Git stash apply?

Oh jeez, there we go again…

mathieu@ml recovery (master)$ git fsck −−lost-found
dangling commit 24e3752f7a73ae98b361ce1c260e1f285d653447
dangling commit 93b0c51cfea8c731aa385109b8e99d19b38a55be

Ok, we still see the one we lost earlier, 93b0c51… Let’s look at the other one.

mathieu@ml recovery (master)$ git show 24e3752f7a73ae98b361ce1c260e1f285d653447
commit 24e3752f7a73ae98b361ce1c260e1f285d653447
Merge: 6da726f... c90f079...
Author: Mathieu Martin <webmat@gmail.com>
Date:   Sat Jun 7 16:02:57 2008 -0400

On master: Questioning the universe

diff --cc file
index 557db03,557db03..f2a8bf3
--- a/file
+++ b/file
@@@ -1,1 -1,1 +1,2 @@@
Hello World
++What does that mean?

Spot on. Let’s try something wild, while we’re here.

mathieu@ml recovery (master)$ git checkout 24e3752f7a73ae98b361ce1c260e1f285d653447
Note: moving to "24e3752f7a73ae98b361ce1c260e1f285d653447" which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at 24e3752... On master: Questioning the universe

mathieu@ml recovery (24e3752...)$

As you may have noticed, my console always indicates which branch I’m in, so far [2]. But now I seem to be in some kind of twilight zone, which Gitk confirms.

Let’s follow Git’s suggestion and make that a branch.

mathieu@ml recovery (24e3752...)$ git checkout -b recovery
Switched to a new branch "recovery"

mathieu@ml recovery (recovery)$

Looks weird, like stashed items always do, but at least we have our commit.

After fiddling around with what’s been recovered from the stash, I recommend NOT keeping it as a commit.

If you try to replay the change in the recovery branch over master’s most recent commit, you lose the “Questioning the universe” commit. Probably because a stash is a weird kind of commit, or maybe because of a bug. I don’t know.

(Don’t follow this one in your console)

mathieu@ml recovery (recovery)$ git rebase master  #I said don't do this one
First, rewinding head to replay your work on top of it...
HEAD is now at 93b0c51... Now that was cool
Nothing to do.

If instead I checkout master and then rebase its last change over the ‘recovery’ branch it seems to work.

However since I just saw a commit disappear when rebasing the other way around, I get the feeling that this isn’t a normal commit and it may come back to haunt me later.

Recover it by applying a diff

Let’s just apply the diff to master. I’ll do as if it actually was a substantial commit, involving lots of modifications on lots of files, and apply it automatically with ‘git apply’.

First let’s visualize where we’re at, again:

A diff against master is not what we want since master includes a new (very cool) commit.

Instead we just want to see the changes introduced by the current commit. To do this we can compare it with the common ancestor between the master and recovery branches. So let’s start by finding it’s ID.

mathieu@ml recovery (recovery)$ git diff 6da726f37683c83947d54314cd32ca1ee9d490e0
diff --git a/file b/file
index 557db03..f2a8bf3 100644
--- a/file
+++ b/file
@@ -1 +1,2 @@
Hello World
+What does that mean?

Looks good. Now we throw that diff upstairs.

git diff 6da726f37683c83947d54314cd32ca1ee9d490e0 > ../recovery.diff

Then get apply it to our master branch.

mathieu@ml recovery (recovery)$ git checkout master
Switched to branch "master"

mathieu@ml recovery (master)$ git apply ../recovery.diff

And we finally confirm that everything’s under control.

mathieu@ml recovery (master)$ git status
# On branch master
# Changed but not updated:
#   (use "git add <file>..." to update what will be committed)
#
#    modified:   file
#
no changes added to commit (use "git add" and/or "git commit -a")

mathieu@ml recovery (master)$ git diff
diff --git a/file b/file
index 557db03..f2a8bf3 100644
--- a/file
+++ b/file
@@ -1 +1,2 @@
Hello World
+What does that mean?

This change was first stashed rather than committed because I felt it was not complete. Applying it with Git apply only introduces it as an unstaged change, which works perfectly for this situation. Now I can keep banging at the code until I feel this actually deserves to be committed.

mathieu@ml recovery (master)$ echo "I don't know" >> file

mathieu@ml recovery (master)$ git commit -a -m "Conversation of staggering depth"
Created commit 65a4794: Conversation of staggering depth
 1 files changed, 2 insertions(+), 0 deletions(-)

Cleaning up the crud

Ok, so now I still have this weird looking recovery branch.

Since it’s now useless we can get rid of it.

mathieu@ml recovery (master)$ git branch -d recovery
error: The branch 'recovery' is not an ancestor of your current HEAD.
If you are sure you want to delete it, run 'git branch -D recovery'.

Aha! This time everything’s committed correctly, so I know I can delete it for real. Git is complaining because that commit was not included through its normal merge or rebase commands. So it warns me that I may be about to lose something. However I know I got everything through the diff I made and re-applied.

mathieu@ml recovery (master)$ git branch -D recovery
Deleted branch recovery.

Now that I’m aware that commits are reachable even if they’re not in a branch anymore, I wonder about my repo’s size.

mathieu@ml recovery (master)$ git gc
Counting objects: 22, done.
Compressing objects: 100% (14/14), done.
Writing objects: 100% (22/22), done.
Total 22 (delta 7), reused 0 (delta 0)

mathieu@ml recovery (master)$ git prune

Fair enough. I would expect the unused commits to now be unreachable, but strangely enough:

mathieu@ml recovery (master)$ git fsck −−lost-found
dangling commit 49ed65cdea22443af3f1fd400754fe1517421b24
dangling commit 4b1bf4792cba929e88114379d7d5e86a2dc9990f
dangling commit 6cdf88318109dede7bd3c1a75be76c7255708ded
dangling commit 715a6b2cfe797383216d0f9b04fe8f50e90e779f
dangling commit f443703e5060d9f3b4d97504bda5f97e5a0b31e8

If anyone finds out what that’s all about, please let me know!

Maybe Git’s just refusing to do any work unless it’s going to actually save a considerable amount of space? I have no idea.

Conclusion

Once you know how to recover from bad mistakes, you’ll find that Git is not only a very powerful tool, but also a very forgiving one. As opposed to a motocross.

The following commands will help you figure you way out of most bad situations:

• git show
• git fsck −−lost-found
• git diff

And these ones will actually get out of these bad situations:

• git rebase
• git cherry-pick
• git merge
• git apply

As I think I demonstrated, Git gives you the ability to recover from most bad mistakes. The fact that any single commit can be cherry-picked, checked out, rebased or merged makes it really easy to recover from hairy situations.

The only case where you might actually lose information is when something has not been committed or stashed yet, which I think is perfectly reasonable.

So if you take only one thing away from this article, let it be this. Git is much safer than a motocross.

Footnotes

[1] At the time I didn’t know that just having the SHA1 id was enough to save me.

[2] See how to configure your console in the same manner and also get auto-completion for Git here.

<branch>

When this parameter names a non-branch (but still a valid commit object), your HEAD becomes detached.
Junio C. Hamano — the checkout documentation

Git — the only SCM that beheads its users.

• Rubinius for the Layman, Part 1: Rubies All the Way Down
• Rubinius for the Layman, Part 2: How Rubinius is Friendly

In this shorter second installment, I’ll present the ways in which Rubinius will be friendly to your multiple personalities:

• you, the programmer;
• you, the (potential) contributor.

Rubinius is programmer-friendly

Backtraces

The first reason why I consider Rubinius more programmer-friendly is the better backtraces. I’ll show some examples run from each interpreter’s interactive console.

Take the following piece of buggy code. Notice the nil at the end of the array.

ary=['bob', 'mom', nil].inject([]) {|result, element| result << element.to_sym}

With MRI you get the following:

NoMethodError: undefined method `to_sym' for nil:NilClass
        from (irb):5
        from (irb):5:in `inject'
        from (irb):5:in `each'
        from (irb):5:in `inject'
        from (irb):5

Ok, we have basic information on all the method calls that led to the NoMethodError.

Rubinius goes a step further, however:

NoMethodError: No method 'to_sym' on an instance of NilClass.
   from Kernel(NilClass)#to_sym (method_missing_cv) at kernel/core/kernel.rb:606
   from Object#irb_binding {} at (irb):4
   from Enumerable(Array)#inject {} at kernel/core/enumerable.rb:375
   from Array#each at kernel/core/array.rb:573
   from Enumerable(Array)#inject at kernel/core/enumerable.rb:371
   from Object#irb_binding {} at (irb):4

The backtrace tells me in which module each method on the stack was defined!

Q: How awesome is that?

A: Very!

I’m almost looking forward to debugging buggy mixins or method chains. Aren’t you?

These backtraces will definitely come in handy when debugging in situations where a lot of magic is happening behind the scenes.

Bring back the horse!

There’s another interesting detail about the backtraces. Remember what language Rubinius is written in? Heh. The backtraces Rubinius generates go deeper.

Say we have another piece of buggy code:

s = 'ahh'
s[1] = nil

MRI:

TypeError: can't convert nil into String
 from (irb):5:in `[]='
 from (irb):5

Rubinius:

TypeError: Coercion error: nil.to_str => String failed:
(undefined local variable or method `to_str' for nil)
   from Type.coerce_to at kernel/core/kernel.rb:19
   from Kernel(String)#StringValue at kernel/core/kernel.rb:80
   from String#splice! at kernel/core/string.rb:2192
   from String#[]= at kernel/core/string.rb:360
   from Object#irb_binding {} at (irb):8

Not only is the message more descriptive, but as you can see, we can see deeper into the interpreter’s inner workings.

The argument could be made that this could lead to noisier backtraces. On one hand it’s true that adding 5 lines to a 30+ lines backtrace generated in a Rails unit test will rarely help. On the other hand, all this additional information may sometimes help finding subtle bugs (in user code or in Rubinius) or figure out that our use of a feature is not quite correct.

I say the additional information is very welcome. And to help keep the information overload in check, let’s get off our asses and use the quiet_backtrace gem by James Golick and Dan Croak.

The real Rubinius backtraces

It gets even better. The previous examples were only run in irb. So the additionnal information was getting squeezed in IRB’s exception formatting. Say I save both pieces of buggy code to 1.rb and 2.rb. You’ll notice too that I split program 2 in 2.rb and /lib/2.rb (I plan on making it a gem).

Here’s in full colored glory the insults Rubinius will throw back at me:

mat@laptop rubinius $ rbx 1.rb
An exception has occurred:
    No method 'to_sym' on an instance of NilClass. (NoMethodError)Backtrace:
    Kernel(NilClass)#to_sym (method_missing_cv) at kernel/core/kernel.rb:612
                        Object#__script__ {} at 1.rb:1
                 Enumerable(Array)#inject {} at kernel/core/enumerable.rb:375
                                  Array#each at kernel/core/array.rb:573
                    Enumerable(Array)#inject at kernel/core/enumerable.rb:371
                           Object#__script__ at 1.rb:1
                    CompiledMethod#as_script at kernel/core/compiled_method.rb:326
                         Compile.single_load at kernel/core/compile.rb:238
                 Compile.load_from_extension at kernel/core/compile.rb:310
                           Object#__script__ at kernel/loader.rb:190

mat@laptop rubinius $ rbx 2.rb
An exception has occurred:
    Coercion error: nil.to_str => String failed:
(undefined local variable or method `to_str' for nil) (TypeError)
Backtrace:
                 Type.coerce_to at kernel/core/kernel.rb:19
     Kernel(String)#StringValue at kernel/core/kernel.rb:80
                 String#splice! at kernel/core/string.rb:2192
                     String#[]= at kernel/core/string.rb:360
              Object#__script__ at ./lib/2.rb:2
       CompiledMethod#as_script at kernel/core/compiled_method.rb:326
            Compile.single_load at kernel/core/compile.rb:238
        Compile.unified_load {} at kernel/core/compile.rb:149
                     Array#each at kernel/core/array.rb:573
           Compile.unified_load at kernel/core/compile.rb:120
         Kernel(Object)#require at kernel/core/compile.rb:450
              Object#__script__ at 2.rb:1
       CompiledMethod#as_script at kernel/core/compiled_method.rb:326
            Compile.single_load at kernel/core/compile.rb:233
    Compile.load_from_extension at kernel/core/compile.rb:310
              Object#__script__ at kernel/loader.rb:190

Holy f****** sh** Batman! How centering the lines makes a difference!

To the left is the logical location: the class/module and the method name. To the right is just the necessary information about the physical location. Notice the paths. They only contain what you need i.e. the relative path to execution — relative to the VM for Rubinius code, and relative to the execution root of my glorious code.

Don’t tell anyone, but I think I’ll intentionally put in bugs at work. Just to see these in the logs! Too bad our logs don’t support coloring.

Rubinius is contributor-friendly

Commit access

The Rubinius project is friendly for the programmer who uses it. But the project itself is also friendly to potential contributors. They currently have a policy called the free-flowing commit bit. This policy says the following:

Anyone who submits a patch which is accepted is granted commit access.

In case you missed it, here it is again:

Anyone who submits a patch which is accepted is granted commit access.

This is not crazy talk. The patch can be as simple as you want. A correction to documentation or a spec for the behavior of a Ruby class not behaving properly under Rubinius. Any form of submission is admissible: a pastie, a ticket prepended with [PATCH] on lighthouse, or any other traditional way (emails, ransom notes etc.) Of course these methods are only temporary, since you’re going to get commit access afterwards.

The idea in being so liberal with commit access is to put the barrier to contribution as low as possible. This way, when your commit bit is set, you’re going to be tempted to help more. And your next contribution is going to be so much easier to submit.

A masterful adaptation of the timeless ‘gateway drug’ style of recruiting, really. Get them hooked with as little friction as possible. Then let them, uhhh, flourish in the wonderful world they just discovered.

How’s that for being contributor-friendly? This may not last forever, I guess. So hurry up and submit a patch! ;-)

The naysayers might once again intervene to remind us that this isn’t going to scale. Well it’s every open source project’s dream to have too many contributors. The community and the team will adapt in time. The fact that Rubinius is hosted with Git also gives the team much more flexibility to try different committing strategies, or simply gives more power to recover from mistakes from newcomers.

To really have a better understanding of Evan’s vision of the Rubinius community, I highly recommend watching the video of his presentation at MountainWest RubyConf. The presentation’s content is really not the same as the other presentation mentioned in my first article. Both are really worth it.

Conclusion

That’s it for now, folks. This is how much Rubinius will be friendly to you. Delicious stacktraces and a very welcoming project. A project ready to give you commit access if you submit anything good: a spec, some doc, or just a plain ol’ bug fix.

Stay tuned for the next installment, in which I’ll start covering a few mildly technical bits of Rubinius. Unless I change my mind and decide to tackle another aspect of Rubinius.

A few references:

• The links for the videos are 3 paragraphs up, you lazy reader. No wait, I’m the one being lazy here.
• How to write a ticket for the Rubinius project.
• Let’s bug James Golick and Dan Croak so they make sure quiet_backtrace is ready for these deep Rubinius stack traces. Ha! Sorry James :-)

Tonight’s fiddling led me to playing around with my .irbrc file. I guess we’ve all searched for examples of config files for IRB at one time or another. When you spend a lot of time into your interactive console, you naturally end up wanting to tweak it to your liking.

However you’ll start having problems as soon as you start fooling around with the different runtimes. Or when you share your irbrc between different machines. Maybe even on different OSes. Either some gems are unsupported or you haven’t installed them yet (and don’t necessarily care).

So tonight I tried to fix that problem. My solution is only a small method that I define in my irbrc, so I’m not sure it’s worth putting it on github yet :-) Let me know what you think.

Presenting tramp_require

Usage

When my irbrc requires something just so I don’t need to do it manually:

tramp_require 'pp' #=> true/false (same return value as a normal require)

Outcome if the gem is not installed:

** Unable to require 'wirble'
--> LoadError: Did not find file to load: wirble

And you IRB loads without a problem (just without the gem pre-loaded).

When I require something I actually use in my .irbrc:

tramp_require('wirble') do
  Wirble.init(:skip_prompt=>true)
  Wirble.colorize
end

If the gem is loaded successfully, the block is executed.

If the gem isn’t loaded successfully, the block’s not executed and the same warning message is shown.

Note that the user code passed in the block is not being rescued: if it your code fails, it’s your problem :-)

Implementation

Here I’ll paste 2 equivalent implementations. The first one is a clean and understandable version (also on pastie):

def tramp_require(what, &block)
  loaded, require_result = false, nil 

  begin
    require_result = require what
    loaded = true 

  rescue Exception => ex
    puts "** Unable to require '#{what}'"
    puts "--> #{ex.class}: #{ex.message}"
  end 

  yield if loaded and block_given? 

  require_result
end

This second version is the one I actually use, if I set the debug variable to false the result is the same as the previous implementation. If I set it to true, I get much more information, including the full backtrace of the exception (also on pastie):

$debug_irbrc=false 

def tramp_require(what, &block)
  loaded, require_result = false, nil 

  begin
    puts('', "requiring #{what}") if $debug_irbrc
    require_result = require what
    loaded = true
    puts "successfully required #{what}" if $debug_irbrc 

  rescue Exception => ex
    puts "** Unable to require '#{what}'"
    exception_details = "#{ex.class}: #{ex.message}"
    if $debug_irbrc
      ex.backtrace.reverse.each{|l| exception_details < < "\n   #{l}"}
    else
      exception_details.insert(0, "--> ")
    end
    puts exception_details
  end 

  if loaded and block_given?
    puts "executing block for #{what}" if $debug_irbrc
    yield
  end 

  require_result
end

Feel free to use these snippets as you see fit :-)

The obligatory full paste of my irbrc is available on pastie, for the curious.

One word of warning, however. I’m guessing some of you will rename tramp_require to something more boring. Just don’t rename it to irb_require. IRB already defines a method named this way.

• Rubinius for the Layman, Part 1: Rubies All the Way Down
• Rubinius for the Layman, Part 2: How Rubinius is Friendly

In January, Antonio Cangiano wrote an article titled Why Engine Yard, Rubinius and Merb matter. In his article he discussed the financing of Engine Yard. More specifically, he discussed why EY was not just another irrelevant startup being funded by starry-eyed investors. Au contraire! Engine Yard is already recognized as a solid hosting service provider for Ruby on Rails, and has a customer base. He then went on to discuss how they are going to use the money to fund the development of Merb and Rubinius by hiring the main contributors and paying them to work on these projects.

These two projects are very important for the future of the Ruby ecosystem. Rubinius is an attempt at a very powerful Ruby VM while Merb is an attempt at a solid alternative to Rails, with its own distinct advantages. It’s great to hear that EY has taken it upon themselves to basically put both projects on steroids.

Being interested in the different Ruby interpreters / VMs, I was glad to get this overview of the event. On the other hand, I was hoping to get more juicy bits on the technology side of Rubinius. Since I’m already familiar with the project (from afar), I thought maybe I could come up with another more technical view on Rubinius. Well, let’s say a layman’s technical view on Rubinius.

I’ve been putting off this series of articles for a couple of weeks because I took the opportunity to talk about Rubinius at the last Montreal on Rails. I didn’t want to steal my own punch lines on my blog before the presentation.

In this series of articles I’ll try to present the Rubinius project from a technical point of view. A 10 000 ft technical view, that is. Some of what I say here is in part based on Evan Phoenix’ presentation at RubyConf 2007, so some of it could sound familiar to those who attended or watched the video (highly recommended).

Q: What is Rubinius?

A: A Ruby VM and Runtime written as much as possible in Ruby, and as little as possible in C. The goal is eventually to completely eliminate the C code.

Of course, the first question that comes to mind is: WTF? It’s a perfectly understandable reaction. So in this first article I’ll mostly address the 3 letter question. The next articles will address all the other aspects of this fascinating project.

It’s not a bad idea

It’s a traditional milestone for any language: the ultimate level of dogfooding. Implementing a VM or a runtime for a language with this very language means a lot in terms of the language’s maturity and power.

And this is valid for many other technologies, too. It’s been a very important milestone for Subversion as well some years back, for example. Their exponential growth really took off when they started using their own Subversion server to host their repository.

Some Ruby skeptics might say that Ruby is not fast and mature enough to implement a VM.

Let’s first address the assertion that ‘Ruby is slow’. It’s a sentence that doesn’t really make sense, when you think about it: Ruby is a language. Apart from being dynamic, very little of the language’s characteristics fundamentally limit the performance it can get. Its current performance problems rather come from its default interpreter, MRI.

As for the maturity of the language, I agree that some of its features may not be such great ideas. Current work on alternative implementations has brought some of them to light, and from here they mostly look like features that are not in the right place. If a project depends on ObjectSpace, well maybe a better design could circumvent that need. And tainting might not be the most sound security model. Whatever. The parts that don’t work are as of now slowly being weeded out by the different Ruby implementers, including by Matz himself who works on YARV.

So Ruby is maturing right now. The ultimate level of dogfooding to help that process is to use it to implement itself.

It’s actually a great idea

Who wants to program in Ruby?

If you’re interested in working on VM technology and you love the Ruby language, here are your options*:

I finally have a day job writing Ruby (thanks Carl). I’m sorry to tell you that I’m not interested in going back to Java or C# in my spare time if I can help it.

Actually, the case is probably even stronger for my fellow programmers who work with Java or C# during the day, dreaming of coming back to their Ruby-red passionate lover at night. Very few of them will come home and fire up Visual Studio / Eclipse / Netbeans, methinks.

* Actually this is an overstatement: some of these projects are very open to contribution, while others are pretty much closed to outside contribution. More on that later.

** These number are those from Evan’s presentation at the end of 2007, so they’re probably quite outdated. My other option was running sloccount on 5 projects I don’t know closely enough and getting all the numbers wrong :-)

Getting feature-complete with how many lines of code?

Another side effect of using Ruby to implement Rubinius seems to be often overlooked. Rubinius is getting close to feature-complete with under 50 000 LOC. Ruby’s a very expressive language, remember?

The Ruby part is reusable

Here’s what I expect from a feature-complete Ruby interpreter:

Actually, if we look underneath, we see MRI and YARV are implemented that way:

When we look at Rubinius, as mentioned already, we see it’s implemented like that:

So if I want to implement my own revolutionary Ruby interpreter with LOLCODE, I now have the option to implement a core VM and use as much as possible from Rubinius for the rest:

So by reusing Rubinius’ Ruby runtime, 1.0 is gonna come much faster now. If some parts are too slow and for some reason can’t be sped up enough with this approach, I can always get back to the bare metal LOLCODE when comes time to optimize performance. Neat, isn’t it?

But… how is that possible?

Enough about why it’s a great idea. Is it even possible to implement Ruby in Ruby (and end up with something decent)? Actually, yes. It’s been done before for languages with similar characteristics.

Ruby’s a dynamic language. Dynamic languages have the reputation of being slow, right? I can concede that there’s probably an upper bound to how fast a dynamic language can get. And this upper bound is probably lower than that of a statically typed language.

But let’s not forget that Smalltalk and Lisp are dynamic languages as well. There are very fast implementations of both languages. Word on the street is that Smalltalk has been seen running at about half the speed of C. Unfortunately, I have no actual precise reference on that. You should therefore assume that it’s a lie. Just pay attention when you hear the name Strongtalk.

A lot of research has been made in the Smalltalk community in the past 30 years. The Rubinius team is drawing on this research as well as other recent virtual machine research. The VM is loosely based on the Smalltalk-80 architecture as described in the Blue Book.

Still don’t believe me?

Here’s how a recent build of Rubinius compares to MRI.

An overview

The details

These following measurements are in seconds, so lower is better.

For the more visual among us

(click for full size)

So there’s clearly something there, considering the fact that right now the focus is not really on performance, but rather on implementing features and on correctness.

Of course keep in mind that each one of these benchmark is merely a (contrived) measurement of the performance of a very specific operation, executed a bunch of times. Every real life application will have it’s own performance characteristics. So do not read too much in the following numbers.

Leave that horse alone

I’ll stop beating this dead horse, now. Once you get past the ‘Ruby in Ruby’ part, there’s still a lot of very interesting aspects to the Rubinius project, both technical and non-technical.

Stay tuned for the next installments, in which I’ll cover some of these other aspects: specs for Ruby, in what ways Rubinius is friendly, enabling new use cases for Ruby, threading, replacing C and how Rubinius might eventually help Rails deployment.

Until then, here are few of the ways you can get more information about Rubinius:

• The most recent Rubinius presentation of Rubinius at Mountain West RubyConf, which I haven’t even checked yet.
• The previous presentation, at RubyConf 2007, one of the sources of information for this post.
• Evan’s blog, Rubinius’ original creator and project leader.
• Adam Gardiner’s blog, one of the committers to Rubinius, who recently started explaining the internals of Rubinius (way more technical than this article).
• The Rubinius website, of course.
• To keep the pulse of the project, you can of course hang out on #rubinius on irc.freenode.net. If however you want to catch up on what you may have missed, the conversations are archived online.

Be sure to subscribe to the new feed.

The main reason why I make the move now is of course to begin the dogfooding process with Defensio.

I’m not going to bore you with a long post about moving a blog. Sorry for the inconvenience :-)

After you’ve started using Ruby, there’s a significant chance you’ll start loathing whatever code base you currently have to work on. Especially if it’s a statically compiled language. A code base you used to think was ok, except for its few quirks.

After a while of perusing the different Ruby-related blogs, you’ll have heard other Rubyists speak of their work with words like beauty, productivity, expressiveness, conciseness, fun and you’ll realize just how far your current language is taking you from all of these words.

You’ll see

Dictionary<string, string> someDic = new Dictionary<string, string>();

And dream of

some_dic = {}

You’ll see multiple declarations for the same method, trying to emulate optional parameters and think of Ruby’s symbols and options hashes, where such a simple method as:

def method_with_options(options = {})
  encoding = options[:encoding] || 'utf-8'
  puts('Other option detected') if options[:other_option]
  #...
end

Enables all of the following uses

method_with_options :encoding => 'utf-16', :other_option => true
method_with_options :encoding => 'utf-16'
method_with_options :other_option => true
method_with_options

You’ll hear about metaprogramming and the complex syntax or frameworks that can bend Java, C# or C++ to allow a programmer to achieve what he seeks.

In the back of your head, you’ll think of the insane flexibility allowed by

- simple Ruby syntax for method chaining or redefinition;

- dynamic class definition, that lets you add methods to any existing class, even Ruby’s core classes;

- duck typing, where objects of any type can be passed to a method, as long as it responds to the expected method calls in a reasonable fashion;

and oh so many other Ruby niceties.

You’ll encounter twisted method definitions such as

bool SomeMethod(int param1, ref SomeClass someClass, out SomeEnum resultType, out string result)

and think of Ruby’s multiple returns that allows you to clearly define what’s a return value and what’s a parameter:

success, resultType, result = some_method(param1, someClass)

You’ll delve into huge, puzzling class hierarchies that struggle just to use the right abstraction level for class names… You’ll eventually realize that the whole hierarchy was simply there to share a few methods among loosely similar classes.

Then you’ll really get irritated at all the accidental complexity that could have been avoided by simply using mixins, where you define a common method and then include it in any appropriate class. And all of this without ever puzzling over strange abstract names for classes that happen to sit between 2 clear-cut levels of abstraction.

You’ll want to explore a new part of the .Net API by playing with it. You’ll create a dummy project in some random directory, find a name for it, include the proper parts of the API in the generic main class created by default and finally start playing with the construct of interest.

All this time you’ll be thinking of IRB, the Ruby interactive console. It not only allows you to play with an existing API with absolutely no fuss, but thanks to Ruby’s flexibility, you can even define classes in the console and then play with them!

But then you’ll think “Oh yeah, IRB is in fact so flexible that I can use if from a frickin’ web page (with a tutorial)!” And you’ll go play there for a couple minutes (when no one’s looking), just to keep you sane for a couple more hours. Until the C++ / C# / Java drudgery is over for the day.

You’ve been warned. If you learn Ruby, you’ll start thinking it’s impossible for you to keep using the technology you’re currently using at work. If you’re patient you’ll try to introduce it there gently (and most likely get frustrated at the time it takes). If you’re not so patient, you’ll just end up changing job.

Next monday I’m joining the great team at Karabunga to work on Defensio. I’ll be doing Ruby and a bit of Rails. Liberation is coming :-)

In the comments, some people concluded that yes, they should try to write more tests, but didn’t know where to start. In this post I won’t suggest frameworks, or specific tutorials. I’d just like to give one very first step that will help you write code that is easier to test. You’ll benefit from it even if you don’t use a testing framework yet.

As the title suggests, what I’m suggesting is pretty simple. Write side effects free methods/functions. Simple isn’t it? The rest of this article is just about explaining my point. So if the light bulb went off already, you can stop reading now.

I’m kidding, of course! So let’s not take anything for granted, instead let’s make sure we’re on the same page and define “side effect free”. It means that a function (substitute with “method” if you like) receives parameters, spits out a result and has not touched anything outside of it. There are two key parts to this definition:

1. The function does not depend on anything else than it’s parameters: it does not expect a variable to be set outside of it to work properly (at the object, class or global level).
2. It does not modify anything. Its result can be entirely observed either from the return value or from the exception thrown.

Of course you often have to modify the state of the application as a result of a computation. What I’m suggesting is not a substitute for that. What I’m suggesting is simply to put the juicy bits of your computation in a side effect free function. This part will be trivial to test, but you’ll still have the code that uses this function. That other part, which modifies the state of the app will need unit tests or other higher level testing.

Trivial examples of side effect free functions can be found in any good math library supplied with a language. Of course no modern language expects you to set a global variable in order to compute a square root. Those who want to follow the Ruby examples can do so on Try Ruby (fear not, you’ll be able to follow along even if you don’t know any Ruby).

Math.sqrt(4)
  => 2.0

And

Math.sqrt(-1)
Errno::EDOM: Numerical argument out of domain - sqrt
  from (irb):6:in `sqrt'
  from (irb):6

So there we have it. The result of squirt is observed either from the return value or from the exception thrown. We don’t expect any state to have been modified anywhere else in the application. I’ll present a less trivial example in a bit, but first let me just say that the direct consequence of writing this kind of code is that you can test it trivially, whatever your technique of choice.

• If you’re in your debugger or in your interactive console, you can call it as many times as you want, with different parameters and check out if its behavior is what you expect.
• If you use unit tests, you can code the interesting scenarios and verify their expected outcome, only in a repeatable manner.

Now since we all have a Math library of some sort in our language of choice, let’s look at another example: analyzing the parameters that will dictate the execution of your program. This is valid for configuration files with a bit more work, but let’s keep the example simple and just analyze command-line parameters.

Most languages provide us with some kind of array of parameters when entering our main function. The common way of dealing with them is to slap a big if or switch statement somewhere at the beginning of your program, which sets the state of the application accordingly, before actually starting to work on the application’s main task.

A side effect free approach would be to split the process in two parts:

1. parse the parameters
2. set the state / do some work

For example we could define a function that accepts an array and returns a hash (a Dictionary for .Net folks, a Map for Java folks) of the execution parameters:

{
  'arg1' => 'val1',
  'arg2' => 'val2'
}

So let’s say we start with the following method to analyze our arguments :

def analyze_args(arg_list)
  parsed_args = {}
  #We check that each argument is in the form 'arg=value'
  arg_list.each { |arg|
    key, value = arg.split '='
    if (key.nil? || value.nil?)
      raise "Some arguments are not in the 'arg=value' format"
    end
    parsed_args[key] = value
  }
  return parsed_args
end

Now we can trivially test the parsing of the command-line params:

analyze_args(["mom"])

RuntimeError: Some arguments are not in the 'arg=value' format
  from (irb):23:in `analyze_args'
  from (irb):20:in `each'
  from (irb):20:in `analyze_args'
  from (irb):29
  from :0

analyze_args(["mom=food"])
  => {"mom"=>"food"}
analyze_args(["mom=food", "dad=car"])
  => {"mom"=>"food", "dad"=>"car"}

Now that this part is taken care of, I can test it with whatever input I want, trivially.

To reiterate, in order to make your code easier to test, just extract the juicy bits of your program in side effect free functions and keep them apart from the rest of your program, which in turn makes sense of the result. At least the side effect free parts will be trivial to test.

This is just the beginning of the testability and automated tests journey, however. Of course you still have the rest of the program to test, preferably in an automated fashion.

Something unsettled me, however, when I saw it. It has nothing to do with the team or what the narrator says. In the demo, he shows us a couple of features of the designer, and inevitably we end up seeing snippets of code. That’s when the culture shock, uhh, shocked me.

I’ve seen plenty of verbose code in my time. I even work in C# right now, which I find errs on the side of verbosity. So I’m used to seeing verbose code, and I’m certainly used to see WinForms code.

But seeing this WinForms code in Ruby felt very weird.

.

C# colleagues might tell me that this code just satisfies what the API wants to hear, in the way it wants to hear it. But I’m getting used really fast of the Ruby way of doing things. Which consists “sane defaults” for a start, and using DSLs to solve complex problems in a declarative manner rather than in an imperative manner.

WinForms development with Ruby screams for a Rubyish wrapper. Anybody up for that? If it can be done for Java’s Swing (with Profligacy, the Swing reducer), it can certainly be done for WinForms.

For my part, however, I’d be much more interested in exploring XAML with Ruby, but that’s another story.

So if you want to check the presentation again for some reason, here it is: JRuby – On and off Rails.

Also available is my take on Nick Sieger’s, Rails performance numbers.

I suppose some of you might be interested in my other references, as well. So here’s a recap of the links I mentioned at the end of the presentation.

The people

Charles Oliver Nutter works for Sun, he’s the main driving force behind the compiler, among other things. If you’re interested to hear about easy bugs to fix in JRuby, he posts a list once in a while.

Ola Bini works for Thoughtworks and he also works on JRuby full time. His recent feats include speeding up the regex implementation of JRuby.

Nick Sieger also works for Sun, but his job is on the user end of JRuby. He contributes to JRuby as well, however. And by the way, check out the follow-up on the talk he and his team gave at RailsConf. The article discusses implementing some parts of the model of his application as RESTful services, thus giving them more flexibility for scaling. The WAR file deployment approach is mentioned as a tool that helps manage all those independent Rails instances.

Last but not least, Thomas Enebo, also hired by Sun to work full time on JRuby. During the presentation I mentioned that his blog was less technical than the other guys. Turns out this feeling was largely based on the activity on his blog before summer. Recently he posted some very interesting posts on JRuby and Ruby. Looks like I have some catching up to do :-)

The project

The official JRuby site is hosted on codehaus. While the official wiki is on Charles Nutter’s personal site for now.

JRuby on Rails

For 2 good tutorials that take you through the steps of trying out JRuby, go see on ADS’ blog:

Getting JRuby and Rails running. Deploying to Glassfish. Theirs posts are pretty detailed, but since things are moving so fast in the JRuby world, some of the details are a bit out of date already :-) In it they use Goldspike to generate their WAR archive, just like I did.
As I mentioned however, a post by Nick Sieger introduced me to warbler as another very promising replacement to Goldspike. In fact, as Nick describes warbler, I fully expect to try it and stick with it instead of Goldspike. Definitely worth a try.

Check out the Glassfish gem for a better integration of Glassfish in your Rails workflow. I haven’t tried it yet, so I can’t elaborate on it.

The 2 Java web servers I used are:

Jetty, the small, unintrusive yet very scalable web server. I likened it to WEBrick in that it can be used to try out our WAR file with absolutely no fuss. Which is not to say that it can’t be used in a production setting.

Glassfish, the full-featured server with the admin interface. This is the server that ended up being about as fast as the MRI/Mongrel combination, in the performance comparison.

If you’re interested in a JRuby/Mongrel setup, the one which ended up being the fastest in the comparison, check out this post*, mentioned by Nick Sieger, and this post by Ola Bini.

JRuby off Rails

For those that were insterested in the non Rails bits as well, I talked about an unintrusive plugin that allows you to create Swing gui apps, I mentioned (and stumbled over the name of) Profligacy. It’s been created by Zed Shaw, the creator of Mongrel.

I also talked about using JNA to reuse existing C extensions for Ruby, which do not have a JRuby equivalent yet.

I have to stop and go to work now

Hey, I wanted to make this a short post. Looks like I failed :-) If I forgot something of if you have questions, don’t hesitate to comment. I’ll get back to you and update this post, if necessary.

As for me, I might help Carl by checking out whether JRuby could be a good fit for Defensio. I’m looking forward to that, I think it’s a great real world project to dive into JRuby head on. Of course I’ll report back and contribute, based on my findings.

*Sorry, the JRuby/Mongrel link seems dead as I’m posting this. I leave the link in case the problem is temporary.

Tomorrow my presentation at Montreal on Rails is going to be made on my brand new Mac Mini!

In my last post I mentioned I was going to present on an ooooold Win XP laptop, but I decided to make the switch to the Mac world now. Well, one week ago. So fear not, I’m not going to subject our community to a 5 square meters XP/Powerpoint eye bleeding festival.

The Apple world had one little surprise for me, however. Immediately after I bought my Mini I started hearing about the Java 1.6 debacle on Leopard. What? I didn’t event know OS X didn’t come with the standard Java distribution! And here I am, about to make a presentation on JRuby on a Mac :-) Note to self: never switch to a completely new OS (to me) one week before a presentation again.

This won’t affect the presentation, however, since everything works perfectly fine on Java 1.5, preinstalled on Leopard. And isn’t it reasonable to expect Apple to release Java 1.6 shortly anyway? So for those I’ll convince to give JRuby a try, all hope is not lost. Moreover, for the Rails community I think the Java 1.6 story is pretty much a non-issue. Your test and production servers are running on Linux, right?

So with that out of the way, there’s only one bump left in the road. As our host Fred pointed out to me, my mini has only one video out. Doh! I didn’t find any VGA or DVI splitter, so I’ll deal with it. This will be a bit of a pain in the neck, but hopefully just for me :-) Let’s see how the live coding part goes!

* In my best Professor Farnsworth voice (from Futurama)

Tonight, as I tried to set myself up to work with JRuby on Windows*, I bumped into an annoying problem. When I tried to configure jirb, the JRuby version of IRB, it didn’t always seem to find the .irbrc config file in my home directory. I looked a bit on the ‘net and found nothing that helped me solve my problem. I’ve worked out a fix, you can follow along in this wonderful adventure.

So to configure irb, I create the .irbrc file in my home directory.

C:\Documents and Settings\Mat\.irbrc

Note: Windows will yell at you if you try to make a new file starting with a “.” in windows explorer. Just use any good text editor to create it directly.

In it I put some configs for auto-completion and auto indent, as suggested on the Tips and Tricks page on the Rails wiki:

IRB.conf[:AUTO_INDENT] = true
IRB.conf[:USE_READLINE] = true
require 'irb/completion'
puts "Yay! Completion's loaded!" #Uhh, that's my special debugging code :-)


Now I fire up a console and start the interactive Ruby shell with

C:\Documents and Settings\Mat>jirb
Yay! Completion's loaded!
irb(main):001:0>

Hmmm, it works. What am I yelling about? :-) Let’s say we fire it up in an actual project directory instead.

C:\projects\an_actual_project>jirb
irb(main):001:0>

Oops! My “special” debugging code isn’t executing now :-)

If you care to see what’s the code that hints at how to solve the problem, you can open

…\jruby-1.0.1\lib\ruby\1.8\irb\init.rb

Or check it out a snippet of IRB.rc_file_generators online.

As we can see, if the HOME environment variable is set, the method looks for .irbrc in the directory, otherwise it looks for it in the pwd (present working directory).

Nice! That explains why it works in my home and not elsewhere :-)

So now I just have to set a HOME variable and I’m all set:

Now after restarting my console (to have the updated environment variables):

C:\projects\an_actual_project>jirb
Yay! Completion's loaded!
irb(main):001:0>

Note that even though Windows variables are not case sensitive, you must name your HOME variable in capital letters. Ruby expects the ENV hash to contain the HOME variable in caps.

* Q: Why would I want to set this up on Windows, if I’ve got a perfectly good Linux setup at home?

A: I’ll be giving a talk about JRuby at the next Montreal on Rails, and since I haven’t really needed a laptop in at least two years, well, I only have 3 years old laptop. This old geezer can run equally old versions of Linux, but I wouldn’t like my audience to have to suffer the visually challenged distro I currently have on there.

I thought about implementing a small command-line tool just to give me the functionality I was missing, but it felt too patchy for my taste, this time. I had to use it on other test machines so I wanted to be able to just copy the script without having to update some executable utilities as well.

Sadly I could not use Ruby because my company doesn’t use it, and I don’t want to sneak it in the wrong way. And as I said it was to be used on another machine, so it was not like creating a personal script on my own machine. In fact it even had to be documented in a manual testing procedure.

So I decided that it was time to try Powershell. I’ve been hearing about it for quite a while now, and I heard it’s pretty powerful. Since it’s the future of Windows console/scripting anyway, I figured nobody sane at a Windows shop would yell at me for introducing this particular technology.

Now that I’ve been dabbling with it for the past few weeks, my opinion is starting to form and it goes a little bit like this. If you’re a Windows power user (especially an admin or a developer) and you don’t know Powershell, you’re at a disadvantage :-) The “Power” in Powershell is very well deserved. If you’re still flopping along using batch files, you’re using a weak, error-prone tool where a new, very powerful tool is available. It’s free and it’s by Microsoft, so there’s no reason it wouldn’t be well received in most Windows shops :-)

I decided to capture my understanding of Powershell in an introductory article. In it I explain some of the basics of the language and introduce you to all that’s necessary to get started with Powershell. In the reference section at the end I also mention other good sources of information if you want to learn more. Depending on the amount of fiddling around you do while following along, I guess it’s a 30 minutes to 2h read tops.

So without further ado, Discover Powershell (it’s an order!)

Tonight I was inspired and even went to the extent of expressing myself in video :-)

It involves jumping through a couple of hoops, and the book contains a couple of pages containing ads about other Sitepoint books. But nothing too bad, at first sight. And it’s still a free PDF book! So if you know anybody who’s curious about Rails and might want to try it out, you have until the beginning of December to point him/her to rails.sitepoint.com.

The first 2 were very interesting and filled with great people to meet. Tonight promises to be just as great, and as a bonus, we’re leaving behind the small McGill University classroom to be welcome instead by Fred and Ben from Standoutjobs. Can’t wait to see that either!

And great big thanks to Mat and Carl for organizing this monthly meetup!

I thought I’d write a list of the podcasts I listen to. This way I can just point people here when they’re curious. I’ll put this somewhere else soon, to update it when I discover new stuff, but now I’m just in brain-dump mode :-)

Those I listen to religiously

• The Linux Action Show! – Bryan and Chris entertain us with Linux news, the occasional interview, tips on development, system administration, desktop compositing, and So Much More!
• .Net Rocks – Carl and Richard meet the cream of the crop in the .Net world. Great podcast to learn about tons of tools and practices related to .Net development and other MS technologies surrounding it. There’s even good stuff for non .Net developers.
• Hanselminutes – Scott Hanselman’s the ultimate geek. He seems to follow everything in the world of development. He often talks about .Net development, but has a very open mind and covers a broad range of other technologies, including a bit of Ruby on Rails once in a while. He also covers lots of other technology-related topics. He pretty much always manages to keep the podcasts around a no-fuss 30 minutes, hence the name Hanselminutes.
• Security Now – The ultimate security podcast, with host Leo Laporte and security expert Steve Gibson. All episodes are worth going back to, since the formula is about the theory behind concepts of security as well as security tools, rather than being a news-centric podcast.
• Macbreak Weekly – Definitely a news-centric podcast, very entertaining crew that transmit their passion for the Mac (and the iPhone) very well. The discussion often revolves around digital media as well, with very interesting insights. Oh, and Merlin Mann is often one of the cohosts. Very funny guy :-)
• The Ruby on Rails podcast – The podcast extraordinaire if you’re interested in Ruby on Rails. Interviews with the big names in the Rails community. Lots of talk about different tools that can aid you in your quest to create the ultimate Rails app. The summer has unfortunately brought few new shows, but one came out recently. Let’s hope the pace picks up again, I miss the RoR podcast :-)
• FLOSS Weekly – Very promising podcast about Free, Libre and Open Source Software (not about dental floss). Some very big names have been interviewed on the show (Guido van Rossum, “Maddog” Hall, Jeremy Allison, Randall Schwartz…) The first host, Chris, has a very busy schedule and the pace has really slowed down for a while. But there has been interest by Mr Shwartz and Allison to host the show once in a while as well. I can’t wait to see that happen (the most recent show is hosted by Schwartz, actually).
• You Suck at Web Design – Short, funny podcast by Matthew D. Jordan about being a freelance web designer (and the journey to get there). Twisted humor and cringe-inducing stories await you here.

Those I listen to occasionally

• net@night – Ambermac, the ultimate web geek speaks with Leo about what’s going on the internet. Interviews with web entrepreneurs. Shows recorded live on talkshoe.com (can be good or not).
• Windows Weekly – Paul Thurrot speaks with Leo about what’s going on in the world of Windows and Microsoft in general. The subject may seem dry, but the hosts definitely make up for it. A plus for anyone working in IT with lots of Windows machines. Thurrot knows the Windows world inside out, but is very balanced: he uses (and loves) his Mac and uses Ubuntu as well, so you can count on his opinion to always be fair.
• This Week in Tech – News-centric podcast about the world of technology in general. Very interesting hosts, including once in a while Merlin Mann, as well as John Dvorak.org/blog (don’t ask :).
• Penny Arcade Podcast – Every once in a while, Gabe and Tycho take us through the creative process of creating the day’s comic strip.

</braindump>

Huh, well! I appear to be beating my writer’s block little by little :-)

Let me know if you’ve discovered something interesting here or if you’d like to share the podcasts you listen too once in a while.

When your hammer is C++, everything begins to look like a thumb.

This especially strikes a chord, since I’m currently working on something in C++ at work. Ugh!

I had a mini-epiphany recently. Altough I haven’t had the chance yet to program with a functional programming language, it’s a subject I’ve had my eye on for a while. My ear perks up when I hear about Erlang, Haskell, F# or Scala, 4 functional programming languages.

Recently I’ve been dipping in the intricacies of memory management, at work, in .Net. This was in the context of the Dispose pattern (I have a post in the making about the subject). Both subjects are not directly related, but I found a parallel that might help to explain lazy evaluation to programmers used to more traditional programming languages.

I won’t go in depth in the subject or define the concepts precisely here. I might however revisit what follows when I’ve had time to actually get more experience with an FP language. So here was my revelation:

When specifying a computation in a programming language, lazy evaluation is to the detailed, manual sequencing of operations what garbage collection and automatic memory management is to the detailed, manual management of memory.

In other words, with advanced virtual machines such as Java’s or .Net’s, by allowing the VM to manage the memory, we more or less get the advantage of having world-class computer scientists optimize the management of the memory we use. The programmer declares what he needs in term of memory, instead of specifying precisely how he needs it (stack vs heap, who’s responsible of freeing the memory, etc.) The amount of control over the precise performance is lessened to a certain extent, but the resulting performance gains can often be quite stunning. For example, I read recently that Java’s allocation of memory on the heap requires around 10 processor operations instead of 60 to 100, for the best C++ compilers. See Brian Goetz’ excellent article on IBM’s website.

In the same manner, with a functional programming language’s lazy evaluation, optimizations can be made without the programmer needing to state them explicitly. The interpreter can decide when to execute the code, namely if and when its outcome is needed, and not before that.

Since the precise order of execution is not guaranteed, the bits of computing specified by the programmer in an FP can be optimized in other ways, too. For example, if the VM sees that a given instructions in the code is not dependent on the outcome of the previous instruction, it can execute them both simultaneously, if there are more than one cores available. It’s managed automagically by the interpreter, no threading headache necessary. Conversely, the programmer can specify the ordering of instructions only when necessary (e.g. when interacting with an outside system).

This is why FPs are often mentioned in the current debate surrounding the looming concurrency challenge. This kind of optimization is also being done by Java and .Net’s VMs, but to a much lesser extent, because of deep differences between imperative (Java, C++, C#) and functional programming languages.