Aaron Kromer's Blog

Check out the full Testing Scripts with Load series: part 1, part 2, part 3

Previously it was shown how to use load to test scripts. As with all techniques, there are some drawbacks to using load.

Hidden Require Dependency

Take script:

1
2
3
4
5
6
7
8

#!/usr/bin/env ruby
$:.unshift File.join File.dirname(__FILE__), "..", "lib"

require 'local_server'

logger = Logger.new($stderr)

LocalServer.new(logger).run

and passing spec:

1
2
3
4
5
6
7
8
9
10
11
12
13

require 'spec_helper'
require 'logger'
require 'stringio'

describe 'Running the server locally' do
  it 'logs that it is running' do
    io = StringIO.new
    allow(Logger).to receive(:new).and_return(Logger.new io)

    expect{ load 'script/local-server' }.to change{ io.string }
      .to include 'SERVER: starting up'
  end
end

However, when the script is run standalone, it errors with:

1

uninitialized constant Logger (NameError)

Be aware that since load happens in the current spec context, a missing require may not be noticed if it is required by the spec.

Whenever possible have at least one test that shells out as a sanity check.

Check out the full Monitoring Messages in Rspec series: part 1, part 2, part 3

Continuing the series ( part 1, part 2 ) on matching messages in RSpec, the next logical step is custom argument matchers.

1

expect(mechanic).to receive(:fix).with something_broken

Using the RSpec matcher DSL this could simply look like:

1
2
3
4
5

RSpec::Matchers.define :something_broken do
  match do |thing|
    thing.broken?
  end
end

That’s all there is to it. Now it can be used as both a regular matcher and as an argument matcher.

If the matcher needs to be created from scratch, a matches? method must be defined instead:

1
2
3
4
5
6
7
8
9

class SomethingBroken
  def matches?(target)
    target.broken?
  end
end

def something_broken
  SomethingBroken.new
end

This works just fine as a normal matcher, however, when used as an argument matcher, it will always fail. The reason is that argument matchers are invoked with the == operator, which by default, verifies if the objects are the same object.

Attempting to use a normal matcher with the change expectation also oddly fails, due to change invoking the === message, not matches?. Since the default === behavior is ==, the existing argument matchers currently work with it.

There is active talk / changes happening to standardize the matchers to ===. This will allow for a more consistent and composable interface. It also has the added benefit of allowing the matchers to be used in more complex conditionals using case statements.

To fix the class based matcher simply add the necessary alias(es):

1
2
3
4
5
6
7

class SomethingBroken
  def matches?(target)
    target.broken?
  end
  alias_method :==, :matches?
  alias_method :===, :matches?  # Not technically necessary due to default ==
end

Note that with such a simple matcher, there is no reason it cannot be created as a simple composed method using an existing matcher:

1
2
3

def something_broken
  be_broken   # Note: Not all built in matchers have == aliases yet
end

Check out the full Monitoring Messages in Rspec series: part 1, part 2, part 3

Last time it was demonstrated how it is possible to monitor only a desired message expectations. While that technique is useful for a vast majority of use cases, sometimes you need a bit more complexity.

Say for example, the desire is to verify the state of the provided parameter. As in this very contrived example (Update: 2013-07-28 Paired down example to only show test and not implementation):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

describe Factory do
  context 'making sure current stock is ready' do
    it 'only fixes broken widgets' do
      # Setup code

      states = []
      expect(mechanic).to receive(:fix) do |widget|
        states << widget.broken?
      end

      expect{ factory.perform_maintenance }.to change{states}.to [true]
    end
  end
end

This leverages the stub with substitute implementation provided by RSpec mocks. It can also be used with the allow syntax.

While the above version has it’s uses, it tends to hide some of the intent in the closure manipulation. A slightly more expressive method is just to add the state expectation in the block:

1
2
3

expect(mechanic).to receive(:fix) do |widget|
  expect(widget).to be_broken
end

This last technique is great for setting up generic message stubs which require that something with specific state is provided. By adding it to a generic allow, it ensures when the contract is broken anywhere in the code under test, the test will properly fail. (Update 2013-07-30: seems there is an issue with this technique when combined with and_call_original)

Check out the full Monitoring Messages in Rspec series: part 1, part 2, part 3

A common question I see asked in the #rspec IRC channel is:

How do I verify only a specific message is received?

The context of this question is usually a method that sends the same message multiple time, but with different arguments.

Take the following contrived example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

def do_stuff(thing)
  thing.do :having_fun
  thing.do :drink_coffee
  thing.do :code
end

describe 'Coffee time!' do
  it 'drinks the coffee' do
    thing = double('Thing')

    expect(thing).to receive(:do).with(:drink_coffee)

    do_stuff thing
  end
end

Alas, it fails:

1
2
3

Double "Thing" received :do with unexpected arguments
         expected: (:drink_coffee)
              got: (:having_fun)

The solution is to add a generic stub (a catchall) for the desired message:

1
2
3
4
5
6
7
8
9
10

describe 'Coffee time!' do
  it 'drinks the coffee' do
    thing = double('Thing')

    allow(thing).to receive(:do)
    expect(thing).to receive(:do).with(:drink_coffee)

    do_stuff thing
  end
end

There is another solution which can also be used: as_null_object.

The issue with as_null_object is that it will happily hum along even for invalid and unexpected messages. The double stub pattern above is explicit in the test, matching the catchall only with the message expectation.

1
2
3
4
5
6
7
8
9

describe 'Coffee time!' do
  it 'drinks the coffee' do
    thing = double('Thing').as_null_object

    expect(thing).to receive(:do).with(:drink_coffee)

    do_stuff thing
  end
end

Stay tuned for RSpec3 when as_null_object doubles will be smart enough to only stub matching messages. Or check out one of the many plugins: rspec-fire, bogus, and spy.

Check out the full Testing Scripts with Load series: part 1, part 2, part 3

A standard method for testing Ruby scripts is to shell out to the script using Kernel#` or Kernel#system. Both of these follow a fork and exec pattern to run the command.

However, there are instances where it is necessary to mock/stub something in the script. The above technique fails in this situation due to exec replacing the current process; the stubs simply cease to exist.

The solution: use load

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Script
#!/usr/bin/env ruby
$:.unshift File.join File.dirname(__FILE__), "..", "lib"

require 'logger'
require 'monitor'
require 'pathname'
require 'temperature_api'
require 'tracker'

logger    = Logger.new($stderr)
file_path = Pathname.new(ARGV[0])
tracker   = Tracker.new(backup: file_path)
api       = TemperatureApi.new('api.server.com')

Monitor.new(tracker, api, logger).run

1
2
3
4
5
6
7
8
9
10
11
12
13

# Spec
describe 'Monitoring cpu heat' do

  it 'uploads the temperature to the server' do
    stub_const('ARGV', ['monitor-integration.bak'])
    stub_request(:put, 'https://api.server.com/temp').to_return(status: 200)

    expect(load 'script/monitor')
      .to have_requested(:put, 'https://api.server.com/temp')
      .with(body: /\A{"temp":\d+}\z/)
  end

end

In conclusion, no more excuses for not have integration tests for your scripts.