Aaron Kromer's Blog

In the past, testing JSON APIs tended to be a bit painful for me. Most of this pain revolved around setting expectations on the response body.

If you treat the response as a raw string, attempting to use regular expressions ends up being an exercise in how you handle frustration. While a JSON body is a string, it has structure. Using regular expressions for parsing them is akin to using a hammer on a screw. It’ll get the job done, but it’s the wrong tool for the job.

Ruby gives us JSON.parse. Which will convert a valid JSON string into a more familiar object structure. Now comes the “fun” part of actually verifying that structure:

• Sometimes you only care about part of the response
• Sometimes you care about validating the entire response
• Sometimes the response is very complicated consisting of many smaller, more logically meaningful, structures
• Sometimes you only care about the general structure (e.g. this value must be a number, that value must be either an empty array or an array of strings, etc.)

It is possible to do all of these validations out of the box. In my experience, writing them tended to be tedious. Often the resulting code left something to be desired in terms of readability. This was especially true when validating the general response structure.

I like to follow the “one expectation per spec” guideline. However, this lead to writing many small specs. Normally, this is perfectly fine and something I advocate you do. However, in terms of a JSON response, it means I need to have more discipline to keep everything explicitly organized.

Naturally in the Ruby community, many gems have sprouted up to help with this problem set. I’ve had a bit of success with some of those gems in the past. However, with the release of RSpec 3, several new features have eliminated my need for these JSON gems.

Expectations on a JSON response is a great fit for composing matchers. When I need to logically group checking several options, the compound matchers are the perfect tool.

Often people don’t realize that the matcher messages (i.e. exist, be, eq, include, etc) are just factories helpers (see endnotes). They are just helper methods which create the matcher object for you. That means, we can easily write our own using our app’s domain language.

Let’s jump right into an example!

These examples are assuming a JSON structure like one of the ones listed on the jsonapi.org site. Though I am assuming integer value are represented as numbers and not strings, since that is valid JSON and more meaningful:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

require 'rails_helper'
# Use common JSON helpers such as: `json_response`, `be_an_empty`, `all_match`
require 'support/json_api_helpers'

RSpec.describe "/api/kits", type: :request do
  def be_kits_root_json
    be_kits_json.and(
      include(
        'meta' => {
          'first'   => anything,
          'last'    => anything,
          'current' => anything,
        }
      )
    )
  end

  def be_kits_json
    include(
      'version' => '1.0',
      'links'   => {
        'kits.beacons'       => "#{beacons_url}/{kits.beacons}",
        'kits.overlays'      => "#{overlays_url}/{kits.overlays}",
        'beacons.attributes' => "#{beacon_attributes_url}/{beacons.attributes}",
      },
      'kits'    => be_an_empty(Array).or(
        all_match(
          'id'        => Fixnum,
          'name'      => be_nil.or(be_a String),
          'api_token' => String,
          'account'   => be_nil.or(
            match(
              'id'   => Fixnum,
              'name' => be_nil.or(be_a String),
            )
          ),
          'links'     => {
            'self'     => /\A#{kits_url}\/\d+\z/,
            'beacons'  => be_an_empty(Array).or(all be_a Fixnum),
            'overlays' => be_an_empty(Array).or(all be_a Fixnum),
          },
        ),
      ),
    )
  end

  def include_linked_resources(*resources)
    resource_maps = resources.each_with_object({}) { |resource, mappings|
      mappings.store(resource.to_s, be_an(Array))
    }
    include('linked' => resource_maps)
  end

  context "a basic user", "with a kit having no beacons or maps" do
    # Setup world state

    describe "requesting the kits root" do
      it "conforms to the expected JSON structure" do
        get kits_path, *options
        expect(json_response).to be_kits_root_json
      end

      # More specific specs
    end

    describe "requesting a kit" do
      it "conforms to the expected JSON structure" do
        get kit_path(kit), *options
        expect(json_response).to be_kits_json
      end

      # More specific specs
    end
  end

  # More state specs

  context "a developer user", "sending request with parameter 'include'" do
    # Setup world state

    describe "requesting the kits root" do
      it "conforms to the expected JSON structure with included resources" do
        get kits_path(include: "beacons,beacon_attributes"), *options
        expect(json_response).to be_kits_root_json.and(
          include_linked_resources(:beacons, :beacon_attributes)
        )
      end
    end

    describe "requesting a beacon" do
      it "conforms to the expected JSON structure with included resources" do
        get kit_path(kit, include: "beacons,beacon_attributes"), *options
        expect(json_response).to be_kits_json.and(
          include_linked_resources(:beacons, :beacon_attributes)
        )
      end
    end
  end
end

The possibilities are fairly endless. We could improve this further by allowing the factories to take model instances or attribute hashes. We can use those to check specific content when available:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

def account_resource(account = nil, allow_nil: false)
  return nil unless account || !allow_nil
  if account
    {
      'id'   => account.id,
      'name' => account.name
    }
  else
    {
      'id'   => Fixnum,
      'name' => be_nil.or(be_a String),
    }
  end
end

def kit_resource(kit = nil, allow_nil: false)
  return nil unless kit || !allow_nil
  if kit
    {
      'id'        => kit.id,
      'name'      => kit.name,
      'api_token' => kit.api_token,
      'account'   => account_resource(kit.account, allow_nil: true),
    }
  else
    {
      'id'        => Fixnum,
      'name'      => be_nil.or(be_a String),
      'api_token' => String,
      'account'   => be_nil.or(match account_resource),
    }
  end
end

context "a basic user", "with a kit having no beacons or maps" do
  # Setup world state

  describe "requesting the kits root" do
    it "conforms to the expected JSON structure" do
      get kits_path, *options
      expect(json_response).to be_kits_root_json
    end

    it "has only the expected kit" do
      get kits_path, *options
      expect(json_response).to include 'kits' => [kit_resource(basic_users_kit)]
    end
  end

  describe "requesting a beacon" do
    it "conforms to the expected JSON structure" do
      get kit_path(kit), *options
      expect(json_response).to be_kits_json(basic_users_kit)
    end
  end
end

Happy RSpec’ing!

Updates 2014-09-30

Thanks to everyone who provided feedback on this post. I’ve take it all into consideration and made the following changes:

• The first code sample now shows the full spec file structure. This hopefully makes the important distinction that the helper methods are not being defined on main.

• The line require 'support/json_api_helpers' isn’t loading another library. Instead it is loading an extracted set of shared helper methods common to nearly all JSON API request specs for this project. These have been extracted to a module to keep them off of main and placed in spec/support.

  This follows the new guidance that specs should only load those files which they need. It also makes it easier for your future self and your co-workers to come back to the file later and try to find where things are defined.

  I’m including the file below for completeness:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# spec/support/json_api_helpers.rb
module MyApp
  module RSpec
    module JsonApiHelpers

      def json_response
        JSON.parse response.body
      end

      def be_an_empty(klass)
        be_a(klass).and(be_empty)
      end

      def all_match(*args)
        all match(*args)
      end

    end
  end
end

RSpec.configure do |c|
  c.include MyApp::RSpec::JsonApiHelpers, type: :request
end

• It was pointed out that all match will handle the empty Array case. It is possible to amend the above be_an_empty(Array).or(all_match()) to instead read: be_an(Array).and(all_match()).

• This is a helper method. My reference to it as a factory was more explicitly attempting to describe it as a helper method which instantiates another object. In Rails, people often know of “factories” from the “factory vs fixture” debate. Often those factories are relatively simple wrappers around constructors. After researching this a little more it seems in the larger programming world “factory” is not the proper term. Perhaps “creator” is. I will move to calling them helpers in the future.

The past week was spent dealing with an infuriating data crunching task. Chris and I decided to celebrate by killing our poorly written Ruby cli scripts with fire. Printing the scripts and going out back to set them on fire would be more dramatic. Though it probably would be appreciated by the other businesses which we share space with.

I took a breather and looked into zsh functions to accomplish this. Here’s what I came up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

rageflip() { echo " ┻━┻ ︵ヽ(\`Д´)ﾉ︵ ┻━┻  $*" }

has_utility() { hash $1 2>/dev/null }

# Wrapper around `rm -rf` expressing callers loathing and disdain for the
# provided files.
#
# Inspiration was Homebrew's output:
# 🍺  /usr/local/Cellar/tmux/1.9: 15 files, 628K, built in 25 seconds
#
# For additional file stats, specific to code files, consider installing the
# Count Lines of Code (cloc) tool: http://cloc.sourceforge.net/
#
#   brew install cloc
#   sudo apt-get install cloc
#
ignite() {
  if (( $# == 0 )) then echo "USAGE: ignite file [file] ..."; return; fi
  local total_lines
  local human_size
  local lc_blank
  local lc_comment
  local lc_code
  echo "BURN IT ALL!!! $(rageflip)"
  for i do
    # If the file is empty we have 0 lines
    human_size=$(ls -lh $i | awk '{ print $5 }')
    total_lines=${$(sed -n '$=' $i):-0}
    stats="$total_lines lines"

    if has_utility cloc; then
      # Setup some local variables regarding file stats
      lc_blank=
      lc_comment=
      lc_code=
      eval $(
        cloc $i --quiet | tail -2 | head -1 |
        awk '{ print "lc_blank="$3, "lc_comment="$4, "lc_code="$5 }'
      )
      if [ ! -z $lc_blank ]; then
        stats="$lc_code loc, $lc_comment comments, $lc_blank whitespace lines"
      fi
    fi

    rm -rf $i
    echo "🔥  $i: $stats, $human_size"
  done
}

Now we can sit back and watch it all burn:

1
2
3
4
5
6

$ ignite script/crunch-*
BURN IT ALL!!!  ┻━┻ ︵ヽ(`Д´)ﾉ︵ ┻━┻
🔥  script/crunch-dump-sessions: 5 loc, 5 comments, 2 whitespace lines, 330B
🔥  script/crunch-normalize: 47 loc, 14 comments, 11 whitespace lines, 2.2K
🔥  script/crunch-import-csv: 101 loc, 2 comments, 20 whitespace lines, 2.7K
🔥  script/crunch-timestamp: 203 loc, 44 comments, 38 whitespace lines, 8.4K

Though at this point, it might as well be a full script file. If I do that, I might as well just write it as a Ruby cli…

Check out the full Testing Scripts with Load series: part 1, part 2, part 3

Scope Smashing

Take the following script and spec:

1
2
3
4
5

#!/usr/bin/env ruby

def current_user
  'Bob'
end

1
2
3
4
5
6
7
8
9
10
11
12
13

require 'spec_helper'

def current_user
  'Alice'
end

describe 'Running the server locally' do
  it 'logs that it is running' do
    load 'script/current-user-smash'

    expect(current_user).to eq 'Alice'
  end
end

Alas, the spec fails with:

1
2

expected: "Alice"
     got: "Bob"

This is due to how load works:

If the optional wrap parameter is true, the loaded script will be executed under an anonymous module, protecting the calling program’s global namespace. In no circumstance will any local variables in the loaded file be propagated to the loading environment.

While it is easy to spot the issue this time, that’s not normally the case. Say if the additional method is define by a gem or in supporting file. Or if you are testing multiple scripts that each define the same top-level methods. These conditions will result in very strange and difficult to debug failures. Of course, it’s always a good idea to not define top-level methods to begin with.

Instead always pass the additional wrap parameter. Here I’ve named it as a descriptive inline variable to reveal it’s intent:

1
2
3
4
5

it 'logs that it is running' do
  load 'script/current-user-smash', _in_sandbox = true

  expect(current_user).to eq 'Alice'
end

While at RetroRuby 2014 a very good question was asked in the newbie track. The following code sample had just been shown on some basic awesomeness of Ruby:

1
2

"Words" * 2
# => "WordsWords"

One of the participants then asked why that worked but the inverse didn’t:

1
2

2 * "Words"
# TypeError: String can't be coerced into Fixnum

Oops what just happened?

In Ruby, virtually everything is accomplished by sending a message to another object. Above we would say:

Send the message * to "Words" with parameter 2

Send the message * to 2 with parameter "Words"

“Sending a message” is the Java equivalent of calling a method. Another way to write the above is:

1
2

"Words".*(2)
# => "WordsWords"

Here we used the normal “method” calling syntax: obj.method(args)

You’ll also probably see the following:

1
2

"Words".send(:*, 2)
# => "WordsWords"

This time we explicitly sent the message: obj.send(message, args)

With send Ruby doesn’t check if the message you passed was supposed to be public or private. Generally, what you wanted to do was dynamically send the message while still making sure to respect the public API of the object. To do this you should use public_send instead: obj.public_send(message, args).

So back to the original issue. Both String and Fixnum respond to the message *.

• String#*
• Fixnum#*

However, String’s implementation knows what to do when the argument is a Fixnum. When we reverse it, the Fixnum implementation doesn’t understand what to do with a String argument.

How to fix this?

Well you probably shouldn’t. But just for fun we’ll use Ruby’s open class behavior to monkey patch Fixnum’s * implementation.

1
2
3
4
5
6
7
8
9
10
11

class Fixnum # We just re-opened the class

  def *(arg) # We're redefining the * message - this destroys the
             # previous implementation!!!
    arg * self
  end

end

2 * "Words"
# => WordsWords

It worked!! Before you go doing this to everything, be aware we’ve now lost the ability to do normal multiplication. Additionally, we’ll overflow the stack trying to multiply two Fixnums.

1
2

2 * 2
# SystemStackError: stack level too deep

Wrap Up

In Ruby, most things are messages sent from one object to another. Some times the language gives a little extra syntactic sugar to make sending the message more readable.

Additionally, open classes and monkey patching have their uses. Just be aware that with great power comes great responsibility. Always stop to ask yourself if this is really a good idea; and try to never change existing behavior if you can.

Level Up!

So what if we just wanted to extend the existing functionality. There are a lot of ways to do this and covering each, along with the pros and cons is very out of scope for this post. I’m just going to demonstrate one way to illustrate how it could be done.

You’ll need a new irb or Pry session so we get the original implementation for Fixnum back.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class Fixnum # We just re-opened the class

  # We need to store a reference to the original implementation One way to do
  # this is to create an alias for the original implementation so we can call
  # it later. Be sure to pick a descriptive name so that it isn't accidentally
  # overwritten by something else.
  alias :star_without_string_support :*

  def *(arg) # Redefine the message destroying the previous implementation!!!
    if arg.is_a? String
      arg * self
    else
      star_without_string_support arg  # use the original implementation
    end
  end

end

Recently at work, we’ve been spinning up a bunch of demo apps on Heroku. This has worked out well for us so far. However, some of these apps require background data crunching. This hasn’t been a problem as we just add a Rake task, or two, and set the Heroku scheduler to take care of things.

Then comes the day when the sales team says:

“Oh hey. Can we re-run all the numbers but with different configurations?”

Sure, that’s not a problem you think:

1

$ heroku run rake redo_all_the_things[config.yml]

However, there’s a few issues with this:

1. The config.yml needs to be on Heroku
2. This counts against your active dyno time

See the thing is, Heroku doesn’t let you create files that aren’t already in your VC system from outside the app. It’s now sorta possible on Cedar, but not really for this purpose.

Additionally, if these tasks turn out to be long running, that’ll quickly eat up the overhead on the ‘free’ dyno. For demo apps of non-paying customers that’s not always an option.

Enter the live environment.

Just add a new live environment to the Rails application. This allows us to run Rake locally, but against the production database. Additionally, allowing for custom configuration files to just be stored on our local system. And we don’t spin up a new dyno on Heroku. Win, win!

To set this up:

• Update the Gemfile

1
2
3
4
5

# Gemfile

# We use dotenv-rails to help manage environment configs locally.
# Add our new environment:
gem 'dotenv-rails', groups: [:development, :test, :live]

• Add the live specific sample file: .env.live-example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# .env.live-example

# Rename this file to `.env.live` and it will be automatically sourced by
# rails. Uncomment the settings for them to be picked up.
#
# Grab the settings from the output of:
#
#  $ heroku config
#
# Running against the Heroku production DB directly
# BE CAREFUL!! YOU HAVE BEEN WARNED!!
#DATABASE_NAME=WARNING
#DATABASE_HOST=DANGER
#DATABASE_PORT=WILL
#DATABASE_USER=ROBINSON
#DATABASE_PASSWORD=SERIOUSLY

• Update the database.yml file

1
2
3
4
5
6
7
8
9
10

# config/database.yml

live:
  adapter: postgresql
  encoding: unicode
  database: <%= ENV['DATABASE_NAME'] %>
  host: <%= ENV['DATABASE_HOST'] %>
  port: <%= ENV['DATABASE_PORT'] %>
  username: <%= ENV['DATABASE_USER'] %>
  password: <%= ENV['DATABASE_PASSWORD'] %>

• Copy config/environments/production.rb to config/environments/live.rb. If you wish to symlink it instead be wary of how git treats that.

Now running our tasks is as simple as:

1

$ RAILS_ENV=live rake redo_all_the_things[config-fun-edition.yml]