Integration testing for Rails APIs part 3: Automating tedious things

Validating JSON responses

Manually parsing and checking JSON using JSON.parse is a pain: you need to be aware of the structure in each spec.

Wouldn’t it be nice if we could define the structure of an object in a single file and then validate responses using that file? With JSON schema we can!

JSON schema is a format for describing JSON documents structurally. The latest draft is version 4, which I’ll be using in these examples. Even though the spec is still in draft form, v4 is full-featured and good for most use cases.

Describing a user resource with JSON schema

Now, don’t start thinking of XML here: JSON schema is actually easy to read and write unlike XSD.

We’ll use the GET /users/:id endpoint from the previous part as an example. The returned user resource has three attributes and looks like this:

{
  "id": 1,
  "email": "[email protected]",
  "role": "admin"
}

The role attribute is only shown if the current user has the admin role.

Let’s think about the data types for a second. Our resource document’s root is an object with the following properties:

• id
  • integer
  • required
• email
  • string, looks like an email address
  • required
• role
  • string, either “user” or “admin”
  • optional

The schema for the user resource would then look like this:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "required": [ "id", "email" ],
  "additionalProperties": false,
  "properties": {
    "id":    { "type": "integer" },
    "email": { "type": "email" },
    "role":  { "enum": [ "user", "admin" ] }
  }
}

We’ll store our schemas in the /spec/support/schemas folder so that we can easily use them for testing.

Schema root

As you can see, the schema is a plain ol’ JSON file. The $schema property lets the validator know which version of JSON schema to use. It’s optional: in case it’s missing, the validator will use the most recent version it supports.

Since we’re in the root element of our schema, the type attribute defines the root element of the described document. The root element of our user resource is an object.

Properties

An object’s properties can be described with the same syntax as all elements. They also have a type property, we’re using the integer and email types.

The enum notation is special: it defines an array of values that the property can have. It’s not limited to strings, you can specify all kinds of values, including null.

Required properties are listed in required. Note that extra properties that haven’t been defined in the schema are allowed by default: to disallow them, set additionalProperties to false.

Advanced goodies

The schema format allows for advanced features such as referencing other schemas and defining your own data types.

I’m not going to describe them here, but to learn how to use them, check out the documentation and examples.

Writing a schema matcher

Using the json-schema gem and a custom RSpec matcher, we can now validate responses against that schema.

Below is the matcher that I use.

Note that you don’t have to copy it into your project. I’ve packaged it into a gem called rspec_json_schema_matcher that you can install.

# Validates JSON responses against a schema
RSpec::Matchers.define :match_schema do |schema|
  match do |body|
    schema_directory = "#{Dir.pwd}/spec/support/schemas"
    schema_path = "#{schema_directory}/#{schema}.json"
    opts = { validate_schema: true }

    begin
      @result = JSON::Validator.fully_validate_json(schema_path, body, opts)
    rescue JSON::Schema::ValidationError => e
      @result = "Schema '#{schema}' did not pass validation:\n\n- #{e.message}"
    end

    expect(@result).to eq []
  end

  failure_message do |body|
    "#{messages}\n\n#{body}"
  end

  description do
    "match the JSON schema '#{schema}'"
  end

private

  def messages
    if @result.respond_to?(:join)
      @result.join("\n")
    else
      @result
    end
  end
end

Assuming that you have a schema called user.json in /spec/support/schemas, you can validate a response body against it like this:

subject { response }

context 'when the user exists' do
  its(:status) { should eq 200 }
  its(:body)   { should match_schema 'user' }
end

What JSON schema doesn’t do

Your API might be returning valid data, but does it make sense? Remember, the schema validates only the structure, not the content.

When testing your API, you should have some tests in place to ensure that the data actually makes sense:

• Are you returning the right record?
• Does the record change when you update it?

To help test these, it’s useful to have some utilities installed for working with JSON. I’ve found the json_spec gem to be good for most of my use cases.

Shared examples for less duplication

If you output helpful JSON summaries of errors, you’ll soon start to notice that you’re writing the same code again and again when testing for them.

RSpec lets you write shared examples to eliminate duplicated code. I’ve found them to be useful when checking for responses that are pretty much the same everywhere. Errors are prime candidates here.

If you’ve written schemas for your error summaries, validating them becomes suddenly quite easy:

RSpec.shared_examples 'no content' do
  its(:status) { should eq 204 }
  its(:body)   { should be_empty }
end

RSpec.shared_examples 'unauthorized' do
  its(:status) { should eq 401 }
  its(:body)   { should match_schema 'errors/unauthorized' }
end

RSpec.shared_examples 'forbidden' do
  its(:status) { should eq 403 }
  its(:body)   { should match_schema 'errors/forbidden' }
end

RSpec.shared_examples 'not found' do
  its(:status) { should eq 404 }
  its(:body)   { should match_schema 'errors/not_found' }
end

RSpec.shared_examples 'unprocessable entity' do
  its(:status) { should eq 422 }
  its(:body)   { should match_schema 'errors/unprocessable_entity' }
end

Usually I have a bunch of shared examples under /spec/support/shared_examples that I require in spec_helper.rb.

Now testing an error response becomes a one-liner in the spec:

subject { response }

context 'when the user does not exist' do
  it_behaves_like 'not found'
end

What’s next

This concludes my series on API testing tips. I hope you’ve learned something useful that you can use in your projects!

Let me know in the comments below if you have any questions or feedback.

Even though this series ends here, I’ve got all sorts of goodies lined up. Don’t forget to subscribe to the newsletter below so that you’ll be the first one to know about them!

Related posts

• Integration testing for Rails APIs part 2: Writing less code
• Integration testing for Rails APIs part 1: Structuring specs
• Farewell, Panel Tabs
• A better way for dynamically choosing a readable text colour with Sass