Contract Testing with Pact
As we migrate our systems to a microservices-oriented architecture at DoorDash, we have taken care to balance the excitement around anticipated benefits (development velocity, deployment independence, etc.) with a more measured assessment of our preparedness to work with the complexities introduced by more distributed systems.
One of the most common pain points with microservices — or, for that matter, any distributed system, finer or coarser-grained — is ensuring that service deployments maintain backwards compatibility and that the whole ecosystem plays well together in all environments, not just production.
An integration environment where teams can bring microservices together to ensure compatibility is critical to both development velocity and system correctness; but as the number of services grows, the coordination required to maintain this environment can become a heavy burden for development and infrastructure/operations teams.
At DoorDash, we looked towards applying consumer-driven contract testingas a technique to enforce compatibility testing. Conveniently, a mature and well-supported contract testing framework called Pact exists for enabling contract testing between REST services and consumers of services.
For our initial use case, we chose to adopt Pact to enforce the contracts between our mobile apps and the mobile backend-for-frontend (BFF).
Development Workflow with Pact
Pact works by connecting a set of unit tests. One set of tests for the service consumer and one set of tests for the service provider. The consumer tests contain specifications about an HTTP request and the shape of the response. When the tests are run, the Pact framework turns the specifications into a contract that is uploaded to a repository (known as a broker). The broker is a web service that contains a collection of contracts and associations between consumers and producers. With that in mind, the overall flow looks like this:
On the consumer:
- Write unit tests specifying the request and the response shape
- Run the tests, which, if successful, generate a Pact file
- Upload the Pact file to the broker
On the producer:
- Download the Pact files that involve the producer
- Run the tests, which, if successful, generate verification results
- Upload the verification results to the broker
In our case, the consumers are the iOS and Android mobile apps. Next, we’ll see how to specify the consumer API contract.
iOS App: Pact on Swift
Integrating Pact testing with an iOS application is pretty straightforward, consisting of a few simple steps:
- Install the Pact mock service
- Install the PactConsumerSwift library
- Configure Xcode
- Write your test
- Upload the verification results to your Pact Broker
Install the Pact Mock Service
The Pact mock service spins up a local server that will generate Pacts when your tests are run. Your tests will provide a series of expectations (fields, http status, etc.) for an endpoint (for example, /v1/auth) that are compiled into a Pact file.
To install the Pact mock service, simply run the following:
Install the PactConsumerSwift Library
The PactConsumerSwift library is used by your application to conveniently interact with the mock service to generate pact files. It provides objects and methods that can be used to easily define expectations and pass these expectations to the mock server.
To incorporate the service into your project, simply add the following line to your Podfile:
In order to have the local pact server spin up when your tests run, we’ll add pre-actions and post-actions to your project target.
Select your target and click Edit Scheme, select the Test phase, and add a new Run Script Action for both pre-actions and post-actions. It should look something like this:
For pre-actions, enter the following and save:
This will spin up the mock service on port 1234.
For post-actions, enter the following and save:
This simply stops the mock service.
Write Your Tests
A sample test is as follows:
This sets up the services that will be making the network calls. The MockService, provided by PactConsumerService, is initialized with provider and consumer keys — make sure these match the ones you previously used to set up your Pact broker.
The BootstrapClient is the network layer you are using in your application. Configure it with the base url provided by the MockService (which will point network calls to your local server).
We use the MockService to start building our expectations. These are customizable and you can add or remove expectations that fit your particular use case. We can specify a response body that we expect, headers, HTTP status, and other related fields.
Make sure the request parameters are configured properly for the request that you want to test as these will be used by the broker to reconcile your Pact with the backend results.
This part of the test runs your network call against the local server (remember that earlier we initialized an instance of our network service with the base url provided by the MockService). We run it within a closure passed into the runfunction on MockService so that the MockService can perform any configuration that it needs before running the actual call.
And that’s it!
Upload the Verification Results to Your Pact Broker
If you run your tests and they complete successfully, you’ll see that a new Pact file has been generated for you in your /tmp/pacts folder at the root of your project. Now, all we need to do is to upload this to your Pact broker. We’ll do it manually with curl but this can be added to the post-actions of your test phase as well.
And that’s it! If the upload is successful, you’ll see the new Pact file in your Pact broker. Next, we’ll look at a similar example from the perspective of the Android client.
Android App: Pact on Kotlin
Add the Pact gradle plugin:
Add a task to let Pact know how to publish to the broker:
Add these 2 test dependencies:
Define a Pact test which describe the interaction between the consumer and the provider. First, we define a Pact rule:
Create a test function to define the interaction:
Finally, we can define a pact verification test for this pact:
Now that the consumer contracts have been defined, we can start to write tests to ensure the contracts are fulfilled by the mobile BFF.
On the Mobile BFF: Pact in Kotlin
Once you’ve defined the contract via the consumer tests, you can move on to verifying the contract. In order to verify the contract, we need a way to start the application and send it the HTTP requests that are part of the contract. In our architecture, the provider is the mobile BFF.
There are two main approaches to verification:
- Run the mobile BFF via the command line
- Run the mobile BFF via an integration test
Pact provides tooling to accomplish both approaches. Since the mobile BFF makes requests to downstream services, we chose to verify using an integration test, since it would be easier to mock the outgoing HTTP requests. We also considered verifying the contract via the command line, but that would have necessitated building in a “test” mode to the mobile BFF and we thought that would be too much effort compared to simply using an integration test.
To set up Pact for the provider, start by configuring build.gradle:
This block of configuration sets up the system properties that Pact needs to find and download the necessary contracts from the broker. Since the contracts come from the broker, Pact will automatically try and upload the verification results when the tests are run. This should only happen on CI, otherwise, running the tests on your local workstation will overwrite the values stored in the broker.
Next, add the required dependencies:
Then, we can start writing a test to verify the contract. Start by creating a JUnit test (we’re using JUnit 5 in this example) and add the necessary annotations to indicate that it is a Pact test.
The @PactBroker annotation indicates that the test should fetch the contract from the broker using the system properties defined in build.gradle. @Provider tells Pact which provider this test is verifying. The last two annotations indicate that this is a Spring Boot integration test and instruct Spring to start the application on a random port. The selected port will be injected into the field annotated with @LocalServerPort.
The actual test is pretty simple.
We first configure the target in the PactVerificationContext. This is the URL where our application is running. Since the port field is injected at runtime, we use that port number to create the URL.
Next, we perform one last piece of configuration. Since we expect the consumers to make authenticated requests, we add the Authentication header into the requests that Pact makes to the mobile BFF. Finally, we call context.verifyInteraction to execute the request.
It may seem strange that a test doesn’t actually have any test methods or assertions. This is because the tests themselves live in the Pact files. The contract created by the consumer is the actual test. It specifies what a request looks like and what the response is expected to contain. The Pact framework takes this specification and automatically turns it into a more traditional unit test. The benefit of this approach is that new contracts can be added by the consumer and they will automatically get executed by the provider when its test suite is run.
Contract testing is a strong supplement to traditional functional testing (see contract vs. functional testing) and stays true even if API documentation becomes outdated.
Pact is relatively easy to set up and well-supported within the community. Stay-tuned for follow-up posts on the expanding use of contract-based testing in our systems and how we expand the concept to gRPC-based services!