While working on a recent project, a teammate introduced to me a little known testing pattern that I had never used before in Golang that for our purposes we’ll call Export_Test. At first I was a little skeptical as to it’s readability but soon came around once I realised how useful this pattern became in order to test all the edge cases we wanted without sacrificing private methods or fields. And as you may or may not be aware, at the company formally known as Pivotal, we love Test Driven Development.

The Problem

For our project, we were building an API that needed to request information from various microservices that exposed endpoints as REST APIs. Consequently, we designed a lot of packages that communicated over HTTPS. This meant that we needed a lot of tests around edge cases involving HTTP requests.

Let’s take a look back at the testing pyramid:

testing pyramid: slow, costly UI; service; fast cheap unit

Now the words we use to describe these layers vary between domains, and admittedly in our project we tended to talk about:

  • unit
  • contract
  • integration
  • system
  • acceptance

    Ultimately, the principles are the same. What matters is that:

    Unit tests are cheap and fast and the bulk of your code should be testing in this section.

    So our problem became, if we wanted to test packages who’s singular job was to be an HTTP client, how could we test as much as possible in the unit tests?

    The Solution

    In Golang, files with the suffix _test.go are only compiled for test. It is this functionality that we are exploiting in this pattern.

    1. Injecting with Methods

    Let’s say we have a client we are building called foo_http.go which allows us to choose whether we are going to allow encrypted or non-encrypted http requests (non-encrypted clients are handy for sandbox environments when customers are testing a new setup or for demonstration purposes).

    foo_http.go

    package foo_http
    
    import (
    	"crypto/tls"
    	"fmt"
    	"net/http"
    )
    
    //go:generate counterfeiter -o fakes/fake_http_client.go . Client
    type Client interface {
    	Do(*http.Request) (*http.Response, error)
    }
    
    type FooHTTP struct {
    	AllowUnencrypted bool
    	client           Client
    }
    
    func New(allowUnencrypted, skipTLSValidation bool) FooHTTP {
    	return FooHTTP{
    		AllowUnencrypted: allowUnencrypted,
    		client: &http.Client{
    			Transport: &http.Transport{
    				TLSClientConfig: &tls.Config{
    					InsecureSkipVerify: skipTLSValidation,
    					MaxVersion:         tls.VersionTLS12,
    				},
    			},
    		},
    	}
    }
    
    func (fh FooHTTP) Do(req *http.Request) (*http.Response, error) {
    	if req.URL.Scheme == "http" && !fh.AllowUnencrypted {
    		return nil, fmt.Errorf("http is not allowed, url: %s", req.URL.String())
    	}
    
    	return fh.client.Do(req)
    }
    

    Here we’ve wrapped the default HTTP client in a new struct FooHTTP that holds the value of whether it’s configured with allowing unencrypted connections or not and the client itself. In order to test this simple wrapper, we can use a file we called export_test.go in the same package that allows us to use methods on this struct to inject a mock http client.

    export_test.go

    package foo_http
    
    func (fh *HTTP) WithClient(client Client) *HTTP {
    	fh.client = client
    	return fh
    }
    
    func (fh *HTTP) HTTPClient() Client {
    	return fh.client
    }
    

    This allows us to then use the counterfeiter mock we set up in foo_http.go to mock the http client responses.

    foo_http_test.go

    package foo_http_test
    
    import (
    	"myProject/foo_http"
    	"myProject/foo_http/fakes"
    	"crypto/tls"
    	"fmt"
    	"net/http"
    	"net/url"
    
    	. "github.com/onsi/ginkgo"
    	. "github.com/onsi/gomega"
    )
    
    var _ = Describe("FooHTTP", func() {
    [...]
        var (
            fakeClient *fakes.FakeClient
            req        *http.Request
        )
    
        BeforeEach(func() {
            fakeClient = new(fakes.FakeClient)
            req = &http.Request{URL: &url.URL{Scheme: "https"}}
        })
    
        When("the http call fails", func() {
            It("returns an error", func() {
                fakeClient.DoReturns(nil, fmt.Errorf("i'm an error"))
                fooClient := new(fooHTTP.HTTP).WithClient(fakeClient)
    
                _, err := fooClient.Do(req)
                Expect(err).To(MatchError("i'm an error"))
            })
        })
    [...]
    })
    

    This allows us to instantiate a FooHTTP client with our mock using: client := new(fooHTTP.HTTP).WithClient(fakeClient).

    2. Using Getter Methods to access private variables

    Also of notes in the export_test.go from above is the use of a method to access the private client using a getter.

    export_test.go

    func (fh *HTTP) HTTPClient() Client {
    	return fh.client
    }
    

    We can then use this method to test that our New() function is returning the client instantiated in the way we expect:

    foo_http_test.go

    Describe("New", func() {
        It("returns an HTTP struct", func() {
            fooClient := FooHTTP.New(true, true)
    
            Expect(fooClient.AllowUnencrypted).To(BeTrue())
            httpClient, ok := fooClient.HTTPClient().(*http.Client)
            Expect(ok).To(BeTrue())
            transport, ok := httpClient.Transport.(*http.Transport)
            Expect(ok).To(BeTrue())
            Expect(transport.TLSClientConfig.InsecureSkipVerify).To(BeTrue())
            Expect(transport.TLSClientConfig.MaxVersion).To(Equal(uint16(tls.VersionTLS12)))
        })
    })
    

    This allows us to be extremely granular in the way we unit test every bit of code including our constructors.

    3. Using Public and Private Constructors for Dependency Injection

    Consider again our foo_http.go. Instead of using just the Public function of New(). Perhaps we use a pattern that allows for dependency injection like so:

    foo_http.go

    package foo_http
    
    import (
    	"crypto/tls"
    	"fmt"
    	"net/http"
    )
    
    [...]
    
    type FooHTTP struct {
    	AllowUnencrypted bool
    	client           Client
    }
    
    func BuildClient(allowUnencrypted, skipTLSValidation bool) FooHTTP {
        client := &http.Client{
    			Transport: &http.Transport{
    				TLSClientConfig: &tls.Config{
    					InsecureSkipVerify: skipTLSValidation,
    					MaxVersion:         tls.VersionTLS12,
    				},
    			}
            }    
    	return BuildClient(allowUnencrypted, skipTLSValidation, client)
    }
    
    func buildClient(allowUnencrypted, skipTLSValidation bool, httpClient *http.Client) FooHTTP {
    	return FooHTTP{
    		AllowUnencrypted: allowUnencrypted,
    		client: httpClient,
    	}
    }
    
    [...]
    

    Now our BuildClient function defaults the http client to how we want it to behave and the private function builds the actual struct. This allows us to now inject that client as mock just for our testing.

    export_test.go

    package foo_http
    
    var BuildClientImpl = buildClient
    

    And subsequently we can now inject this dependency in the tests like so:

    foo_http_test.go

    package foo_http_test
    
    [...]
        var (
            fakeClient *fakes.FakeClient
            req        *http.Request
        )
    
        BeforeEach(func() {
            fakeClient = new(fakes.FakeClient)
            req = &http.Request{URL: &url.URL{Scheme: "https"}}
        })
    
        When("the http call fails", func() {
            It("returns an error", func() {
                fakeClient.DoReturns(nil, fmt.Errorf("i'm an error"))
                fooClient := FooHTTP.BuildClientImpl(true, true, fakeClient)
    
                _, err := fooClient.Do(req)
                Expect(err).To(MatchError("i'm an error"))
            })
        })
    [...]
    })
    

    Conclusion

    And that’s it!

    Once I wrapped my head around this Export Test pattern I could see some really strong benefits such as:

  • it helped strengthen our ability to unit test
  • it improved readability in that our testing logic was clearly separated from code logic
  • it removed the need to make methods, functions and struct fields public for the sake of testing