Vendoring Golang Code

Saturday, 29 November 2014

---

One of the issues that many companies face when looking at Golang is what to do with the Code that you find out on the “interwebs”. The basic consensus is to either do:

• leave it alone, use go get to simply get the current version
• “copy”, or “vendor” the code into a local repository for consistent builds

For any sort of repeatable and consistent build to be available, the second is really the only option. How exactly the public (or vendor specific) code moves from one place into your company’s source code control is something that is very different from company to company. Every company has procedures and tools (you hope) to make this process their version of “correct”.

Build Artifact Information

In the end, the important things are less how exactly you vendor your code, but knowing what version of a code base that is currently in use. Being able to quickly inspect a running system or artifact to see the versions of all the vendor’d pieces is paramount to knowing if anything needs to be updated. In this vein, Golang provides an interesting ability with its linker. The Golang linker can inject a string value into un-initialized string variables. The Golang ld Command Docs say:

-X symbol value
    Set the value of an otherwise uninitialized string variable.
    The symbol name should be of the form importpath.name,
    as displayed in the symbol table printed by "go tool nm".

This snippet, along with the Golang go Command Docs give all the pieces necessary to embedd the current build version into a Golang binary. Using this ability, we can inject information into a Golang build, such as Git SHA, build user, etc. These variables can then be used to inspect the running or built artifact to discern what was built. The method to build official artifacts for deployment would then be modified to be something like:

SHA=$(git rev-parse HEAD)
go install -a -ldflags "-X github.com/weingart/vendor.buildSHA ${SHA}" example

The vendor package provides the foloowing variables that can be set via the above method:

buildSHA
buildTag
buildUser
buildTime
buildComment

Vendor’d Package Information

Unfortunately, expanding the go build or go install command line to include a Git SHA for every vendor’d piece of code gets very unwieldy very quickly. In order to make this a little easier vendor is a small package that keeps track of vendor’d code versions as well as provide a placeholder for the uninitilized string variables for the local build information. As part of the vendoring of Golang code, your procedure would add a file to each package that is being vendor’d. In particular, <package>/vendor.go or something similar, that looks something like:

package vendor-code

import "github.com/weingart/vendor"

func init() {
    vendor.Add(&vendor.Info{
        "git-repo": "github.com/vendor/repo",
        "git-sha":  "9e69b5e2e8d4d042f67d9b24f0f69ffb3ab35687",
        "git-tag":  "sometag",
    })
}

As packages can have multiple init() functions, there are no conflicts with any existing init() functions within vendor’d packages. The built artifact can then export this information using the expvar package, or via the vendor.Get*() functions.

package main

import (
    _ "expvar"
    "fmt"
    "github.com/weingart/vendor"
    "net/http"
)

func main() {
    fmt.Println(vendor.GetInfo())
    http.ListenAndServe(":8080", nil)
}

Using the above as an example, the build and vendor information can be found by going to: http://localhost:8080/debug/vars

The code can be found on GitHub along with the documentation at GoDoc.

---