Understanding Go mod

This tutorial will explain some of the pitfalls in migrating to go modules from $GOPATH to accommodate package management in Go and why they happen.

Introduction

In this tutorial, you are going to learn:

How to create a simple C4ID digest calculator program using an available package on Github.
Line by line code walkthrough.
Create the application using $GOPATH.
Migrate from $GOPATH to $ go mod.
Troubleshoot import issues post $ go mod migration.

Creating the program

Migrating to Go mod from $GOPATH can be a little confusing, especially if you learned the language being reliant on $GOPATH to handle the external modules that your Go program uses. It’s straight forward. Install Go, Set $GOPATH, invoke $ go get and the package would be downloaded straight to ‘$GOPATH/src/path/to/pkg’. Let’s take the github.com/Avalanche-io/c4/ repository as an example.

Be aware that I am on Windows 10 using MSYS2. Hence the use of both $ and % syntax when referencing variables.

# Note that my $GOPATH resolves to C:\Users\%username%\go
# Change to our $GOPATH Directory and create our sandbox folder.
$ cd $GOPATH && mkdir -p /src/tutorials/gomod
# Change to our gomod directory.
$ cd /src/tutorials/gomod
# Create our main.go file
$ touch main.go

Now let’s flesh out this main.go file to satisfy the program scope which is to import the github.com/Avalanche-io/c4/ module and generate a C4ID for multiple arguments given from the command line. Error handling should be in place to instruct our user that at least one (1) argument is given.

package main
import (
 "fmt"
 "os"
 "strings"
"github.com/Avalanche-io/c4"
)
func main() {
if len(os.Args) <=1 {
  fmt.Println("Usage: $ main.exe <some-string>")
 } else {
  for x := range os.Args[1:] {
   fmt.Println(c4.Identify(strings.NewReader(os.Args[x])))  
  }
 }
}

With our code written, let’s $go get the c4 package.

Note: Use $ git init to initiate a git repo in order to only use $ go get -u .

$ go get -u -v github.com/Avalanche-io/c4/

If you want to browse it, you can find it here: $GOPATH/src/guthub.com/Avalance-io/c4.

And finally, let’s build and execute our program.

$ go build -o main.exe main.go
$ ./main.exe “Understanding go.mod”
c421U3MrDfX4imH3jjTEFwRfDrvitFcmj9bqohbFvKFi6apVjHGRQezEqVUdRw4xWA6rHFbHD51nxUHDoifwYRMKJ6
# Be sure to check our user misuse handling
$ ./main.exe
Usage: $ main.exe <some-string>

Code Walkthrough

There are a few things happening in the main function. It is also summarised in Figure 1.

Note: If you are familiar with Go syntax and type convention, feel free to skip this breakdown.

We are taking the second (2nd) argument or more that are given on the command line and passing it to the strings.NewReader function.
os.Args registers all command-line arguments as a slice of strings. Of which the Index value of Zero (0); os.Args[0] actually equals the main.exe binary. In order to pass the string argument that we typed, we wrap our expression in a range clause.
As per the specification, we are going to use a for range clause in the pointer semantic form that will iterate over os.Arg which is a slice of strings. Note that I am omitting index 0 by using [1:]. Therefore, for every string value found, it is assigned to xand given as the index position of (x) hence os.Args[x] for each iterative pass.
Strings PKG NewReader takes a string as an argument and returns a type reader interface, which is exactly what we need as the Identify function in the c4 package is expecting a io.Reader interface as an input value which, in turn, returns an interface of type ID. Println accepts an interface and prints the output to stdout.
In order to ensure proper use of our program by the user, we have the for range clause as an else statement to an if conditional governing the handling of the os.Args slice of strings. As per point 1. We are only interested in arguments after the first index.

image Figure — 1 Is aflow diagram showcasing the functionality of func Main()
Figure. 1: Understanding Go mod func Main().

Introducing Go mod to our program

Alright, we’ve created a functioning program using $GOPATH. Now, let’s add go.mod and rebuild our executable.

# delete our previous executable
$ rm main.exe
# add a go.mod
$ go mod init
go: creating new go.mod: module tutorials/gomod
# for good practice, lets run go tidy to get and prune pkg dependencies
$ go mod tidy
go: finding module for package github.com/Avalanche-io/c4
go: finding module for package golang.org/x/image/math/f64
go: found github.com/Avalanche-io/c4 in github.com/Avalanche-io/c4 v0.7.0
go: found golang.org/x/image/math/f64 in golang.org/x/image v0.0.0-20200618115811-c13761719519

Go.mod is now added to our project. Two additional files have been added to our working directory. A go.mod as a result of $ go mod init and go.sum as a result of $ go mod tidy. Now, let’s rebuild our binary and execute our program once more.

$ go build -o main.exe main.go
# command-line-arguments
.\main.go:16:16: undefined: c4.Identify

Uh oh. We have a problem. The compiler is telling us that c4.Identify is undefined. We haven’t edited our code. Our c4 import path is valid as per the repo and pkg layout at $GOPATH.

Launching godoc and navigating to the c4 package confirms this. The Identify function is right there in c4/id.go line 73, func Identify().

$ godoc http=:6060

After executing the above, navigate to the following in your browser of choice.

http://localhost:6060/src/github.com/Avalanche-io/c4/id.go?s=1195:1231#L63

NOTE: URL may be different based on your $GOPATH

image -1 shows the presence of function Identify in c4/id.go on line 73

Image — 1: godoc Avalanche-io c4 URL.

So what exactly has happened?

Understanding Go Mod

File Breakdown

go.mod

module tutorials/gomod
go 1.14
require (
 github.com/Avalanche-io/c4 v0.7.0
 golang.org/x/image v0.0.0-20200618115811-c13761719519 // indirect
)

As per the documentation the go.mod file adheres to the following syntax.

module <module> => module tutorials/gomod

require <module> <version> => [in line below]

require github.com/Avalanche-io/c4 v0.7.0 golang.org/x/image v0.0.0–20200618115811-c13761719519 // indirect

<module> = the module in question that you require. Normally the domain address of the repository.
<version> = the repository tag of the latest ‘Release’.
// indirect indicates that the module is a required dependency of the preceding listed module.

go.sum

github.com/Avalanche-io/c4 v0.7.0 h1:q1+QFR8KVJGPkZZhcZHmqjWPQSBMEiKo6k61Atw0miI=
github.com/Avalanche-io/c4 v0.7.0/go.mod h1:NKmOoDq2g/auYP8t0S99LWQkP0gDUuIB8vmajMJw358=
golang.org/x/image v0.0.0-20200618115811-c13761719519 h1:1e2ufUJNM3lCHEY5jIgac/7UTjd6cgJNdatjPdFWf34=
golang.org/x/image v0.0.0-20200618115811-c13761719519/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
It follows a similar convention to go.sum with the following syntax;

<module> <version>[/go.mod] <hash>

Further details are elaborated on in the documentation. But two excerpts are of interest to us.

“Each known module version results in two lines in the go.sum file. The first line gives the hash of the module version’s file tree. The second line appends “/go.mod” to the version and gives the hash of only the module version’s (possibly synthesized) go.mod file”
“The begins with an algorithm prefix of the form “h:”. The only defined algorithm prefix is “h1:”, which uses SHA-256.”

To add, the hash is stored as a Base64 representation of the SHA-256 digest.

Path Comparisons

Notice in go.mod, our module resides at ‘github.com/Avalanche-io/c4’. With go.mod, go is no longer looking at ‘$GOPATH/src/pkg/github.com/Avalanche-io/c4’. But rather it is now looking at ‘$GOPATH/pkg/mod/github.com/Avalanche-io/c4’. In this directory, you will find; ‘!avalance-io/c4@v0.7.0’. This is the directory that our program is pointing to.

Image -2 lists the contents of the directory c4@v0.7.0 Image — 2: Go mod Path.

Looking into this directory, we can see that it looks very different to the contents of the ‘$GOPATH/src’ equivalent; shown below.

Image -3 lists the directory contents of $GOPATH c4 pkg.

Image —3: $GOPATH Path.

You will notice however, that the $GOPATH path directory mirrors that of the Avalanche-io/c4 master branch.

Image -4 lists the contents of the c4’s module master branch. Image — 04: Avalanche-io Master Repo.

So why is the ‘go.mod’ reference different? What exactly is downloading? One answer; ‘Version Control’.

The field indicates which version will be downloaded. This is normally the latest release by Default. As noted in our ‘go.mod’ file.

go.mod

github.com/Avalanche-io/c4 v0.7.0

Our version of interest is v0.7.0 of which there is a corresponding release here, which also happens to be the latest release. Let’s compare it to our downloaded module.

image —5 shows a module comparison between both remote repo’s release branch and local downloaded module. Image — 5 Module Comparison.

We can clearly see that they both match and that this is the module we are importing in our program. With the exception of doc.go, there are no *.go files to interpret and import into our program at “github.com/Avalanche-io/c4”. However, navigate one more directory down to “github.com/Avalanche-io/c4/id”, we can find our Identify function in ‘id.go’ This means that we need to change our import statement to reflect the module layout of this version’s packages .

Code Amendments

Ok, now that we have established what is going on. Let’s update the import statement in our code to reflect the layout of the v0.7.0 release. Simply add ‘id’ to the import path and append c4 to the import string to ‘import as’ in order to preserve the ‘c4’ prefix of all existing expressions where the c4 package is called.

package main
import (
 "fmt"
 "os"
 "strings"
c4 "github.com/Avalanche-io/c4/id"
)
func main() {
if len(os.Args) <=1 {
  fmt.Println("Usage: $ main.exe <some-string>")
 } else {
  for x := range os.Args[1:] {
   fmt.Println(c4.Identify(strings.NewReader(os.Args[x])))  
  }
 }
}

Build our binary and run the program.

$ go build -o main.exe main.go
$ ./main.exe “Understanding go.mod”
c421U3MrDfX4imH3jjTEFwRfDrvitFcmj9bqohbFvKFi6apVjHGRQezEqVUdRw4xWA6rHFbHD51nxUHDoifwYRMKJ6

Success!

Summary

In this tutorial, you learned:

How to create a simple C4ID digest calculator program using an available package on github.
Line by line code walkthrough.
Create the application using $GOPATH.
Migrate from $GOPTH to $ go mod.
Troubleshoot import issues post $ go modmigration.

References

Go Command Documentation, https://golang.org/cmd/go/
Go Language Specification, https://golang.org/ref/spec
Go PKG strings, https://pkg.go.dev/strings
Go PKG io, https://pkg.go.dev/io
Go PKG fmt, https://pkg.go.dev/fmt
Avalanche-io C4ID, https://github.com/Avalanche-io/c4
IETF RFC 6234, https://tools.ietf.org/rfc/rfc6234.txt
IETF RFC 4648, https://tools.ietf.org/rfc/rfc4648.txt