This tutorial will explain some of the pitfalls in migrating to go modules from $GOPATH to accommodate package management in Go and why they happen.
Introduction
In this tutorial, you are going to learn:
- How to create a simple C4ID digest calculator program using an available package on Github.
- Line by line code walkthrough.
- Create the application using
$GOPATH
. - Migrate from
$GOPATH
to$ go mod
. - Troubleshoot import issues post
$ go mod
migration.
Creating the program
Migrating to Go mod from $GOPATH
can be a little confusing, especially if you learned the language being reliant on $GOPATH to handle the external modules that your Go program uses. It’s straight forward. Install Go, Set $GOPATH
, invoke $ go get
and the package would be downloaded straight to ‘$GOPATH/src/path/to/pkg’. Let’s take the github.com/Avalanche-io/c4/ repository as an example.
Be aware that I am on Windows 10 using MSYS2. Hence the use of both $
and %
syntax when referencing variables.
# Note that my $GOPATH resolves to C:\Users\%username%\go
# Change to our $GOPATH Directory and create our sandbox folder.
$ cd $GOPATH && mkdir -p /src/tutorials/gomod
# Change to our gomod directory.
$ cd /src/tutorials/gomod
# Create our main.go file
$ touch main.go
Now let’s flesh out this main.go file to satisfy the program scope which is to import the github.com/Avalanche-io/c4/ module and generate a C4ID for multiple arguments given from the command line. Error handling should be in place to instruct our user that at least one (1) argument is given.
package main
import (
"fmt"
"os"
"strings"
"github.com/Avalanche-io/c4"
)
func main() {
if len(os.Args) <=1 {
fmt.Println("Usage: $ main.exe <some-string>")
} else {
for x := range os.Args[1:] {
fmt.Println(c4.Identify(strings.NewReader(os.Args[x])))
}
}
}
With our code written, let’s $go get
the c4 package.
Note: Use $ git init to initiate a git repo in order to only use $ go get -u .
$ go get -u -v github.com/Avalanche-io/c4/
If you want to browse it, you can find it here: $GOPATH/src/guthub.com/Avalance-io/c4
.
And finally, let’s build and execute our program.
$ go build -o main.exe main.go
$ ./main.exe “Understanding go.mod”
c421U3MrDfX4imH3jjTEFwRfDrvitFcmj9bqohbFvKFi6apVjHGRQezEqVUdRw4xWA6rHFbHD51nxUHDoifwYRMKJ6
# Be sure to check our user misuse handling
$ ./main.exe
Usage: $ main.exe <some-string>
Code Walkthrough
There are a few things happening in the main function. It is also summarised in Figure 1.
Note: If you are familiar with Go syntax and type convention, feel free to skip this breakdown.
- We are taking the second (2nd) argument or more that are given on the command line and passing it to the strings.NewReader function.
os.Args
registers all command-line arguments as a slice of strings. Of which the Index value of Zero (0);os.Args[0]
actually equals the main.exe binary. In order to pass the string argument that we typed, we wrap our expression in a range clause.- As per the specification, we are going to use a for range clause in the pointer semantic form that will iterate over
os.Arg
which is a slice of strings. Note that I am omitting index 0 by using[1:]
. Therefore, for every string value found, it is assigned to xand given as the index position of(x)
henceos.Args[x]
for each iterative pass. - Strings PKG
NewReader
takes a string as an argument and returns a typereader
interface, which is exactly what we need as the Identify function in the c4 package is expecting aio.Reader
interface as an input value which, in turn, returns an interface of typeID
.Println
accepts an interface and prints the output to stdout. - In order to ensure proper use of our program by the user, we have the
for range
clause as anelse
statement to an if conditional governing the handling of theos.Args
slice of strings. As per point 1. We are only interested in arguments after the first index.
Figure. 1: Understanding Go mod func Main().
Introducing Go mod to our program
Alright, we’ve created a functioning program using $GOPATH. Now, let’s add go.mod and rebuild our executable.
# delete our previous executable
$ rm main.exe
# add a go.mod
$ go mod init
go: creating new go.mod: module tutorials/gomod
# for good practice, lets run go tidy to get and prune pkg dependencies
$ go mod tidy
go: finding module for package github.com/Avalanche-io/c4
go: finding module for package golang.org/x/image/math/f64
go: found github.com/Avalanche-io/c4 in github.com/Avalanche-io/c4 v0.7.0
go: found golang.org/x/image/math/f64 in golang.org/x/image v0.0.0-20200618115811-c13761719519
Go.mod is now added to our project. Two additional files have been added to our working directory. A go.mod as a result of $ go mod init
and go.sum as a result of $ go mod tidy
. Now, let’s rebuild our binary and execute our program once more.
$ go build -o main.exe main.go
# command-line-arguments
.\main.go:16:16: undefined: c4.Identify
Uh oh. We have a problem. The compiler is telling us that c4.Identify is undefined. We haven’t edited our code. Our c4 import path is valid as per the repo and pkg layout at $GOPATH.
Launching godoc
and navigating to the c4 package confirms this. The Identify
function is right there in c4/id.go line 73, func Identify().
$ godoc http=:6060
After executing the above, navigate to the following in your browser of choice.
http://localhost:6060/src/github.com/Avalanche-io/c4/id.go?s=1195:1231#L63
NOTE: URL may be different based on your $GOPATH
Image — 1: godoc Avalanche-io c4 URL.
So what exactly has happened?
Understanding Go Mod
File Breakdown
go.mod
module tutorials/gomod
go 1.14
require (
github.com/Avalanche-io/c4 v0.7.0
golang.org/x/image v0.0.0-20200618115811-c13761719519 // indirect
)
As per the documentation the go.mod file adheres to the following syntax.
module <module>
=> module tutorials/gomod
require <module> <version>
=> [in line below]
require github.com/Avalanche-io/c4 v0.7.0 golang.org/x/image v0.0.0–20200618115811-c13761719519 // indirect
<module>
= the module in question that you require. Normally the domain address of the repository.<version>
= the repository tag of the latest ‘Release’.- // indirect indicates that the module is a required dependency of the preceding listed module.
go.sum
github.com/Avalanche-io/c4 v0.7.0 h1:q1+QFR8KVJGPkZZhcZHmqjWPQSBMEiKo6k61Atw0miI=
github.com/Avalanche-io/c4 v0.7.0/go.mod h1:NKmOoDq2g/auYP8t0S99LWQkP0gDUuIB8vmajMJw358=
golang.org/x/image v0.0.0-20200618115811-c13761719519 h1:1e2ufUJNM3lCHEY5jIgac/7UTjd6cgJNdatjPdFWf34=
golang.org/x/image v0.0.0-20200618115811-c13761719519/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
It follows a similar convention to go.sum with the following syntax;
<module> <version>[/go.mod] <hash>
Further details are elaborated on in the documentation. But two excerpts are of interest to us.
- “Each known module version results in two lines in the go.sum file. The first line gives the hash of the module version’s file tree. The second line appends “/go.mod” to the version and gives the hash of only the module version’s (possibly synthesized) go.mod file”
- “The begins with an algorithm prefix of the form “h:”. The only defined algorithm prefix is “h1:”, which uses SHA-256.”
To add, the hash is stored as a Base64 representation of the SHA-256 digest.
Path Comparisons
Notice in go.mod, our module resides at ‘github.com/Avalanche-io/c4’. With go.mod, go is no longer looking at ‘$GOPATH/src/pkg/github.com/Avalanche-io/c4’. But rather it is now looking at ‘$GOPATH/pkg/mod/github.com/Avalanche-io/c4’. In this directory, you will find; ‘!avalance-io/c4@v0.7.0’. This is the directory that our program is pointing to.
Image — 2: Go mod Path.
Looking into this directory, we can see that it looks very different to the contents of the ‘$GOPATH/src’ equivalent; shown below.
Image —3: $GOPATH Path.
You will notice however, that the $GOPATH path directory mirrors that of the Avalanche-io/c4 master branch.
Image — 04: Avalanche-io Master Repo.
So why is the ‘go.mod’ reference different? What exactly is downloading? One answer; ‘Version Control’.
The field indicates which version will be downloaded. This is normally the latest release by Default. As noted in our ‘go.mod’ file.
go.mod
github.com/Avalanche-io/c4 v0.7.0
Our version of interest is v0.7.0 of which there is a corresponding release here, which also happens to be the latest release. Let’s compare it to our downloaded module.
Image — 5 Module Comparison.
We can clearly see that they both match and that this is the module we are importing in our program. With the exception of doc.go, there are no *.go files to interpret and import into our program at “github.com/Avalanche-io/c4”. However, navigate one more directory down to “github.com/Avalanche-io/c4/id”, we can find our Identify function in ‘id.go’ This means that we need to change our import statement to reflect the module layout of this version’s packages .
Code Amendments
Ok, now that we have established what is going on. Let’s update the import statement in our code to reflect the layout of the v0.7.0 release. Simply add ‘id’ to the import path and append c4 to the import string to ‘import as’ in order to preserve the ‘c4’ prefix of all existing expressions where the c4 package is called.
package main
import (
"fmt"
"os"
"strings"
c4 "github.com/Avalanche-io/c4/id"
)
func main() {
if len(os.Args) <=1 {
fmt.Println("Usage: $ main.exe <some-string>")
} else {
for x := range os.Args[1:] {
fmt.Println(c4.Identify(strings.NewReader(os.Args[x])))
}
}
}
Build our binary and run the program.
$ go build -o main.exe main.go
$ ./main.exe “Understanding go.mod”
c421U3MrDfX4imH3jjTEFwRfDrvitFcmj9bqohbFvKFi6apVjHGRQezEqVUdRw4xWA6rHFbHD51nxUHDoifwYRMKJ6
Success!
Summary
In this tutorial, you learned:
- How to create a simple C4ID digest calculator program using an available package on github.
- Line by line code walkthrough.
- Create the application using
$GOPATH
. - Migrate from
$GOPTH
to$ go mod
. - Troubleshoot import issues post $ go modmigration.
References
Go Command Documentation, https://golang.org/cmd/go/
Go Language Specification, https://golang.org/ref/spec
Go PKG strings, https://pkg.go.dev/strings
Go PKG io, https://pkg.go.dev/io
Go PKG fmt, https://pkg.go.dev/fmt
Avalanche-io C4ID, https://github.com/Avalanche-io/c4
IETF RFC 6234, https://tools.ietf.org/rfc/rfc6234.txt
IETF RFC 4648, https://tools.ietf.org/rfc/rfc4648.txt
Further Reading
Official Go Website, Golang.org
Keeping Your Modules Compatible, Jean de Klerk and Jonathan Amsterdam, 7 July 2020, https://blog.golang.org/module-compatibility
Go Modules Reference, https://golang.org/ref/mod