Definition:
Modules:
- A module is a collection of related Go packages that are versioned together as a single unit.
- Modules must be semantically versioned in the form v(major).(minor).(patch), such as v0.1.0, v1.2.3, or v3.0.1.
pre-release version
- The versioning leading v is required.
- The go command automatically updates go.mod each time it uses the module graph, to make sure go.mod always accurately reflects reality and is properly formatted.
- Because the module graph defines the meaning of import statements, any commands that load packages also use and therefore update go.mod, including go build, go get, go install, go list, go test, go mod graph, go mod tidy, and go mod why.
$ go mod why -m rsc.io/binaryregexp
Commands outside modules:
When module-aware mode is enabled explicitly (by setting
GO111MODULE=on), most module commands have more limited functionality if no go.mod file is present.
For example,
go build,
go run, and other build commands can only build packages in the standard library and packages specified as .go files on the command line.
Previously, the go command would resolve each package path
to the latest version of a module but would not record the module path or version. This resulted in slow, non-reproducible builds.
go get continues to work as before, as do
go mod download and
go list -m with explicit versions.
$
go list -m // list module instead of packages
Default Proxy:
https://proxy.golang.org
Default checksum db:
https://sum.golang.org
- $GOPRIVATE (show: $ go env GOPRIVATE)
controls which modules the go command considers to be private (not available publicly) and should therefore not use the proxy or checksum database. The variable is a comma-separated list of glob patterns (in the syntax of Go's path.Match) of module path prefixes.
- $GOMOD (show: $ go env GOMOD)
The absolute path to the go.mod of the main module,
or the empty string if not using modules.
- $GOPROXY (show: $ go env GOPROXY)
URL of Go module proxy. See 'go help goproxy'.
- $GOTOOLDIR (show: $ go env GOTOOLDIR)
The directory where the go tools (compile, cover, doc, etc...) are installed.
- $GOHOSTOS (show: $ go env GOHOSTOS)
The operating system (GOOS) of the Go toolchain binaries.
- $GOHOSTARCH (show: $ go env GOHOSTARCH)
The architecture (GOARCH) of the Go toolchain binaries.
- $GOEXE (show: $ go env GOEXE)
The executable file name suffix (".exe" on Windows, "" on other systems).
- $GOTMPDIR (show: $ go env GOTMPDIR)
The directory where the go command will write temporary source files, packages, and binaries.
- $GOINSECURE is a new environment variable that instructs the go command to not require an HTTPS connection, and to skip certificate validation, when fetching certain modules directly from their origins.
Like the existing GOPRIVATE variable, the value of GOINSECURE is a comma-separated list of glob patterns.
go.mod example:
--
module my/thing
go 1.12
require other/thing v1.0.2
require new/thing/v2 v2.3.4
exclude old/thing v1.2.3
replace bad/thing v1.4.5 => good/thing v1.4.5
--
verbs explain:
--
module, to define the module path;
go, to set the expected language version;
require, to require a particular module at a given version or later;
exclude, to exclude a particular module version from use; and
replace, to replace a module version with a different module version.
--
like go import's syntax, verbs can be coded as:
--
require (
new/thing v2.3.4
old/thing v1.2.3
)
--
GOPATH and Modules:
When using modules, GOPATH is no longer used for resolving imports.
However, it is still used to store downloaded source code (in GOPATH/pkg/mod)
and compiled commands (in GOPATH/bin).
'go get' checks out or updates a git repository also updates submodules.
Module proxy protocol:
- If GOPROXY is unset, is the empty string, or is the string "direct", downloads use the default direct connection to version control systems.
- Setting GOPROXY to "off" disallows downloading modules from any source.
- Otherwise, GOPROXY is expected to be the URL of a module proxy, in which case the go command will fetch all modules from that proxy.
- No matter the source of the modules, downloaded modules must match existing entries in go.sum (see 'go help modules' for discussion of verification).
golang build cache path:
The go command also caches successful package test results.
See 'go help test' for details.
GOCACHE, defaulting to $HOME/.cache/go-build, we'll set to "/tmp/on"
Print current cache directory
$
go env GOCACHE
Setting the environment variable
GOCACHE=off will cause go commands that write to the cache to fail.
$
GOCACHE=off
Clean all cache
$
go clean -cache
Removes all cached test results
$
go clean -testcache
The
GODEBUG environment variable can enable printing of debugging information about the state of the cache:
- GODEBUG=gocacheverify=1 causes the go command to bypass the use of any cache entries and instead rebuild everything and check that the results match existing cache entries.
- GODEBUG=gocachehash=1 causes the go command to print the inputs for all of the content hashes it uses to construct cache lookup keys. The output is voluminous but can be useful for debugging the cache.
- GODEBUG=gocachetest=1 causes the go command to print details of its decisions about whether to reuse a cached test result.
golang module cache path:
$
GOPATH/pkg/mod
We can remove the cached package manually here.
Clean downloaded cache:
$
go clean -modcache
reference:
https://tip.golang.org/cmd/go/#hdr-Remove_object_files_and_cached_files
Backward compatible:
Set GO111MODULE=on if want to use go module within $GOPATH
Modules and vendoring
When the main module contains a top-level vendor directory and its go.mod file specifies go 1.14 or higher, the go command now defaults to -mod=vendor
When -mod=vendor is set (explicitly or by default), the go command now verifies that the main module's vendor/modules.txt file is consistent with its go.mod file.
By default, the go command satisfies dependencies by downloading modules from their sources and using those downloaded copies (after verification).
-mod=mod causes the go command to instead load modules from the module cache.
To allow interoperation with older versions of Go, or to ensure that all files used for a build are stored together in a single file tree, 'go mod vendor' creates a directory named vendor in the root directory of the main module and stores there all the packages from dependency modules that are needed to support builds and tests of packages in the main module.
To build using the main module's top-level vendor directory to satisfy dependencies (disabling use of the usual network sources and local caches), use 'go build -mod=vendor'. (This is not recommended)
i.e put all dependencies under:
vendor/ folder
Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored.
Readonly Modules
If invoked with -mod=readonly, the go command is disallowed from the implicit automatic updating of go.mod described above.
When -mod=readonly is set, go commands will no longer fail due to a missing go directive or an erroneous // indirect comment.
-mod=readonly is now set by default when the go.mod file is read-only and no top-level vendor directory is present.
Instead, it fails when any changes to go.mod are needed. This setting is most useful to check that go.mod does not need updates, such as in a continuous integration and testing system.
The "go get" command remains permitted to update go.mod even with -mod=readonly, and the "go mod" commands do not take the -mod flag (or any other build flags).
Cache Control
-modcacherw is a new flag that instructs the go command to leave newly-created directories in the module cache at their default permissions rather than making them read-only.
The use of this flag makes it more likely that tests or other tools will accidentally add files not included in the module's verified checksum. However, it allows the use of rm -rf (instead of go clean -modcache) to remove the module cache.
Use other go.mod file
-modfile=file is a new flag that instructs the go command to read (and possibly write) an alternate go.mod file instead of the one in the module root directory.
A file named go.mod must still be present in order to determine the module root directory, but it is not accessed.
When -modfile is specified, an alternate go.sum file is also used: its path is derived from the -modfile flag by trimming the .mod extension and appending .sum.
Fresh up your git tag skills:
go module relies on git tag to do versioning.
Two kinds of git tags:
https://git-scm.com/book/en/v2/Git-Basics-Tagging
Annotated tag, as a commit object with tagger's information:
$
git tag -a v1.0.1 -m "version 1.0.1"
$
git show v1.0.1
Lightweight Tags, no information attached:
$
git tag v1.0.1-lw
Push local tags to remote:
$
git push origin <tagname>
push all tags:
$
git push origin --tags
Checking out Tags:
Will be in detached state. i.e not under current branch's HEAD commit.
$
git checkout 2.0.0
or checkout tag into branch: (better, thus you could do some changes and commit)
$
git checkout -b version2 v2.0.0
delete local tag:
https://stackoverflow.com/questions/5480258/how-to-delete-a-git-remote-tag
$
git tag -d v1.4
delete remote tag:
$
git push origin :refs/tags/v1.4
or
$
git push origin :tagname
or
$
git push --delete origin tagname
Go module follow semantic versioning:
https://semver.org/
Go uses repository tags to lookup for versions.
Only version >= 2 will have different import path.
i.e
github.com/buddhavs/module_test <= module base path, which contains 'util' package
import "
github.com/buddhavs/module_test/util"
// for version 0 or 1
import "
github.com/buddhavs/module_test/
v2/util"
// for version 2
import "
github.com/buddhavs/module_test/
v3/util"
// for version 3
Steps for init new module:
1.
$
go mod init github.com/buddhavs/module_test
Generates 'go.mod'
go.mod contains current projects path/version
i.e
module
github.com/buddhavs/module_test <= module base path, which contains 'util' package
2.
Import current module needed by the application.
$
go mod tidy
Generates updated
go.mod and new
go.sum
go commands other than
go mod tidy no longer remove a require directive that specifies a version of an indirect dependency that is already implied by other (transitive) dependencies of the main module.
go commands other than
go mod tidy no longer edit the go.mod file if the changes are only cosmetic.
3.
Series of git init commands:
$
git init
$
git add * // includes go.mod
$
git commit -am "1 commit"
$
git push -u origin master
4.
Anyone who uses the package can now do follow commands as before:
$
go get github.com/buddhavs/module_test
5.
$
git tag v1.0.0 // tag the version
6.
$
git checkout -b v1
$
git push -u origin v1 --tags
Steps for create new minor/patch version module:
$
git commit -m "patch version."
$
git tag v1.0.1
$
git push --tags origin v1.0.1
Steps for create new major version module:
$
git commit -m "v2"
$
git checkout -b v2 # Recommended.
$
echo "module github.com/buddhavs/module_test/v2" > go.mod
$
git commit go.mod -m "Bump to v2"
$
git tag v2.0.0
$
git push --tags origin v2.0.0 # or master if don't have a branch
Consume modules:
1.
If using module v0 or v1 with minor or patch, the import path is as usual.
i.e
package main
import mt "github.com/buddhavs/module_test"
func main() {
mt.Run()
}
2.
If using module
> v1, import path with version number.
i.e
package main
import mt "github.com/buddhavs/module_test/v2" // it's mentioned in the module's go.mod
func main() {
mt.Run()
}
3.
go build will use the latest version of the main version of the module.
Consume modules from local file path:
https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem
Modify go.mod with 'replace' key word:
go.mod:
--
module module_main
require module_test/v2 v2.0.1
replace module_test/v2 => ../module_test
--
Cmd:
list current project's needed modules:
$
go list -m all
Fails explicitly if
-mod=vendor is set and information is requested for a module not mentioned in vendor/modules.txt
find updated modules:
$
go list -m -u all
get all modules source code only:
$
go get -d
upgrade all modules:
$
go get -u
get the latest version:
$
go get -u github.com/vsdmars/project@latest
show module versions:
$
go list -m rsc.io/sampler
use 'go get' to upgrade/downgrade modules:
$
go get -u // use the latest minor or patch releases (i.e. it would update from 1.0.0 to, say, 1.0.1 or, if available, 1.1.0)
$
go get -u=patch // use the latest patch releases (i.e., would update to 1.0.1 but not to 1.1.0)
$
go get package@version //update to a specific version (say, github.com/robteix/testmod@v1.0.1)
use local changes:
$
go mod edit -replace 'rsc.io/quote=../quote'
use remote version:
$
go mod edit -replace 'rsc.io/quote=github.com/myitcv/london-gophers-quote-fork@v0.0.0-myfork' // @TAG
Go Clean:
https://manpages.debian.org/testing/golang-go/go-clean.1.en.html
# -r clean to be applied recursively to all the dependencies of the packages named by the import paths.
$
go clean -r
Go Module Proxy
Download Protocol
Reference:
https://docs.gomods.io/intro/protocol/
go command determines needing a module, it first looks at the local cache
(under
$GOPATH/pkg/mods ). If it can’t find the files there, it then goes from the network.
To control what files go can download, setting the
GOPROXY environment variable to point to our proxy’s URL.
For instance:
$
export GOPROXY=http://gproxy.association.local:8080
The proxy is a web server that responds to the module download protocol, which is an API to query and fetch modules. The web server may serve static files.
( figure from
https://roberto.selbach.ca/go-proxies/ )
We can set
GOPROXY to local directory which has the correct directory layout of the API protocol also works:
$ export GOPROXY=file://home/robteix/devel/go-proxy-blog
Project Athens, a go module proxy project:
https://github.com/gomods/athens
Go 1.12
https://golang.org/doc/go1.12#modules
Go Install doesn't work anymore if
GO111MODULE=ON
There will be a work around on golang v1.12
https://github.com/golang/go/issues/24250
$
go run x.go
$
go get rsc.io/2fa@v1.1.0
Both can now(go v.1.12) operate in
GO111MODULE=on mode without an explicit go.mod file.
Commands such as
- go get
- go list
- go mod download
behave as if in a module with initially-empty requirements.
In this mode, go env GOMOD reports the system's null device (/dev/null or NUL).
The go directive in a go.mod file now indicates the version of the language used by the files within that module.
This changed use of the go directive means that if you use Go 1.12 to build a module, thus recording go 1.12 in the go.mod file, you will get an error when attempting to build the same module with Go 1.11 through Go 1.11.3. Go 1.11.4 or later will work fine, as will releases older than Go 1.11. If you must use Go 1.11 through 1.11.3, you can avoid the problem by setting the language version to 1.11, using the Go 1.12 go tool, via go mod edit -go=1.11
When an import cannot be resolved using the active modules, the go command will now try to use the modules mentioned in the main module's replace directives before consulting the module cache and the usual network sources.
If a matching replacement is found but the replace directive does not specify a version, the go command uses a pseudo-version derived from the zero time.Time (such as v0.0.0-00010101000000-000000000000).
Go 1.13
Reference:
- GO111MODULE environment variable continues to default to auto, but the auto setting now activates the module-aware mode of the go command whenever the current working directory contains, or is below a directory containing, a go.mod file — even if the current directory is within GOPATH/src.
- GOPATH will be deprecated in go v.1.13
- the GOPROXY (by default in 1.13) environment variable allows comma-separated list. It’ll try the first proxy before falling back to the next path.
- The default value of GOPROXY is set as https://proxy.golang.org,direct
Anything after the direct token is ignored.
If don’t want to use Go proxy , set it to off.
- A new GOPRIVATE environment variable is introduced, contains a comma-separated list of glob patterns.
This can be used to bypass the GOPROXY proxy for certain paths,
i.e private modules in a company
(e.g: GOPRIVATE=*.internal.company.com).
- The new GOSUMDB environment variable identifies the name, and optionally the public key and server URL, of the database to consult for checksums of modules that are not yet listed in the main module's go.sum file.
If GOSUMDB does not include an explicit URL, the URL is chosen by probing the GOPROXY URLs for an endpoint indicating support for the checksum database, falling back to a direct connection to the named database if it is not supported by any proxy.
If GOSUMDB is set to off, the checksum database is not consulted and only the existing checksums in the go.sum file are verified.
- Users who cannot reach the default proxy and checksum database (for example, due to a firewalled or sandboxed configuration) may disable their use by setting GOPROXY to direct, and/or GOSUMDB to off. go env -w can be used to set the default values for these variables independent of platform:
$ go env -w GOPROXY=direct
$ go env -w GOSUMDB=off
- proxy list:
- https://proxy.golang.org
- https://goproxy.io
- opensource projects:
- https://github.com/gomods/athens
- https://github.com/goproxy/goproxy
- https://thumbai.app/
Go Module Index
Module Authentication
go command uses go.sum to verify that dependencies are bit-for-bit identical to the expected versions before using them in a build.
Module Discovery
Reference:
