Showing posts with label golang_debug. Show all posts
Showing posts with label golang_debug. Show all posts

Mar 8, 2021

[Go] relase v1.16 highlight

Reference:
https://golang.org/doc/go1.16




386:
As announced in the Go 1.15 release notes, Go 1.16 drops support for x87 mode compilation (GO386=387). Support for non-SSE2 processors is now available using soft float mode (GO386=softfloat). Users running on non-SSE2 processors should replace GO386=387 with GO386=softfloat.


Module:
Module-aware mode is enabled by default, regardless of whether a go.mod file is present in the current working directory or a parent directory.


More precisely, the GO111MODULE environment variable now defaults to on. To switch to the previous behavior, set GO111MODULE to auto.

Build commands like go build and go test no longer modify go.mod and go.sum by default.
Instead, they report an error if a module requirement or checksum needs to be added or updated (as if the -mod=readonly flag were used). Module requirements and sums may be adjusted with go mod tidy or go get.

go install now accepts arguments with version suffixes (for example, go install example.com/cmd@v1.0.0 or go install example.com/cmd@latest). This causes go install to build and install packages in module-aware mode, ignoring the go.mod file in the current directory or any parent directory, if there is one. This is useful for installing executables without affecting the dependencies of the main module.

go install, with or without a version suffix (as described above), is now the recommended way to build and install packages in module mode.
go get should be used with the -d flag to adjust the current module's dependencies without building packages, and use of go get to build and install packages is deprecated. In a future release, the -d flag will always be enabled.

retract directives may now be used in a go.mod file to indicate that certain published versions of the module should not be used by other modules. A module author may retract a version after a severe problem is discovered or if the version was published unintentionally.

The "retract" directive comes in two forms:

retract v1.0.0 // single version
retract [v1.1.0, v1.2.0] // closed interval

The go directive for specifying a Go version does not seem to matter when using this. E.g. I can have a file like this:
module awesomeProject37

go 1.11 // not Go 1.16 above

retract v1.0.0 // single version
retract [v1.1.0, v1.2.0] // closed interval
As long as using Go 1.16+ as the SDK.


The go mod vendor and go mod tidy subcommands now accept the -e flag, which instructs them to proceed despite errors in resolving missing packages.

The go command now ignores requirements on module versions excluded by exclude directives in the main module. Previously, the go command used the next version higher than an excluded version, but that version could change over time, resulting in non-reproducible builds.

In module mode, the go command now disallows import paths that include non-ASCII characters or path elements with a leading dot character (.)
Module paths with these characters were already disallowed (see Module paths and versions), so this change affects only paths within module subdirectories.


Embedding Files:
The go command now supports including static files and file trees as part of the final executable, using the new //go:embed directive. See the documentation for the new embed package for details.


go test:
When using go test, a test that calls os.Exit(0) during execution of a test function will now be considered to fail. This will help catch cases in which a test calls code that calls os.Exit(0) and thereby stops running all future tests. If a TestMain function calls os.Exit(0) that is still considered to be a passing test.

go test reports an error when the -c or -i flags are used together with unknown flags. Normally, unknown flags are passed to tests, but when -c or -i are used, tests are not run.


go get:
The go get -insecure flag is deprecated and will be removed in a future version. This flag permits fetching from repositories and resolving custom domains using insecure schemes such as HTTP, and also bypasses module sum validation using the checksum database.

To permit the use of insecure schemes, use the GOINSECURE environment variable instead. 
To bypass module sum validation, use GOPRIVATE or GONOSUMDB. See go help environment for details.


go get example.com/mod@patch now requires that some version of example.com/mod already be required by the main module. (However, go get -u=patch continues to patch even newly-added dependencies.)


GOVCS environment variable:
GOVCS is a new environment variable that limits which version control tools the go command may use to download source code. This mitigates security issues with tools that are typically used in trusted, authenticated environments. By default, git and hg may be used to download code from any repository. svn, bzr, and fossil may only be used to download code from repositories with module paths or package paths matching patterns in the GOPRIVATE environment variable. See go help vcs for details.


The all pattern:
When the main module's go.mod file declares go 1.16 or higher, the all package pattern now matches only those packages that are transitively imported by a package or test found in the main module. (Packages imported by tests of packages imported by the main module are no longer included.) This is the same set of packages retained by go mod vendor since Go 1.11.


The -toolexec build flag:
When the -toolexec build flag is specified to use a program when invoking toolchain programs like compile or asm, the environment variable TOOLEXEC_IMPORTPATH is now set to the import path of the package being built.


The -i build flag:
The -i flag accepted by go build, go install, and go test is now deprecated. 


The list command:
When the -export flag is specified, the BuildID field is now set to the build ID of the compiled package.
This is equivalent to running go tool buildid on go list -exported -f {{.Export}}, but without the extra step.


The -overlay flag:
The -overlay flag specifies a JSON configuration file containing a set of file path replacements. The -overlay flag may be used with all build commands and go mod subcommands. It is primarily intended to be used by editor tooling such as gopls to understand the effects of unsaved changes to source files. The config file maps actual file paths to replacement file paths and the go command and its builds will run as if the actual file paths exist with the contents given by the replacement file paths, or don't exist if the replacement file paths are empty.


Cgo:
The cgo tool will no longer try to translate C struct bitfields into Go struct fields, even if their size can be represented in Go. The order in which C bitfields appear in memory is implementation dependent, so in some cases the cgo tool produced results that were silently incorrect.


Runtime:
The new runtime/metrics package introduces a stable interface for reading implementation-defined metrics from the Go runtime. It supersedes existing functions like runtime.ReadMemStats and debug.GCStats and is significantly more general and efficient. See the package documentation for more details.

Setting the GODEBUG environment variable to inittrace=1 now causes the runtime to emit a single line to standard error for each package init, summarizing its execution time and memory allocation.

This trace can be used to find bottlenecks or regressions in Go startup performance. The GODEBUG documentation describes the format.

On Linux, the runtime now defaults to releasing memory to the operating system promptly (using MADV_DONTNEED), rather than lazily when the operating system is under memory pressure (using MADV_FREE).
This means process-level memory statistics like RSS will more accurately reflect the amount of physical memory being used by Go processes. Systems that are currently using GODEBUG=madvdontneed=1 to improve memory monitoring behavior no longer need to set this environment variable.

Go 1.16 fixes a discrepancy between the race detector and the Go memory model. The race detector now more precisely follows the channel synchronization rules of the memory model. As a result, the detector may now report races it previously missed.


Compiler:
The compiler can now inline functions with 
  • non-labeled for loops, 
  • method values, 
  • and type switches.
The inliner can also detect more indirect calls where inlining is possible.



Core library:
The new io/fs package defines the fs.FS interface.
Deprecation of io/ioutil.

Nov 18, 2019

[Go][Delve] Remote debugging command

While debugging, use this argument as in go compile command:
$ go build -gcflags="all=-N -l"  // -N: disable optimizations -l: disable inlining

Delve on remote machine should be the same version of local machine, as well as the go compiler.

Start the process through delve on remote machine
$ dlv --listen=:2345 --headless=true --api-version=2 exec ./application

or attach to the running process on remote machine:
$ dlv --listen=:2345 --headless=true --api-version=2 attach <pid>


Docker run command:
$ dk run --security-opt="apparmor=unconfined" --cap-add=SYS_PTRACE

Docker file:
FROM golang:1.11.5-alpine3.8 AS build-env
 
ENV CGO_ENABLED 0
 
# Allow Go to retreive the dependencies for the build step
RUN apk add --no-cache git
 
WORKDIR /goland-debugging/
ADD . /goland-debugging/
 
RUN go build -o /goland-debugging/srv .
 
# Get Delve from a GOPATH not from a Go Modules project
WORKDIR /go/src/
RUN go get github.com/go-delve/delve/cmd/dlv
 
# final stage
FROM alpine:3.8
 
WORKDIR /
COPY --from=build-env /goland-debugging/srv /
COPY --from=build-env /go/bin/dlv /
 
EXPOSE 8080 40000
 
CMD ["/dlv", "--listen=:40000", "--headless=true", "--api-version=2", "exec", "/srv"]

Sep 2, 2019

[Go] instrumentation

Reference:
GopherCon 2019: Jason Keene - Dynamically Instrumenting Go Programs
http://www.brendangregg.com/blog/2016-03-05/linux-bpf-superpowers.html

Book:
BPF Performance Tools


Probe Effect (be aware)

Types of instrument:
  • Static and Always ON
    Logging, metrics, distributed tracing
  • Static and Requires Activation
    Leveled logging, pprof, usdt probes( https://lwn.net/Articles/753601 )
  • Dynamic Instrumentation
    Debuggers, tracers, uprobes/bpf


Streetlight Effect (where static trace is ON, be aware)

Thus we need dynamic instrumentation.

  1. ptrace/debuggers
    GDB ( http://vsdmars.blogspot.com/2018/10/clinux-debugging.html )
  2. go-bpf
    https://github.com/iovisor/gobpf

Jun 25, 2018

[Go] stack memory size

runtime/debug/#SetMaxStack

stacks are not essential for Go programs.
Go compiler/runtime can allocate all memory block on heap.
Supporting stacks is just to make Go programs run more efficiently:
  • allocating memory blocks on stacks is much faster than on heap. 
  • memory blocks allocated on a stack don't need to be garbage collected. 
  • stack memory blocks are more CPU cache friendly than heap ones.

Some facts:
  • If a field of a struct value escapes to heap, then the whole struct value will also escape to heap.
  • If an element of an array value escapes to heap, then the whole array value will also escape to heap.
  • If an element of a slice value escapes to heap, then all the elements of the slice will also escape to heap.
  • If a non-nil slice value escapes to heap, then its elements will also escape to heap.
A memory block created by calling new function may be allocated on heap or stacks. This is different to C++.

runtime.KeepAlive function calls are often needed if unsafe pointers are involved.

To check which local values (value parts) will escape to heap at run time:
$ go build -gcflags -m