more text
This commit is contained in:
tarun-elango
@@ -0,0 +1,486 @@
|
||||
# Go: Introduction and Setup
|
||||
|
||||
## Learning Objectives
|
||||
|
||||
- Understand what Go is and why it was created.
|
||||
- Build a mental model of the Go compiler, runtime, and toolchain.
|
||||
- Install Go and verify that the environment is correct.
|
||||
- Understand modules, packages, and the shape of a simple Go project.
|
||||
- Read and run a minimal Go program with confidence.
|
||||
- Recognize the kinds of systems Go is especially good at building.
|
||||
|
||||
## Why Learn Go
|
||||
|
||||
Go, also called Golang, is a statically typed compiled language designed for software that needs to be simple to read, fast to build, easy to deploy, and reliable under load.
|
||||
|
||||
At first glance, Go can look smaller than languages like Java, C++, or Rust. That is not an accident. Go was deliberately designed to remove a lot of language surface area so engineers spend less time debating style and more time shipping understandable systems.
|
||||
|
||||
### What Problem Go Was Trying to Solve
|
||||
|
||||
Go came from a practical frustration inside large software teams. The designers wanted a language that would:
|
||||
|
||||
- compile quickly even for large codebases
|
||||
- make dependency management and builds straightforward
|
||||
- support concurrency without forcing every engineer to become a threads expert
|
||||
- produce binaries that are easy to deploy in servers and containers
|
||||
- encourage code that many people can read, not just the original author
|
||||
|
||||
This makes Go especially strong in infrastructure and backend work:
|
||||
|
||||
- HTTP APIs and web services
|
||||
- reverse proxies and gateways
|
||||
- distributed systems components
|
||||
- command-line tools
|
||||
- data pipelines and background workers
|
||||
- cloud-native control planes, schedulers, and operators
|
||||
|
||||
Projects like Docker, Kubernetes, Terraform, Prometheus, and many internal backend platforms rely on Go for exactly these reasons.
|
||||
|
||||
### Why Go Feels Different
|
||||
|
||||
Many languages try to give you more expressive power by adding more features. Go often does the opposite. It removes features that create ambiguity or deep complexity.
|
||||
|
||||
For example:
|
||||
|
||||
- there are no classes in the traditional Java sense
|
||||
- inheritance is replaced by composition
|
||||
- exceptions are replaced by explicit error values
|
||||
- formatting is standardized by tooling rather than team debate
|
||||
|
||||
That tradeoff matters. Go is not trying to be the most flexible language for every programming style. It is trying to be a dependable language for teams building production systems.
|
||||
|
||||
## The Go Philosophy in Practical Terms
|
||||
|
||||
Before learning syntax, it helps to understand the values the language is optimized for.
|
||||
|
||||
### Simplicity Over Cleverness
|
||||
|
||||
Go code is meant to be read quickly. If a solution is slightly more verbose but much easier to understand, Go generally prefers that version.
|
||||
|
||||
In real systems this matters more than beginners often expect. Most production code is maintained by someone who did not originally write it. The simpler the code reads, the lower the long-term cost.
|
||||
|
||||
### Fast Feedback Loops
|
||||
|
||||
Go's toolchain is intentionally fast. Building, testing, and formatting are part of the normal workflow rather than optional extras.
|
||||
|
||||
That speed changes engineering behavior. Developers run tests more often, refactor with more confidence, and keep tighter iteration loops.
|
||||
|
||||
### Built-In Tooling Culture
|
||||
|
||||
Some ecosystems depend heavily on third-party tools for basic workflow consistency. Go bakes a large part of that workflow into the language toolchain itself.
|
||||
|
||||
Common tasks use the standard `go` command:
|
||||
|
||||
- `go run`
|
||||
- `go build`
|
||||
- `go test`
|
||||
- `go fmt`
|
||||
- `go mod`
|
||||
- `go doc`
|
||||
|
||||
This is one reason Go projects often feel operationally clean compared with ecosystems that require many layers of build tooling.
|
||||
|
||||
## Where Go Fits in a System
|
||||
|
||||
Go is not the answer to every problem. It shines in a particular band of workloads.
|
||||
|
||||
### Strong Fits
|
||||
|
||||
- backend services that need predictable performance
|
||||
- network servers handling many concurrent requests
|
||||
- tools distributed as a single binary
|
||||
- microservices that need fast startup and straightforward containerization
|
||||
- platform engineering components such as controllers, schedulers, and sidecars
|
||||
|
||||
### Weaker Fits
|
||||
|
||||
- highly dynamic scripting where a REPL-first workflow matters more than static guarantees
|
||||
- extremely low-level systems programming where full control over memory layout is critical
|
||||
- domains where advanced compile-time type programming is a core need
|
||||
|
||||
That does not mean Go cannot be used there. It means the language was optimized for another center of gravity.
|
||||
|
||||
## Mental Model: From Source Code to Running Program
|
||||
|
||||
Many beginners treat a language as just syntax. That is too shallow for systems work. You should understand the path from source files to a running process.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[.go source files] --> B[go fmt]
|
||||
B --> C[go build]
|
||||
C --> D[Compiler]
|
||||
D --> E[Linker]
|
||||
E --> F[Single binary]
|
||||
F --> G[OS process]
|
||||
G --> H[Go runtime starts]
|
||||
H --> I[main.main executes]
|
||||
```
|
||||
|
||||
### What Happens Internally
|
||||
|
||||
When you run `go build`, several important things happen:
|
||||
|
||||
1. The compiler parses and type-checks your code.
|
||||
2. It compiles packages into machine code for the target platform.
|
||||
3. The linker combines your code, the standard library, and runtime support into a binary.
|
||||
4. When the binary starts, the Go runtime initializes memory management, the scheduler, and other low-level runtime state.
|
||||
5. Your `main` package starts executing from `main.main()`.
|
||||
|
||||
This is one reason Go is attractive operationally. The result is often a single deployable binary with few moving parts.
|
||||
|
||||
### Why the Runtime Exists in a Compiled Language
|
||||
|
||||
Go is compiled, but it still has a runtime. That runtime is not a virtual machine like the JVM. It is a support layer linked into the binary.
|
||||
|
||||
It is responsible for things such as:
|
||||
|
||||
- garbage collection
|
||||
- goroutine scheduling
|
||||
- stack growth
|
||||
- map and channel internals
|
||||
- panic handling
|
||||
- parts of reflection and interface support
|
||||
|
||||
In practice, this means Go gives you a native binary while still providing higher-level language features that would be painful to implement manually.
|
||||
|
||||
## Installing Go
|
||||
|
||||
The official distribution is available from the Go project site, and package managers also work well on macOS and Linux.
|
||||
|
||||
After installation, confirm the toolchain is available:
|
||||
|
||||
```bash
|
||||
go version
|
||||
go env GOROOT GOPATH
|
||||
```
|
||||
|
||||
### What These Values Mean
|
||||
|
||||
- `GOROOT` points to the Go installation itself.
|
||||
- `GOPATH` is the old workspace model and still exists for cache and tool behavior, but modern projects should use modules.
|
||||
|
||||
The most important shift to understand is this:
|
||||
|
||||
- old Go development often centered around `GOPATH`
|
||||
- modern Go development centers around `go.mod`
|
||||
|
||||
If you are learning Go today, think in modules first.
|
||||
|
||||
## Your First Module
|
||||
|
||||
A Go module is the unit of versioning and dependency management.
|
||||
|
||||
Create a project:
|
||||
|
||||
```bash
|
||||
mkdir hello-go
|
||||
cd hello-go
|
||||
go mod init example.com/hello-go
|
||||
```
|
||||
|
||||
This creates a `go.mod` file. That file tells Go two important things:
|
||||
|
||||
- the module path
|
||||
- the dependency set for the project
|
||||
|
||||
Example:
|
||||
|
||||
```go
|
||||
module example.com/hello-go
|
||||
|
||||
go 1.25.0
|
||||
```
|
||||
|
||||
The exact Go version may differ, but the idea is the same.
|
||||
|
||||
### Why Modules Exist
|
||||
|
||||
Without a module system, dependency versions become fragile and hard to reproduce. A module gives Go enough information to:
|
||||
|
||||
- resolve imports
|
||||
- fetch dependencies
|
||||
- build the same project consistently on other machines
|
||||
|
||||
In real backend systems, reproducible dependency state is not optional. It is part of shipping dependable software.
|
||||
|
||||
## A Minimal Go Program
|
||||
|
||||
Create `main.go`:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
fmt.Println("hello, Go")
|
||||
}
|
||||
```
|
||||
|
||||
Run it:
|
||||
|
||||
```bash
|
||||
go run .
|
||||
```
|
||||
|
||||
Build it:
|
||||
|
||||
```bash
|
||||
go build .
|
||||
```
|
||||
|
||||
### Read the Program Line by Line
|
||||
|
||||
`package main`
|
||||
|
||||
- Every Go file belongs to a package.
|
||||
- The special package `main` produces an executable program.
|
||||
|
||||
`import "fmt"`
|
||||
|
||||
- Packages must be imported explicitly.
|
||||
- `fmt` is part of the standard library and handles formatted I/O.
|
||||
|
||||
`func main()`
|
||||
|
||||
- Functions are declared with `func`.
|
||||
- `main` is the entry point for an executable.
|
||||
|
||||
`fmt.Println(...)`
|
||||
|
||||
- A package-qualified function call.
|
||||
- The standard library is intentionally strong, so you will use packages like `fmt`, `net/http`, `context`, `time`, and `encoding/json` constantly.
|
||||
|
||||
### Why Go Is Strict About Unused Imports and Variables
|
||||
|
||||
Go rejects unused local variables and unused imports. At first this can feel annoying. In practice it keeps code cleaner and reduces confusion while refactoring.
|
||||
|
||||
In long-lived services, that strictness is useful. It prevents stale code from quietly accumulating.
|
||||
|
||||
## Understanding Packages and Files Early
|
||||
|
||||
A common beginner mistake is to think each file is independent. In Go, files in the same folder and package are compiled together.
|
||||
|
||||
That means this is one logical package:
|
||||
|
||||
```text
|
||||
myservice/
|
||||
handlers.go
|
||||
server.go
|
||||
config.go
|
||||
```
|
||||
|
||||
if all files declare the same package name.
|
||||
|
||||
### Simple Project Shape
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[myservice] --> B[go.mod]
|
||||
A --> C[main.go]
|
||||
A --> D[internal]
|
||||
D --> E[httpapi]
|
||||
D --> F[store]
|
||||
A --> G[pkg]
|
||||
G --> H[client]
|
||||
```
|
||||
|
||||
This diagram introduces a pattern you will see often:
|
||||
|
||||
- `main.go` or `cmd/...` starts the program
|
||||
- `internal/...` holds application-private packages
|
||||
- `pkg/...` is sometimes used for reusable exported packages, though many teams avoid it unless it adds real clarity
|
||||
|
||||
Do not overcomplicate layout early. Start simple and split packages only when the structure earns its keep.
|
||||
|
||||
## Core Tooling You Should Use Immediately
|
||||
|
||||
Go learning goes faster when you treat tooling as part of the language.
|
||||
|
||||
### `go fmt`
|
||||
|
||||
```bash
|
||||
go fmt ./...
|
||||
```
|
||||
|
||||
This formats your code according to the standard Go style.
|
||||
|
||||
Why it exists:
|
||||
|
||||
- removes formatting debates
|
||||
- keeps diffs smaller and more readable
|
||||
- makes code look familiar across projects
|
||||
|
||||
### `go test`
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
This runs tests across packages.
|
||||
|
||||
Even before you know advanced testing, you should get used to this command. In Go, running the full package test set is normal, not exceptional.
|
||||
|
||||
### `go doc`
|
||||
|
||||
```bash
|
||||
go doc fmt.Println
|
||||
```
|
||||
|
||||
This helps you inspect package and symbol documentation from the command line.
|
||||
|
||||
### `go env`
|
||||
|
||||
```bash
|
||||
go env
|
||||
```
|
||||
|
||||
This prints environment details the toolchain is using. It is extremely helpful when debugging build or dependency issues.
|
||||
|
||||
## Zero Values: A Go Idea You Should Learn Early
|
||||
|
||||
Go gives every variable a default zero value.
|
||||
|
||||
Examples:
|
||||
|
||||
- `0` for integers
|
||||
- `false` for booleans
|
||||
- `""` for strings
|
||||
- `nil` for pointers, slices, maps, interfaces, channels, and function values
|
||||
|
||||
Why this exists:
|
||||
|
||||
- it reduces uninitialized-memory style bugs
|
||||
- it makes declarations cheap and predictable
|
||||
- it encourages data structures that are usable in a default state when designed well
|
||||
|
||||
Example:
|
||||
|
||||
```go
|
||||
var retries int
|
||||
var enabled bool
|
||||
var name string
|
||||
|
||||
fmt.Println(retries, enabled, name)
|
||||
```
|
||||
|
||||
In production code, zero values matter all the time. For example, a `sync.Mutex` or `bytes.Buffer` works correctly without manual initialization. That is a subtle but powerful ergonomics win.
|
||||
|
||||
## The Standard Library Is Part of the Language Experience
|
||||
|
||||
One reason Go feels productive in backend systems is that you can build a lot with the standard library alone.
|
||||
|
||||
Packages you will quickly rely on include:
|
||||
|
||||
- `fmt` for formatting and printing
|
||||
- `errors` for error handling helpers
|
||||
- `time` for deadlines, timers, and durations
|
||||
- `context` for cancellation and request scope
|
||||
- `net/http` for servers and clients
|
||||
- `encoding/json` for JSON encoding and decoding
|
||||
- `os` and `io` for file and stream operations
|
||||
- `sync` for mutexes and synchronization primitives
|
||||
|
||||
This matters because fewer external dependencies often means:
|
||||
|
||||
- easier upgrades
|
||||
- fewer version conflicts
|
||||
- less supply chain risk
|
||||
- more consistent team knowledge
|
||||
|
||||
## How Go Is Used in Real Systems
|
||||
|
||||
It helps to attach the language to actual engineering tasks rather than seeing it as abstract syntax.
|
||||
|
||||
### Backend API Service
|
||||
|
||||
A Go service might:
|
||||
|
||||
- listen for HTTP requests
|
||||
- parse JSON into structs
|
||||
- validate input
|
||||
- call a database or downstream service
|
||||
- return a JSON response
|
||||
|
||||
Go is strong here because:
|
||||
|
||||
- request handling maps naturally to goroutines
|
||||
- binaries are simple to deploy
|
||||
- startup is fast
|
||||
- memory and CPU use are usually predictable enough for service operation
|
||||
|
||||
### Distributed Systems Component
|
||||
|
||||
A scheduler, controller, queue worker, or service discovery agent often needs:
|
||||
|
||||
- concurrency
|
||||
- networking
|
||||
- serialization
|
||||
- low operational complexity
|
||||
- strong observability hooks
|
||||
|
||||
Go's standard library and runtime model fit that space very well.
|
||||
|
||||
### CLI and Platform Tooling
|
||||
|
||||
Internal developer tools are another strong Go use case. A single statically linked binary is easy to ship across machines and CI environments.
|
||||
|
||||
## Common Mistakes and Misconceptions
|
||||
|
||||
### Mistake: Treating Go Like Tiny Java or Tiny Python
|
||||
|
||||
Go is its own language with its own design center. If you constantly try to recreate class-heavy Java patterns or highly dynamic Python patterns, the code usually becomes awkward.
|
||||
|
||||
### Mistake: Ignoring the Toolchain
|
||||
|
||||
Go is not just syntax plus a compiler. The standard workflow is a major part of the language experience. Learn `go build`, `go test`, `go fmt`, and `go mod` early.
|
||||
|
||||
### Mistake: Overengineering Project Structure on Day One
|
||||
|
||||
Beginners sometimes create many directories and interfaces before the project has real complexity. Start with a small module and grow structure as the codebase proves it needs it.
|
||||
|
||||
### Mistake: Thinking "Compiled" Means "No Runtime"
|
||||
|
||||
Go produces native binaries, but those binaries include runtime support for garbage collection, scheduling, and other language features.
|
||||
|
||||
### Mistake: Treating Modules and Packages as the Same Thing
|
||||
|
||||
They are related but different.
|
||||
|
||||
- a module is a versioned collection of packages
|
||||
- a package is a unit of code organization and namespace
|
||||
|
||||
That distinction becomes important once projects grow.
|
||||
|
||||
## Practical Intuition to Carry Forward
|
||||
|
||||
At this stage, the most important thing is not memorizing every command. It is building a mental model:
|
||||
|
||||
- Go is optimized for readable, deployable, concurrent systems software.
|
||||
- The toolchain is part of the language culture.
|
||||
- Modules manage dependencies.
|
||||
- Packages organize code.
|
||||
- A Go program becomes a native process with runtime support linked in.
|
||||
|
||||
If you understand those ideas, the language details in the next file will make much more sense.
|
||||
|
||||
## Real-World Use Cases
|
||||
|
||||
- Building a JSON API server for a mobile app backend.
|
||||
- Writing a queue consumer that processes jobs concurrently.
|
||||
- Creating an internal deployment CLI distributed as one binary.
|
||||
- Implementing a control-plane component that watches cluster state and reconciles resources.
|
||||
|
||||
## Summary
|
||||
|
||||
Go exists to make production engineering simpler, especially for backend and infrastructure software. Its power is not just in syntax. It comes from the combination of a clear language, a fast toolchain, a strong standard library, native binaries, and a runtime designed for concurrency.
|
||||
|
||||
You should now be comfortable with the big picture:
|
||||
|
||||
- why Go exists
|
||||
- how source code becomes a running binary
|
||||
- how to install and verify the toolchain
|
||||
- how modules and packages fit together
|
||||
- how to create and run a first Go program
|
||||
|
||||
The next step is learning the language itself: values, types, control flow, data structures, functions, methods, structs, interfaces, and error handling.
|
||||
@@ -0,0 +1,717 @@
|
||||
# Go: Core Language Fundamentals
|
||||
|
||||
## Learning Objectives
|
||||
|
||||
- Understand Go's type system and value model.
|
||||
- Use variables, constants, control flow, and core data structures idiomatically.
|
||||
- Understand how slices, maps, strings, and pointers actually behave.
|
||||
- Write functions, methods, structs, and interfaces with practical clarity.
|
||||
- Handle errors the Go way instead of forcing exception-style thinking onto the language.
|
||||
- Build intuition for memory, receivers, and abstraction choices in real backend code.
|
||||
|
||||
## Start with the Right Mental Model
|
||||
|
||||
Go is a statically typed language centered on values. That sounds simple, but it shapes almost everything.
|
||||
|
||||
In practice, most Go code is about:
|
||||
|
||||
- creating values
|
||||
- transforming values
|
||||
- passing values between functions
|
||||
- attaching behavior to named types
|
||||
- returning explicit errors when something goes wrong
|
||||
|
||||
Go is not a class hierarchy language. It is not an exception-driven language. It is not a macro-heavy metaprogramming language. It is a language that tries to keep data flow and control flow visible.
|
||||
|
||||
That visibility is why production Go code can be easy to reason about even when the system itself is large.
|
||||
|
||||
## Variables, Constants, and Zero Values
|
||||
|
||||
### What They Are
|
||||
|
||||
Variables store values whose contents can change. Constants are fixed compile-time values.
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import "fmt"
|
||||
|
||||
func main() {
|
||||
var retries int = 3
|
||||
timeoutSeconds := 10
|
||||
const serviceName = "billing-api"
|
||||
|
||||
fmt.Println(retries, timeoutSeconds, serviceName)
|
||||
}
|
||||
```
|
||||
|
||||
### Why Go Has Multiple Declaration Styles
|
||||
|
||||
Go gives you a few common forms:
|
||||
|
||||
- `var name Type` when the type matters or you want the zero value
|
||||
- `var name Type = value` when you want explicitness
|
||||
- `name := value` inside functions when the type is obvious
|
||||
- `const` for fixed values known at compile time
|
||||
|
||||
Idiomatic Go uses short declaration heavily inside functions because it keeps code readable without losing type safety.
|
||||
|
||||
### Zero Values Matter More Than They Seem
|
||||
|
||||
Every variable has a useful default value.
|
||||
|
||||
- numbers become `0`
|
||||
- booleans become `false`
|
||||
- strings become `""`
|
||||
- pointers, slices, maps, interfaces, channels, and functions become `nil`
|
||||
|
||||
This matters because many Go types are designed so the zero value is already valid.
|
||||
|
||||
Examples:
|
||||
|
||||
- a `bytes.Buffer` can be used immediately
|
||||
- a `sync.Mutex` can be locked immediately
|
||||
- a `time.Time` has a well-defined zero state
|
||||
|
||||
That design lowers initialization friction and reduces a whole class of bugs.
|
||||
|
||||
### When to Use `const`
|
||||
|
||||
Use constants for values that are conceptually fixed:
|
||||
|
||||
- protocol names
|
||||
- status labels
|
||||
- numeric tuning knobs that should not change at runtime
|
||||
|
||||
Do not use `const` just because a variable happens not to change in one function. Prefer clarity over ceremony.
|
||||
|
||||
### `iota` and Enumerated Constants
|
||||
|
||||
Go does not have enums in the Java sense, but it uses typed constants effectively.
|
||||
|
||||
```go
|
||||
type JobState int
|
||||
|
||||
const (
|
||||
JobPending JobState = iota
|
||||
JobRunning
|
||||
JobDone
|
||||
JobFailed
|
||||
)
|
||||
```
|
||||
|
||||
Why this exists:
|
||||
|
||||
- it gives you readable symbolic values
|
||||
- it keeps the type distinct from unrelated integers
|
||||
- it works well with switches, logging, and serialization helpers
|
||||
|
||||
## Control Flow: Small Set, High Clarity
|
||||
|
||||
Go intentionally keeps control flow simple.
|
||||
|
||||
### `if`
|
||||
|
||||
```go
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
```
|
||||
|
||||
This pattern appears constantly in Go because errors are explicit return values.
|
||||
|
||||
You will also see scoped initialization inside `if`:
|
||||
|
||||
```go
|
||||
if user, err := repo.Find(ctx, id); err != nil {
|
||||
return err
|
||||
} else {
|
||||
fmt.Println(user.Name)
|
||||
}
|
||||
```
|
||||
|
||||
Use this sparingly. It is useful when the scope should remain local, but overused it can make code denser than necessary.
|
||||
|
||||
### `for`
|
||||
|
||||
Go has one looping keyword: `for`.
|
||||
|
||||
```go
|
||||
for i := 0; i < 3; i++ {
|
||||
fmt.Println(i)
|
||||
}
|
||||
|
||||
for condition {
|
||||
fmt.Println("acts like while")
|
||||
break
|
||||
}
|
||||
|
||||
for {
|
||||
break
|
||||
}
|
||||
```
|
||||
|
||||
Why this is useful:
|
||||
|
||||
- fewer looping forms to remember
|
||||
- easier syntax surface for reading code
|
||||
- the language avoids duplicated constructs with minor differences
|
||||
|
||||
### `range`
|
||||
|
||||
`range` iterates over slices, arrays, strings, maps, and channels.
|
||||
|
||||
```go
|
||||
nums := []int{10, 20, 30}
|
||||
for index, value := range nums {
|
||||
fmt.Println(index, value)
|
||||
}
|
||||
```
|
||||
|
||||
Important practical details:
|
||||
|
||||
- ranging over a map does not guarantee order
|
||||
- ranging over a string gives runes, not raw bytes
|
||||
- ranging over a channel continues until the channel is closed
|
||||
|
||||
### `switch`
|
||||
|
||||
Go's `switch` is more flexible than many beginners expect.
|
||||
|
||||
```go
|
||||
switch state {
|
||||
case JobPending:
|
||||
fmt.Println("queued")
|
||||
case JobRunning:
|
||||
fmt.Println("in progress")
|
||||
default:
|
||||
fmt.Println("terminal state")
|
||||
}
|
||||
```
|
||||
|
||||
There is also expression-less `switch`, which is a readable alternative to long `if` ladders.
|
||||
|
||||
## Core Data Structures
|
||||
|
||||
### Arrays
|
||||
|
||||
Arrays have fixed length and are value types.
|
||||
|
||||
```go
|
||||
var ports [3]int
|
||||
ports[0] = 8080
|
||||
```
|
||||
|
||||
Arrays exist, but in day-to-day Go you use slices much more often.
|
||||
|
||||
Why arrays still matter:
|
||||
|
||||
- they are the underlying foundation for slices
|
||||
- fixed-size data can be useful for performance-sensitive code
|
||||
- array values emphasize that size is part of the type
|
||||
|
||||
### Slices
|
||||
|
||||
A slice is a small descriptor pointing at an underlying array.
|
||||
|
||||
```go
|
||||
ids := []int{101, 102, 103}
|
||||
ids = append(ids, 104)
|
||||
```
|
||||
|
||||
Internally, a slice conceptually contains:
|
||||
|
||||
- a pointer to backing storage
|
||||
- a length
|
||||
- a capacity
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Slice Header] --> B[Pointer]
|
||||
A --> C[Length]
|
||||
A --> D[Capacity]
|
||||
B --> E[Backing Array]
|
||||
```
|
||||
|
||||
### Why Slices Exist
|
||||
|
||||
They give you dynamic-seeming sequences without hiding memory behavior completely. They are a practical middle ground:
|
||||
|
||||
- easier to use than raw arrays
|
||||
- cheaper than many boxed collection abstractions
|
||||
- explicit enough that performance behavior is still understandable
|
||||
|
||||
### How `append` Works Internally
|
||||
|
||||
If capacity is available, `append` writes into the existing backing array. If capacity is exhausted, Go allocates a new array, copies the old contents, and returns a new slice header.
|
||||
|
||||
That means two things:
|
||||
|
||||
- appending may reallocate
|
||||
- slices sharing backing storage can affect each other unexpectedly
|
||||
|
||||
Example:
|
||||
|
||||
```go
|
||||
base := []int{1, 2, 3, 4}
|
||||
a := base[:2]
|
||||
b := base[:3]
|
||||
|
||||
a = append(a, 99)
|
||||
fmt.Println(base, a, b)
|
||||
```
|
||||
|
||||
This surprises many beginners because `a` may overwrite data visible through `b` if both still share the same backing array.
|
||||
|
||||
### Maps
|
||||
|
||||
Maps are Go's hash table type.
|
||||
|
||||
```go
|
||||
counts := map[string]int{
|
||||
"ok": 12,
|
||||
"error": 3,
|
||||
}
|
||||
|
||||
counts["retry"]++
|
||||
```
|
||||
|
||||
Why maps exist:
|
||||
|
||||
- fast key lookup
|
||||
- natural representation for counters, indexes, sets, and lookup tables
|
||||
|
||||
Important details:
|
||||
|
||||
- reading a missing key returns the value type's zero value
|
||||
- use the two-result form when absence matters
|
||||
- map iteration order is deliberately not stable
|
||||
|
||||
```go
|
||||
value, ok := counts["missing"]
|
||||
fmt.Println(value, ok)
|
||||
```
|
||||
|
||||
Maps are reference-like structures managed by the runtime. A nil map can be read from, but writing to it panics.
|
||||
|
||||
### Strings, Bytes, and Runes
|
||||
|
||||
Go strings are immutable sequences of bytes, usually holding UTF-8 encoded text.
|
||||
|
||||
This distinction matters:
|
||||
|
||||
- `byte` is an alias for `uint8`
|
||||
- `rune` is an alias for `int32` and represents a Unicode code point
|
||||
|
||||
```go
|
||||
message := "Go cafe"
|
||||
fmt.Println(len(message))
|
||||
|
||||
for _, r := range message {
|
||||
fmt.Printf("%c\n", r)
|
||||
}
|
||||
```
|
||||
|
||||
Why this matters in real systems:
|
||||
|
||||
- HTTP payloads and file formats are byte-oriented
|
||||
- user-visible text is Unicode-oriented
|
||||
- confusing the two leads to subtle bugs in validation, truncation, and indexing
|
||||
|
||||
If you need mutable byte data, use `[]byte`, not `string`.
|
||||
|
||||
## Functions: Multiple Return Values and Defer
|
||||
|
||||
### Functions as the Unit of Composition
|
||||
|
||||
Go prefers straightforward function composition over deep inheritance trees.
|
||||
|
||||
```go
|
||||
func parsePort(value string) (int, error) {
|
||||
port, err := strconv.Atoi(value)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("parse port %q: %w", value, err)
|
||||
}
|
||||
|
||||
return port, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Why Multiple Return Values Exist
|
||||
|
||||
This is one of the most important Go design choices. Instead of exceptions for normal failure, functions can return both a result and an error.
|
||||
|
||||
Benefits:
|
||||
|
||||
- the failure path is visible in the function signature
|
||||
- callers must consciously handle errors
|
||||
- control flow stays explicit
|
||||
|
||||
In backend systems, this reduces the hidden control-flow jumps that exception-heavy code can create.
|
||||
|
||||
### `defer`
|
||||
|
||||
`defer` schedules a function call to run when the surrounding function returns.
|
||||
|
||||
```go
|
||||
func readConfig(path string) error {
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// read file
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
Why it exists:
|
||||
|
||||
- resource cleanup should be hard to forget
|
||||
- cleanup code often belongs near acquisition code
|
||||
|
||||
Internally, deferred calls are recorded and executed in last-in, first-out order when the function exits.
|
||||
|
||||
Use `defer` for correctness first. In very hot paths, you may care about its cost, but in most business logic the clarity win is worth it.
|
||||
|
||||
## Pointers and Memory Intuition
|
||||
|
||||
### What a Pointer Is
|
||||
|
||||
A pointer stores the address of another value.
|
||||
|
||||
```go
|
||||
func increment(value *int) {
|
||||
*value++
|
||||
}
|
||||
```
|
||||
|
||||
Go pointers exist so you can:
|
||||
|
||||
- mutate shared state intentionally
|
||||
- avoid copying very large values when appropriate
|
||||
- define methods that update a receiver
|
||||
|
||||
### Why Go Pointers Feel Safer Than C Pointers
|
||||
|
||||
Go allows pointers, but it removes several sharp edges:
|
||||
|
||||
- no pointer arithmetic
|
||||
- garbage collection manages object lifetime
|
||||
- type safety is preserved
|
||||
|
||||
This is a very deliberate design choice. Go wants the practical utility of pointers without turning everyday backend code into manual memory management.
|
||||
|
||||
### Stack, Heap, and Escape Analysis
|
||||
|
||||
Do not think of Go as "everything is on the heap." The compiler decides where values live.
|
||||
|
||||
- if a value can stay local safely, it may live on the stack
|
||||
- if it must outlive the local frame or be referenced elsewhere, it may escape to the heap
|
||||
|
||||
This compiler decision process is called escape analysis.
|
||||
|
||||
Why it matters:
|
||||
|
||||
- heap allocation can increase GC pressure
|
||||
- unnecessary pointer-heavy designs can make code slower and harder to reason about
|
||||
|
||||
You usually do not hand-place objects yourself. Instead, you write clear code and learn enough about allocation behavior to avoid obviously wasteful patterns.
|
||||
|
||||
## Structs and Methods
|
||||
|
||||
### Structs: Go's Primary Data Modeling Tool
|
||||
|
||||
Structs group related fields.
|
||||
|
||||
```go
|
||||
type User struct {
|
||||
ID int64
|
||||
Name string
|
||||
Email string
|
||||
}
|
||||
```
|
||||
|
||||
Why structs matter:
|
||||
|
||||
- they model domain entities cleanly
|
||||
- they work naturally with JSON, databases, and configuration
|
||||
- they keep data layout explicit
|
||||
|
||||
### Methods
|
||||
|
||||
Methods are functions attached to a type.
|
||||
|
||||
```go
|
||||
type Counter struct {
|
||||
value int
|
||||
}
|
||||
|
||||
func (c *Counter) Inc() {
|
||||
c.value++
|
||||
}
|
||||
|
||||
func (c Counter) Value() int {
|
||||
return c.value
|
||||
}
|
||||
```
|
||||
|
||||
### Value Receiver or Pointer Receiver
|
||||
|
||||
Use a pointer receiver when:
|
||||
|
||||
- the method mutates the receiver
|
||||
- the struct is large enough that copying is undesirable
|
||||
- consistency across methods is clearer with pointers
|
||||
|
||||
Use a value receiver when:
|
||||
|
||||
- the value is small and conceptually immutable
|
||||
- copying is cheap and expected
|
||||
|
||||
In real projects, consistency matters. If a type usually uses pointer receivers, use them across the method set unless there is a strong reason not to.
|
||||
|
||||
### Composition Over Inheritance
|
||||
|
||||
Go does not have class inheritance. Instead, it leans on composition and embedding.
|
||||
|
||||
```go
|
||||
type Logger struct{}
|
||||
|
||||
func (Logger) Info(msg string) {
|
||||
fmt.Println("INFO:", msg)
|
||||
}
|
||||
|
||||
type Server struct {
|
||||
Logger
|
||||
addr string
|
||||
}
|
||||
```
|
||||
|
||||
Embedding can promote fields and methods, but use it to express a real structural relationship, not to imitate inheritance mechanically.
|
||||
|
||||
## Interfaces: Small, Behavioral, and Implicit
|
||||
|
||||
### What an Interface Is
|
||||
|
||||
An interface describes behavior, not concrete data layout.
|
||||
|
||||
```go
|
||||
type Store interface {
|
||||
Save(ctx context.Context, user User) error
|
||||
}
|
||||
```
|
||||
|
||||
Any type with a matching method set satisfies the interface automatically.
|
||||
|
||||
### Why Implicit Satisfaction Exists
|
||||
|
||||
Go avoids the ceremony of explicit `implements` declarations. The benefit is that interfaces are lightweight and decoupled from concrete types.
|
||||
|
||||
That means you can define interfaces where they are needed, usually at the consumer side.
|
||||
|
||||
Example:
|
||||
|
||||
```go
|
||||
type UserService struct {
|
||||
store Store
|
||||
}
|
||||
```
|
||||
|
||||
The service depends on behavior, not a particular database implementation.
|
||||
|
||||
### How Interfaces Work Internally
|
||||
|
||||
Conceptually, an interface value holds:
|
||||
|
||||
- the dynamic concrete type
|
||||
- the concrete value or pointer value
|
||||
|
||||
This is why the "nil interface" pitfall exists. An interface can contain a typed nil pointer and still be non-nil as an interface value.
|
||||
|
||||
That pitfall shows up in logging, error returns, and optional dependency wiring.
|
||||
|
||||
### Keep Interfaces Small
|
||||
|
||||
Good Go interfaces are often tiny.
|
||||
|
||||
```go
|
||||
type Clock interface {
|
||||
Now() time.Time
|
||||
}
|
||||
```
|
||||
|
||||
Why small interfaces are better:
|
||||
|
||||
- easier to implement
|
||||
- easier to test
|
||||
- less coupling
|
||||
- behavior stays focused
|
||||
|
||||
Large interfaces usually signal that an abstraction was designed too early or at the wrong level.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[HTTP Handler] --> B[UserService interface]
|
||||
B --> C[PostgresUserService]
|
||||
B --> D[InMemoryUserService]
|
||||
```
|
||||
|
||||
## Generics: Useful, But Not the Center of Go
|
||||
|
||||
Modern Go supports type parameters.
|
||||
|
||||
```go
|
||||
func MapSlice[T any, U any](items []T, fn func(T) U) []U {
|
||||
result := make([]U, 0, len(items))
|
||||
for _, item := range items {
|
||||
result = append(result, fn(item))
|
||||
}
|
||||
return result
|
||||
}
|
||||
```
|
||||
|
||||
Why generics exist:
|
||||
|
||||
- avoid repetitive boilerplate for reusable containers and algorithms
|
||||
- preserve static type safety
|
||||
- reduce reliance on `interface{}` or reflection for generic utilities
|
||||
|
||||
When to use them:
|
||||
|
||||
- collections
|
||||
- reusable helpers where type-specific duplication is obvious
|
||||
- infrastructure libraries
|
||||
|
||||
When not to use them:
|
||||
|
||||
- when a plain interface or a concrete type is simpler
|
||||
- when the abstraction is more confusing than the duplication it removes
|
||||
|
||||
Go generics are powerful, but idiomatic Go still prefers the simplest tool that expresses the problem clearly.
|
||||
|
||||
## Error Handling Idioms
|
||||
|
||||
### Errors Are Values
|
||||
|
||||
This is a core Go idea. Failure is usually represented as an `error` return value.
|
||||
|
||||
```go
|
||||
func loadUser(ctx context.Context, repo Store, user User) error {
|
||||
if err := repo.Save(ctx, user); err != nil {
|
||||
return fmt.Errorf("save user %d: %w", user.ID, err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Why Go Chooses Explicit Errors
|
||||
|
||||
In systems code, unexpected control flow is expensive to reason about. Explicit errors keep the failure path visible.
|
||||
|
||||
This helps with:
|
||||
|
||||
- tracing the source of failures
|
||||
- adding context at each layer
|
||||
- making retry and fallback decisions deliberately
|
||||
|
||||
### Wrapping Errors
|
||||
|
||||
Use `%w` with `fmt.Errorf` when you want to preserve the underlying cause.
|
||||
|
||||
```go
|
||||
return fmt.Errorf("read config: %w", err)
|
||||
```
|
||||
|
||||
Then callers can inspect it with `errors.Is` or `errors.As`.
|
||||
|
||||
### Sentinel Errors and Typed Errors
|
||||
|
||||
Sentinel errors are package-level variables for meaningful known cases.
|
||||
|
||||
```go
|
||||
var ErrNotFound = errors.New("not found")
|
||||
```
|
||||
|
||||
Typed errors are useful when the caller needs structured details.
|
||||
|
||||
Use them carefully. Too many special-case errors can make APIs harder to use.
|
||||
|
||||
### When to Panic
|
||||
|
||||
`panic` is for truly unrecoverable programmer or runtime states, not normal business failures.
|
||||
|
||||
Good uses are rare:
|
||||
|
||||
- impossible invariants that indicate a bug
|
||||
- startup-time failures in very small programs where continuing makes no sense
|
||||
|
||||
Bad uses are common:
|
||||
|
||||
- validation failure from user input
|
||||
- database connection hiccups during a request
|
||||
- file-not-found in normal control flow
|
||||
|
||||
In production services, panicking on ordinary errors creates instability and poor operability.
|
||||
|
||||
## Real-World Usage Patterns
|
||||
|
||||
### Request and Response Structs
|
||||
|
||||
Structs commonly model HTTP payloads, database rows, queue messages, and config.
|
||||
|
||||
```go
|
||||
type CreateOrderRequest struct {
|
||||
CustomerID string `json:"customer_id"`
|
||||
AmountCents int64 `json:"amount_cents"`
|
||||
}
|
||||
```
|
||||
|
||||
### Interfaces at Boundaries
|
||||
|
||||
A service often depends on an interface for storage or external calls, while the concrete implementation stays in another package.
|
||||
|
||||
### Explicit Error Returns for Every I/O Layer
|
||||
|
||||
Anything involving the network, disk, serialization, or databases should return precise errors with context. This is normal, not noisy. It is how production Go code stays debuggable.
|
||||
|
||||
## Common Mistakes and Misconceptions
|
||||
|
||||
### Mistake: Treating Slices Like Independent Dynamic Arrays
|
||||
|
||||
Slices can share backing storage. Appends and sub-slices can interact in surprising ways if you ignore capacity and aliasing.
|
||||
|
||||
### Mistake: Using Pointers Everywhere
|
||||
|
||||
Beginners sometimes assume pointer-heavy code is more advanced. Often it is just harder to read and increases allocation pressure. Use pointers for a reason, not by default.
|
||||
|
||||
### Mistake: Designing Huge Interfaces Up Front
|
||||
|
||||
Go interfaces work best when they are small and shaped by actual use. Large "service interfaces" often become rigid and awkward.
|
||||
|
||||
### Mistake: Ignoring Unicode Details
|
||||
|
||||
Indexing a string operates on bytes, not necessarily user-visible characters. This matters for APIs, validation, and text handling.
|
||||
|
||||
### Mistake: Using `panic` for Routine Error Handling
|
||||
|
||||
That is usually a sign you are importing habits from another language rather than using Go idioms.
|
||||
|
||||
### Mistake: Confusing Nil Values
|
||||
|
||||
Nil slices, nil maps, nil pointers, and nil interfaces do not all behave the same. Learn their semantics explicitly.
|
||||
|
||||
## Summary
|
||||
|
||||
Go's core language is intentionally compact, but it is not shallow. The important ideas are practical:
|
||||
|
||||
- values and types are central
|
||||
- slices, maps, and strings have concrete runtime behavior worth understanding
|
||||
- functions return errors explicitly
|
||||
- structs and methods model data and behavior
|
||||
- interfaces describe behavior and stay best when small
|
||||
- pointers and memory behavior matter, but Go shields you from manual memory management
|
||||
|
||||
If you can read and write these fundamentals fluently, you are ready for the most distinctive part of Go: concurrency with goroutines, channels, synchronization primitives, and request-scoped cancellation.
|
||||
@@ -0,0 +1,465 @@
|
||||
# Go: Concurrency and Goroutines
|
||||
|
||||
## Learning Objectives
|
||||
|
||||
- Understand the difference between concurrency and parallelism.
|
||||
- Learn how goroutines work and why they are cheaper than OS threads.
|
||||
- Use channels, `select`, mutexes, and synchronization primitives appropriately.
|
||||
- Understand the `context` package as the control plane for cancellation and deadlines.
|
||||
- Build a light but correct mental model of Go's memory model and data races.
|
||||
- Recognize common production concurrency patterns and the bugs that come with them.
|
||||
|
||||
## Why Concurrency Matters in Go
|
||||
|
||||
Go became popular partly because it made concurrent programming feel accessible.
|
||||
|
||||
Backend and systems software naturally deals with many things at once:
|
||||
|
||||
- handling multiple HTTP requests
|
||||
- waiting on databases and other services
|
||||
- processing jobs from queues
|
||||
- streaming data through pipelines
|
||||
- watching timers, sockets, and shutdown signals
|
||||
|
||||
If you handle all of that in a single linear flow, the program spends a lot of time idle. Concurrency lets you structure work so independent tasks can make progress without blocking each other unnecessarily.
|
||||
|
||||
### Concurrency vs Parallelism
|
||||
|
||||
- concurrency is about structuring many tasks in progress
|
||||
- parallelism is about tasks literally running at the same time on multiple CPU cores
|
||||
|
||||
Go helps with both, but it starts with concurrency as a programming model.
|
||||
|
||||
## Goroutines: Lightweight Concurrent Execution
|
||||
|
||||
### What They Are
|
||||
|
||||
A goroutine is a function executing independently from other goroutines.
|
||||
|
||||
```go
|
||||
go sendEmail(userID)
|
||||
```
|
||||
|
||||
That single keyword starts a concurrent unit of execution.
|
||||
|
||||
### Why Goroutines Exist
|
||||
|
||||
OS threads are powerful, but they are relatively expensive to create and manage directly. Go wanted a lighter abstraction so programs could comfortably run thousands or even millions of concurrent tasks, as long as the workload and memory usage made that reasonable.
|
||||
|
||||
### How Goroutines Work Internally
|
||||
|
||||
Go uses an M:N scheduler. In simplified terms:
|
||||
|
||||
- many goroutines are multiplexed onto fewer OS threads
|
||||
- the runtime scheduler decides which goroutine runs where
|
||||
- the scheduler cooperates with the runtime and system calls to keep work moving
|
||||
|
||||
The common mental model is `G`, `M`, and `P`:
|
||||
|
||||
- `G` is a goroutine
|
||||
- `M` is an OS thread, called a machine in runtime terminology
|
||||
- `P` is a processor token that lets Go code execute and carries scheduler state
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
G1[Goroutine] --> P1[Processor P]
|
||||
G2[Goroutine] --> P1
|
||||
G3[Goroutine] --> P2[Processor P]
|
||||
P1 --> M1[OS Thread M]
|
||||
P2 --> M2[OS Thread M]
|
||||
```
|
||||
|
||||
This model lets Go keep concurrency cheap while still using real CPU parallelism when available.
|
||||
|
||||
### Why Goroutines Feel Cheap
|
||||
|
||||
They start with small stacks that can grow as needed. That is very different from traditional thread models where each thread may reserve a much larger stack up front.
|
||||
|
||||
Still, "cheap" does not mean "free."
|
||||
|
||||
Each goroutine has:
|
||||
|
||||
- scheduler overhead
|
||||
- stack memory
|
||||
- potential references keeping heap data alive
|
||||
|
||||
Launching goroutines without bounds in a busy server can still create memory pressure and operational problems.
|
||||
|
||||
## Waiting for Goroutines to Finish
|
||||
|
||||
When goroutines need coordination, a common tool is `sync.WaitGroup`.
|
||||
|
||||
```go
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for _, id := range ids {
|
||||
wg.Add(1)
|
||||
|
||||
go func(userID int64) {
|
||||
defer wg.Done()
|
||||
processUser(userID)
|
||||
}(id)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
```
|
||||
|
||||
Why it exists:
|
||||
|
||||
- lets one part of the program wait for a known set of concurrent tasks
|
||||
- keeps coordination explicit without using channels for every case
|
||||
|
||||
In production code, `WaitGroup` is often simpler than a custom done channel when you only need task completion, not data transfer.
|
||||
|
||||
## Channels: Communication and Coordination
|
||||
|
||||
### What a Channel Is
|
||||
|
||||
A channel is a typed conduit used to send values between goroutines.
|
||||
|
||||
```go
|
||||
jobs := make(chan int)
|
||||
results := make(chan string)
|
||||
```
|
||||
|
||||
### Why Channels Exist
|
||||
|
||||
Go popularized the idea "share memory by communicating." The point is not that shared memory is forbidden. The point is that ownership transfer through communication is often easier to reason about than unrestricted shared mutation.
|
||||
|
||||
Channels are useful for:
|
||||
|
||||
- handing work to workers
|
||||
- propagating results
|
||||
- signaling completion
|
||||
- coordinating pipelines
|
||||
|
||||
### Unbuffered Channels
|
||||
|
||||
An unbuffered channel requires sender and receiver to synchronize.
|
||||
|
||||
```go
|
||||
done := make(chan struct{})
|
||||
|
||||
go func() {
|
||||
fmt.Println("work complete")
|
||||
done <- struct{}{}
|
||||
}()
|
||||
|
||||
<-done
|
||||
```
|
||||
|
||||
Why this matters:
|
||||
|
||||
- send and receive form a handoff point
|
||||
- it is both data transfer and synchronization
|
||||
|
||||
### Buffered Channels
|
||||
|
||||
A buffered channel can hold a fixed number of values without an immediate receiver.
|
||||
|
||||
```go
|
||||
queue := make(chan string, 100)
|
||||
queue <- "task-1"
|
||||
```
|
||||
|
||||
Why buffered channels exist:
|
||||
|
||||
- smooth over short bursts
|
||||
- decouple producer and consumer timing somewhat
|
||||
- model bounded queues naturally
|
||||
|
||||
Do not treat buffering as magic. A large enough producer can still fill the buffer and block.
|
||||
|
||||
### Closing Channels
|
||||
|
||||
Closing a channel means no more values will be sent.
|
||||
|
||||
```go
|
||||
close(queue)
|
||||
```
|
||||
|
||||
Rules that matter:
|
||||
|
||||
- only close from the sending side when it owns completion
|
||||
- do not close a channel just because you are done receiving from it
|
||||
- sending on a closed channel panics
|
||||
|
||||
Receivers can use the two-result form:
|
||||
|
||||
```go
|
||||
value, ok := <-queue
|
||||
```
|
||||
|
||||
When `ok` is false, the channel is closed and drained.
|
||||
|
||||
### When Not to Use Channels
|
||||
|
||||
Channels are excellent, but not universal. If you just need to protect a shared map or counter, a mutex may be simpler. Overusing channels can make code look concurrent while actually becoming harder to understand.
|
||||
|
||||
## `select`: Wait on Multiple Communication Paths
|
||||
|
||||
`select` lets a goroutine wait on multiple channel operations.
|
||||
|
||||
```go
|
||||
select {
|
||||
case result := <-results:
|
||||
fmt.Println("got result", result)
|
||||
case <-time.After(200 * time.Millisecond):
|
||||
fmt.Println("timed out")
|
||||
}
|
||||
```
|
||||
|
||||
Why it exists:
|
||||
|
||||
- real systems often wait on multiple events
|
||||
- timeouts and cancellation are first-class concerns
|
||||
- many concurrent flows need to react to whichever signal arrives first
|
||||
|
||||
### Real-World Use: Timeout and Cancellation
|
||||
|
||||
```go
|
||||
select {
|
||||
case msg := <-incoming:
|
||||
handle(msg)
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
```
|
||||
|
||||
This is the backbone of responsive concurrent systems in Go: do work if possible, but remain interruptible.
|
||||
|
||||
## The `context` Package: Cancellation, Deadlines, and Scope
|
||||
|
||||
### What It Is
|
||||
|
||||
`context.Context` carries request-scoped cancellation, deadlines, and small pieces of request metadata across API boundaries.
|
||||
|
||||
```go
|
||||
func FetchUser(ctx context.Context, id string) (User, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://example.internal/users/"+id, nil)
|
||||
if err != nil {
|
||||
return User{}, err
|
||||
}
|
||||
|
||||
// send request with ctx-aware client
|
||||
return User{}, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Why It Exists
|
||||
|
||||
In distributed systems, work rarely lives in a single function. An HTTP request may trigger:
|
||||
|
||||
- JSON parsing
|
||||
- database queries
|
||||
- downstream HTTP calls
|
||||
- cache lookups
|
||||
- logging and tracing
|
||||
|
||||
If the client disconnects or a deadline expires, you want the whole chain to stop promptly. Context gives the program a standard way to express that control signal.
|
||||
|
||||
### How It Works Internally
|
||||
|
||||
Contexts form a tree.
|
||||
|
||||
- a parent context can be derived into child contexts
|
||||
- canceling the parent cancels all children
|
||||
- deadlines propagate downward
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Background Context] --> B[HTTP Request Context]
|
||||
B --> C[DB Query Context]
|
||||
B --> D[Downstream API Context]
|
||||
B --> E[Worker Task Context]
|
||||
```
|
||||
|
||||
### Rules for Using Context Correctly
|
||||
|
||||
- pass it as the first parameter by convention
|
||||
- do not store it inside structs for long-lived use
|
||||
- do not use it as a bag of optional business parameters
|
||||
- respect cancellation by checking `ctx.Done()` or using context-aware APIs
|
||||
|
||||
### Common Misuse
|
||||
|
||||
Putting every random value into context makes code opaque. Use context values only for request-scoped metadata that crosses process boundaries or middleware layers, such as trace IDs or auth claims when your framework expects it.
|
||||
|
||||
## Mutexes, RWMutexes, and Atomics
|
||||
|
||||
### Why These Exist Alongside Channels
|
||||
|
||||
The slogan "share memory by communicating" is helpful, but it is not a religion. Some problems are fundamentally shared-state problems.
|
||||
|
||||
Example:
|
||||
|
||||
- protecting a cache map
|
||||
- incrementing metrics counters
|
||||
- updating a shared in-memory registry
|
||||
|
||||
For these, a mutex is often clearer than designing a special manager goroutine and channel protocol.
|
||||
|
||||
### `sync.Mutex`
|
||||
|
||||
```go
|
||||
type Counter struct {
|
||||
mu sync.Mutex
|
||||
value int64
|
||||
}
|
||||
|
||||
func (c *Counter) Inc() {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
c.value++
|
||||
}
|
||||
```
|
||||
|
||||
Why it works:
|
||||
|
||||
- only one goroutine can hold the lock at a time
|
||||
- the critical section becomes explicit
|
||||
|
||||
### `sync.RWMutex`
|
||||
|
||||
Useful when reads are much more frequent than writes, but do not assume it is always faster. Its benefits depend on workload and contention patterns.
|
||||
|
||||
### `sync/atomic`
|
||||
|
||||
Atomic operations are useful for low-level counters, flags, and lock-free coordination where the semantics are simple and precise.
|
||||
|
||||
Use atomics carefully. They are powerful but easy to misuse if you do not understand memory ordering and invariants.
|
||||
|
||||
## The Go Memory Model, Lightly Explained
|
||||
|
||||
The memory model answers a critical question: when one goroutine writes data, when is another goroutine guaranteed to see it?
|
||||
|
||||
If two goroutines touch the same variable without proper synchronization and at least one access is a write, you have a data race.
|
||||
|
||||
This is not a style issue. It is a correctness bug.
|
||||
|
||||
### Synchronization Creates Visibility Guarantees
|
||||
|
||||
Common happens-before edges include:
|
||||
|
||||
- sending on a channel before the corresponding receive completes
|
||||
- unlocking a mutex before a later lock on that mutex
|
||||
- closing a channel before receives observe closure
|
||||
- `WaitGroup` and other primitives coordinating completion
|
||||
|
||||
If you rely on plain timing, such as "the other goroutine will probably run first," you do not have a guarantee.
|
||||
|
||||
### Why This Matters in Production
|
||||
|
||||
Data races can pass tests and still fail under load, on different CPUs, or only once every few days. That is why race bugs are among the most frustrating backend failures.
|
||||
|
||||
Use the race detector early:
|
||||
|
||||
```bash
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
## Concurrency Patterns You Will Actually Use
|
||||
|
||||
### Worker Pool
|
||||
|
||||
Useful when you have many jobs but want bounded concurrency.
|
||||
|
||||
```go
|
||||
func worker(id int, jobs <-chan int, results chan<- int) {
|
||||
for job := range jobs {
|
||||
results <- job * 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Job Producer] --> B[Buffered Jobs Channel]
|
||||
B --> C[Worker 1]
|
||||
B --> D[Worker 2]
|
||||
B --> E[Worker 3]
|
||||
C --> F[Results Channel]
|
||||
D --> F
|
||||
E --> F
|
||||
```
|
||||
|
||||
Why it exists:
|
||||
|
||||
- prevents unbounded goroutine creation
|
||||
- smooths throughput
|
||||
- matches CPU or downstream capacity constraints
|
||||
|
||||
### Fan-Out and Fan-In
|
||||
|
||||
This pattern sends work to multiple goroutines and merges results back together. It is common in API aggregation, search, and parallel I/O.
|
||||
|
||||
### Pipelines
|
||||
|
||||
Each stage reads from an input channel, transforms data, and sends to an output channel. This is useful for streaming transformations, though you must design cancellation carefully or you can leak goroutines when downstream stops consuming.
|
||||
|
||||
### Bounded Semaphores with Channels
|
||||
|
||||
A buffered channel can act as a semaphore controlling how many operations run at once. This is handy for limiting downstream API calls or database work.
|
||||
|
||||
## Real-World Usage Patterns
|
||||
|
||||
### HTTP Request Fan-Out
|
||||
|
||||
An API gateway might receive one request, then concurrently ask a profile service, inventory service, and pricing service for data. Context cancellation ensures that if the client goes away or a deadline expires, those downstream calls stop too.
|
||||
|
||||
### Background Job Processing
|
||||
|
||||
A worker service reading from a queue often uses:
|
||||
|
||||
- one intake goroutine
|
||||
- a bounded worker pool
|
||||
- retry logic
|
||||
- context cancellation for shutdown
|
||||
- metrics on success, failure, and latency
|
||||
|
||||
### Streaming and Event Processing
|
||||
|
||||
Go is good at managing concurrent streams from sockets, brokers, or internal pipelines because goroutines map well to independent flows of work.
|
||||
|
||||
## Common Mistakes and Misconceptions
|
||||
|
||||
### Mistake: Spawning Unbounded Goroutines
|
||||
|
||||
If every request starts many goroutines without a limit, memory and scheduler pressure can explode under load.
|
||||
|
||||
### Mistake: Forgetting Cancellation
|
||||
|
||||
Goroutines that wait forever on channels, I/O, or timers become leaks. In servers, leaked goroutines are a real operational bug.
|
||||
|
||||
### Mistake: Closing Channels from the Wrong Side
|
||||
|
||||
Channel closure should usually be owned by the sender that knows when production is complete.
|
||||
|
||||
### Mistake: Using Channels for Everything
|
||||
|
||||
Sometimes a mutex is the simplest and most correct tool.
|
||||
|
||||
### Mistake: Assuming Concurrent Means Safe
|
||||
|
||||
Starting work in multiple goroutines does not automatically make the code synchronized. Shared state still needs a correctness story.
|
||||
|
||||
### Mistake: Ignoring the Race Detector
|
||||
|
||||
If you write concurrent Go and do not run `go test -race`, you are skipping one of the most useful safety tools in the ecosystem.
|
||||
|
||||
### Mistake: Misusing Context Values
|
||||
|
||||
Context is for cancellation, deadlines, and narrow request-scoped metadata. It is not general dependency injection.
|
||||
|
||||
## Summary
|
||||
|
||||
Go concurrency is powerful because it combines a simple source-level model with strong runtime support.
|
||||
|
||||
- goroutines make concurrent work cheap to express
|
||||
- channels coordinate ownership transfer and signaling
|
||||
- `select` handles multiple events, timeouts, and cancellation
|
||||
- mutexes and atomics remain essential for shared-state problems
|
||||
- `context` is the control plane for request-scoped work
|
||||
- the memory model and race detector protect correctness when multiple goroutines interact
|
||||
|
||||
The next step is learning how to organize real Go codebases: packages, modules, tests, benchmarks, and the toolchain that keeps production Go code clean and maintainable.
|
||||
@@ -0,0 +1,565 @@
|
||||
# Go: Packages, Testing, and Tools
|
||||
|
||||
## Learning Objectives
|
||||
|
||||
- Understand how Go packages and modules organize code and dependencies.
|
||||
- Learn the visibility rules that shape API design in Go.
|
||||
- Write unit tests, table-driven tests, handler tests, benchmarks, and fuzz tests.
|
||||
- Use the Go toolchain to format, vet, benchmark, and inspect code.
|
||||
- Understand common package layout patterns for growing services.
|
||||
- Recognize how tooling discipline keeps Go codebases maintainable in production.
|
||||
|
||||
## Packages and Modules: Two Different Layers
|
||||
|
||||
One of the easiest ways to get confused in Go is to blur packages and modules together. They are connected, but they are not the same thing.
|
||||
|
||||
### Package
|
||||
|
||||
A package is a unit of code organization and namespace.
|
||||
|
||||
Examples:
|
||||
|
||||
- `fmt`
|
||||
- `net/http`
|
||||
- `time`
|
||||
- your own package like `internal/store`
|
||||
|
||||
All files in a directory usually belong to the same package and compile together.
|
||||
|
||||
### Module
|
||||
|
||||
A module is the versioned dependency unit defined by `go.mod`.
|
||||
|
||||
One module can contain many packages.
|
||||
|
||||
This distinction matters in real projects because:
|
||||
|
||||
- packages organize design inside the codebase
|
||||
- modules organize dependency and version boundaries across codebases
|
||||
|
||||
## Why Go Organizes Code This Way
|
||||
|
||||
Go wants dependency structure to stay visible and simple.
|
||||
|
||||
Packages make it easy to answer questions like:
|
||||
|
||||
- what code belongs together
|
||||
- what API surface is exported
|
||||
- what dependencies are allowed here
|
||||
|
||||
Modules make it easy to answer questions like:
|
||||
|
||||
- what external libraries does this project depend on
|
||||
- what version of a dependency are we building against
|
||||
- how can another project import this code reproducibly
|
||||
|
||||
In large backend systems, those questions are not administrative details. They directly affect build speed, deploy safety, and team comprehension.
|
||||
|
||||
## Package Naming and Visibility
|
||||
|
||||
### Package Names
|
||||
|
||||
Go package names are usually short, lower-case, and simple.
|
||||
|
||||
Good examples:
|
||||
|
||||
- `store`
|
||||
- `auth`
|
||||
- `queue`
|
||||
- `config`
|
||||
|
||||
Less ideal examples:
|
||||
|
||||
- `storeutils`
|
||||
- `commonhelpers`
|
||||
- `myAmazingPackage`
|
||||
|
||||
Why the simplicity matters:
|
||||
|
||||
- package names appear at every call site
|
||||
- short names keep code readable
|
||||
- packages should represent clear concepts, not junk drawers
|
||||
|
||||
### Exported vs Unexported Identifiers
|
||||
|
||||
Go uses capitalization to control visibility.
|
||||
|
||||
- identifiers starting with an uppercase letter are exported
|
||||
- identifiers starting with a lowercase letter are package-private
|
||||
|
||||
```go
|
||||
package store
|
||||
|
||||
type User struct {
|
||||
ID int64
|
||||
Name string
|
||||
}
|
||||
|
||||
type repository struct {
|
||||
db *sql.DB
|
||||
}
|
||||
|
||||
func New(db *sql.DB) *repository {
|
||||
return &repository{db: db}
|
||||
}
|
||||
```
|
||||
|
||||
Why this rule exists:
|
||||
|
||||
- it keeps visibility obvious at the point of declaration
|
||||
- it avoids separate access modifiers like `public` and `private`
|
||||
- it encourages small package APIs
|
||||
|
||||
### Internal APIs with `internal/`
|
||||
|
||||
Go has a special `internal` directory rule. Packages inside `internal/` can only be imported by code within the parent module tree.
|
||||
|
||||
This is useful when you want:
|
||||
|
||||
- separation between application-private code and reusable libraries
|
||||
- freedom to refactor internals without pretending they are public APIs
|
||||
|
||||
That makes `internal/` a strong default for service code.
|
||||
|
||||
## Common Package Layout Patterns
|
||||
|
||||
There is no single mandatory project structure in Go. That is intentional. The language tries to avoid framework-enforced layout.
|
||||
|
||||
A common service layout looks like this:
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[payments-service] --> B[go.mod]
|
||||
A --> C[cmd/api/main.go]
|
||||
A --> D[internal/httpapi]
|
||||
A --> E[internal/service]
|
||||
A --> F[internal/store]
|
||||
A --> G[internal/config]
|
||||
A --> H[internal/worker]
|
||||
```
|
||||
|
||||
### `cmd/`
|
||||
|
||||
Holds entry points for executables.
|
||||
|
||||
If one repository produces multiple binaries, `cmd/` keeps them separate cleanly.
|
||||
|
||||
### `internal/`
|
||||
|
||||
Holds application-private packages.
|
||||
|
||||
For most services, this is where the real code lives.
|
||||
|
||||
### `pkg/`
|
||||
|
||||
Some repositories use `pkg/` for packages intended to be imported by other modules. This is a convention, not a language rule. Use it only if it creates actual clarity.
|
||||
|
||||
### Keep Layout Earned, Not Decorative
|
||||
|
||||
Beginners often copy large directory trees too early. If a project has only one executable and a few packages, keep it smaller until complexity genuinely arrives.
|
||||
|
||||
## Modules, Dependencies, and Reproducibility
|
||||
|
||||
### `go.mod`
|
||||
|
||||
`go.mod` declares the module path and dependency requirements.
|
||||
|
||||
```go
|
||||
module example.com/payments
|
||||
|
||||
go 1.25.0
|
||||
|
||||
require (
|
||||
github.com/google/uuid v1.6.0
|
||||
)
|
||||
```
|
||||
|
||||
### `go.sum`
|
||||
|
||||
`go.sum` records checksums for module content so builds can verify dependency integrity.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- catches unexpected module tampering
|
||||
- helps keep builds reproducible
|
||||
- gives the module downloader integrity data
|
||||
|
||||
### How Version Resolution Works
|
||||
|
||||
Go uses module version selection designed to be reproducible and understandable. The details can get deep, but the practical takeaway is:
|
||||
|
||||
- dependency versions are declared in `go.mod`
|
||||
- the toolchain resolves one version per module in the build graph
|
||||
- commands like `go mod tidy` keep the graph clean
|
||||
|
||||
Useful commands:
|
||||
|
||||
```bash
|
||||
go get github.com/google/uuid@latest
|
||||
go mod tidy
|
||||
go list -m all
|
||||
```
|
||||
|
||||
### Why Import Cycles Are Forbidden
|
||||
|
||||
Go does not allow cyclic package imports.
|
||||
|
||||
That may feel restrictive, but it enforces architectural clarity. Cycles usually signal packages that were split at the wrong boundary or abstractions that are too tangled.
|
||||
|
||||
In large systems, this restriction prevents dependency graphs from collapsing into spaghetti.
|
||||
|
||||
## Designing Package Boundaries Well
|
||||
|
||||
### Group by Responsibility, Not by Type Name Alone
|
||||
|
||||
Better:
|
||||
|
||||
- `auth`
|
||||
- `store`
|
||||
- `httpapi`
|
||||
- `billing`
|
||||
|
||||
Weaker:
|
||||
|
||||
- `models`
|
||||
- `utils`
|
||||
- `helpers`
|
||||
|
||||
Packages should usually represent behavior or domain areas, not just a pile of vaguely related structs.
|
||||
|
||||
### Define Interfaces Where They Are Consumed
|
||||
|
||||
Instead of putting every interface next to every implementation, define small interfaces at the consumer boundary when that improves decoupling and testing.
|
||||
|
||||
### Keep APIs Small
|
||||
|
||||
If another package imports yours, what do they truly need? In Go, smaller package surfaces are easier to maintain and easier to refactor safely.
|
||||
|
||||
## Testing in Go: A Built-In Culture
|
||||
|
||||
Testing is part of normal Go workflow, not an optional afterthought.
|
||||
|
||||
### Where Tests Live
|
||||
|
||||
Tests live in files ending with `_test.go`.
|
||||
|
||||
```text
|
||||
store/
|
||||
store.go
|
||||
store_test.go
|
||||
```
|
||||
|
||||
Functions that begin with `Test` are discovered by `go test`.
|
||||
|
||||
```go
|
||||
func TestParsePort(t *testing.T) {
|
||||
port, err := parsePort("8080")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
|
||||
if port != 8080 {
|
||||
t.Fatalf("got %d want %d", port, 8080)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why Go's Testing Model Is Simple
|
||||
|
||||
The standard library's `testing` package is deliberately small. That simplicity means:
|
||||
|
||||
- the barrier to writing tests is low
|
||||
- teams do not need a huge framework to get started
|
||||
- test execution integrates naturally with the language toolchain
|
||||
|
||||
## Table-Driven Tests
|
||||
|
||||
This is one of the most common idioms in Go.
|
||||
|
||||
```go
|
||||
func TestParsePort(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
want int
|
||||
wantErr bool
|
||||
}{
|
||||
{name: "valid", input: "8080", want: 8080},
|
||||
{name: "invalid", input: "abc", wantErr: true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, err := parsePort(tt.input)
|
||||
|
||||
if tt.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("expected error, got nil")
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
|
||||
if got != tt.want {
|
||||
t.Fatalf("got %d want %d", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Why this pattern is so useful:
|
||||
|
||||
- test cases stay compact
|
||||
- edge cases become easy to enumerate
|
||||
- adding new scenarios becomes mechanical rather than repetitive
|
||||
|
||||
## Testing HTTP Handlers
|
||||
|
||||
For APIs, `net/http/httptest` is essential.
|
||||
|
||||
```go
|
||||
func TestHealthHandler(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodGet, "/health", nil)
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(`{"status":"ok"}`))
|
||||
})
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("got %d want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Why this matters:
|
||||
|
||||
- lets you test handlers without a real network listener
|
||||
- keeps tests fast and deterministic
|
||||
- makes request/response behavior easy to inspect
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Performance-sensitive code can be benchmarked with the same toolchain.
|
||||
|
||||
```go
|
||||
func BenchmarkParsePort(b *testing.B) {
|
||||
for i := 0; i < b.N; i++ {
|
||||
_, _ = parsePort("8080")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run it with:
|
||||
|
||||
```bash
|
||||
go test -bench=. -benchmem ./...
|
||||
```
|
||||
|
||||
### Why Benchmarks Matter
|
||||
|
||||
In backend systems, intuition about performance is often wrong. Benchmarks let you compare alternatives with data.
|
||||
|
||||
Be careful though:
|
||||
|
||||
- microbenchmarks can miss real I/O behavior
|
||||
- unrealistic inputs produce misleading results
|
||||
- one benchmark is not a system profile
|
||||
|
||||
### Allocation Awareness
|
||||
|
||||
`-benchmem` shows allocation counts and bytes per operation. This is particularly helpful in Go because excess allocation often increases garbage collection pressure.
|
||||
|
||||
## Fuzz Testing
|
||||
|
||||
Modern Go supports fuzz testing through the standard toolchain.
|
||||
|
||||
```go
|
||||
func FuzzParsePort(f *testing.F) {
|
||||
f.Add("8080")
|
||||
f.Add("abc")
|
||||
|
||||
f.Fuzz(func(t *testing.T, input string) {
|
||||
_, _ = parsePort(input)
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
Why fuzzing is valuable:
|
||||
|
||||
- explores unexpected input combinations
|
||||
- finds parser and validation edge cases
|
||||
- is especially useful for network protocols, text parsing, and serialization code
|
||||
|
||||
## Tooling That Should Be in Your Daily Workflow
|
||||
|
||||
### `go fmt`
|
||||
|
||||
```bash
|
||||
go fmt ./...
|
||||
```
|
||||
|
||||
Standard formatting is one of Go's biggest team-level productivity wins.
|
||||
|
||||
### `go test`
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
Run it constantly. Fast feedback is part of idiomatic Go engineering.
|
||||
|
||||
### `go test -race`
|
||||
|
||||
```bash
|
||||
go test -race ./...
|
||||
```
|
||||
|
||||
This detects many data races in concurrent code. If you write goroutines and shared state, this is a critical tool.
|
||||
|
||||
### `go vet`
|
||||
|
||||
```bash
|
||||
go vet ./...
|
||||
```
|
||||
|
||||
`go vet` looks for suspicious constructs that compile but are likely wrong.
|
||||
|
||||
### `go test -cover`
|
||||
|
||||
```bash
|
||||
go test -cover ./...
|
||||
```
|
||||
|
||||
Coverage is useful as a signal, not a religion. High coverage does not guarantee meaningful tests, but very low coverage may show untested risk.
|
||||
|
||||
### Useful External Tools
|
||||
|
||||
- `staticcheck` for deeper linting and bug finding
|
||||
- `govulncheck` for known vulnerability detection in dependencies and reachable code
|
||||
- `goimports` for formatting plus import cleanup
|
||||
- `pprof` for CPU and memory profiling
|
||||
|
||||
These are not built into the core language, but they fit naturally into Go's tooling culture.
|
||||
|
||||
## Build and Release Workflows
|
||||
|
||||
Go makes builds operationally simple.
|
||||
|
||||
### Build a Binary
|
||||
|
||||
```bash
|
||||
go build ./cmd/api
|
||||
```
|
||||
|
||||
### Cross-Compile
|
||||
|
||||
```bash
|
||||
GOOS=linux GOARCH=amd64 go build ./cmd/api
|
||||
```
|
||||
|
||||
This is a practical reason Go is so common in infrastructure tools. Cross-platform binaries are relatively easy to produce.
|
||||
|
||||
### Why Single-Binary Delivery Matters
|
||||
|
||||
In deployment pipelines, simplicity is leverage.
|
||||
|
||||
- containers become smaller and simpler
|
||||
- startup is predictable
|
||||
- dependency packaging becomes easier
|
||||
- CI artifacts are easier to reason about
|
||||
|
||||
## Documentation as Part of the API Surface
|
||||
|
||||
Go places real value on package and symbol documentation.
|
||||
|
||||
Exported identifiers should usually have comments when the package is intended for reuse.
|
||||
|
||||
Why this matters:
|
||||
|
||||
- tooling can surface docs automatically
|
||||
- package APIs become easier to consume without reading implementation details
|
||||
- maintainers communicate intent, not just mechanics
|
||||
|
||||
## Real-World Usage Patterns
|
||||
|
||||
### Service Repository Layout
|
||||
|
||||
A production service often has:
|
||||
|
||||
- one or more `cmd/` entry points
|
||||
- internal packages for handlers, business logic, storage, and config
|
||||
- tests next to each package
|
||||
- benchmark and race-check jobs in CI
|
||||
|
||||
### Testing Strategy
|
||||
|
||||
Healthy Go services usually combine:
|
||||
|
||||
- unit tests for pure logic
|
||||
- handler tests with `httptest`
|
||||
- integration tests for DB and network boundaries
|
||||
- benchmarks for hot paths
|
||||
- race detection for concurrent subsystems
|
||||
|
||||
### Tooling in CI
|
||||
|
||||
A practical CI pipeline often runs:
|
||||
|
||||
- `go fmt` or formatting checks
|
||||
- `go test ./...`
|
||||
- `go test -race ./...`
|
||||
- `go vet ./...`
|
||||
- optional static analysis and vulnerability scanning
|
||||
|
||||
## Common Mistakes and Misconceptions
|
||||
|
||||
### Mistake: Creating Huge Project Layouts Too Early
|
||||
|
||||
Structure should reflect real complexity. Decorative folders make learning and maintenance harder.
|
||||
|
||||
### Mistake: Treating `pkg/` as Mandatory
|
||||
|
||||
It is only a convention. Many good Go services never use it.
|
||||
|
||||
### Mistake: Exporting Too Much
|
||||
|
||||
Large public package surfaces make refactoring harder and couple packages unnecessarily.
|
||||
|
||||
### Mistake: Ignoring Import Cycles Until Late
|
||||
|
||||
If your packages keep wanting to import each other, the boundaries are probably wrong.
|
||||
|
||||
### Mistake: Writing Only Happy-Path Tests
|
||||
|
||||
Most production failures happen in error paths, edge inputs, and timeout scenarios.
|
||||
|
||||
### Mistake: Optimizing from Guesswork Instead of Benchmarks
|
||||
|
||||
Measure before changing code for performance reasons.
|
||||
|
||||
### Mistake: Treating Coverage as the Goal
|
||||
|
||||
Coverage can be useful, but well-chosen tests matter more than inflated percentages.
|
||||
|
||||
## Summary
|
||||
|
||||
Go packages and modules keep code organization and dependency management explicit. The testing package and standard toolchain make quality checks part of ordinary development rather than an extra framework burden.
|
||||
|
||||
The big practical lessons are:
|
||||
|
||||
- packages shape design boundaries
|
||||
- modules shape dependency boundaries
|
||||
- small APIs and clean imports keep code maintainable
|
||||
- table-driven tests and `httptest` are core testing patterns
|
||||
- benchmarks, race checks, and tooling provide objective feedback
|
||||
- operational simplicity is one of Go's biggest strengths
|
||||
|
||||
The next step is to bring all of this together in production system design: HTTP servers, request lifecycles, context propagation, timeouts, graceful shutdown, observability, and distributed systems patterns in Go.
|
||||
@@ -0,0 +1,470 @@
|
||||
# Go: Real-World System Design in Go
|
||||
|
||||
## Learning Objectives
|
||||
|
||||
- Understand how Go is used to build production HTTP services and distributed systems.
|
||||
- Learn the request lifecycle in `net/http` and how handlers interact with context.
|
||||
- Design services with clear boundaries, sane package structure, and operational safety.
|
||||
- Apply timeouts, cancellation, connection reuse, and graceful shutdown correctly.
|
||||
- Recognize common backend patterns in Go for workers, queues, caches, and external service calls.
|
||||
- Reason about real-world tradeoffs rather than just writing syntax-correct code.
|
||||
|
||||
## Why Go Works Well for Production Systems
|
||||
|
||||
By the time you reach system design, you should stop thinking of Go as just a language and start thinking of it as an operating model.
|
||||
|
||||
Go is attractive in production because it combines:
|
||||
|
||||
- native binaries that are easy to ship
|
||||
- a concurrency model that fits network services well
|
||||
- a standard library strong enough to build real servers
|
||||
- explicit errors and visible control flow
|
||||
- tooling that supports fast feedback and straightforward CI
|
||||
|
||||
That combination makes Go common in:
|
||||
|
||||
- REST and JSON APIs
|
||||
- RPC services
|
||||
- control-plane components
|
||||
- stream and queue consumers
|
||||
- gateways and reverse proxies
|
||||
- schedulers and automation tooling
|
||||
|
||||
## The HTTP Request Lifecycle in Go
|
||||
|
||||
### What `net/http` Gives You
|
||||
|
||||
Go's standard library includes both an HTTP server and HTTP client. You do not need a framework to build a real API.
|
||||
|
||||
At a high level:
|
||||
|
||||
1. the server listens on a socket
|
||||
2. connections are accepted
|
||||
3. requests are parsed
|
||||
4. a handler is invoked
|
||||
5. the handler writes a response
|
||||
|
||||
Conceptually, the flow looks like this:
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Client] --> B[Load Balancer]
|
||||
B --> C[Go HTTP Server]
|
||||
C --> D[Middleware]
|
||||
D --> E[Handler]
|
||||
E --> F[Service Layer]
|
||||
F --> G[Database]
|
||||
F --> H[Cache]
|
||||
F --> I[Downstream API]
|
||||
```
|
||||
|
||||
### Why This Model Is Powerful
|
||||
|
||||
The standard library model is deliberately small:
|
||||
|
||||
- `http.Handler` is just an interface
|
||||
- middleware is ordinary function composition
|
||||
- routing can be simple or sophisticated depending on need
|
||||
|
||||
This keeps the underlying mechanics easy to understand. Even if you later use a router or framework, it usually plugs into the same `http.Handler` shape.
|
||||
|
||||
### Internal Behavior That Matters
|
||||
|
||||
You do not need to memorize the internals, but you should know the operational consequences:
|
||||
|
||||
- the server can handle many requests concurrently
|
||||
- handler code must therefore be safe under concurrency
|
||||
- request bodies and response writers are tied to request lifetime
|
||||
- request contexts are canceled when clients disconnect or the server shuts down the request
|
||||
|
||||
That last point is critical. Context is not decoration. It is how Go propagates lifecycle control through the call stack.
|
||||
|
||||
## Building an Idiomatic HTTP Service
|
||||
|
||||
Here is a small but production-minded shape for a service:
|
||||
|
||||
```go
|
||||
type UserStore interface {
|
||||
Create(ctx context.Context, user User) error
|
||||
}
|
||||
|
||||
type App struct {
|
||||
logger *slog.Logger
|
||||
store UserStore
|
||||
}
|
||||
|
||||
func NewApp(logger *slog.Logger, store UserStore) *App {
|
||||
return &App{logger: logger, store: store}
|
||||
}
|
||||
|
||||
func (a *App) routes() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/health", a.handleHealth)
|
||||
mux.HandleFunc("/users", a.handleCreateUser)
|
||||
return a.logging(a.recover(mux))
|
||||
}
|
||||
|
||||
func (a *App) handleHealth(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
||||
}
|
||||
|
||||
func (a *App) handleCreateUser(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodPost {
|
||||
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
Email string `json:"email"`
|
||||
}
|
||||
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
http.Error(w, "invalid json", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
user := User{Name: req.Name, Email: req.Email}
|
||||
|
||||
if err := a.store.Create(r.Context(), user); err != nil {
|
||||
a.logger.Error("create user", "err", err)
|
||||
http.Error(w, "internal server error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusCreated, user)
|
||||
}
|
||||
```
|
||||
|
||||
### What This Example Demonstrates
|
||||
|
||||
- constructor-based dependency injection
|
||||
- interfaces at the boundary where behavior matters
|
||||
- handlers that stay thin and pass context downward
|
||||
- standard library routing and JSON handling
|
||||
- explicit error handling rather than hidden control flow
|
||||
|
||||
This shape scales well. You can add middleware, tracing, validation, auth, metrics, and graceful shutdown without replacing the whole architecture.
|
||||
|
||||
## Context in Production Request Paths
|
||||
|
||||
### Why Context Is Central
|
||||
|
||||
When an HTTP request comes in, the context attached to it should usually flow through all downstream operations.
|
||||
|
||||
Example chain:
|
||||
|
||||
- HTTP handler receives request
|
||||
- service layer validates business logic
|
||||
- repository executes SQL query
|
||||
- service makes a downstream HTTP call
|
||||
- background operation respects cancellation if appropriate
|
||||
|
||||
If the client disconnects or the server deadline is exceeded, that context cancellation should stop the rest of the work.
|
||||
|
||||
### Practical Rules
|
||||
|
||||
- pass `ctx` explicitly as the first parameter
|
||||
- use `NewRequestWithContext` for outbound HTTP
|
||||
- use database APIs that accept context
|
||||
- never replace request context with `context.Background()` in the middle of request processing unless you are intentionally detaching work
|
||||
|
||||
### Timeouts and Deadlines
|
||||
|
||||
Timeouts are not just protection against slowness. They are protection against resource exhaustion.
|
||||
|
||||
Without timeouts:
|
||||
|
||||
- goroutines can pile up waiting on I/O
|
||||
- file descriptors remain occupied
|
||||
- request latency can become unbounded
|
||||
- downstream incidents can cascade back into your service
|
||||
|
||||
Good Go services apply timeouts at multiple layers:
|
||||
|
||||
- incoming server read and header timeouts
|
||||
- request-scoped deadlines via context
|
||||
- outbound client timeouts
|
||||
- database query timeouts
|
||||
|
||||
## Graceful Shutdown
|
||||
|
||||
### What It Means
|
||||
|
||||
Graceful shutdown means the process stops accepting new work, gives in-flight work a chance to finish within a bounded time, and then exits cleanly.
|
||||
|
||||
This matters for:
|
||||
|
||||
- rolling deployments
|
||||
- autoscaling events
|
||||
- node drains in orchestration platforms
|
||||
- operator-triggered restarts
|
||||
|
||||
### Example
|
||||
|
||||
```go
|
||||
func run() error {
|
||||
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
||||
app := NewApp(logger, newInMemoryStore())
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: ":8080",
|
||||
Handler: app.routes(),
|
||||
ReadHeaderTimeout: 2 * time.Second,
|
||||
}
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
_ = srv.Shutdown(shutdownCtx)
|
||||
}()
|
||||
|
||||
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
return err
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Why It Matters Internally
|
||||
|
||||
If you just kill the process abruptly:
|
||||
|
||||
- in-flight requests get dropped
|
||||
- partial writes may occur
|
||||
- background jobs may stop mid-operation
|
||||
- queue acknowledgments may be inconsistent
|
||||
|
||||
Graceful shutdown is an operational correctness feature, not just polish.
|
||||
|
||||
## Outbound Clients, Connection Pools, and Resource Reuse
|
||||
|
||||
### HTTP Clients
|
||||
|
||||
Creating a new `http.Client` per request is usually a mistake. Clients and transports manage connection reuse.
|
||||
|
||||
Why reuse matters:
|
||||
|
||||
- avoids needless TCP and TLS setup costs
|
||||
- improves latency
|
||||
- reduces load on downstream services
|
||||
|
||||
At the same time, the zero-value default client setup is not enough for every production case. You usually want explicit timeouts and transport tuning.
|
||||
|
||||
### Database Pools
|
||||
|
||||
Packages like `database/sql` manage connection pools. Your job is to configure them sanely and use context-aware operations.
|
||||
|
||||
Important operational knobs include:
|
||||
|
||||
- max open connections
|
||||
- max idle connections
|
||||
- connection lifetime
|
||||
|
||||
These are part of system design, not just code details. Wrong pool settings can overload databases or starve your service.
|
||||
|
||||
## Service Architecture Patterns in Go
|
||||
|
||||
### Composition Root in `main`
|
||||
|
||||
In Go, `main` often acts as the composition root where you:
|
||||
|
||||
- load config
|
||||
- initialize logging
|
||||
- create clients and stores
|
||||
- wire dependencies together
|
||||
- start the server or worker
|
||||
|
||||
This keeps wiring visible and avoids magic containers.
|
||||
|
||||
### Thin Handlers, Clear Services
|
||||
|
||||
A healthy pattern is:
|
||||
|
||||
- handlers translate transport concerns
|
||||
- services handle business logic
|
||||
- repositories or clients handle I/O boundaries
|
||||
|
||||
Do not turn this into rigid architecture theater. The point is clarity, not layers for their own sake.
|
||||
|
||||
### Interfaces at Edges
|
||||
|
||||
Use interfaces where they help isolate external systems or enable tests. Do not create an interface for every struct just because a pattern from another language told you to.
|
||||
|
||||
## Background Workers and Queues
|
||||
|
||||
Go is also strong for worker processes.
|
||||
|
||||
Common worker responsibilities:
|
||||
|
||||
- poll or receive jobs
|
||||
- decode payloads
|
||||
- apply business logic
|
||||
- talk to storage or downstream services
|
||||
- retry or dead-letter on failure
|
||||
|
||||
A production worker often combines:
|
||||
|
||||
- bounded concurrency
|
||||
- context-driven shutdown
|
||||
- idempotent processing
|
||||
- metrics and tracing
|
||||
- retry with backoff
|
||||
|
||||
### Why Idempotency Matters
|
||||
|
||||
Distributed systems retry. That means your worker or API should behave safely when the same logical operation arrives more than once.
|
||||
|
||||
Examples:
|
||||
|
||||
- charging an order only once
|
||||
- ignoring duplicate event delivery with a deduplication key
|
||||
- using upserts or unique constraints to protect state transitions
|
||||
|
||||
## Resilience Patterns in Go Services
|
||||
|
||||
### Retries
|
||||
|
||||
Retries can improve reliability, but they are dangerous when used carelessly.
|
||||
|
||||
Use retries when:
|
||||
|
||||
- the error is transient
|
||||
- the operation is safe to retry
|
||||
- you apply limits and backoff
|
||||
|
||||
Do not blindly retry every failure. That can turn a partial outage into a full overload event.
|
||||
|
||||
### Backpressure and Bounded Concurrency
|
||||
|
||||
Every service has finite CPU, memory, DB connections, and downstream quota. Good Go systems acknowledge this with:
|
||||
|
||||
- worker pool limits
|
||||
- channel buffer sizing based on real capacity, not guesswork
|
||||
- request timeouts
|
||||
- queue sizing and shedding strategies
|
||||
|
||||
### Caching
|
||||
|
||||
Caches reduce latency and downstream load, but they introduce staleness, invalidation complexity, and memory pressure.
|
||||
|
||||
In Go services, a cache may be:
|
||||
|
||||
- in-memory with mutex protection
|
||||
- external like Redis
|
||||
- layered with local plus remote caching
|
||||
|
||||
Choose based on consistency needs and failure modes, not just speed.
|
||||
|
||||
## Observability: Systems Need to Explain Themselves
|
||||
|
||||
### Logging
|
||||
|
||||
Structured logs are easier to query and correlate than ad hoc strings. Go's `log/slog` is a good default in modern code.
|
||||
|
||||
### Metrics
|
||||
|
||||
Metrics help answer:
|
||||
|
||||
- how many requests or jobs are happening
|
||||
- how often errors occur
|
||||
- how long operations take
|
||||
- whether queues, pools, or workers are saturating
|
||||
|
||||
### Tracing
|
||||
|
||||
Tracing becomes valuable once a request crosses multiple services. Go's context propagation model fits tracing naturally because trace metadata can move alongside request lifecycle.
|
||||
|
||||
### Profiling
|
||||
|
||||
When a service is slow or memory-hungry, use profiling rather than guesswork. Go's pprof ecosystem is one of the language's strongest practical advantages.
|
||||
|
||||
## A Realistic Service Architecture Example
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Client] --> B[API Gateway]
|
||||
B --> C[Go API Service]
|
||||
C --> D[Auth Middleware]
|
||||
D --> E[Business Service]
|
||||
E --> F[(Postgres)]
|
||||
E --> G[(Redis Cache)]
|
||||
E --> H[Message Broker]
|
||||
H --> I[Go Worker Service]
|
||||
I --> F
|
||||
C --> J[Observability Stack]
|
||||
I --> J
|
||||
```
|
||||
|
||||
Why Go fits this architecture well:
|
||||
|
||||
- API and worker components can share libraries and tooling
|
||||
- binaries are easy to containerize
|
||||
- concurrency model fits request handling and job processing
|
||||
- context propagation helps with cancellation and tracing
|
||||
|
||||
## Real-World Usage Patterns
|
||||
|
||||
### JSON API Service
|
||||
|
||||
Go is widely used for services that accept JSON, validate input, call storage or other APIs, and return typed responses.
|
||||
|
||||
### Internal Platform Components
|
||||
|
||||
Controllers, schedulers, reconcilers, and long-running agents are natural Go workloads because they need networking, concurrency, and operational predictability.
|
||||
|
||||
### Data and Event Processing
|
||||
|
||||
Consumers and workers benefit from Go's lightweight concurrency and straightforward deployment model.
|
||||
|
||||
## Common Mistakes and Misconceptions
|
||||
|
||||
### Mistake: Starting with a Framework Instead of Understanding `net/http`
|
||||
|
||||
Frameworks can help, but you should first understand the handler model underneath them.
|
||||
|
||||
### Mistake: Ignoring Timeouts
|
||||
|
||||
Untimed network calls are operational liabilities.
|
||||
|
||||
### Mistake: Creating New Clients Per Request
|
||||
|
||||
That defeats connection reuse and often harms performance badly.
|
||||
|
||||
### Mistake: Letting Handlers Contain All Business Logic
|
||||
|
||||
This makes testing harder and transport concerns bleed into domain behavior.
|
||||
|
||||
### Mistake: Launching Background Goroutines Without Shutdown Strategy
|
||||
|
||||
Every long-lived goroutine in a service should have a lifecycle story.
|
||||
|
||||
### Mistake: Overabstracting Everything into Interfaces
|
||||
|
||||
Use interfaces deliberately at boundaries, not as decoration.
|
||||
|
||||
### Mistake: Forgetting That Handlers Run Concurrently
|
||||
|
||||
Shared state in a server must be synchronized properly.
|
||||
|
||||
## Summary
|
||||
|
||||
Production Go system design is about making the whole service lifecycle explicit:
|
||||
|
||||
- request entry through `net/http`
|
||||
- context propagation through each downstream operation
|
||||
- bounded concurrency and sensible resource reuse
|
||||
- graceful shutdown during deploys and failures
|
||||
- clear package and dependency boundaries
|
||||
- observability and performance measurement built into the operational model
|
||||
|
||||
The most important mindset shift is this: idiomatic Go systems are not built around hidden magic. They are built around visible control flow, explicit dependencies, clear boundaries, and operationally honest concurrency.
|
||||
|
||||
That is exactly why Go remains such a strong language for backend and distributed systems engineering.
|
||||
Reference in New Issue
Block a user