Computer-Fundamentals/go/04_packages_testing_and_tools.md

# Go: Packages, Testing, and Tools

## Learning Objectives

- Understand how Go packages and modules organize code and dependencies.
- Learn the visibility rules that shape API design in Go.
- Write unit tests, table-driven tests, handler tests, benchmarks, and fuzz tests.
- Use the Go toolchain to format, vet, benchmark, and inspect code.
- Understand common package layout patterns for growing services.
- Recognize how tooling discipline keeps Go codebases maintainable in production.

## Packages and Modules: Two Different Layers

One of the easiest ways to get confused in Go is to blur packages and modules together. They are connected, but they are not the same thing.

### Package

A package is a unit of code organization and namespace.

Examples:

- `fmt`
- `net/http`
- `time`
- your own package like `internal/store`

All files in a directory usually belong to the same package and compile together.

### Module

A module is the versioned dependency unit defined by `go.mod`.

One module can contain many packages.

This distinction matters in real projects because:

- packages organize design inside the codebase
- modules organize dependency and version boundaries across codebases

## Why Go Organizes Code This Way

Go wants dependency structure to stay visible and simple.

Packages make it easy to answer questions like:

- what code belongs together
- what API surface is exported
- what dependencies are allowed here

Modules make it easy to answer questions like:

- what external libraries does this project depend on
- what version of a dependency are we building against
- how can another project import this code reproducibly

In large backend systems, those questions are not administrative details. They directly affect build speed, deploy safety, and team comprehension.

## Package Naming and Visibility

### Package Names

Go package names are usually short, lower-case, and simple.

Good examples:

- `store`
- `auth`
- `queue`
- `config`

Less ideal examples:

- `storeutils`
- `commonhelpers`
- `myAmazingPackage`

Why the simplicity matters:

- package names appear at every call site
- short names keep code readable
- packages should represent clear concepts, not junk drawers

### Exported vs Unexported Identifiers

Go uses capitalization to control visibility.

- identifiers starting with an uppercase letter are exported
- identifiers starting with a lowercase letter are package-private

```go
package store

type User struct {
    ID   int64
    Name string
}

type repository struct {
    db *sql.DB
}

func New(db *sql.DB) *repository {
    return &repository{db: db}
}
```

Why this rule exists:

- it keeps visibility obvious at the point of declaration
- it avoids separate access modifiers like `public` and `private`
- it encourages small package APIs

### Internal APIs with `internal/`

Go has a special `internal` directory rule. Packages inside `internal/` can only be imported by code within the parent module tree.

This is useful when you want:

- separation between application-private code and reusable libraries
- freedom to refactor internals without pretending they are public APIs

That makes `internal/` a strong default for service code.

## Common Package Layout Patterns

There is no single mandatory project structure in Go. That is intentional. The language tries to avoid framework-enforced layout.

A common service layout looks like this:

```mermaid
graph TD
    A[payments-service] --> B[go.mod]
    A --> C[cmd/api/main.go]
    A --> D[internal/httpapi]
    A --> E[internal/service]
    A --> F[internal/store]
    A --> G[internal/config]
    A --> H[internal/worker]
```

### `cmd/`

Holds entry points for executables.

If one repository produces multiple binaries, `cmd/` keeps them separate cleanly.

### `internal/`

Holds application-private packages.

For most services, this is where the real code lives.

### `pkg/`

Some repositories use `pkg/` for packages intended to be imported by other modules. This is a convention, not a language rule. Use it only if it creates actual clarity.

### Keep Layout Earned, Not Decorative

Beginners often copy large directory trees too early. If a project has only one executable and a few packages, keep it smaller until complexity genuinely arrives.

## Modules, Dependencies, and Reproducibility

### `go.mod`

`go.mod` declares the module path and dependency requirements.

```go
module example.com/payments

go 1.25.0

require (
    github.com/google/uuid v1.6.0
)
```

### `go.sum`

`go.sum` records checksums for module content so builds can verify dependency integrity.

Why this matters:

- catches unexpected module tampering
- helps keep builds reproducible
- gives the module downloader integrity data

### How Version Resolution Works

Go uses module version selection designed to be reproducible and understandable. The details can get deep, but the practical takeaway is:

- dependency versions are declared in `go.mod`
- the toolchain resolves one version per module in the build graph
- commands like `go mod tidy` keep the graph clean

Useful commands:

```bash
go get github.com/google/uuid@latest
go mod tidy
go list -m all
```

### Why Import Cycles Are Forbidden

Go does not allow cyclic package imports.

That may feel restrictive, but it enforces architectural clarity. Cycles usually signal packages that were split at the wrong boundary or abstractions that are too tangled.

In large systems, this restriction prevents dependency graphs from collapsing into spaghetti.

## Designing Package Boundaries Well

### Group by Responsibility, Not by Type Name Alone

Better:

- `auth`
- `store`
- `httpapi`
- `billing`

Weaker:

- `models`
- `utils`
- `helpers`

Packages should usually represent behavior or domain areas, not just a pile of vaguely related structs.

### Define Interfaces Where They Are Consumed

Instead of putting every interface next to every implementation, define small interfaces at the consumer boundary when that improves decoupling and testing.

### Keep APIs Small

If another package imports yours, what do they truly need? In Go, smaller package surfaces are easier to maintain and easier to refactor safely.

## Testing in Go: A Built-In Culture

Testing is part of normal Go workflow, not an optional afterthought.

### Where Tests Live

Tests live in files ending with `_test.go`.

```text
store/
  store.go
  store_test.go
```

Functions that begin with `Test` are discovered by `go test`.

```go
func TestParsePort(t *testing.T) {
    port, err := parsePort("8080")
    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }

    if port != 8080 {
        t.Fatalf("got %d want %d", port, 8080)
    }
}
```

### Why Go's Testing Model Is Simple

The standard library's `testing` package is deliberately small. That simplicity means:

- the barrier to writing tests is low
- teams do not need a huge framework to get started
- test execution integrates naturally with the language toolchain

## Table-Driven Tests

This is one of the most common idioms in Go.

```go
func TestParsePort(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    int
        wantErr bool
    }{
        {name: "valid", input: "8080", want: 8080},
        {name: "invalid", input: "abc", wantErr: true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := parsePort(tt.input)

            if tt.wantErr {
                if err == nil {
                    t.Fatalf("expected error, got nil")
                }
                return
            }

            if err != nil {
                t.Fatalf("unexpected error: %v", err)
            }

            if got != tt.want {
                t.Fatalf("got %d want %d", got, tt.want)
            }
        })
    }
}
```

Why this pattern is so useful:

- test cases stay compact
- edge cases become easy to enumerate
- adding new scenarios becomes mechanical rather than repetitive

## Testing HTTP Handlers

For APIs, `net/http/httptest` is essential.

```go
func TestHealthHandler(t *testing.T) {
    req := httptest.NewRequest(http.MethodGet, "/health", nil)
    rec := httptest.NewRecorder()

    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
        _, _ = w.Write([]byte(`{"status":"ok"}`))
    })

    handler.ServeHTTP(rec, req)

    if rec.Code != http.StatusOK {
        t.Fatalf("got %d want %d", rec.Code, http.StatusOK)
    }
}
```

Why this matters:

- lets you test handlers without a real network listener
- keeps tests fast and deterministic
- makes request/response behavior easy to inspect

## Benchmarks

Performance-sensitive code can be benchmarked with the same toolchain.

```go
func BenchmarkParsePort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _, _ = parsePort("8080")
    }
}
```

Run it with:

```bash
go test -bench=. -benchmem ./...
```

### Why Benchmarks Matter

In backend systems, intuition about performance is often wrong. Benchmarks let you compare alternatives with data.

Be careful though:

- microbenchmarks can miss real I/O behavior
- unrealistic inputs produce misleading results
- one benchmark is not a system profile

### Allocation Awareness

`-benchmem` shows allocation counts and bytes per operation. This is particularly helpful in Go because excess allocation often increases garbage collection pressure.

## Fuzz Testing

Modern Go supports fuzz testing through the standard toolchain.

```go
func FuzzParsePort(f *testing.F) {
    f.Add("8080")
    f.Add("abc")

    f.Fuzz(func(t *testing.T, input string) {
        _, _ = parsePort(input)
    })
}
```

Why fuzzing is valuable:

- explores unexpected input combinations
- finds parser and validation edge cases
- is especially useful for network protocols, text parsing, and serialization code

## Tooling That Should Be in Your Daily Workflow

### `go fmt`

```bash
go fmt ./...
```

Standard formatting is one of Go's biggest team-level productivity wins.

### `go test`

```bash
go test ./...
```

Run it constantly. Fast feedback is part of idiomatic Go engineering.

### `go test -race`

```bash
go test -race ./...
```

This detects many data races in concurrent code. If you write goroutines and shared state, this is a critical tool.

### `go vet`

```bash
go vet ./...
```

`go vet` looks for suspicious constructs that compile but are likely wrong.

### `go test -cover`

```bash
go test -cover ./...
```

Coverage is useful as a signal, not a religion. High coverage does not guarantee meaningful tests, but very low coverage may show untested risk.

### Useful External Tools

- `staticcheck` for deeper linting and bug finding
- `govulncheck` for known vulnerability detection in dependencies and reachable code
- `goimports` for formatting plus import cleanup
- `pprof` for CPU and memory profiling

These are not built into the core language, but they fit naturally into Go's tooling culture.

## Build and Release Workflows

Go makes builds operationally simple.

### Build a Binary

```bash
go build ./cmd/api
```

### Cross-Compile

```bash
GOOS=linux GOARCH=amd64 go build ./cmd/api
```

This is a practical reason Go is so common in infrastructure tools. Cross-platform binaries are relatively easy to produce.

### Why Single-Binary Delivery Matters

In deployment pipelines, simplicity is leverage.

- containers become smaller and simpler
- startup is predictable
- dependency packaging becomes easier
- CI artifacts are easier to reason about

## Documentation as Part of the API Surface

Go places real value on package and symbol documentation.

Exported identifiers should usually have comments when the package is intended for reuse.

Why this matters:

- tooling can surface docs automatically
- package APIs become easier to consume without reading implementation details
- maintainers communicate intent, not just mechanics

## Real-World Usage Patterns

### Service Repository Layout

A production service often has:

- one or more `cmd/` entry points
- internal packages for handlers, business logic, storage, and config
- tests next to each package
- benchmark and race-check jobs in CI

### Testing Strategy

Healthy Go services usually combine:

- unit tests for pure logic
- handler tests with `httptest`
- integration tests for DB and network boundaries
- benchmarks for hot paths
- race detection for concurrent subsystems

### Tooling in CI

A practical CI pipeline often runs:

- `go fmt` or formatting checks
- `go test ./...`
- `go test -race ./...`
- `go vet ./...`
- optional static analysis and vulnerability scanning

## Common Mistakes and Misconceptions

### Mistake: Creating Huge Project Layouts Too Early

Structure should reflect real complexity. Decorative folders make learning and maintenance harder.

### Mistake: Treating `pkg/` as Mandatory

It is only a convention. Many good Go services never use it.

### Mistake: Exporting Too Much

Large public package surfaces make refactoring harder and couple packages unnecessarily.

### Mistake: Ignoring Import Cycles Until Late

If your packages keep wanting to import each other, the boundaries are probably wrong.

### Mistake: Writing Only Happy-Path Tests

Most production failures happen in error paths, edge inputs, and timeout scenarios.

### Mistake: Optimizing from Guesswork Instead of Benchmarks

Measure before changing code for performance reasons.

### Mistake: Treating Coverage as the Goal

Coverage can be useful, but well-chosen tests matter more than inflated percentages.

## Summary

Go packages and modules keep code organization and dependency management explicit. The testing package and standard toolchain make quality checks part of ordinary development rather than an extra framework burden.

The big practical lessons are:

- packages shape design boundaries
- modules shape dependency boundaries
- small APIs and clean imports keep code maintainable
- table-driven tests and `httptest` are core testing patterns
- benchmarks, race checks, and tooling provide objective feedback
- operational simplicity is one of Go's biggest strengths

The next step is to bring all of this together in production system design: HTTP servers, request lifecycles, context propagation, timeouts, graceful shutdown, observability, and distributed systems patterns in Go.