How do you diagnose goroutine leaks and upgrade safely to Go 1.26?

1. Scenario and success criteria

The service starts several workers for each batch and aggregates results over a channel. When one worker returns an error, the caller returns early. During peaks, memory and goroutine counts climb until restarting an instance temporarily helps. The upgrade must find goroutines that cannot unblock, preserve cancellation semantics, compare Go 1.26 GC behavior, and provide a reversible rollout.

Start with baselines: request latency percentiles, throughput, heap size, GC CPU, active goroutines, leak slope, and error rate. runtime.NumGoroutine alone cannot tell which goroutines are actually leaked.

2. Relevant Go 1.26 changes

Go 1.26 introduces an experimental goroutineleak pprof profile for a class of goroutines permanently blocked and unable to wake. Enable it with GOEXPERIMENT=goroutineleakprofile and expose it through net/http/pprof. It relies on garbage-collector reachability and cannot identify every leak, so treat it as a diagnostic signal rather than proof of absence.

The release also provides go fix modernizers, lets new accept an expression, and enables Green Tea GC by default. Evaluate language conveniences, experimental diagnostics, and runtime changes separately so one benchmark does not mix unrelated variables.

3. Build a minimal leak reproduction

This pattern returns on the first error while other workers still send to an unbuffered channel and eventually block forever:

func processWorkItems(ctx context.Context, ws []WorkItem) ([]Result, error) {
    ch := make(chan result)
    for _, w := range ws {
        go func() {
            value, err := process(ctx, w)
            ch <- result{value: value, err: err}
        }()
    }

    results := make([]Result, 0, len(ws))
    for range ws {
        r := <-ch
        if r.err != nil {
            return nil, r.err
        }
        results = append(results, r.value)
    }
    return results, nil
}

Inject deterministic errors, repeat the run, and increase batch size while watching goroutine count. Include caller cancellation, timeout, and successful completion paths so the fix does not only cover the happy path.

4. Fix cancellation, closing, and backpressure

Make senders honor ctx.Done() and ensure that receiver cancellation cannot leave senders behind. A buffered channel with an explicit capacity can work, as can one coordinator goroutine responsible for closing and aggregation. Multiple workers must not race to close one channel.

select {
case ch <- result{value: value, err: err}:
case <-ctx.Done():
}

A complete fix cancels a derived context on the first error, waits for all workers to exit, and then returns the error. errgroup.WithContext can coordinate that lifetime, but every worker still has to propagate the context and apply timeouts to external I/O.

5. Enable the goroutineleak profile and gather evidence

Build an isolated binary with GOEXPERIMENT=goroutineleakprofile and sample a protected pprof endpoint. Compare the leak profile, goroutine stacks, heap profile, trace, and business metrics over the same window. An empty profile does not prove there is no leak: blocking primitives reachable from globals may not be classified by this mechanism.

Sample during load and fault injection, recording Go version, experiment flags, request load, and sampling interval. In production, restrict endpoint access, sampling frequency, and retention so diagnostics do not become an information-exposure surface.

6. Evaluate Green Tea GC and upgrade compatibility

Go 1.26 enables Green Tea GC by default. The release notes describe goals of better locality and CPU scalability when marking and scanning small objects, but the benefit depends on workload. Separate GC CPU, pause time, peak heap, and request latency in benchmarks and compare with the old version on identical hardware, compiler flags, and traffic.

If a regression appears, temporarily use GOEXPERIMENT=nogreenteagc as a diagnostic control before deciding whether to report it. Do not turn an experimental opt-out into a permanent architecture. Validate race behavior, cgo, plugins, and dependencies in a canary before increasing traffic.

7. Use go fix as an auditable modernization

go fix uses the same analysis framework as go vet to apply behavior-preserving modernizers; Go 1.26's new(expr) syntax is a representative target. Run it on a branch and review every diff, then use compilation, unit tests, race detection, and benchmarks to confirm semantics.

Do not rewrite the entire codebase merely to follow a new version. Keep automatic fixes separate from leak repairs and GC upgrades so one risk can be rolled back without removing another fix. Use explicit go directives and build constraints for generated code and cross-version modules.

8. Rubric and follow-ups

Must explain

• Explain the blocking cause and repair worker lifetime with cancellation, closing, and backpressure.
• Know that goroutineleak is an experimental, reachability-based profile that cannot cover every leak.
• Separate Green Tea GC, go fix, and version upgrade into independent baselines, canary checks, and rollback steps.

Follow-up questions

• If a leaked channel is held by a global registry, why might the profile miss it, and what evidence would you add?
• After the first error, how do you ensure a blocked network call also exits?
• When would you use GOEXPERIMENT=nogreenteagc, and how do you avoid masking the real issue?

Scoring guide

An excellent answer connects reproduction, lifecycle repair, diagnostic evidence, and upgrade governance: make every worker cancellable, confirm with profiles and stacks, then prove runtime safety with isolated benchmarks and a canary.