Three ways /proc will ruin your afternoon

· 4 min read · go, linux
Contents

Building a process viewer in Go sounds simple. Read /proc, parse some numbers, render a table. The kind of project that takes a weekend and teaches nothing new.

It took longer than a weekend.

The parenthesis trap

/proc/<pid>/stat is a single line of space-separated fields. PID first, then the process name in parentheses, then state, ppid, and about 50 more numbers. Parsing it is a strings.Fields() call and some indexing.

Until a JVM thread shows up in /proc/<pid>/task/<tid>/stat as Main Thread, or a container sets its process name to worker (busy) via prctl. The kernel wraps the name in (parentheses) but does not escape spaces or parens inside it. So this:

100 (my cool (app)) R 1 100 100 ...

becomes this after strings.Fields():

["100", "(my", "cool", "(app))", "R", "1", "100", ...]

Every field after the name shifts. State reads as a number. PPID reads as something that was never a PID. Nothing panics — the values are syntactically valid, just completely wrong. The parser happily produces garbage for exactly the processes with interesting names.

The fix is older than Go itself. Tools like procps and htop have done it forever: find the last ) in the line, not the first space.

lp := bytes.IndexByte(data, '(')
rp := bytes.LastIndexByte(data, ')')
comm := string(data[lp+1 : rp])
rest := bytes.TrimLeft(data[rp+1:], " ")
fields := strings.Fields(string(rest))
// fields[0] = state, fields[1] = ppid, ...

The test that pins this needs a fixture with both a space and a paren in the process name. my cool (app) is the minimum viable stress test. Without it, every refactor that “simplifies” the parser back to strings.Fields() will pass the test suite and break in production.

The NUL byte surprise

/proc/<pid>/cmdline contains the argv of a process. Looks like a string. Reads like a string. Is not a string.

The kernel separates arguments with \0, not spaces. A program invoked as mybin "hello world" appears in /proc as mybin\0hello world\0. Split on spaces and hello and world become separate entries — but they were one argument. Don’t split at all and you get an opaque blob with embedded NULs. Either way, argument-level filtering breaks silently.

data, _ := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid))
data = bytes.TrimRight(data, "\x00")
argv := bytes.Split(data, []byte{0})

The more annoying part is writing test fixtures. Go string literals don’t have NUL bytes in a way that feels natural. printf '\0' in a shell or []byte{0} in Go test setup — either way, the fixture needs to contain the actual byte the kernel produces, not a space-separated approximation of it.

The 1.84e+19% CPU usage

Per-process CPU usage is a delta calculation: read cumulative ticks from /proc/<pid>/stat, wait, read again, divide by elapsed time. Store previous values in a map[int]uint64 keyed by PID.

This works until a process exits and its PID gets recycled. The new process has fresh counters — lower than the old ones. The subtraction new_ticks - old_ticks wraps around uint64, producing a number near math.MaxUint64. Divide by elapsed time and the UI renders 1.84e+19% CPU utilization for a process that just started.

The number is technically correct for unsigned arithmetic. uint64 has no overflow panic. The compiler is happy. The runtime is happy. Only the human reading the terminal output notices something is off.

if cur.ticks < prev.ticks {
    // pid reuse or counter reset — skip this tick
    continue
}
delta := float64(cur.ticks - prev.ticks)
cpuPercent := delta / (float64(userHZ) * dt) * 100

This generalizes to any monotonic counter sampled by a recyclable identifier. Network byte counters when an interface is recreated. Disk I/O counters when a device rotates out. Anything where the identity looks stable but the backing resource got swapped underneath.

The regression test injects a fake previous sample with 9,999,999 ticks against a current fixture of 30, and asserts the resulting percentage is zero. Without that test, the guard is one “cleanup” refactor away from disappearing.

The pattern

All three bugs share a trait: the code works on well-behaved inputs and breaks silently on real ones. No panics, no error returns, no obviously wrong types. Just quiet corruption that looks plausible until a specific process name, a specific argument format, or a specific PID lifecycle triggers it.

The /proc filesystem is documented. The edge cases are known. They’re just not the cases anyone writes first.

Sources