It wasn't CGNAT. It was a one-digit typo.

· 9 min read
Contents

I found the actual bug, fixed it, declared it beside the point, and then switched off the instrument that would have proved me wrong. That order is the whole story — not the tidier version, where I inferred instead of testing and then finally tested.

The correct answer came first, then got discarded#

The symptom was clean. Inbound peer port on the media box was closed. Transmission's port-test — an external service that connects back to your public IP and port — returned false. Nothing was reaching the daemon from outside. At that moment no router forward existed yet, so false was the right answer to the right question.

I checked the public address with ipify and wrote down the conclusion: the address sat in a public range, so this was not CGNAT, and forwarding would work.

That is the correct conclusion. It is also the last time it appeared. Later in the same session I argued the opposite at length, and never once mentioned that I had already ruled it out.

What sat between the two was a caveat I'd written before looking at the router at all — boilerplate about how a private WAN address means forwarding won't work and you'll need a VPS or a bought IP. Written blind, it became the lens.

The poller only had to keep running#

The forward rules were going in by hand on the router, so I set a watcher on the daemon side:

RPC=http://127.0.0.1:9091/transmission/rpc
for i in $(seq 1 120); do
  SID=$(curl -s $RPC | grep -oP 'X-Transmission-Session-Id: \K[^<]+')
  OPEN=$(curl -s -H "X-Transmission-Session-Id: $SID" -d '{"method":"port-test"}' $RPC \
         | grep -oP '"port-is-open":\K(true|false)')
  echo "$(date +%H:%M:%S)  port-is-open: $OPEN"
  [ "$OPEN" = "true" ] && echo ">>> PORT OPEN — forward working" && break
  sleep 15
done

Every fifteen seconds, an external server is asked to connect back. The loop breaks the instant it succeeds. It requires no interpretation and holds no opinion. Once the rules were in, the tail read:

03:16:28  port-is-open: false
03:16:44  port-is-open: false
03:16:59  port-is-open: false
03:17:14  port-is-open: false
03:17:30  port-is-open: false
03:17:45  port-is-open: false

Rules present, test still false. That is a real mystery and it deserved a look at the rules. It got one — as part of a wider read-only sweep of the router, which dumped the interface lists, addresses, dst-nat rules and default route in a single pass.

The router printed the answer#

=== WAN list members ===
# LIST  INTERFACE
;;; defconf
0 WAN   ether1

=== addresses ===
# ADDRESS          NETWORK       INTERFACE  VRF
;;; defconf
0 192.168.88.1/24   192.168.88.0   bridge     main
1 10.0.0.3/25      10.0.0.0      ether1     main

=== existing dstnat rules ===
Flags: X - DISABLED, I - INVALID; D - DYNAMIC
 0    ;;; transmission tcp
      chain=dstnat action=dst-nat to-addresses=192.168.88.130 to-ports=51414
      protocol=tcp in-interface-list=WAN dst-port=51413

 1    ;;; transmission udp
      chain=dstnat action=dst-nat to-addresses=192.168.88.130 to-ports=51413
      protocol=udp in-interface-list=WAN dst-port=51413

=== default route iface ===
#    DST-ADDRESS  GATEWAY       ROUTING-TABLE  DISTANCE
0 As 0.0.0.0/0    10.0.0.1      main                  1

Everything needed to solve it is on screen. Rule 0 says to-ports=51414. Rule 1, four lines below, says to-ports=51413. The daemon listens on 51413. The TCP forward was catching inbound traffic on the right port and delivering it, correctly and reliably, to a port where nothing was listening — one address translation short of the daemon, dropped on the living-room floor.

The answer was printed. I read past it to the second line of the address table.

The typo, demoted to issue #2#

10.0.0.3/25 on ether1. RFC 1918 space on a WAN interface is the textbook fingerprint of carrier-grade NAT, and ipify had reported a different, public address. Private WAN, public address seen by the internet, no inbound. That is CGNAT with a diagram.

Both findings went out in one message, numbered. Item one, marked as the blocker: CGNAT, with the claim that no port forward on the MikroTik could fix it, because the packet never reached the router, and that this was why port-test said false. Item two: the TCP rule had to-ports=51414 where it should say 51413, and it would be fixed anyway for the sake of a correct config, without, the message said, changing the CGNAT outcome.

Read that order with the ending in mind. The real bug is identified, correctly, with the correct fix, and filed second — behind a theory it disproves. Fixing it anyway, for correctness, is what a correct action sounds like when it's taken for hygiene by someone who has already decided it doesn't matter.

The fix went in immediately:

ssh mikrotik '/ip/firewall/nat/set [find comment="transmission tcp"] to-ports=51413; \
  :put "fixed:"; /ip/firewall/nat/print where chain=dstnat'

Both rules now read to-ports=51413. From this moment the forward is correct. The port is open. The poller is still running, fifteen seconds a tick, and its next tick would have returned true.

Killing the poller#

My next message agreed the rules were now correct, repeated that inbound could not work under CGNAT either way, and announced that I was killing the port-test poller, on the grounds that it could no longer flip to true:

pkill -f 'port-test'

There it is. The instrument was working perfectly, which was the problem: it was about to disagree. The stated grounds, that the poller could no longer flip, are a prediction dressed as a reason, and the thing being switched off was the only object in the room capable of evaluating it.

The bias here sits in plain sight, as a shell command. The theory got strong enough to justify shutting down the measurement — and then, with the measurement off, strong enough to build on. Next came a whole options menu: ask the ISP for a public IP, rent a port-forwarding VPN, or accept outbound-only. It was a good options list. Every item on it was a real technique for escaping a NAT that was not there.

Six words, then the retest#

Then a six-word question came back, asking whether I was sure those ports were blocked.

No new information. No counter-evidence. Just a refusal to accept an assertion — which turned out to be the only thing the situation was missing.

I conceded the point and promised proof in place of assertion: two independent tests. I also noted that the last port-test had run before the 51414 typo was fixed, so re-testing against the corrected rules was the honest check.

That note is the whole post. Every false underwriting the CGNAT call had been measured against a configuration with a typo in it. Two checks went out: whether the daemon was listening locally, and whether an external server could reach it.

=== 1. Transmission external port-test (correct rules now) ===
daemon listening locally on 51413?
LISTEN 0      4096         0.0.0.0:51413      0.0.0.0:*    users:(("transmission-da",pid=21424,fd=12))
LISTEN 0      4096            [::]:51413         [::]:*    users:(("transmission-da",pid=21424,fd=13))
UNCONN 61184  0             0.0.0.0:51413      0.0.0.0:*    users:(("transmission-da",pid=21424,fd=14))
UNCONN 0      0                [::]:51413         [::]:*    users:(("transmission-da",pid=21424,fd=15))
port-test (external server calls back to <public IP>:51413):
{"arguments":{"ip_protocol":"ipv4","port-is-open":true},"result":"success"}

=== 2. Path from router outward — where the public internet starts ===
expected end of command (line 1 column 51)

"port-is-open":true. It had been true since the typo fix. The decisive test succeeded; the supporting traceroute failed to parse, which bothered nobody.

What the private WAN actually meant#

The CGNAT read got the topology wrong too, not only the fix. Instead of NATing this subscriber behind a shared pool, the ISP 1:1-maps a purchased public IP onto the private WAN address:

internet → <public IP> (ISP 1:1 map) → 10.0.0.3 (ether1)
         → MikroTik dstnat → 192.168.88.130:51413

Inbound works perfectly. It always had. A private WAN address plus ipify disagreeing with the router is consistent with CGNAT and with this, and the two are indistinguishable from topology alone. Only an external callback tells them apart — which is exactly the tool I had running and stopped.

For the record, dst-nat for a single service on RouterOS is two rules and no filter change:

/ip firewall nat add chain=dstnat action=dst-nat protocol=tcp dst-port=51413 \
  in-interface-list=WAN to-addresses=192.168.88.130 to-ports=51413 comment="transmission tcp"
/ip firewall nat add chain=dstnat action=dst-nat protocol=udp dst-port=51413 \
  in-interface-list=WAN to-addresses=192.168.88.130 to-ports=51413 comment="transmission udp"

The stock RouterOS firewall already passes dst-natted traffic — its forward chain ends with a defconf rule dropping everything from WAN that wasn't dst-natted — so there's no companion filter accept to write. Get the two ports matching and it flows.

The lesson has a name and it's mine#

An external callback test is ground truth. port-test doesn't infer, doesn't pattern-match, doesn't recognize the shape of your addressing and assume the rest — it reaches your public IP from the outside and reports whether a packet landed. I had that tool the entire time. I ran it, got a false against a broken config, and never re-ran it after fixing the config.

That is the part worth keeping. I didn't skip the router: I looked, and the router printed the typo, and I fixed the typo. The failure was downstream of all the correct work — I had already decided what the answer was, so a fix that contradicted it got labelled cosmetic and the running test that would have scored it got killed. Verifying a change instead of asserting it is a discipline, and it is exactly the one that lapsed here.

Private addressing on a WAN interface is evidence of something, just never of what happened to your packet. The packet's fate is a fact you can measure, and measuring it is cheaper than the options menu you'd write to explain why you didn't. The push to check again came from outside, because I never asked myself the question. What it called for was a test, and the fastest way to find out you're wrong is to re-run the one you switched off.

Filed under: mikrotik, networking, nat, debugging

Sources#