Troubleshooting

Common issues and their solutions when running NAT46 on the NPA Publisher.

DNS Issues

AAAA Query Returns an A Record

Symptom: dig @127.0.0.1 <host> AAAA shows an A record in the answer section.

;; ANSWER SECTION:
v6app.internal.  300  IN  A  240.0.0.1

Cause: The old nat46 CoreDNS plugin is deployed. The fixed plugin returns NODATA for AAAA queries.

Fix: Rebuild the Docker image with the updated coredns-nat46-netlink plugin. Verify the fix is present:

grep "aaaa_nodata" ~/build/coredns/plugin/nat46/nat46.go

If this returns no match, the plugin source hasn't been updated.

SNAT DNS Resolution Failed

Symptom: Publisher logs show:

snatResolveFinish():0x0 SNAT DNS resolution failed host=v6app.internal

Cause: Usually the AAAA bug above. getaddrinfo(AF_UNSPEC) sends both A and AAAA queries. If the AAAA response contains an A record (DNS protocol violation), glibc discards both responses.

Fix: Deploy the fixed nat46 plugin. After rebuilding, A queries return synthesized addresses and AAAA queries return empty NODATA.

A Query Returns Nothing for IPv6-Only Host

Symptom: dig @127.0.0.1 <host> A +short returns no result.

Checks:

1. Verify the Corefile has a nat46 { ... } block:

   docker exec <container> cat /etc/coredns/Corefile

2. Verify CoreDNS can reach upstream DNS:

   dig @127.0.0.1 google.com A +short

3. Verify the host actually has AAAA records:

   dig @8.8.8.8 <host> AAAA +short

4. Check CoreDNS logs for errors:

   docker logs <container> 2>&1 | grep -i "nat46\|error"

CoreDNS Not Responding

Symptom: dig @127.0.0.1 . NS times out.

Checks:

# Is CoreDNS running?
docker exec <container> ps aux | grep coredns

# Check CoreDNS startup logs
docker logs <container> 2>&1 | tail -30

# Is port 53 bound?
docker exec <container> netstat -ulnp | grep 53

Network Issues

Jool SIIT Not Translating

Symptom: Packets reach tun0 with synthesized destination but never appear as IPv6 on the host interface.

Checks:

# Module loaded?
lsmod | grep jool_siit

# Instance exists?
sudo jool_siit instance display

# EAMT has mapping?
sudo jool_siit eamt display

# Global config
sudo jool_siit global display

If the EAMT is empty after DNS queries succeed, the nat46 plugin's netlink communication with Jool may be failing. Check CoreDNS logs for netlink errors.

Source EAMT Entry Missing

Symptom: IPv6 packets leave the host with an unroutable pool6 source address (e.g., fd8f:...:191.1.0.1) instead of the host's real IPv6. Responses never come back.

Check:

sudo jool_siit eamt display | grep 191.1.0.1

Should show 191.1.0.1/32 mapped to the host's IPv6 address.

Fix:

HOST_IPV6=$(jq -r '.host_ipv6' ~/resources/.nat46_config.json)
sudo jool_siit eamt add "$HOST_IPV6"/128 191.1.0.1/32

Legacy ip6tables MASQUERADE Rules

Symptom: NPA_NAT46_POSTROUTING chain exists in ip6tables but has zero packet counts.

Cause: Older wizard versions set up MASQUERADE rules. These are now unused because Jool bypasses netfilter.

Fix: The latest wizard cleans these up automatically on enable/disable. To remove manually:

sudo ip6tables -t nat -D POSTROUTING -j NPA_NAT46_POSTROUTING 2>/dev/null
sudo ip6tables -t nat -F NPA_NAT46_POSTROUTING 2>/dev/null
sudo ip6tables -t nat -X NPA_NAT46_POSTROUTING 2>/dev/null

Pool6 Route Missing

Symptom: Jool translates packets but they never reach the IPv6 destination.

Check:

ip -6 route show table local | grep fd

Fix:

POOL6=$(jq -r '.pool6' ~/resources/.nat46_config.json)
sudo ip -6 route add local "$POOL6" dev lo

No Traffic on tun0

Symptom: tcpdump -i tun0 shows no packets for synthesized destinations.

Checks:

1. Publisher connected to stitcher? Check agent.txt logs
2. Client connected and steering traffic? Check client logs
3. Private app configured in admin console?
4. App published to this specific publisher?

Container Issues

ip: command not found

Symptom: Cannot run ip addr, ip route, etc. inside the container.

Cause: Container built before iproute2 was added to the Dockerfile.

Temporary fix:

sed -i 's/^#//' /etc/apt/sources.list
apt-get update && apt-get install -y iproute2

Permanent fix: Rebuild the Docker image (iproute2 is now in the Dockerfile).

NAT46 Config File Not Mounted

Symptom: /home/resources/.nat46_config.json doesn't exist inside the container.

Check:

docker inspect <container> | grep -A5 resources

The host path ~/resources must be volume-mounted to /home/resources in the container.

apt-get Cannot Find Packages

Symptom: apt-get install fails with "Unable to locate package".

Cause: The Dockerfile comments out /etc/apt/sources.list to limit package installation in production.

Workaround:

sed -i 's/^#//' /etc/apt/sources.list
apt-get update

Publisher Process Issues

Publisher Not Starting

Check:

docker logs <container> 2>&1 | tail -50
ps aux | grep npa_publisher

Restarting the Publisher

Inside the container:

killall npa_publisher
# automatic.sh should restart it, or manually:
LD_PRELOAD=/home/libjemalloc.so.2 /home/npa_publisher \
  -a <address> -p <port> -l <loglevel> &

Diagnostic Commands Reference