Scaling
nl6 is tested up to 30,000 concurrent simulated devices on a single host. Each device has its own IP, SNMP listener, SSH server, and flow exporter, so the working set is dominated by file descriptors and the Go runtime's goroutine / heap footprint rather than raw CPU.
Resource envelope
| Dimension | Footprint |
|---|---|
| Memory | About 16 KiB of live heap per device: 78.5 MiB measured at 5,000 devices, see Profiling. |
| CPU | Minimal during steady state; bursts during device bring-up. |
| File descriptors | Dominated by per-device sockets. nl6 raises its soft limit to the hard limit at startup and the packaged systemd unit sets LimitNOFILE=1048576; intervene only when the hard limit is capped (see below). |
| Network | nl6sim namespace isolation prevents systemd-networkd overhead. |
Optimisations already in place
The simulator ships the following out of the box, with no tuning required:
- Pre-generated 100-point metric arrays. CPU / memory / temperature / GPU metrics are computed once at startup and indexed on poll.
- Lock-free
sync.Mapfor O(1) OID lookups under concurrent SNMP load. - Pre-computed next-OID mappings for
GETNEXT/WALKwithout table scans. - Buffer pool for SNMP reads, which reduces GC pressure on sustained traffic.
- Shared SSH / TLS keys across all devices, which avoids per-device key generation.
- Parallel TUN pre-allocation.
prealloc.gospins up 100 to 200 workers by default (sized by batch;max_workersin the create request overrides it, clamped to 500) to bring a large fleet online in seconds.
See Architecture for the component map.
Prepare the host
Run these before a large deployment:
- File-descriptor limits. Each device opens several sockets, so a large fleet needs a high
nofile. The Go runtime nl6 is built on raises the process's soft limit to the hard limit at startup, so on a typical host no action is needed. You only have to intervene when the hard limit is capped low, for example in a restrictive container or a systemd unit withLimitNOFILE=…:1024. The packaged unit underdeploy/packages/systemd/setsLimitNOFILE=1048576, so a.deb/.rpminstall is covered. Otherwise raise the hard ceiling:For a persistent limit, raiseulimit -Hn 1048576 # current shell (then nl6 lifts the soft limit to it)LimitNOFILEon the systemd unit ornofilein/etc/security/limits.conf, preserving any existing PAM limits. - Keep network namespaces enabled (default). Only pass
-no-namespacefor debugging. Running in the root namespace pulls systemd-networkd into every interface change and kills throughput. - Prefer the container path for repeatable setup. The Docker image bundles the dependencies and the tuning above.
Monitor a running fleet
ulimit -Hn/ulimit -Snmust exceed the device count by a comfortable margin (each device opens several sockets).htopduring bring-up shows a short CPU spike, which is normal as TUN interfaces come up in parallel. Steady-state load should be near-idle.ip netns exec nl6sim ip addrconfirms TUN interfaces exist inside the namespace. Unexpected entries in the host namespace usually mean-no-namespacewas used./api/v1/system-statsreturns the current file-descriptor count, memory, and load average. See Web API.
Scale in a container
When running under Docker, pair the host tuning above with:
--cap-add=NET_ADMIN+--device=/dev/net/tunso the container can manage TUN / netns.--network=hostso per-device TUN IPs are reachable from outside the container.- A memory budget of about
16 KiB * device_countof live heap plus a comfortable buffer for the Go runtime.
See Docker for the full bring-up recipe and Troubleshooting for bring-up failures.
Kubernetes is not a supported deployment target.
deploy/helm/nl6-minion/ is a single-node lab chart, not a supported path.
Why Kubernetes is out of scope lists the constraints.