Load-test scenario report schema
The report is the machine-readable output of a finalized scenario.
It is the authoritative record of what nl6 sent, which an operator diffs against a monitor's received counts to localize missed or duplicated telemetry.
It is built once at stop/abort, immutable thereafter, and served by GET /api/v1/scenarios/{id}/report (also returned by stop).
See the API reference and the scenarios guide.
Shape
{
"summary": {
"id": "s-000001",
"phase": "stopped",
"protocol": "syslog",
"metadata": {
"config_sha256": "9b8d8c9c…969314",
"resolved_participants_sha256": "3f1c…a204",
"seed": 42,
"nl6_version": "v0.16.0",
"t0": "2026-07-18T09:00:05.000Z",
"t1": "2026-07-18T09:00:07.000Z",
"drain_end": "2026-07-18T09:00:07.500Z",
"sub_window_count": 10,
"sub_window_duration": "200ms",
"run_tags": {
"protocol": "syslog", "mechanism": "window_source_ip", "value": "",
"pen": 0, "pen_required": true, "degraded": true,
"note": "no PEN configured (-scenario-pen); SD-PARAM lever unavailable, isolate by source IP + [t0,t1)"
},
"fidelity": {"silent_at_start": true, "changed_during_window": false}
},
"duration": "2s",
"participants_armed": 2,
"participants_excluded": 1,
"emitted": 40, "in_window": 40, "drain": 0,
"suppressed_pre_window": 0, "send_failures": 0, "dropped": 0,
"informational": {"background_suppressed": 0, "requested": 40, "deferred": 0,
"informs_acked": 0, "informs_pending": 0},
"sub_windows": [4, 4, 4, 4, 4, 4, 4, 4, 4, 4],
"excluded": [
{"device": "10.42.0.9", "reason": "device not found",
"remediation_hint": "create the device before arming, or remove it from the scenario"}
]
},
"counters": [
{"protocol": "syslog", "source_ip": "10.42.0.1", "collector": "10.0.0.9:514",
"emitted": 20, "sent": 20, "in_window": 20, "drain": 0,
"suppressed_pre_window": 0, "send_failures": 0, "dropped": 0,
"informational": {"background_suppressed": 0, "requested": 20, "deferred": 0,
"informs_acked": 0, "informs_pending": 0},
"sub_windows": [2, 2, 2, 2, 2, 2, 2, 2, 2, 2]}
],
"applications": [],
"l7_values": []
}
(The example is a syslog run, so applications and l7_values are empty.
See applications[] and l7_values[] for populated flow-scenario examples.)
The top-level blocks always serialize in the order summary, counters, applications, l7_values, so a streaming consumer sees the aggregate first and can rely on the trailer position of the two flow blocks.
summary
| Field | Type | Meaning |
|---|---|---|
id | string | Scenario ID (s-000001). |
phase | string | Terminal phase: stopped (window elapsed / explicit stop) or aborted (graceful shutdown). |
protocol | string | Participating protocol. |
metadata | object | The reproducibility fingerprint + actual window timestamps (see below). |
duration | string | t1 − t0 as a Go duration, from the monotonic clock. |
| It is the window that actually ran. | ||
participants_armed | number | Devices that armed and ran. |
participants_excluded | number | Declared participants that did not arm. This is the true total, derived from the exclusion counter rather than from len(excluded), which is capped (see excluded_truncated). |
emitted … dropped | number | Fleet-wide sums of the per-device identity buckets below. |
informational | object | Fleet-wide sum of the disclosure counters (see counters[].informational). Outside the identity. |
sub_windows | array | Fleet-wide loss localization: in-window sends per time bucket, as the element-wise sum of the counters[].sub_windows rows. Length metadata.sub_window_count; sums to in_window. See Loss localization. |
excluded | array | Arm-time exclusions, each {device, reason, remediation_hint}. Capped at 1,000 rows (900 for arm-time exclusions + 100 reserved for start-time ones). Uncapped, one row per unresolved participant would be ~14 MB at the participant ceiling. |
excluded_truncated | bool | Present (true) only when the row cap dropped rows, i.e. participants_excluded > len(excluded). Absent otherwise, so the common case is unchanged. |
excluded_by_reason | object | reason → count over every exclusion, capped or not. This is where the complete disclosure lives when excluded is a sample. Absent when there are no exclusions. |
summary.metadata
The reproducibility fingerprint plus the timestamps the run actually observed.
Copy the (config_sha256, seed) back into a resubmit on the same nl6_version to re-run a scenario exactly.
Resubmitting an archived body: if it carries a drain key, strip it.
That field is refused with a 400, because it configured nothing.
A body that never carried one hashes to the same config_sha256 as it always did, so baselines stay comparable.
Two fingerprints appear here and they answer different questions.
config_sha256 pins what you declared.
It is the submit-time idempotency key.
resolved_participants_sha256 pins what actually ran.
They diverge as soon as membership is derived rather than enumerated: a participants_cidr scenario resolves against the live fleet, so one config_sha256 legitimately produces different participant sets on different days, or against a half-built fleet.
When two runs share a config fingerprint but disagree on counts, comparing the resolved digest tells you in one step whether you are looking at a pipeline change or a different fleet.
| Field | Type | Meaning |
|---|---|---|
config_sha256 | string | SHA-256 over the canonicalized submit config, which is the declared intent. |
resolved_participants_sha256 | string | SHA-256 identifying the devices that actually participated. Computed over post-prune membership (the same set counters[] serializes and participants_armed counts), so a run that lost devices between arm and start cannot digest identically to a whole one. Independent of how membership was declared: an explicit list and a prefix selector resolving to the same live devices agree, which is what keeps a baseline comparable to a re-declared repeat. |
seed | number | The seed that pinned every random draw. |
nl6_version | string | Simulator version that produced the report. Reproduction is guaranteed only on the same version. |
t0 | RFC3339-ms | Actual window open (emission start). |
t1 | RFC3339-ms | Actual window close. This is the planned T1, or the abort instant for an early abort (never later than planned T1). |
drain_end | RFC3339-ms | When the drain barrier finished and the report was finalized. Observed, not configured: the barrier returns as soon as the writes admitted before T1 return, typically single-digit milliseconds after t1. It is also bounded: if those writes have not returned within the ceiling, the barrier gives up and drain_end is the give-up instant. See drain_stragglers. |
incomplete_joins | array | Absent on a healthy run. Names the finalize waits that did not complete within the budget: syslog-scheduler, trap-ticker, flow-ticker. These run ahead of the drain barrier, and the goroutines behind them are not cancelled, so they may have moved counters after this report was taken. Same lower-bound caveat as drain_stragglers, for the same reason. |
drain_stragglers | number | Absent on a healthy run. Present only when the drain barrier hit its ceiling and gave up, and then it is an upper bound on the sends still in flight at that moment (a send completing while the barrier reads the counter is still counted). One unit is one admission, not one record: the syslog and trap paths admit a single write, while the flow paths admit a whole paginated Tick batch, so one flow straggler can stand for hundreds of records. A run that reports it was finalized while its counters were still moving, so the totals are a lower bound rather than a settled figure and an unidentified subset of participants may not satisfy the ledger identity. The count is fleet-wide and carries no attribution, so the report cannot say which participants those are. Deliberately its own field and never folded into dropped: a straggler was admitted and may still have reached the collector, so calling it dropped would assert an outcome nothing observed. See The drain barrier is bounded. |
sub_window_count | number | Loss-localization granularity: the number of equal time buckets [T0,T1) is sliced into (currently 10). |
sub_window_duration | string | Width of one bucket as a Go duration, computed as the planned window / sub_window_count (the basis fires were bucketed against). Bucket i covers [T0 + i·d, T0 + (i+1)·d). For an aborted run the buckets after the abort instant are empty (bucketing uses the planned t1, not the shortened actual one). |
rate | object | Rate disclosure: {requested_per_device, paced, achieved_per_device}. paced=false means this protocol's emission cadence is not driven by the scenario rate at all (gnmi-dial-out streams at its own SAMPLE interval), so achieved_per_device still reports what happened but the request explains none of it. achieved_per_device counts in-window records only, so a capture it is compared against must be bounded to [t0, t1). See achieved_per_device is an in-window rate. |
fidelity | object | Fleet-silence disclosure, always present: {silent_at_start, changed_during_window}. silent_at_start is the fidelity value in force at t0. changed_during_window is true when fidelity was toggled, or a timed revert fired, between t0 and finalize. A true there means non-participant devices resumed or ceased autonomous push part-way through the window, so the collector-side accept rate covers a mixture. The ledger stays exact either way, which is why this needs saying out loud. |
rate_cap | object | Shared-cap disclosure, present only when this run's protocol has a fleet-wide ceiling in force ({per_second, shared_with}). Rate limiters are per protocol (syslog and SNMP trap each own one, flow protocols and gNMI dial-out have none), so only same-protocol runs contend. shared_with names the same-protocol scenarios whose windows overlapped this one, sequence-ordered. An empty list means no peer scenario overlapped. It does not mean the bucket was uncontended: the scenario scheduler shares the fleet limiter, and background firing spends a token per pop even for fires the scenario gate suppresses, so on a busy non--fidelity fleet a solo run is still throttled by background traffic. The cap is the one in force when the run began, not when the report was fetched. Currently emitted for syslog only, because it is the one protocol whose scenario emission provably passes through the fleet limiter. -trap-global-cap governs background trap firing, which the scenario trap path does not go through. A run that shared its bucket did not measure what it would have measured alone, and this is what makes that visible in the artifact instead of silently changing the numbers. Overlaps are recorded as they begin, not reconstructed at finalize, so a peer stopped and deleted before this run finishes is still named. |
run_tags | object | Run tagging: how this run's traffic is isolated from background noise per its protocol's lever, as {protocol, mechanism, value, pen, pen_required, degraded, note}. See Run tagging. mechanism is one of syslog_sd_param, snmp_enterprise_varbind, netflow9_source_id, ipfix_odid, sflow_sub_agent_id, gnmi_synthetic_path, window_source_ip. degraded=true means a PEN-dependent lever fell back to window_source_ip because no -scenario-pen was set. |
achieved_per_device is an in-window rate
achieved_per_device = sum(counters[].in_window) / (t1 - t0) / len(counters)
Both halves of that are deliberate.
in_window counts the records whose socket write returned inside [t0, t1), and the denominator is that same window.
Nothing else belongs in either half.
in_window excludes records that were produced during the window but written after it.
Those are counted under drain instead.
Attributing them to the window would divide them by the window's own duration, which inflates the rate by exactly the records the window did not have time to emit.
So the exclusion is correct, and it is also tiny: post-T1 fires are suppressed at generation, so the drain bucket can only catch work already admitted at the T1 instant.
That is one write on the syslog and trap paths, or one paginated batch on the flow paths (the flow exporter admits around a whole Tick).
On syslog, with a 30 s drain configured on the then-existing knob, drain_end landed 9 ms after t1 and drain was 0.
To compare against a capture, bound the capture to [t0, t1). That is the whole correction.
Do not adjust the figure by a drain: the tail is bounded by that admitted work rather than by any duration, so it cannot move a 120 s window by percent.
drain is not a configurable field, and submitting one is a 400.
The drain barrier is bounded
The barrier that produces drain_end waits for every send admitted before T1 to return, and the shutdown path runs it, so the wait is capped.
Uncapped, one admitted send that never returned would hold shutdown open indefinitely.
Two cases reach that state.
A stream transport whose write sets no deadline blocks for as long as it blocks.
And an admitted send that never completes at all, because its write path panicked or a callback was dropped, would never return no matter what deadline the transport carried.
That second case is why the barrier is bounded rather than the transports. A per-transport write deadline cannot see it, and it could not bound the total anyway: syslog TCP serialises behind a per-connection mutex, so a device's worst case is its own 2 s write timeout times the sends queued behind it.
The whole waiting phase shares one budget. Finalize joins the scenario's scheduler and its trap and flow tickers before it reaches the barrier, and every one of those joins is bounded too: the syslog and trap schedulers fire inline, so a stalled scheduler-driven write parked finalize with the barrier ceiling never armed.
All four waits share a single 60 s budget, and whatever did not complete is named in incomplete_joins.
Nothing is cancelled in either case, so a report carrying incomplete_joins or drain_stragglers is a lower bound over a set that was still moving.
The barrier now gives up after a fixed ceiling of 60 s and records how many sends were still outstanding in drain_stragglers.
The presence of that field is the only reliable signature of a give-up. A drain_end 60 s past t1 is neither necessary nor sufficient: the ceiling is armed only once finalize reaches the barrier, which is after the unbounded joins described below, so on a truncated run drain_end is at least 60 s past t1 and can be arbitrarily further.
The stragglers are not cancelled, because nothing can interrupt a write parked in the kernel.
They keep running and keep moving ledger counters after the report was snapshotted.
So a report carrying drain_stragglers should be read as a lower bound taken over a set that was still moving, not as a settled measurement.
The counters are atomics, so nothing is corrupted and no participant's row is invalid on its own; what is lost is the guarantee that a participant's buckets add up, because emitted is incremented when a record is generated and sent only when its write returns.
A straggler sits between those two increments at the instant of the snapshot.
On a healthy run the field is absent and none of this applies.
An earlier version of this section claimed the figure carried a bias proportional to the drain's share of the window, and advised dividing sent by the window plus the drain instead.
Both were wrong, and the advice made measurements worse rather than better: sent already includes the drain bucket, and that bucket is ~0, so lengthening the denominator by a drain that nothing emitted into deflates the result by drain ÷ (window + drain).
On a 120 s window with a 5 s drain that is 5/125 = 4.0 % of pure, self-inflicted error.
(The superseded sentence quoted 4.2 %, which is 5/120.
That is the drain's share of the window, the quantity its own wrong model was about, not the error its own remedy introduced.)
Setup for both parts: netflow9, five participants, 120 s window, capture taken on the emitting node over a veth (loopback would not fragment, which is why an earlier capture misled). Template FlowSets subtracted, non-first fragments skipped.
Part one ran at rate 4/device, seed 42, with capture and report taken on the same clock.
It found 2349 wire data records against 2349 ledger records in [t0, t1), zero ledger error, with 0 records in [t1, drain_end) and 0 after it.
Part two re-analysed the four original cells that had produced the −3 % to −8 % figures.
Every one of them lands on its published achieved_per_device at published precision once three measurement-side terms are removed.
Take the rate-8 cell.
Its capture holds 4327 data records, the denominator for both percentages below:
| term | records | share of the 4327 captured |
|---|---|---|
| template FlowSets counted as data records ("phantoms") | 10 | 0.23 % |
| emission after the window, counted as wire¹ | 320 | 7.4 % |
| ledger error | 0 | 0 % |
¹ Those 320 were separated by a rule fixed before the comparison (records after the largest inter-datagram gap past 80 % of the window), because these older captures have no report JSON alongside them, so t1 is inferred rather than read.
Four independent cells landing on four different published values is what carries the conclusion, not the rule.
The third term is not a record count and so is not a row above: the capture span was used as the denominator instead of the window. That is 121.2 s against 120 s, a further 1 % deflation.
The post-window burst is not drain. It is traffic emitted after the scenario stopped and the gate came off, which the ledger never counts and should not: it belongs to no window.
What this means in practice. achieved_per_device answers "did pacing hit its target" directly, and it is comparable across runs of different window lengths: it is an in-window count over its own window, and with the drain model gone no term in it scales with window length.
What a short window still costs is precision, not bias.
With fewer records, first-fire alignment and scheduler jitter are a larger share of the total.
For an absolute rate, measure on the wire with the capture bounded to [t0, t1) and template records excluded.
Reproducing resolved_participants_sha256
The encoding is deliberately the dumbest one that a checker can reproduce without a parser: each participating address followed by \n, in byte order, SHA-256, hex.
So a collector that recorded which sources it received from can answer "did I receive from the same fleet nl6 sent from?" with one string comparison:
# "$IPS" = the source addresses your collector saw during [t0,t1)
printf '%s\n' "$IPS" | LC_ALL=C sort | sha256sum
LC_ALL=C is required, not decoration.
Under a UTF-8 locale, glibc's collation ignores punctuation at the primary level, so sort orders 10.42.10.1 before 10.42.1.2 while byte order puts 10.42.1.2 first.
That yields a different digest and an operator reads a false "different fleet" from the very comparison this section exists to enable.
macOS/BSD sort happens to agree with byte order, so the mistake reproduces only on the Linux collector hosts that are the actual audience.
Byte order is a deliberate departure from the address order used for the excluded[] rows.
Those rows are read by humans, where 10.42.0.10 sorting before 10.42.0.2 looks broken; this is input to a hash function, where any total order does, and byte order is the one every language sorts strings in by default.
The trailing newline after the final address is part of the encoding.
Run-tag levers
metadata.run_tags.mechanism names the in-band lever a run's traffic can be filtered by on a shared collector.
Every protocol is also bounded by the window [T0,T1) and the participant source IPs; the lever is the second discriminator where the wire format has one.
| Protocol | mechanism | Lever | PEN? |
|---|---|---|---|
| NetFlow v9 | netflow9_source_id | filter received flows by the device's Source ID | no |
| IPFIX | ipfix_odid | filter by the Observation Domain ID (enterprise IE is a secondary, PEN-only lever) | no |
| sFlow v5 | sflow_sub_agent_id | filter by sub_agent_id | no |
| gNMI dial-out | gnmi_synthetic_path | dial-out stamps the device IP in Notification.Prefix.Target; filter by target | no |
| Syslog 5424 | syslog_sd_param | RFC 5424 SD-PARAM [nl6@<PEN> runId="<id>"] | yes |
| SNMP trap/inform | snmp_enterprise_varbind | enterprise varbind under the nl6 PEN | yes |
| NetFlow v5 | window_source_ip | no taggable field, so isolate by participant source IPs + [T0,T1) | n/a |
A PEN-dependent lever needs -scenario-pen.
Without it, mechanism is window_source_ip and degraded is true.
How to use the levers when reconciling is in Scenarios → Run tagging.
counters[] (per participant)
One row per participant, keyed by the join tuple (protocol, source_ip, collector).
This is the same tuple a collector groups its received counts by, so the two sides line up row-for-row.
Every ledger field is always present (explicit zeros, never omitted), so a zero-valued row still diffs cleanly.
| Field | Meaning |
|---|---|
protocol | Participating protocol. |
source_ip | Device management IP (the emitter). |
collector | Configured collector host:port for this device (empty if the device was deleted post-window). |
emitted | Records generated: gate-passed fires + emission-suppressed pre-window fires. |
sent | in_window + drain, the loss denominator for reconciliation (convenience; derived). |
in_window | Records sent (write returned success) with the write-return timestamp in [T0, T1). |
drain | Records whose write returned at or after T1, the drain barrier's tail. Bounded by the writes already in flight at T1, so on a healthy run it is 0; it is not a configurable grace period. |
suppressed_pre_window | State-driven / on-demand fires that occurred before T0. They are counted but not emitted on the wire. |
send_failures | Resolve / encode / write errors (nl6 could not send). |
dropped | Records generated but never confirmed on the wire. The barrier waits for every fire it admitted, so this is not a "slow write" bucket: the real causes are a fire that reached the barrier after it closed (the detach/teardown race) and a shutdown-race socket drop. |
informational.background_suppressed | Informational, quarantined in its own sub-object. Counts background-cadence fires the gate suppressed for this participant during the scenario. Deliberately not a flat sibling of the identity buckets and not part of the ledger identity. |
informational.informs_acked / informs_pending | SNMP INFORM ack settlement (best-effort, collector-side). An origination counts sent at first-transmit; informs_pending = originations − acked at report time (still-awaiting-ack). Zero for fire-and-forget traps and non-trap protocols. Outside the identity. |
informational.requested | Scheduler demand, meaning every fire the scenario scheduler popped (pre-limiter). requested = sent + deferred + send_failures + dropped. |
informational.deferred | Fires the shared global cap had no token for. They are throttled, not fired, NOT lost. Outside the identity and the loss denominator, so a cap throttle never masquerades as pipeline loss. |
sub_windows | Loss localization: this participant's in-window sends per time bucket. Length metadata.sub_window_count; sums to in_window. See Loss localization. |
The six identity fields (emitted + the five loss buckets) are flat siblings.
The nested informational object carries five disclosure counters, always present: background_suppressed, requested, deferred, informs_acked, informs_pending.
A consumer computing the identity iterates the flat fields and never has to know which keys to exclude.
applications[] (fleet-wide flow traffic ground truth)
For scenarios on a flow protocol (netflow5 / netflow9 / ipfix), one row per distinct (l4_proto, dst_port, application_id) across all sent flow records, the trusted-sender reference for validating a collector's per-application aggregation.
Rows are sorted ascending by the numeric (protocol, dst_port, application_id) key, so on one port a plain row sorts before the NBAR2 rows.
The block is always present ([] for non-flow and sflow scenarios).
Additive block per the evolution policy below.
"applications": [
{"l4_proto": "tcp", "dst_port": 443, "app_hint": "https",
"records": 3, "bytes": 3000, "packets": 30,
"avg_bytes_per_second": 500.0,
"sub_window_bytes": [300, 300, 300, 300, 300, 300, 300, 300, 300, 300]},
{"l4_proto": "tcp", "dst_port": 443, "application_id": 50332091, "application_name": "ssl",
"app_hint": "https",
"records": 2, "bytes": 2000, "packets": 20,
"avg_bytes_per_second": 333.3,
"sub_window_bytes": [200, 200, 200, 200, 200, 200, 200, 200, 200, 200]}
]
(A 6-second window carrying 3000 in-window bytes → 3000 / 6 = 500.0 B/s.)
The join key has three parts, and the third is what separates NBAR2 traffic from plain traffic on the same port. The first row above is plain IPFIX (or NetFlow) traffic to 443/tcp; the second is Cisco AVC traffic to the same port whose records carried an applicationId.
A consumer that groups only on (l4_proto, dst_port) sees both rows for one port and must sum them to get the old single row.
application_id is the 32-bit value the record's IE 95 carries: the RFC 6759 classification engine id in the top 8 bits and the selector in the low 24, so engine 3 (IANA-L4, port-based) with selector 443 is 3 × 2^24 + 443 = 50332091.
It is resolved from the sending device's NBAR2 catalog at the moment the record is counted, never from a per-record index, so two device types whose catalogs list applications in different orders still fold into one row per wire id.
A collector that decodes applicationId as a big-endian integer (IPFIXcol2 with octetArrayAsUint, for instance) compares the number directly.
| Field | Type | Meaning |
|---|---|---|
l4_proto | string | Transport protocol name (tcp / udp / icmp; numeric fallback). Join key, part 1. |
dst_port | number | Flow destination port. Join key, part 2. ICMP records carry zero source and destination ports (ICMP has no transport ports), so ICMP traffic aggregates under a single (icmp, 0) row. |
application_id | number | Wire applicationId (IE 95) as an integer. Join key, part 3. Omitted on a row of plain records (no NBAR2), so a fleet with no NBAR2 device serializes exactly as before this field existed. |
application_name | string | The catalog's applicationName for application_id, the same string the device advertises in its RFC 6759 application table. Omitted with application_id. Informational, like app_hint. |
app_hint | string | Convenience label from a built-in well-known-port map (443 → https, 53 → domain, …); "" for unmapped ports. Informational: collector classification is user-configurable, so join on the key, never on the hint. |
records | number | Sent flow records for this row (sent basis: in-window + drain). For a pure flow scenario, Σ applications[].records == summary.sent. |
bytes | number | Sum of the records' flow byte counters, exactly what a conforming collector sums for the same window. |
packets | number | Sum of the records' flow packet counters. |
avg_bytes_per_second | number | In-window bytes ÷ (t1 − t0) (actual window). The headline rate reference. Drain bytes stay in bytes (the reconciliation total) but are excluded here: the denominator is the window, and a drain byte was written outside it, so counting it would credit the window with bytes it did not carry. (There is no "drain time" to add to the denominator; the tail is a barrier, not a span.) In-window bytes = Σ sub_window_bytes. |
sub_window_bytes | array | In-window bytes per localization bucket (drain bytes excluded, the same convention as sub_windows vs sent). Informational: collectors interpolate a flow's bytes across its [start, end] interval, so per-bucket comparison is approximate; reconcile on totals (see validation methodology). |
sflow scenarios are excluded by design: an sFlow collector derives byte volumes by sampling extrapolation (frame_length × sampling_rate), not by summing record byte counters, so these totals are not the numbers a correct sFlow collector would report.
l7_values[] (layer-7 values from NBAR2 records)
One row per distinct (application_id, field, value) across all sent records that carried that value, positioned after applications.
field is http_host (the HTTP host, Cisco IE 12235 under PEN 9) or http_uri (the URI half of the HTTP URI statistics field, IE 9357).
Rows are sorted ascending by (application_id, field, value).
The block is always present and is [] unless an NBAR2 participant sent a record carrying a host or a URI.
"l7_values": [
{"application_id": 50331728, "application_name": "http", "field": "http_host",
"value": "www.example.com", "records": 6, "bytes": 6000, "packets": 60,
"avg_bytes_per_second": 1000.0},
{"application_id": 50331728, "application_name": "http", "field": "http_uri",
"value": "/index.html", "records": 10, "bytes": 10000, "packets": 100,
"avg_bytes_per_second": 1666.7}
]
| Field | Type | Meaning |
|---|---|---|
application_id | number | Wire applicationId of the records, as in applications[]. Join key, part 1. |
application_name | string | The catalog's name for it. Informational. |
field | string | http_host or http_uri. Join key, part 2. |
value | string | The hostname or URI as a string. For http_host this is the hostname WITHOUT the six-byte prefix 03 00 00 50 34 02 the wire carries in front of it (Cisco's layout); for http_uri it is the URI without the NUL and hit count that follow it. Join key, part 3. |
records / bytes / packets | number | Sent-basis totals of the records carrying this value, the applications[] convention. |
avg_bytes_per_second | number | In-window bytes ÷ actual window, the applications[] convention. |
The invariant is an inequality. For each field, Σ l7_values[field].records ≤ Σ applications[].records, and equality is not required: a DNS or SSL record carries no host and no URI, so it contributes to applications[] and to no row here.
Equality holds only when every sent record carried exactly one value of that field.
To reconcile http_host against a collector, strip the six-byte prefix from the decoded IE 12235 value first; a record whose value is exactly the six bytes carried no host and belongs to no http_host row.
libfds types both IE 12235 and IE 9357 as strings, and IPFIXcol2 drops non-printable bytes from a string unless its nonPrintableChar output option is on: off, it shows the URI alone (the NUL and count vanish) and a host as P4www.example.com (the prefix's two printable bytes and then the name); on, every byte is a \u00XX escape.
The 2-byte hit count that follows the URI on the wire is not part of the key.
The ledger identity
Every counter row (and the summary) satisfies, exactly:
emitted = in_window + drain + send_failures + dropped + suppressed_pre_window
sent = in_window + drain
informational.background_suppressed sits outside this identity by design.
It counts generation-suppressed background fires that were never generated as scenario records, which is exactly why it lives in its own sub-object rather than as a flat sibling.
Use sent (in_window + drain) as the number to reconcile against a collector's received count.
Field / semver evolution policy
The report is a versioned contract. Consumers should tolerate unknown fields.
- Patch / minor (backward-compatible): new fields may be added to
summaryorcounters; a new top-level block may be added aftercounters. Existing field names, types, units, and thesummary-before-countersorder never change within a major. - Major (breaking): renaming/removing a field, changing a type or unit, or reordering the top-level blocks. Breaking changes are gated on a major
nl6_versionbump and called out in the changelog. - The ledger identity above is a stability guarantee. It holds in every version that ships these fields.
config_sha256covers only the submit config, not the report shape; the report contract is tracked bynl6_version.- The submit config has no separate version of its own; it moves with
nl6_versiontoo, and a removed request field is a breaking change for harnesses. Removals are listed in the API reference's Removed request fields with the release that removed them, so a reader on an older binary can tell which one changed. So far:drain, refused with a400.
Future projections (additional protocols in counters, richer loss-localization blocks) are additive under this policy.
One assumption a consumer may have held is now false, and is called out rather than left to be discovered: participants_excluded == len(excluded) no longer holds once the 1,000-row exclusion cap bites.
excluded_truncated marks exactly that case and excluded_by_reason carries the complete breakdown.
The field names, types and order are unchanged, so this is minor under the policy above.
Still, read the count from participants_excluded, never from the array length.
Loss localization
A fleet total answers how much was lost; sub_windows answers where, by device-set (the per-participant counters[] rows, keyed on the join tuple) and by time (the array within each row).
The planned window [T0,T1) is sliced into metadata.sub_window_count equal buckets, each metadata.sub_window_duration wide; sub_windows[i] counts the in-window sends whose write-return time fell in bucket i ([T0 + i·d, T0 + (i+1)·d)).
An early abort shortens the run but not the bucket basis.
Buckets past the abort instant are empty.
- Bucketing choice. A fixed count (10), not a fixed duration, so the report stays bounded and window-length-independent (a 10 s window → 1 s buckets; a 10 min window → 60 s buckets).
- Scope. Localizes in-window sends only:
sum(sub_windows) == in_windowfor every row and the summary. Drain-tail sends are post-T1by definition and carry no sub-window (reconcile them via thedraintotal). - How to use it. Bucket your collector's received records the same way (receive-time relative to
T0, same bucket width) and diff per bucket: "1,204 records lost, all from[T0+30s, T0+45s]" points at a time window, not just a fleet total. Combined with the per-rowsource_ip, loss narrows to a device-set and a time span. - JSON only. The flat CSV projection is unchanged, so index-keyed CSV parsers keep working; localization lives in the JSON report.
Additive under the semver policy, which explicitly anticipates "richer loss-localization blocks."
CSV projection
GET /api/v1/scenarios/{id}/report?format=csv serves a flat text/csv projection of counters[], with one header row plus one row per participant, join-ready on the first three columns:
protocol,source_ip,collector,emitted,in_window,drain,suppressed_pre_window,send_failures,dropped,background_suppressed
syslog,10.42.0.1,10.0.0.9:514,20,20,0,0,0,0,0
syslog,10.42.0.2,10.0.0.9:514,20,20,0,0,0,0,0
The informational.background_suppressed counter flattens to a trailing background_suppressed column.
summary-level fields are not in the CSV.
It is purely the per-device counter projection, so it joins directly against a collector's received-counts export on (protocol, source_ip, collector).