I/O Latency

Analyzing Network Latency with BCC

Network latency involves measuring packet processing delays across the kernel stack. BCC can trace packet arrival and processing times using xdp or tc (traffic control) programs.

Example: XDP Latency Measurement

// xdp_latency.c
#include <uapi/linux/ptrace.h>
#include <linux/skbuff.h>

struct latency {
    u64 start;
};

BPF_PERCPU_ARRAY(latency_map, struct latency, 1);

int xdp_prog(struct xdp_md *ctx) {
    struct latency *lat = latency_map.lookup(&cpu);
    if (!lat) return XDP_ABORTED;
    lat->start = ctx->rx_timestamp;
    return XDP_PASS;
}

int xdp_prog_end(struct xdp_md *ctx) {
    struct latency *lat = latency_map.lookup(&cpu);
    if (!lat) return XDP_ABORTED;
    u64 latency = ctx->rx_timestamp - lat->start;
    bpf_trace_printk("latency: %d ns\n", latency);
    return XDP_ABORTED;
}

Compile and load this program using clang and bpftool, then monitor the output to identify delays in packet processing. Use bpftool prog detach to cleanly exit the program.

Note: The ctx->rx_timestamp field requires the CONFIG_XDP_USE_QDISC kernel configuration to be enabled. This may not be available on all systems.

Using tc for Network Latency

For traffic control-based analysis:

# Measure packet processing delay with tc
tc qdisc add dev eth0 root netem delay 10ms

Note: netem simulates network delay, not actual packet processing latency. Actual measurement requires tracing specific tc events like tc_queueing_enqueue or tc_queueing_discard. Combine this with BCC's tc tracing tools to correlate delays with specific traffic patterns:

# Trace tc qdisc with bpftrace
bpftrace -e '
tracepoint:tc:tc_queueing_enqueue {
    printf("Enqueued packet: %d bytes\n", args->len);
}
tracepoint:tc:tc_queueing_discard {
    printf("Discarded packet: %d bytes\n", args->len);
}
'