Skip to content

Performance Monitoring

Linux network performance optimization requires a deep understanding of system metrics and tools to identify bottlenecks. Tools like sar, iostat, and netstat provide critical insights into CPU, disk I/O, and network stack behavior. This section explores their usage for diagnosing latency, packet loss, and resource contention.

Network and System Activity Monitoring with sar

The sar (System Activity Reporter) tool collects and reports system-wide metrics, including network traffic, CPU utilization, and disk I/O. It is part of the sysstat package and can log data over time for trend analysis.

Key Use Cases

  • Monitoring network throughput and packet loss.
  • Identifying CPU or disk contention affecting network performance.

Example Commands

# Check network device statistics (e.g., eth0)
sar -n DEV 1 5  # 1-second intervals for 5 samples

# Analyze packet loss and retransmissions
sar -n EDEV 1 5  # Ethernet device statistics

Interpretation

  • Look for high rx/tx (receive/transmit) rates indicating saturation.
  • Check dropped packets or retrans (retransmissions) for network errors.
  • Use sar -u to correlate CPU utilization with network activity.

Disk I/O Monitoring with iostat

The iostat tool (part of the sysstat suite) measures disk and CPU usage, helping identify I/O bottlenecks that indirectly impact network performance (e.g., slow storage causing application latency).

Key Use Cases

  • Detecting disk saturation or slow read/write speeds.
  • Monitoring latency for storage-backed network services (e.g., databases).

Example Commands

# Monitor disk I/O with extended statistics
iostat -x 1  # 1-second intervals, extended output

# Focus on specific devices (e.g., /dev/sda)
iostat -x /dev/sda 1

Interpretation

  • High await (average request wait time) or %util (disk utilization) indicates I/O contention.
  • Use iostat alongside sar to isolate whether disk I/O is the root cause of network latency.

Network Statistics with netstat

The netstat tool (or its modern replacement, ss) provides visibility into active network connections, routing tables, and protocol-specific statistics. It is invaluable for diagnosing connection issues or protocol-specific bottlenecks.

Key Use Cases

  • Identifying open connections, listening ports, or dropped packets.
  • Analyzing TCP/UDP statistics for congestion or errors.

Example Commands

# Show TCP connection states and statistics
netstat -snt  # Summary statistics for TCP

# List active network connections
ss -tuln  # Show listening UDP/TCP ports

Interpretation

  • Look for high Retransmit or Local Port exhaustion in TCP statistics.
  • Use netstat -s to check for packet loss (SegsRecv vs. SegsSent).
  • Combine with sar to correlate connection states with network throughput.

Combining Tools for Comprehensive Analysis

For optimal performance tuning: 1. Use sar to capture baseline network and system metrics. 2. Run iostat to isolate disk I/O as a potential bottleneck. 3. Query netstat or ss to diagnose connection or protocol-specific issues.

For example, if a server experiences high latency, you might: - Check sar -n DEV for network saturation. - Run iostat -x to rule out disk I/O contention. - Use netstat -snt to identify TCP retransmissions or connection drops.

Key takeaways

  • Use sar to monitor network throughput, CPU, and disk I/O over time.
  • Leverage iostat to diagnose disk I/O bottlenecks affecting network services.
  • Analyze netstat or ss output to troubleshoot connection states and protocol errors.
  • Combine these tools to isolate and resolve performance issues systematically.