31 — Traffic Analysis and IDS Tools
Level: Advanced · Time: ~22 min · Prerequisites: Lesson 30 — Vulnerability Scanning and Exploitation Tools
Why this matters
Logs tell you what a machine believed happened. Traffic tells you what actually crossed the wire — including the devices that keep no logs at all, the guests and printers nobody manages, and the two workstations that have no business talking to each other. It is the only evidence source that is independent of the endpoint, which matters enormously when the endpoint is compromised and its logs are lying to you. Universal TLS changed the deal: you now see metadata rather than content, and nothing else in this lesson works unless you accept that. Metadata is still the most productive detection surface a small team owns.
The mental model: three levels of network visibility
You do not choose between "capture everything" and "capture nothing". You choose a level, and each level answers different questions at a different cost.
| Level | What you get | Typical volume and retention | Answers |
|---|---|---|---|
| Full packet capture | every byte, replayable | very high; days | "show me exactly what was sent" |
| Rich metadata records | connection, DNS, HTTP and TLS facts as structured rows | medium; weeks | "which hosts did this, and when" |
| Flow records | five-tuple, bytes, duration, timing | low; months | "is this normal for this host" |
The practical consequence: capability declines as retention grows. You keep weeks of queryable facts and only days of packets, so make sure the facts you keep are the ones you will want to query. That is the argument for the logging engines described later in this lesson.
What TLS took, and what it left: gone are URLs, request bodies, file contents, credentials in transit, and anything at all inside an encrypted tunnel. What remains is the envelope — destination address and port, packet sizes and timing, the server name requested (SNI) in most connections, the certificate subject, issuer and validity dates, the client's TLS fingerprint where you compute it, and the hosting provider behind the address.
Capturing packets from the command line
tcpdump is the command-line capture tool on every Linux and macOS system, and the first thing you install on a sensor. The flags matter more than the filters.
# 1. What is even on this wire? 200 packets, no name or port resolution
sudo tcpdump -i eth0 -nn -c 200
# 2. Capture full packets to a file for later analysis (-s 0 = whole packet)
sudo tcpdump -i eth0 -nn -s 0 -w /var/tmp/cap-$(date +%F-%H%M).pcap
# 3. Read the file back — analysis never touches the live wire
tcpdump -r /var/tmp/cap-2026-01-14-0930.pcap -nn | head
# 4. Only the conversation you care about
sudo tcpdump -i eth0 -nn -s 0 'host 10.20.30.40 and port 445'
# 5. Connection attempts only: SYN set, ACK clear — a scan or brute force in progress
sudo tcpdump -i eth0 -nn 'tcp[tcpflags] & tcp-syn != 0 and tcp[tcpflags] & tcp-ack = 0'
| Flag | What it does, and why you care |
|---|---|
-i eth0 |
the interface. Use -D to list them, any to capture across all of them |
-nn |
no DNS or service-name lookups. Faster, quieter, and stops your sensor generating traffic |
-s 0 |
snaplen unlimited — full packets, not just headers |
-w / -r |
write raw packets to a file / read them back from one |
-c 200 |
stop after N packets, so a command does not run forever |
-A / -X |
print payload as ASCII / as hex plus ASCII — how you see cleartext credentials |
-G 300 -W 24 |
rotate to a new file every 300 seconds, keeping 24 files |
The capture filter language is BPF, and a small vocabulary covers most work: host, src host, dst host, net 10.20.0.0/16, port, portrange 1-1024, not, and, or, tcp, udp, icmp, ether host, vlan 20, and greater 128 for packet length. Build it as a sentence: "from this host, but not my own management noise" is host 10.20.30.40 and not port 22 and not port 53.
[!TIP] Capture with
-wand analyse the file. Live analysis on a busy interface drops packets, and a capture you cannot re-read cannot be handed to anyone else or attached to a case.
Reading a capture: filters, streams and the investigation checklist
The graphical analyser (Wireshark) has two filter languages, and confusing them wastes hours.
| Capture filter | Display filter | |
|---|---|---|
| Language | BPF, the same as tcpdump |
Wireshark's own field syntax |
| Applied | before anything is written | after capture, over whatever you have |
| Use it for | reducing what you record | asking whatever question you like, repeatedly |
Display filters you will actually type: ip.addr == 10.20.30.40, tcp.port == 445, dns, dns.qry.name contains "xyz", tls.handshake.extensions_server_name == "example.com" (older versions use the ssl. prefix), http.request.method == "POST", tcp.flags.syn == 1 and tcp.flags.ack == 0, tcp.stream eq 4, frame.time >= "2026-01-14 22:00:00". Follow > TCP Stream reassembles one conversation into a readable transcript and is usually the fastest way to understand a session. Statistics > Conversations, Endpoints and Protocol Hierarchy give you the shape of the capture before you read a single packet.
The checklist for an investigation, in order:
| Question | Where the answer is |
|---|---|
| Which hosts talked at all? | Statistics > Endpoints, sorted by packets |
| Which pairs talked, and on which ports? | Statistics > Conversations |
| How much data moved, in which direction? | Conversations sorted by bytes; compare upload to download |
| Who initiated? | the SYN, or filter tcp.flags.syn == 1 and tcp.flags.ack == 0 |
| What looks malformed or retransmitted? | Analyse > Expert Information |
| What was actually said? | right-click a packet > Follow > TCP Stream |
tshark, the command-line equivalent, is what you use for counting rather than reading:
# Top 20 DNS names by frequency
tshark -r cap.pcap -Y dns -T fields -e dns.qry.name | sort | uniq -c | sort -rn | head -20
# Every server name requested over TLS
tshark -r cap.pcap -Y 'tls.handshake.type == 1' -T fields -e tls.handshake.extensions_server_name | sort -u
# Connections per minute for one host — the beaconing view
tshark -r cap.pcap -q -z io,stat,60,"tcp.port==443 and ip.addr==10.20.30.41"
What to look for: five anomaly families
This table is the working core of the lesson. Every row is visible in metadata alone, with the content encrypted.
| Family | What it looks like | Why it matters |
|---|---|---|
| DNS anomalies | long random-looking labels; high-entropy subdomains under one parent; heavy TXT querying; a single internal host resolving hundreds of unique domains; DNS to a resolver that is not yours; DNS on a non-standard port | tunnelling and exfiltration ride in queries, and algorithmically generated domains show up as volume against one parent |
| Beaconing | connections at regular intervals with small jitter; small, consistent payloads; a long-lived session to one destination; intervals at suspiciously exact periods | that is command-and-control keeping in touch, and it is hard for an attacker to make look like browsing |
| Data volume | a workstation uploading gigabytes; transfers at 03:00; outbound to a hosting or VPS provider rather than a content network; upload far exceeding download | the exfiltration stage, usually after the attacker has already been present for days |
| Protocol anomalies | SMB or RDP between two workstations; cleartext protocols still in use; a protocol on a port that does not match it (SSH on 443); obsolete protocol versions offered | lateral movement, and traffic built to survive a naive port-based rule |
| Cleartext evidence | USER/PASS in FTP, an Authorization: Basic header in HTTP, Telnet sessions, MySQL or PostgreSQL unencrypted, SNMP community strings, LDAP simple binds |
credentials in the clear are a finding on their own, whether or not an attack is under way |
Judgement decides which of these matter. A workstation doing a 40 GB offsite backup at 02:00 is a scheduled job; the same pattern from an unmanaged laptop is an incident.
Rule-based network intrusion detection
A network IDS reads the same packets you just captured and compares them, in order, against a rule set. It keeps connection state, so rules can say "established, to the server" rather than matching single packets. Two deployment modes, and the difference is the whole point:
| Mode | How it sits | What it can do | Risk |
|---|---|---|---|
| Passive (IDS) | receives a copy from a span port or tap | alert only | none to availability |
| Inline (IPS) | traffic passes through it | alert and drop | a bad rule or a full queue now costs you the network |
Suricata is the engine a small team should learn. Deployment is a config file plus an interface:
sudo suricata-update list-sources # which rule feeds are available
sudo suricata-update enable-source et/open # the free community rule set
sudo suricata-update # download and merge rules
sudo suricata -T -c /etc/suricata/suricata.yaml # test config and rules, then exit
sudo systemctl restart suricata
sudo tail -f /var/log/suricata/eve.json | jq 'select(.event_type=="alert")'
The rule format, in outline — action, protocol, source, port, direction, destination, port, then options:
alert http $HOME_NET any -> $EXTERNAL_NET any (
msg:"LOCAL PowerShell user agent to the internet";
flow:established,to_server;
content:"WindowsPowerShell"; http_header;
sid:1000001; rev:1;
)
content matches bytes, pcre matches a regular expression, flow constrains direction and state, threshold and detection_filter limit how often a rule fires, and sid plus rev give it identity and version. Community rule sets exist and are worth running: the Emerging Threats Open set is what suricata-update pulls by default, and Snort has an equivalent community set from Cisco. The engine itself ships with almost nothing.
[!WARNING] A default rule set is tuned for the whole internet, not for you. Expect a large volume of noise on day one — rules for software you do not run, alerts on your own vulnerability scanner, matches on things you have decided to accept. Clear that backlog before anyone else looks at the console, because a console nobody trusts is a console nobody reads.
Tuning responsibly means three things: disable what cannot apply to your estate, alert for weeks before you block anything, and record why each rule changed so the next person can reverse it. Suricata supports this with disable.conf, enable.conf and modify.conf alongside the rules, and every change should be dated and attributed.
Structured network logging beats raw packets
The insight that changed network detection: most of what you need is a record of a connection, not the packets inside it. Logging engines decode traffic and emit one structured row per event — small enough to keep for weeks, and queryable in seconds.
Zeek is the reference implementation. It writes tab-separated logs named by protocol, each with a #fields header, and runs live or over a capture file:
sudo zeek -i eth0 # live
zeek -r /var/tmp/cap.pcap # replay a capture into logs
cat conn.log | zeek-cut id.orig_h id.resp_p duration orig_bytes resp_bytes
| Log | The question it answers |
|---|---|
conn.log |
every connection: who, to whom, how long, how many bytes each way |
dns.log |
every query and answer, including TXT records and response codes |
http.log |
requests, user agents and what was fetched, where traffic is not encrypted |
ssl.log / x509.log |
TLS sessions with server name, certificate issuer and validation result |
notice.log / weird.log |
its own detections, and protocol that does not parse as it should |
A TLS client fingerprint (the JA3 approach, and its successors) condenses the details of a client's handshake into a short string. The same fingerprint recurs for the same malware family or the same library even when the destination and certificate change, so it survives infrastructure rotation. Fingerprinting arrives through extra packages and richer products rather than every engine out of the box; check what yours supports before relying on it.
Full packet capture platforms (Arkime is the common open-source one, paired with a search cluster) keep the packets themselves for the case where you must prove exactly what was sent. Size them honestly: packet storage is measured in days, not months. Lighter tools that simply rank who is talking to whom have their place too, and are worth running when you have no other visibility.
Where sensors go. Follow the zones in Lesson 18 — Network Security Controls: inside the internet edge so you see what survived the firewall, and between zones so you see internal movement. One fact catches everyone out: on a switched network you do not see other hosts' traffic unless you configure it. You need a span/mirror port, a physical tap, or the sensor placed inline. A sensor plugged into an ordinary switch port sees broadcast, multicast and its own traffic, and nothing else.
From observation to detection
Finish by writing the sentence. Suppose you see a workstation open a 4 KB TLS session to one address every 60 seconds with a few seconds of jitter, for six hours, uploads only, always the same client fingerprint. That is a beacon. Say it precisely enough to query: host, destination, connections per hour, median duration, bytes out, consistency of interval — every one of those is a column in the logs above, so the observation is already a rule: a condition on records, evaluated on a schedule, with a threshold on the count. Thresholds, exclusions and what happens when it fires are the bridge into Lesson 23.
Attack it / Defend it
| The attack | How it works | The control that stops it |
|---|---|---|
| Passive sniffing | capture credentials from cleartext protocols | encrypt everything; retire FTP, Telnet and cleartext HTTP; switching and segmentation |
| ARP or rogue DHCP man-in-the-middle | insert yourself on the segment and relay traffic | Dynamic ARP Inspection, DHCP snooping, 802.1X, encrypted protocols |
| DNS tunnelling and exfiltration | encode data in query names, receive replies in TXT records | route all DNS to your own resolver, watch query volume and label entropy, filter egress |
| Beaconing to command-and-control | small, regular outbound sessions to one host | connection-count baselining on conn.log-style records, TLS fingerprinting, egress allowlists |
| Exfiltration over an allowed port | send data out on 443 to a hosting provider | per-host volume baselining, destination category blocking |
| IDS evasion | fragment, encode or obfuscate to dodge signatures | engine reassembly, and narrow expectations using flow and content alongside pcre |
| Encrypted traffic blind spot | do everything inside TLS | accept it and detect on metadata, volume, timing and certificates instead of content |
| Unauthorised device on the wire | plug in and join a flat network | 802.1X, port security, network access control, and knowing what normal looks like |
Key takeaways
- Traffic is the independent witness. When the endpoint lies, the wire does not — and hosts you cannot put an agent on still generate metadata.
- TLS moved you from content to behaviour. Volume, timing, destination, certificate and fingerprint now carry the detection, and they are enough for most cases a small team sees.
- Default IDS rules are noisy by design. Silence what cannot apply, alert before you block, and record why you changed each rule.
- Structured connection records beat packets at scale. Keep weeks of queryable facts, not days of bytes.
- A switched network hides everything until you configure a span port, a tap, or an inline sensor.
Check yourself
- You need six weeks of "which host connected to which address, when, and how much data moved". What do you deploy, and why is full packet capture the wrong answer?
- A new rule set gives you 4,000 alerts on day one. Name three steps you take before telling anyone the IDS is working.
- What does the SNI field tell you about an HTTPS connection, and what does it not tell you?
- One internal host makes 200 queries a minute to subdomains of a single domain, each label 30 characters of random-looking text. What are the two plausible explanations, and how do you tell them apart?
- Write the sentence that describes a beacon you found, in the form of a query. Which log will you run it against, and where does that rule live next?