44 — Detecting Intrusion with Open-Source Tools

Level: Advanced · Time: ~25 min · Prerequisites: Lesson 43 — Securing a Small Business

Why this matters

This is the lesson the rest of the course has been building towards: someone is in your network, or you suspect it, and you need to know how you would find out with tools you can install this afternoon. Start from the right assumption — assume breach, and stop asking whether you are compromised. Ask instead what normal looks like on your systems and where the evidence of a deviation would be written down, because detection is a comparison against a baseline rather than a scan for something evil. An attacker does not need a signature to be noticed: they need to fail to look like your ordinary Tuesday. Everything below is about making that comparison possible, then acting on it.

The mental model: detection is a comparison, not a scan

Part of the model	What it means
The baseline	a record of what normal activity looks like per source — which hosts exist, which services listen, which destinations are used, which accounts log in, at what hours, in what volumes
The evidence	the artefacts where a deviation is written down: identity events, endpoint events, network records, data movements, cloud audit trails
The pipeline	collection, shipping, storage, query, alert — the weakest stage decides the whole thing, and in small environments the weakest stage is always collection (Lesson 22 — Logging and Monitoring Foundations)
The fallback	where the endpoint cannot be trusted, the identity and network evidence still survives, because it is recorded somewhere the attacker does not control. Every recipe below reduces to six comparisons: new hosts, new services, new destinations, new accounts, changed volumes, changed hours

[!IMPORTANT] Record at least a week of normal activity per source before you need it, and a month if the environment is busy or seasonal. Retention is the other half of the same decision: an alert you cannot investigate because the raw events expired three days ago is worse than no alert, because it cost you the time to build it.

Where the evidence lives

Evidence class	The artefact that matters	Open-source tooling
Identity	authentication events, second-factor enrolment and disablement, privilege and group changes, service-account use, dormant accounts waking up	Wazuh with agents or API collection; the identity provider's own audit log; `auth.log` and `journalctl` on Linux hosts
Endpoints	process creation with command line and parent process, service and scheduled-task creation, autorun entries, cron and systemd timers, file integrity of system and web paths, and the state of the security tooling	Sysmon and `auditd` collected by Wazuh; osquery for point-in-time questions; Velociraptor for on-demand hunting; Hayabusa or Chainsaw over exported Windows event logs; Sigma rules for portable logic
Network and data	name-resolution queries, connection records, firewall allow and deny, IDS alerts, traffic volume by host and destination, and the bulk reads and staging locations that precede a transfer out	Zeek (`conn.log`, `dns.log`, `http.log`, `ssl.log`) with RITA for beaconing; Suricata (`eve.json`); Arkime for session-level search; `tcpdump`; the resolver's query log; endpoint file-access auditing
Cloud	control-plane audit trail, identity and policy changes, storage access, access-key creation	the provider's audit log (CloudTrail on AWS); Prowler or ScoutSuite for configuration; the cloud modules in Wazuh

The baseline does not need to be a project. For each source, keep a short record and refresh it monthly — and remember that these are the six comparisons every recipe below makes.

Comparison	What "new" looks like	Why it matters
New hosts	a hostname, address or device in the logs that is not in the inventory	an unmanaged machine is where an attacker works undisturbed
New services	a listening port on a host that never had one	a backdoor, or a service someone installed without telling anyone
New destinations	a domain or address no host in your estate has contacted before	download, command and control, exfiltration
New accounts	an account created, enabled, or added to a group	persistence and privilege, and the cheapest thing for an attacker to leave behind
Changed volumes	one host uploading or reading far more than it ever has	staging and exfiltration
Changed hours	activity at 03:00 from an account that works 09:00–17:00	automated or remote access using someone else's credentials

The detection recipe list

Fifteen recipes, each in four lines: what it detects, the source and the logic, the usual innocent explanation, and what to do. Everything here is open source, and everything is achievable with a single collection platform, a resolver log and a firewall log.

1. Authentication from a new country or an unusual source, including impossible travel Detects a successful sign-in from a country, address range or network the account has never used, or two sign-ins from places too far apart in time to be one person. Source and logic identity, VPN and mail sign-in logs, collected in Wazuh or read in the provider's console; first-seen country and address per account, alerting on the first occurrence rather than the fourth. Benign and action travel, mobile data, a newly installed VPN client — confirm with the user on a known channel; if unconfirmed, revoke the sessions and reset the credentials.

2. Repeated failures followed by a success from the same address Detects the moment a password guesser gets in, which is the point at which most other detection becomes retrospective. Source and logic authentication logs in Wazuh; count failures per source per account inside a window, and alert on a success from a source that just generated them (Windows 4625 followed by 4624; SSH failures followed by an accepted login). Benign and action a user who mistyped then succeeded, or a device retrying an old password after a change — verify with the user; if they did not do it, treat the session as hostile.

3. A new privileged account, or a change to a privileged group Detects persistence and privilege escalation, which is the step that turns a foothold into an incident. Source and logic directory and endpoint events (Windows 4720, 4728, 4732, 4672; Linux group file changes and sudo entries detected by file integrity monitoring); alert on any addition to an administrative group, and on new accounts outside a change window. Benign and action a legitimate new administrator, or an application installing a service account — check the change record first, and if there is no record, treat it as confirmed attacker activity.

4. A second factor being disabled or re-enrolled Detects an attacker who already has the password removing the control that would stop them, or re-enrolling the factor to their own device. Source and logic identity provider audit events; alert on any factor removal, re-enrolment or authentication-method change outside a documented request. Benign and action a new phone, which is why the request should exist in a ticket — if it does not, revoke the sessions, force re-enrolment with the user on the phone, and treat the password as burnt.

5. A new service, scheduled task or autorun entry Detects persistence, and it is one of the few detections that works even when the initial access is unknown. Source and logic endpoint events (Windows 4697 and 7045 for services, 4698 for a task, Sysmon 11 and 13 for file and registry writes, 1 for the process that created them; Linux systemctl list-timers --all, /etc/systemd/system, /etc/cron.d, /var/spool/cron); alert on creation, and on services whose binary lives in a temporary or user-writable path. Benign and action software installation, which is why change windows exist — verify the software against the installation record; if it is unexplained, treat the host as compromised and preserve the evidence before removing anything (Lesson 25 — Digital Forensics Basics).

6. A suspicious process chain Detects exploitation and malware execution: an office document or mail client spawning a command interpreter, or a built-in signed system utility abused to download or execute something. Source and logic Sysmon event 1 through Wazuh, with command lines enabled; alert on unusual parent–child pairs (a document reader, mail client or web browser spawning a shell), on interpreters with encoded or download-and-execute arguments, and on the signed utilities commonly used to fetch files, such as a certificate utility, a background transfer service or a registered scripting host. Benign and action an administrative script or a software deployment, and rarely true when a document spawned it — capture the command line and the file it fetched, then isolate the host.

7. A large outbound transfer, or an obvious staging pattern Detects data being collected for removal: a host reading far more than it normally does, or uploading a volume it has never uploaded before. Source and logic connection and byte counts from Zeek conn.log and the firewall, plus endpoint file-access auditing; alert on volume per host per hour against the baseline, and on a host talking to an unexpected destination at unexpected volume. Benign and action a backup, a cloud sync, a large download, or a legitimate migration — check what the host is and whether the destination is one it has used before; if the destination is new, treat it as exfiltration until proven otherwise.

8. Periodic, beacon-like outbound connections Detects command-and-control channels, which are usually small, regular and easy to miss in a total-volume view. Source and logic Zeek conn.log analysed with RITA, or grouped by destination with the interval between connections measured; alert on connections with regular spacing and low jitter to the same destination from the same host. Benign and action software update checks, time synchronisation, telemetry and monitoring agents — the pattern is the prompt to look, not the verdict, so identify the process behind the connection before acting.

9. Name-resolution anomalies Detects tunnelling and algorithmically generated domains: unusually long high-entropy subdomain labels, a high proportion of failed lookups, or one host suddenly querying a breadth of unique domains it has never touched. Source and logic the resolver query log (Pi-hole, AdGuard Home or the router) and Zeek dns.log; alert on label length and entropy, on unique-domain count per host per hour against baseline, and on large TXT or unusual record types. Benign and action content delivery networks, anti-tracking quirks and a new application are the usual explanations — identify the host and the process; if the host is an internet-of-things device or a server that should not be browsing, treat it as an incident.

10. File-sharing or remote-desktop traffic between two workstations Detects lateral movement, because workstations rarely serve each other files or accept each other's remote desktop connections in a healthy network. Source and logic Zeek connection records and firewall allow logs for the file-sharing and remote-desktop ports; alert on workstation-to-workstation connections on those ports, and on the network logon (Windows 4624 logon type 3) and remote interactive logon (type 10) that accompany them. Benign and action an administrator working from a desktop, or a peer-to-peer application — confirm which workstation and which user; the spread of a network tool across the estate looks exactly like this.

11. A new listening service on a host that did not have one Detects a backdoor, a proxy, or an internal service bound to the wrong interface. Source and logic ss -tulpn on Linux, Get-NetTCPConnection -State Listen on Windows, osquery listening_ports, or Zeek's service observation; compare against the recorded baseline per host rather than scanning by hand. Benign and action a new application, a development server, or a database someone exposed for convenience — ask the owner; an unexplained listener on a host whose owner has no answer is a containment decision.

12. An unexplained integrity change to a system binary or a web root Detects tampering, web-shells and rootkits: the modification of something that should never change between patches. Source and logic file integrity monitoring (Wazuh syscheck, AIDE, or osquery file-integrity queries) over the system paths and the web root; alert on any change outside a patching window. Benign and action an operating system update or a legitimate deployment, both of which are dated in a change record — a changed binary with no patch behind it is one of the highest-confidence findings in this list.

13. Anything interacting with a honeypot Detects an intruder with near-perfect confidence, because nothing legitimate should ever touch a decoy. Source and logic a decoy service (OpenCanary is the established open-source option) or a decoy document, link or credential placed where only an attacker would find it; alert on any interaction at all. Benign and action there is no innocent explanation for a decoy account being used — treat the source host as compromised and start from the network evidence, because the attacker was probably active for some time before touching the decoy.

14. A security tool disabled, or a log cleared Detects the attacker's first move after gaining administrative access, and it is also the alert that tells you why you have seen nothing else. Source and logic agent health and tamper events from the platform, Windows 1102 for a cleared security log and 104 in the System log, service stop events, and audit-policy changes (Windows 4719); alert on the event and on the absence of heartbeats from a host. Benign and action a technician's maintenance, which should be in a change record — if it is not, assume the host is compromised and treat the gap in its logs as a fact to preserve.

15. Cloud: a new access key, a broad policy change, or a storage bucket becoming publicly readable Detects the cloud equivalent of privilege escalation and exposure, often reached with nothing more than a stolen long-lived key. Source and logic the provider's audit trail; alert on access-key creation, new or modified identity policies that grant broad access, changes to trust relationships, audit-logging being disabled, and policy changes that make storage public. Benign and action a developer's automation or a legitimate infrastructure change — verify against the change record; a key created by a departed user's credentials is a confirmed incident. Note that detailed data-event logging is usually a paid feature in the major providers, so check current pricing rather than assuming it is included.

Triage, escalation and the one-afternoon starter path

When one of these fires

Confirm the alert is real by reading the raw event, not the summary. Parsing errors, duplicated collection and timezone confusion all produce convincing nonsense.
Identify the asset and the user. Which machine, which account, which address — before deciding anything.
Establish whether the action was expected. Ask the person on a known channel, and check the change record and the ticket system.
Look around the event: the same host and the same account for an hour either side, then the same indicator across the estate.
Decide which of three things this is — a confirmed incident, a suspicious lead worth more data, or benign.
Act in proportion, and record what you did. Benign: close it with a note. Suspicious: collect evidence first, then contain. Confirmed: contain, then preserve, and follow the playbook in Lesson 45 — Incident Response Playbooks for Small Teams. Stopping the loss beats preserving the timeline, but take a copy of what you can first, and write down the decision, the reasoning and what you ruled out.
Feed it back. If the same alert was benign for the fifth time, tune it or document the exception; if it fired on something genuinely bad, write down what you would need next time.

When to stop improvising and bring in help

The small set of findings that means you stop improvising and bring in help: confirmed attacker access to a system that matters, because an adversary with a foothold is a race you have not trained for and containment is easy to get wrong; any indication that data has been taken, because notification duties, evidence preservation and legal consequences start here; ransomware anywhere in the estate, where the first encrypted file is a countdown and most of the decisions are business decisions rather than technical ones; anything touching regulated personal data, where the reporting deadline is short — take that to your adviser and, where required, your supervisory authority, and note that this is not legal advice; and a system you do not understand and cannot safely inspect, where improvised forensics destroys evidence and can trigger the attacker early.

One afternoon: from nothing to five working alerts

Deploy the platform on one host — the all-in-one installation or the single-node container deployment — and open the dashboard. One host, not a cluster: the objective today is one alert you trust.
Add agents to the two most important machines. On Windows, enable Sysmon with a published configuration rather than the default, turn on Include command line in process creation events, and enable PowerShell script block logging. On Linux, enable an execution audit rule.

# Linux: record every process execution with its arguments
echo '-a always,exit -F arch=b64 -S execve -k exec' | sudo tee /etc/audit/rules.d/exec.rules
sudo augenrules --load && sudo systemctl restart auditd
sudo ausearch -k exec -ts recent | head -5     # confirm it is recording

Turn on query logging on the resolver, and forward it with the firewall's logs into the same store by syslog. Pi-hole and AdGuard Home keep a query log by default; Unbound needs log-queries: yes, and a dnsmasq-based router needs log-queries. Search name resolution, deny decisions and host events together in one place — this single step is what makes most of the recipes above possible.
Create the first five alerts: repeated authentication failures followed by a success from one address; a new privileged group member or local administrator; a new service, scheduled task or autorun entry; the security agent stopping or a log being cleared; and a first-seen outbound destination from a host, with unusual volume.
Prove the pipeline works by failing something deliberately and finding it.

ssh wronguser@localhost                                       # a deliberate failure
sudo grep -i "failed password" /var/log/auth.log | tail -3     # find it locally
# then search the same host and the same minute in the platform interface.
# Present locally and absent centrally means your shipping is broken (Lesson 22).

The honest limits of open-source detection

Limit	What it means in practice
No baseline, and short retention	without a week or more of recorded normal you cannot tell "unusual" from "Tuesday", and a patient attacker is invisible if your logs last seven days — you discover the intrusion only when it is used, months after it began. This is the most common reason a small detection programme fails
Encryption	most traffic is encrypted, so network analysis gives you destinations, volumes, timing and certificate metadata rather than content
The endpoint agent can be defeated	an attacker with administrative control can hide from it, which is exactly why identity and network evidence matter — and a silenced agent is itself a detection
You may hear about it from outside first	a customer, a bank, law enforcement or a ransom note. That is the argument for this whole lesson: it turns a phone call from your bank into an incident you were already tracking

Symptom to evidence

What you notice	What it usually means	Where to look first	The tool
A slow computer	a resource-hungry mining or scanning process, a browser hijack, or failing hardware	the process list, and process creation events with command lines	Sysmon and Wazuh, osquery `processes`, Velociraptor
A user locked out of an account	their own device retrying an old password, or someone guessing at their account	lockout and failed-logon events for that account, with source addresses	Windows 4740 and 4625 in Wazuh; `auth.log` on Linux
A spike in outbound traffic	a backup or cloud sync that finally ran, a large download — or staging and exfiltration	connection records and byte counts by host and destination	Zeek `conn.log` in Arkime, the firewall session table, RITA for periodic patterns
Unexpected mailbox rules or forwarding	a classic post-compromise step: the rule hides the replies that would expose the fraud	the mailbox's rules, forwarding and delegate settings, and the sign-in log	the mail provider's audit log and admin console
An invoice nobody recognises	invoice fraud, usually through a compromised supplier mailbox, sometimes through yours	the sending account's sign-in history and mailbox rules, then your finance procedure	the mail provider's audit log; then a phone call to a number you already had
A security tool disabled	either an administrative change or an attacker's first move — assume the second until a named person confirms the first	agent status and tamper events, and the host's event log	the platform's agent health view, the endpoint protection console, Windows 1102
An unknown device on the network	a guest, a device you forgot, a neighbour on a weak passphrase, or an intruder	the DHCP lease list and the wireless client list	the router's admin page, the DHCP server's leases, then the resolver log for what it queries

[!TIP] The test you should be able to answer after reading this lesson is one sentence long: given this alert, what would I check first, and why. If you can answer that for the five alerts you actually built, you are doing detection. If you can only answer it for the hundred alerts a vendor shipped you, you are doing triage on someone else's guesses.

Attack it / Defend it

The attack	How it works	The control that stops it
Creating a privileged account	persistence and escalation in one step	alert on every administrative group change and on account creation outside a change window
Disabling the second factor	an attacker with the password removes the obstacle	alert on factor removal or re-enrolment with no ticket behind it
Service or scheduled-task persistence	an attacker installs something that survives a reboot	service and task creation events, with attention to binaries in temporary paths
Malicious document macro	an office file spawns an interpreter and downloads a payload	unusual parent–child process pairs, encoded command lines, and the signed utilities used to fetch files
Command-and-control beaconing	small, regular outbound connections that never look like a volume spike	connection records analysed for periodic patterns (Zeek with RITA), and first-seen destinations
Name-resolution tunnelling	data carried inside long, high-entropy queries	resolver query logging, with label length and entropy alerting, and unique-domain counts per host
Lateral movement between workstations	file sharing or remote desktop from one workstation to another	workstation-to-workstation port alerting, and network and remote-interactive logon events
Web shell on a web root	a small file added to a server that already accepts traffic	file integrity monitoring over the web root, alerted outside patching windows
Log clearing after gaining administrative access	history erased to remove the evidence of everything above	shipping logs off the host, plus the cleared-log event and agent-silence alerting
Cloud key creation and policy change	a stolen long-lived key creates its own access, or opens storage to the world	provider audit-trail alerting, short-lived credentials, and configuration auditing (Prowler, ScoutSuite)

Key takeaways

Assume breach and compare against a baseline. Detection is not a scan for evil; it is noticing that something does not match what you recorded as normal.
Five evidence classes, six comparisons. Identity, endpoint, network, data and cloud artefacts, compared for new hosts, services, destinations, accounts, volumes and hours, cover almost every recipe here.
Build five alerts you trust, not a hundred you ignore. Platform, two agents, resolver logging, forwarded firewall logs, and a deliberate-failure test are an afternoon's work.
A quiet attacker is found in the correlation, not in a single event. The alert is the prompt; the surrounding hour on the same host and account is the answer.
Know the limits honestly. Without a baseline, with short retention, or with the endpoint under the attacker's control, open-source detection will miss the patient intruder — which is why identity and network evidence carry the weight.

Check yourself

An alert fires: a successful logon from a country the account has never used. State your first four checks in order, and what makes you escalate.
Why is a new scheduled task a stronger finding than a new outbound connection?
Your resolver logs last three days and your endpoint events last ninety. What will you miss, and what would you change first?
A workstation connects to another workstation on the file-sharing port at 02:00. What is the likely explanation, what evidence do you correlate, and what is the usual benign story?
Within an afternoon, which four sources would you collect and which five alerts would you write — and how would you prove any of it works?