Skip to content

06 β€” Operating System Security Basics

Level: Beginner Β· Time: ~18 min Β· Prerequisites: Lesson 5 β€” Identity, Authentication and Authorisation


Why this matters

Every credential an attacker steals is only worth what the operating system will let it do. The OS is where privilege is defined, where services wait for connections, where logs are written and where persistence is installed. When an intruder escalates from a normal user to root or SYSTEM, that is an operating system failure rather than a network one β€” an unpatched service, a setuid binary, an over-broad sudo rule, a group membership nobody reviewed. Hardening is mostly subtractive: fewer accounts, fewer services, fewer permissions, more logging. Most of this lesson is about what to take away.


The mental model: who the kernel trusts

Account type Linux Windows Rule
Superuser / administrator root (UID 0) Administrator, members of Administrators (S-1-5-32-544) breaks every restriction; for administration only, never daily work
Ordinary user your login, e.g. UID 1000 your domain account where your work happens, with least privilege
Service account www-data, postgres, nobody LOCAL SERVICE (S-1-5-19), NETWORK SERVICE (S-1-5-20), virtual service accounts runs one service, owns only that service's files, no interactive login
System / kernel kernel, running as UID 0 SYSTEM (S-1-5-18) not for humans at all

Linux identifies subjects by numeric UID and GID; the names in /etc/passwd and /etc/group are only labels. Windows identifies them by SID, a string like S-1-5-21-…-1004, where a name is just a lookup. The consequence is identical on both: renaming an account changes none of its rights, and a service account that inherited a human's group memberships is a privilege escalation waiting to happen.

Working as an administrator all day is the most common own-goal in this lesson. Every document you open, every script you run, every browser tab and mail attachment inherits administrator authority β€” so one phishing attachment becomes full system compromise. On Windows an administrator logging in interactively receives a filtered token and must consent to elevation; on Linux sudo plays the same role. Both exist to make privilege a deliberate act.

[!TIP] The habit that prevents the majority of endpoint incidents: browse, read mail and write documents as a normal user. Administration happens in a separate session, with a separate account, for a defined task.


Linux permissions in depth

Every file has an owner, a group, and three permission triplets: user, group, other.

Symbol On a file On a directory
r read the contents list the names inside
w modify the contents create, delete or rename entries
x execute the file traverse it and reach entries by name
Mode Meaning Where you see it
644 rw-r--r-- normal file: owner writes, everyone reads
600 rw------- private file β€” SSH private keys, secrets, key material
755 rwxr-xr-x programs and directories everyone may use
1777 rwxrwxrwt /tmp: everyone writes, only owners delete
chmod 640 secrets.conf          # numeric: all three triplets at once
chown app:app /srv/app/data     # owner and group
umask                           # 022 by default β†’ new files 644, new directories 755
umask 027                       # 027 β†’ new files 640, new directories 750

The special bits are where privilege escalation lives.

Bit Numeric Effect Security relevance
setuid 4000 the program runs with the owner's privileges, not the caller's a root-owned setuid binary runs as root β€” the classic local escalation target
setgid 2000 files run with the owner's group; on a directory, new files inherit the group useful for shared directories, dangerous when the group is privileged
sticky 1000 on a directory, only the owner may delete their own files the reason /tmp is shareable without chaos
find / -perm -4000 -type f 2>/dev/null     # setuid binaries β€” should be a short, known list

If that list contains anything you did not expect, it is your investigation for the day: attackers add setuid copies of shells, or exploit existing ones, precisely because the list is rarely reviewed.


Windows access control and UAC

Windows does not use rwx triplets. Every object carries a discretionary access control list (DACL): an ordered list of access control entries, each naming a SID and allowing or denying specific rights β€” Read, Write, Execute, Full Control, Take Ownership. Order matters, because deny entries are evaluated before allow entries, which is why a badly ordered ACL grants access nobody intended.

Concept What it does
Share permissions gate an SMB share; the narrower of share and NTFS permissions wins
NTFS permissions (DACL) per-file and per-folder rights, inherited down the tree
Groups the unit of administration β€” grant to a group, never to a person
Inheritance children inherit parent permissions unless explicitly broken

UAC (User Account Control) exists because of the split-token problem: an administrator logged in interactively gets a normal filtered token, and earns a full administrator token only after an explicit consent prompt. Run as administrator is that elevation, and the prompt is not an obstacle to work around β€” it is the control that stops silent escalation from a malicious attachment. At a high level, a process requests elevation, the shell shows a prompt on the secure desktop (so malware cannot draw a fake one), and on approval a new process is created holding the full token. The residual risks are the obvious two: users who approve anything that pops up, and admin accounts that never needed to exist.


Raising privilege safely

Tool Used for Pitfall
sudo <command> run one command as another user blanket rules and wildcards
su - <user> start a login shell as another user needs the target's password and leaves no per-command accountability
sudo -i / sudo -s an interactive root shell everything afterwards is unlogged shell activity; prefer one command at a time
sudo -l list what you are allowed to run the first thing to check on any host you administer
runas /user:DOMAIN\admin cmd.exe run a process as another user on Windows the child process inherits the elevated authority entirely

/etc/sudoers β€” edited only with visudo, plus drop-ins under /etc/sudoers.d/ β€” is one of the most commonly misconfigured files on a Linux system.

Pitfall Why it is dangerous Better approach
ALL=(ALL) ALL full root for a user or group grant named commands only
NOPASSWD: everywhere no deliberate act, and any process running as that user is root require the password, or a PAM-backed second factor
jane ALL=(root) /usr/bin/vi /var/log/* vi has a shell escape (:!sh) and the wildcard allows any path allowlist a fixed command with no arguments, or a wrapper that validates its own input
Granting editors, pagers, interpreters, find, tar each can execute or write arbitrary content never grant sudo on a tool that can run other tools

[!WARNING] A wildcard in a sudo rule is a shell escape in disguise. sudo vi, sudo less, sudo tar, sudo find and sudo python all let the user run a shell as root β€” so a rule naming an editor or interpreter is effectively ALL=(ALL) ALL.


Services, isolation and application hardening

  • Every listening service is a door, and doors you do not need should be bricked up. An unused service still gets patched, still parses untrusted input, and still counts as attack surface.
  • Run each service as its own account with minimum rights. www-data should not write to /etc, and a database should not run as root.
  • systemd unit files are where that is configured β€” User=, Group= and the sandboxing directives are cheap and effective:
[Service]
User=appsvc
Group=appsvc
NoNewPrivileges=yes
ProtectSystem=strict
PrivateTmp=yes
  • Mandatory access control adds what permissions cannot express. SELinux labels every process and file with a context and permits only the interactions a policy allows, even for root (getenforce, sestatus, ls -Z, setsebool). AppArmor does the same job per program with path-based profiles in /etc/apparmor.d/ (aa-status). Both turn "root can do anything" into "this program can do exactly what its policy says".
  • Defender and application control. Microsoft Defender's Attack Surface Reduction rules block risky behaviour patterns rather than matching file hashes β€” for example, stopping Office applications from spawning child processes, or blocking script execution from downloads. Application allowlisting (WDAC or AppLocker on Windows, fapolicyd or SELinux policy on Linux) inverts the default so only approved programs run.
  • Disable what executes implicitly. Office macros from internet-sourced files, autorun from removable media, and script hosts nobody uses are three delivery mechanisms you can switch off today.
ss -tulpn                                    # Linux: what is listening, and as which process
systemctl list-unit-files --state=enabled    # what is enabled to start at boot

On Windows, Get-NetTCPConnection -State Listen lists the same thing, Get-Service | Where-Object Status -eq 'Running' lists the services, and Get-LocalGroupMember -Group Administrators answers the question that matters most: who holds the keys.


Integrity, patching and audit

Control What it does
Secure Boot firmware verifies the bootloader signature, so a tampered boot chain does not load
TPM holds keys and measures boot state; a disk key is released only if the measurements match
LUKS Linux full-disk encryption; the passphrase never leaves the machine (cryptsetup luksFormat)
BitLocker Windows full-disk encryption, key sealed to the TPM (manage-bde -status)

Patching is a control with a schedule and an owner. Unpatched services are the most common initial-access vector: the flaw is known, the fix has been available for months, and the exploit is public. Prioritise internet-facing services, escalate by exploitability and exposure rather than CVSS alone (Lesson 27 covers prioritisation), then verify the reboot happened β€” a patched kernel is not running until the machine restarts.

Disk encryption is not about sophisticated attackers: it is about the laptop left on a train, the disk pulled from a decommissioned server, and the backup drive in a desk drawer. Without it, physical access is total data access. Visibility is the last layer, and it is worthless without retention:

auditctl -w /etc/passwd -p wa -k identity     # watch writes and attribute changes
auditctl -w /etc/sudoers -p wa -k privilege
Windows event ID Meaning Why you care
4624 successful logon baseline what normal logons look like, including type 3 (network) and type 10 (RDP)
4625 failed logon spraying and brute force; pattern and volume matter more than single events
4672 special privileges assigned to a new logon an administrator logged on β€” track it, especially outside change windows
4720 user account created either a change record or persistence
7045 a service was installed (System log) services are a favoured persistence mechanism

File integrity monitoring (AIDE or an OSSEC-style agent on Linux, Defender or a commercial FIM on Windows) hashes critical files and alerts when they change β€” the cheapest way to notice that a binary, a unit file or a web root has been modified. Keep log retention long enough to investigate: 90 days is a common floor, and an attacker who knows your logs roll off in seven days will simply wait. Centralise logs off the host, because clearing local logs is an early step for a competent intruder β€” see Lesson 22.

A one-paragraph aside on containers. Containers use namespaces and cgroups for isolation, plus capabilities and seccomp to limit what they may call. That is isolation, not a security boundary: a container running as root with the runtime socket mounted is effectively root on the host. Run them as non-root, drop capabilities, and never mount the Docker socket into a workload.

# Linux Command or setting
1 Patch, then reboot apt update && apt full-upgrade, then systemctl reboot
2 Know what is listening ss -tulpn β€” close or firewall anything unexplained
3 Keys only for SSH PasswordAuthentication no, PermitRootLogin no in /etc/ssh/sshd_config
4 Review setuid binaries find / -perm -4000 -type f 2>/dev/null
5 Protect keys and home directories chmod 700 ~/.ssh && chmod 600 ~/.ssh/*
6 Review sudo rules visudo -c, inspect /etc/sudoers.d, remove wildcards and NOPASSWD
7 Enforce MAC getenforce β†’ enforcing, or aa-status for AppArmor
8 Encrypt the disk LUKS at install time; verify with lsblk -f
# Windows Setting or command
1 Patch, then reboot Windows Update; check the last boot time
2 Separate admin from daily use standard user for work, elevate per task
3 Audit the Administrators group Get-LocalGroupMember -Group Administrators
4 Encrypt the disk BitLocker with TPM; escrow the recovery key and test it
5 Secure Boot and TPM enabled firmware settings; verify with Confirm-SecureBootUEFI
6 Defender and ASR rules on Get-MpPreference; ASR rules in block mode
7 Unique local admin passwords LAPS
8 RDP restricted, never internet-facing off by default, or behind VPN with MFA and a jump host

Attack it / Defend it

The attack How it works The control that stops it
Local privilege escalation via a setuid binary exploit a flawed setuid program, or leave a setuid shell behind patch promptly, review setuid binaries, mount /home and /tmp with nosuid, MAC policy
Over-broad sudo rule use a wildcarded or editor-based rule to open a root shell allowlist fixed commands, no wildcards, require a password, log sudo
UAC bypass get a privileged process to launch something the user never approved UAC at the highest setting, no admin account for daily work, application control
Credential dumping read secrets from LSASS memory or /etc/shadow as a privileged user Credential Guard and PPL on Windows, restrict access to /etc/shadow, no domain admin sessions on workstations
Service persistence install a service or unit that restarts after reboot monitor event 7045, FIM on unit files, application allowlisting
Cron or scheduled-task persistence add a periodic job that re-establishes access baseline cron and scheduled tasks, alert on new entries, restrict write access
Unpatched internet-facing service exploit a public vulnerability for initial access patch cadence with deadlines, minimise exposed services, network controls (Lesson 18)
Log destruction clear or alter local logs to hide the intrusion centralised log forwarding, restrictive log permissions, alert on log-clearing events
Unpatched kernel abuse a kernel bug to become root, bypassing every user-space control patching with enforced reboots, live patching where available, MAC as a second layer

Key takeaways

  • Privilege is defined by the OS, so the OS is where privilege escalation is won or lost β€” not on the network.
  • Subtract rather than add. Fewer accounts, services, permissions and interpreters make an incident harder to start and easier to see.
  • Never do daily work as an administrator. UAC and sudo exist because standing privilege turns one phishing attachment into a full compromise.
  • A wildcard in a sudo rule is a root shell, and so is any rule granting an editor, pager or interpreter.
  • Integrity and audit catch what prevention missed: Secure Boot, disk encryption, file integrity monitoring, auditd, and centralised logs with real retention.

Check yourself

  1. What does a root-owned setuid binary do, and why is the list of them worth reviewing?
  2. Give three sudoers patterns that effectively grant full root, and the correct alternative to each.
  3. A service writes to /srv/app/data and runs as www-data. Name two unit-file directives that limit what it can do.
  4. Which Windows event ID tells you a new service was installed, and why does that matter for persistence?
  5. You recover a stolen laptop. Which control determines whether the disk contents are safe, and which additional control protects against a tampered bootloader?

Next

Lesson 7 β€” How Attacks Actually Happen β€” Kill Chain and MITRE ATT&CK