Security

Linux Audit Daemon (auditd) Configuration

Set up kernel-level syscall auditing for compliance, forensics, and incident investigation

The Linux Audit Daemon (auditd) provides kernel-level system call auditing for compliance, forensics, and security investigations. It captures detailed records of user and system activities for later analysis.

What is auditd

auditd is the userspace daemon that interacts with the Linux audit subsystem to:

  • Log all system calls at the kernel level
  • Track file modifications and access
  • Monitor privilege escalation
  • Record user authentication events
  • Support compliance frameworks (PCI-DSS, SOC 2, HIPAA)
  • Provide forensic evidence for incidents

Installation

Ubuntu/Debian

# Install auditd and plugins
sudo apt update
sudo apt install -y auditd audispd-plugins

# Start and enable service
sudo systemctl start auditd
sudo systemctl enable auditd

# Verify status
sudo systemctl status auditd
sudo auditctl -l  # List rules

Audit Rule Management

Two Methods

  1. auditctl (temporary): Rules lost on reboot
  2. /etc/audit/rules.d/ (persistent): Rules survive reboot

Configure Persistent Rules

Edit /etc/audit/rules.d/audit.rules:

sudo nano /etc/audit/rules.d/audit.rules

# Then reload:
sudo systemctl restart auditd

Core Rule Types

Watch Rules (-w flag)

Monitor file/directory modifications:

# Watch file for any changes
-w /etc/passwd -p wa -k passwd_changes

# Parameters:
# -w: watch path
# -p: permissions (r=read, w=write, x=execute, a=attribute change)
# -k: key for searching logs

Syscall Rules (-a flag)

Audit specific system calls with filters:

# Log all execve (execute) syscalls
-a always,exit -F arch=b64 -S execve -k execute

# Parameters:
# -a: action (always,exit or always,task)
# -F: filter (arch, euid, uid, success, etc.)
# -S: syscall name or number
# -k: key for searching

Key Names (-k flag)

Use meaningful key names for searching:

-k passwd_changes    # Key: passwd_changes
-k sudo_commands     # Key: sudo_commands
-k failed_logins     # Key: failed_logins

Common Audit Rules

Monitor System Configuration Files

# Watch critical system files
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /etc/sudoers -p wa -k sudoers_changes
-w /etc/sudoers.d/ -p wa -k sudoers_changes
-w /etc/hosts -p wa -k hosts_changes
-w /etc/hostname -p wa -k hostname_changes
-w /etc/ssh/sshd_config -p wa -k sshd_config_changes

Monitor SSH Keys

# Track authorized_keys modifications
-w /home -p wa -k ssh_keys_changed
-w /root/.ssh/authorized_keys -p wa -k root_ssh_keys
-w /root/.ssh/authorized_keys.d/ -p wa -k root_ssh_keys

Log All Root Command Execution

# 64-bit systems (b64), execute new processes with euid 0 (root)
-a always,exit -F arch=b64 -S execve -F euid=0 -k root_commands

# 32-bit systems (if needed)
-a always,exit -F arch=b32 -S execve -F euid=0 -k root_commands

# Log ALL sudo usage
-a always,exit -F arch=b64 -S execve -F uid!=0 -F auid!=-1 -F auid!=4294967295 -k sudo_commands

Monitor Failed Login Attempts

# Login and sudo failure tracking
-a always,exit -F arch=b64 -S adjtimex,settimeofday -k time-change
-a always,exit -F arch=b64 -S clock_settime -F a0=0x0 -k time-change
-a always,exit -F arch=b64 -S sethostname,setdomainname -k network-modifications

Monitor iptables/firewall Changes

# Track firewall rule changes
-a always,exit -F arch=b64 -S iptables -k firewall_changes
-a always,exit -F arch=b64 -S ip6tables -k firewall_changes
-a always,exit -F arch=b64 -S nftables -k firewall_changes
-w /etc/iptables/ -p wa -k firewall_config

Monitor User/Group Changes

# Track system user and group modifications
-w /etc/group -p wa -k group_changes
-w /etc/gshadow -p wa -k group_changes
-w /etc/passwd -p wa -k user_changes
-w /etc/shadow -p wa -k user_changes

Example Complete Rule Set

Add to /etc/audit/rules.d/audit.rules:

# Remove any existing rules
-D

# Buffer Size
-b 8192

# Failure handling
-f 2

# System configuration monitoring
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /etc/sudoers -p wa -k sudoers_changes
-w /etc/sudoers.d/ -p wa -k sudoers_changes
-w /etc/ssh/sshd_config -p wa -k sshd_config_changes

# Root command execution
-a always,exit -F arch=b64 -S execve -F euid=0 -k root_commands

# Sudo usage
-a always,exit -F arch=b64 -S execve -F uid!=0 -F auid!=-1 -k sudo_commands

# SSH keys
-w /root/.ssh/authorized_keys -p wa -k root_ssh_keys

# Firewall changes
-a always,exit -F arch=b64 -S iptables -k firewall_changes
-w /etc/iptables/ -p wa -k firewall_config

# Make configuration immutable (load last)
-e 2

Apply Rules

# Restart auditd to load rules
sudo systemctl restart auditd

# Verify rules loaded
sudo auditctl -l

# Check for errors
sudo auditctl -a test

Searching Audit Logs

ausearch Command

Search by key name:

# Find all passwd changes
sudo ausearch -k passwd_changes

# Find all sudo commands
sudo ausearch -k sudo_commands

# Find all root executions
sudo ausearch -k root_commands

# Search by time (today only)
sudo ausearch -k passwd_changes -ts today

# Search by time range
sudo ausearch -k passwd_changes -ts 2024-03-20 -te 2024-03-21

Filter by User ID

# All root (uid 0) actions
sudo ausearch -ui 0

# All actions by specific user
sudo ausearch -u username

# All auid (audit uid) 1000 actions
sudo ausearch -au 1000

Advanced ausearch

# Find failed system calls
sudo ausearch -m SYSCALL -sv no

# Find file modifications
sudo ausearch -m EXECVE -sv yes

# Get raw output
sudo ausearch -k sudo_commands -i

Generate Reports

aureport

Summarize audit data:

# Authentication report (logins, sudo)
sudo aureport --auth

# Failed authentication attempts
sudo aureport --failed

# Summary of all events
sudo aureport --summary

# User summary
sudo aureport --user

# File modification summary
sudo aureport --file

# Time-based report
sudo aureport --daily

Custom aureport

# Events this hour
sudo aureport --start thismonth --end now

# User activity over 7 days
sudo aureport --user --start today-7d

# Full event listing
sudo aureport -x

Forwarding to Central Log Server

Ensure audit logs are forwarded to a central server to prevent tampering. Local-only logs can be deleted by attackers with root access.

Configure auditd with rsyslog

Forward audit logs to a remote syslog server:

# Install rsyslog (usually pre-installed)
sudo apt install -y rsyslog

# Enable dispatcher plugin
sudo nano /etc/audit/plugins.d/syslog.conf

# Change to:
active = yes
direction = out
path = builtin_syslog
type = builtin
arg_format = string
arg = LOG_INFO
format = string

Configure rsyslog for remote forwarding

# Edit rsyslog config
sudo nano /etc/rsyslog.d/30-forward.conf

# Add:
# Forward audit logs to remote server
:programname, isequal, "audispd" @logserver.example.com:514

# Or use TCP (more reliable):
:programname, isequal, "audispd" @@logserver.example.com:514

Restart Services

sudo systemctl restart auditd
sudo systemctl restart rsyslog

# Verify logs arriving on remote server
tail -f /var/log/audit/audit.log

Query Logs with jq and awk

Convert audit.log to JSON

# Use ausearch with -i flag for interpretation
sudo ausearch -k sudo_commands -i | head -30

# Export specific fields
sudo ausearch -k sudo_commands -i | grep "exe=" | awk -F'exe=' '{print $2}'

Parse with grep and awk

# Get all files modified by root in last hour
sudo tail -10000 /var/log/audit/audit.log | \
  grep 'type=EXECVE' | \
  grep -E 'uid=0|euid=0' | \
  tail -20

# Count events by type
sudo grep "type=" /var/log/audit/audit.log | \
  awk -F'type=' '{print $2}' | \
  awk '{print $1}' | \
  sort | uniq -c | sort -rn

Log Rotation and Storage

auditd manages its own log rotation separately from rsyslog. Configure retention in /etc/audit/auditd.conf.

Configure auditd Log Rotation

Edit /etc/audit/auditd.conf:

sudo nano /etc/audit/auditd.conf

# Key settings:
log_file = /var/log/audit/audit.log
log_format = RAW
priority_boost = 4
flush = INCREMENTAL
freq = 20
num_logs = 5
max_log_file = 50
max_log_file_action = ROTATE

Backup Audit Logs

# Archive old logs
sudo tar czf audit-backup-$(date +%Y%m%d).tar.gz /var/log/audit/audit.log.*

# Check disk space used
sudo du -sh /var/log/audit/

# Compress old logs to save space
sudo find /var/log/audit/ -name "audit.log.*" -mtime +30 -exec gzip {} \;

Troubleshooting

Check auditd Status

# Service status
sudo systemctl status auditd

# Check rules loaded
sudo auditctl -l

# View configuration
sudo cat /etc/audit/auditd.conf

# Check for errors
sudo auditctl -a test

No Events Recorded

# Verify rules exist
sudo auditctl -l | wc -l

# Check if auditd is running
sudo systemctl status auditd

# Check audit log file permissions
ls -la /var/log/audit/audit.log

# Restart auditd
sudo systemctl restart auditd

High Audit Volume

If auditd is overwhelming logs:

# Adjust buffer size in /etc/audit/rules.d/audit.rules
-b 4096

# Remove less critical rules
# Reduce frequency with -a never flag for less important rules

# Reload:
sudo systemctl restart auditd

# Monitor disk usage
watch -n 5 'du -sh /var/log/audit/'

Performance Impact

For high-throughput systems:

# Use immutable flag at end of rules (prevents removal without reboot)
# -e 2 in rules file

# Reduce syscall auditing to critical operations only
# Focus on:
# - execve (new processes)
# - chmod/chown (permission changes)
# - unlink (file deletion)

# Avoid auditing read-heavy operations (too noisy)

Integration with SIEM

Forwarding to ELK Stack

Configure Filebeat to ship audit logs to Elasticsearch:

# /etc/filebeat/modules.d/auditd.yml
- module: auditd
  log:
    enabled: true
    var.paths: ["/var/log/audit/audit.log"]

# Enable the module:
sudo filebeat modules enable auditd
sudo systemctl restart filebeat

Create Audit Alerts

Set up alerting for critical events:

# In your SIEM/ELK, alert on:
# - type: SYSCALL AND name: "sethostname" (hostname changes)
# - auid: 0 AND type: EXECVE (root command execution)
# - type: PERMISSION AND denied (permission denied attempts)
# - key: "sudoers_changes" (sudo config modifications)

Best Practices

  • Enable early: Configure at system deployment, not after incident
  • Forward logs: Use rsyslog or Filebeat to centralize to avoid tampering
  • Archive regularly: Rotate logs and archive to cold storage
  • Alert on critical: Set up alerts for sudoers, passwd, SSH key changes
  • Review regularly: Analyze aureport summaries weekly
  • Test rules: Use -a always,task first to test without impacting kernel
  • Document keys: Use clear key names for easier searching
  • Baseline first: Capture normal behavior before looking for anomalies

AppArmor Mandatory Access Control

Implement kernel-level access control policies with AppArmor to restrict application privileges

Trivy - Container Vulnerability Scanner

Scan Docker images, filesystems, and IaC configs for vulnerabilities and misconfigurations with Trivy.

On this page

What is auditdInstallationUbuntu/DebianAudit Rule ManagementTwo MethodsConfigure Persistent RulesCore Rule TypesWatch Rules (-w flag)Syscall Rules (-a flag)Key Names (-k flag)Common Audit RulesMonitor System Configuration FilesMonitor SSH KeysLog All Root Command ExecutionMonitor Failed Login AttemptsMonitor iptables/firewall ChangesMonitor User/Group ChangesExample Complete Rule SetApply RulesSearching Audit Logsausearch CommandFilter by User IDAdvanced ausearchGenerate ReportsaureportCustom aureportForwarding to Central Log ServerConfigure auditd with rsyslogConfigure rsyslog for remote forwardingRestart ServicesQuery Logs with jq and awkConvert audit.log to JSONParse with grep and awkLog Rotation and StorageConfigure auditd Log RotationBackup Audit LogsTroubleshootingCheck auditd StatusNo Events RecordedHigh Audit VolumePerformance ImpactIntegration with SIEMForwarding to ELK StackCreate Audit AlertsBest Practices