Monitoring root actions on Linux using Auditd and Wazuh

| by Francisco Navarro | Wazuh 3.10
Post icon

The Linux Audit system takes care of keeping track of what is happening in the operating system by listening to events based on pre-configured rules. Nevertheless, Audit does not provide additional security itself, it is used with other tools to enhance security. In this article we will focus on how monitoring root actions on Linux using Auditd and Wazuh.

Using Wazuh we can analyze the events reported by Audit and generate alerts when required, allowing us to be aware of what’s happening in the endpoints, for example, what commands are being executed with root privileges, and deal with possible security risk if required.

First steps with Linux Audit system

The Linux Audit System is installed by default on most Linux systems. If needed, you may install and enable it with the following commands:

apt-get install auditd audispd-plugins
yum install audit audit-libs
systemctl enable auditd.service && systemctl start auditd.service

Audit’s configuration file is stored at /etc/audit/auditd.conf and it controls the behavior of the Audit daemon according to our needs. For more information, see Linux man: auditd.conf(8).

Audit events to be monitored are selected using rules defined at /etc/audit/rules.d/audit.rules.

We can use the auditctl command to control its behavior, get its status, and add or delete rules into the kernel’s audit system. You can list the rules used by audit with the following command:

auditctl -l
No rules

By default, no auditd rules are configured, but if you’re using the Wazuh syscheck module with who data enabled to monitor any directory, you will have some rules like the following one:

auditctl -l
-w /etc -p wa -k wazuh_fim

These rules are created and maintained by Syscheck itself, which uses the Linux Audit subsystem to get the information about who makes changes in a monitored directory.

For the purpose of this blog post we want to monitor audit logs directly, so we will need to manually create rules at /etc/audit/rules.d/audit.rules and load them using:

auditctl -R /etc/audit/rules.d/audit.rules

After doing so, Auditd will log all the detected events to /var/log/audit/audit.log. We will make Wazuh monitor this file to generate relevant security events using the log data collection module.

Auditing root commands execution

The following rules are used to track the execution of any binary in the system with effective user (eudid) root. Just add them at the end of the audit.rules file and load them using auditctl

-a exit,always -F arch=b64 -F euid=0 -S execve -k  audit-wazuh-c
-a exit,always -F arch=b32 -F euid=0 -S execve -k  audit-wazuh-c

Note that the first rule is for x86-64 architectures and will not work on i386 systems.
If your system has i386 architecture you should only add the second line.

These two rules will control the exit of a process through system calls and will create an audit event when the effective user is root.

Audit generates numerous events and if we were to look directly at the audit.log it is hard to distinguish if those events correspond to write access, read access, execute access, attribute change, or system call rule. For this reason, a list of audit keys known by Wazuh decoders are listed at /var/ossec/etc/lists/audit-keys, these keys are used to differentiate the Audit events expected to be monitored by Wazuh. Specifically, audit-wazuh-c is used to keep track of command execution events.

Read System call monitoring, Audit configuration for further information.

cat /var/ossec/etc/lists/audit-keys
audit-wazuh-w:write
audit-wazuh-r:read
audit-wazuh-a:attribute
audit-wazuh-x:execute
audit-wazuh-c:command

In this case, we’re monitoring command executions, so we will observe the audit-wazuh-c key.

If auditd is present when the Wazuh agent is installed, it will by default monitor the audit.log file, otherwise we can configure Wazuh to monitor auditd log files using Logcollector. This module will forward the messages to the analysisd daemon which will use the audit decoders available in the Wazuh ruleset to automatically extract the relevant audit information from the /var/log/audit/audit.log file and will trigger alerts if they match the criteria of any rule in the ruleset.

If not present already, we need to add the following to our configuration file (/var/ossec/etc/ossec.conf) and restart it.

<localfile>
    <location>/var/log/audit/audit.log</location>
    <log_format>audit</log_format>
</localfile>

After verifying the logcollector configuration is tracking changes in audit.log, if we log into the monitored system with a non-root user (for example, John user, with uid 1001) and run some commands with sudo, or perform a sudo su - followed by some commands run as superuser, an alert will be generated based on the following rule:

<!-- System call rules -->
<rule id="80792" level="3">
     <if_sid>80700</if_sid>
     <list field="audit.key" lookup="match_key_value" check_value="command">etc/lists/audit-keys</list>
     <description>Audit: Command: $(audit.exe)</description>
     <group>audit_command,gdpr_IV_30.1.g,</group>
</rule>

This rule will check that the audit.key in the resulting log message matches the command key defined for Wazuh decoders. This will result in a level 3 alert that includes within its description the command that was executed (audit.exe).

The audit information collected by the audit decoder comes from the /var/log/audit/audit.log file, which will have some entries like the following ones:

type=SYSCALL msg=audit(1574420226.095:1325): arch=c000003e syscall=59 success=yes exit=0 a0=13ca310 a1=13c46e0 a2=14370a0 a3=7ffc7d90b060 items=2 ppid=3635 pid=3653 auid=1001 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=2 comm="touch" exe="/usr/bin/touch" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="audit-wazuh-c"
type=EXECVE msg=audit(1574420226.095:1325): argc=2 a0="touch" a1="/tmp/malicious_file"

Example

If we run the following commands:

[John@wazuhmanager ~]id
uid=1001(John) gid=1001(John) groups=1001(John),10(wheel) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[John@wazuhmanager ~]sudo ls /etc
[sudo] password for John: 
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  swapfile  sys  test  tmp  usr  var
[John@wazuhmanager ~]sudo su -
Last login: Tue Nov  5 13:50:01 UTC 2019 from 192.168.0.154 on pts/1
[root@wazuhmanager ~]# touch /tmp/malicious_file

Some alerts will be generated at /var/ossec/logs/alerts/alert.log with the collected audit information. For example:

** Alert 1574332831.605292: - audit,audit_command,gdpr_IV_30.1.g,
2019 Nov 21 10:40:31 wazuhmanager->/var/log/audit/audit.log
Rule: 80792 (level 3) -> 'Audit: Command: /usr/bin/ls'
type=SYSCALL msg=audit(1574332829.641:625): arch=c000003e syscall=59 success=yes exit=0 a0=70d060 a1=77c890 a2=710440 a3=7ffe09d71b60 items=2 ppid=4406 pid=4425 auid=1001 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=3 comm="ls" exe="/usr/bin/ls" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="audit-wazuh-c" type=EXECVE msg=audit(1574332829.641:625): argc=3 a0="ls" a1="--color=auto" a2="/" type=CWD msg=audit(1574332829.641:625):  cwd="/root" type=PATH msg=audit(1574332829.641:625): item=0 name="/bin/ls" inode=100737708 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:bin_t:s0 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0 type=PATH msg=audit(1574332829.641:625): item=1 name="/lib64/ld-linux-x86-64.so.2" inode=6205 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0 type=PROCTITLE msg=audit(1574332829.641:625): proctitle=6C73002D2D636F6C6F723D6175746F002F
audit.type: SYSCALL
audit.id: 625
audit.arch: c000003e
audit.syscall: 59
audit.success: yes
audit.exit: 0
audit.ppid: 4406
audit.pid: 4425
audit.auid: 1001
audit.uid: 0
audit.gid: 0
audit.euid: 0
audit.suid: 0
audit.fsuid: 0
audit.egid: 0
audit.sgid: 0
audit.fsgid: 0
audit.tty: pts1
audit.session: 3
audit.command: ls
audit.exe: /usr/bin/ls
audit.key: audit-wazuh-c
audit.execve.a0: ls
audit.execve.a1: --color=auto
audit.execve.a2: /
audit.cwd: /root
audit.file.name: /bin/ls
audit.file.inode: 100737708
audit.file.mode: 0100755

The alerts generated in the WUI would look like the following image:

Monitoring Root

Tracking the user running root commands

This configuration tracks our root command execution. However, if we add additional Audit rules to track other users’ actions or give different alert levels according to the user’s UID, we can create custom rules at /var/ossec/etc/rules/local_rules.xml with a richer description and a higher alert level to differentiate between different events of the same kind.

For example, with the following custom rule, we will receive a detailed description (not only of the command executed but also the login id of the user behind such root login, as well as a message indicating that this command has been executed with root privileges). Also, we’ve assigned level 6 for this rule in order to indicate its greater importance.

<rule id="100002" level="6">
    <if_sid>80792</if_sid>
    <field name="audit.euid">0</field>
    <description>Audit: Root command execution: $(audit.exe) with loginuid user $(audit.auid)</description>
    <group>audit_command,</group>
</rule>

After adding such rule (and restarting the manager to make it effective), if we launch another root command with our user John, the alert generated will be the same as the previous one but its header will be:

** Alert 1574332960.1054203: - local,syslog,sshd,audit_command,
2019 Nov 21 10:42:40 wazuhmanager->/var/log/audit/audit.log
Rule: 100002 (level 6) -> 'Audit: Root command execution: /usr/bin/ls with loginuid user 1001'

As we know the user John has uid: 1001 we would know that it is him the one behind such root command execution.

Remember that you can always manage your custom rules and decoders using the WUI, as shown in the next image:

Custom rules and decoders WUI

Tracking and monitoring root-specific command execution

Once we define our rule to detect execution of commands as root, we can create additional rules that detect the use of certain malicious commands, based on this one.

For example, as referred by compliance standards such as CIS, DSS, NIST, a potentially dangerous event on a Linux system is the modification of kernel modules. If we want to check for those events, we need to keep an eye on the execution of commands such as modprobe, rmmod and insmod.

Wazuh allows you to maintain flat-file CDB lists which are compiled into a special binary format to facilitate high-performance lookups in Wazuh rules. Such lists must be created as files, added to the Wazuh configuration, and then compiled. After that, rules that search for decoded fields in those CDB lists as part of their match criteria can be built. Read Using CDB lists to know more about this capability.

We have a list of commands to be monitored so we may create a file /var/ossec/etc/lists/kernel_control_commands with the following content:

insmod:
rmmod:
modprobe:

Then, we must add the following line in the ruleset section of /var/ossec/etc/ossec.conf in the manager.

<list>etc/lists/kernel_control_commands</list>

These CDB lists can also be managed using the WUI, in the Management dashboard:

CDB lists

After that, we can add the following custom rule:

<rule id="100010" level="8">
    <if_sid>100002</if_sid>
    <list field="audit.command" lookup="match_key">etc/lists/kernel_control_commands</list>
    <description>Audit: [Kernel modification] ($(audit.command)) Executed with loginuid user $(audit.auid): $(audit.execve.a0) $(audit.execve.a1) $(audit.execve.a2) </description>
    <group>audit_command,</group>
</rule>

Notice that we have given it a level 8 because we consider this event more relevant for our system’s security, and that this rule is a child of the previous custom rule (if_sid field), this rule will only trigger if the previous one has triggered and the command executed matches the CDB list specified.

Finally, we have to run /var/ossec/bin/ossec-makelists to compile the CDB list and then restart the manager to make the new rule effective.

Now, we can test the new rule by loading a kernel module, for example: modprobe speedstep-lib. An alert with the following header will then be generated:

** Alert 1574333826.3665357: - local,syslog,sshd,audit_command,
2019 Nov 21 10:57:06 wazuhmanager->/var/log/audit/audit.log
Rule: 100010 (level 8) -> 'Audit: [Kernel modification] (modprobe) Executed with loginuid user 1001: modprobe speedstep-lib  '

Using these indications you could write different custom rules to control and monitoring specific root-execution related commands in our systems.

Control and monitoring root

From this point on, we are prepared to monitoring root actions and analyze events of high relevance to our system. For further information you may read the following references:

References

If you have any questions about monitoring root actions on Linux using Auditd and Wazuh, don’t hesitate to check out our documentation to learn more about Wazuh or join our community where our team and contributors will help you.