Persistence techniques are mechanisms or configurations threat actors use to maintain illicit access to compromised endpoints after gaining initial access. Persistence guarantees that attackers have endpoint access regardless of system restarts, changed credentials, or other interruptions that may potentially terminate illegal access.
Once persistence techniques are set up, threat actors no longer have to perform exploitation steps to regain access to the endpoint. Examples of persistence techniques include the use of startup scripts, the creation of user accounts, and the modification of existing binaries.
This blog post covers a few common Linux persistence techniques an adversary uses to establish permanent access to compromised endpoints.
MITRE ATT&CK persistence techniques and detection
The MITRE Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) Framework provide real-world adversary tactics and techniques for modeling the behavior of threat actors. Attackers commonly use some of the following Linux persistence techniques:
- T1098.004 – Account Manipulation: SSH Authorized Keys
- T1037.004 – Boot or Logon Initialization Scripts: RC Scripts
- T1136.001 – Create Account: Local Account
- T1078.003 – Valid Accounts: Local Accounts
- T1546.004 – Event Triggered Execution: Unix Shell Configuration Modification
- T1574.006 – Hijack Execution Flow: Dynamic Linker Hijacking
- T1053.003 – Scheduled Task/Job: Systemd Timers
- T1505.003 – Server Software Component: Web Shell
Note
We implemented the detection techniques using Wazuh 4.3.10 and simulated the attacks against an Ubuntu 22.04 LTS endpoint.
T1098.004 – Account Manipulation: SSH Authorized Keys
This persistence technique uses SSH key-based authentication to maintain access to compromised endpoints. SSH, or secure shell, is an encrypted protocol used to administer and securely communicate with servers. SSH uses a file called authorized_keys
that defines the SSH keys used for logging into a user account. You must configure this file in any user account that needs SSH keys for permanent access. The default configuration in most SSH implementations allows you to deploy new authorized keys.
Threat actors can leverage this technique to import their public keys into the file ~/.ssh/authorized_keys
to establish communication with compromised endpoints at any point in time without specifying a password. Before threat actors can leverage this technique, they need to ensure PubkeyAuthentication
is enabled in the SSH configuration file /etc/ssh/sshd_config
. Additionally, threat actors need to ensure PermitRootLogin
is enabled when logging in as root.
To detect this persistence technique with Wazuh, configure the file integrity monitoring (FIM) module to monitor the creation and modification of the ~/.ssh/authorized_keys
and /etc/ssh/sshd_config
files. The Wazuh FIM module monitors selected file paths on a filesystem and triggers alerts when files are created, modified, or deleted.
Perform the following steps on the Linux and Wazuh server endpoints to detect this persistence technique.
Note
Ensure to restart the Wazuh agent after creating a new user’s /.ssh/
directory so that the FIM module can adequately monitor it.
Linux endpoint configuration
1. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file:
<ossec_config> <syscheck> <directories check_all="yes" realtime="yes">/root/.ssh/</directories> <directories check_all="yes" realtime="yes">/home/*/.ssh/</directories> <directories check_all="yes" realtime="yes">/var/*/.ssh/</directories> <directories check_all="yes" realtime="yes">/etc/ssh/sshd_config</directories> </syscheck> </ossec_config>
2. Restart the Wazuh agent to apply the configuration changes:
$ sudo systemctl restart wazuh-agent
Wazuh server configuration
1. Create a custom rules file comm_persist_tech_rules.xml
in the /var/ossec/etc/rules/
directory and add the following rules to trigger alerts for the SSH authorized keys persistence technique:
<group name="common_persistence_techniques,sshd,"> <rule id="100100" level="10"> <if_sid>554</if_sid> <field name="file" type="pcre2">\/authorized_keys$</field> <description>SSH authorized_keys file "$(file)" has been added</description> <mitre> <id>T1098.004</id> </mitre> </rule> <rule id="100101" level="10"> <if_sid>550</if_sid> <field name="file" type="pcre2">\/authorized_keys$</field> <description>SSH authorized_keys file "$(file)" has been modified</description> <mitre> <id>T1098.004</id> </mitre> </rule> <rule id="100102" level="10"> <if_sid>550</if_sid> <field name="file" type="pcre2">\/sshd_config$</field> <description>SSH config file "$(file)" has been modified</description> <mitre> <id>T1098.004</id> </mitre> </rule> </group>
Where:
- Rule ID
100100
is triggered when the~/.ssh/authorized_keys
file is added to the monitored endpoint. - Rule ID
100101
is triggered when the~/.ssh/authorized_keys
file is modified on the monitored endpoint. - Rule ID
100102
is triggered when the/etc/sshd_config
file is modified on the monitored endpoint.
2. Restart the Wazuh manager to apply the configuration changes:
The following alerts are generated on the Wazuh dashboard when FIM detects changes in the monitored files.
T1037.004 – Boot or Logon Initialization Scripts: RC Scripts
Boot or Logon Initialization Scripts are scripts automatically executed at boot or logon initialization without user interaction. RC scripts include rc.local
, rc.common
, and other RC scripts specific to the Unix-like distribution.
The /etc/rc.local
is a script that Linux administrators use to define startup scripts or custom services. However, the RC script has been replaced with systemd
, and it is now mainly used for compatibility purposes. The generator systemd-rc-local-generator
included in the current version of systemd checks for the existence of /etc/rc.local
and executes it on boot. You should monitor this script, as modifications may be malicious.
Threat actors can leverage this script to create backdoors that give them continued access to compromised endpoints.
We use the following methods to detect this persistence technique:
- Auditd: An auditing framework that collects and stores system events such as operating system calls, functions, and file access events. It is used to monitor the creation, modification, or deletion of the
/etc/rc.local
file. - Command monitoring: This Wazuh module monitors events that are not logged on the endpoint by executing commands and analyzing their output. It is used to monitor for suspicious process creation, for example, the
/etc/rc.local
process with aparent process ID (PPID)
of1
. Processes with PPID of 1 indicate that the parent process terminated without waiting for the child to finish. It is also used to check the status ofrc-local.service
. A state other than theinactive
state may indicate an attack attempt.
Perform the following steps on the Linux and Wazuh server endpoints to detect this persistence technique.
Linux endpoint configuration
1. Install “auditd” using the following command:
# apt install auditd -y
2. Add the following auditd rules to the /etc/audit/rules.d/audit.rules
file to detect the creation, modification, or deletion of the /etc/rc.local
file using the following command:
# echo "-w /etc/rc.local -p wa -k possible_rcscript_attack" >> /etc/audit/rules.d/audit.rules
3. Reload the auditd rules file and verify the above configuration is applied:
# auditctl -R /etc/audit/rules.d/audit.rules # auditctl -l
4. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file. This configuration forwards auditd logs to the Wazuh server and configures command monitoring.
<ossec_config> <!-- Forwarding auditd logs to the Wazuh server --> <localfile> <log_format>audit</log_format> <location>/var/log/audit/audit.log</location> </localfile> <!-- Command monitoring (command executes every 180 seconds) --> <localfile> <log_format>command</log_format> <command>ps -ef | grep "[/]etc/rc.local" | awk '{print }'</command> <alias>ppid of rc_local</alias> <frequency>180</frequency> </localfile> <localfile> <log_format>command</log_format> <command>systemctl show rc-local.service --property=ActiveState | awk -F"=" '{print}'</command> <alias>state of rc_local service</alias> <frequency>180</frequency> </localfile> </ossec_config> audit /var/log/audit/audit.log command ps -ef | grep "[/]etc/rc.local" | awk '{print $3}' ppid of rc_local 180 command systemctl show rc-local.service --property=ActiveState | awk -F"=" '{print $2}' state of rc_local service 180
5. Restart the Wazuh agent to apply the configuration changes:
$ sudo systemctl restart wazuh-agent
Wazuh server configuration
1. Add the following rules to the /var/ossec/etc/rules/comm_persist_tech_rules.xml
file to trigger alerts for this persistence technique:
<group name="common_persistence_techniques,"> <rule id="100105" level="10"> <if_sid>80700</if_sid> <field name="audit.key">possible_rcscript_attack</field> <description>[RC script "$(audit.file.name)" has been added, modified, or deleted]: Possible RC scripts persistence attack</description> <mitre> <id>T1037.004</id> </mitre> </rule> <rule id="100106" level="10"> <if_sid>530</if_sid> <match>'ppid of rc_local': 1$</match> <description>[PPID of rc-local.service is 1]: Possible RC scripts persistence attack</description> <mitre> <id>T1037.004</id> </mitre> </rule> <rule id="100107" level="10"> <if_sid>530</if_sid> <regex type="pcre2">'state of rc_local service': (?!inactive$)</regex> <description>[rc-local.service is not inactive]: Possible RC scripts persistence attack</description> <mitre> <id>T1037.004</id> </mitre> </rule> </group>
Where:
- Rule ID
100105
is triggered when the/etc/rc.local
file is added, modified, or deleted on the monitored endpoint. - Rule ID
100106
is triggered when the PPID ofrc-local.service
is1
. - Rule ID
100107
is triggered when the status ofrc-local.service
is not inactive.
2. Restart the Wazuh manager to apply the configuration changes:
$ sudo systemctl restart wazuh-manager
The following alerts are generated on the Wazuh dashboard when the rules are triggered.
T1136.001 – Create Account: Local Account
This persistence technique involves the creation of new user accounts on compromised endpoints to maintain endpoint access. Adversaries can create users using the useradd
, usermod
, and passwd
commands. Activities of these commands modify the /etc/shadow
, /etc/gshadow
, /etc/passwd
, /etc/group
, and /etc/login.defs
files and are logged in /var/log/auth.log
.
By default, Wazuh detects when a new user is added to Linux endpoints by monitoring user creation-related commands. However, threat actors can manually modify these files to create users without using the commands.
You can use the Wazuh FIM module to detect this technique by monitoring modifications to the above files.
Perform the following steps on the Ubuntu and Wazuh server endpoints to detect this persistence technique.
Linux endpoint configuration
1. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file:
<ossec_config> <syscheck> <directories check_all="yes" realtime="yes">/etc/shadow</directories> <directories check_all="yes" realtime="yes">/etc/gshadow</directories> <directories check_all="yes" realtime="yes">/etc/passwd</directories> <directories check_all="yes" realtime="yes">/etc/group</directories> <directories check_all="yes" realtime="yes">/etc/login.defs</directories> </syscheck> </ossec_config>
2. Restart the Wazuh agent to apply the configuration changes:
$ sudo systemctl restart wazuh-agent
Wazuh server configuration
1. Add the following rules to the /var/ossec/etc/rules/comm_persist_tech_rules.xml
to trigger alerts for this persistence technique:
<group name="common_persistence_techniques,"> <rule id="100115" level="10"> <if_sid>550</if_sid> <field name="file" type="pcre2">\/etc\/passwd$|\/etc\/shadow$|\/etc\/gshadow$|\/etc\/group$|\/etc\/login.defs$</field> <description>[File "$(file)" has been modified]: Possible local account manipulation</description> <mitre> <id>T1136.001</id> <id>T1078.003</id> </mitre> </rule> </group>
Where:
- Rule ID
100115
is triggered when the/etc/shadow
,/etc/gshadow
,/etc/passwd
,/etc/group
, and/etc/login.defs
files are modified.
2. Restart the Wazuh manager to apply the configuration changes:
$ sudo systemctl restart wazuh-manager
The following alerts are generated on the Wazuh dashboard when the rules are triggered.
T1078.003 – Valid Accounts: Local Accounts
Local accounts are those configured for users of an endpoint or service. They are locally stored on an endpoint and have assigned rights and permissions for the endpoint. Local accounts can be used for remote administration. This persistence technique involves the manipulation of local accounts to maintain persistence on compromised endpoints.
Cyber threat actors can modify existing account credentials to persist on compromised endpoints. Local accounts or users are stored in the /etc/passwd
file of Linux endpoints. The usermod
, chage
, and passwd
commands modify existing user account details, such as username, password, shells, and more. These commands modify the /etc/shadow
, /etc/gshadow
, /etc/passwd
, /etc/group
, and /etc/login.defs
configuration files containing user account information.
You can monitor file modifications using the Wazuh FIM module to detect this persistence technique. By default, Wazuh detects remote SSH connections in case modified accounts are used to access compromised endpoints remotely.
The same rules in the T1136.001 – Create Account: Local Account provide coverage for this persistence technique.
T1546.004 – Event Triggered Execution: Unix Shell Configuration Modification
This persistence technique involves executing malicious commands through scripts triggered by a user’s shell. When Unix shells launch, they execute several configuration scripts. Threat actors can leverage these scripts to maintain persistence on compromised endpoints. They achieve this by inserting malicious commands that automatically get executed by shells. These scripts are /etc/profile
, /etc/profile.d
, /etc/bash.bashrc
, /etc/bash.bash_logout
, ~/.bash_profile
, ~/.bash_login
, ~/.profile
, ~/.bash_profile
, ~/.bashrc
, and ~/.bash_logout
.
To detect this persistence technique, monitor for changes in shell configuration scripts using the Wazuh FIM module.
Perform the following steps on the Linux and Wazuh server endpoints to detect this persistence technique.
Linux endpoint configuration
1. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file:
<ossec_config> <syscheck> <directories check_all="yes" realtime="yes">/etc/</directories> <directories check_all="yes" realtime="yes">/home/*/.bash_profile</directories> <directories check_all="yes" realtime="yes">/home/*/.bash_login</directories> <directories check_all="yes" realtime="yes">/home/*/.profile</directories> <directories check_all="yes" realtime="yes">/home/*/.bash_profile</directories> <directories check_all="yes" realtime="yes">/home/*/.bashrc</directories> <directories check_all="yes" realtime="yes">/home/*/.bash_logout</directories> <directories check_all="yes" realtime="yes">/root/.bash_profile</directories> <directories check_all="yes" realtime="yes">/root/.bash_login</directories> <directories check_all="yes" realtime="yes">/root/.profile</directories> <directories check_all="yes" realtime="yes">/root/.bash_profile</directories> <directories check_all="yes" realtime="yes">/root/.bashrc</directories> <directories check_all="yes" realtime="yes">/root/.bash_logout</directories> </syscheck> </ossec_config>
2. Restart the Wazuh agent to apply the configuration changes:
$ sudo systemctl restart wazuh-agent
Wazuh server configuration
1. Add the following rules to the /var/ossec/etc/rules/comm_persist_tech_rules.xml
to trigger alerts for this persistence technique:
<group name="common_persistence_techniques,"> <rule id="100120" level="10"> <if_sid>554</if_sid> <field name="file" type="pcre2">\/etc\/profile$|\/etc/profile.d\/|\/etc\/bash.bashrc$|\/etc\/bash.bash_logout$|.bash_profile$|.bash_login$|.profile$|.bash_profile$|.bashrc$|.bash_logout$</field> <description>Unix shell config "$(file)" has been added</description> <mitre> <id>T1546.004</id> </mitre> </rule> <rule id="100121" level="10"> <if_sid>550</if_sid> <field name="file" type="pcre2">\/etc\/profile$|\/etc/profile.d\/|\/etc\/bash.bashrc$|\/etc\/bash.bash_logout$|.bash_profile$|.bash_login$|.profile$|.bash_profile$|.bashrc$|.bash_logout$</field> <description>Unix shell config "$(file)" has been modified</description> <mitre> <id>T1546.004</id> </mitre> </rule> </group>
Where:
- Rule IDs
100120
and100121
are triggered when the/etc/profile
,/etc/profile.d
,/etc/bash.bashrc
,/etc/bash.bash_logout
,~/.bash_profil
e,~/.bash_login
,~/.profile
,~/.bash_profile
,~/.bashrc
, and~/.bash_logout
scripts are added or modified respectively.
2. Restart the Wazuh manager to apply the configuration changes:
$ sudo systemctl restart wazuh-manager
The following alert is generated on the Wazuh dashboard when the /root/.bashrc
and /etc/bash.bashrc
files are modified.
T1574.006 – Hijack Execution Flow: Dynamic Linker Hijacking
The dynamic linker is the program that loads shared objects (shared libraries) required by an executable. It loads the shared library into system memory and prepares the executable for use at runtime. The dynamic linker uses environment variables such as LD_PRELOAD
, or configuration files such as /etc/ld.so.preload
to load shared libraries.
Threat actors can trick the dynamic linker into loading malicious libraries by hijacking the execution flow of an application. The LD_PRELOAD
environment variable and the /etc/ld.so.preload
file usually contain user-specified shared objects to be loaded first by the dynamic linker. By default, the LD_PRELOAD
environment variable is not set, and the /etc/ld.so.preload
file does not exist in a vanilla installation of Linux. A malicious library is likely to be in use if this is the case.
To detect the Dynamic Linker Hijacking persistence technique, monitor the creation or modification of /etc/ld.so.preload
using the Wazuh FIM module. You can also check if the LD_PRELOAD
environment variable is set by using the command monitoring module.
Perform the following steps on the Linux and Wazuh server endpoints to detect this persistence technique.
Linux endpoint configuration
1. With auditd already installed and configured, add the following auditd rules to the /etc/audit/rules.d/audit.rules
file to monitor the creation, modification, or deletion of the /etc/ld.so.preload
file using the following command:
# echo "-w /etc/ld.so.preload -p wa -k possible_preload_hijack" >> /etc/audit/rules.d/audit.rules
2. Reload the auditd rules file and verify the above configuration is applied:
# auditctl -R /etc/audit/rules.d/audit.rules # auditctl -l
3. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file to monitor when the LD_PRELOAD
environment variable is set:
<ossec_config> <!-- Command monitoring (command executes every 180 seconds) --> <localfile> <log_format>command</log_format> <command>printenv LD_PRELOAD</command> <alias>check for LD_PRELOAD envar</alias> <frequency>180</frequency> </localfile> </ossec_config>
4. Restart the Wazuh agent to apply the configuration changes:
# systemctl restart wazuh-agent
Wazuh server configuration
1. Add the following rules to the /var/ossec/etc/rules/comm_persist_tech_rules.xml
file to trigger alerts for this persistence technique:
<group name="common_persistence_techniques,"> <rule id="100125" level="10"> <if_sid>80700</if_sid> <field name="audit.key">possible_preload_hijack</field> <description>[Config file "ld.so.preload" has been added, modified, or deleted]: Possible dynamic linker hijacking</description> <mitre> <id>T1574.006</id> </mitre> </rule> <rule id="100126" level="10"> <if_sid>530</if_sid> <match>'check for LD_PRELOAD envar'</match> <description>[LD_PRELOAD envar has been set]: Possible dynamic linker hijacking</description> <mitre> <id>T1574.006</id> </mitre> </rule> </group>
Where:
- Rule ID
100125
is triggered when the/etc/ld.so.preload
file is added, modified, or deleted from the monitored endpoint. - Rule ID
100126
is triggered when theLD_PRELOAD
environment variable is set to preload a user-defined shared library.
2. Restart the Wazuh manager to apply the configuration changes:
$ sudo systemctl restart wazuh-manager
The following alerts are generated on the Wazuh dashboard when the rules are triggered.
T1053.003 – Scheduled Task/Job: Systemd Timers
Systemd timers are unit files used to schedule jobs in Linux environments. Timers can be used as an alternative to Cron. Systemd timers control system services and are appended with the file extension .timer
. Systemd timers and systemd services work together in scheduling system services. This implies each .timer
file must have a corresponding .service
file with the same name. The files are generally stored in the /etc/systemd/system/
, /usr/lib/systemd/system/
, /usr/local/lib/systemd/system/
, and /lib/systemd/system/
directories.
Threat actors can schedule the execution of malicious code by creating or modifying systemd timers.
To detect this persistence technique, configure the Wazuh FIM module to detect the creation or modification of .timer
and .service
files in the directories mentioned above.
Perform the following steps on the Linux and Wazuh server endpoints to detect this persistence technique.
Linux endpoint configuration
1. Append the following configuration to the Wazuh agent /var/ossec/etc/ossec.conf
file:
<ossec_config> <syscheck> <directories check_all="yes" realtime="yes">/etc/systemd/system/</directories> <directories check_all="yes" realtime="yes">/usr/lib/systemd/system/</directories> <directories check_all="yes" realtime="yes">/usr/local/lib/systemd/system/</directories> <directories check_all="yes" realtime="yes">/lib/systemd/system/</directories> </syscheck> </ossec_config>
2. Restart the Wazuh agent to apply the configuration changes:
$ sudo systemctl restart wazuh-agent
Wazuh server configuration
1. Add the following rules to the /var/ossec/etc/rules/comm_persist_tech_rules.xml
file to trigger alerts for this persistence technique:
<group name="common_persistence_techniques,"> <rule id="100130" level="12"> <if_sid>554</if_sid> <field name="file" type="pcre2">\/systemd\/system\/.*\.timer$|\/systemd\/system\/.*\.service$</field> <description>[Systemd "$(file)" has been added]: Possible task/job scheduling</description> <mitre> <id>T1053.006</id> </mitre> </rule> <rule id="100131" level="12"> <if_sid>550</if_sid> <field name="file" type="pcre2">\/systemd\/system\/.*\.timer$|\/systemd\/system\/.*\.service$</field> <description>[Systemd "$(file)" has been modified]: Possible task/job scheduling</description> <mitre> <id>T1053.006</id> </mitre> </rule> </group>
Where:
- Rule IDs
100130
and100131
are triggered when.timer
and.service
files are added and modified respectively in the monitored/etc/systemd/system/
,/usr/lib/systemd/system/
,/usr/local/lib/systemd/system/
, and/lib/systemd/system/
directories.
2. Restart the Wazuh manager to apply the configuration changes:
$ sudo systemctl restart wazuh-manager
The following alerts are generated on the Wazuh dashboard when the rules are triggered.
The detection for T1053.003 – Scheduled Task/Job: Cron can be found on section 3 of detecting illegitimate crypto miners on Linux endpoints.
T1505.003 – Server Software Component: Web Shell
A web shell is a malicious script that enables threat actors to compromise web servers and launch additional attacks. Threat actors use this technique to maintain persistent access to compromised endpoints.
Kindly visit our blog post on web shell attack detection with Wazuh for detailed information about its detection.
Conclusion
This blog post shows several techniques to establish persistence on compromised Linux endpoints. Following the MITRE attack framework, we covered how cyber threat actors use these Linux persistence techniques to maintain connectivity across compromised endpoints.
We have been able to detect the presence of persistence activities on Linux endpoints using the Wazuh FIM and command monitoring capabilities. Additionally, we integrated the Linux auditing system, auditd to monitor specific file events.
If you have any questions on this blog post or Wazuh in general, kindly join our Slack community.