Exploring security alerting options for improved threat detection in Wazuh – Part 2

An improved security alerting system enhances data breach prevention, ensures compliance, and streamlines operations through timely detection and efficient alert management. In exploring security alerting options for improved threat detection in Wazuh – Part 1, we covered the basic configuration and the first two monitor types. Now, let’s explore the remaining monitor types for better security alerting.

A robust alert system is crucial in today’s fast-changing cyber landscape. Wazuh offers a range of options to tailor alerts to your needs, making it easier to detect and respond to threats effectively.

In this blog post, we focus on configuring Per cluster metrics monitor, Per document monitor, and Composite monitor types in Wazuh. We also show their practical implementation that benefits security analysts in their work to strengthen security measures.

Infrastructure

• A pre-built, ready-to-use Wazuh OVA 4.8.1 that includes the Wazuh core components (Wazuh server, Wazuh indexer, and Wazuh dashboard). Follow the virtual machine (OVA) – installation alternatives to download and set up the Wazuh virtual machine.
• An Ubuntu 22.04.3 endpoint with Wazuh agent 4.8.1 installed and enrolled to the Wazuh server.

Notification channels configuration

In our previous blog post, we configured Slack as a notification channel. In this blog post, we show how to configure Discord and Telegram notification channels by choosing the Custom webhook option in the Notifications plugin. We use the Discord channel for the Per cluster metrics monitor and the Per document monitor. Additionally, we will use the Telegram channel for the Compositor monitor.

Wazuh dashboard

Perform the steps below to configure Discord and Telegram webhooks in the Notifications plugin in your Wazuh dashboard.

Discord channel configuration

Complete the below steps on your Wazuh dashboard to configure a Discord channel.

1. Create a Discord incoming webhook to send security notifications from the Wazuh dashboard to a Discord channel. Follow the intro to webhooks guide to create a Discord incoming webhook.

Note: Copy the Discord webhook URL for later use.

2. Navigate to Notifications under the Explore section in the Wazuh dashboard.

3. Click on Create channel to create a new Discord channel.

4. Enter Discord channel as a Name and provide a suitable Description. You can use any name of your choice.

5. Select Custom webhook as the Channel type and POST as Method under Configurations.

6. Select Webhook URL as Define endpoints by and insert the Discord webhook URL in the Webhook URL option.

7. Click on Create to complete the new Discord notification channel creation process.

After successfully creating the Discord channel, you should see results similar to the below image. Make sure the Notification status shows Active.

Telegram channel configuration

Complete the below steps on your Wazuh dashboard to configure a Telegram channel.

1. Follow the obtain your bot token guide to create your Telegram bot token. Copy the token and we will use it to receive a chat_id for the Telegram bot. We also need the Telegram bot token to push security alert notifications to the Telegram bot.

2. Open a browser and access the below webpage to get your Telegram bot chat_id. Copy the chat_id to construct our notification message later.

https://api.telegram.org/bot<TELEGRAM_BOT_TOKEN>/getUpdates

Where:

• Replace <TELEGRAM_BOT_TOKEN> with your token received in Step 1.

Expected output:

{
    "ok": true,
    "result": [
        {
            "update_id": 132555888,
            "message": {
                "message_id": 1,
                "from": {
                    "id": <chat_id>,
                    "is_bot": false,
                    "first_name": "Abdullah Al",
                    "last_name": "Noman",
                    "username": "x0k3r",
                    "language_code": "en"
                },
                "chat": {
                    "id": <chat_id>,
                    "first_name": "Abdullah Al",
                    "last_name": "Noman",
                    "username": "<username>",
                    "type": "private"
                },
                "date": 1722324221,
                "text": "/start",
                "entities": [
                    {
                        "offset": 0,
                        "length": 6,
                        "type": "bot_command"
                    }
                ]
            }
        }
    ]
}

3. Navigate to Explore > Notifications in the Wazuh dashboard.

4. Click on Create channel to create a new Telegram channel.

5. Define the Telegram channel as a Name and provide a suitable Description.

6. Select Custom webhook as the Channel type and POST as Method under Configurations.

7. Select Webhook URL as Define endpoints by and insert the below constructed Telegram webhook URL in the Webhook URL option.

https://api.telegram.org/bot<TELEGRAM_BOT_TOKEN>/sendMessage

Where:

• Replace <TELEGRAM_BOT_TOKEN> with your token received earlier.

8. Click on Create to complete the new Telegram notification channel creation process.

After successfully creating the Telegram channel, you should be able to see the below image. Make sure the Notification status shows Active.

Wazuh alerting use cases

In part one of our blog series on Wazuh security alerting, we presented two use cases. We discuss three more use cases extensively in this blog post:

• Monitoring unusual memory utilization of Wazuh indexer using Per cluster metrics monitor.
• Detecting unauthorized access to sensitive servers using Per document monitor.
• Potential security breach detection using Composite monitor.

Monitoring unusual memory utilization of Wazuh indexer using Per cluster metrics monitor 

Memory utilization plays a critical role in IT security, particularly in the availability principle of the CIA (Confidentiality, Integrity, and Availability) triad that ensures data and resources are accessible and usable when needed. The efficient management of memory resources is important to maintain system availability. 

High memory usage within the Wazuh indexer can trigger system crashes and downtime, impacting the availability of Wazuh. Monitoring the memory usage of the Wazuh indexer is imperative to preventing these problems.

In this use case, we guide you through setting up a Per cluster metrics monitor on the Wazuh dashboard to monitor the Wazuh indexer’s memory usage. The following steps show how to configure the Alerting plugin to send a notification when the Wazuh indexer memory utilization exceeds a pre-defined threshold.

Wazuh dashboard

Perform the steps below to configure the Per cluster metrics monitor to generate security notifications for Wazuh indexer memory utilization.

1. Navigate to Explore > Alerting > Monitors on the Wazuh dashboard.

2. Click on the Create monitor button to create a new monitor for Wazuh indexer memory utilization.

2. Specify the Monitor name “Unusual memory utilization of Wazuh indexer monitoring” under the Monitor details section. You can use any name of your choice.

3. Select Per Cluster metrics monitor in the Monitor details section under the Monitor type.

4. Select By interval from the drop-down at the Frequency block of the Schedule section. 

5. Select 1 and Minute(s) at the Run every block of the Schedule section.

6. Choose Cluster stats from the Request type dropdown in the Query section.

7. Specify node-1 as the Path parameters which is the name of the Wazuh indexer to be monitored.

8. Click on the Preview query to get a valid response for node-1.

We establish predefined trigger conditions that will notify us when the free memory of the indexer node falls below 30%.

Follow the steps below to configure a trigger to define trigger conditions that generate security notifications.

9. Select Add trigger to define conditions and actions in the Triggers section.

10. Specify the Trigger name as “Unusual memory utilization trigger”.

11. Specify the Severity level as 1 (Highest). 

12. Copy and paste the painless script below for the Trigger condition.

ctx.results[0].nodes.os.mem.free_percent <= 30

13. Specify “Wazuh indexer usual memory utilization notification” as the Action name in the Action section.

14. Select [Channel] Discord channel from the dropdown as the Channels.

15. Replace the existing contents with the following in the Message box.

{
  "username": "Wazuh Alert Notification",
  "embeds": [
	  {
	    "title": "Monitor _{{ctx.monitor.name}}_ just entered alert status. Please investigate the issue.\n",
		"description": "- **Trigger**: {{ctx.trigger.name}}\n - **Severity**: {{ctx.trigger.severity}}\n - **Period start**: {{ctx.periodStart}}\n - **Period end**: {{ctx.periodEnd}}\n",
		"color": 16711680
	  }
    ]
}

16. Click on the Create button to complete the Per cluster metrics monitor configuration.

Result

Since we are monitoring Wazuh indexer metrics, we do not need to perform any additional activities to generate security alerts. Based on the resource usage of the Wazuh indexer, once the threshold is met, a security notification will be triggered. 

Navigate to Explore > Alerting > Monitors and select Unusual memory utilization of Wazuh indexer monitoring on the Wazuh dashboard to view generated alerts.

The below image shows notifications sent to the preconfigured Discord channel.

Detecting unauthorized access to sensitive servers using Per document monitor

Successful authentication to servers from unauthorized locations indicates a possible intrusion. The Per document monitor can be used as a security measure to alert on successful login attempts to an endpoint from pre-defined unauthorized locations.

The concept of this detection lies in observing instances of successful authentication to all servers scoped to specific groups. We establish monitors to alert when such successful authentications occur in locations not sanctioned by us. We use the GeoLocation.country_name field of all indexed documents to track where the users originate from.

In this use case, we guide you through setting up a Per document monitor on your Wazuh dashboard to keep track of suspicious authentication to sensitive servers from unauthorized locations. The following steps will help you configure the alerting plugin to notify you of such activities in your environment.

Wazuh dashboard

Perform the steps below to configure the Per document monitor to generate security notifications for detecting unauthorized access to the sensitive servers.

1. Navigate to Explore > Alerting > Monitors on the Wazuh dashboard.

2. Click on the Create monitor button to create a new monitor for detecting unauthorized access to sensitive servers.

3. Specify “Unauthorized access to sensitive servers monitoring” as the Monitor name under the Monitor details section.

4. Select Per document monitor under Monitor type in the Monitor details section.

5. Select Visual editor as the preferred method at the Monitoring defining method for specifying the query and trigger.

6. Select By interval from the drop-down at the Frequency block of the Schedule section. 

7. Select 1 and  Minute(s) at the run every block of the Schedule section.

Now, we define the data source from where the security data will be collected to our monitor.

8. Enter the Index wazuh-alerts-* as the Data Source.

In this step, we will define three unique queries to detect SSH access from Location_1 and Location_2 marked as unauthorized locations. We consider Singapore and Japan as the unauthorized locations in this use case.

Query A

9. Specify Security_event as a query name.

10. Select  predecoder.program_name and then is from the dropdown, type sshd to define the exact string to be captured from the field.

11. Click on Add another query to add our next query.

Query B

12. Specify Location_1 as a query name.

13. Select  GeoLocation.country_name and then is from the dropdown, type Singapore to define the exact string to be captured from the field. 

14. Click on Add another query to add our final query.

Query C

15. Specify Location_2 as a query name.

16. Select  GeoLocation.country_name and then is from the dropdown type Japan to define the exact string to be captured from the field.

We establish predefined trigger conditions that will notify us when there are successful authentications to a sensitive server from unauthorized locations.

17. Specify the Trigger name as “Unauthorized access to sensitive servers trigger”.

18. Specify the Severity level as 1 (Highest).

19. Select Security_event from the dropdown as the first query under Specify queries or tags.

20. Click on Add condition to specify another trigger condition.

21. Select Location_1 with AND condition as the second query.

22. Click on Add condition to specify another trigger condition.

23. Select Location_2 with OR condition as the third query.

24. Specify “Unauthorized access to sensitive servers monitoring notification” as the Action name in the Action section.

25. Select [Channel] Discord channel under Channels.

26. Replace the existing contents with the following in the Message box.

{
  "username": "Wazuh Alert Notification",
  "embeds": [
	  {
	    "title": "Monitor _{{ctx.monitor.name}}_ just entered alert status. Please investigate the issue.\n",
		"description": "- **Trigger**: {{ctx.trigger.name}}\n - **Severity**: {{ctx.trigger.severity}}\n - **Period start**: {{ctx.periodStart}}\n - **Period end**: {{ctx.periodEnd}}\n",
		"color": 16711680
	  }
    ]
}

27. Select Per execution as the Action configuration.

28. Click on the Create button to complete the Per cluster metrics monitor configuration.

Attack emulation

In this section, we manually ingest some logs to the Wazuh server to generate security alerts that trigger our newly created monitor based on the Per document monitor. 

Perform the following steps on the enrolled Ubuntu endpoint.

1. Create a per_document_monitor.log file in the root directory (/):

# touch /per_document_monitor.log

2. Add the following line in the <ossec_config> block of the Wazuh agent configuration /var/ossec/etc/ossec.conf file:

  <localfile>
    <log_format>syslog</log_format>
    <location>/per_document_monitor.log</location>
  </localfile>

3. Restart the Wazuh agent to apply the changes:

# systemctl restart wazuh-agent

4. Ingest the following SSH logs to generate relevant security alerts:

# echo "Jul 15 17:08:34 Ubuntu_22 sshd[13858]: Accepted password for user from 101.0.26.28 port 64679 ssh2
Jul 15 17:08:34 Ubuntu_22 sshd[13858]: Accepted password for user from 121.7.124.190 port 64679 ssh2
Jul 15 17:08:34 Ubuntu_22 sshd[13858]: Accepted password for user from 101.0.26.88 port 64679 ssh2
Jul 15 17:08:34 Ubuntu_22 sshd[13858]: Accepted password for user from 121.7.125.157 port 64679 ssh2" >> /per_document_monitor.log

Result

Once the activities in the Attack emulation section are performed, you should see the relevant security alerts generated in your Wazuh dashboard. Navigate to Explore > Alerting > Monitors and select Unauthorized access to sensitive servers monitoring on the Wazuh dashboard to view generated alerts.

The below image shows a notification sent to the preconfigured Discord channel.

Potential security breach detection using Composite monitor

Security breaches can lead to unauthorized access to sensitive data, loss of confidential information, financial damage, and disruption of services. To safeguard against security breaches, one of the most crucial methods is to have a better monitoring system that promptly informs users about any potential cyber attack on a system.

In this use case, we first detect unauthorized access to a sensitive server and then detect abnormal file deletion activities in the same server. These are high-severity attacks that indicate potential security breaches. To demonstrate this, we generate more focused security alerts using a Composite monitor by combining Per bucket monitor and Per document monitor types.

Previously, we have configured a Per document monitor to detect unauthorized access to sensitive servers from unauthorized locations. We have also configured a Per bucket monitor to detect abnormal file deletion activities in part one of this blog post series. Now, we construct a Composite monitor to produce a chained alert when both of the above monitors generate an alert. The chained alert will be sent to the configured Telegram channel.

Wazuh dashboard

Perform the steps below to configure the Composite monitor to generate security notifications for possible security breach detection.

1. Navigate to Explore > Alerting > Monitors on your Wazuh dashboard.

2. Click on the Create monitor to create a new Composite monitor.

3. Name “Possible security breach detection” as the Monitor name.

4. Select Composite monitor as the Monitor type.

5. Select Visual editor as the Monitor defining method.

6. Select By interval as Frequency under Schedule that should run every 1 minute.

7. Select Unauthorized access to sensitive servers monitoring and Abnormal file deletion monitoring as the Delegate monitors.

Now, we create a trigger. Click on the Add trigger to configure conditions and actions.

8. Name “Possible security breach detection trigger” as the Trigger name.

9. Set the Trigger conditions as Unauthorized access to sensitive servers monitoring and Abnormal file deletion monitoring.

10. Set 1(Highest) as the Alert severity.

11. Select the Telegram channel as the Notification channel under Notifications.

12. Click on Configure notification to construct a notification message.

13. Replace the existing contents with the following in the Message box and click Update to set the notification message:

{
  "chat_id": "<chat_id>",
  "text": "Monitor <code>{{ctx.monitor.name}}</code> just entered alert status. Please investigate the issue.\n
  - <code>Trigger</code>: {{ctx.trigger.name}}
  - <code>Severity</code>: {{ctx.trigger.severity}}
  - <code>Period start</code>: {{ctx.periodStart}}
  - <code>Period end</code>: {{ctx.periodEnd}}",
  "parse_mode": "HTML"
}

Where:

• Replace <chat_id> with the chat_id received in the Telegram channel configuration section.

14. Click on Create to finish the composite monitoring configuration process.

Attack emulation

Follow the steps below in your monitored Ubuntu endpoint.

1. Perform the Attack emulation section in detecting unauthorized access to sensitive servers using Per document monitor.

2. Perform the Attack emulation section under detecting abnormal file deletion activities using Per bucket monitor.

Result

Once the activities in the Attack emulation section are performed, you should see the relevant security alerts generated in your Wazuh dashboard. 

Navigate to Explore > Alerting > Monitors and select Potential security breach detection on the Wazuh dashboard to view generated alerts.

Conclusion

This blog post demonstrates how to explore Wazuh with its security alerting options that enable improved security operations. We configured the Per query monitor and Per bucket monitor in our first security alerting blog post. In this part 2, We covered the remaining monitor types – Per cluster metrics monitor, Per document monitor, and Composite monitor with real-world use cases that help organizations take prompt actions against security incidents.

Wazuh is an open source solution designed for robust threat detection, incident response, compliance management, and more. It boasts a thriving community dedicated to supporting its users. For further insights into Wazuh, we encourage you to explore our comprehensive documentation and insightful blog posts.

Reference

• Exploring security alerting for improved threat detection in Wazuh – Part 1
• Composite monitors