-
AIOps & SecOps Case Study — Part 1

Building a Zero-Cloud AIOps & Security Digest Engine

Executive Summary

  • Objective: Establish a 100% self-hosted, private SOC/NOC reporting system that aggregates network telemetry and threat logs without relying on external cloud infrastructure.
  • Core Stack: Zabbix 7, n8n, Suricata IDS/IPS, MikroTik REST API, Ollama (Qwen), and SMTP.
  • Key Impact: Reduced daily log review time from 30+ minutes down to a 1-minute email digest, eliminating third-party API costs while keeping internal IP architectures fully isolated.

The Problem: Alert Fatigue vs. Data Sovereignty

  • Log Telemetry Overload: Manually reviewing thousands of daily Suricata entries (fast.log / eve.json) alongside Zabbix metrics induces severe operator fatigue.
  • Data Sovereignty & Privacy Constraints: Transmitting internal IP schemas, network topology, and vulnerability vectors to public cloud LLM APIs poses significant security risks.
  • Context Window Exhaustion: Raw JSON payloads and verbose log outputs contain excessive noise that quickly exceeds the context window limits of lightweight local models.

What is n8n?
n8n is an extensible, fair-code workflow automation platform that connects self-hosted tools and infrastructure APIs via custom node-based logic. Unlike cloud-locked SaaS alternatives (e.g., Zapier, Make), n8n can be deployed 100% on-premise, making it the ideal orchestration engine for private SecOps pipelines and sovereign AIOps workflows.

System Architecture & Data Pipeline

A centralized n8n workflow orchestrates the entire telemetry pipeline through five distinct stages:

Pipeline Stage Component Execution Logic
1. Trigger Schedule Trigger Executes an automated workflow run at scheduled intervals (e.g., Daily at 07:00 AM).
2. Ingestion Zabbix Request Nodes Queries the Zabbix 7 REST API to pull MikroTik system metrics, Suricata threat alerts, and SCA compliance tables.
3. Transformation Custom JS Code Nodes Executes Fix Suricata Logs and Fix SCA Table nodes to strip structural noise and extract core fields.
4. Context Merge Merge Node Unifies sanitized network metrics and threat telemetry into a structured JSON payload.
5. AI Inference & Delivery Ollama (Qwen) & SMTP The local Qwen LLM processes the payload to synthesize an executive summary, which is then dispatched via the SMTP node.

Log Sanitization & Model Optimization

To maximize inference performance on low-resource local LLMs without causing hallucination or context window truncation, strict preprocessing guardrails were deployed:

  • Regex Sanitization: Stripped redundant string headers, metadata wrappers, and repetitive timestamp tokens from raw Suricata event logs.
  • Array Aggregation: Grouped duplicate threat SIDs into frequency counters rather than passing repeated raw log lines.
  • Prompt Engineering: Enforced strict system instructions to guarantee structured, concise executive summaries and actionable remediation items.

Complete n8n Workflow Blueprint

The production-ready, sanitized n8n workflow JSON—complete with pre-configured JS transformation nodes and LLM system prompts—is available on GitHub.

📦 Download Workflow JSON Blueprint Visit GitHub Repository →

How to Import & Setup:

  1. Download workflow_clean.json from the repository.
  2. Open your n8n dashboard and go to Workflows → Import from File.
  3. Replace all <placeholder-here> tags (e.g., <zabbix-ip-here>, <ollama-ip-here>) with your local infrastructure details.

Outcomes & Future Roadmap

The core ingestion engine is fully operational, delivering daily structured infrastructure digests directly to the inbox.

With telemetry ingestion and data normalization successfully established, the foundation is set for Part 2: Mini Self-Healing Infrastructure & Dynamic AIOps Remediation.

Comments

Post a Comment

Popular posts from this blog

AdGuard Home DNS for Newbies - Part 3

-
Image
🧩 Part 3: Connecting Your Network to AdGuard Home Configure Your Router or Devices to Use AdGuard Home Why This Step Matters Now that AdGuard Home is installed and running, it’s time to make your network actually  use  it. Right now, your devices are probably still asking Google’s (8.8.8.8), Cloudflare’s (1.1.1.1), or your ISP’s DNS servers for website lookups — which means no ad-blocking, no caching, and no privacy benefits yet. Connecting your network to AdGuard Home means rerouting those DNS lookups through your own local resolver — one you control. This unlocks: Ad-blocking and tracking protection  for every device on your network. Faster lookups  thanks to local caching. Full visibility  into what domains your devices are contacting. And don’t worry — you can test all of this safely, one step at a time. Method 1: Set DNS via Your Router (Whole-Network Filtering) This is the recommended setup because it’s  set-and-forget . Every device that joins your ...
Read more

Suricata on Mikrotik(IDS+IPS) = Part 4 - Configuration of the IPS Part

-
Image
  Configuration Disclaimer:  this one again is only for  ubuntu 18.04  and login as  root  user to be sure Make sure you have a good understanding of the suricata rules before doing this This might cause some disconnection from the site or services you are visiting and even the internet if not properly configured. In the real environment it is best not to run as root user but for the sake of just testing it I opted to run as root configure the following rules on your mikrotik router: /ip firewall filter add action=drop chain=input comment="Block bad actors" src-address-list=Blocked /ip firewall filter add action=drop chain=forward comment="Drop any traffic going to bad actors" dst-address-list=Blocked create a user group with all policies checked except telnet create a user allowing your local addresses with the group you created earlier Lets Start ... 1.      Install PHP         apt install php -y 2.   Navigate...
Read more

DHCP for Dummies: How Your Devices Get Online Without You Lifting a Finger

-
Image
DHCP for Dummies: How Your Devices Get Online Without You Lifting a Finger Introduction Ever wondered how your phone, laptop, or smart TV connects to the internet without you having to do anything? One minute you’re tapping "Connect to Wi-Fi," and the next, boom—you’re online. That magic? It’s all thanks to DHCP. DHCP (Dynamic Host Configuration Protocol) is what hands out IP addresses to your devices, kind of like a hotel receptionist giving guests room keys. It’s automatic, invisible, and keeps everything running smoothly—until something breaks. In this guide, you’ll learn: ✔️ What DHCP is (in plain English). ✔️ How it works behind the scenes. ✔️ Why it’s better than manually assigning IP addresses. ✔️ How to fix common DHCP problems without calling IT. By the end, you’ll have a solid grip on DHCP—even if tech stuff usually makes your head hurt. Let’s go! --- What is DHCP? (The No-Nonsense Explanation) Okay, imagine you walk into a hotel. You don’t pick a room or set up a k...
Read more