Most teams have the logs. Almost none of them have detection.
Walk into a security operations review at almost any mid-sized bank or telecom and you will hear the same reassurance: "We log everything." Storage is cheap, agents are deployed, the SIEM licence is paid up, dashboards glow on the wall. Then you ask the question that actually matters. When was the last time one of those logs turned into an alert that a human acted on before a customer or a regulator noticed the problem? The room usually goes quiet.
We have run enough SOC assessments across banking, telecom, and critical national infrastructure to say this plainly: collecting telemetry and detecting attacks are two different disciplines, and the gap between them is where most breaches live. Observability, a term the site-reliability crowd has used for years, is finally being borrowed by security teams. That is a good thing. But borrowing the word without the engineering behind it just produces very expensive log archives.
This article is about closing that gap: what observability means when the goal is catching an intruder rather than debugging a slow checkout page, which signals actually earn their storage cost, and how a pile of logs becomes a detection program that holds up under both an incident and an audit.
Logs, metrics, and traces, seen through a security lens
The observability world talks about three pillars. Each one maps to something a defender needs, and each one is misused in a predictable way.
Logs are the record of what happened
A log is a timestamped statement that an event occurred: a user authenticated, a process spawned, a firewall dropped a packet, an S3 bucket policy changed. For security this is the primary evidence base. When you reconstruct an intrusion after the fact, you are reading logs. The failure mode is volume without meaning. A team ships forty log sources into the SIEM, nobody normalises the fields, and a single logon event looks different coming from a domain controller, a VPN concentrator, and a cloud identity provider. You end up unable to write one rule that catches the same behaviour across all three.
Metrics are the shape of normal
Metrics are numbers measured over time: authentication failures per minute, outbound bytes per host, API calls per key. SRE teams use them for capacity and latency. Security teams should use them for baselines. A brute-force attempt, a data-staging operation before exfiltration, or a compromised service account hammering an API all show up as a metric moving off its normal band long before anyone reads the underlying logs. Most SOCs never build these baselines, so they miss the cheapest early-warning signal available to them.
Traces show how a request moved through the system
A trace follows a single transaction across the services it touches. This is the least-used pillar in security and, in modern distributed and micro-service estates, increasingly the most valuable. When an attacker abuses a legitimate API path, chains internal service calls, or pivots through a payment flow, the individual logs on each service look benign. The trace that stitches them together is what reveals the abuse. If your application teams already run OpenTelemetry for performance, that same data is a security asset you are probably ignoring.
Why "more logs" is not "more security"
There is a stubborn belief that ingesting more data makes an organisation safer. It does not. It makes the SIEM bill larger and the analyst's day longer. What matters is coverage of the right sources and the ability to reason across them.
The useful way to think about coverage is to start from attacker behaviour, not from whatever happens to produce a log file. The MITRE ATT&CK framework catalogues the techniques adversaries actually use and, for each one, the data sources that would reveal it. Work backwards from the techniques that threaten your business and you get a shopping list of telemetry that is defensible to a board and to an auditor. Everything outside that list is a candidate for sampling, shorter retention, or cold storage, not for a hot, indexed, alert-generating pipeline.
Here is the coverage most organisations actually need, and the common gap we find against it.
Telemetry domainWhat it catchesWhere teams fall short
Identity and authentication
Credential abuse, MFA fatigue, impossible-travel logons, privilege escalation
Cloud IdP logs collected, but on-prem AD and VPN auth left out, so the attacker's path is invisible
Endpoint (EDR/process)
Malware execution, living-off-the-land binaries, lateral movement
Coverage on laptops but not on servers, jump hosts, or OT gateways
Network and DNS
Command-and-control callbacks, data exfiltration, tunnelling
Perimeter firewall logs only; east-west and DNS traffic unmonitored
Cloud control plane
Policy changes, key creation, resource tampering, disabled logging
Data-plane noise ingested while the audit trail of who changed what is missed
Application and API
Business-logic abuse, injection, token theft, scraping
Treated as an app-team concern, never routed to the SOC
SaaS and email
Phishing, mailbox rules, OAuth grant abuse, data sharing
No API-based logging from the platforms staff actually work in
Notice a theme. The gaps are rarely about a missing product. They are about partial deployment, unmonitored internal traffic, and telemetry that stops at an organisational boundary because "that belongs to the application team." Attackers do not respect those boundaries, and neither should your detection strategy.
The pipeline that turns telemetry into detection
A working security-observability capability is a pipeline, and each stage does real work. Skip a stage and the whole thing degrades into search-after-the-fact.
Collection. Agents, syslog, cloud-native log exports, and OpenTelemetry collectors pull events from every domain in the table above. The engineering goal here is reliability, not completeness for its own sake. A source that silently stops sending is worse than one you never had, because the dashboard still looks green.
Normalisation. Raw events get mapped to a common schema so a "user logged in" from three vendors becomes one comparable field set. This is unglamorous and it is where most in-house SIEM projects stall. Without it, every detection rule has to be rewritten per source, and rule maintenance quietly becomes a full-time job nobody was hired for.
Enrichment. Events gain context: threat-intelligence matches on an IP, asset criticality, data classification, whether the user is a privileged account. Enrichment is what lets an analyst triage in seconds instead of minutes, and at real alert volumes that difference decides whether the queue is ever cleared.
Detection. Rules, correlation, and analytics run against the enriched stream. Good programs treat detections as code: version-controlled, peer-reviewed, tested against known-bad samples, and written in a portable format such as Sigma so a rule is not locked to one vendor. This is detection engineering, and it is the single highest-return investment a SOC can make.
Response. A detection that does not drive an action is a report, not a control. The last stage wires alerts into a runbook, a case, an on-call rotation, and where it is safe, an automated containment step. The measure of the whole pipeline is not how much it ingests. It is time to detect and time to respond.
Detection engineering, not dashboard collecting
The shift that separates a mature SOC from an expensive one is treating detections as an engineering product with a lifecycle. New detections are proposed against a threat model, built, tested, tuned, and eventually retired when they no longer earn their keep. False positives are treated as bugs to be fixed, not background noise to be endured, because every needless alert erodes the analyst's trust in the queue.
Concretely, a program worth the name will do a few things consistently. It writes detections in a shareable rule language rather than clicking them into a single console. It keeps those rules in version control with a review step, the same way you would treat production code. It maps its detection library to ATT&CK so leadership can see, honestly, which techniques are covered and which are blind spots. And it runs adversary-emulation or purple-team exercises to confirm the rules fire against real behaviour, because a rule that has never caught anything is a rule you cannot trust.
None of this requires a specific brand of SIEM. We have built effective detection pipelines on Microsoft Sentinel, Elastic, Splunk, and open tooling such as Wazuh and the OpenSearch stack, depending on what a client already owned and what their budget and data-residency constraints allowed. The platform is a smaller decision than most vendors would have you believe. The engineering discipline around it is the whole game.
Logging and monitoring are also a compliance requirement
For regulated businesses in the UAE, security observability is not only good practice. It is written into the standards you are measured against, and auditors ask for evidence.
ISO/IEC 27001:2022 makes this explicit. Control A.8.15 requires that logs recording activities, exceptions, and security events be produced, kept, and protected. Control A.8.16 requires that networks, systems, and applications be monitored for anomalous behaviour and that appropriate action follow. PCI-DSS Requirement 10 goes further for anyone handling cardholder data, mandating that access to systems and data be logged, that logs be reviewed, and that they be retained for at least a year with ninety days immediately available. The NIST Cybersecurity Framework devotes its entire Detect function to exactly this, with categories for continuous monitoring and for analysing detected events. The CIS Controls place audit-log management and malware defences among the foundational safeguards.
The pattern across all of them is the same, and it is worth stating directly: it is not enough to collect. You must protect the logs from tampering, retain them for a defined period, review them, and demonstrate that monitoring leads to action. A SIEM full of unread data satisfies none of those requirements, and a good auditor will find that out in one interview.
Retention is the detail that trips teams up. Detection needs recent, hot data. Investigation and compliance need older data that is intact and searchable. Design the tiers deliberately: hot storage for the detection window, warm for investigation, cold for the retention obligation, with integrity protection so the archive stands up as evidence if it ever reaches a regulator or a court.
The cloud changes where the interesting signals live
When workloads move to AWS, Azure, or Google Cloud, the most valuable security telemetry shifts to the control plane, the audit trail of management actions. AWS CloudTrail, Azure Activity and Entra sign-in logs, and Google Cloud Audit Logs record who did what to the environment itself: who created an access key, changed a security group, assumed a role, or, tellingly, turned logging off. Attackers who reach a cloud environment work through these APIs, so this is where their fingerprints appear.
Two failures recur in cloud SOC work. First, teams drown in high-volume data-plane logs while the control-plane audit trail, which is smaller and far more revealing, is never centralised. Second, the shared-responsibility line is misread: the provider secures the infrastructure, but detecting misuse inside your account is your job, and the provider will not do it for you. A cloud migration is the right moment to design this in, rather than discovering the gap during an incident eighteen months later.
A short readiness check
Before the next audit or the next incident, these are the questions we ask first. If any answer is uncomfortable, that is where to start.
- Can you list your log sources from memory, and does that list cover identity, endpoint, network, cloud control plane, and your critical applications?
- If a key source stopped sending data tonight, would anyone know by morning?
- Are your detections written down, version-controlled, and mapped to the techniques that actually threaten your business?
- When an alert fires at 3 a.m., is there a runbook and an owner, or does it wait for the day shift?
- Can you produce, on request, tamper-protected logs for the retention period your regulator requires?
- Have you tested your detections against real attacker behaviour in the last year, or are you trusting rules that have never caught anything?
How Aydahwa Enterprise can help
Aydahwa Enterprise builds and matures security-observability capabilities for organisations in banking, telecom, and critical national infrastructure across the UAE and the wider region. Our work is led by hands-on architects, not a script, and it is grounded in the standards regulators actually apply, including ISO/IEC 27001, PCI-DSS, the NIST Cybersecurity Framework, and CIS Benchmarks, backed by certifications up to Microsoft Cybersecurity Architect Expert.
In practice that means assessing your current telemetry coverage against attacker behaviour, closing the log-source gaps that leave you blind, and standing up a detection-engineering practice your team can own rather than a dashboard they cannot maintain. We design retention and integrity tiers that satisfy both your analysts and your auditors, and we do it on the platform you already run wherever that is sensible.
If you want to know where you stand, start with our free cybersecurity self-assessment or work through the cybersecurity readiness checklist. To go deeper, our cybersecurity services cover SOC design, SIEM and detection engineering, and compliance, while our cloud services team builds this monitoring into migrations from the start, and our IT support practice keeps it running. When you are ready to talk specifics, get in touch and we will start with your real environment, not a template.
