Agentic AI Is the New SOAR: Everyone's Excited, Most Will Be Disappointed

I keep having conversations with folks that start something like “We’re thinking about dumping our Splunk logs into an AI agent and having it find the bad stuff.”

Every time, I nod. I understand the appeal. Truly. I also know how it’s most likely going to end.

This is the SOAR moment for AI agents. And if you’ve never heard that acronym, SOAR stands for Security Orchestration, Automation, and Response. If you lived through the first wave of SOAR adoption, or perhaps still are, you already know what’s coming.

The Pattern Should Sound Familiar

When SOAR platforms hit the market, every vendor made promises that would make any Security Operations Manager or CISO weep. Promises that sounded like “We will automate your playbooks, reduce analyst fatigue, and just let the platform handle the noise.” Sounds great. Where do I sign up?

For a handful of organizations with mature data pipelines, clean telemetry, and dedicated engineering resources, it worked. For most, it became an expensive integration project that consumed more time than it saved and required a full-time engineer just to keep it running.

The technology wasn’t the problem. The assumptions were.

Today, we are watching the same thing happen with agentic AI. The demos are compelling. Every AI influencer on YouTube and Reddit is screaming from the rafters about how their new AI thing can solve all of your problems. Security vendors are now integrating agentic AI into their products to “improve the lives of SOC analysts.” Their vendor pitch decks are immaculate. And organizations are setting themselves up for the same disappointment. Not because AI agents don’t work, or that even some of what is being sold isn’t true, but because agents and agent-based tools are being deployed in environments that still can’t get SOAR to work.

Why? Because their data is a mess, their standard operating procedures are five years out of date, and the one guy who knows how it all works is already overloaded, with both the SOC manager and the CISO breathing down their neck about the new shiny thing that is supposed to do a better job than SOAR, reduce risk, and save the world. By the way, is the SOAR platform working yet?

What “Agentic” Actually Means in the Real World

Here’s what the research shows.

RAND Corporation data puts the overall AI project failure rate at over 80 percent. 33.8 percent are abandoned before production. 28.4 percent complete but delivers no value. And 18.1 percent simply aren’t justifiable on cost alone. MIT reports that 95 percent of generative AI pilots fail to reach production. Gartner predicts that over 40 percent of agentic AI projects will be scrapped by 2027. Not because the models fail, but because organizations struggle to operationalize them.

These aren’t fringe statistics. They’re consistent across multiple independent research organizations tracking thousands of enterprise implementations.

The failure modes are also consistent.

Integration is harder than anyone wants to admit. An agent is only as useful as what it can actually touch. The leading causes of agent failure are poor memory management, brittle tool connections, broken input and output handling, dirty or poorly structured data, non-existent guardrails, and workflows that may work for a human but are not efficient for a machine. Things like not knowing when human verification should actually happen, ineffective token management, and agents constantly polling systems for updates rather than using event-driven architecture. More often than not, these systems are deployed with so much unnecessary complexity that an agent is given access to ungoverned rate limits, half-baked open-source middleware, tools they don’t need, and duplicate logic throughout multiple steps. You might as well give a new hire keys to your server room without any documentation, support, or ever thinking to check in on them. By Friday, your network will be offline.

Organizations will be more successful when they realize that demos rarely survive first contact. Technical teams can now rapidly build impressive, immersive proofs of concept using tools like Base44, OpenClaw, Lovable, and more. The demo works. Leadership gets excited. Then the system hits a real environment with real data volume, real API failures, and real edge cases. And now, what practitioners are calling “Innovation Theater” takes hold. Prototypes that work in sandboxes but lack standardization for integration with a live environment. This is the same issue that every platform has ever faced in every environment on the planet. Sales sells the shiny object. Product support turns it on. Engineering flails. Every. Time. Insert professional services pitch deck here.

Most enterprise AI agent initiatives don’t fail spectacularly. They fail through escalating operational costs, poor human oversight, inconsistent outcomes, and a gradual loss of trust. Once agents move beyond demos, knowledge workers become de facto validators, reviewing outputs without formal accountability or metrics. And the lack of education and experience in interpreting output, or in blindly trusting something that can be so confidently wrong, is killing projects and teams left and right. The result often isn’t a catastrophic collapse. It’s chronic inefficiency.

You think the agent is working. Instead, you’re just reviewing everything it does. At the end of the day, most teams have automated nothing. Instead, they have just added a step. And when the agent messes something up, someone has to go back and fix it to get the right result, likely taking more time than it would have taken to do the work without the agent in the first place.

Back to the Splunk Logs

The “dump logs into an agent” instinct isn’t stupid. It comes from a real problem. The volume of security telemetry has outpaced human capacity to review it. The question isn’t whether to use AI for this. Splunk is already investing heavily in solving this problem with AI, so there is obviously a use case here. The question is whether an unstructured firehose of log data is the right input for an agent architecture.

Let me help you. It isn’t.

What makes an agent effective is almost the opposite of what most people feed into them. Effective agents run on well-structured, contextually enriched data with explicit success criteria and clearly bounded action spaces. A Splunk index of raw logs is none of these things. It’s high-volume, heterogeneous, and often noisy data with no inherent priority signal. That’s exactly the kind of environment where agent reasoning quickly degrades, and false-positive rates spike - faster than gas prices when the Strait of Hormuz is closed.

Most teams still struggle to get consistently good output from a simple ChatGPT session. Expecting those same organizations to reliably solve complex detection problems with agents is, frankly, shortsighted. It isn’t that the goal is impossible in the long term; it’s that there’s a basic gap in understanding what these systems need to work well. The good news is that this gap is fixable with deliberate training and education, ideally before anyone starts wiring agents into production pipelines.

The organizations making AI work for security operations aren’t pointing agents at raw logs. They’re building enrichment pipelines that normalize and score data before it reaches the agent layer. They’re defining explicit task boundaries. Not asking agents to “find the badness,” but instead asking something like “given these ten ranked alerts with their enriched context, identify which two require immediate analyst review and explain why.” They’re building human checkpoints before anything irreversible happens.

That’s not a product you buy. That’s an engineering project you staff, scope, and maintain.

What’s Actually Working

AI is delivering very real value in very narrow, well-engineered, well-documented use cases. The problem is, most teams gloss over documenting use cases, user stories, and the functional and non-functional requirements that engineers succeed with. AI agents love that documentation, though. Ask me how I know.

A 2025 Cloud Security Alliance benchmark study found AI-assisted investigations were 45 percent faster for cloud security alerts and 61 percent faster for identity and access alerts. Those gains are real. But they came from tightly scoped integrations with clean data pipelines, not from pointing an agent at a raw data lake and waiting for intelligence to self-assemble. Skynet didn’t become self-aware by ingesting garbage.

Coding agents are the clearest success story overall. The gains are absolutely measurable, the feedback loop is fast, enabling rapid iteration, and the consequences of a bad output are generally recoverable. Thank you, Git. The most consistent pattern in mature agentic development is treating planning as an actual engineering phase. Success here looks like short bursts of research and design that narrow ambiguity before any files are touched. Every. Time.

Constrained, event-driven workflows with clear inputs and outputs are working, though they remain challenging due to large language models’ inherent non-determinism. Despite this, fields that are seeing the benefits include finance reconciliation, IT ticket triage, and specific enrichment steps in intelligence pipelines. Tasks where “done” is unambiguous and the blast radius of a failure is bounded. The secret to success is human-in-the-loop evaluation, and even having agents check other agents’ work against known good information or examples.

Anthropic published guidance on building effective agents that contained a candid admission. The most successful agents met their customers’ needs without using complex frameworks. They used simple chains, well-engineered inputs, and explicit human checkpoints. Take that into consideration. The company that stands to profit from you building more complex systems just told you that complexity is often the wrong call.

What isn’t working is the general-purpose autonomous agent dream that everyone is trying to sell. The one where you describe a goal, walk away, and come back to a finished product. That vision only exists in a demo dreamland. It does not yet exist in any production environment, and I am betting we are far off from that becoming a reality anytime soon.

The reason? Most people are bad at documentation, and good, successful agents need accurate documentation. They need context and understanding without having to infer every step of the reasoning process. Too much inference in high-stakes fields leads to catastrophic outcomes.

What This Means for the Security Workforce

The impact of AI on security operations and cyber intelligence isn’t coming. It’s already here. Some functions are being compressed. Others are expanding. Where you sit in that division today will determine your relevance over the next five years.

Tier 1 SOC work is already the most exposed. AI and machine learning systems can now automatically classify and respond to security alerts with 95% accuracy across common threat scenarios. Gartner predicted that by the end of 2025, half of all Tier 1 SOC positions would be eliminated or fundamentally transformed, and that prediction has at least in part come true. While it hasn’t happened at the speed Gartner predicted, the reality is that work within a SOC is fundamentally changing. If you are spending the majority of your shift manually checking indicators of compromise against VirusTotal, copy-pasting IP addresses into threat intelligence platforms, and writing the same incident summary you’ve written a hundred times, that work is being automated. And your window to reposition is narrowing.

CrowdStrike eliminated approximately 500 positions in 2025, with CEO George Kurtz stating directly that “AI is flattening our hiring curve.” The cuts were tied to AI-driven automation, making certain operational roles redundant. But the company emphasized it would continue hiring in product engineering and customer-facing roles. That’s not a contradiction. It’s a signal about where value is shifting.

The skills crisis is real, but it’s been misdiagnosed. Most coverage frames this as a headcount problem. It isn’t. 59 percent of organizations now report critical or significant skills gaps, up from 44 percent in 2024, and what’s missing isn’t more certified candidates. It’s analysts who can reason, communicate, adapt, and be willing to learn. AI is not replacing the analyst. It is raising the baseline of what every analyst is expected to bring to the table. Critical thinking and analytical curiosity are the currencies of the future, and there are too few people who are good at them.

64 percent of cybersecurity job listings now require AI, machine learning, or automation skills. That number was negligible three years ago. The job descriptions haven’t changed because the tools are smarter. They’ve changed because the tools require someone smarter to operate them well. If you are not spun up on AI, if you don’t know how it works, why it works the way it does, and how it can replace you, you have very little time to catch up.

The adversary is already there. This isn’t a hypothetical. In November 2025, Anthropic revealed that a Chinese state-sponsored group used AI to automate a significant portion of an intrusion attempt, performing reconnaissance, exploit generation, and lateral movement preparation, automating an estimated 80 to 90 percent of the attack workflow. While the scope of this event is contested, the fact remains that while the operation still required human oversight, AI tooling drastically reduced the attacker workload and accelerated timelines for effect. When the threat actor uses AI to operate at machine speed, defenders working with manual workflows are structurally at a disadvantage. Full stop.

For Cyber Threat Intelligence practitioners specifically, the picture is more nuanced. AI techniques now support the full intelligence lifecycle. From natural language processing-based automated crawlers that extract structured intelligence from open-source sources, to machine learning-based classifiers that enrich raw indicators with contextual information, to graph-based learning that infers relationships between indicators, campaigns, and threat actors. The analyst who understands how those pipelines work, how to build them, who can evaluate the quality of the enrichment, catch the attribution error the model made, and translate machine output into actionable finished intelligence, that person is not being replaced.

The meta-trend is becoming clear. **Cybersecurity is evolving into a data science problem, and the workforce is shifting accordingly. **That’s not pessimism. It’s a navigation coordinate, and you should be checking your map.

How to Stay Relevant

Certifications are table stakes. The practitioners who remain indispensable are the ones building skills that AI cannot replicate at scale.

Become the engineer who builds the pipeline, not just the analyst who reviews the output. The people who understand data enrichment architecture, who can design an event-driven collection system, who can evaluate and tune an agent’s performance, those roles are expanding, and the industry is starving for them. If you understand what your security information and event management platform is actually doing under the hood, you’re positioned to help design something better. If you only know how to use the interface, you’re quickly becoming replaceable. Period.

Develop hypothesis-driven thinking. Analysts are no longer primarily alert processors. They’re expected to reason about what an attacker is trying to do and think critically about context and patterns rather than following playbooks. This is a trainable skill. Practice it deliberately. Work through historical incident reports. Map adversary tactics, techniques, and procedures without the SIEM. Understand attacker behavior at a conceptual level that no tool can shortcut. AI inference is getting better, but it isn’t perfect. The strength of an analyst will always be their ability to think abstractly because AI, at least for now, “reasons" linearly.

Learn to evaluate AI output, not just consume it. The analyst who can identify when an AI enrichment is wrong, when the MITRE ATT&CK mapping is off, when attribution is thin, and when a confidence score doesn’t match the evidence is doing something the model cannot do for itself. That’s judgment. It’s built through years of handling cases manually. Don’t skip that foundation in a rush to operate entirely through AI tooling.

Learn to write and communicate well, and learn to understand the business and where your stakeholder sits within it. At the senior levels, the critical skill is translating intelligence from tactical and technical details into the strategic language of risk management. That capability is specifically what the threat intelligence career ladder rewards at the top of the profession. AI can produce a draft. It cannot replace a practitioner who understands the stakeholder, knows what decision they need to make, and knows how to frame uncertainty in a way that drives action rather than paralysis. This gap is closing quickly, though. With an effective agentic workflow, good documentation, and good context, even this role is short-lived for simpler problems at some companies.

The analysts who are thriving are the ones who use AI aggressively where speed is cheap. Tasks like initial enrichment, summary generation, and pattern surfacing. They reserve rigorous human judgment for the places where mistakes are expensive. The most valuable security analysts going forward will be those who combine technical depth with critical thinking that goes beyond pattern matching.

The analysts who will lose their jobs aren’t the ones “replaced by AI.” They’re the ones who refuse to work with it or refuse to think independently. They refuse to dig deep, to be curious, to challenge the output. If you’re an analyst who just follows a standard operating procedure or template, optimizes for maximum repeatability, and treats the role like a punch-in, punch-out eight-hour shift, your days are numbered.

Get excited. Build something. Get curious. Think about the problem. Use AI as a tool, not a crutch. If you know, deep down, that the answer it’s giving you is wrong, don’t let it make the decision for you.

The Takeaway

Agentic AI is not vaporware. There are real use cases with the potential to reshape how we work completely. The productivity gains are very achievable. But the path to them runs directly through boring, unglamorous engineering and documentation work with clean data pipelines, explicit task scoping, integration resilience, evaluation frameworks, and an honest assessment of where human judgment cannot be removed from the loop.

The organizations that figure this out will have a genuine operational advantage. The ones chasing the demo will spend the next 18 months debugging in production and wondering why the vendor’s case study doesn’t resemble their reality. Unfortunately, that will be the case for most organizations. Job security for consultants, I suppose.

For practitioners in security operations and intelligence, the threat is real, but it’s targeted. Routine work is being automated. Strategic, judgment-intensive, adversarially-aware work is not, and demand for that work is growing. The question isn’t whether to engage with these tools. The question is whether you’re building the skills to operate at the next level.

We’ve seen this movie. We know how it ends.

Govern yourselves accordingly.