Key Takeaways
|
Customers want to jump straight into AIOps and self-healing because the technology is exciting and the operational pain is real.
Teams are dealing with on-call fatigue, alert storms, and a lot of manual work. So it’s easy to start treating autonomous operations as a tooling problem: wire in some AI, add automation, and “the pain goes away.”
But AI only augments what’s currently in place. If the monitoring and observability are weak, the alerts won’t be trustworthy.
I’ve watched organizations spend six months building Ansible runbooks around alerts nobody should’ve trusted in the first place. The problem was still there. They just automated the wrong decisions faster.
Before AIOps or self-healing can work safely at scale, teams need the proper foundation: end-to-end visibility across the application and infrastructure, enough context to understand the business impact, and confidence in the response they’re automating.
The Observability Maturity Model helps teams assess each application’s readiness, identify missing capabilities, and determine what they need to build before adding more intelligence or automation.
If your logs and metrics are clean, AI can help teams respond faster. If they’re sparse, noisy, or incomplete, it’ll amplify those weaknesses.
Garbage in, garbage out isn’t new. The difference now is that the garbage can execute at 2 a.m. without a human verifying it.
I’ve seen customers run playbooks from false-positive alerts and automate responses based on phantom signals. A server restarts in the middle of the night, and leadership asks why. The team can’t explain the decision because they never had enough information to make the call in the first place.
That’s the risk of moving too quickly.
Before you automate the response, you have to trust the signal and understand what’s causing the problem.
One of the first things I ask is whether they could get a critical application back up within an hour.
Most teams say, “No.”
They have to jump between three or four tools and correlate the logs, infrastructure, and user experience. They may have a theory, but not a true root cause.
The team should be able to pinpoint the root cause without switching platforms, validate it within minutes, and have very few manual runbooks because the remaining work is repeatable.
When teams reach that point, we’ve seen mean time to recovery drop by roughly 75% through faster root cause analysis and automation.
If they still depend on manual investigation and runbooks, autonomous action is premature.
The Observability Maturity Model is a 5-stage framework for assessing whether an application has the foundation, visibility, intelligence, automation, and feedback required for autonomous operations.
It isn’t an end-all, be-all checklist.
Levels blend depending on the application, workload, and team. For example, an e-commerce platform might be at level four while an internal reporting application is at level two.
The framework helps teams understand where each application is today and what needs to be in place before they add more intelligence or automation.
It starts with your metrics, logs, and traces in one place.
The team must trust the data coming into the system. If it’s incomplete or unreliable, everything built on top of it will be questionable.
The second level moves beyond application performance and infrastructure monitoring into true observability.
Now you can see how the infrastructure and applications behave together, understand the dependencies, and find the root cause through end-to-end visibility.
I compare it to navigation.
Paper maps showed you the route, but not why traffic stopped. Basic GPS added turn-by-turn directions and red, yellow, and green indicators. That’s closer to traditional monitoring. You know there’s a problem, but you don’t have the full context.
Modern navigation applications can tell you a truck flipped over, three lanes are blocked, and give you another route. Observability does the same thing for the business. You can say the application is down, explain why, and understand what to do next.
Once you trust the observability, you can add intelligence.
That includes anomaly detection, alert correlation, identifying the cause of an issue, and recognizing patterns across the environment.
At the automation level, findings start triggering actions.
That might mean running a playbook, rolling back a change, updating a configuration, or scaling the infrastructure. But executing the action doesn’t mean the process is complete.
I once watched an engineer spend almost two hours every Friday closing ServiceNow tickets. The observability platform found the issue and the automation executed the change, but each action opened a ticket that remained open after the problem was resolved.
The business ran, but the closed loop wasn’t there.
At the autonomous level, the system detects the issue, diagnoses it, takes action, verifies the result, and feeds that result back into the observability and service-management systems.
It doesn’t run a playbook and assume everything is fine. It checks whether the metric stabilized, the anomaly disappeared, or the service returned to the expected state.
That feedback also allows the system to recognize patterns over time.
For example, if CPU and memory usage spike every Monday morning, it can add capacity for those hours and scale it back down afterward.
Each of the five maturity stages has its own crawl, walk, and run progression:
• Crawl: The work is largely manual while the team learns the system and builds trust in the data.
• Walk: Teams introduce intelligence or automation but continue validating alerts, outcomes, and fixes.
• Run: Intelligence and automation operate with verification and a closed feedback loop, so the team can confirm the problem was actually resolved.
An application can be run in observability but only crawl in automation. Different applications, workloads, and teams also progress at their own pace.
The goal is to keep advancing each capability rather than treating maturity as one organization-wide score or a journey that ends.
Automated remediation is one shot. You detect the issue, open a ticket, run a trusted playbook, and the symptom stops.
Self-healing closes the loop. The system diagnoses the issue, takes action, verifies the result, provides feedback, and learns from what happened.
It doesn’t just assume the runbook fixed the problem.
I’ve seen teams automate the immediate fix and miss the underlying problem. For example, in one environment, a latency spike triggered a ticket. Ansible restarted the service at 2 a.m. The on-call engineers stopped getting woken up, but the service kept crashing every night. Nobody asked why.
A self-healing process would recognize the pattern and say, “I keep restarting this service every night at 2 a.m. You may want to look into this.”
After investigating, we found backups and database jobs running at the wrong times. The automation had stopped the calls, but it hadn’t fixed the process.
Successfully automating the wrong thing can be worse than doing it manually because it hides what’s really happening.
When you build that loop correctly, the impact goes beyond fewer overnight calls. We’ve seen the number of war rooms required for incident response fall by roughly 60% as more issues are diagnosed and resolved without coordinated manual intervention.
A closed-loop observability and automation architecture usually involves several platforms.
An observability platform like Dynatrace detects and diagnoses the issue, including the cause and business impact. ServiceNow or Jira opens the ticket, connects it to deployment changes, and provides the runbook context. Ansible executes the rollback, configuration change, or scaling action.
Then the observability platform verifies the result:
If the same issue appears again a few days later, the feedback tells you the response wasn’t permanent.
Self-healing might first identify the problem and recommend a fix for human review, or it may eventually take the action itself.
I’d start with a human in the loop, especially in production. In QA or pre-production, let the agent come back and say, “I think this is the problem. You should fix this.” Verify the diagnosis and the response once or twice. Once the team is comfortable, automate that action into the process the next time it happens.
That’s how you move from a human in the loop toward what we call a human at the helm.
Apply the model application by application, starting with the systems whose failure creates the greatest business impact.
When deciding which applications to advance first, ask: “What do you get measured for, hired for, or fired for?”
In one environment, the answer was our partner portal, where partners bought and renewed licenses. We eventually learned that between $300 million and $350 million moved through it every quarter. Even leadership didn’t know how much money depended on those systems.
For each application, assess three things:
1. Business criticality: What hurts when it goes down?
2. Observability readiness: Can the team see the system clearly and identify the root cause?
3. Automation opportunity: What does the team still do manually? If operations runs the same five-step runbook every week, that’s a candidate for automation.
Start with work the team understands. Don’t build ambitious automation around applications you barely understand. Start small, build trust, and then expand. I tell teams to “automate yourself out of your current role and into your next one.”
Before automating, document the baseline. Track whether mean time to recovery drops, on-call interruptions decline, and the same issues stop recurring. Then connect those improvements to downtime, customer impact, and revenue.
The ultimate measure is whether your best engineers and architects are still fighting the same fires or building what the business needs next. We’ve seen automation and faster root cause analysis improve production update and DevOps delivery timelines by around 40%, giving those teams more capacity to move the business forward.
The maturity model gives teams a way to make that progression deliberately, without automating decisions before they can trust the underlying signals and responses.
Ready to build toward autonomous operations?
Connect with Arctiq to discuss bringing observability and automation together with Dynatrace and Red Hat Ansible Automation Platform.