Australian organisations are rapidly adopting artificial intelligence, but new research has identified a governance gap beneath that adoption, with few organisations able to confidently explain what their AI systems are doing, who is responsible when something goes wrong, or how quickly problems can be identified and resolved.

For boards that have approved AI initiatives over the past year, the findings point to an immediate oversight issue rather than a future risk.

The 2026 Ecosystm Developer Experience in Australia Study, commissioned by Avocado and Dynatrace, surveyed 154 technology leaders and engineering practitioners working at Australian organisations with more than 500 employees.

The research is based specifically on Australian organisations rather than broader regional data that combines Australia with larger Asia-Pacific markets.

Boards face an AI accountability question

Only 9% of organisations surveyed reported having end-to-end visibility into how their AI applications behave in production.

That leaves 91% operating with only partial visibility, raising questions about whether organisations can adequately explain AI behaviour when challenged by a board, regulator or customer.

Those questions include why an AI system produced a particular response, what data it relied on, whether sensitive information was exposed and what financial or operational impact resulted from the decision.

Security and guardrail violations are already among the most closely monitored AI behaviours, cited by 47% of respondents and ranking second only to retrieval performance.

The findings suggest organisations are increasingly aware of AI-related risks, but detecting a potential violation is different from having sufficient visibility to explain, contain and remediate it.

Zana Stojanovski, General Manager of Business Solutions at Avocado, said the 9% figure should concern organisations currently approving AI initiatives.

“That 9% figure is a concern for any board approving an AI initiative this year,” Stojanovski said.

“It isn’t an engineering problem but rather a governance gap. You need to understand how your AI solution arrives at its answers, so that you can explain it to anyone that asks.”

“Monitoring violations of your governance is a very different proposition to understanding what happened.”

AI maturity falls as incidents progress

The research found AI operations maturity declines as incidents move from detection through diagnosis and resolution.

Around 25% of organisations said they could detect AI-related issues quickly or proactively.

That fell to 23% for organisations capable of diagnosing root causes in near real time, while only 17% said they could resolve issues quickly or through automation.

The findings suggest many Australian organisations may be able to identify that an AI system has failed, but fewer can explain the cause quickly enough to limit potential regulatory, financial or reputational consequences.

Stojanovski said detection alone was insufficient if organisations could not rapidly diagnose and resolve the underlying problem.

“Detection without fast diagnosis and resolution isn’t real accountability, it’s just an early warning that a problem is already underway,” Stojanovski said.

“Boards should be asking not ‘can we tell something’s wrong,’ but ‘how long between something going wrong and us being able to explain and fix it,’ because that’s the window where cost, damage and reputational risk actually accumulate.”

AI risk ownership remains fragmented

Responsibility for AI application performance is also fragmented across many organisations, creating uncertainty around who ultimately owns the risk when an AI system fails.

Traditional operations functions, including site reliability engineering teams, continue to handle AI monitoring within some businesses, while specialist functions including MLOps, AI engineering and data science teams are increasingly taking responsibility elsewhere.

The study suggests genuinely cross-functional ownership remains uncommon.

Such an approach would require model behaviour, data quality, application logic, infrastructure and security to be brought together under a coordinated accountability framework rather than treated as separate technical functions.

Observability data is not yet shaping development

The research also found that collecting telemetry is no longer the main challenge for large Australian organisations. Instead, the issue is whether organisations are making effective use of the information they already collect.

Observability is increasingly being introduced earlier in the software development lifecycle, with organisations looking to use production data to inform software design, development and testing rather than relying on it primarily for troubleshooting after deployment.

However, only 27% of organisations surveyed said production telemetry had significantly changed how software decisions were made from the beginning of the development process.

For many organisations, telemetry continues to be used reactively after applications have already entered production.

Developers also reported difficulty accessing the information they need.

Around 53% of respondents said easier access to logs, traces and runtime data during debugging would improve their daily development experience.

The finding suggests organisations may already possess substantial operational data, but that information is not always reaching engineers at the point when it could influence development decisions.

“Most organisations think they’ve solved observability once the dashboards exist,” Stojanovski said.

“The real test is whether that data changes a decision before something breaks, in design, in code review, in testing, not just after an incident, when it’s too late to prevent the cost.”

“That’s the gap between having telemetry and actually operating on it.”

Governance and observability need to develop together

The report suggests AI governance policies alone are unlikely to address the visibility problem.

Instead, organisations may need clearly defined ownership and accountability structures alongside AI-specific observability systems capable of providing the evidence those governance frameworks require.

Traditional application monitoring platforms were not originally designed around AI-specific failure modes including model drift, hallucinated responses, unexpected model behaviour or sudden increases in inference costs.

Without visibility into those behaviours, organisations may struggle to answer questions from regulators, customers or boards even where formal AI governance policies exist.

“Governance without observability is a policy with no evidence behind it,” Stojanovski said.

“Observability without governance is data nobody’s accountable for acting on.

“Organisations closing this gap are treating the two as one initiative, not two separate line items.”

Boards being urged to clarify ownership

The findings raise another question for boards and technology executives: who is responsible for ensuring an organisation can explain an AI incident before one occurs?

Technical capabilities including explainability, monitoring and faster incident response can be developed, but the report indicates organisational ownership remains less clearly defined.

Darian Bird, Principal Advisor at Ecosystm and author of the report, said observability needed to become part of the engineering process much earlier.

“Observability needs to move upstream,” Bird said.

“Engineering teams need production context when they are designing, building and testing software, not after an issue reaches production.”

“As AI increases the complexity of development and operations, that visibility becomes an engineering capability as well as a governance requirement.”

Stojanovski said Australian boards were increasingly discussing AI strategy and return on investment, but operational accountability needed greater attention.

“We’re seeing Australian boards ask good questions about AI strategy and ROI, and far fewer asking whether their organisation could actually explain an AI failure if a regulator or customer demanded an answer tomorrow,” Stojanovski said.

“That’s the question this data should put on the agenda.”

Research examines broader software engineering challenges

The Ecosystm research also examines differences between how technology leaders and developers assess their organisation’s AI operations maturity.

It explores how production telemetry is being used to inform software design and testing before problems reach production, as well as barriers slowing platform engineering adoption within Australian organisations.

The report also considers the role observability gaps play in platform engineering and provides recommendations for implementing AI-specific instrumentation and clearer operational ownership as enterprise AI adoption increases.

Avocado and Dynatrace said their work together is focused on reducing complexity across Australian engineering environments, improving operational resilience and providing developers with greater visibility into the applications and AI systems they operate.