How Java Is Evolving for Enterprise AI: A Talk by Fábio Velloso

At Avalia Tech Forum 2026 in São Paulo, Fábio Velloso explored how Java is evolving to support AI in enterprise applications, from model integration to the runtime. His talk took a technical look at the frameworks and platform capabilities available to developers, with practical examples of what happens when AI becomes part of a working application. This article summarises the main points.

Fábio outlined the engineering challenges of integrating a probabilistic AI component into enterprise software.

Bringing AI into an enterprise application

Fábio opened with an example of an unexpected response from a language model. It illustrated a challenge that runs through the talk: developers are introducing a probabilistic component into software, where the same input can produce different answers.

He showed how LangChain4j and Spring AI help integrate language models into Java applications. Through these frameworks, AI becomes a dependency that developers can configure and connect to the application’s existing infrastructure.

That integration gives teams access to established mechanisms for handling failures.

Fábio demonstrated how timeouts and circuit breakers can prevent a slow or unavailable model from disrupting the application. A fallback can redirect a request to a local model when the remote service fails, allowing the application to continue operating.

Understanding what happens in production

Testing remains necessary, but Fábio stressed the need to observe AI behaviour after deployment. Real users introduce inputs that developers may not have anticipated, while response times and model outputs can vary.

Using monitoring dashboards from a classification application, he showed how teams can track model latency and token consumption alongside metrics specific to their use case. In his example, most of the response time came from the model call and the network. Optimising the Java code would therefore have had little effect on the overall wait.

Measurements from Fábio’s application showed that most response time came from the model and network.

The measurements also revealed a tradeoff between cost and quality. When the team shortened a prompt by roughly half, nearly half of the responses changed, and the results became worse. The example showed why a change intended to reduce consumption needs to be evaluated against the quality of the application’s answers.

Fábio also described using smaller models for classification tasks, supported by processing in Java. His example illustrated how model selection depends on the work the application needs to perform.

From integration to runtime performance

The final part of the talk examined changes in the Java platform itself. Fábio explained how virtual threads support applications that spend substantial time waiting for external services, including AI models.

Virtual threads allow Java applications to handle more concurrent operations while waiting for external services.

He also introduced structured concurrency as a way to organise related operations and handle their completion or failure together.

He then covered the Foreign Function & Memory API and the Vector API, explaining their role in connecting Java to native libraries and supporting specialised computation. A vector similarity example illustrated the potential performance gains, while his discussion of Project Babylon looked ahead to work on running Java-defined computations in other execution environments.

Across the talk, Fábio connected AI integration to the engineering work needed to operate it reliably. His examples showed how Java’s existing ecosystem, together with developments in the platform, can help teams understand and manage AI inside enterprise software.