Azure AI

How Agentic Software Development is Rewriting the Rules for Databases

Content type Blog Post

Author Thiago Menezes

Publication Date 18 Sep, 2026

Reading Time 3 minutes

Contents

Introduction
Evolutionary Development Cycles
Cost and Elasticity Imperatives
Openness as Operational Must-Have
Implications for Data Architects
References
About the author

Introduction
Evolutionary Development Cycles
Cost and Elasticity Imperatives
Openness as Operational Must-Have
Implications for Data Architects
References
About the author

Introduction

I’ve been deep in data architecture for nearly two decades, and watching agentic AI take over software/data development feels like the biggest shift since the cloud boom. A recent Databricks blog post nails it: agents are exploding the volume of databases created for rapid experimentation, demanding infrastructure that branches instantly, scales to zero, and stays open (that’s amazing!). In my experience building lakehouses on Databricks and other platforms, this isn’t hype, it’s telemetry reality, with agents already creating 4x more databases than humans on Lakebase. [1]

Evolutionary Development Cycles

Traditional dev was linear, costly, and human-paced. Agents flip that to evolutionary loops: generate app variants, test schemas, evaluate, iterate, in seconds to hours. Databricks reports average projects hit 10 branches, some nesting over 500 deep, slashing cycles 100x-1000x. I’ve seen this in my own pipelines, where Databricks Unity Catalog lets agents spin Delta tables without friction, mirroring git for data. [1]

Gartner predicts 40% of enterprise apps will embed AI agents by end-2026, up from under 5% now. That’s an 8x surge, forcing data stacks to evolve or get left behind. [2]

Cost and Elasticity Imperatives

Agent-born apps are often ephemeral prototypes or narrow tools, running seconds before discard. Half last under 10 seconds on Lakebase, per Databricks data. Legacy databases with fixed overhead kill economics here; you need true serverless that scales compute to zero idle cost.

AWS echoes this, pushing databases like Aurora for agentic grounding with vector search and stateful memory. In my Azure work, we’ve used similar elastic patterns for bursty ML workloads, avoiding baseline fees on experimental agent outputs. [3]

Openness as Operational Must-Have

Agents thrive on open-source training data, excelling with Postgres APIs over proprietary ones. Lakebase stores in open Postgres pages on object storage, letting agents query natively without vendor lock. [1]

Forbes notes agentic AI crosses org boundaries, amplifying this need for interoperable stacks. I’ve learned that firsthand migrating Hadoop to Databricks, where openness cut integration pains 50%. [4]

Implications for Data Architects

This agentic wave means rethinking medallion architectures for branching lakes. Gartner warns 40% of agentic projects may flop by 2027 without robust infra, but successes like AWS DevOps Agent or Azure Pipelines Agents for auto-optimizing databases show the upside. As a mentor, I tell teams: build for agents now, or refactor later. [3][4]

Exciting times, but demands ethical governance too. We can talk about it in a future article!

What excites you most about agentic dev?