The State & Future of AI Agents
What works in production, challenges & future predictions
A year ago I wrote The State of AI Agents, thinking about agents and the biggest topics like reliability. The market was very early, many products were just about cool demos, and a lot of agents were unreliable.
A lot happened in a year, from LLMs using computers to growth in generative UI apps and agent-powered automation. This blog post is an update of my last-year AI agents summary.
1. Agents are NOT the products
As we anticipated, many agents have become an “invisible” feature, seen more as a feature than the product itself.
We are shifting from agentic projects like AutoGPT to more complex products where agentic properties may not even be obvious to the end user. People still ask about an example of an agent that is really useful, without realizing they might already be using them within products like Perplexity. It’s not surprising that end users don’t care much about agents, but whether the app works, is reliable, and brings value.
2. There are established use cases of agents in production
In my post from a year ago, I said: “Today’s most common use cases are coding, personal daily tasks, or research.“
This is still true, but I also see a new pattern in a few use cases that are actually used in production and already have good (and growing) amounts of paying customers, even on the enterprise level.
A) AI-powered workflow automation
B) Spreadsheets where cells are powered by AI
Maybe also Clay
C) Data analysis agents
D) Research and web scraping agents
(Note that these categories overlap a lot.)
Why are they successful? I think these use cases are low-hanging fruits, as opposed to autonomous use cases like AI software engineers or “life copilots” (e.g., agents planning a whole vacation for you). Of course, these are being developed too, but they might take more time to achieve reliability.
What a lot of the products I listed have in common is that they let agents operate only in parts of their product. They limit its freedom and build software around it. Maybe the winning strategy at least for now is to let the agent control only one part of the product. I talked about this with Max, the founder of Gumloop.
3. Autonomous AI software engineers are popular again
Agents are going in cycles and waves. There is sometimes a big hype like there was around Devin, the autonomous software engineer who was claimed to code complex programs and replace developers. Then people started hating Devin, saying that it is overrated.
A few weeks later, Anthropic launched a demo of LLM-powered computer use, and that inspired other developers. Are very autonomous LLM agents back? Developers are now letting LLMs work in their own virtual computer and do very autonomous tasks, like navigating to browser, looking for specific sites, images, or videos, creating new files, or even booking a flight.
E2B launched an open-source sandbox environment that can be used with any LLM to let it use a computer in an isolated way.
Most computer use examples are now very experimental. However, I think no matter whether a project is “over-hyped” at the beginning, the most important is that it sparks the interest of developers, and makes the community built on top of it.
4. Agents are built with multiple LLMs combined
When asking developers about building with LLMs, I discovered they regularly combine different LLMs in one agentic app. Faster models like Llama are popular for basic tasks like summarization, while more powerful ones like Claude Opus or GPT-4 are useful for complex writing.
This isn't just about preference—each model has unique strengths, leading some developers to use even dozens of LLMs in a single workflow to maximize performance.
Evaluating and benchmarking LLMs is a whole different topic, but the point is, that there are significant differences in LLMs, and it’s becoming a norm to combine them in production.
Read more about how you can test differences between different LLMs in a fun way:
5. Generative UI use cases are growing
The challenge of agents is how to let humans supervise the agents’ decisions. With generative UI, agents show their output directly through interactive interfaces - letting humans see the immediate output. Every AI engineer probably knows v0 by Vercel or the concept of artifacts".
I think the natural next step is being able to make changes directly in the produced code, and creating something like “code editor 2.0”.
6. Big companies are doubling down on agents
We can’t keep saying “the market is too early” anymore because bigger players are noticing things. Just a few pieces of news:
Salesforce’s pivot to agents - they announced an AI agent platform called Agentforce. Marc Benioff, the CEO, said “The next wave of AI will be all about agents”
Microsoft launched Windows Agent Arena is an environment for evaluating open-source AI agents. It’s targeting agents that can act on your own computer.
Replit has launched an AI agent capable of building entire applications from scratch. Community reactions have been great so far. Replit is not exactly a big corporation, but still, millions of developers are building with it. They recently announced integration with LangGraph too.
LangChain made a big “State of AI Agents” survey. They asked over 1,300 engineers, PMs, and other professionals about AI agents. Agentic products mentioned most times were Perplexity, Replit, and Cursor.
I was surprised that only ~55% of respondents have tracing/observability for agents, even though many of them reported using agents in production.
7. Agent frameworks remain popular
For frameworks, there is a tradeoff between being able to customize the program and using prepared concepts with less freedom. You face the initial investment of learning the framework logic and understanding the concepts. Other challenges lay in maintainability, e.g., dependencies that need to be monitored for updates.
I noticed that LangGraph has been receiving a lot of praise recently, even compared to LangChain. LangGraph is a library and a low-level visual studio used to build agents and allows you to visualize, interact, and debug even more complex agentic workflows.
Other popular (multi-agent) frameworks are AutoGen and CrewAI. If you consider GitHub stars a good metric, these two are still going strong.
A user survey by AgentOps shows that 40%+ of engineers are using frameworks to build agents. (Here, LangGraph probably falls under LangChain).
8. Good evals for agents are an ongoing challenge
From my anecdotal evidence, hearing from AI developers, and evals of their agents is still painful. Developers want to continuously compare their agents, and different versions of them, the strong category is codegen evals.
There are different benchmarks like SWE bench, but I heard AI founders say that they need to evaluate very very niche things their agents and LLMs are doing within their apps.
The challenge for LLM evals is to create environments that are both realistic and reproducible.
Common examples of benchmarks are SWE bench, LMArena, Web Arena, WorkArena++ for web agents, or Windows Agent Arena. Cognition published A review of OpenAI o1 and how we evaluate coding agents where they collaborated with OpenAI on evaluating their o1 model. They are thinking about what’s the best strategy to evaluate agents.
I think for an individual user/developer, the best way to compare different LLMs is to try it yourself. There was the very exciting launch of WebDev Arena, where you can test for free how different LLMs create whole web apps.
9. AI agents use tools made specifically for them
AI agents increasingly depend on tools made specially for them, within the expanding AI stack ecosystem. It seems the vision of a new wave of LLM-powered software is becoming a reality.
Like Letta for memory, E2B for code execution, AgentOps for testing, or Composio for tool integration. While developers often started by building these components in-house, many are switching to existing solutions, once they have evolved enough.
Code interpreters given to LLMs have become especially crucial. Without a code interpreter, an agent has to rely only on LLM, which is non-deterministic by nature and might hallucinate or avoid more complicated tasks
What’s next?
Dead internet theory says that a significant portion of internet content and traffic will be (or is) generated by bots and AI rather than real humans. Some experts estimate that as much as 90% of online content may be synthetically generated by 2026, according to a Europol report.
I think we are getting into an interesting phenomenon where:
We experience much more noise in the form of internet content created by AI
We then use AI agents to centralize information and filter out that noise - e.g., Manaflow for summarizing news each day and presenting them in a concise way, Perplexity for providing key info supported by links…
So essentially AI agents are both fighting and contributing to the dead internet.
Read more about agents…
Earlier posts on AI agents:
4 Reasons Your AI Agent Needs Code Interpreter - This one we coauthored with Vasek Mlejnsky. (Follow him, he’s super-smart)
