Bringing Agents into Production at Coinbase

The Starting Point

When Focused partnered with Coinbase, the organization had already moved beyond asking whether AI agents were worth exploring. Multiple engineering teams were actively experimenting with agent-based systems using LangChain and LangGraph, and there was genuine momentum behind the approach.

The harder question was whether those agents could survive in production.

Coinbase operates at a scale where AI systems must meet strict standards for reliability, security, observability, and operational control. Historically, many promising AI prototypes stalled at the same point. They worked in demos, but lacked the foundations required for long-term operation, monitoring, and trust.

To change that dynamic, Coinbase formed a small, cross-functional Tiger Team made up of engineers from several product groups. The mandate was ambitious: over roughly six weeks, design, build, and deploy multiple AI agents to production. The goal was to prove that agentic systems could operate safely, predictably, and repeatedly inside Coinbase’s existing platform.

Just as importantly, the team aimed to pave the road for future adoption by establishing shared patterns, tooling, and best practices that other teams could reuse.

Focused joined as a hands-on technical partner to help turn early momentum into durable, production-grade systems.

The Challenge

At the start of the engagement, Coinbase was in a strong position. The Tiger Team was composed of highly capable engineers who were already aligned on modern AI tooling and agent-based design.

What was missing were the foundations required for production.

Agents could be run locally and demonstrated compelling behavior, but once they left a developer’s laptop, visibility dropped off sharply. LangSmith was not yet set up, which meant there was no consistent tracing, no shared view into tool usage or failure modes, and no reliable way to debug complex, multi-step agent workflows in production.

Formal eval pipelines were also absent. Without evals, teams had no systematic way to reason about correctness, detect regressions, or build confidence in non-deterministic systems. At Coinbase’s scale, subjective confidence was not sufficient for production approval.

The work was also high-visibility. Senior leadership was closely watching the outcome to determine whether agentic systems were something Coinbase should continue investing in across the organization.

Focused’s Role

Focused embedded directly alongside Coinbase engineers as a hands-on advisor and consulting engineer. Rather than operating as an external delivery team, we paired day-to-day with engineers across the Tiger Team and adjacent product groups.

Our role spanned architecture, execution, and decision-making. We helped teams clarify product requirements, determine whether specific problems were well-suited for agents, and design systems that could meet enterprise operational standards.

Technically, we advised on structuring agents using LangChain and LangGraph, helped get LangSmith fully operational, and introduced formal evaluation workflows. We contributed to production agents, built proofs of concept, and helped teams reason about quality, safety, and long-term maintainability.

The goal was not just to ship agents, but to leave Coinbase teams with stronger foundations and clearer standards than they had before.

Technical Approach

From the outset, the technical strategy was deliberate: standardize on proven agent infrastructure and treat observability and evaluation as first-class concerns.

One of the most important architectural decisions was to center the work around LangSmith. When we joined, teams had begun building custom logging and tracking code to understand agent behavior. While thoughtful, this approach duplicated existing functionality and added long-term maintenance risk.

Instead, we made a conscious decision to delete complexity rather than add more. By adopting LangSmith, teams gained structured traces, metrics, and detailed visibility into agent execution without owning custom infrastructure. Engineers could inspect how agents behaved across steps, tools, and model calls, making debugging faster and confidence higher.

Formalizing evaluations was equally critical. We worked with teams to operationalize evals, defining datasets, running experiments, and using evaluation results to catch regressions and validate changes over time. For non-deterministic systems, evals became the mechanism that turned intuition into evidence.

We also introduced annotation queues in LangSmith to capture human feedback and turn it into structured evaluation data, enabling continuous improvement without bespoke tooling.

Across the engagement, the approach emphasized:

• LangChain and LangGraph for explicit agent composition and control flow
• LangSmith for observability, evaluation, and feedback
• Clear separation between agent logic and infrastructure
• Reusable templates and patterns for future teams
  ‍

Example Agent Architecture

**‍

Execution and Outcomes**

One of the most important moments of execution came early, when the team decided not to pursue an agent for a proposed use case that was not a good fit. Rather than forcing an agent into a place it did not belong, the team stepped back and redirected effort toward better-aligned problems. That decision avoided wasted time and reduced downstream risk.

By the end of the Tiger Team effort, two agents had been deployed to production and were operating inside real workflows. More importantly, the team established shared foundations that other teams

could adopt.

LangSmith became an active infrastructure rather than a one-off tool. Engineers gained consistent visibility into agent behavior, cost, and failures. Leadership gained confidence by seeing working agents backed by real observability and evaluation data.

The result was not just deployed agents, but a shift in how Coinbase approached agentic development. Agents moved from isolated experiments to a repeatable, production-ready architectural pattern.

The Takeaway

This engagement reinforced a core belief at Focused: the success of agent systems depends far more on system design than on the model itself.

When agents are treated as first-class software components and supported by strong observability and evaluation, they stop being demos and start becoming durable parts of production systems.