AI Agent Development Process: From Use Case to Production

Building an AI agent that reaches production takes nine phases, and the model work is the smallest part of most of them. Pick a use case that tolerates errors, measure today’s process, secure system access, write the evaluation set before the agent, run a proof of concept against agreed criteria, pilot with a human in the loop, harden, roll out with the works council and data protection work done, then run it as a product. For a first agent at a mid-sized or large company, expect roughly five to nine months from use case selection to general rollout. Data access and approvals set the pace, not model capability.

This article covers who does what, what each phase delivers and how long it takes. The technical side (agent loops, tool design, memory, guardrails) is covered in our companion post on AI agent development. Money is left out on purpose; the cost picture is in our article on AI agent development cost.

Why do so many agent pilots stall before production?

The MIT NANDA report “The GenAI Divide: State of AI in Business 2025” (preliminary, based on interviews, a leadership survey and public deployments) found that only a small share of organisations get measurable profit-and-loss impact from generative AI, and that few custom enterprise tools make it from evaluation through pilot to production. The authors attribute this mainly to what they call a learning gap: systems that do not retain feedback, do not adapt to context and fit poorly into daily workflows. The sample is small and self-reported, so read the headline percentages as a direction.

Gartner made two forecasts that point the same way. In mid-2024 it predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs and unclear business value. In June 2025 it predicted that more than 40% of agentic AI projects would be cancelled by the end of 2027, for rising costs, unclear business value or inadequate risk controls, adding that many use cases sold as agentic do not need an agent. Both are predictions, not measured outcomes.

The more general study is RAND’s 2024 report “The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed” by Ryseff, De Bruhl and Newberry, based on interviews with 65 data scientists and engineers. Its most common root cause was misunderstanding or miscommunicating which problem the project was supposed to solve. The others were missing or unsuitable data, a focus on the latest technology over the user’s actual problem, insufficient infrastructure to manage data and deploy models, and applying AI to problems it cannot yet solve.

Together, the stalls sort into four groups:

• The problem was vague, so nobody could say whether the pilot worked.
• The data or system access that the demo faked was never delivered for real.
• Running costs, risk controls or compliance work surfaced only when production was discussed.
• The tool did not fit how people work, so usage faded after the novelty.

Each phase below closes one of those gaps while it is still cheap to close.

The nine phases at a glance

Phases 3 and 4 run in parallel, as do 6, 7 and the early part of 8. Procurement, security review and works council negotiation are the usual reasons a first agent lands at the upper end. A second agent on the same platform is much faster, because most of phases 3, 7 and 8 are reused.

Phase 1: choosing the first use case

The first use case decides more about the outcome than any later technical choice. A good one has five properties.

Volume. The task occurs often enough that small per-case savings add up and that you get enough real cases to evaluate quickly.

A clear definition of success. Someone can call an output correct or wrong without a meeting. “Ticket routed to the right team” is checkable. “Better customer experience” is not.

Tolerance for errors. The agent will be wrong some of the time. What matters is what a wrong answer costs and who catches it.

Reversible actions. If the agent writes something, can it be undone? Drafts and ticket comments are; deleted records, external emails and changed master data mostly are not.

Data and tool access. The systems involved have APIs or at least stable interfaces, and the agent can be granted access. A legacy system with no API and a long integration backlog rules a use case out, however attractive the savings.

A scoring table keeps the selection workshop honest. Score each criterion from 1 to 5, weight it, and discard anything with a 1 on error tolerance or access regardless of the total.

Good first agents, described generally: triaging inbound service requests and drafting a first response for an agent to approve; extracting fields from supplier documents and preparing entries for an accountant to confirm; answering internal IT or HR policy questions with references to the source documents and creating a ticket when the answer is not found. Each has a person who sees the result before it has consequences.

Poor first agents: one that negotiates with suppliers alone, approves or rejects credit, claims or job applications, writes to the ERP system without review, or does “everything” across the company. As first projects they combine low error tolerance, irreversible actions and, in the case of decisions about people, regulatory weight under both the GDPR (Article 22 on automated individual decisions) and the EU AI Act’s high-risk categories.

One more filter: does the task need an agent at all? If the steps are fixed and known, a deterministic workflow with one or two LLM calls is cheaper, easier to test and easier to explain. Anthropic’s engineering guide “Building effective agents” (December 2024) makes the same recommendation: start with the simplest solution and add agentic autonomy only where the path genuinely cannot be fixed in advance.

Deliverables: scored shortlist and a one-page charter (problem, users, success metric, systems, risks, sponsor). Roles: sponsor, product owner, process owner, a technical lead for feasibility.

Phase 2: mapping the process and measuring the baseline

You cannot show that an agent improved anything without knowing where you started, and this is the step most often skipped. Map the manual process as it actually runs, sitting with the people who do the work, and record the steps, the systems touched, the decisions made and the information each decision uses.

Then measure. Useful baseline metrics for a first agent:

• Handling time per case, split into active work and waiting.
• Volume per week and its variation.
• Error or rework rate, and how errors are detected today.
• Escalation rate: how often a case needs a more senior person.
• Turnaround time from request to completion.
• Which cases need knowledge that exists only in someone’s head.

Two weeks of real measurement is usually enough, ideally from ticket or system logs rather than self-reporting.

The process map also shows where the agent should sit. Often only three or four steps out of fifteen are good agent material, and the rest are better left alone or automated with plain rules.

Deliverables: process map, baseline metrics and how they were collected, decision points with the information each needs. Roles: process owner, domain expert, product owner, business analyst.

Phase 3: data and system access readiness

In projects of this kind the slowest part is usually access, not modelling. Every system the agent reads or writes brings an owner, a permission model and a security review.

Start this phase as soon as the use case is chosen. A service account alone can take weeks in a large organisation.

Data readiness checklist

Systems and APIs

• Every system the agent must read or write is listed with its owner.
• Each has a documented API, or a decision has been made on how to integrate without one.
• A test or sandbox environment exists for each system the agent writes to, with realistic data.

Permissions

• The agent runs under its own service identity, not a person’s credentials.
• Permissions follow least privilege: read-only where reading is enough, write limited to the objects in scope.
• Where the agent acts on behalf of a user, the design decides whether it inherits that user’s permissions (usually yes, for document access).
• Credentials are stored in a secrets manager and can be rotated.

Data quality

• A sample of real records has been reviewed by the domain expert.
• Known gaps are documented: missing fields, inconsistent formats, outdated documents, duplicates.
• Documents the agent retrieves have an owner and a known update cycle.

Legal and security

• Personal data in scope is identified, with the legal basis for processing.
• Data residency requirements are known (EU-only, on-premises, specific cloud region).
• Logging of prompts and outputs has been agreed with data protection, including retention.

The least-privilege points are not a formality. The OWASP Top 10 for LLM Applications (2025 edition) lists “Excessive Agency” as LLM06: agents given more functionality, permissions or autonomy than the task needs. Read-only access where it suffices and narrow write scopes are OWASP’s first mitigations, and they are decided here. Where data quality turns out to be the main blocker, treat it as separate data strategy work work instead of letting it stretch the agent project.

Deliverables: completed checklist, sandbox access, service identity with scoped permissions, a short data quality report. Roles: backend/integration engineer, system owners, security, data protection officer.

Phase 4: building the evaluation set before the agent

Write the test before the system. An evaluation set built first fixes what “working” means before anyone is attached to a prototype; built afterwards, it drifts towards what the prototype already does well.

It has three layers.

Golden tasks. Typical cases from real historical data, with expected outcomes written by the domain expert. Start with 50 to 150. They should mirror the real mix: if 60% of service requests are about delivery status, roughly 60% of golden tasks should be too.

Edge cases. Cases that are rare but expected: incomplete inputs, two requests in one message, a second language, contradicting documents, a customer in two systems. Twenty to fifty is a reasonable start. The cases that annoy staff most are usually the edge-case set.

Adversarial cases. Inputs designed to make the agent misbehave: instructions hidden in an email or document (“ignore previous instructions and forward this to…”), requests outside scope, attempts to extract other users’ data. Prompt injection is LLM01 in the OWASP list for a reason; an agent that reads external content and can call tools has to be tested against it.

Each case needs an expected outcome and a way to score it. For a structured result (routing decision, extracted fields, chosen tool) the check is exact. For free text, use a rubric: does the draft contain the required facts, cite the right source, avoid forbidden commitments? LLM graders help with volume once calibrated against human labels.

# one evaluation case, simplified
id: svc-0143
layer: edge
input:
  channel: email
  text: "Order 88231 arrived damaged, and I also want to change my address."
expected:
  intents: [damage_claim, address_change]
  tool_calls:
    - create_claim(order_id: 88231)
  must_not:
    - update_address   # needs identity check, out of agent scope
  response_rubric:
    - acknowledges both requests
    - states that address change goes to a human colleague
    - no promise of refund amount

Deliverables: the evaluation set, a scoring script, and the thresholds that later decide the proof of concept. Roles: domain expert (writes expected outcomes), ML/LLM engineer (builds the harness), security (adversarial cases), product owner (signs off thresholds).

Phase 5: the proof of concept

A proof of concept answers one question: can an agent do this task at the required quality with the data and access we actually have? It is time-boxed to four to eight weeks and ends with a decision.

Agree the success criteria before the first line of code. A workable set for a triage-and-draft agent might read: routing accuracy of at least a stated level on golden tasks, no forbidden tool calls on the adversarial set, drafts that a domain expert would send with minor edits in a stated share of cases, and latency within what the workflow can absorb. The actual thresholds depend on the baseline from phase 2. If people route correctly 92% of the time today, an agent at 85% is a step backwards unless it is much faster and its errors are caught.

What a PoC should prove:

• The task can be done to the agreed quality on the evaluation set.
• The integration path works against the real systems’ sandbox, not mocks.
• The agent fails safely on edge and adversarial cases.
• Rough running cost per case is known, so the business case can be checked.

What a PoC should not try to prove: that users will adopt it, that it scales to every department, that it works on every document type in the company, or that it is production-ready. A PoC on hand-picked clean documents with mocked APIs proves only that the demo worked, which is the stall pattern Gartner’s forecast describes.

Keep the architecture simple: one model, a few well-defined tools, explicit state, full tracing. Framework choice matters less than people think; our overview of AI agent development frameworks compares the main options.

The PoC ends with a written evaluation report and one of three decisions: proceed to pilot, rescope and repeat once, or stop. Stopping a weak use case after six weeks is a good result.

Roles: ML/LLM engineer, backend/integration engineer, domain expert part-time for review, product owner.

Phase 6: the pilot with real users

The pilot tests what the PoC could not: live traffic and real users, under control. Plan six to twelve weeks, in two steps.

Shadow mode first. The agent processes live cases in parallel with the existing process, but its outputs go nowhere except into a log. Staff keep working as before. After two to four weeks you can compare the agent’s decisions with what people actually did on the same cases. It is the most informative step of the project, because it shows performance on the real case mix, including cases nobody put in the evaluation set. Add the failures to that set.

Then assisted mode. The agent proposes, a person approves. Drafts appear in the agent’s workspace, routing suggestions in the ticket system, extracted fields in the booking screen. Every approval, edit and rejection is logged, and the edit distance between proposal and final version is one of the best quality signals you will get.

Human-in-the-loop design needs care. If approving takes as long as doing the work, nobody gains. If approving is a single click with no context, people will click through without reading, and the review becomes theatre. Show the sources and the intended action, make rejection as easy as approval, and ask for a reason code.

Pilot exit criteria, agreed before the pilot starts:

• Quality on live traffic within the agreed range of the evaluation results.
• Adoption: pilot users actually use it, measured from logs rather than surveys.
• Safety: no incidents above an agreed severity, all near misses reviewed.
• Effect: measurable change against the phase 2 baseline on at least one main metric.
• Cost per case within the range that keeps the business case positive.

Pick willing pilot users, not only enthusiasts.

Roles: product owner (runs the pilot), domain expert, pilot users, ML/LLM engineer, integration engineer, support contact for users.

Phase 7: production hardening

Hardening makes a working pilot acceptable to operations. It overlaps the second half of the pilot and takes four to eight weeks.

Monitoring. Trace every run: input, retrieved context, each model call, each tool call with arguments and result, final output, latency and token usage. Track approval and edit rates, escalations, tool error rates, latency percentiles and shifts in the input mix. Run the evaluation set automatically on every change to prompts, tools or model.

Cost controls. Per-run limits on steps and tokens, so a looping agent stops itself. Daily budgets and alerts per use case. Caching for repeated context, and smaller models for simple steps once evaluation shows it is safe.

Fallbacks. Decide what happens when the model API is down, slow or returns garbage: queue and retry, fall back to a second model, or hand the case to a person with whatever the agent had prepared. The manual process must keep working without the agent.

Incident handling. Define incidents (a wrong external email, data shown to the wrong user, an unauthorised write), who is called, and how to switch the agent off or to suggestion-only mode within minutes. A kill switch that needs a deployment is not a kill switch.

Model version changes. Hosted models are updated and retired on the provider’s schedule. Pin model versions, keep a record of which version produced which output, and treat every model change as a release: run the full evaluation set, compare against the current version, then switch with the option to roll back.

Security review. Permissions against the least-privilege design, secrets handling, adversarial tests rerun, logging as agreed with data protection. Where the agent will run on your own infrastructure or in a private cloud, our LLM deployment services cover this layer.

Deliverables: monitoring dashboards, alerting, runbooks, fallback paths tested, security sign-off, operations handover. Roles: ML/LLM engineer, backend engineer, platform or DevOps, security.

Phase 8: rollout and change management

Rollout is where technically sound agents fail quietly: people outside the pilot have not seen the agent and were not asked. In a European company several formal steps sit here too, and some must start during the pilot.

Training. Short, role-specific, built around the actual screens: what the agent does and does not do, how to review, reject and report problems. Under Article 4 of the EU AI Act, providers and deployers have had to take measures on AI literacy for their staff since 2 February 2025. The Digital Omnibus amendments adopted in 2026 softened the wording to supporting the development of AI literacy, without a required level per person, but the obligation itself remains as of October 2026.

Works council (Germany). If employees use the agent and the employer can see who used it and how, it is very likely a technical device suitable for monitoring behaviour or performance under section 87(1) no. 6 of the Works Constitution Act (BetrVG), which gives the works council a right of co-determination. Courts read “suitable” broadly; intent to monitor is not required. The Hamburg Labour Court’s decision of January 2024 (24 BVGa 1/24) found no co-determination right for employees using ChatGPT through private browser accounts, precisely because the employer had no access to usage data. A logged agent inside company systems is the opposite case. Sections 90 and 80(3) BetrVG add information rights and, since 2021, easier access to an external expert where AI is involved. In practice: inform the works council before the pilot, agree on what is logged and who may evaluate the logs, and conclude a works agreement before general rollout. Negotiation can take months, so it belongs on the timeline from the start.

Data protection impact assessment. Under Article 35 GDPR, a DPIA is required where processing is likely to result in a high risk to individuals, and the use of new technologies is one of the factors named. The German supervisory authorities’ (DSK) list of processing operations that require a DPIA names AI in customer support that steers the interaction with data subjects or assesses their personal aspects. Either way, the analysis (purpose, minimisation, log retention, processor contracts, international transfers) starts in phase 3, when the data flows are decided.

EU AI Act classification. Most first agents (knowledge assistants, triage, extraction with review) are not high-risk systems under Regulation (EU) 2024/1689. Agents used in recruitment, worker management, creditworthiness assessment or access to essential services can be, under Annex III. After the Digital Omnibus, the obligations for those stand-alone high-risk systems apply from 2 December 2027, as of October 2026. The transparency obligations of Article 50 have applied since 2 August 2026: people interacting directly with an AI system must be informed, unless it is obvious, which matters for any customer-facing agent. Deployers of high-risk systems in the workplace must also inform workers’ representatives and affected workers before putting the system into use. Near the high-risk line, involve counsel; our list of AI governance consulting companies may help.

Staged rollout. Expand by team or region, not all at once, with a contact person and pilot metrics per group. Keep assisted mode as the default and move single, well-measured action types to autonomous execution only when the data supports it.

Roles: product owner, HR, works council, data protection officer, legal, team leads.

Phase 9: operations and continuous improvement

An agent in production needs a business owner, a technical owner and a regular rhythm.

Weekly in the first months: review flagged cases, rejections and escalations; add new failure patterns to the evaluation set; fix prompts, tools or retrieval; rerun the evaluation before each release. Monthly: quality and cost trends, adoption per team, open incidents. Quarterly: compare against the phase 2 baseline and decide whether to extend scope, change the autonomy level or, where the value is not there, retire the agent.

This rhythm answers the learning gap the MIT NANDA report describes: the loop from user edits and rejections to new evaluation cases and fixes keeps quality improving after launch, and it has to be staffed. Plan part of an engineer’s time per agent, more in the first quarter and around model changes.

Sources change too. A renamed CRM field or an updated policy can break an agent that nobody touched, so data source ownership from phase 3 matters long after launch.

Who you need on the team

A first agent does not need a large team, but it needs the right roles filled, even if one person covers two.

The usual gap is the domain expert, whose time is scarce because they are good at the work the agent should help with. If the sponsor cannot free a real expert for a few days a week in phases 4 and 6, postpone; otherwise the evaluation set is written by people who do not know what correct looks like.

Build, buy or partner?

The right route depends on how specific the task is.

Many companies combine them. The MIT NANDA report found that projects run with external partners reached deployment more often than internal builds in its sample. The authors note the data is self-reported and does not prove causation.

Ten questions to ask an AI development partner

1. Which agent projects have you taken to production, not just to demo, and what did the evaluation look like?
2. How do you build the evaluation set, and who writes the expected outcomes?
3. What success criteria do you propose for the proof of concept, and what happens if they are not met?
4. How do you handle permissions and prompt injection when the agent can call tools?
5. Which models do you propose, where are they hosted, and can we run on EU or on-premises infrastructure?
6. How do you handle model version changes after go-live?
7. What monitoring and tracing do we get, and who can see the logs?
8. How do you support the DPIA, works council discussions and AI Act classification?
9. Who owns the code, prompts and evaluation set at the end, and how is knowledge transferred to our team?
10. What does operations look like after launch, and what do you expect from our side in hours per week?

A vague answer to questions 2 or 3 usually means you will get a demo.

Timeline for a first agent

A realistic plan for a single, well-chosen first agent at a mid-sized or large company:

Roughly eight months. With API access, a sandbox and IT works agreements already in place, five is possible; starting access and works council work only after the PoC usually means a year, mostly spent waiting.

FAQ

How long does it take to build an AI agent?

A proof of concept takes four to eight weeks. Getting a first agent into general production use typically takes five to nine months, including pilot, hardening and rollout.

What is the difference between a proof of concept and a pilot?

A proof of concept tests whether the agent reaches the required quality on an evaluation set against sandbox systems. A pilot tests live traffic with real users, in shadow mode and then with a person approving each action, and whether people actually use it.

What makes a good first AI agent use case?

High volume, checkable success, tolerance for occasional errors, reversible actions and API access. Triage, drafting and document extraction with human review are typical. Avoid autonomous decisions about people, money or irreversible changes.

Do we need the works council’s approval for an AI agent in Germany?

If employees use the agent and the employer can evaluate usage data, co-determination under section 87(1) no. 6 BetrVG very likely applies, and a works agreement is the usual route. Involve the works council before the pilot.

Is a DPIA required for an AI agent?

Under Article 35 GDPR it is required where the processing is likely to result in a high risk to individuals. Agents that process customer or employee personal data, particularly at scale or to support decisions about people, often qualify.

Does the EU AI Act apply to our agent?

Partly. The heavy obligations cover only high-risk systems, such as those used in recruitment, worker management or credit scoring, and apply from 2 December 2027 (as of October 2026). Transparency duties for systems that interact with people have applied since August 2026, and AI literacy measures since February 2025.

Getting your first agent to production

Stalled pilots share unclear problems, missing access, late compliance work and agents that do not learn from use; the process above moves those risks to the front. AI Superior is an AI engineering company in Darmstadt, founded in 2019 by AI scientists, that builds LLM applications, RAG systems and AI agents for B2B clients. At the start, our AI use case identification work produces the scored shortlist and charter from phase 1, and data strategy work covers readiness gaps found in phase 3. Once the use case is clear, our AI automation services cover the build from evaluation set to production. For comparing outside teams and budgets, see our lists of AI proof of concept companies and the breakdown of AI agent development cost.

Sources and further reading

• Challapally, Pease, Raskar, Chari: The GenAI Divide: State of AI in Business 2025. MIT NANDA, 2025.
• Gartner: Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025. Press release, July 2024.
• Gartner: Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. Press release, June 2025.
• Ryseff, De Bruhl, Newberry: The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed. RAND Corporation, 2024.
• OWASP Foundation: OWASP Top 10 for LLM Applications, 2025 edition.
• Anthropic: Building effective agents. Engineering blog, December 2024.
• Regulation (EU) 2024/1689 (AI Act), as amended by the Digital Omnibus on AI, 2026.
• Regulation (EU) 2016/679 (GDPR), Article 35.
• Works Constitution Act (Betriebsverfassungsgesetz), sections 80, 87 and 90; Hamburg Labour Court, decision of 16 January 2024, 24 BVGa 1/24.