How to Build AI Agents That Run for Hours, Not Minutes

A single LLM API call takes 30 seconds and returns a neat answer. That part is solved. Now try building an agent that processes 500 legal documents over three hours, extracting clauses, cross referencing them, and producing a structured report. That is a fundamentally different engineering challenge, and most teams discover this the hard way.

At Ulltra, we have been building these kinds of long running agents for clients across finance and compliance. The lessons we have learned are consistent: the agent logic is the easy part. The harness around the agent is what determines whether it succeeds or fails in production.

What goes wrong at runtime

Short lived agents hide a lot of problems. When your agent runs for hours instead of seconds, every fragile assumption breaks:

• Context window overflow. After processing 50 documents, the accumulated context exceeds the model's window. The agent starts losing track of earlier work or hallucinating connections.
• API rate limits. Provider rate limits will interrupt your agent mid task. Without retry logic, the entire run fails.
• Transient failures. Network timeouts, 503 errors, temporary outages. Over a three hour window, these are not edge cases. They are certainties.
• Lost state on crash. Your agent processes 400 out of 500 documents, then the process crashes. Without checkpointing, you start from zero.

The harness pattern

The solution is a concept Anthropic calls the "agent harness": infrastructure that wraps your agent and manages everything outside the core reasoning loop. The agent focuses on thinking. The harness focuses on surviving.

A well designed harness handles four responsibilities:

1. Checkpoint and resume

After every meaningful unit of work, the harness saves a snapshot of the agent's progress. If the process crashes, it reads the last checkpoint and resumes from there instead of restarting.

2. Heartbeat monitoring

The harness emits periodic signals confirming the agent is alive and making progress. If heartbeats stop, an external monitor can alert your team or trigger an automatic restart from the last checkpoint.

3. Graceful shutdown

When you need to stop an agent (deployment, scaling, budget limit), the harness finishes the current work unit, saves state, and exits cleanly. No corrupted half finished results.

4. Cost budgets

Long running agents can burn through API credits fast. The harness tracks cumulative token usage and enforces a spending ceiling. When the budget is exhausted, the agent saves its progress and stops.

A minimal harness in practice

Here is a simplified Python harness that illustrates these concepts:

import json
import time
from pathlib import Path

class AgentHarness:
    def __init__(self, checkpoint_path: str, max_cost_usd: float = 50.0):
        self.checkpoint_path = Path(checkpoint_path)
        self.max_cost = max_cost_usd
        self.total_cost = 0.0
        self.state = self._load_checkpoint()

    def _load_checkpoint(self) -> dict:
        if self.checkpoint_path.exists():
            return json.loads(self.checkpoint_path.read_text())
        return {"processed": [], "results": [], "last_heartbeat": None}

    def _save_checkpoint(self):
        self.state["last_heartbeat"] = time.time()
        self.checkpoint_path.write_text(json.dumps(self.state))

    def run(self, documents: list[str], process_fn):
        remaining = [d for d in documents if d not in self.state["processed"]]
        print(f"Resuming: {len(self.state['processed'])} done, {len(remaining)} left")

        for doc in remaining:
            if self.total_cost >= self.max_cost:
                print(f"Budget exhausted at ${self.total_cost:.2f}")
                self._save_checkpoint()
                return self.state["results"]

            result, cost = process_fn(doc)
            self.total_cost += cost
            self.state["processed"].append(doc)
            self.state["results"].append(result)
            self._save_checkpoint()  # checkpoint after every document

        return self.state["results"]

This is deliberately simple. Production harnesses add retry logic with exponential backoff, parallel processing, structured logging, and integration with monitoring systems. But even this basic version solves the most critical problem: your three hour job no longer restarts from scratch after a failure.

Real world example: compliance document processing

One of our clients needed to analyze 2,000+ regulatory filings for specific clause patterns. The naive approach (one long agent loop) failed consistently around the 90 minute mark due to API timeouts. Here is how we restructured it:

1. Chunked processing. Each document became an independent work unit. The agent processed one document at a time, keeping its context window clean.
2. Persistent checkpoints. After each document, the harness wrote results and progress to a database. Restarts picked up exactly where they left off.
3. Adaptive rate limiting. The harness monitored API response headers and throttled requests before hitting provider limits, rather than crashing into them.
4. Cost tracking. The system logged token usage per document, giving the client full visibility into processing costs and letting them set daily spending caps.

The result: a system that processes the full document set reliably in under four hours, surviving multiple transient failures per run without any manual intervention.

The architecture shift

Building long running agents is not about writing cleverer prompts or choosing a better model. It is about treating your agent as a process that needs the same operational infrastructure as any other critical workload: state management, observability, fault tolerance, and cost controls.

The teams that get this right early will be the ones running AI workflows that their competitors cannot match in reliability or scale. The teams that skip the harness will keep restarting failed runs and wondering why AI "doesn't work in production."

If you are planning to deploy agents that go beyond simple request/response patterns, we would be happy to help you design the architecture. Get in touch and let's talk about what reliable, long running AI looks like for your use case.