daBongo LMS AI Training Courses

Lesson Content

Prototype vs. production gaps.

A prototype makes a happy-path API call. A production integration handles:

• API errors (429 rate limit, 500 server error, network timeout)
• Unexpected or malformed responses
• Cost at scale (token usage, billing alerts)
• Observability (logging, monitoring, alerting)
• Graceful degradation (what does the application do when the API is unavailable?)

Most developers shipping their first API integration discover these gaps when they encounter production traffic.

Error handling.

The Anthropic API returns standard HTTP error codes. Critical ones to handle:

• 429 Too Many Requests: Rate limit exceeded. Implement exponential backoff with jitter – retry after a delay that increases with each retry attempt
• 500/529: Server errors. Retry with backoff; surface an error to the user if retries fail
• Network timeouts: The SDK's auto-retry handles some of these, but implement a circuit breaker for extended outages

The Anthropic SDK includes auto-retry logic for some error types. Review current SDK retry behavior in the documentation and configure it to match your application's needs.

Cost controls.

• Set max_tokens on every request – prevent runaway completions that consume unexpected tokens
• Implement usage alerts in the Anthropic console (notify when monthly spend crosses a threshold)
• Log token counts per request in production – identify unexpectedly large requests early
• For user-facing applications: implement application-level request limits per user/session

Logging strategy.

Minimum production logging for API calls:

• Request ID (from response)
• Model used
• Input and output token counts
• Response latency
• Error type and retry count (if applicable)
• Application-level context (user ID, session ID, feature name)

This logging enables cost attribution, latency monitoring, and debugging of production failures.

Graceful degradation.

For critical-path features backed by the API, define what happens when the API is unavailable:

• Return a cached or static response?
• Surface an error message?
• Fall back to a non-AI path?

Applications with no degradation plan go fully down when the API is unavailable. Applications with degradation plans degrade gracefully and maintain partial function.