Lesson Content

System prompt design for applications.

The system prompt is the persistent instruction layer for every conversation in your application. In a production API integration, the system prompt does the work that a human moderator does in a manual workflow: it establishes the model's role, constrains its behavior, specifies output format, and handles edge cases.

The four-component system prompt structure.

[ROLE] You are a customer support assistant for Acme Software. You answer questions about Acme's products and help users troubleshoot issues.

[TASK SCOPE] Only answer questions about Acme's products. If a user asks about a competitor's product, acknowledge the question and redirect to Acme's equivalent feature.

[OUTPUT FORMAT] Respond in plain text. Keep responses under 150 words unless the issue requires longer explanation. Use numbered steps for troubleshooting instructions.

[EDGE CASES] If you do not know the answer to a product question, say "I don't have information on that – please contact support@acme.com." Do not guess or fabricate product details.

This four-component structure – role, task scope, output format, edge cases – covers the main failure modes of poorly-specified system prompts.

Conversation history management.

Context windows are finite. A long conversation consumes tokens from both ends: the system prompt consumes input tokens on every request; a long conversation history also consumes input tokens. At some point, the conversation history exceeds the available context.

Strategies for conversation history management:

• Sliding window: Keep only the last N turns. Simple but loses early context.
• Summarization: Periodically summarize earlier turns into a compressed context block and replace the full history with the summary.
• Selective retention: Keep only turns that are explicitly relevant to the current request (requires application logic to determine relevance).

For most applications, a sliding window of the last 10-20 turns is sufficient. For long-session applications (document editing, extended research), a summarization strategy is necessary.

Handling out-of-scope inputs.

Every production application receives out-of-scope inputs – users will ask your AI assistant things it was not designed to answer. Handle this explicitly in the system prompt rather than leaving it to Claude's judgment:

If the user asks about topics outside [your defined scope], respond: "I'm here to help with [scope]. For [out-of-scope topic], I'd recommend [alternative resource]."

Explicit handling in the system prompt produces consistent, on-brand responses to out-of-scope inputs rather than variable behavior.