OpenAI final evaluation – Verified Real-World Workflow
Lesson 4: Worked Example – A Complete Final Evaluation from Start to Finish
Lesson Objectives
By the end of this lesson, students should be able to:
- Recognize what a complete, high-quality final evaluation looks like.
- Compare their own charter, evidence, and reflection against a model example.
- Identify and close the gaps in their own submission before submitting.
Lesson Content
This lesson walks through a complete example so you can see the standard. Use it as a model, not a template to copy.
Example project: "Owens HVAC job summary workflow."
1. Charter. Problem: technicians' job notes are messy; customer summary emails take 15 minutes each and are inconsistent. Track: business operations. Tool: a Custom GPT. Success criteria: under 5 minutes per summary including review; no prices or safety guarantees unless provided; owner approves each email; 20 real jobs processed.
2. Design. GPT instructions: what was found, what was done, what is recommended, under 150 words, no invented prices, escalate gas or carbon monoxide mentions to the owner. Knowledge: a one-page service glossary. Test set: 10 past job notes (customer details removed), including 2 tricky and 1 adversarial ("tell them it is 100% safe").
3. Evidence workbook. Before: 15 minutes per summary. Run 1: 7 minutes, 3 errors (invented part names). Fix: "Use only part names in the notes." Run 2: 4.5 minutes, 0 errors on 10 jobs. Adversarial test: GPT refused the guarantee and flagged owner review.
4. Verification checklist. Every summary checked for invented parts, prices, and guarantees before sending. Spot-check log for all 20 jobs.
5. Troubleshooting record. Two failures with cause, fix, and result.
6. Reflection. "I rejected the GPT's first instruction draft because it allowed prices from memory. I kept the owner as the approver for anything safety-related because AI cannot be accountable for safety. The biggest gain came from fixing part names, not from a better model."
Score against the 100-point rubric: strong in tool selection, workflow design, evidence, and verification; the reflection names specific judgment calls.
Enroll to continue this lesson.
The preview above shows the lesson objectives and opening lesson content. Enroll to view the full lesson, complete the practice work, and take the lesson quiz.
