Quality and safety: QA and guardrails

Quality and safety: QA and guardrails

Why it matters: Quality and safety are what separate “AI experiments” from reliable systems.

Goals

• Define QA checks for your outputs.
• Add guardrails for sensitive content.
• Create a rollback/review rule.

Key definitions

• QA (quality assurance): A repeatable set of checks that catches mistakes before release.
• Guardrail: A constraint that prevents unsafe or policy-violating outputs.
• Red flag: A signal that requires human review (legal/medical claims, sensitive data, uncertainty).
• Escalation: A rule for what happens when the model is uncertain or refuses.

Workflow (step-by-step)

1. List “never ship without review” categories for your work.
2. Create a QA checklist (facts, numbers, tone, compliance, formatting).
3. Add guardrails to the prompt (no invented facts; ask for sources; flag uncertainty).
4. Add an escalation step: when unsure, ask clarifying questions or stop.
5. Run a “bad case” test: intentionally ambiguous inputs and see what fails.

Example (good vs bad)

✅ Good: You ship only after passing QA, and you have a clear path when the model is uncertain.

❌ Bad: You rely on the model to self-police without explicit rules or review.

Checklist

• QA checklist exists and is used.
• Red flags trigger human review.
• Prompt bans fabrication.
• There is an escalation path.

Metrics / criteria

• Incidents: 0 policy violations shipped.
• QA pass rate increases as prompts improve.
• Measurable: You can say whether the output is correct/complete (pass/fail or a score).
• Effort: You can produce the result in 15–30 minutes using the workflow.
• Clarity: Another person can run your prompt and get a similar outcome.

Common mistakes + fixes

• Pitfall: Assuming “it sounds right” is true. Fix: Require sources, checks, or tests.
• Pitfall: No red-flag rules. Fix: Write explicit “stop and ask” or “human review” conditions.
• Pitfall: Vague instructions. Fix: Add constraints, examples, and a success checklist.
• Pitfall: One-shot prompting. Fix: Iterate: draft → critique → revise → verify.
• Pitfall: No validation. Fix: Add QA steps, tests, and explicit “what counts as done”.

Student tasks

1. Apply the workflow to one real work task related to quality and safety: qa and guardrails.
2. Create a small artifact: prompt, checklist, rubric, table, draft, or decision note.
3. Run one quality check and record what changed after the check.
4. Write a 3-sentence reflection: what worked, what failed, and what you will reuse.

Useful external sources

• OpenAI prompt engineering guide: https://platform.openai.com/docs/guides/prompt-engineering — Prompt structure, iteration, constraints, and evaluation habits.
• NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework — Risk framing, governance language, and verification mindset.
• OWASP Top 10 for Large Language Model Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/ — Practical AI safety risks such as prompt injection, leakage, and insecure output handling.
• Google People + AI Guidebook: https://pair.withgoogle.com/guidebook/ — Human-centered AI design, user control, feedback, and failure modes.

Bibliography

• OpenAI prompt engineering guide. https://platform.openai.com/docs/guides/prompt-engineering
• NIST AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
• OWASP Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/
• Google People + AI Guidebook. https://pair.withgoogle.com/guidebook/