AI Guardrails and Safety: What Engineers Must Know

A helpful answer in a demo does not establish that an AI system is safe to release. Start by naming the risks, limiting what the system can do and testing the controls. This guide explains common failure modes and practical checks, with OWASP and NIST as references. It connects to the skills in our AI engineering overview.

Why models need guardrails

Models can be tricked, can leak information, can invent facts and can be pushed to produce harmful content. They also behave a little differently each time. Traditional software fails in predictable ways, so tests catch most problems. Model output is variable, so you need extra controls around it, and you need to assume that some input will be hostile.

The reference list: OWASP Top 10 for LLM Applications

The Open Worldwide Application Security Project publishes a widely used list of risks for applications built on large language models. The 2025 version lists:

1. LLM01 Prompt Injection
2. LLM02 Sensitive Information Disclosure
3. LLM03 Supply Chain
4. LLM04 Data and Model Poisoning
5. LLM05 Improper Output Handling
6. LLM06 Excessive Agency
7. LLM07 System Prompt Leakage
8. LLM08 Vector and Embedding Weaknesses
9. LLM09 Misinformation
10. LLM10 Unbounded Consumption

You do not need to memorise the numbers. The value is that it turns a vague worry, "is it safe?", into a checklist you can review a design against.

The risks that matter most in practice

Prompt injection

Text in the input tries to change what the model does. Direct injection comes from the user. Indirect injection is hidden in something the model reads: a web page, an email, a document, or even text embedded in an image. Microsoft's AI-103 outline names this last case directly, listing the need to "detect and mitigate indirect prompt injection by using embedded text in images". Assume that anything the model reads could contain instructions from someone else.

Excessive agency

An agent with more access than it needs can cause more harm if it is tricked or simply wrong. See what AI agents are for the design habits that limit this.

Improper output handling

Treating the model's output as safe to run or display. If code executes or renders what the model wrote, an attacker who can influence the model can influence your system.

Sensitive information disclosure and system prompt leakage

The model reveals data it should not, or reveals its own hidden instructions. Do not put secrets in prompts, and filter what data the model can reach.

Misinformation

Fabricated answers presented as fact. Grounding in real sources through RAG helps, but it does not remove the problem, so answers need evaluating.

Unbounded consumption

Loops, retries or very large requests that run up cost or bring the service down. Set limits on tokens, steps and spend.

Practical controls

• Least privilege. Give the system the smallest set of tools and data it needs.
• Approval for risky actions. A human confirms anything that spends money, deletes data or contacts people.
• Input and output filters. Screen for unsafe content, and validate that output matches the format you expect before using it.
• Separate trusted and untrusted content. Mark what came from users and documents so it is not treated as instructions.
• Keep secrets out of prompts and out of anything the model can echo back.
• Limits. Caps on steps, tokens, time and cost.
• Logging and audit trails. Record what the model was given and what it did, so you can investigate.
• Evaluation. Test with adversarial examples, not only friendly ones.

The AI-103 outline covers many of these under responsible AI: safety filters, guardrails, risk detection and content moderation; evaluators and safety evaluations; auditing through trace logging and approval workflows; and governing agent behaviour with oversight modes, constraints and tool-access controls.

The governance side: NIST AI RMF

Safety is also about process. The US National Institute of Standards and Technology released its AI Risk Management Framework on January 26, 2023 to help manage risks to individuals, organisations and society from AI. It is intended for voluntary use and is organised around four functions: Govern, Map, Measure and Manage. Even if you never adopt it formally, it is a useful vocabulary for the questions a team should be asking: who is accountable, what could go wrong, how do we measure it and how do we respond.

Where this appears in certifications

Responsible AI is now a graded topic. Microsoft's AI-103 outline includes it as a major area, and AWS's AIF-C01 gives responsible AI 14% and security, compliance and governance a further 14% of scored content. The certification roadmap shows the route to both.

Learn it by breaking things

Safety is easiest to understand by seeing a system fail. Build a small assistant, feed it a document with a hidden instruction, watch what happens and add the control that stops it. That is how learning by doing applies here, and it is how Code Nexus teaches it: episodes put you in the middle of an incident and ask you to contain it. Create a free account to start with the fundamentals.