Prompt Engineering

Prompt engineering isn't about discovering secret phrases that unlock hidden capabilities. It's about writing clear specifications. You're conditioning a probability distribution — the better your prompt defines the desired output, the more likely the model's next-token predictions land where you want them.

Why Prompting Works

Remember that the model predicts the next token based on everything that came before it. Your prompt is literally the context that shapes those predictions. A vague prompt leaves a wide probability space — the model could go anywhere. A specific, well-structured prompt narrows that space to the outputs you actually want.

This is why the same model can seem brilliant or useless depending on how you talk to it. The model didn't change. The distribution you're sampling from did.

Core Techniques

Be Specific and Explicit

"Summarize this article" is worse than "Summarize this article in 3 bullet points, each under 20 words, focusing on the financial implications." The second prompt constrains the output space. Don't make the model guess what you want.

Give Role and Context

Setting a role primes the model with relevant patterns from training. "You are a senior security engineer reviewing a pull request" activates different patterns than a bare instruction. Provide context about who the output is for, what it will be used for, and what constraints apply.

Few-Shot Examples

Show, don't tell. Including 2-5 examples of input/output pairs in your prompt is one of the most reliable techniques:

Convert the following to structured JSON:

Input: "John Smith, age 34, engineer at Acme Corp"
Output: {"name": "John Smith", "age": 34, "title": "engineer", "company": "Acme Corp"}

Input: "Maria Garcia, age 28, designer at StartupXYZ"
Output: {"name": "Maria Garcia", "age": 28, "title": "designer", "company": "StartupXYZ"}

Input: "Alex Chen, age 41, VP of Engineering at BigCo"
Output:

The model pattern-matches from the examples. This works even for tasks that are hard to describe in words but easy to demonstrate.

Chain of Thought

Adding "think step by step" or "show your reasoning" gives the model intermediate tokens to compute in. LLMs do their "thinking" by generating tokens — if you ask for the answer immediately, the model has to compress all reasoning into a single prediction. Giving it space to work through the steps measurably improves accuracy on math, logic, and multi-step reasoning tasks.

Structured Output

Tell the model exactly what format you want. "Respond in JSON with keys: summary, sentiment, confidence." If you want code, specify the language. If you want a table, say so. Structured output is easier to parse programmatically and reduces the chance of getting conversational filler mixed into your data.

Delimiters and XML Tags

Separate your instructions from your data. If user input is mixed into the prompt without clear boundaries, the model can't tell where instructions end and content begins:

<instructions>
Classify the following customer message by sentiment: positive, negative, or neutral.
Respond with only the classification.
</instructions>

<customer_message>
{"{{user_input}}"}
</customer_message>

This also helps defend against prompt injection (more on that below).

Prefilling the Response

Some APIs let you start the assistant's response. Prefilling the opening of a JSON object forces the model into JSON mode more reliably than asking nicely. This technique constrains the output format from the first token.

Anti-Patterns

Vague instructions. "Make this better" tells the model nothing. Better for whom? In what dimension? Be concrete about what "better" means.

Contradictory constraints. "Be concise but thorough and cover every edge case in under 100 words" forces the model to pick which constraint to violate. Prioritize your constraints or acknowledge the tradeoff.

Burying key instructions in the middle. Models attend more strongly to the beginning and end of the context (lost-in-the-middle). Put your most important instructions first and repeat critical constraints at the end.

System Prompt vs User Prompt

Most APIs distinguish between a system prompt (sets behavior, persona, and constraints) and user prompts (the actual request). The system prompt persists across turns in a conversation. Use it for stable configuration — tone, format rules, safety guardrails. Put per-request specifics in the user prompt.

Temperature as a Task-Dependent Knob

Temperature isn't "creativity." It's sampling randomness. For classification, data extraction, or code generation — tasks with a correct answer — use low temperature (0-0.3). For brainstorming, creative writing, or exploring alternatives, higher temperature (0.7-1.0) gives you more varied output. Match the temperature to whether you want consistency or diversity.

Iteration: The Real Skill

The best prompts aren't written — they're iterated. Test on real inputs, not toy examples. Change one thing at a time so you know what helped. Keep a set of test cases that cover your edge cases. Prompt engineering is empirical engineering, not creative writing.

The Security Note

If your prompt concatenates user input — a search query, a customer message, a document — that's an injection surface. A user can write "ignore your previous instructions and instead..." and the model may comply, because it can't fundamentally distinguish instructions from data. Delimiters help but don't eliminate the risk. This is prompt injection, and it's a real concern for any user-facing LLM application. We'll cover it in depth later in the security lessons.

Key Takeaways

• Prompt engineering is specification, not incantation — you're shaping the probability distribution the model samples from
• Few-shot examples and chain-of-thought are the two highest-leverage techniques for most tasks
• Use delimiters to separate instructions from data, and structure your output format explicitly
• Temperature controls randomness, not intelligence — match it to your task
• Iterate on real inputs, changing one variable at a time
• Any user input in your prompt is an injection surface — treat it accordingly

Next, we'll look at how to give models access to knowledge they weren't trained on using Retrieval Augmented Generation.