Prompts, Tokens, Context Windows and AI Memory

A prompt is the information an AI application gives a model for the current task. It may combine system instructions, a user request, examples, retrieved documents, tool results and earlier conversation. Good prompting is less about a magic phrase and more about supplying a clear working contract.

Build a prompt from useful parts

1. Goal: state the result the model should produce.
2. Context: provide the audience, business situation and approved source material.
3. Constraints: define what must not happen, including data and action boundaries.
4. Output: specify the structure, level of detail and evidence required.
5. Examples: show representative input and acceptable output when format or judgement is difficult to describe.

Keep instructions internally consistent. If a prompt asks for both a one-sentence answer and a comprehensive report, the model must guess which requirement matters more.

Tokens and the context window

Models process tokens rather than pages or words. Instructions, chat history, retrieved passages and the generated answer all consume capacity. The context window is the amount the application can provide for a model invocation. Larger context can help, but irrelevant or contradictory material can reduce clarity and increase cost or latency.

Context is not the same as memory

A model invocation does not automatically remember every prior conversation. Applications create memory by storing selected information and adding it back into later requests. That information might be recent messages, a user preference, a conversation summary or records retrieved from a system.

Memory therefore needs product decisions: what is stored, for how long, who can access it, how a user corrects it and when it should be deleted. Do not treat conversational convenience as permission to retain sensitive information.

Prompting needs evaluation

Test prompts against a stable set of ordinary cases, edge cases and prohibited cases. Check factual support, completeness, format, refusal behaviour and whether different users receive equitable outcomes. A prompt that works once in a demonstration is not yet a reliable business process.

For repeatable workflows, version the instructions and examples alongside the application. Record the model, retrieval settings and evaluation result so a later change can be compared rather than guessed.

Next: Embeddings, Vector Search and RAG Explained.

Sources Checked

These primary sources were reviewed on 2026-09-17.

• Google for Developers: Introduction to Large Language Models
• Google for Developers: LLMs: What's a large language model?
• National Institute of Standards and Technology: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile