How LLM apps are built

Before writing code, let's understand what's actually going on inside an AI product like a customer service bot, a document assistant, or a writing tool.

The basic architecture

1. A user interacts with your interface: a website, mobile app, WhatsApp number or internal tool.
2. Your backend code receives the message, adds instructions and any useful data (customer details, relevant documents), and calls the LLM API.
3. The model provider (Anthropic, OpenAI, Google, or your own hosted open model) runs the model and returns a response.
4. Your backend may check the response, save it, call other systems, then send it back to the user.

The model is just one component. Most of the work, and most of the value, is in your code around it: the instructions, the data you provide, the checks, and the user experience.

Stateless calls

LLM APIs don't remember previous calls. Every request must include everything the model needs: instructions, conversation history, and data. If a chatbot "remembers" what you said earlier, it's because the app sends the whole conversation again with each message.

Tokens and pricing

Providers charge per token, separately for input (what you send) and output (what the model writes). Output tokens usually cost several times more than input tokens. Roughly, 1,000 tokens is about 750 English words. Prices differ a lot between models, and change over time, so check the provider's pricing page.

Ways to keep costs down:

• Use a smaller, faster model for simple tasks such as classification or short replies. Reserve the most capable models for hard reasoning.
• Send only the relevant parts of documents, not everything.
• Limit response length with the max_tokens setting.
• Trim long conversation histories.
• Set spending limits in your provider account.

Choosing a provider and model

Option	Strengths	Trade-offs
Commercial APIs (Anthropic Claude, OpenAI GPT, Google Gemini)	Most capable models, easy to start, no servers to manage	Pay per use, data goes to the provider, need a payment card
Open-weight models (Llama, Mistral, Qwen and others)	Run on your own servers, more control over data	Need GPUs and engineering skill, usually less capable at the same size
Hosted open models (via platforms like Hugging Face or Together)	Open models without managing GPUs	Varying reliability and pricing

In this course our examples use the Anthropic Python SDK. The concepts (messages, system prompts, tools, retrieval) transfer directly to other providers, whose SDKs look very similar.

API keys are secrets

An API key identifies your account and is charged for every request made with it. If someone gets it, they can run up your bill.

• Never put keys directly in your code, in GitHub, or in a mobile app or website's front-end code.
• Store them in environment variables or a secrets manager (Colab Secrets, Streamlit Secrets, your host's settings).
• All API calls must go through your backend, never directly from users' browsers or phones.
• If a key leaks, delete it in your provider dashboard immediately and create a new one.

Getting set up

1. Create a developer account with a provider (for Anthropic: the Claude Console at console.anthropic.com).
2. Add billing and set a low monthly spending limit while learning. A few US dollars is enough for this whole course if you use smaller models.
3. Create an API key and keep it somewhere safe, such as a password manager.

Key takeaways

• Your app sends instructions, history and data to a stateless model API on every call.
• You pay per input and output token. Choose models and context sizes wisely.
• API keys stay on the server, in secrets storage, never in client code.