OpenAI API in Production: Keys, Responses, Rate Limits

The OpenAI API is OpenAI's metered HTTP developer product for calling models, tools, and embeddings from application code. Documentation lives on developers.openai.com. ChatGPT is the consumer assistant.

Creating a key is free; billed usage is not. Official quickstart samples call client.responses.create with gpt-6-astra.

People searching "ChatGPT API" or "GPT API" mean this product. Assistants sunset in August 2026. This page joins the production path official docs split across separate URLs: project keys and workload identity, Responses versus Completions, model pinning, 429 flavors, and SDK choice.

Key Takeaways

• The API is a billed HTTP product. ChatGPT is the consumer app. The Free usage tier is a $100/month cap in allowed geographies, not unlimited production capacity.
• Production auth is project-scoped keys, service accounts, RBAC, and workload identity federation, not a key pasted into a browser.
• Responses is the default request surface. Chat Completions stays the compatibility format for gateways.
• Pin a live model ID from the models catalog. Leftover IDs such as gpt-4 can still accept traffic.
• Treat 429s as different events. Ramp (slow_down) and overload (HTTP 503) can retry. Spend-limit and credit 429s must not.

What Is the OpenAI API?

The OpenAI API is HTTP endpoints plus official SDKs, billed per token (and per image or audio minute on those SKUs).

OpenAI (announced December 2015) ships it next to ChatGPT. The nonprofit OpenAI Foundation governs the for-profit OpenAI Group, which operates as a public benefit corporation.

The docs host is developers.openai.com. The console is platform.openai.com. The marketing landing is openai.com/api/.

Older platform.openai.com/docs URLs resolve into the same corpus.

"ChatGPT API," "GPT API," and the spaced "open ai api" queries all point here. GPT-6 Astra also ships through Microsoft Azure and Amazon Bedrock; those are distribution paths, not the definition of api.openai.com.

Why This API Matters in 2026

The request primitive moved. Official samples across JavaScript, Python, C#, Java, Go, and Ruby now call Responses.

New capabilities (connectors, Conversations, Codex-oriented SKUs) land there. Chat Completions remains supported.

Assistants is gone. It sunset in August 2026 with no automated migration. Map Assistants to dashboard Prompts, Threads to Conversations, Runs to Responses, and Run steps to Items.

Auth grew past .env. Production orgs now split staging and production projects, mint service accounts, and exchange OIDC, SPIFFE, or X.509 identities for hour-capped Bearer tokens.

A June 2020 launch post still ranks for the key query. That post is history, not the current key model.

How a Production Call Works: Identity, Surface, Model

Three decisions sit on every request. Get them wrong and you leak keys, burn leftover model IDs, or retry a spend-limit 429 until the card declines.

Identity

Local development reads OPENAI_API_KEY from the environment. Production maps a project service account or a federated workload to that project. The browser never sees the secret.

Request Surface

Default new work to client.responses.create. Stay on Chat Completions when a gateway, a vLLM server, or a vendor "OpenAI-compatible" endpoint only implements /v1/chat/completions. Modular's handbook states that compatibility in this ecosystem means Completions; Responses coverage varies.

Model ID

Copy the string from the models page the day you ship, store it in config, and fail closed if someone types a marketing name. Hit your accuracy bar on the strongest SKU, then buy latency and cost with a smaller ID or distillation. That is OpenAI's own model-selection order.

Authentication: Project Keys, Service Accounts, and Federation

The help article still answers where the key is. Production auth is a different job: projects, service accounts, and federation.

Local Keys

Create the secret at platform.openai.com/api-keys. The full value appears once. Lose it, mint another.

Official SDKs read OPENAI_API_KEY. Put that variable in a secrets manager or a local env file that never lands in git.

Project-Scoped Production

Give each teammate a unique key. Sharing keys violates the terms. Never ship a key in a browser, a mobile binary, or a public repo.

Key modes are All, Restricted, Read Only.

Split staging and production projects. Each project gets its own spend cap and rate-limit pool. The default project cannot take members or service accounts.

Preset org roles are owner and reader; preset project roles are owner and member. The current RBAC table also lists a project viewer preset and custom roles.

Role changes can take up to 30 minutes. If a user sits in multiple orgs, send the org header.

Service accounts are the Terraform path. The secret is shown once, same as a user key. Optional controls include IP allowlisting and mTLS.

A 401 can mean a wrong or revoked key, the wrong org or project, missing membership, or an IP missing from the allowlist.

Workload Identity Federation

Agents and CI should not hold a static secret for months. Workload identity federation lets the workload present an OIDC JWT, a SPIFFE JWT-SVID, or an X.509 certificate. OpenAI exchanges that identity for a Bearer access token mapped to a Platform project service account.

Access tokens expire in at most one hour. JWT exchange POSTs to auth.openai.com/oauth/token. X.509 exchange POSTs to mtls.auth.openai.com/oauth/token. Both endpoints are in the token-exchange reference.

Token-exchange responses may include a scope property that mirrors mapping permissions. Federated tokens do not bypass endpoint authorization. An X.509 bearer replaces the API key, not the client certificate: you still send an accepted client certificate to the mTLS API host.

This is not Azure Entra Managed Identity. Azure MI belongs to Azure OpenAI. OpenAI's federation maps onto a Platform service account.

Leak Playbook

Check Usage, rotate the key immediately, scan git history, and update every runtime that still holds the old secret. Key-safety guidance is the checklist. A gateway that centralizes secrets (see Tailscale Aperture) is a complement, not a substitute for rotation.

Responses Versus Chat Completions

Responses is the current default. Chat Completions is the compatibility surface.

The Python SDK still calls Completions the previous standard, supported indefinitely. The migrate guide only says Completions remains supported.

Steve on the API team described the split this way:

"The responses API at its core is what we call an agentic loop. It needs to be able to do multiple things in the span of one API request. Chat completions only allows us to sample for the model one time per request. But in responses we can sample from the model multiple times."

Steve in Build Hour (OpenAI, 5:22)

Responses returns Items in output, not choices[].message. Built-in tools run inside the request. store plus previous_response_id keep state on the server.

OpenAI's internal tests claimed prompt-cache utilization moving from 40% to 80% on Responses versus Completions, plus a vendor-claimed ~3% SWE-bench lift on the same prompt. Treat those as vendor numbers.

Prashant Mital (Head of Applied AI) called Responses a superset of Completions and noted that teams on LiteLLM or custom Completions harnesses leave thinking-model performance on the table. Responses still works stateless: set store: false and replay encrypted_content for zero-data-retention orgs.

GPT-6 Astra tool calling requires Responses. Completions still generates text on Astra; tools on Astra do not. Drop temperature and top_p on Astra.

EU residency rejects service_tier: fast (and the older priority alias) on Astra. Fast mode has no latency SLA.

Rebuild Assistants flows on Responses and Conversations. Agent Builder is on the deprecations list.

Conversation State and Retention

Forum threads disagree on TTL. Use the your-data page.

previous_response_id still bills all prior input tokens in the chain. Conversations last until you delete them, not forever like ChatGPT.

As of March 1, 2023, data sent to the API is not used to train or improve OpenAI models unless you opt in.

EU and residency constraints to budget for: Completions cannot set store=true in non-US regions; Responses cannot set background=true in the EU; Realtime tracing is not EU-residency compliant.

Model Selection and Leftover IDs

Don't pin a marketing name. IDs move. The latest-model nav currently titles the flagship page "Using GPT-6 Astra." The quickstart samples gpt-6-astra.

The models catalog currently leads with GPT-6 Astra for complex reasoning and coding, GPT-5.6 Terra as the cost/intelligence balance, and GPT-5.6 Luna for high-volume work. GPT-5.5 and GPT-5.4 snapshots still exist.

Pin whatever ID you evaluated. Re-read both pages before you freeze a default.

Official order: reach an accuracy target on the strongest model, then buy latency and cost with a smaller SKU or a distilled student. LangSmith Signal (LangChain, September 2026, two-week snapshot) reported gpt-4o-mini in 13% of orgs and 7 of 10 most-used models as mini, nano, or flash. That is production mix, not OpenAI's catalog.

On the pricing page at write time, standard short-context rates for the GPT-6 / GPT-5.6 ladder were gpt-6-astra at $10 / $50 per 1M input / output tokens.

gpt-5.6-sol was $4 / $20, gpt-5.6-terra $2 / $12, and gpt-5.6-luna $0.20 / $1.20. Sol promotional pricing is listed through at least 21 November 2026.

Regional processing adds a 10% uplift on eligible post-5 March 2026 models. Those dollars move. Reopen the page before you freeze a budget.

Pinning leftover IDs is a spend control. Marketing can hide a name. The API may still accept it.

jason (@jxnlco, November 2025) called leftover gpt-4 by accident and spent about $1,000. The deprecations page is the source of record: GA models get at least six months of notice, specialized models at least three, previews as little as two weeks.

ChatGPT product retirements are not API retirements.

Reliability: Rate Limits, 429 Flavors, and Batch

Limits are RPM, RPD, TPM, TPD, IPM, and audio minutes. You hit whichever fires first. They attach at org and project, not at the user.

Some families share a pool. Long-context SKUs have a separate bucket.

Usage tiers (official spend thresholds) run Free ($100/mo cap) through Tier 5 ($1,000 paid, $200,000 approved usage).

The in-between rungs are Tier 1 ($5 paid, $100), Tier 2 ($50, $500), Tier 3 ($100, $1,000), and Tier 4 ($250, $5,000). Third-party "wait N days" tables are unverified. Use the official thresholds.

Watch Retry-After plus x-ratelimit-limit-requests, x-ratelimit-limit-tokens, remaining, reset, and the project-token variants.

After 1M input TPM, increase throughput by at most 50% every 15 minutes. Official SDKs already honor Retry-After for eligible 429 and 503 responses. Do not wrap them in a second unbounded loop.

Elastic (July 2026) documented the two-layer shape in production: OpenAI enforces RPM/TPM/IPM per project, capped at the org ceiling. A noisy project 429s while the org still has headroom.

Elastic polls Admin List project rate limits every five minutes and alerts at 80% of peak one-minute utilization for three consecutive checks. That poll needs an organization Admin API key. Pair it with LLM observability traces so a 429 has a request ID next to it.

A hard spend limit stops affected traffic with a 429, but enforcement is not instantaneous. On r/OpenAI, practitioners report that native hard limits lag 5-10 minutes, which agent loops can outrun (thread).

Community answers are virtual cards, in-process cost estimation, and a local proxy that 429s. Do not open extra accounts to dodge limits.

Batch is 50% cheaper, uses a separate higher pool, and carries a 24-hour completion SLA. Submit JSONL with custom_id. /v1/responses is in the endpoint list.

stream=true is rejected. Batch is the offline and eval path, not the interactive one.

Instrument every response. Log usage.prompt_tokens and usage.completion_tokens with the model ID and a service name.

"easiest quick win if you're calling the SDK directly: both OpenAI and Anthropic return usage.prompt_tokens and usage.completion_tokens in every response. just log those alongside a service name in whatever you're already using - CloudWatch, Datadog, whatever. … for multi-team attribution without touching service code: separate API keys per service is the zero-infra approach."

u/marcusbell95 in r/devops (July 2026)

Watch status.openai.com during incidents. 2023 community figures such as "3 RPM" are not current limits.

Security: Key Safety and Tool Privileges

Key safety and product safety are different jobs.

Key Safety

Unique keys, never client-side, never committed, env or KMS, IP allowlist, rotate on leak. That is the help-center list. Federation shortens the life of anything that does leak.

Product Safety

Run the Moderation API, set safety_identifier where you need abuse clustering, and keep a human in the loop on irreversible actions.

Function calling is a privilege boundary. Sentry's write-up on tool-call exploitation and OWASP LLM06 point at the same failure: the model is not the security kernel.

Do not expose messages[], privileged roles, tools, or tool_choice to the client. Allow-list tools.

Least privilege. Human confirm on refunds, deletes, and anything that can exfiltrate.

Structured Outputs (strict: true plus JSON Schema) make parsers reliable. They do not make the model trustworthy. Prompt injection is a harness problem.

Optional: OpenAI Guardrails' prompt-injection check. Remote MCP servers expand the blast radius; trace tool errors the way you would trace a misbehaving microservice.

If you put LiteLLM or another gateway in front of the SDK, pin the version and audit it. LiteLLM's own advisory confirmed PyPI packages 1.82.7 and 1.82.8 were compromised on 24 March 2026. An AI-suggested package is not inert.

HTTP SDKs, REST, and the Agents SDK

Google's "openai sdk" query ranks the Agents SDK, not the HTTP client. Those are different products.

Harrison Chase (@hwchase17) (April 2025) called "use Agents SDK" bad advice and put the original LangChain in the same camp.

None of those libraries, in his view, were reliable enough for 99% of production use cases.

Treat Agents SDK as an option beside HTTP, not as the production default. For agent loops that must survive a crash, wrap the Responses (or Agents) runner as a workflow so a restart does not reburn paid tokens. Cornelia Davis put it this way at AI Engineer:

"When you're on the 1,350 second turn to the LLM and your application crashes, no sweat. We have kept track of every single LLM call and return and you will not be reburning those tokens. That's what it means. That's what durability means in this space."

Cornelia Davis in OpenAI + Temporal (AI Engineer, 11:15)

That is the job AI harness tools exist to do. LangChain is not required for schemas. The official Python SDK already returns Pydantic models from Completions and Responses.

A first Responses call:

import OpenAI from "openai";

const client = new OpenAI();

const response = await client.responses.create({
  model: "gpt-6-astra",
  input: "Summarize this incident in two sentences.",
});

console.log(response.output_text);

Keep retries at the SDK. Log rate-limit headers.

Claude, Azure, Bedrock, and Completions Gateways

This is failover, not a versus comparison. For a product-level ChatGPT comparison, see Claude vs ChatGPT.

The Claude API sits in the same search-demand band. REST lives at api.anthropic.com. The primary primitive is Messages.

Claude also runs on Bedrock, Vertex, and Azure Foundry. Teams pick Claude for Anthropic's long-context and coding defaults.

In-process dual SDKs are the failure mode r/mlops described: two schemas, two tool-call formats, two token counters.

"Sounds easy until you actually try to load-bear with it. Anthropic Messages and OpenAI Chat Completions have different schemas, different tool call formats, different ways to count tokens. We ended up with two parallel codepaths that had to stay in sync any time either provider shipped a change."

u/nona_jerin in r/mlops (May 2026)

Push failover into a gateway.

Azure OpenAI / Microsoft Foundry. Same model families, Azure bill and SLA, Entra, private endpoints and VNet, residency, provisioned throughput. Foundry is the control-plane choice.

platform.openai.com is the day-zero GA and Batch choice. Dual-path is legitimate. Use Microsoft Learn, not consulting "Azure vs OpenAI" blogs.

Amazon Bedrock. The GPT-6 Astra post names Bedrock as a distribution path next to the API and Azure: cloud bill and IAM instead of Platform keys.

LiteLLM. LiteLLM says it fronts 140+ providers behind one OpenAI-compatible API. Useful for virtual keys, budgets, and failover. Completions-shaped by default, which is Mital's point about leaving Responses performance on the table.

OpenRouter. A multi-model router. Use it as a router, not as a price table.

Gemini and Claude OpenAI-compat layers. A three-line base_url plus chat.completions.create swap works as an eval or migration test. That proves Completions is the interchange format. It is not a reason to skip a native client.

Databricks Unity Gateway. Day-zero Astra with the same governance as Claude, Gemini, and Grok on that platform. A gateway anecdote, not a replacement for api.openai.com.

Open Responses (January 2026 spec) is the ecosystem attempt to standardize /v1/responses. Treat it as the portable agentic shape.

On r/OpenAI, the recurring advice after org bans is to keep the application platform-agnostic so a single vendor kill switch does not take the product down. A gateway is how you do that without maintaining two SDKs.

What a Request Costs

Cost is tokens times the model price. Shrink either side. Prompt cache reads bill at 0.1× when the prefix hits the minimum (1,024 tokens on GPT-5.6+ families, 2,048 on older ones).

Cache writes bill at 1.25×.

Hidden system tokens do not count toward the minimum. Put static instructions and tool schemas first, variable user input last.

prompt_cache_key routes similar requests onto the same backend. Caching does not make outputs identical. Compaction can break the prefix.

Alert on the cached_tokens / prompt_tokens ratio, not only on HTTP error rate. In July 2026, OpenAI staff acknowledged a Completions defect where cache writes billed at 1.25× while cached_tokens stayed at zero.

u/TedSanders (OpenAI, July 2026) localized the bug to Chat Completions and said Responses was working as expected. Confirm against current billing before you treat that incident as still open.

ChatGPT plan prices are seats. API prices are tokens.

For a full model catalog, use the official pricing page.

Databricks published a company-authored estimate of $1.2M/year wasted from seven MCP-server bugs (about $499K tokens, about 12,000 engineering hours, 1,409 tool errors in 24 hours).

Treat the dollars as vendor narrative. The mechanism (trace tool errors before you scale the loop) is the usable part.

Embeddings (text-embedding-3-small / 3-large), Realtime, Batch, image, files, and fine-tuning share the same key. They are separate guides.

Common Production Mistakes to Avoid

Putting the Key in the Client

A browser or mobile binary will leak. Use a backend, a project service account, or WIF. Rotate on any suspicion.

Treating Every 429 as "Slow Down and Retry"

Spend-limit, credit, and usage-quota 429s will not recover if you retry. Inspect error.code. Retry slow_down and 503 overload only.

Calling Leftover Model IDs

gpt-4 still being callable is not a feature you want. Pin IDs. Alert when the model string is not in an allow-list.

Assuming Completions Cache Credits Match Responses

Staff already showed a Completions-only cache-credit bug. If you stay on Completions, monitor cached_tokens. If you need built-in tools or Astra tool calling, use Responses.

Running Agents SDK as a Single Process Without Durable State

A crash on turn 1,350 reburns every paid token unless an orchestrator recorded the turns. HTTP plus a workflow engine is the boring path.

Dual-SDK Failover Inside the App

Two schemas drift. Put Azure, Bedrock, LiteLLM, or OpenRouter in front. Keep application code on one request shape.