How to rate limit an AI endpoint for each user

To rate limit an AI endpoint for each user, check the login first, then count the calls of that user ID on the server, and return a 429 status when the count passes the limit. Count the user ID and not the IP address. Add a cap on the input size and the output tokens, because one request can cost much more than another.

This guide shows how to confirm that your endpoint has no limit and how to add one in about 20 minutes. It gives two working options: a rate-limit service and a database counter. It also shows how to add a token budget and a global cap.

What does it mean to rate limit an AI endpoint for each user?

It means that each user has a fixed number of calls in a fixed time. When the user reaches the number, the route refuses the next call. Other users are not affected.

An AI endpoint needs this more than a normal endpoint. A normal request costs a fraction of a cent. An AI request can cost many times more, and the cost depends on the input and the output length. A loop in a script can spend your budget in an hour.

Two limits solve two different problems:

You need at least the rate limit and the global cap. The quota and the token budget make the cost predictable.

How do I confirm that my AI endpoint has no limit?

Send many requests with a short loop. Use a test account on your own app.

1. Sign in to your live app with a test account.
2. Open DevTools, select the Network tab, and use the AI feature once.
3. Right-click the request to your AI route and select Copy as cURL.
4. Paste the command in a file named loop.sh.
5. Wrap the command in a loop and print only the status code.

for i in $(seq 1 60); do
  curl -s -o /dev/null -w "%{http_code}\n" -X POST https://your-app.example/api/ask \
    -H "Content-Type: application/json" \
    -H "Cookie: <your test session cookie>" \
    -d '{"text":"Say hello."}'
done | sort | uniq -c

If the output shows 60 lines with the status 200, the route has no limit. If the output shows a mix of 200 and 429, a limit exists. Check that the limit is low enough for your budget.

Warning: This test costs real money. Use a short message and a low count. Run it only on your own app.

Then check these three cases:

1. Send the request with no cookie. The route must return 401. If it returns 200, the route has no login. Add the login before you add the limit.
2. Send the loop with a second test account. The second account must have its own counter. If the first account blocks the second, the limit is global and not per user.
3. Send the loop from a changed IP address. Use a phone hotspot. If the limit resets, the limit counts the IP address and not the user.

How do I add the limit now? Fix it in order of urgency

Do these steps in order. Steps 1 and 2 stop the worst loss first.

1. Set a hard spend limit at the provider. In the OpenAI dashboard, open the project settings and turn on the hard limit. The OpenAI spend limits guide has the steps. This is your last control and not your main control.
2. Add the login check to the route. Return 401 for a request with no valid session. Without a user ID, a per-user limit is not possible.
3. Cap the input and the output. Reject long input. Set max_output_tokens on the server.
4. Add a per-user rate limit. Use option A or option B below.
5. Return a 429 status with a Retry-After header. The client then knows how long to wait.
6. Add a global daily cap. Count all calls and refuse new calls at your budget.
7. Run the loop test again. Confirm that the count of 429 statuses is correct.

Both options need the user ID. The examples use the placeholder getUserFromSession. Replace it with the session check of your auth library.

Option A: Upstash Ratelimit

Upstash Ratelimit stores the counters in Redis and works over HTTP. The Upstash getting started page shows the setup. Redis.fromEnv() reads two variables: UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN. Set both on the server and do not use a public prefix.

The Upstash algorithm page lists three algorithms. A fixed window is cheap, but it lets bursts pass at the window edge. A sliding window removes that problem and costs more storage. A token bucket smooths bursts and allows a higher first burst. For an AI route, the sliding window is a good default.

import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";
import { NextResponse } from "next/server";

const perMinute = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(5, "1 m"),
  prefix: "ai:minute",
});

const perDay = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(100, "1 d"),
  prefix: "ai:day",
});

export async function POST(req: Request) {
  const user = await getUserFromSession(req);
  if (!user) return NextResponse.json({ error: "Unauthorized" }, { status: 401 });

  const minute = await perMinute.limit(user.id);
  const day = await perDay.limit(user.id);
  if (!minute.success || !day.success) {
    return NextResponse.json(
      { error: "Too many requests" },
      { status: 429, headers: { "Retry-After": "60" } },
    );
  }

  // read the text, check the size, call the model
}

The code uses the user ID as the identifier. The docs use limit(identifier) and read the success field. The two limiters make one burst limit and one daily limit.

Note: The Retry-After value of 60 is correct for the minute limit only. If the day limit blocked the call, the wait is longer. Set a longer value when the day limit blocks the call.

Option B: A counter in Postgres

Use this option if you already use Supabase or another Postgres database and you do not want a new service. The function counts one call for each user and each day. It is atomic, so parallel requests cannot pass the limit.

create table public.ai_usage (
  user_id uuid not null references auth.users (id),
  day date not null default current_date,
  calls int not null default 0,
  primary key (user_id, day)
);

alter table public.ai_usage enable row level security;

create function public.take_ai_call(p_user uuid, p_limit int)
returns boolean
language plpgsql
security definer
set search_path = ''
as $$
declare n int;
begin
  insert into public.ai_usage (user_id, day, calls)
  values (p_user, current_date, 1)
  on conflict (user_id, day)
  do update set calls = public.ai_usage.calls + 1
  returning calls into n;
  return n <= p_limit;
end;
$$;

revoke execute on function public.take_ai_call(uuid, int) from public, anon, authenticated;

The table has RLS on and no policy, so a browser client cannot read or write it. The revoke line removes the right to call the function from the browser roles. Only server code that uses the service_role key can call it. Keep that key on the server. See the service_role guide for the risks. For more on policies, read the Supabase RLS guide.

Call the function from the route, before the model call:

const { data: allowed } = await supabaseAdmin.rpc("take_ai_call", {
  p_user: user.id,
  p_limit: 100,
});
if (!allowed) {
  return NextResponse.json({ error: "Daily limit reached" }, { status: 429 });
}

This option limits the calls for each day. It does not limit the speed of calls inside the day. Add a second counter for the minute, or use Option A for the burst limit.

What status code and headers should the route return?

Return 429 Too Many Requests. The MDN page for status 429 says that the status means the client “has sent too many requests in a given amount of time”. The page also says that a Retry-After header can tell the client how long to wait.

Use the same shape for every limit. The body must have a short message. The header must have the wait in seconds. Do not return 500 or 403. A client or a monitoring tool cannot tell these apart from real errors.

On the frontend, catch the 429 status and show the user a clear message. Show the time when the user can try again. Do not retry in a loop, because a loop increases the load.

How do I limit the cost of one request?

A request limit does not control the cost of each request. One request with a very long input and a long output can cost as much as hundreds of short requests. Add three more controls:

1. Cap the input size. Reject the request if the text is longer than a fixed number of characters. A cap of a few thousand characters fits most chat features.
2. Cap the output. Set max_output_tokens on every call. The caller must not set it.
3. Keep a token budget. The API response contains the number of tokens that the call used. Add this number to a counter of the user. Refuse the next call when the counter passes the budget.

The budget needs one more column in the table. Add tokens int not null default 0 to ai_usage. After each call, add the used tokens to the row of the user for the day. Before each call, read the row and refuse if the value is too high.

Do not let the caller choose the model. A caller who can choose the model can choose the most expensive one. Fix the model on the server.

How do I add a global daily cap?

A per-user limit does not stop an attack with many accounts. If each account has a limit of 100 calls, 1,000 new accounts can make 100,000 calls. A global cap stops this.

1. Count all AI calls for the day in one counter. Use the same Upstash limiter with a fixed identifier such as "global".
2. Set the limit to the number of calls that your daily budget allows.
3. Return 503 Service Unavailable when the cap is reached. Show a short message to the user.
4. Send yourself an alert when the count reaches 80 percent of the cap.

Also add friction to account creation. Require a confirmed email address before the first AI call. For free plans, set a lower global cap than for paid plans.

How do I check the damage if the endpoint had no limit?

Read these places for the period in which the route had no limit.

1. The provider usage page. Look at the daily cost and the request count. Find the first day with a large spike.
2. The model list. Look for models that your app does not use.
3. Your host logs. Count the requests to the AI route for each IP address and each user agent. A script often uses the same user agent for all requests.
4. Your database. If you use Option B, group the rows by user.

select user_id, sum(calls) as calls
from public.ai_usage
where day > current_date - 30
group by user_id
order by calls desc
limit 20;

Compare the top accounts with the average account. Block the accounts that are far above it. Do not delete their rows yet, because you need them as evidence for the provider if you ask for a refund.

How do I prevent the problem in the next release?

1. Add the limit to the route template. Every route that calls a paid API must have the login check, the limit, and the size cap. Write this in the rules of your coding tool.
2. Search each pull request for model calls. Search for api.openai.com, anthropic, and the names of your SDK clients. Check that each one sits behind the limit.
3. Keep the limits in one place. Use one constant for each limit. A changed number then appears in the review.
4. Test the limit in your test suite. Send six requests in a row and assert that the sixth returns 429.
5. Set an alert on the provider spend. Set it below the hard limit.
6. Scan the live app after each release. A change by a coding tool can remove the limit.

For more patterns, read the API abuse protection guide. If the key of your provider is in the browser, fix that first with the exposed key guide.

When should I get help?

Get help in these cases:

• The usage page shows a large bill, and the loop test shows that the route was open. Contact the support of your provider with the project ID and the dates.
• The traffic comes from thousands of IP addresses and new accounts. A counter for each user does not stop it. Ask your host about bot protection.
• Your route passes private data to the model. Ask a security engineer to review the route.
• You run more than one server instance and the counters differ between them. Use a shared store, such as Redis or Postgres. Do not keep counters in the memory of the process.

After the fix, run the loop test one more time. The output must show the 429 status after the limit.