Skip to content

Add cooldown estimates and prepare limiter 4.1 #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jhurliman merged 1 commit into from
Sep 10, 2026
Merged

Add cooldown estimates and prepare limiter 4.1 #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 1 addition & 9 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,8 @@ jobs:
node-version: ${{ matrix.node-version }}
cache: "yarn"

- name: Cache node_modules
uses: actions/cache@v4
with:
path: node_modules
key: ${{ runner.os }}-node_modules-${{ hashFiles('**/yarn.lock') }}
restore-keys: |
${{ runner.os }}-node_modules-

- name: Install dependencies
run: yarn install --immutable
run: yarn install --frozen-lockfile

- name: Lint projects
run: yarn lint:ci
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## [4.1.0] - Unreleased

- Add `RateLimiter.getWaitTime(count)` to estimate cooldowns across both the bucket and interval allowance (#88).
- Add regressions for positive bucket balances during interval exhaustion and the 5,000-request backlog report.
- Replace placeholder README snippets with complete examples and clarify sequential limiter composition.
- Allow npm to build distribution archives without requiring Yarn globally.

## [4.0.0] - 2026-09-08

This is a major release because input validation and asynchronous request ordering
Expand Down
253 changes: 131 additions & 122 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,161 +1,119 @@
# limiter

![Build Status](https://github.com/jhurliman/node-rate-limiter/actions/workflows/ci.yml/badge.svg)
[![NPM Downloads](https://img.shields.io/npm/dm/limiter.svg?style=flat)](https://www.npmjs.com/package/limiter)
[![CI](https://github.com/jhurliman/node-rate-limiter/actions/workflows/ci.yml/badge.svg)](https://github.com/jhurliman/node-rate-limiter/actions/workflows/ci.yml)
[![npm](https://img.shields.io/npm/v/limiter.svg)](https://www.npmjs.com/package/limiter)

Provides a generic rate limiter for the web and node.js. Useful for API clients,
web crawling, or other tasks that need to be throttled. Two classes are exposed,
RateLimiter and TokenBucket. TokenBucket provides a lower level interface to
rate limiting with a configurable burst rate and drip rate. RateLimiter sits on
top of the token bucket and adds a restriction on the maximum number of tokens
that can be removed each interval to comply with common API restrictions such as
"150 requests per hour maximum".
Control when work starts in Node.js and browsers. **limiter** provides an
interval rate limiter and a hierarchical token bucket, with Promise-based
waiting, synchronous admission checks, and no runtime dependencies.

## Installation
Use it to pace API calls, throttle messages, or budget bytes. Waiting requests
run in FIFO order on each instance with one active timer, even with thousands
of callers. CommonJS, ES modules, and TypeScript declarations are included.

yarn add limiter
| Choose | When you need |
| --- | --- |
| `RateLimiter` | A maximum token count per interval, with continuous refill underneath |
| `TokenBucket` | Separate burst capacity and refill rate, optionally shared through parent buckets |

## Upgrading from 3.x

Version 4 fixes concurrent accounting and waiting behavior. It keeps the existing
constructors, methods, CommonJS/ESM imports, and fractional-token support, but is
a major release because these observable behaviors change:
## Install

- Token counts and capacities must be finite, non-negative numbers no greater
than `Number.MAX_SAFE_INTEGER`. Numeric intervals must be finite and positive.
Invalid values now throw `RangeError`, or reject an asynchronous call. Validate
configuration and handle rejected `removeTokens()` promises; do not use
negative values, `NaN`, or `Infinity` as sentinels.
- Waiting `removeTokens()` calls on the same instance run in FIFO order. A large
request at the front can delay smaller requests behind it. Synchronous calls
and `fireImmediately` requests can still consume capacity ahead of the queue;
children sharing a parent do not have a global FIFO order.
- Concurrent requests now respect the interval allowance and charge a hierarchy
only once per successful removal. Workloads that previously exceeded their
configured limits may wait longer or receive an immediate rejection signal.
- Remaining balances retain fractional millisecond timing precision. Avoid exact
equality checks on fractional balances; use `tryRemoveTokens()` to check
whether a request can proceed immediately.

Zero-value conventions are unchanged: a standalone `TokenBucket` with
`bucketSize: 0` is unlimited and bypasses parents; `tokensPerInterval: 0` refills
a finite bucket on every attempt. A `RateLimiter` with `tokensPerInterval: 0`
accepts only zero-token requests. These settings are not a general off switch.

See the [changelog](CHANGELOG.md) for the release history and
[Additional Notes](#additional-notes) for the full timing and queue semantics.

## Usage

A simple example allowing 150 requests per hour:

```javascript
import { RateLimiter } from "limiter";

// Allow 150 requests per hour. Also understands
// 'second', 'minute', 'day', or a number of milliseconds
const limiter = new RateLimiter({ tokensPerInterval: 150, interval: "hour" });

async function sendRequest() {
// This call will throw if we request more than the maximum number of requests
// that were set in the constructor
// remainingRequests is the underlying bucket balance; the interval counter
// may impose a lower allowance.
const remainingRequests = await limiter.removeTokens(1);
callMyRequestSendingFunction(...);
}
```sh
npm install limiter
```

Another example allowing one message to be sent every 250ms:
## Pace work

```javascript
Save as `example.mjs` and run `node example.mjs`. The first message starts
immediately; subsequent messages start at least 250 ms apart.

```js
import { RateLimiter } from "limiter";

const limiter = new RateLimiter({ tokensPerInterval: 1, interval: 250 });

async function sendMessage() {
const remainingMessages = await limiter.removeTokens(1);
callMyMessageSendingFunction(...);
}
await Promise.all(["first", "second", "third"].map(async (message) => {
await limiter.removeTokens(1);
console.log(message);
}));
```

The default behaviour is to wait for the duration of the rate limiting that's
currently in effect before the promise is resolved, but if you pass in
`"fireImmediately": true`, the promise will be resolved immediately with
`remainingRequests` set to -1:
For CommonJS, use `const { RateLimiter, TokenBucket } = require("limiter")`.
Create and reuse a limiter for each resource whose budget should be shared.
Each token represents an application-defined unit: a request, a message, a byte,
or a fractional cost. Work starts after `removeTokens()` resolves.

```javascript
import { RateLimiter } from "limiter";
## Reject instead of waiting

const limiter = new RateLimiter({
tokensPerInterval: 150,
interval: "hour",
fireImmediately: true
});
`tryRemoveTokens()` atomically checks capacity and consumes tokens on success.
This complete HTTP example applies one shared budget to all incoming requests:

async function requestHandler(request, response) {
// Immediately send 429 header to client when rate limiting is in effect
const remainingRequests = await limiter.removeTokens(1);
if (remainingRequests < 0) {
response.writeHead(429, {'Content-Type': 'text/plain;charset=UTF-8'});
response.end('429 Too Many Requests - your IP is being rate limited');
} else {
callMyMessageSendingFunction(...);
}
}
```

A synchronous method, tryRemoveTokens(), is available in both RateLimiter and
TokenBucket. This will return immediately with a boolean value indicating if the
token removal was successful.

```javascript
```js
import { createServer } from "node:http";
import { RateLimiter } from "limiter";

const limiter = new RateLimiter({ tokensPerInterval: 10, interval: "second" });

if (limiter.tryRemoveTokens(5))
console.log('Tokens removed');
else
console.log('No tokens removed');
createServer((request, response) => {
if (!limiter.tryRemoveTokens(1)) {
const seconds = Math.max(1, Math.ceil(limiter.getWaitTime(1) / 1000));
response.writeHead(429, { "Retry-After": String(seconds) });
response.end("Too many requests\n");
return;
}
response.end("Accepted\n");
}).listen(3000);
```

To get the number of remaining tokens **outside** the `removeTokens` promise,
simply use the `getTokensRemaining` method.
`getWaitTime(count)` returns an estimated delay in **milliseconds** without
consuming tokens. On `RateLimiter` it accounts for both refill and the interval
allowance. It returns zero when capacity is currently available. This
`RateLimiter` helper is new in 4.1; `TokenBucket.getWaitTime()` is available in 4.0.

```javascript
import { RateLimiter } from "limiter";
For the Promise API with immediate rejection signaling, construct a limiter with
`fireImmediately: true`: `removeTokens()` resolves to `-1` when denied, otherwise
to the remaining bucket balance. Invalid inputs still reject the promise.

const limiter = new RateLimiter({ tokensPerInterval: 1, interval: 250 });

// Prints 1 since we did not remove a token and our number of tokens per
// interval is 1
console.log(limiter.getTokensRemaining());
```
## Budget bytes and bursts

Using the token bucket directly to throttle at the byte level:
A standalone bucket starts empty and refills as time passes. This example
allows a 150 KiB burst after enough idle time and refills at 50 KiB per second:

```javascript
```js
import { TokenBucket } from "limiter";

const BURST_CAPACITY = 1024 * 150; // 150 KiB maximum burst
const FILL_RATE = 1024 * 50; // 50 KiB per second sustained rate

// We could also pass a parent token bucket in to create a hierarchical token
// bucket
// bucketSize, tokensPerInterval, interval
const bucket = new TokenBucket({
bucketSize: BURST_CAPACITY,
tokensPerInterval: FILL_RATE,
bucketSize: 150 * 1024,
tokensPerInterval: 50 * 1024,
interval: "second"
});

async function handleData(myData) {
await bucket.removeTokens(myData.byteLength);
sendMyData(myData);
const chunks = [new Uint8Array(1024), new Uint8Array(2048)];
for (const chunk of chunks) {
await bucket.removeTokens(chunk.byteLength);
console.log(`Ready to send ${chunk.byteLength} bytes`);
}
```

Split any chunk larger than `bucketSize` before requesting tokens. To share a
budget, pass another `TokenBucket` as `parentBucket`. A successful removal charges
the child and every finite ancestor together; a failed attempt charges none.

## API at a glance

| API | Result |
| --- | --- |
| `new RateLimiter({ tokensPerInterval, interval, fireImmediately? })` | A full bucket plus an interval allowance |
| `new TokenBucket({ bucketSize, tokensPerInterval, interval, parentBucket? })` | An empty bucket with independent burst and refill settings |
| `removeTokens(count)` | `Promise<number>`: wait, consume, and return remaining balance |
| `tryRemoveTokens(count)` | `boolean`: consume immediately if possible |
| `getWaitTime(count)` | Estimated milliseconds until capacity is available; never consumes tokens |
| `RateLimiter.getTokensRemaining()` | Underlying bucket balance, which can exceed the current interval allowance |

Intervals accept positive milliseconds or `"second"`, `"minute"`, `"hour"`, and
`"day"` (also `"sec"`, `"min"`, and `"hr"`). Token amounts may be fractional.
Always handle rejected promises for invalid or oversized requests.

## Additional Notes

Waiting calls to `removeTokens()` are processed in FIFO order on each instance,
Expand All @@ -175,7 +133,14 @@ rolling-window algorithm separately when those are your requirements.

`getTokensRemaining()` reports the underlying bucket's balance (possibly
fractional), not the interval counter's remaining allowance. Use
`tryRemoveTokens()` to test whether a request can proceed immediately.
`getWaitTime(count)` for a non-consuming cooldown estimate, or
`tryRemoveTokens(count)` to check and consume in one step. Estimates ignore
queued requests and are not reservations; other calls can change availability.

Awaiting two independent rate limiters in sequence does not atomically enforce
both limits at the eventual work start: work can collect behind the second
limiter. For atomic burst and sustained-rate budgets, use parent/child
`TokenBucket` instances. Those implement token-bucket limits, not rolling windows.

Token counts and capacities must be finite, non-negative numbers at most
`Number.MAX_SAFE_INTEGER`; fractional tokens are supported. Numeric intervals
Expand All @@ -194,6 +159,50 @@ the instance and is not shared across processes, workers, or machines. Configure
instances at the scope of the resource you want to limit; creating a new limiter
for every request defeats a shared rate limit.


## Upgrading from 3.x

Version 4 fixes concurrent accounting and waiting behavior. It keeps the existing
constructors, methods, CommonJS/ESM imports, and fractional-token support, but is
a major release because these observable behaviors change:

- Token counts and capacities must be finite, non-negative numbers no greater
than `Number.MAX_SAFE_INTEGER`. Numeric intervals must be finite and positive.
Invalid values now throw `RangeError`, or reject an asynchronous call. Validate
configuration and handle rejected `removeTokens()` promises; do not use
negative values, `NaN`, or `Infinity` as sentinels.
- Waiting `removeTokens()` calls on the same instance run in FIFO order. A large
request at the front can delay smaller requests behind it. Synchronous calls
and `fireImmediately` requests can still consume capacity ahead of the queue;
children sharing a parent do not have a global FIFO order.
- Concurrent requests now respect the interval allowance and charge a hierarchy
only once per successful removal. Workloads that previously exceeded their
configured limits may wait longer or receive an immediate rejection signal.
- Remaining balances retain fractional millisecond timing precision. Avoid exact
equality checks on fractional balances; use `tryRemoveTokens()` to check
whether a request can proceed immediately.

Zero-value conventions are unchanged: a standalone `TokenBucket` with
`bucketSize: 0` is unlimited and bypasses parents; `tokensPerInterval: 0` refills
a finite bucket on every attempt. A `RateLimiter` with `tokensPerInterval: 0`
accepts only zero-token requests. These settings are not a general off switch.

See the [changelog](CHANGELOG.md) for the release history and
[Additional Notes](#additional-notes) for the full timing and queue semantics.

## Development

```sh
yarn install --frozen-lockfile
yarn lint:ci
yarn test
yarn prepack
```

CI checks the source and both module distributions. Tests cover concurrent
accounting, hierarchical buckets, FIFO backlogs, timing boundaries, and invalid
inputs using deterministic clocks.

## License

MIT License
[MIT](LICENSE.txt).
21 changes: 14 additions & 7 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
{
"name": "limiter",
"description": "A generic rate limiter for the web and node.js. Useful for API clients, web crawling, or other tasks that need to be throttled",
"version": "4.0.0",
"version": "4.1.0",
"author": "John Hurliman <jhurliman@jhurliman.org>",
"main": "./dist/cjs/index.js",
"module": "./dist/esm/index.js",
"browser": "./dist/esm/index.js",
"exports": {
"import": "./dist/esm/index.js",
"require": "./dist/cjs/index.js"
"import": {
"types": "./dist/esm/index.d.ts",
"default": "./dist/esm/index.js"
},
"require": {
"types": "./dist/cjs/index.d.ts",
"default": "./dist/cjs/index.js"
}
},
"repository": "git://github.com/jhurliman/node-rate-limiter",
"repository": "https://github.com/jhurliman/node-rate-limiter.git",
"bugs": {
"url": "http://github.com/jhurliman/node-rate-limiter/issues"
"url": "https://github.com/jhurliman/node-rate-limiter/issues"
},
"license": "MIT",
"files": [
Expand All @@ -30,7 +36,7 @@
"scripts": {
"lint": "eslint --fix src",
"lint:ci": "eslint src",
"prepack": "yarn tsc -p tsconfig.json && tsc -p tsconfig.cjs.json && node ./create-package-json.js",
"prepack": "tsc -p tsconfig.json && tsc -p tsconfig.cjs.json && node ./create-package-json.js",
"test": "jest src"
},
"devDependencies": {
Expand All @@ -54,5 +60,6 @@
"limiting",
"throttling"
],
"packageManager": "yarn@1.22.19+sha512.ff4579ab459bb25aa7c0ff75b62acebe576f6084b36aa842971cf250a5d8c6cd3bc9420b22ce63c7f93a0857bc6ef29291db39c3e7a23aab5adfd5a4dd6c5d71"
"packageManager": "yarn@1.22.19+sha512.ff4579ab459bb25aa7c0ff75b62acebe576f6084b36aa842971cf250a5d8c6cd3bc9420b22ce63c7f93a0857bc6ef29291db39c3e7a23aab5adfd5a4dd6c5d71",
"types": "./dist/cjs/index.d.ts"
}
Loading
Loading