Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ evm-version-flag = --evm-version $(call evm-version,$(1))
GROUP_DIR := $(if $(GROUP),$(GROUP)/,)

.DEFAULT_GOAL := help
.PHONY: probe-chain adopt-token help tools discover add-chain add-lane remove-lane sync sync-preview sync-all sync-check doctor fmt-config clean-scratch snapshot-chain roles-check roles-check-all deploy-token deploy-pool deploy-lockbox deploy-lockrelease-pool deploy-new-chain preflight verify verify-args verify-execution
.PHONY: probe-chain adopt-token help tools discover discover-tokens add-chain add-lane remove-lane sync sync-preview sync-all sync-check doctor fmt-config clean-scratch snapshot-chain roles-check roles-check-all deploy-token deploy-pool deploy-lockbox deploy-lockrelease-pool deploy-new-chain preflight verify verify-args verify-execution

# Deploy-time parameters are read by the forge scripts from the environment (vm.env*). Forward a value
# passed on the make command line (make deploy-token TOKEN_NAME=...) to the forge subprocess; a value
Expand Down Expand Up @@ -99,6 +99,9 @@ tools: ## Check the required tools are installed (forge, curl, jq)
discover: tools ## List the CCIP API chain catalog vs local configs, both planes (FILTER=<term> narrows; ENVIRONMENT=<testnet|mainnet> narrows the plane)
@FILTER="$(FILTER)" ENVIRONMENT="$(ENVIRONMENT)" bash script/config/sync-discover.sh

discover-tokens: tools ## List the CCIP API token catalog for an operator (ADMIN=, SYMBOL=, CHAIN_SELECTOR=, ENVIRONMENT= all narrow; unreviewed tokens included)
@ADMIN="$(ADMIN)" SYMBOL="$(SYMBOL)" CHAIN_SELECTOR="$(CHAIN_SELECTOR)" ENVIRONMENT="$(ENVIRONMENT)" bash script/config/discover-tokens.sh

add-chain: tools ## Generate config/chains/<CHAIN>.json from the live API (CHAIN= and SELECTOR= required)
$(if $(CHAIN),,$(error CHAIN is required: make add-chain CHAIN=<selectorName> SELECTOR=<selector> - both from the make discover API NAME + SELECTOR columns))
$(if $(SELECTOR),,$(error SELECTOR is required - find it with: make discover FILTER=<term>))
Expand Down
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Multi-step flows with a machine-readable manifest each, under [`docs/workflows/`
- [Pool-version behavior deltas](reference/pool-behavior-matrix.md) - rate-limit validation, pause, decimals metering, and fast-finality differences, each fixture-backed.
- [Config and project-store schema](config-schema.md) - every field of `config/chains` and `project/`.
- [Config architecture](config-architecture.md) - the layering, sync data-flow, and writers.
- [The CCIP REST API v2 spec](reference/ccip-api.md) - the source of truth for the `script/config/` API callers, and why an omitted filter is a silent wrong answer.
- [Deployed addresses](deployed-addresses.md) - the project store, resolution ladder, and redeploy guard.
- [Reading a write script's outcome line](reference/script-output.md) - what `SENDING` and the two `NOT SENT` lines mean, and what actually confirms the state.

Expand Down
6 changes: 5 additions & 1 deletion docs/primitives/catalog.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"note": "Generated from the script/ tree and the Makefile. Do not edit by hand; run `npm run docs:catalog`.",
"counts": {
"primitives": 61,
"makeTargets": 28
"makeTargets": 29
},
"primitives": [
{
Expand Down Expand Up @@ -1025,6 +1025,10 @@
"target": "discover",
"help": "List the CCIP API chain catalog vs local configs, both planes (FILTER=<term> narrows; ENVIRONMENT=<testnet|mainnet> narrows the plane)"
},
{
"target": "discover-tokens",
"help": "List the CCIP API token catalog for an operator (ADMIN=, SYMBOL=, CHAIN_SELECTOR=, ENVIRONMENT= all narrow; unreviewed tokens included)"
},
{
"target": "add-chain",
"help": "Generate config/chains/<CHAIN>.json from the live API (CHAIN= and SELECTOR= required)"
Expand Down
50 changes: 50 additions & 0 deletions docs/reference/ccip-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
type: reference
---

# The CCIP REST API v2 spec

Source of truth for every script under `script/config/` that talks to the API. Read it before adding
or changing a call; do not guess parameter names.

- Swagger UI: <https://api.ccip.chain.link/docs>
- Machine-readable spec: <https://api.ccip.chain.link/docs/swagger-ui-init.js> (OpenAPI document
inlined under the `swaggerDoc` key; `curl` it and slice that value out)

Base URL is `https://api.ccip.chain.link/v2`, overridable with the non-secret `CCIP_API_BASE`
(see `.env.example`). Every call this repo makes is read-only and unauthenticated.

## A defaulted filter is a silent wrong answer

The hazard these scripts keep hitting: an omitted parameter is not neutral. The API applies its own
default, answers HTTP 200 with a well-formed body and a confident `totalCount`, and the operator sees
"nothing found" instead of an error.

Two instances, both real:

- `GET /tokens` has **`reviewedOnly`, defaulting to `true`** - only tokens whose projects Chainlink
Labs has reviewed. `?environment=testnet` returns 22 tokens; adding `&reviewedOnly=false` returns
~25,900. Filtered by `admin=`, the default returns `totalCount 0` for an operator who has 169
tokens. `script/config/discover-tokens.sh` therefore always sends `reviewedOnly=false`.
- `GET /chains` was queried as `?environment=testnet`, hiding every mainnet chain. Unset returns both
planes, which is what `script/config/sync-discover.sh` now does.

So: check the spec's `default` for every parameter you omit, and prefer sending the value explicitly
over relying on the default. `GET /tokens` has one more defaulted boolean, `expand` (default `false`),
which decides whether each `remoteChains[]` entry carries `status` and `remoteTokenAddress` at all.

Name enumeration is cheap but not sufficient: an unknown parameter gives
`400 {"error":"BAD_REQUEST","message":"Unknown query parameter 'zzz'"}`, which finds typos but never
finds a parameter you did not think to try. That is how `reviewedOnly` was missed. Read the spec.

## Routes this repo calls

| Route | Caller | Query parameters |
| --- | --- | --- |
| `GET /chains` | [`sync-discover.sh`](../../script/config/sync-discover.sh) | `environment` only (unset = both planes). No paging. |
| `GET /chains/{selector}` | [`ccip-config-source.sh`](../../script/config/ccip-config-source.sh), [`ccip-chain-meta.sh`](../../script/config/ccip-chain-meta.sh) | none accepted. |
| `GET /tokens` | [`discover-tokens.sh`](../../script/config/discover-tokens.sh) | `chainSelector`, `remoteChainSelector`, `groupId`, `symbol`, `address`, `admin`, `environment`, `reviewedOnly`, `expand`, `limit`, `cursor`. |

`GET /tokens` pages by **keyset, not offset**: `page` and `offset` are rejected with a 400. Follow
`pagination.cursor` while `pagination.hasNextPage` is true. Filters may travel with the cursor only if
they match the ones encoded in it; a differing filter is a 400.
1 change: 1 addition & 0 deletions script/config/ccip-chain-meta.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
# non-EVM selectorName rung) via vm.tryFfi — config-file generation stays Foundry-side
# (vm.serialize* + vm.writeFile / vm.writeJson); this script only fetches + selects.
#
# Spec (read it before changing a query): docs/reference/ccip-api.md
# Exit-code contract (stderr becomes the Solidity revert reason):
# 0 OK | 2 MISSING_TOOL | 4 NOT_FOUND (no chain for this selector) | 5 API_UNREACHABLE
set -euo pipefail
Expand Down
1 change: 1 addition & 0 deletions script/config/ccip-config-source.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
# 5 API_UNREACHABLE network error / timeout / 5xx (flake, NOT drift — retry later)
# 6 BAD_BODY 200 but chainConfig lacks an active entry (partial/unsupported chain)
#
# Spec (read it before changing a query): docs/reference/ccip-api.md
# The API base URL can be overridden with the non-secret CCIP_API_BASE env var (see .env.example).
set -euo pipefail

Expand Down
187 changes: 187 additions & 0 deletions script/config/discover-tokens.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
#!/usr/bin/env bash
# discover-tokens.sh - list the CCIP REST API v2 TOKEN catalog for an operator, with local state
# awareness. The `make discover` sibling: that one lists chains (`GET /chains`), this one lists
# tokens (`GET /tokens`).
#
# Columns: CHAIN SELECTOR | CHAIN | TOKEN | SYMBOL | GROUP | LANES | LOCAL CONFIG. LANES counts the
# remoteChains entries with status CONNECTED (the array also carries DISCONNECTED ones). LOCAL CONFIG
# joins config/chains/*.json **BY SELECTOR**, the numeric identity key, exactly as sync-discover.sh
# does.
#
# Filters, all optional and all env-driven: ADMIN (TokenAdminRegistry administrator), SYMBOL,
# CHAIN_SELECTOR (home chain), ENVIRONMENT (testnet|mainnet; unset lists both planes). There is no
# flag interface: the script takes no arguments at all.
#
# TWO PARAMETERS THIS SENDS EXPLICITLY, because their defaults are wrong for an operator:
# reviewedOnly=false - defaults to TRUE, i.e. only Chainlink-Labs-reviewed projects. Measured:
# ?environment=testnet returns 22 tokens, &reviewedOnly=false returns ~25,900,
# and with an admin= filter the default returns a confident totalCount 0 for
# an operator who has 169. HTTP 200, well-formed, no error - a silent wrong
# answer. See docs/reference/ccip-api.md.
# expand=true - defaults to FALSE, which strips `status` from every remoteChains entry, so
# the CONNECTED count cannot be computed at all.
#
# Paging is KEYSET, not offset (`page`/`offset` are rejected with a 400): follow pagination.cursor
# while pagination.hasNextPage. Stopping after page 1 silently truncates the operator's list.
#
# AN EMPTY RESULT IS A NORMAL ANSWER, NOT AN ERROR - exit 0. The API lists a token only once its mesh
# is fully configured, so a mesh still being wired is legitimately absent. It is never evidence that
# anything is wrong.
#
# Exit codes: 0 OK (including zero matches) | 2 MISSING_TOOL | 3 BAD_ARG | 5 API_UNREACHABLE |
# 6 BAD_BODY. Matches sync-discover.sh, its direct sibling, which in turn shares the
# ccip-config-source.sh contract. 1 is deliberately unused: everywhere in this repo it
# means a verdict against what the operator owns, and a read-only listing has no verdict
# to issue - a transport or parse failure must never read as "you have no tokens".
# `make discover-tokens` is pass/fail only, since GNU make remaps any failing recipe to
# its own exit 2; call this script directly for the codes (same as probe-chain.sh).
set -euo pipefail

err() { echo "[discover-tokens] $*" >&2; }

# A flag-style `--admin 0x...` would otherwise be silently dropped and list the WHOLE catalog
# unfiltered - the same shape of confident wrong answer as the reviewedOnly default.
if [ "$#" -gt 0 ]; then
err "BAD_ARG: unexpected argument '$1' - discover-tokens takes no arguments"
err " filters are environment variables: ADMIN= SYMBOL= CHAIN_SELECTOR= ENVIRONMENT= (make discover-tokens ADMIN=0x...)"
exit 3
fi

for tool in curl jq; do
command -v "$tool" > /dev/null 2>&1 || {
err "MISSING_TOOL: '$tool' not found on PATH - install it (e.g. brew install $tool)"
exit 2
}
done

cd "$(dirname "$0")/../.."

BASE_URL="${CCIP_API_BASE:-https://api.ccip.chain.link/v2}"
ADMIN="${ADMIN:-}"
SYMBOL="${SYMBOL:-}"
CHAIN_SELECTOR="${CHAIN_SELECTOR:-}"
ENVIRONMENT="${ENVIRONMENT:-}"

uri() { jq -rn --arg v "$1" '$v | @uri'; }

# Only the two values the API honours; it answers an unrecognised one with 200 and an EMPTY list,
# which would print a confident "no tokens". Refuse by name before any request (same as discover).
case "$ENVIRONMENT" in
"" | testnet | mainnet) ;;
*)
err "BAD_ARG: ENVIRONMENT='$ENVIRONMENT' - use testnet or mainnet, or leave it unset for both"
exit 3
;;
esac

# The spec constrains chainSelector to ^[0-9]+$; a non-numeric one is a 400 the operator would have
# to decode from the API's wording.
case "$CHAIN_SELECTOR" in
"") ;;
*[!0-9]*)
err "BAD_ARG: CHAIN_SELECTOR='$CHAIN_SELECTOR' - must be the numeric chain selector (make discover lists them)"
exit 3
;;
esac

query="reviewedOnly=false&expand=true&limit=1000"
[ -n "$ENVIRONMENT" ] && query="${query}&environment=${ENVIRONMENT}"
[ -n "$ADMIN" ] && query="${query}&admin=$(uri "$ADMIN")"
[ -n "$SYMBOL" ] && query="${query}&symbol=$(uri "$SYMBOL")"
[ -n "$CHAIN_SELECTOR" ] && query="${query}&chainSelector=${CHAIN_SELECTOR}"

body_file="$(mktemp)"
rows_file="$(mktemp)"
map_file="$(mktemp)"
cfg_file="$(mktemp)"
trap 'rm -f "$body_file" "$rows_file" "$map_file" "$cfg_file"' EXIT

fetch() { # fetch <url> - body into $body_file, exits 5 on anything but 200
local code
code="$(curl -sS --retry 3 --max-time 30 -o "$body_file" -w '%{http_code}' "$1" 2> /dev/null)" || {
err "API_UNREACHABLE: could not reach $1 (network error/timeout after retries) - retry later or fix CCIP_API_BASE"
exit 5
}
[ "$code" = "200" ] || {
err "API_UNREACHABLE: HTTP ${code} from $1 - retry later"
exit 5
}
}

# selector -> selectorName, both planes in one GET (/chains takes only `environment` and does not page).
fetch "${BASE_URL}/chains"
jq -r '(if type == "array" then . else .chains end)[]
| [(.chainSelector | tostring), .name] | @tsv' "$body_file" > "$map_file" || {
err "BAD_BODY: ${BASE_URL}/chains answered 200 with a body this script cannot read"
exit 6
}

# selector -> local config name (skip the gitignored zz-scratch-* test files)
for f in config/chains/*.json; do
[ -e "$f" ] || continue
case "$(basename "$f")" in zz-scratch-*) continue ;; esac
jq -r '[(.chainSelector | tostring), .name] | @tsv' "$f"
done | awk -F'\t' '{ a[$1] = (a[$1] == "" ? $2 : a[$1] "," $2) } END { for (s in a) print s "\t" a[s] }' \
> "$cfg_file"

cursor=""
pages=0
total=""
# A cursor that never clears would spin forever against a broken or mocked API; 1000/page makes 500
# pages a ceiling no real operator reaches.
while [ "$pages" -lt 500 ]; do
url="${BASE_URL}/tokens?${query}"
[ -n "$cursor" ] && url="${url}&cursor=$(uri "$cursor")"
fetch "$url"
pages=$((pages + 1))

jq -r '.data[]
| [(.chainSelector | tostring), .address, (.symbol // "-"), (.groupId // "-"),
([(.remoteChains // [])[] | select(.status == "CONNECTED")] | length | tostring)]
| @tsv' "$body_file" >> "$rows_file" || {
err "BAD_BODY: ${BASE_URL}/tokens answered 200 with a body this script cannot read"
exit 6
}

[ -n "$total" ] || total="$(jq -r '.pagination.totalCount // "?"' "$body_file")"
has_next="$(jq -r '.pagination.hasNextPage // false' "$body_file")"
cursor="$(jq -r '.pagination.cursor // ""' "$body_file")"
[ "$has_next" = "true" ] && [ -n "$cursor" ] || { cursor=""; break; }
done
# Truncation must never be silent - that is the same failure as the reviewedOnly default.
[ -z "$cursor" ] || err "WARNING: stopped at the ${pages}-page ceiling; the list below is INCOMPLETE - narrow it with ADMIN=/SYMBOL=/CHAIN_SELECTOR="

rows="$(wc -l < "$rows_file" | tr -d ' ')"
scope="$([ -n "$ENVIRONMENT" ] && echo "$ENVIRONMENT" || echo "both planes")"
filters="$(printf '%s' "${ADMIN:+ADMIN=$ADMIN }${SYMBOL:+SYMBOL=$SYMBOL }${CHAIN_SELECTOR:+CHAIN_SELECTOR=$CHAIN_SELECTOR }")"

if [ "$rows" -eq 0 ]; then
echo "No tokens match (${scope}${filters:+, ${filters% }})."
echo ""
echo "This is a normal answer, not a failure: the API lists a token only once its mesh is fully"
echo "configured, so a token still being wired through this repo is legitimately absent. Nothing"
echo "is wrong with your setup."
echo ""
echo " widen it: drop ADMIN/SYMBOL/CHAIN_SELECTOR, or leave ENVIRONMENT unset to list both planes"
echo " manual: make adopt-token CHAIN=<name> TOKEN=<addr> POOL=<addr> (record what you already have)"
echo " make deploy-new-chain CHAIN=<name> SELECTOR=<sel> (deploy a new one)"
exit 0
fi

{
printf 'CHAIN SELECTOR\tCHAIN\tTOKEN\tSYMBOL\tGROUP\tLANES\tLOCAL CONFIG\n'
awk -F'\t' -v OFS='\t' -v names="$map_file" -v cfgs="$cfg_file" '
FILENAME == names { nm[$1] = $2; next }
FILENAME == cfgs { cfg[$1] = $2; next }
{ print $1, ($1 in nm ? nm[$1] : "?"), $2, $3, $4, $5,
($1 in cfg ? "configured(" cfg[$1] ")" : "no local config") }
' "$map_file" "$cfg_file" "$rows_file" | sort
} | column -t -s "$(printf '\t')"

echo ""
echo "${rows} token(s) listed (API totalCount ${total}, ${pages} page(s), ${scope}${filters:+, ${filters% }})."
if awk -F'\t' -v cfgs="$cfg_file" 'FILENAME == cfgs { cfg[$1]; next } !($1 in cfg) { n++ } END { exit !(n > 0) }' "$cfg_file" "$rows_file"; then
echo "Rows marked 'no local config' need their chain onboarded first:" \
"make add-chain CHAIN=<CHAIN column> SELECTOR=<CHAIN SELECTOR column>"
fi
echo "raw: curl -s '${BASE_URL}/tokens?${query}' | jq"
1 change: 1 addition & 0 deletions script/config/sync-discover.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# until a mainnet token could not be found through it - `make add-chain` was never restricted (it
# fetches /chains/<selector> directly), so the chain was addable all along, just invisible here.
#
# Spec (read it before changing a query): docs/reference/ccip-api.md
# Exit codes: 0 OK | 2 MISSING_TOOL | 3 BAD_ARG | 5 API_UNREACHABLE
set -euo pipefail

Expand Down
Loading
Loading