Skip to content

Commit ebffd4f

Browse files
mekilismekilis
authored
feat: switch client generation to openapi-python-client (PDE-755) (#20)
speakeasy's free tier allows one generated SDK per workspace and convoy.js holds that slot, so python generation moves to the OSS openapi-python-client (pinned 0.29.0). - .github/workflows/sdk_generation.yaml: rewritten for openapi-python-client; same filename and dispatch inputs so the frain-dev/convoy dispatcher works unchanged; opens a PR only when the regenerated client differs - speakeasy pipeline kept dormant, not deleted: old workflow renamed to speakeasy_generation.yaml (manual-only), .speakeasy/ and .genignore kept current so switching back is a trigger change - scripts/generate.sh: fetches the spec, generates with --fail-on-warning, rsyncs into src/convoy/ excluding hand-written verify (src/convoy/utils/) and py.typed - pyproject.toml: hand-owned packaging (convoy-python 1.0.0a1, httpx + attrs); wheel verified to carry verify + generated client - MIGRATION.md / README.md updated for the generator switch
1 parent 9ec1e94 commit ebffd4f

12 files changed

Lines changed: 248 additions & 49 deletions

File tree

‎.genignore‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,18 @@
1+
# NOTE: .genignore is Speakeasy's protection mechanism and is DORMANT while
2+
# generation runs on openapi-python-client (scripts/generate.sh enforces the
3+
# same hand-owned paths via rsync excludes). Kept current so a switch back to
4+
# Speakeasy needs no rebuild.
5+
16
# Hand-written webhook signature verification — never overwrite with generated
27
# code. It lives inside the generated src/convoy tree so the import path stays
3-
# `from convoy.utils.webhook import Webhook`; only this file is hand-owned.
4-
src/convoy/utils/webhook.py
8+
# `from convoy.utils.webhook import Webhook`; only this dir is hand-owned.
9+
src/convoy/utils/**
10+
src/convoy/py.typed
11+
12+
# Hand-owned packaging + generator config.
13+
pyproject.toml
14+
.openapi-python-client.yml
15+
scripts/**
516

617
# Shared signature contract + verify unit tests (hand-authored).
718
test/signature-vectors.json

‎.github/workflows/run-tests.yml‎

Lines changed: 5 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -19,16 +19,11 @@ jobs:
1919
python-version: "3.11"
2020

2121
- name: Install package
22+
# Installation must succeed — a packaging regression (e.g. wheel
23+
# omitting webhook verify) has to fail CI, not hide.
2224
run: |
2325
python -m pip install --upgrade pip
24-
# pyproject.toml arrives with the first Speakeasy generation. Once
25-
# it exists, installation must succeed — a packaging regression
26-
# (e.g. wheel omitting webhook verify) has to fail CI, not hide.
27-
if [ -f pyproject.toml ]; then
28-
pip install -e .
29-
else
30-
echo "pre-generation: no installable package yet; tests import from src/"
31-
fi
26+
pip install -e .
3227
pip install pytest
3328
3429
- name: Verify hand-written modules are present
@@ -38,17 +33,9 @@ jobs:
3833
test -f test/test_shared_vectors.py
3934
4035
- name: Verify installed package exposes webhook verify
41-
if: hashFiles('pyproject.toml') != ''
4236
# No PYTHONPATH: this must resolve from the installed distribution.
4337
run: python -c "from convoy.utils.webhook import Webhook"
4438

4539
- name: Execute verify + shared vector tests
46-
run: |
47-
if [ -f pyproject.toml ]; then
48-
# Import from the installed distribution so packaging bugs surface.
49-
pytest test/test_webhook.py test/test_shared_vectors.py -q
50-
else
51-
# PEP 420 namespace packages: resolve convoy.utils.webhook from
52-
# src/ before generation adds real __init__.py files.
53-
PYTHONPATH=src pytest test/test_webhook.py test/test_shared_vectors.py -q
54-
fi
40+
# Import from the installed distribution so packaging bugs surface.
41+
run: pytest test/test_webhook.py test/test_shared_vectors.py -q

‎.github/workflows/sdk_generation.yaml‎

Lines changed: 90 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,27 @@
11
name: SDK Generation
22

3+
# Active generator: openapi-python-client (OSS). The Speakeasy pipeline is
4+
# dormant in speakeasy_generation.yaml (free tier allows one generated SDK
5+
# per workspace; convoy.js holds that slot).
6+
#
7+
# Keeps the same workflow filename and dispatch inputs as the Speakeasy
8+
# version so the frain-dev/convoy dispatcher (speakeasy-sdk.yml) works
9+
# unchanged.
10+
311
on:
412
workflow_dispatch:
513
inputs:
614
force:
7-
description: Force SDK generation even if no changes are detected
15+
# Accepted for dispatcher compatibility. Generation is deterministic
16+
# from the spec, so there is nothing to force: no diff means no PR.
17+
description: Accepted for compatibility; regeneration is always run
818
required: false
919
default: "false"
1020
type: string
1121
feature_branch:
1222
description: Branch for SDK changes
1323
required: false
1424
type: string
15-
environment:
16-
description: Environment variables (e.g., TAG=branch-name)
17-
required: false
18-
type: string
1925
schedule:
2026
- cron: "0 6 * * 1"
2127

@@ -28,20 +34,85 @@ concurrency:
2834
permissions:
2935
contents: write
3036
pull-requests: write
31-
checks: write
32-
statuses: write
3337

3438
jobs:
3539
generate:
36-
# Pin to commit SHA for v15 (mutable tags can be retargeted).
37-
uses: speakeasy-api/sdk-generation-action/.github/workflows/workflow-executor.yaml@7276a3ae83eafb7fe37630fb1fefea3f2649debd # v15
38-
with:
39-
mode: pr
40-
force: ${{ inputs.force || 'false' }}
41-
feature_branch: ${{ inputs.feature_branch }}
42-
environment: ${{ inputs.environment }}
43-
secrets:
44-
# Prefer a PAT: PRs opened with GITHUB_TOKEN do not trigger
45-
# pull_request workflows, so verify CI would never run on them.
46-
github_access_token: ${{ secrets.SDK_BOT_PAT || secrets.GITHUB_TOKEN }}
47-
speakeasy_api_key: ${{ secrets.SPEAKEASY_API_KEY }}
40+
runs-on: ubuntu-latest
41+
steps:
42+
- name: Checkout Code
43+
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
44+
with:
45+
# Prefer a PAT: PRs opened with GITHUB_TOKEN do not trigger
46+
# pull_request workflows, so verify CI would never run on them.
47+
token: ${{ secrets.SDK_BOT_PAT || secrets.GITHUB_TOKEN }}
48+
49+
- name: Setup Python
50+
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
51+
with:
52+
python-version: "3.11"
53+
54+
- name: Install generator
55+
# Pin both so regeneration output is reproducible; ruff is the
56+
# generator's post-processing formatter.
57+
run: pip install openapi-python-client==0.29.0 ruff==0.15.22
58+
59+
- name: Regenerate client
60+
run: ./scripts/generate.sh
61+
62+
- name: Detect changes
63+
id: diff
64+
run: |
65+
if git diff --quiet && [ -z "$(git status --porcelain)" ]; then
66+
echo "changed=false" >> "$GITHUB_OUTPUT"
67+
echo "No client changes; skipping PR." >> "$GITHUB_STEP_SUMMARY"
68+
else
69+
echo "changed=true" >> "$GITHUB_OUTPUT"
70+
fi
71+
72+
- name: Prepare feature branch
73+
if: steps.diff.outputs.changed == 'true'
74+
id: branch
75+
env:
76+
# Never interpolate free-form dispatch inputs into run: directly.
77+
FEATURE_BRANCH_INPUT: ${{ inputs.feature_branch }}
78+
run: |
79+
if [ -n "$FEATURE_BRANCH_INPUT" ]; then
80+
# SDK PRs must come from a reviewable feature branch, never a
81+
# protected ref or an option-looking / metacharacter name.
82+
case "$FEATURE_BRANCH_INPUT" in
83+
main|master|release/*|-*|*[!a-zA-Z0-9._/-]*)
84+
echo "::error::Invalid feature_branch '$FEATURE_BRANCH_INPUT'"
85+
exit 1
86+
;;
87+
esac
88+
echo "name=$FEATURE_BRANCH_INPUT" >> "$GITHUB_OUTPUT"
89+
else
90+
echo "name=sdk-regen-$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
91+
fi
92+
93+
- name: Push branch and open PR
94+
if: steps.diff.outputs.changed == 'true'
95+
env:
96+
GH_TOKEN: ${{ secrets.SDK_BOT_PAT || secrets.GITHUB_TOKEN }}
97+
BRANCH: ${{ steps.branch.outputs.name }}
98+
run: |
99+
git config user.name "github-actions[bot]"
100+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
101+
102+
git checkout -B "$BRANCH"
103+
git add -A
104+
git commit -m "feat: regenerate API client from OpenAPI spec"
105+
# Force push is safe: the regen branch is fully derived from main
106+
# plus this deterministic generation; any previous content is stale.
107+
git push --force origin "$BRANCH"
108+
109+
existing=$(gh pr list --head "$BRANCH" --state open --json number --jq '.[0].number // empty')
110+
if [ -z "$existing" ]; then
111+
gh pr create \
112+
--head "$BRANCH" \
113+
--title "feat: regenerate API client from OpenAPI spec" \
114+
--body "Automated regeneration via openapi-python-client from \`docs/v3/openapi3.yaml\` on frain-dev/convoy main. Hand-written webhook verify (\`src/convoy/utils/\`) is untouched by the sync script."
115+
echo "Opened PR for $BRANCH" >> "$GITHUB_STEP_SUMMARY"
116+
else
117+
echo "Updated existing PR #$existing" >> "$GITHUB_STEP_SUMMARY"
118+
fi

‎.github/workflows/speakeasy_generation.yaml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
name: Speakeasy SDK Generation (dormant)
2+
3+
# DORMANT: Python generation moved to openapi-python-client
4+
# (.github/workflows/sdk_generation.yaml) because the Speakeasy free tier
5+
# allows one generated SDK per workspace and convoy.js holds that slot.
6+
# This workflow is kept manual-only so switching back is a trigger change,
7+
# not a rebuild. Config lives on in .speakeasy/ and .genignore.
8+
9+
on:
10+
workflow_dispatch:
11+
inputs:
12+
force:
13+
description: Force SDK generation even if no changes are detected
14+
required: false
15+
default: "false"
16+
type: string
17+
feature_branch:
18+
description: Branch for SDK changes
19+
required: false
20+
type: string
21+
environment:
22+
description: Environment variables (e.g., TAG=branch-name)
23+
required: false
24+
type: string
25+
26+
# Serialize generations: overlapping cron/dispatch runs race on the same
27+
# branch/PR. Queue instead of cancel so a triggered regen is never dropped.
28+
concurrency:
29+
group: sdk-generation
30+
cancel-in-progress: false
31+
32+
permissions:
33+
contents: write
34+
pull-requests: write
35+
checks: write
36+
statuses: write
37+
38+
jobs:
39+
generate:
40+
# Pin to commit SHA for v15 (mutable tags can be retargeted).
41+
uses: speakeasy-api/sdk-generation-action/.github/workflows/workflow-executor.yaml@7276a3ae83eafb7fe37630fb1fefea3f2649debd # v15
42+
with:
43+
mode: pr
44+
force: ${{ inputs.force || 'false' }}
45+
feature_branch: ${{ inputs.feature_branch }}
46+
environment: ${{ inputs.environment }}
47+
secrets:
48+
# Prefer a PAT: PRs opened with GITHUB_TOKEN do not trigger
49+
# pull_request workflows, so verify CI would never run on them.
50+
github_access_token: ${{ secrets.SDK_BOT_PAT || secrets.GITHUB_TOKEN }}
51+
speakeasy_api_key: ${{ secrets.SPEAKEASY_API_KEY }}

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,6 @@ build
33
convoy_python.egg-info
44
dist/
55
__pycache__/
6-
*.pyc
6+
*.pyc
7+
.venv-proof/
8+
.ruff_cache/

‎.openapi-python-client.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# openapi-python-client config (active generator).
2+
# PyPI name stays convoy-python; imports stay `from convoy ...` so the
3+
# hand-written verify path (convoy/utils/webhook.py) keeps working.
4+
package_name_override: convoy
5+
project_name_override: convoy-python

‎MIGRATION.md‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,19 @@
1-
# convoy-python 1.x migration (Speakeasy)
1+
# convoy-python 1.x migration
22

33
## What changed
44

5-
- The **public HTTP API client** will be generated from Convoy's OpenAPI spec (`docs/v3/openapi3.yaml`) via [Speakeasy](https://www.speakeasy.com/).
6-
- **Webhook signature verification stays hand-written.** Generators do not own crypto. `src/convoy/utils/webhook.py` and the shared `test/signature-vectors.json` contract remain the source of truth for verify (see `.genignore`).
5+
- The **public HTTP API client** is generated from Convoy's OpenAPI spec (`docs/v3/openapi3.yaml`) via [openapi-python-client](https://github.com/openapi-generators/openapi-python-client).
6+
- **Webhook signature verification stays hand-written.** Generators do not own crypto. `src/convoy/utils/webhook.py` and the shared `test/signature-vectors.json` contract remain the source of truth for verify. The generation sync script (`scripts/generate.sh`) never touches `src/convoy/utils/`.
7+
8+
## Generator choice
9+
10+
Generation originally bootstrapped on Speakeasy, but the Speakeasy free tier allows one generated SDK per workspace and `convoy.js` holds that slot. The Speakeasy pipeline is kept **dormant** (`.speakeasy/`, `.genignore`, `.github/workflows/speakeasy_generation.yaml`) so the provider can be switched back without a rebuild; the active pipeline is openapi-python-client (`.github/workflows/sdk_generation.yaml`).
711

812
## Breaking change policy
913

10-
Shipping the Speakeasy client is an intentional **1.x** break from the hand-written `0.x` surfaces. Method shapes are **not** silently preserved.
14+
Shipping the generated client is an intentional **1.x** break from the hand-written `0.x` surfaces. Method shapes are **not** silently preserved.
1115

12-
1. This bootstrap PR wires Speakeasy, removes the deprecated hand-written HTTP client, and relocates verify to `src/convoy/utils/webhook.py` (inside the generated module tree, so `from convoy.utils.webhook import Webhook` keeps resolving — `moduleName: convoy` in `.speakeasy/gen.yaml`).
16+
1. This PR wires openapi-python-client generation; the hand-written HTTP client stays removed and verify lives at `src/convoy/utils/webhook.py` (inside the generated module tree, so `from convoy.utils.webhook import Webhook` keeps resolving — `package_name_override: convoy` in `.openapi-python-client.yml`).
1317
2. The first `sdk_generation.yaml` run opens a PR that adds the OpenAPI-generated client and publishes as `1.x`.
1418
3. Consumers pin `0.x` until they migrate call sites.
1519

@@ -28,8 +32,9 @@ if not webhook.verify_signature(payload, signature):
2832

2933
## Regenerating the API client
3034

31-
CI on `frain-dev/convoy` triggers Speakeasy when OpenAPI artifacts change. Locally (requires `SPEAKEASY_API_KEY`):
35+
CI on `frain-dev/convoy` dispatches `sdk_generation.yaml` when OpenAPI artifacts change. Locally:
3236

3337
```bash
34-
speakeasy run
38+
pip install openapi-python-client==0.29.0 ruff==0.15.22
39+
./scripts/generate.sh
3540
```

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,6 @@ Please see [CONTRIBUTING](CONTRIBUTING.MD) for details.
122122

123123
The MIT License (MIT). Please see [License File](LICENSE) for more information.
124124

125-
## Speakeasy-generated API client
125+
## Generated API client
126126

127-
The HTTP API client is generated from Convoy OpenAPI via Speakeasy. **Webhook signature verification remains hand-written** (`convoy/utils/webhook.py`) and is covered by shared `test/signature-vectors.json`. See [MIGRATION.md](./MIGRATION.md).
127+
The HTTP API client is generated from Convoy's OpenAPI spec via [openapi-python-client](https://github.com/openapi-generators/openapi-python-client). **Webhook signature verification remains hand-written** (`convoy/utils/webhook.py`) and is covered by shared `test/signature-vectors.json`. See [MIGRATION.md](./MIGRATION.md).

‎pyproject.toml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
[project]
2+
name = "convoy-python"
3+
version = "1.0.0a1"
4+
description = "Convoy Python SDK: OpenAPI-generated API client and hand-written webhook signature verification"
5+
readme = "README.md"
6+
license = { text = "MIT" }
7+
authors = [{ name = "Frain Inc." }]
8+
requires-python = ">=3.11"
9+
dependencies = [
10+
"httpx>=0.23.1,<0.29.0",
11+
"attrs>=22.2.0",
12+
]
13+
14+
[project.urls]
15+
Homepage = "https://github.com/frain-dev/convoy-python"
16+
Documentation = "https://getconvoy.io/docs"
17+
18+
[build-system]
19+
requires = ["setuptools>=68"]
20+
build-backend = "setuptools.build_meta"
21+
22+
# namespaces=true (the default) so `convoy` resolves both before the first
23+
# generation run (only convoy/utils exists) and after (generated __init__.py).
24+
[tool.setuptools.packages.find]
25+
where = ["src"]
26+
27+
[tool.setuptools.package-data]
28+
convoy = ["py.typed"]

‎scripts/generate.sh‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# Regenerate the API client from Convoy's OpenAPI spec with
5+
# openapi-python-client, then sync it into src/convoy/ without touching
6+
# hand-owned paths (webhook verify, py.typed).
7+
#
8+
# Requires: openapi-python-client, rsync, curl. Run from the repo root.
9+
10+
SPEC_URL="${SPEC_URL:-https://raw.githubusercontent.com/frain-dev/convoy/main/docs/v3/openapi3.yaml}"
11+
12+
tmp="$(mktemp -d)"
13+
trap 'rm -rf "$tmp"' EXIT
14+
15+
curl -fsSL "$SPEC_URL" -o "$tmp/openapi3.yaml"
16+
17+
# --meta none emits only the package contents; project metadata is hand-owned
18+
# in pyproject.toml.
19+
# --fail-on-warning: a warning means the generator skipped part of the spec;
20+
# never mirror a partial client into src/convoy (rsync --delete would drop
21+
# previously generated modules).
22+
openapi-python-client generate \
23+
--path "$tmp/openapi3.yaml" \
24+
--config .openapi-python-client.yml \
25+
--meta none \
26+
--fail-on-warning \
27+
--output-path "$tmp/gen"
28+
29+
# --delete keeps src/convoy an exact mirror of generator output; excluded
30+
# paths are hand-written and never created or removed by this script.
31+
# .ruff_cache is a side effect of the generator's post-processing formatter.
32+
rsync -a --delete \
33+
--exclude 'utils/' \
34+
--exclude 'py.typed' \
35+
--exclude '.ruff_cache/' \
36+
"$tmp/gen/" src/convoy/
37+
38+
echo "Generated client synced into src/convoy/"

0 commit comments

Comments
 (0)