From 0f618869c09e8412b4bfc36c99e79e80ffd5460e Mon Sep 17 00:00:00 2001 From: bornunique911 Date: Thu, 23 Jul 2026 21:47:03 +0530 Subject: [PATCH 1/3] docs: clarify local contributor setup for upstream sync and MyOpenCRE (#599) [ci skip] --- README.md | 29 ++++++++++++++++++++++++++--- docs/developmentSetup.md | 33 ++++++++++++++++++++++++++++++++- 2 files changed, 58 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index e08a12127..a8bafb0e1 100644 --- a/README.md +++ b/README.md @@ -68,12 +68,17 @@ Install dependencies: make install ``` -Download the latest CRE graph from upstream by running: +Create the local schema and download the latest CRE graph from upstream: ```bash +make migrate-upgrade make upstream-sync ``` +For contributors, this is the supported local data bootstrap path. You do +**not** need access to the internal OpenCRE Google Sheet to work on the +project. + You can precompute local gap-analysis cache after imports with: ```bash @@ -109,18 +114,32 @@ Notes: - `--csv` is required when using `--export`. - This mode exports report data from the OpenCRE API and does not mutate the local DB. -To download a remote CRE spreadsheet locally you can run: +If you need to review a remote spreadsheet locally, you can run: ```bash python cre.py --review --from_spreadsheet ``` -To add a remote spreadsheet to your local database you can run: +If you need to add a remote spreadsheet to your local database, you can run: ```bash python cre.py --add --from_spreadsheet ``` +Those spreadsheet commands are not required for ordinary contribution and local +development. If you want to import your own mappings locally, use MyOpenCRE or +the CSV import endpoint with imports enabled: + +```bash +export CRE_ALLOW_IMPORT=true +make dev-flask +``` + +Then use: + +- `GET /rest/v1/cre_csv` to download a template +- `POST /rest/v1/cre_csv_import` to upload your CSV + To run the web application for development you can run: ```bash @@ -239,9 +258,13 @@ xcode-select --install Sync upstream CRE data (requires internet access): ```bash +make migrate-upgrade make upstream-sync ``` +You do not need Google Sheet access for this workflow. `make upstream-sync` +pulls the public OpenCRE graph into your local database. + Then start the local server: ```bash diff --git a/docs/developmentSetup.md b/docs/developmentSetup.md index b51909c51..aeb8e6858 100644 --- a/docs/developmentSetup.md +++ b/docs/developmentSetup.md @@ -109,7 +109,38 @@ If the tests pass, the project should be operational. You can run tests with ### Import the database -You can run `make migrate-upgrade` first and then `make upstream-sync` +For local development, you do **not** need access to the OpenCRE Google Sheet. +That spreadsheet is part of the maintainer workflow for bulk standards imports. + +The supported contributor workflow is: + +1. create the local database schema +2. sync the public OpenCRE graph from upstream +3. import your own mappings locally through MyOpenCRE if needed + +Run: + +```bash +make migrate-upgrade +make upstream-sync +``` + +`make upstream-sync` downloads the OpenCRE graph from the public API into your +local `standards_cache.sqlite`. + +If you want to map your own standard locally, enable imports and use MyOpenCRE +or the CSV import endpoint: + +```bash +export CRE_ALLOW_IMPORT=true +make dev-flask +``` + +Then you can: + +- open the local MyOpenCRE UI +- download a CSV template from `GET /rest/v1/cre_csv` +- upload your CSV to `POST /rest/v1/cre_csv_import` ## Running locally From df44de34ab848358a9025a81a3e4f3fd4f045e0a Mon Sep 17 00:00:00 2001 From: bornunique911 Date: Sun, 2 Aug 2026 10:51:25 +0530 Subject: [PATCH 2/3] docs: label spreadsheet import as maintainer-only (#599) [ci skip] --- README.md | 12 ++++-------- docs/developmentSetup.md | 7 +++++++ 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index a8bafb0e1..b53af634b 100644 --- a/README.md +++ b/README.md @@ -114,13 +114,8 @@ Notes: - `--csv` is required when using `--export`. - This mode exports report data from the OpenCRE API and does not mutate the local DB. -If you need to review a remote spreadsheet locally, you can run: - -```bash -python cre.py --review --from_spreadsheet -``` - -If you need to add a remote spreadsheet to your local database, you can run: +Maintainer-only: if you need to add a remote spreadsheet to your local +database, you can run: ```bash python cre.py --add --from_spreadsheet @@ -132,13 +127,14 @@ the CSV import endpoint with imports enabled: ```bash export CRE_ALLOW_IMPORT=true -make dev-flask +make start-containers ``` Then use: - `GET /rest/v1/cre_csv` to download a template - `POST /rest/v1/cre_csv_import` to upload your CSV +- send the multipart file field as `cre_csv` To run the web application for development you can run: diff --git a/docs/developmentSetup.md b/docs/developmentSetup.md index aeb8e6858..18a2bccef 100644 --- a/docs/developmentSetup.md +++ b/docs/developmentSetup.md @@ -141,6 +141,13 @@ Then you can: - open the local MyOpenCRE UI - download a CSV template from `GET /rest/v1/cre_csv` - upload your CSV to `POST /rest/v1/cre_csv_import` +- send the multipart file field as `cre_csv` + +Maintainer-only spreadsheet import example: + +```bash +python cre.py --add --from_spreadsheet +``` ## Running locally From 002d10854ffb15f3edc00e644efb7346db348ef2 Mon Sep 17 00:00:00 2001 From: bornunique911 Date: Sun, 2 Aug 2026 11:32:07 +0530 Subject: [PATCH 3/3] docs: align CSV import curl example with local API usage (#599) [ci skip] --- README.md | 9 ++++++++- docs/developmentSetup.md | 9 ++++++++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b53af634b..d61916f0a 100644 --- a/README.md +++ b/README.md @@ -133,9 +133,16 @@ make start-containers Then use: - `GET /rest/v1/cre_csv` to download a template -- `POST /rest/v1/cre_csv_import` to upload your CSV +- `POST /rest/v1/cre_csv_import` to upload your CSV after starting the local API - send the multipart file field as `cre_csv` +Example, once `make dev-flask` is running: + +```bash +curl -X POST http://localhost:5000/rest/v1/cre_csv_import \ + -F cre_csv=@my_mappings.csv +``` + To run the web application for development you can run: ```bash diff --git a/docs/developmentSetup.md b/docs/developmentSetup.md index 18a2bccef..ad1d12982 100644 --- a/docs/developmentSetup.md +++ b/docs/developmentSetup.md @@ -140,9 +140,16 @@ Then you can: - open the local MyOpenCRE UI - download a CSV template from `GET /rest/v1/cre_csv` -- upload your CSV to `POST /rest/v1/cre_csv_import` +- upload your CSV to `POST /rest/v1/cre_csv_import` after starting the local API - send the multipart file field as `cre_csv` +Example, once `make dev-flask` is running: + +```bash +curl -X POST http://localhost:5000/rest/v1/cre_csv_import \ + -F cre_csv=@my_mappings.csv +``` + Maintainer-only spreadsheet import example: ```bash