diff --git a/README.md b/README.md index e08a12127..d61916f0a 100644 --- a/README.md +++ b/README.md @@ -68,12 +68,17 @@ Install dependencies: make install ``` -Download the latest CRE graph from upstream by running: +Create the local schema and download the latest CRE graph from upstream: ```bash +make migrate-upgrade make upstream-sync ``` +For contributors, this is the supported local data bootstrap path. You do +**not** need access to the internal OpenCRE Google Sheet to work on the +project. + You can precompute local gap-analysis cache after imports with: ```bash @@ -109,16 +114,33 @@ Notes: - `--csv` is required when using `--export`. - This mode exports report data from the OpenCRE API and does not mutate the local DB. -To download a remote CRE spreadsheet locally you can run: +Maintainer-only: if you need to add a remote spreadsheet to your local +database, you can run: ```bash -python cre.py --review --from_spreadsheet +python cre.py --add --from_spreadsheet ``` -To add a remote spreadsheet to your local database you can run: +Those spreadsheet commands are not required for ordinary contribution and local +development. If you want to import your own mappings locally, use MyOpenCRE or +the CSV import endpoint with imports enabled: ```bash -python cre.py --add --from_spreadsheet +export CRE_ALLOW_IMPORT=true +make start-containers +``` + +Then use: + +- `GET /rest/v1/cre_csv` to download a template +- `POST /rest/v1/cre_csv_import` to upload your CSV after starting the local API +- send the multipart file field as `cre_csv` + +Example, once `make dev-flask` is running: + +```bash +curl -X POST http://localhost:5000/rest/v1/cre_csv_import \ + -F cre_csv=@my_mappings.csv ``` To run the web application for development you can run: @@ -239,9 +261,13 @@ xcode-select --install Sync upstream CRE data (requires internet access): ```bash +make migrate-upgrade make upstream-sync ``` +You do not need Google Sheet access for this workflow. `make upstream-sync` +pulls the public OpenCRE graph into your local database. + Then start the local server: ```bash diff --git a/docs/developmentSetup.md b/docs/developmentSetup.md index b51909c51..ad1d12982 100644 --- a/docs/developmentSetup.md +++ b/docs/developmentSetup.md @@ -109,7 +109,52 @@ If the tests pass, the project should be operational. You can run tests with ### Import the database -You can run `make migrate-upgrade` first and then `make upstream-sync` +For local development, you do **not** need access to the OpenCRE Google Sheet. +That spreadsheet is part of the maintainer workflow for bulk standards imports. + +The supported contributor workflow is: + +1. create the local database schema +2. sync the public OpenCRE graph from upstream +3. import your own mappings locally through MyOpenCRE if needed + +Run: + +```bash +make migrate-upgrade +make upstream-sync +``` + +`make upstream-sync` downloads the OpenCRE graph from the public API into your +local `standards_cache.sqlite`. + +If you want to map your own standard locally, enable imports and use MyOpenCRE +or the CSV import endpoint: + +```bash +export CRE_ALLOW_IMPORT=true +make dev-flask +``` + +Then you can: + +- open the local MyOpenCRE UI +- download a CSV template from `GET /rest/v1/cre_csv` +- upload your CSV to `POST /rest/v1/cre_csv_import` after starting the local API +- send the multipart file field as `cre_csv` + +Example, once `make dev-flask` is running: + +```bash +curl -X POST http://localhost:5000/rest/v1/cre_csv_import \ + -F cre_csv=@my_mappings.csv +``` + +Maintainer-only spreadsheet import example: + +```bash +python cre.py --add --from_spreadsheet +``` ## Running locally