Skip to content

addIoTDatabaseIndex races the IoT Agent and kills the services script - #17

Closed
kzangeli wants to merge 1 commit into
FIWARE:NGSI-LDfrom
kzangeli:fix/iot-db-index-race
Closed

kzangeli wants to merge 1 commit into
FIWARE:NGSI-LDfrom
kzangeli:fix/iot-db-index-race

Conversation

@kzangeli

Copy link
Copy Markdown

This repo's own CI hit it on 2026-09-19, in the Run Scorpio step of the PR check for #16:

MongoServerError: Collection iotagentjson.devices already exists.
Adding appropriate MongoDB indexes for IoT-Agent  ...
##[error]Process completed with exit code 1.

It is intermittent — the identical commit passed on re-run — so it shows up as a mystery red rather than as a bug.

The window

addIoTDatabaseIndex does:

db.getCollectionNames().forEach(c=>db[c].drop());
db.createCollection("devices");
...

By the time it runs, the IoT Agent container is already up and connected to the same database — it is started by the same docker compose up. If the agent creates devices between the drop loop and the createCollection call, the call fails.

On mongo 6.0, which is what MONGO_DB_VERSION pins, db.createCollection on an existing collection throws NamespaceExists; mongosh then exits 1; and services runs under set -e, so the whole invocation dies — after the stack is up, before the tutorial has run a single request.

(Worth knowing for later: mongo 8 made createCollection return {ok:1} in this case, so bumping MONGO_DB_VERSION would hide this rather than fix it.)

The fix

The two createCollection calls are redundant — createIndex creates the collection implicitly when it is missing. Dropping them closes the window with no other change.

Verified on mongo 6.0:

current amended
clean database exit 0 exit 0
run twice in a row exit 1 exit 0
collection created concurrently mid-script exit 1 exit 0

and the resulting indexes are identical either way:

devices: {_id}, {_id.service,_id.id,_id.type}, {_id.type}, {_id.id}
groups:  {_id}, {_id.resource,_id.apikey,_id.service}, {_id.type}

Scope

Eight other tutorials carry the same function verbatim and are equally exposed: Big-Data-Spark, Concise-Format, IoT-Agent, IoT-Agent-JSON, IoT-Sensors, Short-Term-History, Subscriptions, Time-Series-Data. Happy to send the same one-hunk change to each if you would rather not sync it by hand — just say.

Independent of #16 — different file.

🤖 Generated with Claude Code

The Scorpio step of this repo's own CI failed on 2026-09-19 with

    MongoServerError: Collection iotagentjson.devices already exists.
    ##[error]Process completed with exit code 1.

`addIoTDatabaseIndex` drops every collection in `iotagentjson` and then
re-creates `devices` and `groups`. The IoT Agent container is already up and
connected by then - it is started by the same `docker compose up` - so it can
create `devices` itself in the window between the drop loop and the
`createCollection` call.

On mongo 6.0, which is what `MONGO_DB_VERSION` pins, `db.createCollection` on
an existing collection throws `NamespaceExists` and `mongosh` exits 1. The
script runs under `set -e`, so the whole `services` invocation dies - after
the stack is up, before the tutorial has done anything.

The two `createCollection` calls are redundant: `createIndex` creates the
collection implicitly if it is missing. Removing them makes the block
idempotent and removes the window entirely, with no other change - verified on
mongo 6.0 that the resulting index sets are identical:

    devices: {_id}, {_id.service,_id.id,_id.type}, {_id.type}, {_id.id}
    groups:  {_id}, {_id.resource,_id.apikey,_id.service}, {_id.type}

and that the amended block exits 0 when run twice in a row and when the
collection has been created concurrently in between - the two cases that make
the current version exit 1.

Note this is intermittent: the same commit passed on re-run. The eight other
tutorials carrying the same function are equally exposed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jason-fox

Copy link
Copy Markdown
Member

Fixed with a420da1

@jason-fox jason-fox closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants