Docker fails to start marquez

[pankajSrivastava54]

I have been trying every single branch but always getting:

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "/opt/marquez/wait-for-it.sh": stat /opt/marquez/wait-for-it.sh: no such file or directory: unknown

[thijs-s]

Just in case anyone is looking for the solution, we fixed it here https://github.com/ilum-cloud/marquez.

[lc-spxl]

I got this after looking into the repository, it's more compact but yet the same as original.

However, marquez seems not happy to start, just run the migration and exits. Stripping from entrypoint.sh this is the command that launch it java ${JAVA_OPTS} -jar marquez-api-0.53.0.jar server marquez.yaml

INFO  [2025-09-19 13:50:12,265] org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory: Repair of failed migration in Schema History table "public"."flyway_schema_history" not necessary as table doesn't exist.
INFO  [2025-09-19 13:50:12,283] org.flywaydb.core.internal.command.DbRepair: Successfully repaired schema history table "public"."flyway_schema_history" (execution time 00:00.018s).
INFO  [2025-09-19 13:50:12,284] marquez.db.DbMigration: Successfully repaired database.
INFO  [2025-09-19 13:50:12,284] marquez.MarquezApp: Stopping app...

networks:
  marquez_net:
  
volumes:
  marquez-data:
  db-data:
  opensearch-data:

services:

  web:
    image: "ilum/marquez-web:0.53.0"
    environment:
      - MARQUEZ_HOST=api
      - MARQUEZ_PORT=5000
      - WEB_PORT=5002
    networks:
      - marquez_net
    depends_on:
      api:
        condition: service_started

  api:
    image: "ilum/marquez:0.53.0"
    environment:

      - MARQUEZ_PORT=5000
      - MARQUEZ_ADMIN_PORT=5001
      - MARQUEZ_CONFIG=/usr/src/app/marquez.yaml
      
      - POSTGRES_HOST=db
      - POSTGRES_PORT=5432
      - POSTGRES_DB=marquez
      - POSTGRES_USER=marquez
      - POSTGRES_PASSWORD=marquez

      - SEARCH_ENABLED=true
      - SEARCH_HOST=opensearch
      - SEARCH_PORT=9200

    volumes:
      - ./config/marquez/marquez.yaml:/usr/src/app/marquez.yaml
      - marquez-data:/opt/marquez

    depends_on:
      db:
        condition: service_healthy
      opensearch:
        condition: service_started

    # entrypoint: "./entrypoint.sh"

    networks:
      - marquez_net
    expose:
      - 5000
      - 5001
    restart: unless-stopped

  db:
    image: postgres:17.4
    networks:
      - marquez_net
    # ports:
    #   - "${POSTGRES_PORT}:${POSTGRES_PORT}"
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
      - MARQUEZ_DB=marquez
      - MARQUEZ_USER=marquez
      - MARQUEZ_PASSWORD=marquez
    volumes:
      - ./config/postgres/postgresql.conf:/etc/postgresql/postgresql.conf:ro
      - ./config/postgres/db-init.sh:/docker-entrypoint-initdb.d/db-init.sh:ro
      - db-data:/var/lib/postgresql/data
    command: ["postgres", "-c", "config_file=/etc/postgresql/postgresql.conf"]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  opensearch:
    image: opensearchproject/opensearch:2.5.0
    hostname: opensearch
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    environment:
      - cluster.name=opensearch-cluster
      - node.name=opensearch
      - discovery.seed_hosts=opensearch
      - cluster.initial_cluster_manager_nodes=opensearch
      - bootstrap.memory_lock=true
      - plugins.security.ssl.http.enabled=false
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
      - OPENSEARCH_PASSWORD=admin
    volumes:
      - opensearch-data:/usr/share/opensearch/data
    cap_add:
      - IPC_LOCK
    # ports:
    #   - "9200:9200"
    #   - "9300:9300"
    networks:
      - marquez_net

[thijs-s]

we will look into this.

[thijs-s]

@lc-spxl can you try running pure ./docker/up.sh ? I will check your config tomorrow.

[lc-spxl]

Ok thanks but I am investigating and I guess that Flyway is exiting here, so it's still a config issue. Will report more

https://github.com/MarquezProject/marquez/blob/main/api/src/main/java/marquez/MarquezApp.java#L120

[lc-spxl]

this is the up to date db-init.sh I am using and now migrations works.

#!/bin/bash
#
# Copyright 2018-2023 contributors to the Marquez project
# SPDX-License-Identifier: Apache-2.0
#
# Usage: $ ./init-db.sh

set -eu

psql -v ON_ERROR_STOP=1 --username "${POSTGRES_USER}" > /dev/null <<-EOSQL
  CREATE USER ${MARQUEZ_USER};
  ALTER USER ${MARQUEZ_USER} WITH PASSWORD '${MARQUEZ_PASSWORD}';
  CREATE DATABASE ${MARQUEZ_DB};
  GRANT ALL PRIVILEGES ON DATABASE ${MARQUEZ_DB} TO ${MARQUEZ_USER};
  ## those lines fixes it
  GRANT ALL ON SCHEMA public TO ${MARQUEZ_USER};
  ALTER DATABASE ${MARQUEZ_DB} SET search_path TO public;

EOSQL

Could I drop opensearch and use the new simple search endpoint now ?

ps Thanks for the work on your branch and the new release

[thijs-s]

Hi @lc-spxl I apologize for the delay. I somehow overlooked your message.

Do you need any more help? I'm just finishing the alternative "full" search, which returns the full object instead of just a simple entity.

So we will have two options:

1. Simple, which will give you name+id of the entity.
2. Full. Full object so you could use to show things like below.