fix: run DEEIX on Nile Postgres without changing other databases by buddhism5080 · Pull Request #780 · DEEIX-AI/DEEIX-Chat

buddhism5080 · 2026-09-24T07:21:39Z

Summary

Hosts under *.thenile.dev take Nile-safe SQL directly. Ordinary Postgres, including a database named nile on another host, is unchanged.

Skip COMMENT ON. Nile returns command tag COMMENT unhandled.
Rewrite UTF-8 to UTF8 and send parameters with the simple protocol, so an empty string is not delivered as NULL.
Replace the slow, empty information_schema constraint join with pg_constraint. Column metadata and format_type lookups read pg_attribute by regclass.
Turn partial unique indexes whose WHERE clause contains AND into expression indexes with the same uniqueness rule.
Use the preinstalled vector extension instead of CREATE EXTENSION.
Store the 4000-dimension HNSW candidate in embedding_hnsw and index that column. Nile rejects functions inside an index expression and rejects CREATE INDEX CONCURRENTLY. The original embedding column is still used for exact reranking.
Migrate Nile tables eight at a time. Other databases stay serial.

摘要

只在数据库地址是 *.thenile.dev 时改写 SQL。普通 Postgres 保持原语句。

Nile 会拒绝 COMMENT ON、CREATE EXTENSION、CREATE INDEX CONCURRENTLY，以及索引表达式里的函数。带 AND 的部分唯一索引会改成语义相同的表达式索引。向量仍按模型原始维度存在 embedding，候选用单独的 halfvec(4000) 列建 HNSW。

Test plan

go test ./internal/infra/persistence/postgres/ ./internal/infra/persistence/schema/ ./internal/infra/persistence/vectorutil/ ./internal/infra/persistence/postgres/conversation/ ./internal/infra/persistence/postgres/memory/
Compared the replacement catalog queries with the original statements on a Nile database
Boot against a Nile host and confirm there is no SQLSTATE during migration

github-actions · 2026-10-29T07:27:51Z

This pull request has had no activity for 7 days. It will be closed in 7 days if there is no further activity.

Gateways that reject the COMMENT command tag fail startup because GORM and the baseline migration treat COMMENT ON as fatal. SCHEMA_COMMENTS=false skips those statements and clears parsed field comments before migrate.

Nile reports client_encoding as UTF-8, so pgx refuses the simple-protocol probe GORM uses during AutoMigrate. current_database() also differs from information_schema.table_catalog, which made AutoMigrate add existing columns.

Only use current_catalog when it differs from current_database() and information_schema is actually catalogued under that other name. UTF-8 client_encoding is rewritten only when the server does not report UTF8.

Ordinary Postgres still creates the partial unique index. A gateway that rejects AND in an index WHERE clause gets the equivalent expression index, which is NULL unless the file is active, not deleted, and has a sha256.

The usage-reference index also puts AND in its WHERE clause. Ordinary Postgres still creates that partial index. The same gateway error now creates the equivalent expression index instead of stopping startup.

Hosts under thenile.dev skip COMMENT, normalize UTF-8, and replace the two slow GORM constraint joins plus the AND partial unique indexes before they run. Other Postgres hosts keep the original statements.

Nile hosts skip COMMENT ON based on the database address. There is no environment variable or config flag for it.

Nile's extended protocol turns a bound empty string into NULL, so the vendor seed insert fails the icon NOT NULL check and the process restarts.

The gateway rejects the CREATE EXTENSION command tag even when vector is already installed. On a Nile host, read pg_extension and continue the vector baseline.

GORM's format_type and information_schema.columns queries were the statements still taking more than a second per table. Read the same values from pg_attribute by regclass.

Per-table catalog lookups are independent once foreign keys are disabled and missing tables already exist. Nile runs eight of them at once. Other databases stay serial.

The gateway rejects CREATE INDEX CONCURRENTLY and DROP INDEX CONCURRENTLY. The HNSW definition stays the same; only the command form changes on a Nile host.

The gateway rejects functions inside an index expression, so the HNSW index cannot use vector_dims or subvector. Nile stores the same 4000-dimension candidate in embedding_hnsw and indexes that column. The original vector remains for exact reranking. Other databases keep the expression index.

The pg_class join used to read file_chunks.embedding took about 1.4s. On Nile, resolve the table with to_regclass instead. Ordinary Postgres keeps the original join.

The workspace quality check rejects interface{} in production Go.

Saving embedding settings waited on a catalog scan that does not return on Nile. Look the HNSW index up by name instead.

The pg_class join still took 1.6s per existing index. Matching pg_index.indexrelid to to_regclass returns in about 0.23s.

Upstream renamed this directory from package db to package postgres. The Nile files still declared package db, so they were a second package in the same directory and were not the package the repositories import.

buddhism5080 force-pushed the feat/nile-postgres branch 4 times, most recently from fa387b3 to 5c4cb0e Compare September 30, 2026 04:03

buddhism5080 force-pushed the feat/nile-postgres branch 2 times, most recently from 1b01275 to 0c05e93 Compare October 10, 2026 02:20

buddhism5080 force-pushed the feat/nile-postgres branch 2 times, most recently from ec2406d to a8f9c2b Compare October 13, 2026 16:25

buddhism5080 force-pushed the feat/nile-postgres branch from a8f9c2b to 8220fd1 Compare October 21, 2026 14:56

github-actions Bot added the stale label Oct 29, 2026

buddhism5080 added 18 commits October 30, 2026 02:56

fix: allow disabling PostgreSQL schema comments

dc8b115

Gateways that reject the COMMENT command tag fail startup because GORM and the baseline migration treat COMMENT ON as fatal. SCHEMA_COMMENTS=false skips those statements and clears parsed field comments before migrate.

fix: keep ordinary Postgres on current_database()

a618308

Only use current_catalog when it differs from current_database() and information_schema is actually catalogued under that other name. UTF-8 client_encoding is rewritten only when the server does not report UTF8.

fix: cover the billing partial unique index Nile rejects

7f47d98

The usage-reference index also puts AND in its WHERE clause. Ordinary Postgres still creates that partial index. The same gateway error now creates the equivalent expression index instead of stopping startup.

fix: use Nile-safe SQL directly when the database host is Nile

777e0ab

Hosts under thenile.dev skip COMMENT, normalize UTF-8, and replace the two slow GORM constraint joins plus the AND partial unique indexes before they run. Other Postgres hosts keep the original statements.

fix: drop the SCHEMA_COMMENTS setting

b04c63a

Nile hosts skip COMMENT ON based on the database address. There is no environment variable or config flag for it.

fix: send Nile text parameters in the simple protocol

b0f1ce6

Nile's extended protocol turns a bound empty string into NULL, so the vendor seed insert fails the icon NOT NULL check and the process restarts.

fix: use Nile's installed pgvector instead of CREATE EXTENSION

856c53d

The gateway rejects the CREATE EXTENSION command tag even when vector is already installed. On a Nile host, read pg_extension and continue the vector baseline.

fix: speed up Nile column catalog lookups

054d6f8

GORM's format_type and information_schema.columns queries were the statements still taking more than a second per table. Read the same values from pg_attribute by regclass.

fix: migrate Nile tables concurrently

87c45d6

Per-table catalog lookups are independent once foreign keys are disabled and missing tables already exist. Nile runs eight of them at once. Other databases stay serial.

fix: build Nile vector indexes without CONCURRENTLY

dc67355

The gateway rejects CREATE INDEX CONCURRENTLY and DROP INDEX CONCURRENTLY. The HNSW definition stays the same; only the command form changes on a Nile host.

fix: look up Nile vector column types by regclass

61e9ab9

The pg_class join used to read file_chunks.embedding took about 1.4s. On Nile, resolve the table with to_regclass instead. Ordinary Postgres keeps the original join.

fix: use any instead of interface{} in Nile SQL rewrite

43172e0

The workspace quality check rejects interface{} in production Go.

fix: check Nile vector indexes without pg_get_indexdef

80cc175

Saving embedding settings waited on a catalog scan that does not return on Nile. Look the HNSW index up by name instead.

fix: look up Nile HNSW indexes by regclass

33cee1c

The pg_class join still took 1.6s per existing index. Matching pg_index.indexrelid to to_regclass returns in about 0.23s.

fix: compile Nile helpers as package postgres

9348a13

Upstream renamed this directory from package db to package postgres. The Nile files still declared package db, so they were a second package in the same directory and were not the package the repositories import.

buddhism5080 force-pushed the feat/nile-postgres branch from 8220fd1 to 9348a13 Compare October 29, 2026 18:58

github-actions Bot removed the stale label Oct 30, 2026

Navigation Menu

Repository navigation

fix: run DEEIX on Nile Postgres without changing other databases - #780

fix: run DEEIX on Nile Postgres without changing other databases#780
buddhism5080 wants to merge 18 commits into
DEEIX-AI:devfrom
buddhism5080:feat/nile-postgres

buddhism5080 commented Sep 24, 2026

Uh oh!

github-actions Bot commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

buddhism5080 commented Sep 24, 2026

Summary

摘要

Test plan

Uh oh!

github-actions Bot commented Oct 29, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant