Skip to content

Commit 78764fd

Browse files
authored
BRICKS-43: Store File API, skills/memories management, conditional loading (#37)
* feat: store file API, skills/memories management, conditional loading via agent namespace - Add Store File API (GET/PUT/DELETE /api/v1/store/files) with LangGraphStoreFileRepository - Add _prepare_agent_namespace: copies selected skills/memories to /agents/{name}/ namespace - Add skill usage tracking endpoint (GET /api/v1/store/skills/{name}/usage) - Remove MiddlewareType, BackendType.FILESYSTEM/COMPOSITE/STATE, root_dir, store_backend - Upgrade deepagents 0.6.12 + langgraph-checkpoint-postgres + psycopg[binary] - Fix StoreBackend deprecated pattern + AsyncPostgresStore CM reference leak - Add backward compat: strip deprecated fields from old YAMLs - Backend: 449 tests pass, SonarQube clean, Trivy 0 vulns * fix: address PR review — body size limit, adelete safety, asearch limit, store lock - PUT /api/v1/store/files: max 10MB content (Field max_length=10_000_000) - _prepare_agent_namespace: try/except on adelete (prevent crash on stale files) - asearch limit 100 → 1000 (prevent silent truncation) - _get_shared_store: asyncio.Lock (prevent concurrent pool creation) * fix: address review round 2 — remove duplicate, log stacktrace, fix test import - yaml_config/adapter.py: remove duplicate _DEPRECATED_BACKEND_FIELDS declaration - factory.py: logger.warning → logger.exception in _prepare_agent_namespace cleanup (keep stacktrace) - test_store_routes.py: import real StoreFileNotFoundError from src.domain.errors.store_file instead of local redefinition * feat: store preview endpoint, path traversal rejection - New endpoint GET /api/v1/store/files/previews?prefix=&chars= returns path+preview (eliminates N+1) - StoreFilePreview dataclass + list_files_with_preview on port/adapter (free — uses asearch values) - ListStoreFilePreviewsUseCase + dependency provider - _normalize_path: reject paths containing '..' (defense in depth)
1 parent 3c7a956 commit 78764fd

27 files changed

Lines changed: 2151 additions & 260 deletions

CONTRIBUTING.md

Lines changed: 65 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ src/
1515
ports/ # Abstract interfaces (AgentRunner, ThreadRepository, AgentConfigLoader)
1616
exceptions.py # Domain-specific exception hierarchy
1717
application/ # Use cases that orchestrate domain logic. Depends only on domain.
18-
use_cases/ # SendMessage, StreamMessage, HITL decisions, thread management
18+
use_cases/ # SendMessage, StreamMessage, HITL decisions, thread management, store file management
1919
requests/ # Pydantic request models for the API layer
20-
routes/ # FastAPI route handlers (thin layer: validate input, call use case, return response)
20+
routes/ # FastAPI route handlers (health, threads, chat, trace, agents, store, websocket)
2121
infrastructure/ # Concrete implementations of domain ports
2222
deepagent/ # LangGraph Deep Agent adapter + factory
2323
yaml_config/ # YAML config file loader
@@ -95,8 +95,8 @@ Key points:
9595
- Validation includes:
9696
- `name` must be 1-100 characters.
9797
- `system_prompt` and `system_prompt_file` are mutually exclusive (enforced by `@model_validator`).
98-
- `middleware` values must match the `MiddlewareType` enum.
99-
- `backend.type` must match the `BackendType` enum.
98+
- `backend.type` must match the `BackendType` enum (`state` or `store`).
99+
- `backend.store_backend` and `backend.checkpoint_backend` must be `"memory"` or `"postgres"`.
100100
- `hitl.rules` values are either `bool` or `InterruptRule` objects.
101101
- `subagents` entries require `name` and `description`.
102102

@@ -108,45 +108,6 @@ uv run python -m src schema > agent-config-schema.json
108108

109109
---
110110

111-
## How to Add a New Middleware
112-
113-
### 1. Add a value to the `MiddlewareType` enum
114-
115-
In `src/domain/entities/agent_config.py`:
116-
117-
```python
118-
class MiddlewareType(StrEnum):
119-
TODO_LIST = "todo_list"
120-
FILESYSTEM = "filesystem"
121-
SUB_AGENT = "sub_agent"
122-
MY_MIDDLEWARE = "my_middleware" # Add your new type
123-
```
124-
125-
### 2. Register it in the factory
126-
127-
In `src/infrastructure/deepagent/factory.py`, add the mapping:
128-
129-
```python
130-
from my_package import MyMiddleware
131-
132-
MIDDLEWARE_MAP: dict[MiddlewareType, type] = {
133-
MiddlewareType.TODO_LIST: FilesystemMiddleware,
134-
MiddlewareType.FILESYSTEM: FilesystemMiddleware,
135-
MiddlewareType.SUB_AGENT: SubAgentMiddleware,
136-
MiddlewareType.MY_MIDDLEWARE: MyMiddleware, # Register here
137-
}
138-
```
139-
140-
### 3. Use it in YAML
141-
142-
```yaml
143-
name: my-agent
144-
middleware:
145-
- my_middleware
146-
```
147-
148-
---
149-
150111
## How to Add a New Backend
151112

152113
### 1. Add a value to the `BackendType` enum
@@ -157,8 +118,6 @@ In `src/domain/entities/agent_config.py`:
157118
class BackendType(StrEnum):
158119
STATE = "state"
159120
STORE = "store"
160-
FILESYSTEM = "filesystem"
161-
COMPOSITE = "composite"
162121
MY_BACKEND = "my_backend" # Add your new type
163122
```
164123

@@ -171,14 +130,10 @@ def _resolve_backend(config: AgentConfig):
171130
match config.backend.type:
172131
case BackendType.STATE:
173132
return None
174-
case BackendType.FILESYSTEM:
175-
return FilesystemBackend(root_dir=config.backend.root_dir or "./workspace")
176133
case BackendType.STORE:
177-
return lambda rt: StoreBackend(rt)
134+
return lambda rt: StoreBackend(store=store, namespace=lambda r: ("filesystem",))
178135
case BackendType.MY_BACKEND:
179-
return MyBackend(config.backend.root_dir) # Your implementation
180-
case BackendType.COMPOSITE:
181-
return None
136+
return MyBackend(config.backend) # Your implementation
182137
```
183138

184139
### 3. Use it in YAML
@@ -187,11 +142,69 @@ def _resolve_backend(config: AgentConfig):
187142
name: my-agent
188143
backend:
189144
type: my_backend
190-
root_dir: "./data"
145+
store_backend: memory
146+
checkpoint_backend: memory
191147
```
192148
193149
---
194150
151+
## Store File API
152+
153+
Files in the LangGraph store (skills, memories, any text blob) are managed through a dedicated set of use cases, a domain port, and an infrastructure adapter, following the same hexagonal pattern as the rest of the codebase.
154+
155+
### Routes (`src/application/routes/store.py`)
156+
157+
| Method | Path | Handler |
158+
|---|---|---|
159+
| `GET` | `/api/v1/store/files` | `list_store_files` (optional `prefix` query param) |
160+
| `GET` | `/api/v1/store/files/{path:path}` | `get_store_file` |
161+
| `PUT` | `/api/v1/store/files/{path:path}` | `put_store_file` (body: `StoreFilePutRequest`) |
162+
| `DELETE` | `/api/v1/store/files/{path:path}` | `delete_store_file` |
163+
164+
Response DTOs: `StoreFileResponse` (`path`, `content`) and `StoreFilePutRequest` (`content`). The `{path:path}` converter allows slashes in the path segment. A missing file on `GET` raises `StoreFileNotFoundError` (`src/domain/errors/store_file.py`), resulting in a `404`.
165+
166+
### Use Cases (`src/application/use_cases/manage_store_file.py`)
167+
168+
Each use case is a thin pass-through to the repository (SRP — one class per action):
169+
170+
| Use Case | Method | Description |
171+
|---|---|---|
172+
| `ListStoreFilesUseCase` | `execute(prefix="/") -> list[str]` | List file paths matching the prefix. |
173+
| `GetStoreFileUseCase` | `execute(path) -> str \| None` | Retrieve a single file's content; `None` if not found. |
174+
| `PutStoreFileUseCase` | `execute(path, content) -> str` | Create or replace a file; returns the stored content. |
175+
| `DeleteStoreFileUseCase` | `execute(path) -> None` | Delete a file (idempotent). |
176+
177+
All use cases are `async` and accept a `StoreFileRepository` via constructor injection.
178+
179+
### Port — `StoreFileRepository` (`src/domain/ports/store_file_repository.py`)
180+
181+
Abstract interface for file CRUD on a namespace-scoped key-value store:
182+
183+
| Method | Signature |
184+
|---|---|
185+
| `list_files` | `(prefix: str) -> list[str]` |
186+
| `get_file` | `(path: str) -> str \| None` |
187+
| `put_file` | `(path: str, content: str) -> None` |
188+
| `delete_file` | `(path: str) -> None` |
189+
190+
`delete_file` is idempotent — implementations must not raise if the path does not exist.
191+
192+
### Adapter — `LangGraphStoreFileRepository` (`src/infrastructure/store_file/adapter.py`)
193+
194+
Implements `StoreFileRepository` on top of a LangGraph `BaseStore` (`InMemoryStore` or `AsyncPostgresStore`):
195+
196+
- Files are stored as `{"content": str, "encoding": "utf-8"}` values keyed by path under the `("filesystem",)` namespace by default (configurable via the `namespace` constructor arg).
197+
- `list_files` uses `asearch(namespace, limit=100)` and filters client-side by `str.startswith(prefix)`.
198+
- `get_file` returns `item.value.get("content")` (or `None` if the item is missing or malformed).
199+
- `put_file` uses `aput` with the content dict.
200+
- `delete_file` uses `adelete` (idempotent).
201+
202+
### Dependency injection
203+
204+
The four use cases are wired in `src/dependencies.py` via `get_list_store_files_use_case`, `get_get_store_file_use_case`, `get_put_store_file_use_case`, and `get_delete_store_file_use_case`. They share a single `LangGraphStoreFileRepository` instance built from the same store used by agent backends.
205+
206+
---
207+
195208
## Running Tests
196209

197210
The project uses **pytest** with **pytest-asyncio** for async test support. All tests are pure unit tests with no external dependencies (LLM calls are fully faked).

0 commit comments

Comments
 (0)