PaperMeld

English | 简体中文

A Codex Skill and plugin for building, searching, and maintaining a local, catalog-first paper library.

PaperMeld gives Codex a disciplined way to use a personal literature library: invoke $papermeld, search library.jsonl, select a few candidate papers, then read the relevant Markdown, PDF, or figure as evidence. It is designed to prevent a common failure mode—an agent blindly scanning a large folder and confusing a filename, title, or catalog field for a supported claim.

The core is a reusable Codex Skill packaged as a Codex plugin. Its Python CLI makes catalog creation, ingestion, and verification deterministic; the Skill reads an existing library.jsonl directly for catalog-first retrieval.

Use it in Codex

After installation, give Codex a concrete retrieval task such as:

Use $papermeld to find 4 papers in my local library about diffusion-based trajectory prediction. Return candidate paths and versions first, then read only the most relevant Introduction and method sections. Do not modify files.

PaperMeld turns this into an evidence-led sequence:

research question -> catalog search -> small candidate set -> source inspection -> evidence-backed answer

The catalog routes the agent to sources; it never replaces the sources as evidence.

Install in Codex

Recommended: install the plugin from GitHub

If your Codex environment supports GitHub marketplaces, run:

codex plugin marketplace add QinKB/PaperMeld --ref main
codex plugin add papermeld@papermeld

Then start a new Codex task and invoke $papermeld. The plugin packages the Skill for catalog-first retrieval.

You can also ask Codex directly:

Install the Codex plugin from https://github.com/QinKB/PaperMeld

Fallback: install only the Skill

If your Codex environment cannot install a community plugin yet, ask it to:

Install the Codex Skill from GitHub repo QinKB/PaperMeld, path plugins/papermeld/skills/papermeld.

When working from a clone, Codex also discovers the repository-scoped Skill at .agents/skills/papermeld automatically. Install the CLI to create, bootstrap, verify, or ingest a library.

Install the CLI

The CLI needs Python 3.11+ and uv. If uv --version is unavailable, install uv with its official instructions, then run:

uv tool install "git+https://github.com/QinKB/PaperMeld.git"
papermeld --help

For a source checkout:

git clone https://github.com/QinKB/PaperMeld.git
cd PaperMeld
uv tool install .
papermeld --help

Use the CLI independently or let Codex call it through $papermeld when it is available. MinerU is used when PaperMeld converts new PDFs.

Optional: install MinerU for PDF conversion

MinerU powers PDF conversion through ingest. Install it in its own environment so its larger ML dependencies remain separate from PaperMeld:

uv venv ~/.venvs/mineru
source ~/.venvs/mineru/bin/activate
uv pip install -U "mineru[all]"
mineru --version

mineru[all] is MinerU's general-purpose official installation. The first parse can download models and take time. While that environment is active, PaperMeld finds mineru on PATH; or pass its location explicitly with --mineru "$(command -v mineru)". See the MinerU Quick Start for platform-specific acceleration and model-source options.

What problem it solves

A directory of PDFs and converted Markdown is readable to a person but opaque to an AI: it does not know which files are relevant, whether an item is a preprint, or where a figure belongs. PaperMeld keeps a library.jsonl catalog with one record per paper version. The correct retrieval pattern is:

question -> catalog search -> select a small evidence set -> read those sources

The catalog is a routing layer, not evidence. An agent must still inspect the chosen Markdown or PDF before it makes a factual claim.

Features

• Local-first: PDF text and conversion artifacts stay on your computer.
• Converter-optional: cataloging and verification use Python's standard library; MinerU adds PDF-to-Markdown conversion.
• Safe ingestion: checks exact PDF hashes, DOI, and arXiv identifiers before a conversion; external PDFs are copied by default rather than moved.
• Existing-library bootstrap: indexes your current Markdown and optionally links PDFs without renaming, moving, or rewriting either.
• Implementation links: connects existing local Git repositories to their papers and records the repository state for later code work.
• Agent-friendly retrieval: stable, concise search output lets an agent open only the relevant source files.
• Portable layout: new libraries have neutral directory names; the older INBOX/PDF/Markdown layout is detected for backward compatibility.

Start a new library

Choose a location outside the PaperMeld source checkout:

papermeld --library ~/Research/Papers init

This creates an empty, portable library:

Papers/
├── inbox/          # optional landing place for PDFs
├── papers/         # PaperMeld-managed PDF copies
├── markdown/       # one readable Markdown file per paper
├── assets/         # converter artifacts and local figures
└── library.jsonl   # one JSON record per paper version

No source PDF, Markdown, or image is uploaded anywhere.

Choose your starting point

I already have Markdown or a converter output folder

First preview the catalog that would be built:

papermeld --library ~/Research/Papers bootstrap \
  --markdown-dir ~/OldLibrary/markdown \
  --pdf-dir ~/OldLibrary/pdfs \
  --dry-run

If the record count and PDF links look right, run the same command without --dry-run:

papermeld --library ~/Research/Papers bootstrap \
  --markdown-dir ~/OldLibrary/markdown \
  --pdf-dir ~/OldLibrary/pdfs

Repeat --markdown-dir or --pdf-dir to join multiple collections. The source folders stay where they are; PaperMeld writes only library.jsonl and its empty library layout. Files it cannot match are marked for review instead of guessed.

I have PDFs and want PaperMeld to convert them

Install and test a local PDF-to-Markdown converter first. PaperMeld supports MinerU out of the box, but keeps it optional:

papermeld --library ~/Research/Papers ingest ~/Downloads/paper.pdf \
  --mineru /absolute/path/to/mineru \
  --label concise-paper-label \
  --dry-run

Review the proposed title, year, venue, filename, and duplicate status. Remove --dry-run only when they are acceptable:

papermeld --library ~/Research/Papers ingest ~/Downloads/paper.pdf \
  --mineru /absolute/path/to/mineru \
  --label concise-paper-label

An external PDF is copied into papers/; the original remains in Downloads. Put a PDF in inbox/ when you want PaperMeld to move it into the managed library. Use --move-source only when you explicitly want an external source file moved.

For another converter, pass a command template with literal {pdf} and {output} placeholders. It must write exactly one Markdown file under the output directory and keep local images reachable with relative paths:

papermeld --library ~/Research/Papers ingest ~/Downloads/paper.pdf \
  --converter 'my-converter --input {pdf} --output {output}' \
  --label concise-paper-label

Retrieve papers outside Codex

Search first, then open a small number of records:

papermeld --library ~/Research/Papers search "causal inference"
papermeld --library ~/Research/Papers verify

The same routing principle applies to scripts and other AI agents: search first and inspect only the sources needed to support the task. PaperMeld preserves library data by default; use the ingestion workflow only when you intend to change the library.

Link local code to papers

If paper implementations already live in a local directory, preview the links that PaperMeld finds from README headings, paper titles, and model names:

papermeld --library ~/Research/Papers link-code \
  --code-root ~/Research/Code \
  --dry-run

Apply the same command without --dry-run to add a code list to matching paper records. Each link stores its local path, origin URL, branch, checked-out commit, README path, and matching evidence. Run verify after a code update to detect a missing repository, changed origin, or changed checkout.

Commands

Command	Purpose	Needs a converter?
init	Create an empty library layout and catalog	No
bootstrap	Index existing Markdown/PDF folders without changing them	No
search QUERY	Route a research question to candidate records	No
link-code	Link existing local Git repositories to paper records	No
verify	Check catalog paths, local image links, and code links	No
ingest PDF	Convert one PDF, organize assets, and append a record	Yes

Run papermeld <command> --help for every option. ingest --dry-run is the recommended first run for any new converter or PDF collection.

Record format and privacy

library.jsonl uses one JSON object per line. Core fields include:

{
  "paper_id": "2025-CVPR-Example",
  "title": "Example paper title",
  "year": 2025,
  "venue": "CVPR",
  "doi": "10.xxxx/example",
  "status": "published",
  "source_pdf": "papers/2025-CVPR-Example.pdf",
  "markdown": "markdown/2025-CVPR-Example.md",
  "artifact_dir": "assets/2025-CVPR-Example",
  "code": [{
    "relationship": "implementation",
    "local_path": "/home/me/Research/Code/Example",
    "repository_url": "https://github.com/example/Example.git",
    "commit": "<checked-out commit>"
  }]
}

When a DOI is available, PaperMeld asks Crossref only for bibliographic metadata. It sends no PDF contents, extracted Markdown, catalog, or API key. Missing or uncertain metadata is recorded for review rather than silently invented.

Do not commit personal PDFs, converted papers, figure assets, or a private catalog to a public repository. The supplied .gitignore excludes the default library layout for that reason.

Compatibility and safety

PaperMeld detects the earlier layout:

INBOX/  PDF/  Markdown/ALL_MARKDOWN/  Markdown/MINERU_OUTPUT/

and continues to index it without migration. Bootstrap never renames or rewrites source Markdown. Ingestion converts in a temporary directory and only adds the PDF, Markdown, artifacts, and catalog record after successful output validation. It stops before conversion on an exact hash, DOI, or arXiv-ID duplicate.

Development

uv sync
uv run python -m unittest discover -s tests -v
uv build

See CONTRIBUTING.md for contribution rules. The project is MIT licensed; see LICENSE.

Acknowledgments

PaperMeld is grateful to the projects and communities that make this workflow possible:

• OpenAI Codex for assistance with implementation, testing, documentation, and the reusable Skill/plugin workflow.
• MinerU for the optional local PDF-to-Markdown conversion path.
• Crossref for DOI-based bibliographic metadata.
• uv for reproducible Python packaging and development workflows.

These projects did not review or endorse PaperMeld. PaperMeld remains an independent, local-first tool; users are responsible for complying with the licenses and access conditions of their source materials and chosen converters.

Community and responsible disclosure

Please read the Code of Conduct before participating. Report security-sensitive local-file or converter issues according to SECURITY.md, not in a public issue.