Lexoid turns messy documents and live websites into agent-ready context.
parse(): High-accuracy document and webpage parsing across static, local LLM, and frontier LLM pipelines.browse()(preview): Goal-driven, evidence-grounded browser automation for live web workflows.
Useful information is buried in documents and scattered across websites. Getting it out shouldn't require expensive a custom pipeline for every task.
- Extract what matters: Turn documents and web pages into usable text and structured data for downstream workflows.
- Automate the search, retain the evidence: Automate browser tasks with evidence.
- Choose your approach: Use static parsing, hosted LLMs, or supported local models.
- Build openly: Collaborate under the Apache 2.0 license.
pip install lexoidTo use LLM-based parsing, define the following environment variables or create a .env file with the following definitions
OPENAI_API_KEY=""
GOOGLE_API_KEY=""For local inference with Ollama, no API key is required. Install Ollama, pull the target model, and keep the local server running:
ollama pull gemma4
export OLLAMA_BASE_URL=127.0.0.1:11434
ollama list
ollama serve
# docker
Reference: https://docs.ollama.com/docker#run-model-locally
CPU example (will most likely be slower; remember to adjust `OLLAMA_TIMEOUT` as needed)
- docker run -d -v ollama:/root/.ollama -p 11434:11434 -e OLLAMA_BASE_URL=0.0.0.0 -e OLLAMA_TIMEOUT=240 --name ollama ollama/ollama
- docker exec -it ollama ollama pull gemma4:latestFor webpage retrieval and URL-to-PDF conversion with parse(), install Playwright's Chromium browser binaries. The Playwright Python package is installed with Lexoid:
playwright install --with-deps --only-shell chromiumTo use the autonomous browser tasks shown below with browse(), install the optional agent dependencies and Patchright's Chromium binaries:
pip install "lexoid[browse]"
patchright install --with-deps chromium[!NOTE] Installing the package from within the virtual environment could cause unexpected behavior, as Lexoid creates and activates its own environment in order to build the wheel.
make build
To install dependencies:
make installor, to install with dev-dependencies:
make devTo activate virtual environment:
source .venv/bin/activateHere's a quick example to parse documents using Lexoid:
from lexoid.api import parse
parsed_md = parse("https://www.justice.gov/eoir/immigration-law-advisor", parser_type="AUTO")["raw"]
# or
pdf_path = "path/to/immigration-law-advisor.pdf"
parsed_md = parse(pdf_path, parser_type="LLM_PARSE")["raw"]
# or
pdf_path = "path/to/immigration-law-advisor.pdf"
parsed_md = parse(pdf_path, parser_type="STATIC_PARSE")["raw"]
print(parsed_md)- path (str): The file path or URL.
- parser_type (str, optional): The type of parser to use ("LLM_PARSE" or "STATIC_PARSE"). Defaults to "AUTO".
- pages_per_split (int, optional): Number of pages per split for chunking. Defaults to 4.
- max_threads (int, optional): Maximum number of threads for parallel processing. Defaults to 4.
- **kwargs: Additional arguments for the parser.
Use parse() when you have a URL to retrieve and parse. Set as_pdf=True to render the page to PDF before parsing; pass save_dir to keep the generated PDF after the call returns:
from lexoid.api import parse
# Retrieve and parse a webpage
page = parse("https://example.com")
# Render the webpage to PDF, then parse the PDF
page_pdf = parse(
"https://example.com",
as_pdf=True,
save_dir="output/",
save_filename="example.pdf",
)
print(page_pdf["pdf_path"])Use browse() when the task requires goal-driven interaction with a website, including searching, applying filters, and completing multi-step non-binding quote, estimate, eligibility, or lookup forms with values supplied in the request. It returns an evidence-backed answer; it does not convert the source URL to PDF:
import asyncio
from lexoid.api import browse
async def main():
result = await browse(
"Visit https://example.com and find the current enterprise plan price."
)
print(result.answer)
asyncio.run(main())Note: The navigator is instructed not to log in, create accounts, accept legal agreements, purchase, bind coverage, pay, or book, and to stop when required information, authentication, or consequential consent is needed. These are navigator instructions, not semantic enforcement for every site control; ambiguous actions should be reviewed. Popups are handled by purpose: promotional overlays can be dismissed, and cookie banners use reject or necessary-only options when available. CAPTCHA solving is planned; until it is implemented, a CAPTCHA that prevents progress is reported as a blocker.
Lexoid provides a command-line interface for document parsing without writing Python code.
The CLI is automatically available after installing Lexoid:
pip install lexoid
lexoid --helpAlternatively, use with Python module syntax:
python -m lexoid --helpConvert documents to markdown or JSON:
# Parse to stdout (default markdown)
lexoid parse --input document.pdf
# Save to file
lexoid parse --input document.pdf --output output.md
# Output as JSON (includes metadata, segments, token usage)
lexoid parse --input document.pdf --format json --output result.json
# Use specific parser (STATIC_PARSE, LLM_PARSE, or AUTO)
lexoid parse --input document.pdf --parser-type STATIC_PARSE
# Use specific LLM model
lexoid parse --input document.pdf --model gpt-4o
# Enable verbose logging
lexoid parse --input document.pdf --verboseExtract data conforming to a JSON schema:
# Inline schema
lexoid schema \
--input document.pdf \
--schema '{"type": "object", "properties": {"title": {"type": "string"}}}' \
--output result.json
# Schema from file
lexoid schema \
--input document.pdf \
--schema schema.json \
--output result.json
# Specify LLM provider
lexoid schema \
--input document.pdf \
--schema schema.json \
--api openai \
--model gpt-4oConvert documents to LaTeX format:
# Convert to stdout
lexoid latex --input document.pdf
# Save to file
lexoid latex --input document.pdf --output output.tex
# Use specific model
lexoid latex --input document.pdf --model gpt-4o--input, -i: Input file path (required) - Supports PDF, images, HTML, DOCX, XLSX, PPTX, or URLs--output, -o: Output file path (optional) - If not specified, output is printed to stdout--verbose, -v: Enable detailed logging
lexoid parse --help
--parser-type, -p: Parser type -AUTO(default),LLM_PARSE, orSTATIC_PARSE--model, -m: LLM model name (default: gemini-3.5-flash-lite, thinking: medium)--pages-per-split: Pages per chunk (default: 4)--max-processes: Parallel processes (default: 4)--framework: Static parser framework -pdfplumberorpaddleocr--format: Output format -markdown(default, plain markdown text) orjson(full result with metadata, segments, token usage)
lexoid schema --help
--schema, -s: JSON schema (file path or inline JSON, required)--model, -m: LLM model (default: gpt-4o-mini)--api: API provider - openai, gemini, anthropic, ollama, etc. (auto-detected if not specified)--example-schema: Provide example data for the schema--fill-single-schema: Auto-fill single schemas
lexoid latex --help
--model, -m: LLM model (default: gpt-4o-mini)--api: API provider - openai, gemini, anthropic, ollama, etc. (auto-detected if not specified)
- OpenAI
- Hugging Face
- Together AI
- OpenRouter
- Fireworks
- Ollama
Lexoid supports local LLM_PARSE inference through Ollama. The initial recommended model is gemma4:latest.
from lexoid.api import parse
result = parse(
"path/to/document.pdf",
parser_type="LLM_PARSE",
api_provider="ollama",
model="gemma4:latest",
max_processes=1,
)
print(result["raw"])Notes:
- Ollama uses the default local endpoint
http://localhost:11434unlessOLLAMA_BASE_URLis set. - Lexoid forces
max_processes=1for Ollama-backed parsing to avoid local multiprocess contention. AUTOrouting does not select Ollama in this first version; choose it explicitly withapi_provider="ollama".
Results aggregated across 14 documents.
Note: Benchmarks are currently done in the zero-shot setting.
| Rank | Model | SequenceMatcher Similarity | Cosine- Similarity | Time (s) | Cost ($) |
|---|---|---|---|---|---|
| 1 | gemini-3.5-flash-lite (thinking: medium) | 0.959 (±0.071) | 0.986 (±0.018) | 9.84 | 0.00659 |
| 2 | claude-sonnet-5-5 (thinking: low) | 0.956 (±0.084) | 0.991 (±0.010) | 6.85 | 0.01478 |
| 3 | claude-opus-5-5 (thinking: low) | 0.952 (±0.088) | 0.983 (±0.022) | 9.13 | 0.02805 |
| 4 | claude-haiku-5-5 (thinking: medium) | 0.949 (±0.074) | 0.980 (±0.025) | 7.96 | 0.00121 |
| 5 | gpt-6.1-sol (thinking: low) | 0.947 (±0.100) | 0.986 (±0.014) | 31.41 | 0.02349 |
| 6 | gemini-3.8-flash (thinking: low) | 0.932 (±0.117) | 0.975 (±0.032) | 20.73 | |
| 7 | gpt-6-astra (thinking: medium) | 0.931 (±0.125) | 0.986 (±0.014) | 30.83 | 0.14173 |
| 8 |  AUTO + gemini-3.5-flash-lite (thinking: medium) |
0.928 (±0.112) | 0.989 (±0.015) | 8.93 | 0.005315 |
| 9 |  AUTO + claude-haiku-5-5 (thinking: medium) |
0.925 (±0.086) | 0.982 (±0.023) | 8.05 | 0.001215 |
| 10 | gemini-4-argon (thinking: high) | 0.890 (±0.229) | 0.969 (±0.071) | 69.63 | |
| 11 | gpt-5.2 | 0.890 (±0.193) | 0.975 (±0.036) | 33.32 | 0.03959 |
| 12 | mistral-ocr-latest | 0.882 (±0.106) | 0.932 (±0.091) | 5.75 | 0.00121 |
| 13 | gpt-5.5 | 0.874 (±0.209) | 0.939 (±0.138) | 72.11 | 0.14495 |
| 14 | gpt-6-luna (thinking: low) | 0.869 (±0.146) | 0.953 (±0.053) | 13.35 | 0.00096 |
| 15 | claude-3-5-sonnet-20241022 | 0.858 (±0.184) | 0.930 (±0.098) | 17.32 | 0.01804 |
| 16 | gpt-5.4-mini | 0.835 (±0.210) | 0.948 (±0.066) | 13.14 | 0.00902 |
| 17 | gpt-5-mini | 0.819 (±0.201) | 0.917 (±0.104) | 52.84 | 0.00811 |
| 18 | gpt-5 | 0.807 (±0.215) | 0.919 (±0.088) | 98.12 | 0.05505 |
| 19 | gpt-5.4 | 0.803 (±0.238) | 0.936 (±0.150) | 31.98 | 0.03887 |
| 20 | claude-sonnet-4-20250514 | 0.801 (±0.188) | 0.905 (±0.136) | 22.02 | 0.02056 |
| 21 | claude-opus-4-20250514 | 0.789 (±0.184) | 0.886 (±0.148) | 29.55 | 0.09513 |
| 22 | accounts/fireworks/models/llama4-maverick-instruct-basic | 0.772 (±0.203) | 0.930 (±0.117) | 16.02 | 0.00147 |
| 23 | gemini-3-flash-preview | 0.766 (±0.293) | 0.858 (±0.210) | 39.38 | 0.00969 |
| 24 | claude-opus-4-8 | 0.764 (±0.254) | 0.863 (±0.154) | 11.10 | 0.03195 |
| 25 | claude-sonnet-4-6 | 0.757 (±0.302) | 0.843 (±0.206) | 16.50 | 0.01804 |
| 26 | gpt-4.1-mini | 0.754 (±0.249) | 0.803 (±0.193) | 23.28 | 0.00347 |
| 27 | accounts/fireworks/models/llama4-scout-instruct-basic | 0.754 (±0.243) | 0.942 (±0.063) | 13.36 | 0.00087 |
| 28 | gpt-4o | 0.752 (±0.269) | 0.896 (±0.123) | 28.87 | 0.01469 |
| 29 | gpt-4o-mini | 0.728 (±0.241) | 0.850 (±0.128) | 18.96 | 0.00609 |
| 30 | claude-haiku-4-5-20251001 | 0.683 (±0.300) | 0.841 (±0.187) | 7.86 | 0.00504 |
| 31 | claude-3-7-sonnet-20250219 | 0.646 (±0.397) | 0.758 (±0.297) | 57.96 | 0.01730 |
| 32 | gpt-4.1 | 0.637 (±0.301) | 0.787 (±0.185) | 35.37 | 0.01498 |
| 33 | google/gemma-3-27b-it | 0.604 (±0.342) | 0.788 (±0.297) | 23.16 | 0.00020 |
| 34 | ds4sd/SmolDocling-256M-preview | 0.603 (±0.292) | 0.705 (±0.262) | 507.74 | 0.00000 |
| 35 | gpt-5.4-nano | 0.600 (±0.309) | 0.856 (±0.119) | 22.51 | 0.00321 |
| 36 | microsoft/phi-4-multimodal-instruct | 0.589 (±0.273) | 0.820 (±0.197) | 14.00 | 0.00045 |
| 37 | qwen/qwen-2.5-vl-7b-instruct | 0.498 (±0.378) | 0.630 (±0.445) | 14.73 | 0.00056 |
If you use Lexoid in production or publications, please cite accordingly and acknowledge usage. We appreciate the support 🙏