Skip to content
atrium

Messy in.

Precise out.

PDF, Office, HTML, EPUB, scans — in. Structure-aware chunks with their page numbers — out.

Read the docsGet the source

Atrium Cloud is this library, hosted and run for you. It is not here yet — the list is what there is until it is. The library is free, MIT licensed, and yours either way.

Ingest a document
curl -X POST http://localhost:8000/ingest \
  -F "files=@./report.pdf"

How Atrium works: parse, chunk, embed, retrieve

Before

A folder of files, none of them tidy.

Point it at a directory. Every file is parsed on the CPU, with no model and no GPU in the loop.

After

The same document, as chunks that know where they came from.

Each chunk carries its page, its heading path and the element type it was cut from.

report.pdf — 2 pages
A parsed page
42 chunks, each with a page
A chunk with its provenance

Every page becomes typed elements.

Seven types come out. A table stays a table.

a section title, with its level
prose
one bullet at a time
rows and cells, never flattened
a fenced block, intact
text that belongs to an image
a note at the foot of the page

Pick a type to find it on the page.

PARSE / TYPED ELEMENTSHEADINGPARAGRAPHLIST_ITEMTABLECODEIMAGE_CAPTIONFOOTNOTE07 TYPES DETECTED
A parsed page. The left column is the document; the right column is what the parser made of each block.

The four stages

Choose a pipeline stageparsechunkembedretrieve

Five ways to cut it. One that respects the document.

Same target, same overlap — only the strategy changes. The strategy that ran is stamped on every chunk, so a corpus with mixed strategies stays auditable.

Boundaries fall on the element edges. The table is one chunk. The heading is a breadcrumb on top of it, not a chunk of its own.

Atrium loads no model.

Embeddings come from your own Ollama — bge-large, 1024 dimensions, in its own process. The chunks travel to it and the vectors come back.

structure_awaretarget 700 · overlap 100

12 chunks · 0 boundaries mid-sentence

front matter
170
Executive Summary
605
Background
698
Project Objectives
652
Project Objectives
377
Phase 1: Data Collection and Preprocessing (Months 1-6)
421
Phase 2: Model Development (Months 7-18)
314
Methodology
446
Research Team
603
Timeline
270
Conclusion
342
Conclusion
393
paragraphtable, kept wholecut mid-sentencebar length = characters

source.docx — 39 elements, 5,563 characters, parsed and chunked by the shipped code. Regenerate with scripts/dump_chunk_packing.py.

What will it take to run?

Pure Python. Your embedder runs in its own process, not ours.

Resident, serving search

88MB

Peak while ingesting the whole corpus is 425 MB. The embedder is not in this number — it is in Ollama's process, and it is 670 MB of that.

Measured on the shipping build: the pdfplumber backend, the local SQLite store, bge-large in its own process
MeasurementValue
Resident, serving search88 MB
Peak, ingesting a 19-document corpus425 MB
Search, p50 / p95 over 1451 chunks27 ms / 36 ms
Ingesting that corpus96.3 s

Apple M4, 3.12.13 · pdfplumber backend · sqlite store · bge-large (1024-d)

  • The embedder is a separate process

    bge-large costs ~670 MB in Ollama's process, and it is in none of the numbers above.

  • RSS does not return to the OS

    A worker that ingests continuously should be recycled between batches.

  • One machine, one configuration

    Apple M4, 16 GB, the container image, bge-large, and the local SQLite store.

Don't want to run it yourself? Atrium Cloud is coming.

clientPOST /ingest200 { jobs: […] }GET /jobs/{job_id}status: completePOST /search"retention policy"top_k: 8200 · took_ms 41atrium/searchparsechunkembedretrievereport.pdfchunks_written 420.781result · rank 10.781page_number14section_pathRetention › Data lifecycleelement_typeparagraphchunking_strategystructure_awarereport.pdf · p. 14

Read the code that parses your documents.

The parsing heuristics, the chunker and the retrieval path are all readable Python. No model checkpoint, no black box.

License
MIT
Python
3.10+
Vector store
SQLite, Qdrant or pgvector
Version
v0.1.0
The benchmark

0.700

R@5 · 23 documents · 700 questions

SystemR@595% CIParsePeak
atrium ·0.7000.594–0.79988 s1016 MB
docling0.6860.565–0.798554 s3482 MB
unstructured0.6140.497–0.725195 s855 MB

pdfplumber (MIT) · atrium smart_chunker / precise (structure-aware) · hybrid — Reciprocal Rank Fusion over dense + BM25 · reranker off

Retrieval accuracy is on par with Docling — the difference is inside the noise band at this corpus size, and the report prints the interval rather than a number without one. The separation is in cost: six times faster parsing, a third of the memory, MIT licensed throughout. The harness, the corpus and the commands are public, so the numbers can be checked.

Every format it reads

PDF, 1 of 18