doocus update and meeting list export

This commit is contained in:
Mariano Gabriel
2026-07-05 11:15:57 -03:00
parent fa3a0e3d1a
commit 300806b936
17 changed files with 614 additions and 628 deletions

View File

@@ -1,37 +1,47 @@
# doocus — local document extraction
# doocus — local drive tree indexer
The document twin of meetus. Turns locally-downloaded files (Drive exports,
PDFs, images, ...) into a shareable **textified** product (`content.md`) plus
structured `meta.json`, mirroring the meetus `output/<run>/` contract.
The document twin of meetus. Replicates a **whole local drive tree** into a
single `index.json`, preserving the folder hierarchy (no flattening). Every file
is a node; the **original file is the artifact** (what a QA/PM opens, what a
package links to). doocus only *extracts text* for the heavy formats
(docx/pdf/pptx/xlsx) — and that text is a **search index only, never a
replacement** for the document.
Extraction is **deterministic and offline** — no cloud AI touches the raw source.
The textified output is what feeds permitted services (Gemini web, NotebookLM)
and the local search index; the raw source stays on disk.
The raw source stays on disk; permitted services (Gemini web, NotebookLM) get the
original (as a link once we have Drive URLs, or re-uploaded).
## Quick start
```bash
# one document
python process_doc.py notes.docx # → docs-output/<YYYYMMDD-NNN-notes>/
python process_doc.py report.pdf --render # also render page 1 → thumb.jpg
python process_doc.py scan.png --ocr # OCR the image into content.md
# index a whole downloaded drive tree → docs-output/index.json
uv run make docs IN="/mnt/win/drive"
uv run make docs IN="..." DOC_EXTRA="--ocr" # OCR scanned pdfs
make docs-dry IN="..." # counts only, no writes
# a whole downloaded folder (mirrors the input tree)
make docs IN="/mnt/win/drive" # → docs-output/...
make docs IN="..." DOC_EXTRA="--render --ocr"
make docs-dry IN="..." # list what would run
# or directly
uv run process_tree.py /mnt/win/drive --output docs-output
# one-off single file (older per-file model, still handy)
uv run process_doc.py notes.docx
```
## Output contract
```
docs-output/<YYYYMMDD-NNN-stem>/
content.md textified document — the shareable product
meta.json source info, sha256, dates, author, title, pages/sheets/slides, word_count, warnings
thumb.jpg optional single preview (image / rendered page / video frame)
manifest.json run config + outputs inventory + source pointer (path, not a copy)
docs-output/
index.json whole tree; every file a node:
{ path, name, ext, family, mode, bytes,
modified, url:null } ◀ url = Drive link (later)
<mirrored path>/<file>.doocus/ sidecar — ONLY for extracted formats:
├── content.md extracted text — SEARCH INDEX ONLY
└── meta.json source/sha/dates/author/pages/... metadata
```
Node **modes**: `extracted` (docx/pdf/pptx/xlsx → text sidecar) · `meeting`
(mp4/mkv/... → belongs to meetus, not a doc) · `link` (everything else → original
is the artifact, nothing duplicated).
## Supported types
| Family | Extensions | Notes |
@@ -65,12 +75,16 @@ uv sync --group doocus --group ocr # + pytesseract (needs system `tesseract`
## Browser UI
`ui/doocus-app/` (sibling of `ui/meetus-app/`, shares `ui/framework/`) browses the
extracted docs with per-type viewers, a metadata panel, and a package builder
that zips the **original file + content.md + meta.json** per doc for a chosen
target (Gemini web / NotebookLM):
`ui/doocus-app/` (sibling of `ui/meetus-app/`, shares `ui/framework/`) reads
`index.json` and shows the **full tree** (folders preserved). Select a file →
detail view: for extracted docs, a split of the markdown-rendered **extracted
text** (clearly labelled "search index, not the document") beside the **native
view of the original** (pdf inline; docx/xlsx → download); for linked files the
original renders directly (image / html / text / markdown). A package builder
zips selected **originals** (+ the `content.md` for extracted ones) for a target
(Gemini web / NotebookLM), and will emit Drive **links** once nodes carry a `url`.
```bash
cd ui/doocus-app && npm install
DOOCUS_OUTPUT=/path/to/docs-output npm run dev
DOOCUS_OUTPUT=/abs/path/to/docs-output npm run dev
```