77 lines
3.2 KiB
Markdown
77 lines
3.2 KiB
Markdown
# doocus — local document extraction
|
|
|
|
The document twin of meetus. Turns locally-downloaded files (Drive exports,
|
|
PDFs, images, ...) into a shareable **textified** product (`content.md`) plus
|
|
structured `meta.json`, mirroring the meetus `output/<run>/` contract.
|
|
|
|
Extraction is **deterministic and offline** — no cloud AI touches the raw source.
|
|
The textified output is what feeds permitted services (Gemini web, NotebookLM)
|
|
and the local search index; the raw source stays on disk.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# one document
|
|
python process_doc.py notes.docx # → docs-output/<YYYYMMDD-NNN-notes>/
|
|
python process_doc.py report.pdf --render # also render page 1 → thumb.jpg
|
|
python process_doc.py scan.png --ocr # OCR the image into content.md
|
|
|
|
# a whole downloaded folder (mirrors the input tree)
|
|
make docs IN="/mnt/win/drive" # → docs-output/...
|
|
make docs IN="..." DOC_EXTRA="--render --ocr"
|
|
make docs-dry IN="..." # list what would run
|
|
```
|
|
|
|
## Output contract
|
|
|
|
```
|
|
docs-output/<YYYYMMDD-NNN-stem>/
|
|
content.md textified document — the shareable product
|
|
meta.json source info, sha256, dates, author, title, pages/sheets/slides, word_count, warnings
|
|
thumb.jpg optional single preview (image / rendered page / video frame)
|
|
manifest.json run config + outputs inventory + source pointer (path, not a copy)
|
|
```
|
|
|
|
## Supported types
|
|
|
|
| Family | Extensions | Notes |
|
|
|---------|-------------------------|-------|
|
|
| text | md, txt, yaml/yml, json | passthrough; json/yaml add a structure summary |
|
|
| tabular | csv | markdown table + row/col metadata (stdlib) |
|
|
| office | docx, pptx, xlsx | text + core properties (author, dates, title) |
|
|
| pdf | pdf | text per page + info dict; `--render` for page-1 thumb |
|
|
| web | html/htm | visible text + title/meta tags |
|
|
| image | jpg/jpeg, png | EXIF (incl. capture date) + thumbnail; `--ocr` for text |
|
|
| media | mp4 | ffprobe metadata + thumbnail; **delegates transcription to meetus** |
|
|
|
|
Each extractor is isolated: a missing optional dependency or a corrupt file is
|
|
recorded as a warning in `meta.json`, never aborting a batch (same resilience as
|
|
`ctrl/batch.sh`).
|
|
|
|
## Dependencies
|
|
|
|
Core text/tabular use only the stdlib. Everything else is a uv group in the
|
|
repo's `pyproject.toml`:
|
|
|
|
```bash
|
|
uv sync --group doocus # Pillow, PyYAML, python-docx, python-pptx,
|
|
# openpyxl, pypdf, beautifulsoup4, lxml
|
|
uv sync --group doocus --group ocr # + pytesseract (needs system `tesseract`)
|
|
```
|
|
|
|
`mp4` uses system `ffmpeg`/`ffprobe` (shared with meetus). PDF page rendering
|
|
(`--render`) additionally needs the `pdf-render` group (`pdf2image`) + system
|
|
`poppler`.
|
|
|
|
## Browser UI
|
|
|
|
`ui/doocus-app/` (sibling of `ui/meetus-app/`, shares `ui/framework/`) browses the
|
|
extracted docs with per-type viewers, a metadata panel, and a package builder
|
|
that zips the **original file + content.md + meta.json** per doc for a chosen
|
|
target (Gemini web / NotebookLM):
|
|
|
|
```bash
|
|
cd ui/doocus-app && npm install
|
|
DOOCUS_OUTPUT=/path/to/docs-output npm run dev
|
|
```
|