Compare commits

...
19 Commits
Author SHA1 Message Date
StommerandClaude Sonnet 5.5 363ecdfd0a Starte test.yml wieder mit dem Push und warte auf den Checkout des Dev-Deploys, weil workflow_run hier nicht auslöst.
Deploy Development / deploy (push) Successful in 51s
Test Suite / pytest-backend (push) Successful in 6m22s
Test Suite / frontend-build (push) Successful in 25s
Test Suite / smoke-dev (push) Successful in 0s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:24:11 +01:00
StommerandClaude Sonnet 5.5 4bd2d047f5 Entferne den abgeschnittenen CSS-Block, der die Layout-Regeln der Bibliothek verschluckt hat.
Deploy Development / deploy (push) Successful in 47s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:20:49 +01:00
StommerandClaude Sonnet 5.5 f925661937 Warne vor bereits eingefügten Quelldokumenten und biete Nicht einfügen, erneut einfügen oder zusätzliches Einbinden an.
Deploy Development / deploy (push) Successful in 44s
Prüfstufen hier, darin, Umfeld stehen als Kontoeinstellung bereit. Der Server antwortet bei einem sicheren Treffer mit 409 und bindet auf Wunsch das vorhandene Element zusätzlich ein.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:06:52 +01:00
StommerandClaude Sonnet 5.5 a6dc24d8fe Lege Einstellungen je Konto mit Erbung vom Hauptnutzer an und biete die Prüfstufe für doppelte Dokumente in Mein Konto an.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 09:18:17 +01:00
StommerandClaude Sonnet 5.5 bee48bf4a5 Behebe die fehlschlagenden Paperless-Tests und lasse test.yml nur noch nach dem Dev-Deploy laufen.
Deploy Development / deploy (push) Successful in 52s
Der Hinweis zu Tags und Zusatzfeld sitzt wieder in paperless_documents, damit die Signatur von _paperless_get bleibt. Der Push-Trigger entfällt, weil er einen zweiten Lauf auf derselben Testdatenbank startete.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 07:29:45 +01:00
StommerandClaude Sonnet 5.5 6e1b0916a6 Erleichtere die Tag-Auswahl der Paperless-Suche mit Suchfeld, Alle-an-/abwählen und Chips für die gewählten Tags.
Deploy Development / deploy (push) Successful in 52s
Test Suite / pytest-backend (push) Failing after 6m4s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 23s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 07:13:59 +01:00
StommerandClaude Sonnet 5.5 ffdaf1c721 Weise bei einem Paperless-Fehler auf die Kombination aus allen Tags und Zusatzfeld hin, die Paperless selbst mit HTTP 500 beantwortet.
Deploy Development / deploy (push) Successful in 49s
Test Suite / pytest-backend (push) Failing after 6m4s
Test Suite / smoke-dev (push) Successful in 1s
Test Suite / frontend-build (push) Successful in 24s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 06:58:07 +01:00
StommerandClaude Sonnet 5.5 0f2b7a8fab Nenne in der Paperless-Fehlermeldung Statuscode, Pfad und Grund, damit sich eine abgelehnte Abfrage zuordnen lässt.
Deploy Development / deploy (push) Successful in 50s
Test Suite / pytest-backend (push) Failing after 7s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 26s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 22:53:46 +02:00
StommerandClaude Sonnet 5.5 38406d3293 Zeige die Zusatzfelder in der Trefferliste und erkenne Vorschaubilder am Inhalt statt am Antwort-Header.
Deploy Development / deploy (push) Successful in 43s
Test Suite / pytest-backend (push) Canceled after 5m58s
Test Suite / smoke-dev (push) Canceled after 0s
Test Suite / frontend-build (push) Canceled after 0s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 22:43:36 +02:00
StommerandClaude Sonnet 5.5 2f7de33049 Ziehe die Fähigkeiten von Paperless und IMAP in ihr Modul und entferne die unbenutzte Filterliste.
Deploy Development / deploy (push) Successful in 46s
Test Suite / pytest-backend (push) Successful in 5m58s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 22s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 22:23:46 +02:00
StommerandClaude Sonnet 5.5 5e32ba547d Verlege die Paperless-Suchparameter und das Vorschaubild in das Modul und korrigiere das Handy-Layout der großen Auswahl.
Deploy Development / deploy (push) Successful in 42s
Test Suite / pytest-backend (push) Successful in 5m50s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 22s
Die gemeinsame Suchroute kennt keine Filternamen mehr; das Modul deklariert sie und unbekannte Parameter werden abgewiesen. Unvollständige Zusatzfeld-Zeilen werden gemeldet statt weggelassen. Vorschaubilder nur als Rastergrafik. Abschlussmatrix und API-Referenz nachgeführt.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 22:10:54 +02:00
StommerandClaude Sonnet 5.5 f580fb282d Filtere Paperless-Dokumente nach Zusatzfeldern und öffne die Auswahl als großen Dialog mit Filtern links und Treffern rechts.
Deploy Development / deploy (push) Successful in 43s
Test Suite / pytest-backend (push) Successful in 5m37s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 22s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 21:23:29 +02:00
StommerandClaude Sonnet 5.5 9089d830b3 Erweitere die Paperless-Suche um Suchart, weitere Filter und eine Vorschau je Treffer, damit sich das richtige Dokument finden und erkennen lässt.
Deploy Development / deploy (push) Successful in 44s
Test Suite / pytest-backend (push) Successful in 6m5s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 22s
Standard ist jetzt Titel und Inhalt ohne Index; Titel, Inhalt und Volltextindex sind wählbar. Neue Filter: Speicherpfad, Tag-Verknüpfung (alle, eines, keines), "ohne" bei Korrespondent, Typ und Pfad, Hinzugefügt-Datum und Archivnummer. Treffer zeigen Vorschaubild, Textauszug, Tags, Typ und Seitenzahl; die Vorschaubilder laufen über den Server.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-24 20:41:06 +02:00
LarsandCursor 4327f5a434 Zeige PDF-Anlagen in der Mail wieder direkt an, statt nur einen Download anzubieten.
Deploy Development / deploy (push) Successful in 42s
Test Suite / pytest-backend (push) Successful in 5m58s
Test Suite / smoke-dev (push) Successful in 1s
Test Suite / frontend-build (push) Successful in 22s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-23 13:25:24 +02:00
Stommer e2475555c0 Aktualisiere die Dokumentation für Download-Regel, HTML-Mail, Bilder und R-11.
Deploy Development / deploy (push) Successful in 41s
Test Suite / pytest-backend (push) Canceled after 3m21s
Test Suite / smoke-dev (push) Canceled after 0s
Test Suite / frontend-build (push) Canceled after 0s
2026-10-23 13:02:36 +02:00
LarsandCursor 7a1206b18c Speichere die neue Position einer verschobenen IMAP-Nachricht, damit der nächste Aufruf sie direkt öffnet.
Test Suite / pytest-backend (push) Canceled after 0s
Test Suite / smoke-dev (push) Canceled after 0s
Test Suite / frontend-build (push) Canceled after 0s
Deploy Development / deploy (push) Successful in 43s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-23 13:00:25 +02:00
LarsandCursor cd239a3ab8 Zeige PDF-Anlagen in der Mail-Ansicht, damit der Browser sie nicht unter einem zufälligen Namen speichert.
Deploy Development / deploy (push) Successful in 47s
Test Suite / pytest-backend (push) Successful in 6m16s
Test Suite / smoke-dev (push) Successful in 1s
Test Suite / frontend-build (push) Successful in 22s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-23 12:59:07 +02:00
StommerandClaude Sonnet 5.5 d0f09830bb Behebe R-11: Übersicht und Suche lesen in Blöcken und prüfen Rechte vor der Begrenzung.
Deploy Development / deploy (push) Successful in 42s
Test Suite / pytest-backend (push) Successful in 5m53s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 21s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 17:26:06 +02:00
StommerandClaude Sonnet 5.5 a3f328832e Lade Bilder aus E-Mails auf Wunsch über den Server nach und verkleinere den Kopf der Leseansicht auf dem Handy.
Deploy Development / deploy (push) Successful in 41s
Test Suite / pytest-backend (push) Successful in 5m32s
Test Suite / smoke-dev (push) Successful in 0s
Test Suite / frontend-build (push) Successful in 21s
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 16:47:24 +02:00
52 changed files with 3437 additions and 545 deletions

No files matched your search

+38 -10
View File
@@ -1,16 +1,15 @@
name: Test Suite name: Test Suite
# One run per push. The deploy starts at the same time and updates the checkout this suite tests,
# so the jobs wait until that checkout contains the pushed commit. A workflow_run trigger
# (after the deploy) did not start reliably on this Gitea, see git log of this file.
on: on:
push: push:
branches: [develop] branches: [develop]
workflow_run:
workflows: ["Deploy Development"]
types: [completed]
workflow_dispatch: workflow_dispatch:
jobs: jobs:
pytest-backend: pytest-backend:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: pytest against Dev Postgres (tsumugu_test) - name: pytest against Dev Postgres (tsumugu_test)
@@ -19,6 +18,19 @@ jobs:
APP_DIR="/home/lars/docker/tsumugu-dev" APP_DIR="/home/lars/docker/tsumugu-dev"
COMPOSE_FILE="docker-compose.dev-env.yml" COMPOSE_FILE="docker-compose.dev-env.yml"
echo "pytest uses Dev Postgres (${APP_DIR}); event=${{ github.event_name }}" echo "pytest uses Dev Postgres (${APP_DIR}); event=${{ github.event_name }}"
if [ "${{ github.event_name }}" = "push" ]; then
for i in $(seq 1 180); do
if git -c safe.directory='*' -C "$APP_DIR" merge-base --is-ancestor "${{ github.sha }}" HEAD 2>/dev/null; then
echo "Dev checkout contains ${{ github.sha }} (attempt $i)"
break
fi
if [ "$i" -eq 180 ]; then
echo "the deploy did not reach ${{ github.sha }} within 15 minutes"
exit 1
fi
sleep 5
done
fi
cd "$APP_DIR" cd "$APP_DIR"
for i in $(seq 1 60); do for i in $(seq 1 60); do
if [ -f "$APP_DIR/backend/pytest.ini" ] && docker compose -f "$COMPOSE_FILE" exec -T backend true 2>/dev/null; then if [ -f "$APP_DIR/backend/pytest.ini" ] && docker compose -f "$COMPOSE_FILE" exec -T backend true 2>/dev/null; then
@@ -59,27 +71,43 @@ jobs:
' '
smoke-dev: smoke-dev:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }} # After pytest, so the deploy that started with the push has normally finished. The health check retries.
needs: pytest-backend
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Non-destructive smoke against the running Dev app - name: Non-destructive smoke against the running Dev app
run: | run: |
set -e set -e
echo "live Dev API (tsumugu_dev), read-only health" echo "live Dev API (tsumugu_dev), read-only health"
curl -sf http://localhost:8094/api/health for url in http://localhost:8094/api/health http://localhost:3094/api/health; do
echo for i in $(seq 1 30); do
curl -sf http://localhost:3094/api/health if curl -sf "$url"; then echo; break; fi
echo if [ "$i" -eq 30 ]; then echo "no health answer from $url"; exit 1; fi
sleep 10
done
done
echo "LAN smoke OK. Public URL after TLS: https://dev.tsumugu.jinkendo.de" echo "LAN smoke OK. Public URL after TLS: https://dev.tsumugu.jinkendo.de"
frontend-build: frontend-build:
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Frontend production build on Dev checkout - name: Frontend production build on Dev checkout
run: | run: |
set -e set -e
APP_DIR="/home/lars/docker/tsumugu-dev" APP_DIR="/home/lars/docker/tsumugu-dev"
if [ "${{ github.event_name }}" = "push" ]; then
for i in $(seq 1 180); do
if git -c safe.directory='*' -C "$APP_DIR" merge-base --is-ancestor "${{ github.sha }}" HEAD 2>/dev/null; then
echo "Dev checkout contains ${{ github.sha }} (attempt $i)"
break
fi
if [ "$i" -eq 180 ]; then
echo "the deploy did not reach ${{ github.sha }} within 15 minutes"
exit 1
fi
sleep 5
done
fi
cd "$APP_DIR/frontend" cd "$APP_DIR/frontend"
npm ci npm ci
npm run build npm run build
+52
View File
@@ -0,0 +1,52 @@
"""Placing a source document into a container: a new reference, or one more binding of an existing element."""
from __future__ import annotations
from fastapi import APIRouter, Depends
from fastapi.responses import JSONResponse
from psycopg import Connection
from app.access.policy import Ctx
from app.api.deps import Principal, get_conn, get_principal
from app.api.schemas import ExistingIn, ExternalIn
from app.ops import bind_existing_external, create_external, existing_externals
router = APIRouter()
def _ctx_model(value) -> Ctx | None:
return None if value is None else Ctx(value.kind, value.id)
def place_external(conn, principal: Principal, body: ExternalIn) -> dict:
context = Ctx(body.context.kind, body.context.id)
placement = _ctx_model(body.placement)
if body.bind_element_id is not None:
return bind_existing_external(conn, principal.account_id, principal.scope_id, body.bind_element_id, context, placement, body.chapter_id)
return create_external(
conn,
principal.account_id,
principal.scope_id,
body.title,
body.source,
body.external_id,
context,
placement,
body.chapter_id,
body.dates.document_on,
body.dates.sent_on,
body.dates.received_on,
body.detail.model_dump(mode="json", exclude_none=True) if body.detail else None,
duplicate=body.duplicate,
)
@router.post("/api/v1/external-refs/existing")
def existing(body: ExistingIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
"""Read only: which of these documents are already present around the target. Nothing is written."""
items = [
{"source": item.source, "external_id": item.external_id, "connector_id": str(item.connector_id) if item.connector_id else None}
for item in body.items
]
result = existing_externals(conn, principal.account_id, Ctx(body.context.kind, body.context.id), _ctx_model(body.placement), items)
return JSONResponse(result, headers={"Cache-Control": "no-store"})
+41
View File
@@ -0,0 +1,41 @@
"""Account preferences: read the effective value, change the own value, bind the team."""
from __future__ import annotations
from fastapi import APIRouter, Depends
from fastapi.responses import JSONResponse
from psycopg import Connection
from app.access.policy import audit
from app.api.deps import Principal, get_conn, get_principal
from app.api.schemas import PreferenceIn
from app.preferences import preference_state, store_preference
router = APIRouter()
_NO_STORE = {"Cache-Control": "no-store"}
@router.get("/api/v1/auth/preferences/{key}")
def get_preference(key: str, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
return JSONResponse(preference_state(conn, principal.account_id, key), headers=_NO_STORE)
@router.put("/api/v1/auth/preferences/{key}")
def put_preference(
key: str,
body: PreferenceIn,
principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn),
):
state = store_preference(conn, principal.account_id, key, body.value, body.team_mode)
audit(
conn,
principal.account_id,
None,
"preference_changed",
"account",
principal.account_id,
detail={"key": key, "value": body.value, "team_mode": body.team_mode},
)
return JSONResponse(state, headers=_NO_STORE)
+45 -33
View File
@@ -34,6 +34,7 @@ from app.access.policy import (
) )
from app.access.limits import ensure_can_add_member from app.access.limits import ensure_can_add_member
from app.api.deps import Principal, get_conn, get_principal, get_session from app.api.deps import Principal, get_conn, get_principal, get_session
from app.api.externals import place_external
from app.api.schemas import ( from app.api.schemas import (
AcceptInviteIn, AcceptInviteIn,
AnnotationIn, AnnotationIn,
@@ -113,7 +114,6 @@ from app.ops import (
create_chapter, create_chapter,
create_cross_link, create_cross_link,
create_deadline, create_deadline,
create_external,
create_file, create_file,
create_matter, create_matter,
create_note, create_note,
@@ -860,21 +860,7 @@ def post_annotation(
@router.post("/api/v1/external-refs") @router.post("/api/v1/external-refs")
def post_external(body: ExternalIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): def post_external(body: ExternalIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
element = create_external( element = place_external(conn, principal, body)
conn,
principal.account_id,
principal.scope_id,
body.title,
body.source,
body.external_id,
Ctx(body.context.kind, body.context.id),
_ctx_model(body.placement),
body.chapter_id,
body.dates.document_on,
body.dates.sent_on,
body.dates.received_on,
body.detail.model_dump(mode="json", exclude_none=True) if body.detail else None,
)
return _created(f"/api/v1/elements/{element['id']}", {"id": str(element["id"]), "title": element["title"]}) return _created(f"/api/v1/elements/{element['id']}", {"id": str(element["id"]), "title": element["title"]})
@@ -1182,37 +1168,61 @@ def post_fulfill(
return deadline_payload(conn, principal.account_id, deadline) return deadline_payload(conn, principal.account_id, deadline)
VIEW_LIMIT = 200
SEARCH_LIMIT = 50
SCAN_BATCH = 200
SCAN_CAP = 20000
def _scan(conn, sql: str, params: tuple = ()):
"""Rows in the order of the query, read in batches.
The rights are checked per row after reading. A fixed LIMIT in the query would count rows
of other people too, so the own ones could fall off the end. The reader stops once it has
enough. `SCAN_CAP` bounds the work on very large data sets.
"""
offset = 0
while offset < SCAN_CAP:
rows = conn.execute(f"{sql} LIMIT %s OFFSET %s", (*params, SCAN_BATCH, offset)).fetchall()
yield from rows
if len(rows) < SCAN_BATCH:
return
offset += SCAN_BATCH
@router.get("/api/v1/views/open-tasks") @router.get("/api/v1/views/open-tasks")
def open_tasks(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): def open_tasks(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
rows = conn.execute( items = []
for task in _scan(
conn,
""" """
SELECT * FROM tasks SELECT * FROM tasks
WHERE deleted_at IS NULL AND task_status = 'open' WHERE deleted_at IS NULL AND task_status = 'open'
ORDER BY due_on NULLS LAST, created_at, id ORDER BY due_on NULLS LAST, created_at, id
LIMIT 200 """,
""" ):
).fetchall()
items = []
for task in rows:
if _task_visible(conn, principal.account_id, task): if _task_visible(conn, principal.account_id, task):
items.append(task_payload(conn, principal.account_id, task)) items.append(task_payload(conn, principal.account_id, task))
if len(items) >= VIEW_LIMIT:
break
return {"items": items, "next_cursor": None} return {"items": items, "next_cursor": None}
@router.get("/api/v1/views/open-deadlines") @router.get("/api/v1/views/open-deadlines")
def open_deadlines(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): def open_deadlines(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
rows = conn.execute( items = []
for deadline in _scan(
conn,
""" """
SELECT * FROM deadlines SELECT * FROM deadlines
WHERE deleted_at IS NULL AND deadline_status = 'open' WHERE deleted_at IS NULL AND deadline_status = 'open'
ORDER BY due_on, created_at, id ORDER BY due_on, created_at, id
LIMIT 200 """,
""" ):
).fetchall()
items = []
for deadline in rows:
if _deadline_visible(conn, principal.account_id, deadline): if _deadline_visible(conn, principal.account_id, deadline):
items.append(deadline_payload(conn, principal.account_id, deadline)) items.append(deadline_payload(conn, principal.account_id, deadline))
if len(items) >= VIEW_LIMIT:
break
return {"items": items, "next_cursor": None} return {"items": items, "next_cursor": None}
@@ -1266,19 +1276,21 @@ def search(q: str, principal: Principal = Depends(get_principal), conn: Connecti
], ],
} }
) )
elements = conn.execute( elements = _scan(
conn,
""" """
SELECT e.* SELECT e.*
FROM elements e FROM elements e
LEFT JOIN note_bodies n ON n.element_id = e.id LEFT JOIN note_bodies n ON n.element_id = e.id
WHERE e.deleted_at IS NULL WHERE e.deleted_at IS NULL
AND (e.title ILIKE %s ESCAPE '\\' OR COALESCE(n.document_json::text, '') ILIKE %s ESCAPE '\\') AND (e.title ILIKE %s ESCAPE '\\' OR COALESCE(n.document_json::text, '') ILIKE %s ESCAPE '\\')
ORDER BY e.updated_at DESC ORDER BY e.updated_at DESC, e.id
LIMIT 100
""", """,
(pattern, pattern), (pattern, pattern),
).fetchall() )
for element in elements: for element in elements:
if len(items) >= SEARCH_LIMIT:
break
from app.access.policy import visible_bindings_for_element from app.access.policy import visible_bindings_for_element
bindings = visible_bindings_for_element(conn, principal.account_id, element) bindings = visible_bindings_for_element(conn, principal.account_id, element)
@@ -1306,7 +1318,7 @@ def search(q: str, principal: Principal = Depends(get_principal), conn: Connecti
], ],
} }
) )
return {"items": items[:50], "next_cursor": None} return {"items": items[:SEARCH_LIMIT], "next_cursor": None}
@router.patch("/api/v1/cases/{case_id}/content-order") @router.patch("/api/v1/cases/{case_id}/content-order")
+23
View File
@@ -352,6 +352,22 @@ class ExternalIn(APIModel):
chapter_id: UUID | None = None chapter_id: UUID | None = None
dates: DatesIn = DatesIn() dates: DatesIn = DatesIn()
detail: ExternalDetail | None = None detail: ExternalDetail | None = None
# "ask": a document that is already present at the account's check level answers 409.
# "insert": the user chose to insert it again. `bind_element_id` binds that element instead of creating one.
duplicate: Literal["ask", "insert"] = "ask"
bind_element_id: UUID | None = None
class ExistingItemIn(APIModel):
source: str = Field(min_length=1, max_length=40)
external_id: str = Field(min_length=1, max_length=200)
connector_id: UUID | None = None
class ExistingIn(APIModel):
context: ContextIn
placement: ContextIn | None = None
items: list[ExistingItemIn] = Field(min_length=1, max_length=100)
class ConnectorIn(APIModel): class ConnectorIn(APIModel):
@@ -521,6 +537,13 @@ class DispatchCorrectIn(APIModel):
clear_counterpart: bool = False clear_counterpart: bool = False
class PreferenceIn(APIModel):
"""`value` replaces the own value; null means: inherit. `team_mode` is for main users and stays as it is when absent."""
value: str | None = Field(default=None, max_length=40)
team_mode: Literal["free", "locked"] | None = None
class DownloadPolicyIn(APIModel): class DownloadPolicyIn(APIModel):
"""Who may save or open the original outside of Tsumugu. `direct` only applies to the account default.""" """Who may save or open the original outside of Tsumugu. `direct` only applies to the account default."""
+70 -117
View File
@@ -4,18 +4,19 @@ from __future__ import annotations
from uuid import UUID from uuid import UUID
from fastapi import APIRouter, Depends from fastapi import APIRouter, Depends, Request
from fastapi.responses import JSONResponse, Response from fastapi.responses import JSONResponse, Response
from psycopg import Connection from psycopg import Connection
from app.access.policy import Ctx, element_rights, get_element, parse_query_context from app.access.policy import element_rights, get_element, parse_query_context
from app.api.deps import Principal, get_conn, get_principal from app.api.deps import Principal, get_conn, get_principal
from app.api.routes import _created, _ctx_model from app.api.routes import _created
from app.api.schemas import ConnectorIn, ExternalIn, SourceAssignmentIn from app.api.schemas import ConnectorIn, ExternalIn, SourceAssignmentIn
from app.core.errors import DOWNLOAD_REFUSED, forbidden, not_found, unprocessable, unsupported from app.core.errors import DOWNLOAD_REFUSED, forbidden, not_found, unprocessable, unsupported
from app.mailpack import parse_mail from app.mailpack import _inline_images, parse_mail, sanitize_html
from app.remoteimages import RemoteImages
from app.media.storage import content_disposition from app.media.storage import content_disposition
from app.ops import create_external from app.api.externals import place_external
from app.retrieval import RetrievalFailure from app.retrieval import RetrievalFailure
from app.sourcebind import assign_reference from app.sourcebind import assign_reference
from app.sourceport import capability_payload, with_capabilities from app.sourceport import capability_payload, with_capabilities
@@ -133,6 +134,7 @@ def view_mail(
"message_cc": parsed.copy, "message_cc": parsed.copy,
"text": parsed.text, "text": parsed.text,
"html": parsed.html, "html": parsed.html,
"remote_images": len(parsed.remote_images),
"text_truncated": parsed.text_truncated, "text_truncated": parsed.text_truncated,
"complete": not parsed.missing, "complete": not parsed.missing,
"notices": parsed.notices, "notices": parsed.notices,
@@ -190,6 +192,25 @@ def _complete_mail(conn, principal: Principal, element, context, ref):
return parsed, raw return parsed, raw
@router.get("/api/v1/external-refs/{element_id}/mail/html-with-images")
def view_mail_with_images(
element_id: UUID,
context_kind: str | None = None,
context_id: str | None = None,
principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn),
):
"""The formatted mail with its pictures from other servers, loaded by the server on request."""
element, context, ref = _opened_ref(conn, principal, element_id, context_kind, context_id)
parsed, _raw = _complete_mail(conn, principal, element, context, ref)
loader = RemoteImages()
rendered = sanitize_html(parsed.html_source, _inline_images(parsed.attachments), remote=loader) if parsed.html_source else ""
return JSONResponse(
content={"html": rendered, "loaded": loader.loaded, "failed": loader.failed},
headers={"Cache-Control": "private, no-store", "Referrer-Policy": "no-referrer"},
)
@router.get("/api/v1/external-refs/{element_id}/mail/original") @router.get("/api/v1/external-refs/{element_id}/mail/original")
def download_mail( def download_mail(
element_id: UUID, element_id: UUID,
@@ -221,13 +242,18 @@ def download_mail_attachment(
if index < 0 or index >= len(parsed.attachments): if index < 0 or index >= len(parsed.attachments):
raise not_found() raise not_found()
item = parsed.attachments[index] item = parsed.attachments[index]
return _mail_bytes(item.data, item.media_type or "application/octet-stream", item.filename, inline=True) name = item.display_name or item.filename or "anlage"
media = (item.media_type or "application/octet-stream").split(";", 1)[0].strip()
if item.data[:1024].lstrip(b"\xef\xbb\xbf\r\n\t ").startswith(b"%PDF"):
media = "application/pdf"
if not name.lower().endswith(".pdf"):
leaf = name.replace("\\", "/").split("/")[-1]
name = name if "." in leaf else f"{name}.pdf"
return _mail_bytes(item.data, media or "application/octet-stream", name, inline=True)
def _mail_bytes(data: bytes, media_type: str, filename: str, inline: bool = False) -> Response: def _mail_bytes(data: bytes, media_type: str, filename: str, inline: bool = False) -> Response:
disposition = content_disposition(filename) disposition = content_disposition(filename, inline=inline)
if inline:
disposition = disposition.replace("attachment;", "inline;", 1)
return Response( return Response(
content=data, content=data,
media_type=media_type, media_type=media_type,
@@ -238,40 +264,23 @@ def _mail_bytes(data: bytes, media_type: str, filename: str, inline: bool = Fals
) )
# These belong to the route. Every other query parameter is a filter the registered module must declare.
_ROUTE_PARAMS = frozenset({"q", "connector_id", "mailbox", "before", "uidvalidity"})
def _filters(request: Request) -> dict:
return {name: value for name, value in request.query_params.items() if name not in _ROUTE_PARAMS}
@router.get("/api/v1/sources/paperless/documents") @router.get("/api/v1/sources/paperless/documents")
def search_paperless( def search_paperless(
request: Request,
q: str = "", q: str = "",
connector_id: UUID | None = None, connector_id: UUID | None = None,
title: str = "",
correspondent: str = "",
document_type: str = "",
tag: str = "",
document_from: str = "",
document_to: str = "",
sort: str = "",
cursor: str = "",
principal: Principal = Depends(get_principal), principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn), conn: Connection = Depends(get_conn),
): ):
return _run_search( return _run_search("paperless", conn, principal.account_id, q.strip(), connector_id, "", None, _filters(request))
"paperless",
conn,
principal.account_id,
q.strip(),
connector_id,
"",
None,
{
"title": title,
"correspondent": correspondent,
"document_type": document_type,
"tag": tag,
"document_from": document_from,
"document_to": document_to,
"sort": sort,
"cursor": cursor,
},
)
@router.get("/api/v1/sources/{kind}/facets") @router.get("/api/v1/sources/{kind}/facets")
@@ -302,6 +311,21 @@ def preview_paperless_document(
) )
@router.get("/api/v1/sources/{kind}/thumbnail")
def thumbnail_source(
kind: str,
connector_id: UUID,
item: str,
principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn),
):
module = require_source(kind)
if not module.capabilities["search"].get("result_preview") or module.thumbnail is None:
raise unsupported("Diese Quelle hat keine Vorschaubilder.")
data, media = module.thumbnail(conn, principal.account_id, connector_id, item)
return Response(content=data, media_type=media, headers=_PRIVATE)
@router.get("/api/v1/sources/imap/mailboxes") @router.get("/api/v1/sources/imap/mailboxes")
def list_mailboxes( def list_mailboxes(
connector_id: UUID, connector_id: UUID,
@@ -313,18 +337,11 @@ def list_mailboxes(
@router.get("/api/v1/sources/imap/messages") @router.get("/api/v1/sources/imap/messages")
def search_mail( def search_mail(
request: Request,
connector_id: UUID, connector_id: UUID,
q: str = "", q: str = "",
mailbox: str = "", mailbox: str = "",
before: int | None = None, before: int | None = None,
title: str = "",
correspondent: str = "",
document_type: str = "",
tag: str = "",
document_from: str = "",
document_to: str = "",
sort: str = "",
cursor: str = "",
uidvalidity: int | None = None, uidvalidity: int | None = None,
principal: Principal = Depends(get_principal), principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn), conn: Connection = Depends(get_conn),
@@ -332,24 +349,7 @@ def search_mail(
if before is not None and before < 1: if before is not None and before < 1:
raise unprocessable("Die Seitenangabe ist ungültig.") raise unprocessable("Die Seitenangabe ist ungültig.")
return _run_search( return _run_search(
"imap", "imap", conn, principal.account_id, q.strip(), connector_id, mailbox.strip(), before, _filters(request), uidvalidity
conn,
principal.account_id,
q.strip(),
connector_id,
mailbox.strip(),
before,
{
"title": title,
"correspondent": correspondent,
"document_type": document_type,
"tag": tag,
"document_from": document_from,
"document_to": document_to,
"sort": sort,
"cursor": cursor,
},
uidvalidity,
) )
@@ -370,21 +370,7 @@ def read_mail(
def link_source(body: ExternalIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): def link_source(body: ExternalIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)):
if get_source(body.source) is None: if get_source(body.source) is None:
raise unprocessable("Verknüpft werden registrierte Quellen.") raise unprocessable("Verknüpft werden registrierte Quellen.")
element = create_external( element = place_external(conn, principal, body)
conn,
principal.account_id,
principal.scope_id,
body.title,
body.source,
body.external_id,
Ctx(body.context.kind, body.context.id),
_ctx_model(body.placement),
body.chapter_id,
body.dates.document_on,
body.dates.sent_on,
body.dates.received_on,
body.detail.model_dump(mode="json", exclude_none=True) if body.detail else None,
)
return _created(f"/api/v1/elements/{element['id']}", {"id": str(element["id"]), "title": element["title"]}) return _created(f"/api/v1/elements/{element['id']}", {"id": str(element["id"]), "title": element["title"]})
@@ -405,22 +391,14 @@ def _connector_parts(body: ConnectorIn, current: dict | None) -> tuple[str, dict
return parse_connector(body.kind, body.name, flat, body.config, current) return parse_connector(body.kind, body.name, flat, body.config, current)
_SEARCH_FLAGS = {
"title": "title",
"correspondent": "correspondent",
"document_type": "document_type",
"tag": "tags",
"document_from": "document_range",
"document_to": "document_range",
"sort": "sort",
"cursor": "continuation",
}
def _run_search(kind: str, conn, owner, query: str, connector_id: UUID | None, mailbox: str, before: int | None, filters: dict, uidvalidity: int | None = None) -> dict: def _run_search(kind: str, conn, owner, query: str, connector_id: UUID | None, mailbox: str, before: int | None, filters: dict, uidvalidity: int | None = None) -> dict:
module = require_source(kind) module = require_source(kind)
search = module.capabilities["search"] search = module.capabilities["search"]
refused = [name for name, value in filters.items() if str(value or "").strip() and not search.get(_SEARCH_FLAGS.get(name, name))] refused = [
name
for name, value in filters.items()
if str(value or "").strip() and not search.get(module.search_params.get(name, ""))
]
if before is not None and not search.get("continuation"): if before is not None and not search.get("continuation"):
refused.append("before") refused.append("before")
if refused: if refused:
@@ -443,43 +421,18 @@ def _run_navigate(kind: str, conn, owner, connector_id: UUID) -> dict:
@router.get("/api/v1/sources/{kind}/search") @router.get("/api/v1/sources/{kind}/search")
def search_source( def search_source(
request: Request,
kind: str, kind: str,
q: str = "", q: str = "",
connector_id: UUID | None = None, connector_id: UUID | None = None,
mailbox: str = "", mailbox: str = "",
before: int | None = None, before: int | None = None,
title: str = "",
correspondent: str = "",
document_type: str = "",
tag: str = "",
document_from: str = "",
document_to: str = "",
sort: str = "",
cursor: str = "",
principal: Principal = Depends(get_principal), principal: Principal = Depends(get_principal),
conn: Connection = Depends(get_conn), conn: Connection = Depends(get_conn),
): ):
if before is not None and before < 1: if before is not None and before < 1:
raise unprocessable("Die Seitenangabe ist ungültig.") raise unprocessable("Die Seitenangabe ist ungültig.")
return _run_search( return _run_search(kind, conn, principal.account_id, q.strip(), connector_id, mailbox.strip(), before, _filters(request))
kind,
conn,
principal.account_id,
q.strip(),
connector_id,
mailbox.strip(),
before,
{
"title": title,
"correspondent": correspondent,
"document_type": document_type,
"tag": tag,
"document_from": document_from,
"document_to": document_to,
"sort": sort,
"cursor": cursor,
},
)
@router.get("/api/v1/sources/{kind}/navigate") @router.get("/api/v1/sources/{kind}/navigate")
+166
View File
@@ -0,0 +1,166 @@
"""Documents of a source that are inserted into a container again.
The same document in different Vorgänge stays allowed. This module only finds where a document is
already present around the target, so the dialog can warn and the user can decide. Levels (account
preference `duplicate_check`):
- here: bound directly to the target.
- inside: also what the target shows: Teilvorgänge of a Vorgang, the linked Vorgänge of an Akte.
- around: also where the target lies: its Vorgang, the Akten it is linked to, the parent Akte.
Only elements the actor may read are reported. A hit outside the actor's rights does not exist here.
"""
from __future__ import annotations
from uuid import UUID
from app.access.policy import (
Ctx,
case_rights,
element_rights,
get_case,
get_matter,
matter_rights,
)
from app.core.errors import Problem
LEVELS = {"off": (), "here": ("here",), "inside": ("here", "inside"), "around": ("here", "inside", "around")}
_ORDER = {"here": 0, "inside": 1, "around": 2}
def _stored_connector(row: dict) -> str | None:
if row["connector_id"] is not None:
return str(row["connector_id"])
stored = (row["detail"] or {}).get("connector_id")
return str(stored) if stored else None
def _candidates(conn, source: str, external_id: str, connector_id: str | None) -> list[dict]:
"""Elements for the same document. `certain` is False when only one side knows the connection."""
rows = conn.execute(
"""
SELECT e.id, e.title, e.owner_subject_id, er.connector_id, er.detail
FROM external_references er
JOIN elements e ON e.id = er.element_id
WHERE er.source = %s AND er.external_id = %s AND e.deleted_at IS NULL
""",
(source, external_id.strip()[:200]),
).fetchall()
found = []
for row in rows:
stored = _stored_connector(row)
if stored is not None and connector_id is not None and stored != connector_id:
continue
found.append(row | {"certain": stored == connector_id})
return found
def _children(conn, matter_id: UUID) -> list[UUID]:
rows = conn.execute("SELECT id FROM matters WHERE parent_matter_id = %s AND deleted_at IS NULL", (matter_id,)).fetchall()
return [row["id"] for row in rows]
def _zones(conn, target: Ctx) -> dict[str, list[Ctx]]:
inside: list[Ctx] = []
around: list[Ctx] = []
if target.kind == "matter":
inside = [Ctx("matter", child) for child in _children(conn, target.id)]
matter = get_matter(conn, target.id)
anchors = [target.id]
if matter is not None and matter["parent_matter_id"] is not None:
around.append(Ctx("matter", matter["parent_matter_id"]))
anchors.append(matter["parent_matter_id"])
links = conn.execute("SELECT DISTINCT case_id FROM matter_case_links WHERE matter_id = ANY(%s)", (anchors,)).fetchall()
around += [Ctx("case", row["case_id"]) for row in links]
else:
links = conn.execute("SELECT matter_id FROM matter_case_links WHERE case_id = %s", (target.id,)).fetchall()
for row in links:
inside.append(Ctx("matter", row["matter_id"]))
inside += [Ctx("matter", child) for child in _children(conn, row["matter_id"])]
case = get_case(conn, target.id)
if case is not None and case["parent_case_id"] is not None:
around.append(Ctx("case", case["parent_case_id"]))
return {"here": [target], "inside": inside, "around": around}
def _blocked(conn, case_id: UUID, element_id: UUID) -> bool:
row = conn.execute(
"""
SELECT 1 FROM case_link_blocks b JOIN matter_case_links l ON l.id = b.link_id
WHERE l.case_id = %s AND b.element_id = %s
""",
(case_id, element_id),
).fetchone()
return row is not None
def _title_if_readable(conn, actor: UUID, container: Ctx) -> str | None:
if container.kind == "case":
case = get_case(conn, container.id)
return case["title"] if case is not None and case_rights(conn, actor, case).read else None
matter = get_matter(conn, container.id)
return matter["title"] if matter is not None and matter_rights(conn, actor, matter, None).read else None
def find_existing(
conn,
actor: UUID,
source: str,
external_id: str,
connector_id: str | None,
target: Ctx,
rights_context: Ctx | None,
level: str,
) -> list[dict]:
"""Matches of one document at the chosen level, readable by the actor, nearest zone first."""
zones = LEVELS.get(level, ())
if not zones:
return []
candidates = _candidates(conn, source, external_id, connector_id)
if not candidates:
return []
layout = _zones(conn, target)
by_element: dict[UUID, dict] = {}
for candidate in candidates:
bindings = conn.execute(
"SELECT b.context_kind, b.context_id, c.title AS chapter FROM bindings b LEFT JOIN chapters c ON c.id = b.chapter_id WHERE b.element_id = %s",
(candidate["id"],),
).fetchall()
for zone in zones:
for container in layout[zone]:
hit = next((b for b in bindings if b["context_kind"] == container.kind and b["context_id"] == container.id), None)
if hit is None:
continue
if target.kind == "case" and zone == "inside" and _blocked(conn, target.id, candidate["id"]):
continue
seen_in = [container] if zone == "around" else [c for c in (target, rights_context) if c is not None]
rights = [element_rights(conn, actor, candidate, ctx) for ctx in seen_in]
if not any(r.read for r in rights):
continue
match = {
"element_id": str(candidate["id"]),
"title": candidate["title"],
"where": zone,
"certain": candidate["certain"],
"chapter": hit["chapter"] if zone == "here" else None,
"container": {
"kind": container.kind,
"title": _title_if_readable(conn, actor, container) if zone != "here" else None,
},
"can_bind": any(r.share for r in rights),
}
known = by_element.get(candidate["id"])
if known is None or _ORDER[zone] < _ORDER[known["where"]]:
by_element[candidate["id"]] = match
return sorted(by_element.values(), key=lambda item: (_ORDER[item["where"]], item["title"]))
def already_present(matches: list[dict]) -> Problem:
where = {"here": "hier", "inside": "in diesem Bestand", "around": "im Umfeld"}[matches[0]["where"]]
return Problem(
409,
"Bereits vorhanden",
f"Dieses Dokument ist {where} schon vorhanden. Füge es nur dann erneut ein, wenn das gewollt ist.",
"already_present",
)
+65 -8
View File
@@ -6,6 +6,7 @@ The same tree feeds preview, reading, attachment download and export.
from __future__ import annotations from __future__ import annotations
import base64
import hashlib import hashlib
import html import html
import re import re
@@ -70,6 +71,8 @@ class ParsedMail:
attachments: list[AttachmentBytes] = field(default_factory=list) attachments: list[AttachmentBytes] = field(default_factory=list)
missing: list[str] = field(default_factory=list) missing: list[str] = field(default_factory=list)
html: str = "" html: str = ""
html_source: str = ""
remote_images: list[str] = field(default_factory=list)
notices: list[str] = field(default_factory=list) notices: list[str] = field(default_factory=list)
text_truncated: bool = False text_truncated: bool = False
document_on: str | None = None document_on: str | None = None
@@ -162,13 +165,19 @@ def parse_mail(raw: bytes) -> ParsedMail:
missing.append(label or f"Anlage {index}") missing.append(label or f"Anlage {index}")
notices.append(f"Die Anlage {label or index} ließ sich nicht vollständig lesen.") notices.append(f"Die Anlage {label or index} ließ sich nicht vollständig lesen.")
continue continue
data = _unwrap_pdf_payload(data, media, label)
if _is_pdf(data):
media = "application/pdf"
label = _pdf_filename(label)
name = _unique_filename(label or f"anlage-{index}.bin", used) name = _unique_filename(label or f"anlage-{index}.bin", used)
inline = disposition == "inline" or bool(content_id) inline = disposition == "inline" or bool(content_id)
attachments.append(_attachment(part, name, label or name, data, inline=inline)) attachments.append(_attachment(part, name, label or name, data, inline=inline, media=media))
simplified = "\n".join(chunk for chunk in plain if chunk).strip() simplified = "\n".join(chunk for chunk in plain if chunk).strip()
rendered = "" rendered = ""
if html_parts: if html_parts:
rendered = sanitize_html("\n".join(html_parts), _inline_images(attachments)) source = "\n".join(html_parts)
remote_found: list[str] = []
rendered = sanitize_html(source, _inline_images(attachments), found=remote_found)
if not simplified: if not simplified:
simplified = simplified_text("\n".join(html_parts)) simplified = simplified_text("\n".join(html_parts))
if missing: if missing:
@@ -183,6 +192,8 @@ def parse_mail(raw: bytes) -> ParsedMail:
attachments=attachments, attachments=attachments,
missing=missing, missing=missing,
html=rendered, html=rendered,
html_source=source if html_parts else "",
remote_images=remote_found if html_parts else [],
notices=notices, notices=notices,
document_on=_message_date(message.get("Date")), document_on=_message_date(message.get("Date")),
) )
@@ -235,8 +246,11 @@ def simplified_text(value: str) -> str:
return text.strip() return text.strip()
def sanitize_html(value: str, images: dict[str, str] | None = None) -> str: def sanitize_html(value: str, images: dict[str, str] | None = None, *, remote=None, found: list[str] | None = None) -> str:
"""Keep a formatted reading. Scripts, active content and remote loads stay out.""" """Keep a formatted reading. Scripts and active content stay out.
Pictures from other servers stay out too, unless `remote` turns an address into an inline picture.
`found` collects the addresses that were met."""
images = images or {} images = images or {}
class Cleaner(HTMLParser): class Cleaner(HTMLParser):
@@ -252,7 +266,7 @@ def sanitize_html(value: str, images: dict[str, str] | None = None) -> str:
return return
if self.skip or name not in _ALLOWED_TAGS: if self.skip or name not in _ALLOWED_TAGS:
return return
rendered = _clean_attrs(name, attrs, images) rendered = _clean_attrs(name, attrs, images, remote, found)
if rendered is None: if rendered is None:
return return
self.parts.append(f"<{name}{rendered}>") self.parts.append(f"<{name}{rendered}>")
@@ -299,7 +313,7 @@ def _safe_style(value: str) -> str:
return "; ".join(kept) return "; ".join(kept)
def _clean_attrs(tag: str, attrs, images: dict[str, str]) -> str | None: def _clean_attrs(tag: str, attrs, images: dict[str, str], remote=None, found: list[str] | None = None) -> str | None:
kept: list[str] = [] kept: list[str] = []
for key, value in attrs: for key, value in attrs:
name = (key or "").lower() name = (key or "").lower()
@@ -332,6 +346,10 @@ def _clean_attrs(tag: str, attrs, images: dict[str, str]) -> str | None:
if source.lower().startswith("cid:"): if source.lower().startswith("cid:"):
token = source[4:].strip().strip("<>") token = source[4:].strip().strip("<>")
source = images.get(token.casefold(), "") source = images.get(token.casefold(), "")
elif re.match(r"https://", source, re.IGNORECASE):
if found is not None and source not in found:
found.append(source)
source = remote(source) if remote is not None else ""
if not _DATA_IMAGE.match(source): if not _DATA_IMAGE.match(source):
continue continue
kept.append(f' src="{source}"') kept.append(f' src="{source}"')
@@ -397,10 +415,49 @@ def _content_id(part: EmailMessage) -> str:
return token return token
def _attachment(part: EmailMessage, filename: str, label: str, data: bytes, inline: bool) -> AttachmentBytes: def _is_pdf(data: bytes) -> bool:
return data[:1024].lstrip(b"\xef\xbb\xbf\r\n\t ").startswith(b"%PDF")
def _unwrap_pdf_payload(data: bytes, media: str, name: str) -> bytes:
"""A PDF that arrived as undecoded Base64 is still the PDF, not the transfer text."""
if _is_pdf(data):
return data
declared = (media or "").split(";", 1)[0].strip().lower()
hinted = "pdf" in declared or (name or "").lower().endswith(".pdf") or declared in {
"application/octet-stream",
"application/x-octet-stream",
"binary/octet-stream",
}
if not hinted:
return data
stripped = b"".join(data.split())
if len(stripped) < 16 or re.fullmatch(rb"[A-Za-z0-9+/=]+", stripped) is None:
return data
try:
decoded = base64.b64decode(stripped, validate=True)
except Exception:
return data
return decoded if _is_pdf(decoded) else data
def _pdf_filename(name: str) -> str:
text = (name or "").strip() or "anlage"
if text.lower().endswith(".pdf"):
return text
if text.lower().endswith(".bin"):
text = text[:-4].rstrip(" .") or "anlage"
leaf = text.replace("\\", "/").split("/")[-1]
if "." in leaf:
return text
return f"{text}.pdf"
def _attachment(part: EmailMessage, filename: str, label: str, data: bytes, inline: bool, media: str | None = None) -> AttachmentBytes:
resolved = (media or part.get_content_type() or "application/octet-stream").split(";", 1)[0].strip()
return AttachmentBytes( return AttachmentBytes(
filename=filename, filename=filename,
media_type=(part.get_content_type() or "application/octet-stream")[:120], media_type=(resolved or "application/octet-stream")[:120],
data=data, data=data,
sha256=hashlib.sha256(data).hexdigest(), sha256=hashlib.sha256(data).hexdigest(),
display_name=label or filename, display_name=label or filename,
+4
View File
@@ -13,7 +13,9 @@ from app.api.accounts import router as accounts_router
from app.api.correspondence import router as correspondence_router from app.api.correspondence import router as correspondence_router
from app.api.downloads import router as downloads_router from app.api.downloads import router as downloads_router
from app.api.export import router as export_router from app.api.export import router as export_router
from app.api.externals import router as externals_router
from app.api.library import router as library_router from app.api.library import router as library_router
from app.api.preferences import router as preferences_router
from app.api.rooms import router as rooms_router from app.api.rooms import router as rooms_router
from app.api.roomshare import router as roomshare_router from app.api.roomshare import router as roomshare_router
from app.api.team import router as team_router from app.api.team import router as team_router
@@ -130,6 +132,8 @@ def create_app() -> FastAPI:
app.include_router(sources_router) app.include_router(sources_router)
app.include_router(export_router) app.include_router(export_router)
app.include_router(downloads_router) app.include_router(downloads_router)
app.include_router(preferences_router)
app.include_router(externals_router)
app.include_router(correspondence_router) app.include_router(correspondence_router)
return app return app
+14 -6
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import hashlib import hashlib
from pathlib import Path from pathlib import Path
from urllib.parse import quote
from uuid import UUID from uuid import UUID
from app.core.errors import storage_unavailable, unprocessable from app.core.errors import storage_unavailable, unprocessable
@@ -66,10 +67,17 @@ def display_filename(name: str) -> str:
return text[:240] return text[:240]
def content_disposition(filename: str) -> str: def content_disposition(filename: str, *, inline: bool = False) -> str:
"""The real name goes along in UTF-8 (RFC 5987), with a plain ASCII fallback for old clients.""" """The real name goes along in UTF-8 (RFC 5987), with a plain ASCII fallback for old clients."""
from urllib.parse import quote name = display_filename(filename)
kind = "inline" if inline else "attachment"
safe = display_filename(filename).replace('"', "") suffix = ""
fallback = safe.encode("ascii", "replace").decode("ascii").replace("?", "_") leaf = name.rsplit(".", 1)
return f"attachment; filename=\"{fallback}\"; filename*=UTF-8''{quote(safe, safe='')}" if len(leaf) == 2 and leaf[1].isascii() and leaf[1].isalnum():
suffix = f".{leaf[1]}"
fallback = name.encode("ascii", "replace").decode("ascii").replace("?", "_").replace('"', "").replace("\\", "_").replace(";", "_")
fallback = " ".join(fallback.split())
if suffix and not fallback.lower().endswith(suffix.lower()):
fallback = f"{fallback}{suffix}".strip()
fallback = fallback or f"datei{suffix}"
return f"{kind}; filename=\"{fallback}\"; filename*=UTF-8''{quote(name, safe='')}"
+55 -1
View File
@@ -36,6 +36,8 @@ from app.access.policy import (
from app.content.documents import clean_document from app.content.documents import clean_document
from app.core.errors import forbidden, not_found, unprocessable, unsupported from app.core.errors import forbidden, not_found, unprocessable, unsupported
from app.core.timeutil import utcnow from app.core.timeutil import utcnow
from app.preferences import effective_preference
from app.duplicates import already_present, find_existing
from app.media.port import LEGACY_PROFILE, StoredHandle, adapter_for, instance_for_write, publish, removal_for from app.media.port import LEGACY_PROFILE, StoredHandle, adapter_for, instance_for_write, publish, removal_for
from app.media.storage import display_filename, sha256, sniff from app.media.storage import display_filename, sha256, sniff
@@ -1142,7 +1144,7 @@ def add_annotation(conn, actor: UUID, binding_id: UUID, body: str, context: Ctx
return row return row
def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str, external_id: str, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, detail: dict | None = None, communication_direction: str | None = None) -> dict: def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str, external_id: str, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, detail: dict | None = None, communication_direction: str | None = None, duplicate: str = "insert") -> dict:
ensure_can_create(conn, actor, scope_id) ensure_can_create(conn, actor, scope_id)
from app.sourceregistry import accepts_reference from app.sourceregistry import accepts_reference
@@ -1150,6 +1152,13 @@ def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str,
raise unprocessable("Diese Quelle ist nicht registriert.") raise unprocessable("Diese Quelle ist nicht registriert.")
target, rights_context = _placement(context, placement) target, rights_context = _placement(context, placement)
require_container_edit(conn, actor, target, rights_context) require_container_edit(conn, actor, target, rights_context)
if duplicate == "ask":
# The level is the account's choice. Only a certain match warns: an unclear connection never blocks.
connector = (detail or {}).get("connector_id")
level = effective_preference(conn, actor, "duplicate_check")
found = [item for item in find_existing(conn, actor, source, external_id, str(connector) if connector else None, target, rights_context, level) if item["certain"]]
if found:
raise already_present(found)
element = _new_element(conn, actor, scope_id, title, "external_ref") element = _new_element(conn, actor, scope_id, title, "external_ref")
conn.execute( conn.execute(
"INSERT INTO external_references (element_id, source, external_id, detail) VALUES (%s, %s, %s, %s)", "INSERT INTO external_references (element_id, source, external_id, detail) VALUES (%s, %s, %s, %s)",
@@ -1167,6 +1176,51 @@ def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str,
return element | {"binding_id": binding["id"]} return element | {"binding_id": binding["id"]}
def existing_externals(conn, actor: UUID, context: Ctx, placement: Ctx | None, items: list[dict]) -> dict:
"""For each document: where it is already present around the target, at the account's check level."""
target, rights_context = _placement(context, placement)
require_container_edit(conn, actor, target, rights_context)
level = effective_preference(conn, actor, "duplicate_check")
return {
"level": level,
"items": [
item
| {"matches": find_existing(conn, actor, item["source"], item["external_id"], item.get("connector_id"), target, rights_context, level)}
for item in items
],
}
def bind_existing_external(conn, actor: UUID, scope_id: UUID, element_id: UUID, context: Ctx, placement: Ctx | None, chapter_id: UUID | None) -> dict:
"""Bind an element that already holds the document into a further container: one element, one more binding.
The element must be a certain match around the target, readable for the actor, and the actor may share it
there. Nothing is copied and nothing is handed over: rights stay those of each context.
"""
ensure_can_create(conn, actor, scope_id)
target, rights_context = _placement(context, placement)
require_container_edit(conn, actor, target, rights_context)
ref = conn.execute(
"SELECT source, external_id, connector_id, detail FROM external_references WHERE element_id = %s",
(element_id,),
).fetchone()
if ref is None:
raise not_found()
connector = ref["connector_id"] or (ref["detail"] or {}).get("connector_id")
matches = find_existing(conn, actor, ref["source"], ref["external_id"], str(connector) if connector else None, target, rights_context, "around")
match = next((item for item in matches if item["element_id"] == str(element_id) and item["certain"]), None)
if match is None:
raise not_found()
if match["where"] == "here":
raise unprocessable("Das Dokument ist hier bereits eingebunden.")
if not match["can_bind"]:
raise forbidden("Du darfst dieses Element nicht in einen weiteren Zusammenhang einbinden.")
element = get_element(conn, element_id)
binding = insert_binding(conn, element_id, target.kind, target.id, chapter_id, actor)
audit(conn, actor, scope_id, "external_ref_bound", "element", element_id, rights_context or target)
return element | {"binding_id": binding["id"]}
def create_file(conn, actor: UUID, scope_id: UUID, title: str, filename: str, data: bytes, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, media_root: str, max_bytes: int, rollback_files: list[Path], communication_direction: str | None = None) -> dict: def create_file(conn, actor: UUID, scope_id: UUID, title: str, filename: str, data: bytes, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, media_root: str, max_bytes: int, rollback_files: list[Path], communication_direction: str | None = None) -> dict:
ensure_can_create(conn, actor, scope_id) ensure_can_create(conn, actor, scope_id)
if not data or len(data) > max_bytes: if not data or len(data) > max_bytes:
+112
View File
@@ -0,0 +1,112 @@
"""Account preferences with inheritance.
An employee inherits the value of the main user who sponsors the account and may change it,
unless the main user made the preference binding for the team. A binding value overrides an own
value without deleting it: it applies again once the binding is lifted.
Allowed values, default and lockability of each key are fixed here.
"""
from __future__ import annotations
from dataclasses import dataclass
from uuid import UUID
from psycopg.types.json import Json
from app.core.errors import forbidden, not_found, unprocessable
@dataclass(frozen=True)
class Preference:
default: str
allowed: tuple[str, ...]
team_lockable: bool = True
PREFERENCES: dict[str, Preference] = {
# Where an element that is inserted again counts as already present.
"duplicate_check": Preference("here", ("off", "here", "inside", "around")),
}
def spec(key: str) -> Preference:
found = PREFERENCES.get(key)
if found is None:
raise not_found()
return found
def _valid(wanted: Preference, value) -> str | None:
return value if isinstance(value, str) and value in wanted.allowed else None
def preference_state(conn, account_id: UUID, key: str) -> dict:
"""The effective value and where it comes from: locked, own, team or default."""
wanted = spec(key)
row = conn.execute(
"""
SELECT a.account_kind,
own.value AS own_value, own.team_mode AS own_mode,
sp.value AS team_value, sp.team_mode AS team_mode
FROM auth_accounts a
LEFT JOIN account_preferences own ON own.account_id = a.id AND own.key = %s
LEFT JOIN account_preferences sp ON sp.account_id = a.sponsor_id AND sp.key = %s
WHERE a.id = %s
""",
(key, key, account_id),
).fetchone()
if row is None:
raise not_found()
employee = row["account_kind"] == "employee"
own = _valid(wanted, row["own_value"])
team = _valid(wanted, row["team_value"]) if employee else None
locked = employee and wanted.team_lockable and row["team_mode"] == "locked"
inherited = team or wanted.default
if locked:
value, source = inherited, "locked"
elif own is not None:
value, source = own, "own"
elif team is not None:
value, source = team, "team"
else:
value, source = wanted.default, "default"
return {
"key": key,
"value": value,
"source": source,
"own": own,
"inherited": inherited,
"locked": locked,
"allowed": list(wanted.allowed),
"default": wanted.default,
"can_lock": row["account_kind"] == "main" and wanted.team_lockable,
"team_mode": row["own_mode"] if row["own_mode"] and not employee else "free",
}
def effective_preference(conn, account_id: UUID, key: str) -> str:
return preference_state(conn, account_id, key)["value"]
def store_preference(conn, account_id: UUID, key: str, value: str | None, team_mode: str | None) -> dict:
"""Own value (None: inherit) and, for a main user, the binding for the team."""
wanted = spec(key)
state = preference_state(conn, account_id, key)
if value is not None and value not in wanted.allowed:
raise unprocessable("Dieser Wert ist für die Einstellung nicht vorgesehen.")
if team_mode is not None and not state["can_lock"]:
raise forbidden("Nur der Hauptnutzer kann eine Einstellung für das Team verbindlich setzen.")
if state["locked"]:
raise forbidden("Der Hauptnutzer hat diese Einstellung für das Team vorgegeben.")
conn.execute(
"""
INSERT INTO account_preferences (account_id, key, value, team_mode)
VALUES (%s, %s, %s, COALESCE(%s, 'free'))
ON CONFLICT (account_id, key) DO UPDATE
SET value = EXCLUDED.value,
team_mode = COALESCE(%s, account_preferences.team_mode),
updated_at = now()
""",
(account_id, key, Json(value) if value is not None else None, team_mode, team_mode),
)
return preference_state(conn, account_id, key)
+2
View File
@@ -1082,4 +1082,6 @@ LABELS = {
"room_default_changed": "Voreinstellung des Datenraums geändert", "room_default_changed": "Voreinstellung des Datenraums geändert",
"room_download_policy_changed": "Download-Regel des Datenraums geändert", "room_download_policy_changed": "Download-Regel des Datenraums geändert",
"download_defaults_changed": "Standard für Downloads geändert", "download_defaults_changed": "Standard für Downloads geändert",
"preference_changed": "Einstellung des Kontos geändert",
"external_ref_bound": "Vorhandene Referenz zusätzlich eingebunden",
} }
+133
View File
@@ -0,0 +1,133 @@
"""Fetching the pictures a mail refers to, on the server and only on request.
The browser never contacts the sender: the server loads each picture, checks that it really is
a PNG, JPEG, GIF or WebP, and hands it over inline. This hides the reader's address and device.
It cannot hide that the mail was opened, because the sender sees the request.
Only public https addresses are fetched. The address is resolved once, every result must be a
public one, and the connection goes to exactly that address, so a changing DNS answer cannot
lead to an internal host. Redirects are followed a few times with the same checks.
"""
from __future__ import annotations
import base64
import http.client
import ipaddress
import socket
import ssl
from urllib.parse import urljoin, urlsplit
MAX_IMAGE_BYTES = 1_500_000
MAX_REDIRECTS = 3
TIMEOUT_SECONDS = 5
MAX_IMAGES = 12
MAX_TOTAL_BYTES = 6_000_000
_SIGNATURES = (
(b"\x89PNG\r\n\x1a\n", "png"),
(b"\xff\xd8\xff", "jpeg"),
(b"GIF87a", "gif"),
(b"GIF89a", "gif"),
)
class _Pinned(http.client.HTTPSConnection):
"""An https connection that goes to a fixed, already checked address."""
def __init__(self, host: str, address: str, port: int) -> None:
super().__init__(host, port, timeout=TIMEOUT_SECONDS, context=ssl.create_default_context())
self._address = address
def connect(self) -> None:
sock = socket.create_connection((self._address, self.port), self.timeout)
self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
def is_public(address: str) -> bool:
try:
ip = ipaddress.ip_address(address)
except ValueError:
return False
return ip.is_global and not ip.is_multicast
def _public_address(host: str, port: int) -> str | None:
try:
found = {item[4][0] for item in socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)}
except OSError:
return None
if not found or not all(is_public(address) for address in found):
return None
return sorted(found)[0]
def _kind(data: bytes) -> str | None:
for signature, kind in _SIGNATURES:
if data.startswith(signature):
return kind
if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
return "webp"
return None
def fetch_image(url: str) -> bytes | None:
"""The bytes of a picture, or None when anything about it is not safe."""
for _ in range(MAX_REDIRECTS + 1):
parts = urlsplit(url)
if parts.scheme != "https" or not parts.hostname or parts.username or parts.password:
return None
port = parts.port or 443
if port != 443:
return None
address = _public_address(parts.hostname, port)
if address is None:
return None
target = (parts.path or "/") + (f"?{parts.query}" if parts.query else "")
connection = _Pinned(parts.hostname, address, port)
try:
connection.request("GET", target, headers={"User-Agent": "Mozilla/5.0", "Accept": "image/png,image/jpeg,image/gif,image/webp", "Connection": "close"})
response = connection.getresponse()
if response.status in {301, 302, 303, 307, 308}:
location = response.getheader("Location")
if not location:
return None
url = urljoin(url, location)
continue
if response.status != 200:
return None
data = response.read(MAX_IMAGE_BYTES + 1)
except (OSError, http.client.HTTPException, ssl.SSLError):
return None
finally:
connection.close()
if len(data) > MAX_IMAGE_BYTES or _kind(data) is None:
return None
return data
return None
class RemoteImages:
"""Loads the pictures of one mail within a budget and remembers each address once."""
def __init__(self) -> None:
self.cache: dict[str, str] = {}
self.total = 0
self.loaded = 0
self.failed = 0
def __call__(self, url: str) -> str:
if url in self.cache:
return self.cache[url]
result = ""
if len(self.cache) < MAX_IMAGES and self.total < MAX_TOTAL_BYTES:
data = fetch_image(url)
if data is not None:
self.total += len(data)
result = f"data:image/{_kind(data)};base64,{base64.b64encode(data).decode('ascii')}"
self.cache[url] = result
if result:
self.loaded += 1
else:
self.failed += 1
return result
+1 -1
View File
@@ -44,7 +44,7 @@ def load_external(conn, actor_id: UUID, element_id: UUID, assigned: UUID | None)
ref = conn.execute( ref = conn.execute(
""" """
SELECT source, external_id, detail, connector_id, resolution SELECT element_id, source, external_id, detail, connector_id, resolution
FROM external_references WHERE element_id = %s FROM external_references WHERE element_id = %s
""", """,
(element_id,), (element_id,),
+39
View File
@@ -217,3 +217,42 @@ def assign_reference(conn, actor_id: UUID, element_id: UUID, connector_id: UUID)
detail = dict(ref["detail"] or {}) detail = dict(ref["detail"] or {})
detail["connector_id"] = str(connector_id) detail["connector_id"] = str(connector_id)
_set_binding(conn, element_id, connector_id, "bound", detail) _set_binding(conn, element_id, connector_id, "bound", detail)
def remember_imap_location(
conn,
element_id: UUID,
external_id: str,
previous_mailbox: str,
previous_uid: int,
mailbox: str,
uid: int,
) -> None:
"""Store the folder and UID of a message found at a new place.
Only the previous place is replaced. The message identity, the connector
and an adopted direction stay as they were.
"""
if (
not mailbox
or any(char in mailbox for char in "\r\n\x00")
or uid < 1
or (mailbox == previous_mailbox and uid == previous_uid)
):
return
conn.execute(
"""
UPDATE external_references
SET detail = jsonb_set(
jsonb_set(detail, '{mailbox}', to_jsonb(%s::text)),
'{uid}',
to_jsonb(%s::bigint)
)
WHERE element_id = %s
AND source = 'imap'
AND external_id = %s
AND detail->>'mailbox' = %s
AND detail->>'uid' = %s
""",
(mailbox, uid, element_id, external_id, previous_mailbox, str(previous_uid)),
)
+2 -63
View File
@@ -1,72 +1,15 @@
"""Declared source and editor capabilities. """Declared editor capabilities and the payloads that carry them.
A common contract does not invent abilities the adapter does not have. The capabilities of a source belong to its module. A filter is offered only when that module implements it.
A filter is offered only when that module implements it.
""" """
from __future__ import annotations from __future__ import annotations
PAPERLESS_CAPABILITIES = {
"kind": "paperless",
"navigate": False,
"read_original": True,
"read_preview": True,
"read_attachment": False,
"edit_content": False,
"edit_metadata": False,
"search": {
"query": True,
"title": True,
"correspondent": True,
"document_type": True,
"tags": True,
"document_range": True,
"sort": True,
"continuation": True,
},
}
IMAP_CAPABILITIES = {
"kind": "imap",
"navigate": True,
"read_original": True,
"read_preview": True,
"read_attachment": True,
"edit_content": False,
"edit_metadata": False,
"search": {
"query": True,
"title": False,
"correspondent": False,
"document_type": False,
"tags": False,
"document_range": False,
"sort": False,
"continuation": True,
},
}
CAPABILITIES = {
"paperless": PAPERLESS_CAPABILITIES,
"imap": IMAP_CAPABILITIES,
}
EDITORS = { EDITORS = {
"note": True, "note": True,
"markdown_file": False, "markdown_file": False,
} }
UNSUPPORTED_SEARCH = (
"title",
"correspondent",
"document_type",
"tag",
"document_from",
"document_to",
"sort",
"cursor",
)
def capability_payload() -> dict: def capability_payload() -> dict:
from app.sourceregistry import registered_sources from app.sourceregistry import registered_sources
@@ -91,10 +34,6 @@ def capability_payload() -> dict:
} }
def unsupported_filters(values: dict[str, str]) -> list[str]:
return [name for name in UNSUPPORTED_SEARCH if str(values.get(name) or "").strip()]
def with_capabilities(kind: str, connector_id: str, payload: dict) -> dict: def with_capabilities(kind: str, connector_id: str, payload: dict) -> dict:
from app.sourceregistry import require_source from app.sourceregistry import require_source
+6 -1
View File
@@ -2,7 +2,7 @@
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass from dataclasses import dataclass, field
from typing import Callable from typing import Callable
from app.core.errors import unprocessable from app.core.errors import unprocessable
@@ -65,6 +65,11 @@ class SourceModule:
navigate: Callable | None = None navigate: Callable | None = None
preview: Callable | None = None preview: Callable | None = None
facets: Callable | None = None facets: Callable | None = None
# Query parameter of a search, mapped to the capability flag that must be true for it.
# The module owns its filter names. Anything not declared here is refused.
search_params: dict[str, str] = field(default_factory=dict)
# thumbnail(conn, owner_id, connector_id, item) -> (bytes, media type), when the module declares it.
thumbnail: Callable | None = None
_MODULES: dict[str, SourceModule] = {} _MODULES: dict[str, SourceModule] = {}
+524 -58
View File
@@ -3,13 +3,18 @@
from __future__ import annotations from __future__ import annotations
import base64 import base64
import hashlib
import imaplib import imaplib
import json import json
import re import re
import ssl import ssl
import threading
import time
from dataclasses import dataclass
from email.header import decode_header from email.header import decode_header
from email.parser import BytesParser from email.parser import BytesParser
from email.utils import parsedate_to_datetime from email.utils import parsedate_to_datetime
from html import unescape
from pathlib import Path from pathlib import Path
from urllib.error import HTTPError, URLError from urllib.error import HTTPError, URLError
from urllib.parse import urlencode, urlparse from urllib.parse import urlencode, urlparse
@@ -33,7 +38,7 @@ from app.mailpack import (
simplified_text, simplified_text,
) )
from app.media.storage import sniff from app.media.storage import sniff
from app.sourcebind import resolve_for_read from app.sourcebind import remember_imap_location, resolve_for_read
from app.sourceaccess import ( from app.sourceaccess import (
ORIGINAL_UNAVAILABLE, ORIGINAL_UNAVAILABLE,
assert_still_owned, assert_still_owned,
@@ -43,7 +48,6 @@ from app.sourceaccess import (
connector_owner as _connector_owner, connector_owner as _connector_owner,
release_for_external, release_for_external,
) )
from app.sourceport import CAPABILITIES
from app.sourceregistry import ConfigField, SourceModule, get_source, parse_fields, register_source, require_source from app.sourceregistry import ConfigField, SourceModule, get_source, parse_fields, register_source, require_source
@@ -182,10 +186,24 @@ def _record_probe(conn, connector_id: UUID, ok: bool, detail: str) -> None:
) )
def _paperless_choice(value, label: str) -> str | None: PAPERLESS_PAGE_SIZE = 20
_PAPERLESS_MODES = {
# Paperless-ngx offers these searches in its own filter bar. The first needs no search index.
"title_content": "title_content",
"title": "title__icontains",
"content": "content__icontains",
"index": "query",
}
_PAPERLESS_TAG_MODES = {"all": "tags__id__all", "any": "tags__id__in", "none": "tags__id__none"}
_PAPERLESS_SORTS = {"created", "-created", "added", "-added", "title", "-title", "score", "-score"}
def _paperless_choice(value, label: str, allow_none: bool = False) -> str | None:
text = str(value or "").strip() text = str(value or "").strip()
if not text: if not text:
return None return None
if allow_none and text == "none":
return text
if not text.isdigit() or len(text) > 12: if not text.isdigit() or len(text) > 12:
raise unprocessable(f"{label} ist keine Auswahl dieser Instanz.") raise unprocessable(f"{label} ist keine Auswahl dieser Instanz.")
return text return text
@@ -200,45 +218,166 @@ def _paperless_day(value, label: str) -> str | None:
return text return text
def _paperless_params(query: str, options: dict | None) -> dict: def _paperless_range(options: dict, prefix: str, field: str, label: str, params: dict) -> None:
start = _paperless_day(options.get(f"{prefix}_from"), f"{label}, von")
end = _paperless_day(options.get(f"{prefix}_to"), f"{label}, bis")
if start and end and start > end:
raise unprocessable(f"{label}: Der Beginn liegt nach dem Ende.")
if start:
params[f"{field}__date__gte"] = start
if end:
params[f"{field}__date__lte"] = end
_CUSTOM_KINDS = {
"string": "text",
"url": "text",
"longtext": "text",
"integer": "number",
"float": "number",
"monetary": "number",
"date": "date",
"boolean": "boolean",
"select": "select",
}
_CUSTOM_OPERATORS = {
"text": {"exact", "icontains", "istartswith", "iendswith"},
"number": {"exact", "gt", "gte", "lt", "lte", "range"},
"date": {"exact", "gt", "gte", "lt", "lte", "range"},
"boolean": {"exact"},
"select": {"exact"},
}
_CUSTOM_LIMIT = 5
def _custom_number(value, label: str):
text = str(value).strip().replace(",", ".")
try:
number = float(text)
except ValueError as exc:
raise unprocessable(f"{label}: Der Wert ist keine Zahl.") from exc
if number != number or number in (float("inf"), float("-inf")):
raise unprocessable(f"{label}: Der Wert ist keine Zahl.")
return int(number) if "." not in text else number
def _custom_value(kind: str, op: str, value, field: dict):
label = field["name"]
if op == "exists":
if not isinstance(value, bool):
raise unprocessable(f"{label}: Gesetzt oder leer ist ungültig.")
return value
if op == "range":
if not isinstance(value, (list, tuple)) or len(value) != 2:
raise unprocessable(f"{label}: Der Bereich braucht Anfang und Ende.")
low, high = (_custom_value(kind, "exact", item, field) for item in value)
if low > high:
raise unprocessable(f"{label}: Der Beginn liegt nach dem Ende.")
return [low, high]
if kind == "text":
text = " ".join(str(value or "").split())
if not text or len(text) > 200:
raise unprocessable(f"{label}: Der Text fehlt oder ist zu lang.")
return text
if kind == "number":
return _custom_number(value, label)
if kind == "date":
day = _paperless_day(value, label)
if not day:
raise unprocessable(f"{label}: Das Datum fehlt.")
return day
if kind == "boolean":
if not isinstance(value, bool):
raise unprocessable(f"{label}: Ja oder Nein ist ungültig.")
return value
allowed = {option["id"] for option in field.get("options") or []}
if str(value) not in allowed:
raise unprocessable(f"{label}: Das ist keine Auswahl dieses Feldes.")
return str(value)
def _paperless_custom_query(raw, fields: list[dict]) -> str | None:
"""Paperless-ngx custom_field_query, built only from fields and operators this instance offers."""
if raw in (None, "", []):
return None
if isinstance(raw, str):
try:
raw = json.loads(raw)
except json.JSONDecodeError as exc:
raise unprocessable("Die Filter der Zusatzfelder sind ungültig.") from exc
if not isinstance(raw, list) or len(raw) > _CUSTOM_LIMIT:
raise unprocessable(f"Es sind höchstens {_CUSTOM_LIMIT} Zusatzfeld-Filter möglich.")
known = {field["id"]: field for field in fields}
atoms = []
for row in raw:
if not isinstance(row, dict):
raise unprocessable("Die Filter der Zusatzfelder sind ungültig.")
field = known.get(str(row.get("field") or ""))
if field is None:
raise unprocessable("Das Zusatzfeld gibt es an diesem Zugang nicht.")
kind = _CUSTOM_KINDS[field["type"]]
op = str(row.get("op") or "")
if op != "exists" and op not in _CUSTOM_OPERATORS[kind]:
raise unprocessable(f"{field['name']}: Diese Bedingung gibt es für den Feldtyp nicht.")
atoms.append([field["name"], op, _custom_value(kind, op, row.get("value"), field)])
if not atoms:
return None
return json.dumps(atoms[0] if len(atoms) == 1 else ["AND", atoms], ensure_ascii=False)
def _paperless_params(query: str, options: dict | None, custom_fields: list[dict] | None = None) -> dict:
"""Paperless-ngx document date is the created field, not the upload or modified time.""" """Paperless-ngx document date is the created field, not the upload or modified time."""
options = options or {} options = options or {}
params = {"page_size": "20"} params = {"page_size": str(PAPERLESS_PAGE_SIZE)}
text = " ".join(str(query or "").split()) text = " ".join(str(query or "").split())
mode = str(options.get("mode") or "").strip() or "title_content"
if mode not in _PAPERLESS_MODES:
raise unprocessable("Diese Suchart gibt es nicht.")
if text: if text:
params["query"] = text params[_PAPERLESS_MODES[mode]] = text
title = " ".join(str(options.get("title") or "").split()) title = " ".join(str(options.get("title") or "").split())
if title: if title:
if "title__icontains" in params and params["title__icontains"] != title:
raise unprocessable("Die Suche nur im Titel und der Titelfilter sind beide gesetzt. Nutze eines davon.")
params["title__icontains"] = title params["title__icontains"] = title
correspondent = _paperless_choice(options.get("correspondent"), "Der Korrespondent") for name, field, label in (
if correspondent: ("correspondent", "correspondent", "Der Korrespondent"),
params["correspondent__id"] = correspondent ("document_type", "document_type", "Der Dokumenttyp"),
document_type = _paperless_choice(options.get("document_type"), "Der Dokumenttyp") ("storage_path", "storage_path", "Der Speicherpfad"),
if document_type: ):
params["document_type__id"] = document_type chosen = _paperless_choice(options.get(name), label, allow_none=True)
if chosen == "none":
params[f"{field}__isnull"] = "true"
elif chosen:
params[f"{field}__id"] = chosen
raw_tags = options.get("tag") or options.get("tags") or "" raw_tags = options.get("tag") or options.get("tags") or ""
if isinstance(raw_tags, (list, tuple)): if isinstance(raw_tags, (list, tuple)):
parts = [str(item) for item in raw_tags] parts = [str(item) for item in raw_tags]
else: else:
parts = str(raw_tags).split(",") parts = str(raw_tags).split(",")
tag_ids = [chosen for chosen in (_paperless_choice(part, "Ein Tag") for part in parts) if chosen] tag_ids = [chosen for chosen in (_paperless_choice(part, "Ein Tag") for part in parts) if chosen]
tag_mode = str(options.get("tag_mode") or "").strip() or "all"
if tag_mode not in _PAPERLESS_TAG_MODES:
raise unprocessable("Diese Tag-Verknüpfung gibt es nicht.")
if tag_ids: if tag_ids:
params["tags__id__all"] = ",".join(tag_ids) params[_PAPERLESS_TAG_MODES[tag_mode]] = ",".join(tag_ids)
start = _paperless_day(options.get("document_from"), "Das Dokumentdatum von") _paperless_range(options, "document", "created", "Dokumentdatum", params)
end = _paperless_day(options.get("document_to"), "Das Dokumentdatum bis") _paperless_range(options, "added", "added", "Hinzugefügt am", params)
if start and end and start > end: custom = _paperless_custom_query(options.get("custom"), custom_fields or [])
raise unprocessable("Das Dokumentdatum von liegt nach dem Dokumentdatum bis.") if custom:
if start: params["custom_field_query"] = custom
params["created__date__gte"] = start asn = str(options.get("asn") or "").strip()
if end: if asn:
params["created__date__lte"] = end if not asn.isdigit() or len(asn) > 10:
raise unprocessable("Die Archivnummer ist keine Zahl.")
params["archive_serial_number"] = asn
sort = str(options.get("sort") or "").strip() sort = str(options.get("sort") or "").strip()
if sort: if sort:
if sort not in {"created", "-created", "title", "-title", "score", "-score"}: if sort not in _PAPERLESS_SORTS:
raise unprocessable("Diese Sortierung gibt es nicht.") raise unprocessable("Diese Sortierung gibt es nicht.")
if sort in {"score", "-score"}: if sort in {"score", "-score"}:
if not text: if not text or mode != "index":
raise unprocessable("Die Relevanz gilt zusammen mit einem Suchtext.") raise unprocessable("Die Relevanz gilt nur für die Suche im Volltextindex.")
# Paperless-ngx 2.20.6 has no ordering field named score. A query without # Paperless-ngx 2.20.6 has no ordering field named score. A query without
# ordering is the relevance order of the full-text index. # ordering is the relevance order of the full-text index.
else: else:
@@ -275,45 +414,254 @@ def _paperless_row(conn, owner_id: UUID, connector_id: UUID | None):
def paperless_documents(conn, owner_id: UUID, query: str, connector_id: UUID | None = None, options: dict | None = None) -> dict: def paperless_documents(conn, owner_id: UUID, query: str, connector_id: UUID | None = None, options: dict | None = None) -> dict:
row = _paperless_row(conn, owner_id, connector_id) row = _paperless_row(conn, owner_id, connector_id)
base = row["settings"]["base_url"].rstrip("/") base = row["settings"]["base_url"].rstrip("/")
params = _paperless_params(query, options)
release_for_external(conn) release_for_external(conn)
payload = _paperless_get(base, row["secret"], "/api/documents/", params) names = _paperless_lookup(row["id"], base, row["secret"])
params = _paperless_params(query, options, names["custom_fields"])
# Observed on a real instance: all-tags together with a custom field makes Paperless answer 500.
try:
payload = _paperless_get(base, row["secret"], "/api/documents/", params)
except Problem as problem:
clash = "," in params.get("tags__id__all", "") and "custom_field_query" in params
if clash and "HTTP 5" in problem.detail:
raise unprocessable(f"{problem.detail} Das tritt auf, wenn „Alle gewählten Tags“ mit einem Zusatzfeld zusammentrifft. Wähle „Mindestens eines“ oder lass das Zusatzfeld weg.") from problem
raise
assert_still_owned(conn, row["id"], owner_id) assert_still_owned(conn, row["id"], owner_id)
terms = _search_terms(query, (options or {}).get("title"))
items = [] items = []
for doc in payload.get("results") or []: for doc in payload.get("results") or []:
created = str(doc.get("created") or "")[:10] created = str(doc.get("created") or "")[:10]
added = str(doc.get("added") or "")[:10]
title = (doc.get("title") or doc.get("original_file_name") or f"Dokument {doc.get('id')}").strip() title = (doc.get("title") or doc.get("original_file_name") or f"Dokument {doc.get('id')}").strip()
pages = doc.get("page_count")
asn = doc.get("archive_serial_number")
items.append( items.append(
{ {
"id": str(doc.get("id")), "id": str(doc.get("id")),
"title": title[:200], "title": title[:200],
"document_on": created if len(created) == 10 else None, "document_on": created if len(created) == 10 else None,
"correspondent": _correspondent(doc), "added_on": added if len(added) == 10 else None,
"correspondent": _correspondent(doc, names),
"document_type": _named(doc.get("document_type"), names["document_types"]),
"storage_path": _named(doc.get("storage_path"), names["storage_paths"]),
"tags": [name for name in (_named(tag, names["tags"]) for tag in doc.get("tags") or []) if name][:12],
"custom_fields": _custom_display(doc.get("custom_fields"), names["custom_fields"]),
"pages": pages if isinstance(pages, int) and pages > 0 else None,
"asn": asn if isinstance(asn, int) else None,
"snippet": _snippet(doc, terms),
"open_url": f"{base}/documents/{doc.get('id')}/", "open_url": f"{base}/documents/{doc.get('id')}/",
} }
) )
page = int(params.get("page") or 1) page = int(params.get("page") or 1)
count = payload.get("count") count = payload.get("count")
if isinstance(count, int): if isinstance(count, int):
next_page = str(page + 1) if page * 20 < count else None next_page = str(page + 1) if page * PAPERLESS_PAGE_SIZE < count else None
else: else:
next_page = str(page + 1) if len(items) == 20 else None next_page = str(page + 1) if len(items) == PAPERLESS_PAGE_SIZE else None
return {"items": items, "connector_id": str(row["id"]), "next_page": next_page, "incomplete": False} return {
"items": items,
"connector_id": str(row["id"]),
"next_page": next_page,
"previous_page": str(page - 1) if page > 1 else None,
"total": count if isinstance(count, int) else None,
"incomplete": False,
}
def paperless_facets(conn, owner_id: UUID, connector_id: UUID) -> dict: def paperless_facets(conn, owner_id: UUID, connector_id: UUID) -> dict:
row = _paperless_row(conn, owner_id, connector_id) row = _paperless_row(conn, owner_id, connector_id)
base = row["settings"]["base_url"].rstrip("/") base = row["settings"]["base_url"].rstrip("/")
token = row["secret"]
release_for_external(conn) release_for_external(conn)
body = { names = _paperless_lookup(row["id"], base, row["secret"])
assert_still_owned(conn, row["id"], owner_id)
return {
"correspondents": names["correspondents"],
"document_types": names["document_types"],
"storage_paths": names["storage_paths"],
"tags": names["tags"],
"custom_fields": names["custom_fields"],
"tag_mode": "all",
"tag_modes": list(_PAPERLESS_TAG_MODES),
"modes": list(_PAPERLESS_MODES),
}
_THUMBNAIL_MEDIA = frozenset({"image/png", "image/jpeg", "image/webp", "image/gif"})
PAPERLESS_SEARCH_PARAMS = {
"mode": "mode",
"title": "title",
"correspondent": "correspondent",
"document_type": "document_type",
"storage_path": "storage_path",
"tag": "tags",
"tag_mode": "tag_mode",
"document_from": "document_range",
"document_to": "document_range",
"added_from": "added_range",
"added_to": "added_range",
"asn": "asn",
"custom": "custom_fields",
"sort": "sort",
"cursor": "continuation",
}
def paperless_thumbnail(conn, owner_id: UUID, document_id: str, connector_id: UUID) -> tuple[bytes, str]:
if not document_id.isdigit() or len(document_id) > 12:
raise unprocessable("Die Paperless-Nummer ist ungültig.")
row = _paperless_row(conn, owner_id, connector_id)
base = row["settings"]["base_url"].rstrip("/")
release_for_external(conn)
fetched = _paperless_file(base, row["secret"], f"/api/documents/{document_id}/thumb/", 2 * 1024 * 1024)
assert_still_owned(conn, row["id"], owner_id)
if fetched is None:
raise not_found()
data, header, _name = fetched
return data, _thumbnail_media(data, header)
def _thumbnail_media(data: bytes, header: str) -> str:
"""Raster pictures only, told by their bytes. A vector file could carry markup, a header can be wrong."""
try:
media = sniff(data)
except Problem:
media = "image/gif" if data[:4] == b"GIF8" else ""
if media not in _THUMBNAIL_MEDIA:
raise unprocessable(f"Paperless hat kein Vorschaubild geliefert (Antworttyp {header or 'unbekannt'}).")
return media
_LOOKUP_TTL = 60.0
_LOOKUP_LIMIT = 32
_lookup_cache: dict[tuple, tuple[float, dict]] = {}
_lookup_lock = threading.Lock()
def _paperless_lookup(connector_id, base: str, token: str) -> dict:
"""Names behind the ids of a result list. Bound to the instance and key that asked, kept for a minute."""
key = (str(connector_id), base, hashlib.sha256(token.encode("utf-8")).hexdigest())
now = time.monotonic()
with _lookup_lock:
hit = _lookup_cache.get(key)
if hit and now - hit[0] < _LOOKUP_TTL:
return hit[1]
names = {
"correspondents": _paperless_names(base, token, "/api/correspondents/"), "correspondents": _paperless_names(base, token, "/api/correspondents/"),
"document_types": _paperless_names(base, token, "/api/document_types/"), "document_types": _paperless_names(base, token, "/api/document_types/"),
"tags": _paperless_names(base, token, "/api/tags/"), "tags": _paperless_names(base, token, "/api/tags/"),
"tag_mode": "all", # An instance without storage paths, or a key that may not read them, still searches.
"storage_paths": _paperless_names_or_none(base, token, "/api/storage_paths/"),
"custom_fields": _paperless_custom_fields(base, token),
} }
assert_still_owned(conn, row["id"], owner_id) with _lookup_lock:
return body if len(_lookup_cache) >= _LOOKUP_LIMIT:
_lookup_cache.clear()
_lookup_cache[key] = (now, names)
return names
def _paperless_custom_fields(base: str, token: str) -> list[dict]:
"""Custom fields the picker can filter on. Types it cannot query, such as document links, are left out."""
found = []
try:
for page in range(1, 6):
payload = _paperless_get(base, token, "/api/custom_fields/", {"page": str(page), "page_size": "100"})
rows = payload.get("results") or []
for item in rows:
kind = item.get("data_type")
if item.get("id") is None or kind not in _CUSTOM_KINDS:
continue
extra = item.get("extra_data") if isinstance(item.get("extra_data"), dict) else {}
options = [
{"id": str(option["id"]), "name": str(option.get("label") or option["id"])[:200]}
for option in extra.get("select_options") or []
if isinstance(option, dict) and option.get("id") is not None
]
found.append({"id": str(item["id"]), "name": str(item.get("name") or item["id"])[:200], "type": kind, "options": options})
if len(rows) < 100:
break
except Problem:
return []
return found
def _paperless_names_or_none(base: str, token: str, path: str) -> list[dict]:
try:
return _paperless_names(base, token, path)
except Problem:
return []
_CUSTOM_SHOWN = 8
def _custom_display(values, fields: list[dict]) -> list[dict]:
"""The custom fields of a hit as name, type and readable value. Empty values and unknown fields are left out."""
if not isinstance(values, list):
return []
known = {field["id"]: field for field in fields}
shown = []
for entry in values:
if not isinstance(entry, dict):
continue
field = known.get(str(entry.get("field")))
value = entry.get("value")
if field is None or value is None or value == "" or isinstance(value, list):
continue
if field["type"] == "boolean":
text = "Ja" if value else "Nein"
elif field["type"] == "select":
text = next((option["name"] for option in field["options"] if option["id"] == str(value)), "")
else:
text = str(value)
if text:
shown.append({"name": field["name"], "type": field["type"], "value": text[:200]})
return shown[:_CUSTOM_SHOWN]
def _named(value, entries: list[dict]) -> str | None:
if value is None or isinstance(value, bool):
return None
wanted = str(value)
for entry in entries:
if entry["id"] == wanted:
return entry["name"]
return None
_TAG_MARKUP = re.compile(r"<[^>]{0,200}>")
_SNIPPET_LENGTH = 260
def _search_terms(query: str, title) -> list[str]:
words = re.findall(r"[\w-]{3,}", f"{query or ''} {title or ''}".casefold())
return list(dict.fromkeys(words))[:8]
def _snippet(doc: dict, terms: list[str]) -> str | None:
"""A short plain-text excerpt: the index hit when Paperless sends one, else the text around the first match."""
hit = doc.get("__search_hit__")
if isinstance(hit, dict):
lines = hit.get("highlights")
if isinstance(lines, str) and lines.strip():
text = " ".join(unescape(_TAG_MARKUP.sub("", lines)).split())
if text:
return text[:_SNIPPET_LENGTH]
content = doc.get("content")
if not isinstance(content, str):
return None
text = " ".join(content.split())
if not text:
return None
folded = text.casefold()
at = min((index for index in (folded.find(term) for term in terms) if index >= 0), default=-1)
if at < 0:
return text[:_SNIPPET_LENGTH] + ("…" if len(text) > _SNIPPET_LENGTH else "")
first = max(0, at - 60)
if first:
first = text.rfind(" ", 0, first) + 1 or first
last = first + _SNIPPET_LENGTH
return ("…" if first else "") + text[first:last] + ("…" if last < len(text) else "")
def _paperless_names(base: str, token: str, path: str) -> list[dict]: def _paperless_names(base: str, token: str, path: str) -> list[dict]:
@@ -509,7 +857,8 @@ def read_linked_mail(conn, actor_id: UUID, element: dict, context: Ctx | None, r
def fetch(row): def fetch(row):
return pull_rfc822(row["settings"], row["secret"], mailbox, uid, external_id, COMPLETE_MAIL_LIMIT) return pull_rfc822(row["settings"], row["secret"], mailbox, uid, external_id, COMPLETE_MAIL_LIMIT)
return complete_borrow(conn, borrowed, fetch) found = complete_borrow(conn, borrowed, fetch)
return _keep_moved_mail(conn, element["id"], external_id, mailbox, uid, found)
def _uuid(value) -> UUID | None: def _uuid(value) -> UUID | None:
@@ -672,15 +1021,28 @@ def _paperless_get(base: str, token: str, path: str, params: dict) -> dict:
except HTTPError as exc: except HTTPError as exc:
if exc.code in {401, 403}: if exc.code in {401, 403}:
raise unprocessable("Paperless hat den Schlüssel abgelehnt.") from exc raise unprocessable("Paperless hat den Schlüssel abgelehnt.") from exc
raise unprocessable("Paperless hat die Anfrage nicht beantwortet.") from exc raise unprocessable(f"Paperless hat die Anfrage nicht beantwortet ({_refusal(exc, path)}).") from exc
except (URLError, TimeoutError, json.JSONDecodeError) as exc: except (URLError, TimeoutError, json.JSONDecodeError) as exc:
raise unprocessable("Paperless ist nicht erreichbar.") from exc raise unprocessable("Paperless ist nicht erreichbar.") from exc
def _correspondent(doc: dict) -> str | None: def _refusal(exc: HTTPError, path: str) -> str:
"""Status and, when Paperless names one, its reason. The body is the owner's own instance, cut short."""
reason = ""
try:
body = json.loads(exc.read(2000).decode("utf-8", "replace"))
reason = json.dumps(body, ensure_ascii=False)[:240] if isinstance(body, (dict, list)) else ""
except (ValueError, OSError):
reason = ""
return f"HTTP {exc.code} auf {path}" + (f": {reason}" if reason else "")
def _correspondent(doc: dict, names: dict | None = None) -> str | None:
value = doc.get("correspondent_name") or doc.get("correspondent") value = doc.get("correspondent_name") or doc.get("correspondent")
if isinstance(value, str) and value.strip(): if isinstance(value, str) and value.strip():
return value.strip()[:200] return value.strip()[:200]
if names is not None:
return _named(value, names["correspondents"])
return None return None
@@ -1470,11 +1832,33 @@ def load_imap_original(conn, owner_id: UUID, connector_id: UUID, mailbox: str, u
row = _owned(conn, owner_id, connector_id) row = _owned(conn, owner_id, connector_id)
if row["kind"] != "imap": if row["kind"] != "imap":
raise unprocessable("Das ist kein E-Mail-Konto.") raise unprocessable("Das ist kein E-Mail-Konto.")
return pull_rfc822(row["settings"], row["secret"], mailbox, uid, expected_id, COMPLETE_MAIL_LIMIT) found = pull_rfc822(row["settings"], row["secret"], mailbox, uid, expected_id, COMPLETE_MAIL_LIMIT)
return found.raw if isinstance(found, LocatedMail) else found
def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id: str, max_bytes: int) -> bytes: @dataclass(frozen=True)
"""Fetch the referenced message. A different message at the old UID is not returned.""" class LocatedMail:
"""Bytes plus the folder they were read from. `relocated` is a complete, unique hit."""
raw: bytes
mailbox: str
uid: int
relocated: bool = False
@dataclass(frozen=True)
class _MailHit:
raw: bytes
mailbox: str
uid: int
def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id: str, max_bytes: int) -> LocatedMail:
"""Fetch the referenced message. A different message at the old UID is not returned.
The stored folder is tried first. A single Message-ID hit elsewhere comes back
marked relocated; the caller stores that place after the access check.
"""
client = _imap_login(settings, secret) client = _imap_login(settings, secret)
try: try:
_select_mailbox(client, mailbox) _select_mailbox(client, mailbox)
@@ -1484,23 +1868,45 @@ def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id
"Die vollständige Nachricht überschreitet die Exportgrenze. " "Die vollständige Nachricht überschreitet die Exportgrenze. "
"Eine gekürzte Vorschau wird nicht als Original übernommen." "Eine gekürzte Vorschau wird nicht als Original übernommen."
) )
alternates: list[bytes] = [] if candidate is not None and identities_match(header_message_id(candidate), expected_id):
if candidate is None or not identities_match(header_message_id(candidate), expected_id): return LocatedMail(candidate, mailbox, uid, relocated=False)
alternates = _search_rfc822(client, expected_id, max_bytes) hits, capped = _search_rfc822(client, expected_id, max_bytes)
try: try:
chosen = choose_message(candidate, alternates, expected_id) chosen = choose_message(None, [hit.raw for hit in hits], expected_id)
except MailIdentityError as exc: except MailIdentityError as exc:
raise unprocessable(exc.message) from exc raise unprocessable(exc.message) from exc
if len(chosen) > max_bytes: match = next(
hit for hit in hits
if hit.raw == chosen and identities_match(header_message_id(hit.raw), expected_id)
)
if len(match.raw) > max_bytes:
raise unprocessable( raise unprocessable(
"Die vollständige Nachricht überschreitet die Exportgrenze. " "Die vollständige Nachricht überschreitet die Exportgrenze. "
"Eine gekürzte Vorschau wird nicht als Original übernommen." "Eine gekürzte Vorschau wird nicht als Original übernommen."
) )
return chosen moved = not capped and (match.mailbox != mailbox or match.uid != uid)
return LocatedMail(match.raw, match.mailbox, match.uid, relocated=moved)
finally: finally:
_imap_logout(client) _imap_logout(client)
def _keep_moved_mail(conn, element_id, external_id: str, mailbox: str, uid: int | None, found) -> bytes:
"""Return the bytes. A unique new place is written only after the fetch has finished."""
if not isinstance(found, LocatedMail):
return found
if found.relocated and element_id is not None and uid is not None:
remember_imap_location(
conn,
element_id,
external_id,
mailbox,
uid,
found.mailbox,
found.uid,
)
return found.raw
def _fetch_rfc822(client: imaplib.IMAP4, uid: int) -> bytes | None: def _fetch_rfc822(client: imaplib.IMAP4, uid: int) -> bytes | None:
try: try:
status, fetched = client.uid("FETCH", str(uid), "(BODY.PEEK[])") status, fetched = client.uid("FETCH", str(uid), "(BODY.PEEK[])")
@@ -1511,11 +1917,12 @@ def _fetch_rfc822(client: imaplib.IMAP4, uid: int) -> bytes | None:
return _literal(fetched) return _literal(fetched)
def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> list[bytes]: def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> tuple[list[_MailHit], bool]:
"""Search every selectable folder. The second value is true when the scan stopped early."""
token = (expected_id or "").strip() token = (expected_id or "").strip()
if not token or token.casefold().startswith("uid:"): if not token or token.casefold().startswith("uid:"):
return [] return [], False
found: list[bytes] = [] found: list[_MailHit] = []
for folder in _imap_mailboxes(client): for folder in _imap_mailboxes(client):
if not folder["selectable"]: if not folder["selectable"]:
continue continue
@@ -1539,10 +1946,10 @@ def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> l
"Die vollständige Nachricht überschreitet die Exportgrenze. " "Die vollständige Nachricht überschreitet die Exportgrenze. "
"Eine gekürzte Vorschau wird nicht als Original übernommen." "Eine gekürzte Vorschau wird nicht als Original übernommen."
) )
found.append(blob) found.append(_MailHit(blob, folder["name"], uid))
if len(found) > 5: if len(found) > 5:
return found return found, True
return found return found, False
PAPERLESS_FIELDS = ( PAPERLESS_FIELDS = (
@@ -1667,13 +2074,15 @@ def _export_imap(conn, actor_id: UUID, external_id: str, detail: dict, assigned:
owner_id = resolved.row["owner_id"] owner_id = resolved.row["owner_id"]
release_for_external(conn) release_for_external(conn)
try: try:
raw = pull_rfc822(settings, secret, mailbox, uid, str(external_id), EXPORT_BYTE_LIMIT) found = pull_rfc822(settings, secret, mailbox, uid, str(external_id), EXPORT_BYTE_LIMIT)
except Problem as exc: except Problem as exc:
return failure_from_problem(exc) return failure_from_problem(exc)
try: try:
assert_still_owned(conn, connector_id, owner_id) assert_still_owned(conn, connector_id, owner_id)
except Problem as exc: except Problem as exc:
return failure_from_problem(exc) return failure_from_problem(exc)
element_id = ref["element_id"] if ref is not None and "element_id" in ref.keys() else None
raw = _keep_moved_mail(conn, element_id, str(external_id), mailbox, uid, found)
if len(raw) > EXPORT_BYTE_LIMIT: if len(raw) > EXPORT_BYTE_LIMIT:
return RetrievalFailure( return RetrievalFailure(
"too_large", "too_large",
@@ -1694,11 +2103,66 @@ def _imap_bytes(conn, actor_id: UUID, element: dict, context: Ctx | None, ref) -
return raw, "message/rfc822", "nachricht.eml" return raw, "message/rfc822", "nachricht.eml"
PAPERLESS_CAPABILITIES = {
"kind": "paperless",
"navigate": False,
"read_original": True,
"read_preview": True,
"read_attachment": False,
"edit_content": False,
"edit_metadata": False,
"search": {
"query": True,
"mode": True,
"title": True,
"correspondent": True,
"document_type": True,
"storage_path": True,
"tags": True,
"tag_mode": True,
"document_range": True,
"added_range": True,
"asn": True,
"custom_fields": True,
"sort": True,
"continuation": True,
"result_preview": True,
},
}
IMAP_CAPABILITIES = {
"kind": "imap",
"navigate": True,
"read_original": True,
"read_preview": True,
"read_attachment": True,
"edit_content": False,
"edit_metadata": False,
"search": {
"query": True,
"mode": False,
"title": False,
"correspondent": False,
"document_type": False,
"storage_path": False,
"tags": False,
"tag_mode": False,
"document_range": False,
"added_range": False,
"asn": False,
"custom_fields": False,
"sort": False,
"continuation": True,
"result_preview": False,
},
}
class _PaperlessAdapter: class _PaperlessAdapter:
kind = "paperless" kind = "paperless"
def capabilities(self) -> dict: def capabilities(self) -> dict:
return CAPABILITIES[self.kind] return PAPERLESS_CAPABILITIES
def probe(self, row) -> dict: def probe(self, row) -> dict:
count = _paperless_count(row["settings"]["base_url"], row["secret"]) count = _paperless_count(row["settings"]["base_url"], row["secret"])
@@ -1719,7 +2183,7 @@ class _ImapAdapter:
kind = "imap" kind = "imap"
def capabilities(self) -> dict: def capabilities(self) -> dict:
return CAPABILITIES[self.kind] return IMAP_CAPABILITIES
def probe(self, row) -> dict: def probe(self, row) -> dict:
_imap_login(row["settings"], row["secret"]) _imap_login(row["settings"], row["secret"])
@@ -1742,7 +2206,7 @@ def _install_sources() -> None:
kind="paperless", kind="paperless",
label="Paperless", label="Paperless",
representation="pdf", representation="pdf",
capabilities=CAPABILITIES["paperless"], capabilities=PAPERLESS_CAPABILITIES,
config_fields=PAPERLESS_FIELDS, config_fields=PAPERLESS_FIELDS,
normalize_config=_paperless_normalize, normalize_config=_paperless_normalize,
probe=paperless.probe, probe=paperless.probe,
@@ -1752,12 +2216,14 @@ def _install_sources() -> None:
navigate=None, navigate=None,
preview=None, preview=None,
facets=paperless.facets, facets=paperless.facets,
search_params=PAPERLESS_SEARCH_PARAMS,
thumbnail=lambda conn, owner, connector_id, item: paperless_thumbnail(conn, owner, str(item), connector_id),
)) ))
register_source(SourceModule( register_source(SourceModule(
kind="imap", kind="imap",
label="E-Mail", label="E-Mail",
representation="email", representation="email",
capabilities=CAPABILITIES["imap"], capabilities=IMAP_CAPABILITIES,
config_fields=IMAP_FIELDS, config_fields=IMAP_FIELDS,
normalize_config=_imap_normalize, normalize_config=_imap_normalize,
probe=imap.probe, probe=imap.probe,
@@ -0,0 +1,18 @@
-- Einstellungen je Konto. Ein Mitarbeiter erbt den Wert seines Hauptnutzers und kann ihn ändern,
-- solange der Hauptnutzer die Einstellung nicht für das Team verbindlich gesetzt hat (team_mode).
-- Erlaubte Werte, Standard und Verbindlichkeit je Schlüssel stehen im Code (app/preferences.py).
-- value NULL: kein eigener Wert, es gilt die Erbung bzw. der Standard.
CREATE TABLE account_preferences (
account_id uuid NOT NULL REFERENCES auth_accounts (id) ON DELETE CASCADE,
key text NOT NULL,
value jsonb,
team_mode text NOT NULL DEFAULT 'free',
updated_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (account_id, key),
CONSTRAINT account_preferences_key_len CHECK (char_length(key) BETWEEN 1 AND 60),
CONSTRAINT account_preferences_team_mode CHECK (team_mode IN ('free', 'locked'))
);
-- Dieselbe Quelle und Kennung: Suche nach bereits eingefügten Dokumenten (app/duplicates.py).
CREATE INDEX external_references_identity ON external_references (source, external_id);
+1 -1
View File
@@ -216,7 +216,7 @@ def test_two_paperless_instances_stay_bound_when_order_changes(client, db, monke
) )
assert filtered["items"][0]["title"] == "Treffer" assert filtered["items"][0]["title"] == "Treffer"
assert calls[-1][3]["correspondent__id"] == "4" assert calls[-1][3]["correspondent__id"] == "4"
assert calls[-1][3]["query"] == "Treffer" assert calls[-1][3]["title_content"] == "Treffer"
def test_ambiguous_legacy_stays_until_assignment(client, db, monkeypatch): def test_ambiguous_legacy_stays_until_assignment(client, db, monkeypatch):
+325 -2
View File
@@ -10,6 +10,9 @@ import io
import zipfile import zipfile
from email.parser import BytesParser from email.parser import BytesParser
import pytest
from app.core.errors import unprocessable
from app.media.port import MemoryAdapter from app.media.port import MemoryAdapter
from app.sourceaccess import borrow_original, complete_borrow from app.sourceaccess import borrow_original, complete_borrow
from app.sourceregistry import SourceModule, unregister_source, register_source from app.sourceregistry import SourceModule, unregister_source, register_source
@@ -355,22 +358,34 @@ def test_paperless_filters_reach_the_api_and_cross_pages(client, db, monkeypatch
params={ params={
"connector_id": first["id"], "connector_id": first["id"],
"q": "nurinhalt", "q": "nurinhalt",
"mode": "index",
"title": "Vertrag", "title": "Vertrag",
"correspondent": "4", "correspondent": "4",
"document_type": "8", "document_type": "8",
"storage_path": "6",
"tag": "2,5", "tag": "2,5",
"tag_mode": "any",
"document_from": "2024-01-01", "document_from": "2024-01-01",
"document_to": "2024-12-31", "document_to": "2024-12-31",
"added_from": "2024-02-01",
"added_to": "2024-02-28",
"asn": "1042",
"sort": "-created", "sort": "-created",
"cursor": "3", "cursor": "3",
}, },
) )
sent = calls[-1]["params"] sent = calls[-1]["params"]
assert sent["query"] == "nurinhalt" assert sent["query"] == "nurinhalt"
assert "title_content" not in sent
assert sent["title__icontains"] == "Vertrag" assert sent["title__icontains"] == "Vertrag"
assert sent["correspondent__id"] == "4" assert sent["correspondent__id"] == "4"
assert sent["document_type__id"] == "8" assert sent["document_type__id"] == "8"
assert sent["tags__id__all"] == "2,5" assert sent["storage_path__id"] == "6"
assert sent["tags__id__in"] == "2,5"
assert "tags__id__all" not in sent
assert sent["added__date__gte"] == "2024-02-01"
assert sent["added__date__lte"] == "2024-02-28"
assert sent["archive_serial_number"] == "1042"
assert sent["created__date__gte"] == "2024-01-01" assert sent["created__date__gte"] == "2024-01-01"
assert sent["created__date__lte"] == "2024-12-31" assert sent["created__date__lte"] == "2024-12-31"
assert sent["ordering"] == "-created" assert sent["ordering"] == "-created"
@@ -378,9 +393,15 @@ def test_paperless_filters_reach_the_api_and_cross_pages(client, db, monkeypatch
ada.request( ada.request(
"GET", "GET",
"/api/v1/sources/paperless/documents", "/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "q": "nurinhalt", "sort": "-score"}, params={"connector_id": first["id"], "q": "nurinhalt", "mode": "index", "sort": "-score"},
) )
assert "ordering" not in calls[-1]["params"] assert "ordering" not in calls[-1]["params"]
ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "q": "nurinhalt", "sort": "-score"},
status=422,
)
assert found["items"][0]["id"] == "41" assert found["items"][0]["id"] == "41"
assert found["connector_id"] == first["id"] assert found["connector_id"] == first["id"]
preview = ada.client.get( preview = ada.client.get(
@@ -431,6 +452,308 @@ def test_paperless_filters_reach_the_api_and_cross_pages(client, db, monkeypatch
) )
def test_paperless_default_search_reads_title_and_content_and_shows_a_preview(client, db, monkeypatch):
import app.sources as sources
calls = []
body = "Vorspann " * 30 + "Die Kündigung des Mietvertrags erfolgt zum Jahresende. " + "Nachspann " * 30
def fake_get(base, token, path, params):
calls.append({"path": path, "params": dict(params)})
if path == "/api/documents/":
return {
"count": 1,
"results": [
{
"id": 7,
"title": "Schreiben",
"created": "2024-03-01T00:00:00+01:00",
"added": "2024-03-05T08:00:00Z",
"correspondent": 4,
"document_type": 8,
"storage_path": None,
"tags": [2, 5, 99],
"archive_serial_number": 1042,
"page_count": 3,
"custom_fields": [
{"field": 1, "value": "EUR100.50"},
{"field": 2, "value": "2024-03-09"},
{"field": 3, "value": "b"},
{"field": 4, "value": True},
{"field": 5, "value": None},
{"field": 99, "value": "fremd"},
],
"content": body,
}
],
}
if path == "/api/storage_paths/":
raise sources.unprocessable("Paperless hat die Anfrage nicht beantwortet.")
if path == "/api/tags/":
return {"results": [{"id": 2, "name": "Miete"}, {"id": 5, "name": "Wohnung"}]}
if path == "/api/correspondents/":
return {"results": [{"id": 4, "name": "Hausverwaltung"}]}
if path == "/api/custom_fields/":
return {
"results": [
{"id": 1, "name": "Rechnungsbetrag", "data_type": "monetary"},
{"id": 2, "name": "bezahlt_am", "data_type": "date"},
{"id": 3, "name": "Status", "data_type": "select", "extra_data": {"select_options": [{"id": "b", "label": "bezahlt"}]}},
{"id": 4, "name": "Geprüft", "data_type": "boolean"},
{"id": 5, "name": "Notiz", "data_type": "string"},
]
}
return {"results": [{"id": 8, "name": "Brief"}]}
monkeypatch.setattr(sources, "_paperless_get", fake_get)
ada = Actor(client, "ada-preview@example.test", "Ada Preview")
register(db, ada.email, ada.name)
ada.login()
first = ada.request(
"POST",
"/api/v1/connectors",
json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"},
status=201,
)
found = ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "q": "kündigung", "correspondent": "none", "tag": "2", "tag_mode": "none"},
)
sent = [call for call in calls if call["path"] == "/api/documents/"][-1]["params"]
assert sent["title_content"] == "kündigung"
assert "query" not in sent
assert sent["correspondent__isnull"] == "true"
assert sent["tags__id__none"] == "2"
item = found["items"][0]
assert item["correspondent"] == "Hausverwaltung"
assert item["document_type"] == "Brief"
assert item["tags"] == ["Miete", "Wohnung"]
assert item["pages"] == 3
assert item["asn"] == 1042
assert item["document_on"] == "2024-03-01"
assert item["added_on"] == "2024-03-05"
assert "Kündigung des Mietvertrags" in item["snippet"]
assert len(item["snippet"]) <= 262
assert item["snippet"].startswith("…")
assert item["custom_fields"] == [
{"name": "Rechnungsbetrag", "type": "monetary", "value": "EUR100.50"},
{"name": "bezahlt_am", "type": "date", "value": "2024-03-09"},
{"name": "Status", "type": "select", "value": "bezahlt"},
{"name": "Geprüft", "type": "boolean", "value": "Ja"},
]
assert found["total"] == 1
assert found["previous_page"] is None
facets = ada.request("GET", "/api/v1/sources/paperless/facets", params={"connector_id": first["id"]})
assert facets["storage_paths"] == []
assert facets["tag_modes"] == ["all", "any", "none"]
# A filter of another source is refused, not ignored.
ada.request(
"GET",
"/api/v1/sources/imap/messages",
params={"connector_id": first["id"], "mailbox": "INBOX", "mode": "title"},
status=422,
)
for params in (
{"mode": "regex"},
{"tag_mode": "x"},
{"asn": "x1"},
{"storage_path": "a"},
{"q": "miete", "mode": "title", "title": "vertrag"},
{"bogus": "1"},
):
ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], **params},
status=422,
)
def test_paperless_custom_field_filters_use_the_instance_fields(client, db, monkeypatch):
import json
import app.sources as sources
calls = []
def fake_get(base, token, path, params):
calls.append({"path": path, "params": dict(params)})
if path == "/api/custom_fields/":
return {
"results": [
{"id": 1, "name": "Rechnungsbetrag", "data_type": "monetary"},
{"id": 2, "name": "bezahlt_am", "data_type": "date"},
{
"id": 3,
"name": "Status",
"data_type": "select",
"extra_data": {"select_options": [{"id": "a", "label": "offen"}, {"id": "b", "label": "bezahlt"}]},
},
{"id": 4, "name": "Verweis", "data_type": "documentlink"},
]
}
if path == "/api/documents/":
if failing:
raise unprocessable(failing[0])
return {"count": 0, "results": []}
return {"results": []}
failing = []
monkeypatch.setattr(sources, "_paperless_get", fake_get)
ada = Actor(client, "ada-custom@example.test", "Ada Custom")
register(db, ada.email, ada.name)
ada.login()
first = ada.request(
"POST",
"/api/v1/connectors",
json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"},
status=201,
)
facets = ada.request("GET", "/api/v1/sources/paperless/facets", params={"connector_id": first["id"]})
assert [item["name"] for item in facets["custom_fields"]] == ["Rechnungsbetrag", "bezahlt_am", "Status"]
assert facets["custom_fields"][2]["options"] == [{"id": "a", "name": "offen"}, {"id": "b", "name": "bezahlt"}]
def ask(rows, status=200):
return ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "custom": json.dumps(rows)},
status=status,
)
ask([{"field": "2", "op": "range", "value": ["2024-01-01", "2024-06-30"]}])
sent = [call for call in calls if call["path"] == "/api/documents/"][-1]["params"]
assert json.loads(sent["custom_field_query"]) == ["bezahlt_am", "range", ["2024-01-01", "2024-06-30"]]
ask(
[
{"field": "1", "op": "gte", "value": "100,5"},
{"field": "3", "op": "exact", "value": "a"},
{"field": "2", "op": "exists", "value": False},
]
)
sent = [call for call in calls if call["path"] == "/api/documents/"][-1]["params"]
assert json.loads(sent["custom_field_query"]) == [
"AND",
[["Rechnungsbetrag", "gte", 100.5], ["Status", "exact", "a"], ["bezahlt_am", "exists", False]],
]
ask([], 200)
assert "custom_field_query" not in [call for call in calls if call["path"] == "/api/documents/"][-1]["params"]
for rows in (
[{"field": "9", "op": "exact", "value": "x"}],
[{"field": "4", "op": "contains", "value": "1"}],
[{"field": "2", "op": "icontains", "value": "2024"}],
[{"field": "3", "op": "exact", "value": "z"}],
[{"field": "1", "op": "gt", "value": "viel"}],
[{"field": "2", "op": "range", "value": ["2024-06-30", "2024-01-01"]}],
[{"field": "2", "op": "exists", "value": "ja"}],
[{"field": "1", "op": "exact", "value": "1"}] * 6,
):
ask(rows, 422)
ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "custom": "kein json"},
status=422,
)
# Paperless answers 500 for all-tags together with a custom field: the message names that pairing.
failing.append("Paperless hat die Anfrage nicht beantwortet (HTTP 500 auf /api/documents/).")
row = json.dumps([{"field": "2", "op": "exists", "value": True}])
def refused(extra):
return ada.request(
"GET",
"/api/v1/sources/paperless/documents",
params={"connector_id": first["id"], "custom": row, **extra},
status=422,
)
assert "Alle gewählten Tags" in str(refused({"tag": "1,2"}))
assert "Alle gewählten Tags" not in str(refused({"tag": "1,2", "tag_mode": "any"}))
assert "Alle gewählten Tags" not in str(refused({"tag": "1"}))
def test_paperless_refusal_names_the_status_and_reason(monkeypatch):
import io
from urllib.error import HTTPError
import app.sources as sources
from app.core.errors import Problem
def refuse(code, body):
def fake(request, timeout=0):
raise HTTPError(request.full_url, code, "x", {}, io.BytesIO(body))
return fake
monkeypatch.setattr(sources, "urlopen", refuse(400, '{"custom_field_query": ["Ungültig"]}'.encode()))
with pytest.raises(Problem) as caught:
sources._paperless_get("https://paper.example.test", "token-alpha", "/api/documents/", {"page_size": "20"})
assert "HTTP 400 auf /api/documents/" in caught.value.detail
assert "custom_field_query" in caught.value.detail
assert "token-alpha" not in caught.value.detail
monkeypatch.setattr(sources, "urlopen", refuse(500, b"<html>Fehler</html>"))
with pytest.raises(Problem) as caught:
sources._paperless_get("https://paper.example.test", "token-alpha", "/api/documents/", {})
assert caught.value.detail.endswith("(HTTP 500 auf /api/documents/).")
monkeypatch.setattr(sources, "urlopen", refuse(403, b"{}"))
with pytest.raises(Problem) as caught:
sources._paperless_get("https://paper.example.test", "token-alpha", "/api/documents/", {})
assert "Schlüssel abgelehnt" in caught.value.detail
def test_paperless_thumbnail_is_bound_to_the_owners_instance(client, db, monkeypatch):
import app.sources as sources
seen = []
def fake_file(base, token, path, max_bytes=0):
seen.append((token, path))
if path.endswith("/9/thumb/"):
return b"RIFFxxxxWEBP", "image/webp", ""
if path.endswith("/8/thumb/"):
return b"<p>", "text/html", ""
if path.endswith("/6/thumb/"):
return b"\x89PNG\r\n\x1a\nxxxx", "application/octet-stream", ""
if path.endswith("/7/thumb/"):
return b"<svg xmlns='http://www.w3.org/2000/svg'/>", "image/svg+xml", ""
return None
monkeypatch.setattr(sources, "_paperless_file", fake_file)
ada = Actor(client, "ada-thumb@example.test", "Ada Thumb")
register(db, ada.email, ada.name)
ada.login()
first = ada.request(
"POST",
"/api/v1/connectors",
json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"},
status=201,
)
url = "/api/v1/sources/paperless/thumbnail"
def get(actor, item, kind_url=url):
return actor.client.get(kind_url, params={"connector_id": first["id"], "item": item}, headers=_headers(actor))
ok = get(ada, 9)
assert ok.status_code == 200
assert ok.headers["content-type"] == "image/webp"
assert ok.headers["cache-control"] == "private, no-store"
assert seen[-1] == ("token-alpha", "/api/documents/9/thumb/")
typed = get(ada, 6)
assert typed.status_code == 200 and typed.headers["content-type"] == "image/png"
assert get(ada, 8).status_code == 422
assert get(ada, 7).status_code == 422
assert get(ada, 5).status_code == 404
assert get(ada, "x1").status_code == 422
# IMAP declares no thumbnails, and the module must declare the capability.
assert get(ada, 9, "/api/v1/sources/imap/thumbnail").status_code == 422
ben = Actor(client, "ben-thumb@example.test", "Ben Thumb")
register(db, ben.email, ben.name)
ben.login()
assert get(ben, 9).status_code == 404
assert len(seen) == 5
def test_revoked_grant_hides_bytes_fetched_during_the_call(client, db, monkeypatch): def test_revoked_grant_hides_bytes_fetched_during_the_call(client, db, monkeypatch):
import app.sources as sources import app.sources as sources
+35 -1
View File
@@ -104,6 +104,40 @@ def test_parse_mail_keeps_names_html_and_attached_messages():
assert named.attachments[0].data == b"pdf" assert named.attachments[0].data == b"pdf"
def test_an_undecoded_pdf_stays_a_named_pdf():
import base64
from app.media.storage import content_disposition
pdf = b"%PDF-1.4\n1 0 obj\n<<>>\nendobj\ntrailer\n<<>>\n%%EOF\n"
raw = (
b"Subject: Scan\r\n"
b"MIME-Version: 1.0\r\n"
b"Content-Type: multipart/mixed; boundary=bound\r\n"
b"\r\n"
b"--bound\r\n"
b"Content-Type: text/plain; charset=utf-8\r\n"
b"\r\n"
b"text\r\n"
b"--bound\r\n"
b"Content-Type: application/octet-stream\r\n"
b"Content-Transfer-Encoding: 8bit\r\n"
b"Content-Disposition: attachment; filename=\"scan-7f3a9c\"\r\n"
b"\r\n"
+ base64.b64encode(pdf) + b"\r\n"
b"--bound--\r\n"
)
item = parse_mail(raw).attachments[0]
assert item.data.startswith(b"%PDF")
assert item.media_type == "application/pdf"
assert item.display_name == "scan-7f3a9c.pdf"
header = content_disposition("\u00dcbergabe.pdf", inline=True)
assert header.startswith("inline;")
header.encode("ascii")
assert ".pdf" in header
assert "filename*=UTF-8''" in header
assert "%C3%9Cbergabe.pdf" in header
class QuotedIMAP: class QuotedIMAP:
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
self.untagged_responses = {} self.untagged_responses = {}
@@ -395,7 +429,7 @@ def test_paperless_filters_change_the_hits(client, db, monkeypatch):
return {"count": 1, "results": [{"id": 4, "name": "Haus"}]} return {"count": 1, "results": [{"id": 4, "name": "Haus"}]}
matched = [] matched = []
for item in catalog: for item in catalog:
query = params.get("query") or "" query = params.get("title_content") or params.get("query") or ""
title = params.get("title__icontains") or "" title = params.get("title__icontains") or ""
if query and query.casefold() not in f"{item['title']} {item['content']}".casefold(): if query and query.casefold() not in f"{item['title']} {item['content']}".casefold():
continue continue
+126
View File
@@ -0,0 +1,126 @@
"""Bereits eingefügte Quelldokumente: Vorabfrage, Warnung (409), erneutes Einfügen und zusätzliches Einbinden."""
from __future__ import annotations
from uuid import uuid4
import pytest
from tests.support import matter_of
from tests.test_team import _main
PREF = "/api/v1/auth/preferences/duplicate_check"
EXISTING = "/api/v1/external-refs/existing"
CONNECTOR = str(uuid4())
@pytest.fixture
def ada(client, db):
return _main(client, db, "ada", main=True)
def _ref(actor, kind, container, external_id="41", connector=CONNECTOR, status=201, **extra):
body = {
"title": f"Dokument {external_id}",
"source": "paperless",
"external_id": external_id,
"context": {"kind": kind, "id": container},
"detail": {"connector_id": connector},
**extra,
}
return actor.request("POST", "/api/v1/external-refs", json=body, status=status)
def _ask(actor, kind, container, external_id="41", connector=CONNECTOR):
body = {
"context": {"kind": kind, "id": container},
"items": [{"source": "paperless", "external_id": external_id, "connector_id": connector}],
}
return actor.request("POST", EXISTING, json=body)["items"][0]["matches"]
def _level(actor, value):
actor.request("PUT", PREF, json={"value": value})
def test_a_document_in_the_same_container_warns_and_can_be_inserted_anyway(ada):
matter = matter_of(ada, "Mietminderung")["id"]
first = _ref(ada, "matter", matter)
found = _ask(ada, "matter", matter)
assert [(m["element_id"], m["where"], m["certain"]) for m in found] == [(first["id"], "here", True)]
refused = _ref(ada, "matter", matter, status=409)
assert refused["code"] == "already_present"
# A deliberate second insert goes through. Another document or another connection is no duplicate.
_ref(ada, "matter", matter, duplicate="insert")
_ref(ada, "matter", matter, external_id="42")
_ref(ada, "matter", matter, connector=str(uuid4()))
assert _ask(ada, "matter", matter, external_id="42")[0]["where"] == "here"
def test_the_same_document_in_another_vorgang_is_no_duplicate(ada):
one = matter_of(ada, "Eins")["id"]
two = matter_of(ada, "Zwei")["id"]
_ref(ada, "matter", one)
assert _ask(ada, "matter", two) == []
_ref(ada, "matter", two)
def test_the_check_can_be_switched_off(ada):
matter = matter_of(ada, "Mietminderung")["id"]
_ref(ada, "matter", matter)
_level(ada, "off")
assert _ask(ada, "matter", matter) == []
_ref(ada, "matter", matter)
def test_levels_reach_into_teilvorgang_vorgang_and_akte(ada):
parent = matter_of(ada, "Hauptvorgang")["id"]
child = ada.request("POST", "/api/v1/matters", json={"title": "Teil", "parent_matter_id": parent}, status=201)["id"]
case = ada.request("POST", "/api/v1/cases", json={"title": "Akte"}, status=201)["id"]
ada.request("POST", f"/api/v1/matters/{parent}/case-links", json={"case_id": case}, status=201)
_ref(ada, "matter", child, external_id="1")
_ref(ada, "matter", parent, external_id="2")
_ref(ada, "case", case, external_id="3")
def where(kind, container, external_id):
return [m["where"] for m in _ask(ada, kind, container, external_id)]
# Standard: only what sits directly in the target.
assert where("matter", parent, "1") == []
_level(ada, "inside")
assert where("matter", parent, "1") == ["inside"]
assert where("matter", child, "2") == []
assert where("case", case, "2") == ["inside"]
assert where("case", case, "3") == ["here"]
_level(ada, "around")
assert where("matter", child, "2") == ["around"]
assert where("matter", parent, "3") == ["around"]
assert where("matter", child, "3") == ["around"]
# The server warns at the same level.
_ref(ada, "matter", child, external_id="3", status=409)
def test_an_existing_element_can_be_bound_instead_of_a_new_one(ada):
parent = matter_of(ada, "Hauptvorgang")["id"]
child = ada.request("POST", "/api/v1/matters", json={"title": "Teil", "parent_matter_id": parent}, status=201)["id"]
original = _ref(ada, "matter", child)
_level(ada, "inside")
assert _ask(ada, "matter", parent)[0]["can_bind"] is True
bound = _ref(ada, "matter", parent, bind_element_id=original["id"])
assert bound["id"] == original["id"]
assert [m["where"] for m in _ask(ada, "matter", parent)] == ["here"]
refused = _ref(ada, "matter", parent, bind_element_id=original["id"], status=422)
assert "bereits" in refused["detail"]
def test_a_stranger_neither_sees_nor_binds_my_element(client, db, ada):
mine = matter_of(ada, "Mein Vorgang")["id"]
element = _ref(ada, "matter", mine)
ben = _main(client, db, "ben")
theirs = matter_of(ben, "Ihr Vorgang")["id"]
_level(ben, "around")
assert _ask(ben, "matter", theirs) == []
_ref(ben, "matter", theirs)
_ref(ben, "matter", theirs, bind_element_id=element["id"], status=404)
+43
View File
@@ -36,3 +36,46 @@ def test_links_and_images_are_limited():
assert 'href="https://ok.example/x"' in cleaned assert 'href="https://ok.example/x"' in cleaned
assert 'rel="noreferrer noopener"' in cleaned and 'target="_blank"' in cleaned assert 'rel="noreferrer noopener"' in cleaned and 'target="_blank"' in cleaned
assert 'src="data:image/png;base64,AAAA"' in cleaned assert 'src="data:image/png;base64,AAAA"' in cleaned
def test_remote_pictures_stay_out_unless_the_server_loads_them():
source = '<img src="https://cdn.example/a.png" alt="a"><img src="http://plain.example/b.png"><p>Text</p>'
found: list[str] = []
assert "<img" not in sanitize_html(source, found=found)
assert found == ["https://cdn.example/a.png"]
loaded = sanitize_html(source, remote=lambda url: "data:image/png;base64,AAAA" if "cdn" in url else "")
assert loaded.count("<img") == 1 and 'src="data:image/png;base64,AAAA"' in loaded
def test_the_server_never_fetches_internal_or_odd_addresses():
from app.remoteimages import RemoteImages, fetch_image, is_public
for address in ("127.0.0.1", "10.0.0.5", "192.168.1.1", "169.254.169.254", "::1", "fd00::1", "224.0.0.1", "not-an-ip"):
assert is_public(address) is False
assert is_public("93.184.216.34") is True
for url in (
"http://example.org/a.png",
"https://127.0.0.1/a.png",
"https://localhost/a.png",
"https://[::1]/a.png",
"https://example.org:8443/a.png",
"https://user:pw@example.org/a.png",
"file:///etc/passwd",
"https:///a.png",
):
assert fetch_image(url) is None
assert RemoteImages()("https://127.0.0.1/a.png") == ""
def test_remote_pictures_are_checked_by_content_and_limited(monkeypatch):
import app.remoteimages as module
png = b"\x89PNG\r\n\x1a\n" + b"0" * 20
monkeypatch.setattr(module, "fetch_image", lambda url: png if url.endswith(".png") else None)
loader = module.RemoteImages()
assert loader("https://a.example/1.png").startswith("data:image/png;base64,")
assert loader("https://a.example/1.png").startswith("data:image/png")
assert loader("https://a.example/1.svg") == ""
assert (loader.loaded, loader.failed) == (1, 1)
assert module._kind(b"<svg xmlns='http://www.w3.org/2000/svg'></svg>") is None
assert module._kind(b"RIFF\x00\x00\x00\x00WEBPVP8 ") == "webp"
+192
View File
@@ -285,3 +285,195 @@ def test_an_adopted_message_loads_in_full_with_its_attachment(client, db, monkey
ada.request("GET", f"/api/v1/external-refs/{wrong['element']['id']}/mail", status=422) ada.request("GET", f"/api/v1/external-refs/{wrong['element']['id']}/mail", status=422)
ben.login() ben.login()
ben.request("GET", f"/api/v1/external-refs/{created['id']}/mail", status=404) ben.request("GET", f"/api/v1/external-refs/{created['id']}/mail", status=404)
def _letter(message_id: str, body: str) -> bytes:
return (
f"From: Ada Beispiel <ada@example.test>\r\n"
f"To: Gegenseite <gegen@example.test>\r\n"
f"Subject: Verschoben\r\n"
f"Date: Thu, 02 May 2024 08:00:00 +0000\r\n"
f"Message-ID: {message_id}\r\n"
f"MIME-Version: 1.0\r\n"
f"Content-Type: text/plain; charset=utf-8\r\n"
f"\r\n"
f"{body}\r\n"
).encode()
class RelocatingIMAP:
"""The saved UID holds another message. The Message-ID lives in Archiv."""
log: list[tuple] = []
copies: dict[tuple[str, int], bytes] = {}
header_uids: dict[str, bytes] | None = None
def __init__(self, *args, **kwargs):
self.selected = ""
def login(self, username, secret):
assert secret == "Postfach-Geheim"
return "OK", [b"logged"]
def list(self):
return "OK", [
b'(\\HasNoChildren) "/" "INBOX"',
b'(\\HasNoChildren) "/" "Archiv"',
]
def select(self, mailbox, readonly=False):
assert readonly is True
self.selected = mailbox
RelocatingIMAP.log.append(("select", mailbox))
return "OK", [b"2"]
def uid(self, command, *args):
if command == "SEARCH":
if args and args[0] == "HEADER":
RelocatingIMAP.log.append(("header", self.selected))
override = (RelocatingIMAP.header_uids or {}).get(self.selected)
if override is not None:
return "OK", [override]
found = [
str(uid)
for (folder, uid), raw in RelocatingIMAP.copies.items()
if folder == self.selected and b"Message-ID: <moved@example.test>" in raw
]
return "OK", [" ".join(found).encode()]
return "OK", [b""]
uid = int(args[0])
spec = " ".join(str(part) for part in args[1:])
if "BODY.PEEK[]" in spec:
RelocatingIMAP.log.append(("fetch", self.selected, uid))
raw = RelocatingIMAP.copies.get((self.selected, uid))
if raw is None:
return "OK", [None]
return "OK", [(b"rfc822", raw)]
return "OK", [None]
def logout(self):
return "BYE", [b""]
def _stored_mail(db, element_id: str) -> dict:
row = db.execute(
"SELECT detail FROM external_references WHERE element_id = %s",
(element_id,),
).fetchone()
return dict(row["detail"])
def _linked_mail(actor: Actor, account: str, matter_id: str) -> dict:
return actor.request(
"POST",
"/api/v1/external-refs",
json={
"title": "Verschoben",
"source": "imap",
"external_id": "<moved@example.test>",
"context": {"kind": "matter", "id": matter_id},
"detail": {
"connector_id": account,
"mailbox": "INBOX",
"uid": 2,
"direction": "in",
},
},
status=201,
)
def test_a_moved_message_remembers_its_new_folder(client, db, monkeypatch):
from uuid import UUID
from app.retrieval import MailPayload, load_external
ada = Actor(client, "ada-move@example.test", "Ada Umzug")
account_row = register(db, ada.email, ada.name)
ada.login()
RelocatingIMAP.log = []
RelocatingIMAP.header_uids = None
RelocatingIMAP.copies = {
("INBOX", 2): _letter("<other@example.test>", "Eine andere Nachricht."),
("Archiv", 9): _letter("<moved@example.test>", "Der Brief liegt jetzt im Archiv."),
}
account = _account(monkeypatch, ada)
monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", RelocatingIMAP)
matter = matter_of(ada, "Ablage")
created = _linked_mail(ada, account, matter["id"])
opened = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail")
assert "Archiv" in opened["text"]
stored = _stored_mail(db, created["id"])
assert stored["mailbox"] == "Archiv"
assert stored["uid"] == 9
assert stored["direction"] == "in"
assert stored["connector_id"] == account
assert any(item[0] == "header" for item in RelocatingIMAP.log)
RelocatingIMAP.log = []
again = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail")
assert again["text"] == opened["text"]
assert ("fetch", "Archiv", 9) in RelocatingIMAP.log
assert not any(item[0] == "header" for item in RelocatingIMAP.log)
assert _stored_mail(db, created["id"])["uid"] == 9
db.execute(
"""
UPDATE external_references
SET detail = jsonb_set(
jsonb_set(detail, '{mailbox}', to_jsonb('INBOX'::text)),
'{uid}',
to_jsonb(2)
)
WHERE element_id = %s
""",
(created["id"],),
)
db.commit()
RelocatingIMAP.log = []
loaded = load_external(db, account_row["id"], UUID(created["id"]), None)
assert isinstance(loaded, MailPayload)
assert "Archiv" in loaded.parsed.text
exported = _stored_mail(db, created["id"])
assert exported["mailbox"] == "Archiv"
assert exported["uid"] == 9
assert exported["direction"] == "in"
assert any(item[0] == "header" for item in RelocatingIMAP.log)
def test_an_ambiguous_or_truncated_move_does_not_replace_the_stored_place(client, db, monkeypatch):
ada = Actor(client, "ada-ambiguous@example.test", "Ada Zweifach")
register(db, ada.email, ada.name)
ada.login()
RelocatingIMAP.header_uids = None
RelocatingIMAP.copies = {
("INBOX", 2): _letter("<other@example.test>", "Altbestand."),
("INBOX", 4): _letter("<moved@example.test>", "Erste Kopie."),
("Archiv", 9): _letter("<moved@example.test>", "Zweite Kopie."),
}
account = _account(monkeypatch, ada)
monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", RelocatingIMAP)
matter = matter_of(ada, "Zweifach")
created = _linked_mail(ada, account, matter["id"])
ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail", status=422)
assert _stored_mail(db, created["id"])["mailbox"] == "INBOX"
assert _stored_mail(db, created["id"])["uid"] == 2
RelocatingIMAP.copies = {
("INBOX", 2): _letter("<other@example.test>", "Altbestand."),
("INBOX", 11): _letter("<moved@example.test>", "Treffer in der abgebrochenen Suche."),
("INBOX", 12): _letter("<a@example.test>", "a"),
("INBOX", 13): _letter("<b@example.test>", "b"),
("INBOX", 14): _letter("<c@example.test>", "c"),
("INBOX", 15): _letter("<d@example.test>", "d"),
("INBOX", 16): _letter("<e@example.test>", "e"),
("Archiv", 9): _letter("<moved@example.test>", "Eigentlich auch hier."),
}
RelocatingIMAP.header_uids = {"INBOX": b"11 12 13 14 15 16"}
RelocatingIMAP.log = []
opened = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail")
assert "abgebrochenen Suche" in opened["text"]
assert not any(item == ("header", "Archiv") for item in RelocatingIMAP.log)
assert _stored_mail(db, created["id"])["mailbox"] == "INBOX"
assert _stored_mail(db, created["id"])["uid"] == 2
+79
View File
@@ -0,0 +1,79 @@
"""Einstellungen je Konto: Erbung vom Hauptnutzer, eigener Wert, Verbindlichkeit für das Team."""
from __future__ import annotations
import pytest
from tests.test_team import _invite, _main
PATH = "/api/v1/auth/preferences/duplicate_check"
@pytest.fixture
def world(client, db):
ada = _main(client, db, "ada", main=True)
room = ada.request("POST", "/api/v1/rooms", json={"name": "Kanzlei"}, status=201)["id"]
ada.use_room(room)
emp = _invite(ada, room, "emp-pref@example.test", name="Eva")
return {"ada": ada, "emp": emp}
def test_the_default_applies_without_any_choice(world):
for person in (world["ada"], world["emp"]):
state = person.request("GET", PATH)
assert (state["value"], state["source"]) == ("here", "default")
assert world["ada"].request("GET", PATH)["can_lock"] is True
assert world["emp"].request("GET", PATH)["can_lock"] is False
def test_an_employee_inherits_the_main_user_and_may_override(world):
ada, emp = world["ada"], world["emp"]
ada.request("PUT", PATH, json={"value": "inside"})
inherited = emp.request("GET", PATH)
assert (inherited["value"], inherited["source"]) == ("inside", "team")
own = emp.request("PUT", PATH, json={"value": "off"})
assert (own["value"], own["source"]) == ("off", "own")
# The main user's change does not touch an own choice.
ada.request("PUT", PATH, json={"value": "around"})
assert emp.request("GET", PATH)["value"] == "off"
assert ada.request("GET", PATH)["value"] == "around"
# Inheriting again.
back = emp.request("PUT", PATH, json={"value": None})
assert (back["value"], back["source"]) == ("around", "team")
def test_a_binding_overrides_an_own_value_without_deleting_it(world):
ada, emp = world["ada"], world["emp"]
emp.request("PUT", PATH, json={"value": "off"})
ada.request("PUT", PATH, json={"value": "inside", "team_mode": "locked"})
locked = emp.request("GET", PATH)
assert (locked["value"], locked["source"], locked["locked"]) == ("inside", "locked", True)
refused = emp.request("PUT", PATH, json={"value": "here"}, status=403)
assert "vorgegeben" in refused["detail"]
# The main user is not bound by the own binding.
assert ada.request("PUT", PATH, json={"value": "here"})["value"] == "here"
assert ada.request("GET", PATH)["team_mode"] == "locked"
assert emp.request("GET", PATH)["value"] == "here"
ada.request("PUT", PATH, json={"value": "here", "team_mode": "free"})
again = emp.request("GET", PATH)
assert (again["value"], again["source"]) == ("off", "own")
def test_only_the_main_user_binds_and_values_are_checked(world):
ada, emp = world["ada"], world["emp"]
emp.request("PUT", PATH, json={"value": "off", "team_mode": "locked"}, status=403)
assert emp.request("GET", PATH)["source"] == "default"
ada.request("PUT", PATH, json={"value": "everywhere"}, status=422)
ada.request("GET", "/api/v1/auth/preferences/unknown", status=404)
ada.request("PUT", "/api/v1/auth/preferences/unknown", json={"value": "off"}, status=404)
def test_a_binding_of_a_stranger_does_not_reach_my_employee(client, db, world):
ben = _main(client, db, "ben")
ben.request("PUT", PATH, json={"value": "off", "team_mode": "locked"})
assert world["emp"].request("GET", PATH)["value"] == "here"
+46
View File
@@ -0,0 +1,46 @@
"""R-11: other people's rows must not push your own out of the overview and the search."""
from __future__ import annotations
from tests.support import Actor, case_of, note_on, register
def test_own_entries_are_found_behind_many_foreign_ones(client, db, monkeypatch):
from fastapi.testclient import TestClient
from app.api import routes
from app.main import app
# A tiny batch size makes the paging visible with few rows.
monkeypatch.setattr(routes, "SCAN_BATCH", 2)
register(db, "ada-scan@example.test", "Ada")
register(db, "ben-scan@example.test", "Ben")
ada = Actor(client, "ada-scan@example.test", "Ada")
ben = Actor(TestClient(app), "ben-scan@example.test", "Ben")
ada.login()
ben.login()
ada_case = case_of(ada, "Adas Akte")
for index in range(7):
matter = ada.request("POST", "/api/v1/matters", json={"title": f"Adas Vorgang {index}"}, status=201)
ada.request("POST", "/api/v1/tasks", json={"title": f"Frühe Aufgabe {index}", "due_on": "2026-01-0%d" % (index + 1), "matter_id": matter["id"]}, status=201)
ada.request("POST", "/api/v1/deadlines", json={"title": f"Frühe Frist {index}", "due_on": "2026-01-0%d" % (index + 1), "matter_id": matter["id"]}, status=201)
note_on(ada, f"Gemeinsames Stichwort {index}", "Zebrastreifen", ada_case["id"])
ben_matter = ben.request("POST", "/api/v1/matters", json={"title": "Bens Vorgang"}, status=201)
ben.request("POST", "/api/v1/tasks", json={"title": "Bens späte Aufgabe", "due_on": "2027-01-01", "matter_id": ben_matter["id"]}, status=201)
ben.request("POST", "/api/v1/deadlines", json={"title": "Bens späte Frist", "due_on": "2027-01-01", "matter_id": ben_matter["id"]}, status=201)
ben_case = case_of(ben, "Bens Akte")
note_on(ben, "Bens Zebrastreifen-Notiz", "Zebrastreifen", ben_case["id"])
assert [item["title"] for item in ben.request("GET", "/api/v1/views/open-tasks")["items"]] == ["Bens späte Aufgabe"]
assert [item["title"] for item in ben.request("GET", "/api/v1/views/open-deadlines")["items"]] == ["Bens späte Frist"]
found = ben.request("GET", "/api/v1/search", params={"q": "Zebrastreifen"})["items"]
assert [item["title"] for item in found] == ["Bens Zebrastreifen-Notiz"]
# The own entries still come in order and the cap on the list holds.
monkeypatch.setattr(routes, "VIEW_LIMIT", 3)
first = ada.request("GET", "/api/v1/views/open-tasks")["items"]
assert [item["title"] for item in first] == ["Frühe Aufgabe 0", "Frühe Aufgabe 1", "Frühe Aufgabe 2"]
monkeypatch.setattr(routes, "SEARCH_LIMIT", 4)
assert len(ada.request("GET", "/api/v1/search", params={"q": "Zebrastreifen"})["items"]) == 4
+3 -3
View File
@@ -30,7 +30,7 @@ Unter `/home/lars/docker/` legt der Lauf nur diese zwei Verzeichnisse an:
|----------|---------|--------| |----------|---------|--------|
| `deploy-dev.yml` | Push `develop` | Deploy nach `/home/lars/docker/tsumugu-dev`, Health `localhost:8094` und `localhost:3094` | | `deploy-dev.yml` | Push `develop` | Deploy nach `/home/lars/docker/tsumugu-dev`, Health `localhost:8094` und `localhost:3094` |
| `deploy-prod.yml` | Push `main` | Deploy nach `/home/lars/docker/tsumugu`, Health `localhost:8007` und `localhost:3007` | | `deploy-prod.yml` | Push `main` | Deploy nach `/home/lars/docker/tsumugu`, Health `localhost:8007` und `localhost:3007` |
| `test.yml` | Push `develop`, nach erfolgreichem Dev-Deploy, manuell | Pytest gegen `tsumugu_test` auf der Dev-Postgres, Smoke der laufenden Dev-App, Frontend-Build | | `test.yml` | Push auf `develop` (wartet, bis der Dev-Checkout den Commit enthält), manuell | Pytest gegen `tsumugu_test` auf der Dev-Postgres, Smoke der laufenden Dev-App, Frontend-Build |
Gearbeitet wird auf `develop`. Production entsteht nur durch einen Merge-Commit `develop` → `main` (`git merge --no-ff develop` auf `main`) und den anschließenden Push von `main`. Kein Fast-Forward, kein direkter Commit auf `main`, kein manuelles `git pull` auf dem Prod-Checkout. Gitea (`deploy-prod.yml`) ist der einzige Prod-Schreibpfad für Code. Gearbeitet wird auf `develop`. Production entsteht nur durch einen Merge-Commit `develop` → `main` (`git merge --no-ff develop` auf `main`) und den anschließenden Push von `main`. Kein Fast-Forward, kein direkter Commit auf `main`, kein manuelles `git pull` auf dem Prod-Checkout. Gitea (`deploy-prod.yml`) ist der einzige Prod-Schreibpfad für Code.
@@ -50,11 +50,11 @@ Diese Punkte sind im [Code-Review](review/06_BEFUNDE.md) beschrieben. Sie stehen
|---|---|---|---| |---|---|---|---|
| Betreiberkonto | Auf einer leeren Datenbank legt die öffentliche Anmeldeseite das Betreiberkonto an. Keine MFA | Nach jedem Neuaufsetzen einer Umgebung sofort selbst anmelden, bevor die Adresse öffentlich erreichbar ist. Alternativ vorher `docker compose exec backend python -m app.cli create-account --email … --name … --password …` ausführen | R-04 | | Betreiberkonto | Auf einer leeren Datenbank legt die öffentliche Anmeldeseite das Betreiberkonto an. Keine MFA | Nach jedem Neuaufsetzen einer Umgebung sofort selbst anmelden, bevor die Adresse öffentlich erreichbar ist. Alternativ vorher `docker compose exec backend python -m app.cli create-account --email … --name … --password …` ausführen | R-04 |
| Sicherung | Es gibt kein Backup von Datenbank und Volume `tsumugu-media`/`dev-tsumugu-media` und keinen geprobten Restore | Laut technischer Spezifikation 8.2 nur synthetische Daten in Produktion, bis Paket P7 umgesetzt ist | R-21 | | Sicherung | Es gibt kein Backup von Datenbank und Volume `tsumugu-media`/`dev-tsumugu-media` und keinen geprobten Restore | Laut technischer Spezifikation 8.2 nur synthetische Daten in Produktion, bis Paket P7 umgesetzt ist | R-21 |
| Release | `deploy-prod.yml` baut bei jedem Push auf `main` neu. Tests laufen nur auf `develop` nach dem Dev-Deploy | Vor dem Merge nach `main` den grünen `test.yml`-Lauf zum selben Stand prüfen | R-21, R-27 | | Release | `deploy-prod.yml` baut bei jedem Push auf `main` neu. Tests laufen nur auf `develop`, sobald der Dev-Deploy den Commit ausgecheckt hat | Vor dem Merge nach `main` den grünen `test.yml`-Lauf zum selben Stand prüfen | R-21, R-27 |
| Datenbankzugang | Das Backend migriert beim Start mit dem Owner-Zugang. Die Runtime-Rolle `tsumugu_runtime` erhält dasselbe Passwort wie `DB_USER` | Ein Wechsel von `DB_PASSWORD` ändert beim nächsten Start beide Rollen | R-09 | | Datenbankzugang | Das Backend migriert beim Start mit dem Owner-Zugang. Die Runtime-Rolle `tsumugu_runtime` erhält dasselbe Passwort wie `DB_USER` | Ein Wechsel von `DB_PASSWORD` ändert beim nächsten Start beide Rollen | R-09 |
| Quellen-Geheimnisse | Paperless-Token und IMAP-Passwörter stehen unverschlüsselt in der Tabelle `connectors` | Datenbank-Dumps wie Zugangsdaten schützen | R-05 | | Quellen-Geheimnisse | Paperless-Token und IMAP-Passwörter stehen unverschlüsselt in der Tabelle `connectors` | Datenbank-Dumps wie Zugangsdaten schützen | R-05 |
| Sessions | 14 Tage gültig, keine Leerlaufabmeldung, kein Passwortwechsel in der Anwendung | Passwort nur über die Datenbank oder ein neues Konto änderbar | R-08 | | Sessions | 14 Tage gültig, keine Leerlaufabmeldung, kein Passwortwechsel in der Anwendung | Passwort nur über die Datenbank oder ein neues Konto änderbar | R-08 |
| Testläufe | `test.yml` startet je Push zweimal auf derselben Datenbank `tsumugu_test` | Ein roter Lauf kann vom parallelen Lauf verursacht sein; Ergebnis des Einzellaufs prüfen | R-27 | | Testläufe | `test.yml` startet je Push einmal, parallel zum Deploy, und wartet auf dessen Checkout (`workflow_run` startete auf dieser Gitea nicht verlässlich); alle Läufe teilen die Datenbank `tsumugu_test`, und das Image bleibt das Laufzeitimage | Zwei manuelle Läufe gleichzeitig vermeiden; sie würden sich die Datenbank wegnehmen | R-27 |
--- ---
+3
View File
@@ -7,6 +7,9 @@ Diese Regeln sind die Arbeitsgrundlage der ersten Lieferung. Sie sind keine endg
3. **Einbindung und Löschen.** „Aus diesem Kontext entfernen“ löscht die Einbindung und ihre Anmerkungen. Das Element bleibt bei der Eigentümerin oder dem Eigentümer, auch ohne Einbindung. „Überall löschen“ darf nur diese Person. Paperless-Originale werden nicht gelöscht. 3. **Einbindung und Löschen.** „Aus diesem Kontext entfernen“ löscht die Einbindung und ihre Anmerkungen. Das Element bleibt bei der Eigentümerin oder dem Eigentümer, auch ohne Einbindung. „Überall löschen“ darf nur diese Person. Paperless-Originale werden nicht gelöscht.
4. **Aktenvererbung.** Die Zuordnung eines Vorgangs gibt die Rechte dieser Akte an den Vorgang und an Inhalte ohne ausdrückliche Ausnahme weiter. Was die zuordnende Person nicht weitergeben darf, bleibt in diesem Aktenkontext unsichtbar, ohne Titel und ohne Anzahl. Querverweise vererben nichts. Die Ownership bleibt unverändert. 4. **Aktenvererbung.** Die Zuordnung eines Vorgangs gibt die Rechte dieser Akte an den Vorgang und an Inhalte ohne ausdrückliche Ausnahme weiter. Was die zuordnende Person nicht weitergeben darf, bleibt in diesem Aktenkontext unsichtbar, ohne Titel und ohne Anzahl. Querverweise vererben nichts. Die Ownership bleibt unverändert.
5. **Herunterladen von Originalen.** Das Recht „Herunterladen“ ist getrennt vom Lesen, standardmäßig überall erlaubt und je Raum und Rolle abschaltbar (Voreinstellung am Konto des Hauptnutzers). Es ist eine Nutzungsregel, kein Kopierschutz: Lesen in der App bleibt möglich, Abfotografieren und Entwicklerwerkzeuge verhindert die Regel nicht. Ob später eine strengere Rechtegrundlage nötig ist, entscheiden Datenschutz und Recht.
6. **Bilder in E-Mails.** Bilder fremder Server werden nicht automatisch geladen. Auf Wunsch lädt der Server sie für die Person nach, das Gerät kontaktiert den Absender nicht. Dass die Nachricht geöffnet wurde, kann der Absender erkennen.
Der gewählte Datenraum legt fest, wo neue Objekte entstehen. Er ist nicht selbst der Rechtepfad. Eine gültige Sitzung muss einen Datenraum gewählt haben; der Server prüft danach den mitgeschickten Handlungskontext. Der gewählte Datenraum legt fest, wo neue Objekte entstehen. Er ist nicht selbst der Rechtepfad. Eine gültige Sitzung muss einen Datenraum gewählt haben; der Server prüft danach den mitgeschickten Handlungskontext.
Technische Anpassung: Das Session-Cookie heißt in Production `__Host-jinkendo-session` und ist `Secure`. Development nutzt diesen Namen nur bei HTTPS. Über die dokumentierten HTTP-Adressen im LAN heißt es `jinkendo-session`, weil ein `__Host-`-Cookie dort nicht gespeichert werden kann. Technische Anpassung: Das Session-Cookie heißt in Production `__Host-jinkendo-session` und ist `Secure`. Development nutzt diesen Namen nur bei HTTPS. Über die dokumentierten HTTP-Adressen im LAN heißt es `jinkendo-session`, weil ein `__Host-`-Cookie dort nicht gespeichert werden kann.
+20 -12
View File
@@ -1,6 +1,6 @@
# Tsumugu – API-Referenz (Ist-Stand) # Tsumugu – API-Referenz (Ist-Stand)
**Stand:** 18.10.2026, ergänzt am 19.10.2026 um Quelleninstanzen, Fähigkeiten und Zuordnung. Erstellt im Code-Review aus `backend/app/main.py`, `api/routes.py`, `api/sources.py`, `api/export.py` und `api/correspondence.py`. **Stand:** 18.10.2026, ergänzt am 19.10.2026 um Quelleninstanzen, Fähigkeiten und Zuordnung und am 23.10.2026 um Download-Regeln, HTML-Mail mit Bildern, Übersicht und Suche. Erstellt im Code-Review aus `backend/app/main.py`, `api/routes.py`, `api/sources.py`, `api/export.py` und `api/correspondence.py`.
Diese Datei beschreibt die tatsächlich registrierten Endpunkte. Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 6.2](TSUMUGU_TECHNICAL_SPEC.md). OpenAPI ist in allen Umgebungen abgeschaltet (`docs_url=None`, `openapi_url=None`). Diese Datei ist deshalb die einzige Übersicht. Sie wird bei jeder Routenänderung mitgepflegt. Diese Datei beschreibt die tatsächlich registrierten Endpunkte. Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 6.2](TSUMUGU_TECHNICAL_SPEC.md). OpenAPI ist in allen Umgebungen abgeschaltet (`docs_url=None`, `openapi_url=None`). Diese Datei ist deshalb die einzige Übersicht. Sie wird bei jeder Routenänderung mitgepflegt.
@@ -33,6 +33,8 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R**
|---|---|---|---| |---|---|---|---|
| POST | `/api/v1/auth/login` | – | E-Mail, Passwort → Session-Sicht und Cookie. 8 Fehlversuche je Bucket (Hash aus IP und E-Mail) in 10 min, gezählt in `auth_login_attempts` | | POST | `/api/v1/auth/login` | – | E-Mail, Passwort → Session-Sicht und Cookie. 8 Fehlversuche je Bucket (Hash aus IP und E-Mail) in 10 min, gezählt in `auth_login_attempts` |
| POST | `/api/v1/auth/logout` | S | Session widerrufen, Cookie löschen, 204 | | POST | `/api/v1/auth/logout` | S | Session widerrufen, Cookie löschen, 204 |
| GET / PUT | `/api/v1/auth/download-defaults` | S+R, Hauptnutzer | Voreinstellung je Rolle und `direct` (direkt freigegebene Personen); neue Räume übernehmen die drei Rollenwerte. Audit `download_defaults_changed` |
| GET / PUT | `/api/v1/auth/preferences/{key}` | S+R | Einstellung des Kontos mit Erbung. GET liefert `value` (wirksam), `source` (`locked`/`own`/`team`/`default`), `own`, `inherited`, `locked`, `allowed`, `can_lock`, `team_mode`. PUT setzt den eigenen Wert (`value`, `null` = erben) und beim Hauptnutzer die Verbindlichkeit für das Team (`team_mode` `free`/`locked`). Ein Mitarbeiter erhält bei verbindlicher Vorgabe 403. Schlüssel: `duplicate_check` (`off`/`here`/`inside`/`around`, Standard `here`). Audit `preference_changed` |
| GET | `/api/v1/auth/session` | S | Konto, bestätigter Datenraum, Räume, CSRF-Token, Epoche | | GET | `/api/v1/auth/session` | S | Konto, bestätigter Datenraum, Räume, CSRF-Token, Epoche |
| POST | `/api/v1/auth/session/scope` | S | Datenraum wechseln (`kind`, `id`); erhöht die Epoche | | POST | `/api/v1/auth/session/scope` | S | Datenraum wechseln (`kind`, `id`); erhöht die Epoche |
| GET | `/api/v1/auth/bootstrap` | – | `{available: true}`, solange kein Betreiber existiert, sonst 404 | | GET | `/api/v1/auth/bootstrap` | – | `{available: true}`, solange kein Betreiber existiert, sonst 404 |
@@ -59,6 +61,7 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R**
| GET | `/api/v1/rooms` | S+R | wählbare Räume; `include_archived=true` ergänzt die archivierten Räume der Person | | GET | `/api/v1/rooms` | S+R | wählbare Räume; `include_archived=true` ergänzt die archivierten Räume der Person |
| POST | `/api/v1/rooms` | S+R | gemeinsamen Raum anlegen, Anlegende Person wird Verwalter | | POST | `/api/v1/rooms` | S+R | gemeinsamen Raum anlegen, Anlegende Person wird Verwalter |
| PATCH | `/api/v1/rooms/{id}` | S+R, M / E | `name` (Verwalter); `status` und `default_visibility` nur der Eigentümer (E). Archivieren setzt Sitzungen im Raum auf den persönlichen Raum zurück | | PATCH | `/api/v1/rooms/{id}` | S+R, M / E | `name` (Verwalter); `status` und `default_visibility` nur der Eigentümer (E). Archivieren setzt Sitzungen im Raum auf den persönlichen Raum zurück |
| GET / PUT | `/api/v1/rooms/{id}/download-policy` | S+R, Mitglied / E | je Rolle (`manager`, `member`, `reader`), ob das Original gespeichert oder geöffnet werden darf; `can_change` nennt, ob die Person ändern darf. Fremde und unbekannte Räume: 404. Audit `room_download_policy_changed` |
| GET | `/api/v1/rooms/{id}/members` | S+R, M | Mitglieder mit Rolle und Kontostatus | | GET | `/api/v1/rooms/{id}/members` | S+R, M | Mitglieder mit Rolle und Kontostatus |
| POST | `/api/v1/rooms/{id}/members` | S+R, M | bestehendes Konto als `member`, `manager` oder `reader` aufnehmen oder dessen Rolle setzen | | POST | `/api/v1/rooms/{id}/members` | S+R, M | bestehendes Konto als `member`, `manager` oder `reader` aufnehmen oder dessen Rolle setzen |
| PATCH | `/api/v1/rooms/{id}/members/{subject_id}` | S+R, M | Rolle ändern; letzter Verwalter bleibt. Rückstufung auf `reader` entzieht Bearbeiten- und Freigeben-Grants auf Objekte des Raums | | PATCH | `/api/v1/rooms/{id}/members/{subject_id}` | S+R, M | Rolle ändern; letzter Verwalter bleibt. Rückstufung auf `reader` entzieht Bearbeiten- und Freigeben-Grants auf Objekte des Raums |
@@ -142,8 +145,9 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R**
| POST | `/api/v1/notes` | Notiz mit Einbindung (`context`, optional `placement`, `chapter_id`, `dates`, `communication_direction`) | | POST | `/api/v1/notes` | Notiz mit Einbindung (`context`, optional `placement`, `chapter_id`, `dates`, `communication_direction`) |
| PATCH | `/api/v1/notes/{id}` | gemeinsamen Text ändern; Antwort mit `visible_bindings` und `notice` | | PATCH | `/api/v1/notes/{id}` | gemeinsamen Text ändern; Antwort mit `visible_bindings` und `notice` |
| POST | `/api/v1/files` | Datei hochladen (Multipart: `title`, `context_kind`, `context_id`, `upload`, optional Platzierung, Kapitel, Daten, Richtung); PDF, PNG, JPEG, WebP bis 20 MiB | | POST | `/api/v1/files` | Datei hochladen (Multipart: `title`, `context_kind`, `context_id`, `upload`, optional Platzierung, Kapitel, Daten, Richtung); PDF, PNG, JPEG, WebP bis 20 MiB |
| GET | `/api/v1/files/{id}/content` | Datei als `attachment`, `no-store` | | GET | `/api/v1/files/{id}/content` | Datei als `attachment` mit Dateiname (UTF-8, `filename*`), `no-store`. Mit `save=true` zusätzlich das Download-Recht, sonst 403 |
| POST | `/api/v1/external-refs` | externe Referenz (`paperless`, `imap`, `url`) mit `detail` | | POST | `/api/v1/external-refs` | externe Referenz (`paperless`, `imap`, `url`) mit `detail`. `duplicate` (`ask` Standard, `insert`): bei `ask` antwortet ein bereits vorhandenes Dokument mit 409 `already_present`, wenn nur sichere Treffer auf der Prüfstufe des Kontos zählen (Einstellung `duplicate_check`). `bind_element_id` bindet stattdessen das vorhandene Element zusätzlich in den Zielcontainer (eine Referenz, eine weitere Einbindung; Audit `external_ref_bound`). Nur ein sicherer, lesbarer Treffer im Umfeld des Ziels, mit Weitergaberecht (sonst 403 bzw. 404) |
| POST | `/api/v1/external-refs/existing` | nur lesend: `context`, optional `placement`, `items` (`source`, `external_id`, `connector_id`) → je Dokument die Treffer (`element_id`, `title`, `where` `here`/`inside`/`around`, `certain`, `chapter`, `container`, `can_bind`) auf der Prüfstufe des Kontos. Nur Lesbares, ein Treffer außerhalb der Rechte gilt als nicht vorhanden |
| POST | `/api/v1/sources/link` | wie oben, nur `paperless`/`imap` (doppelter Weg, siehe Connectoren-Befund) | | POST | `/api/v1/sources/link` | wie oben, nur `paperless`/`imap` (doppelter Weg, siehe Connectoren-Befund) |
| GET | `/api/v1/elements` | eigene, nicht eingebundene Elemente (max. 100) | | GET | `/api/v1/elements` | eigene, nicht eingebundene Elemente (max. 100) |
| POST | `/api/v1/elements/{id}/delete` | überall löschen; nur Element-Owner | | POST | `/api/v1/elements/{id}/delete` | überall löschen; nur Element-Owner |
@@ -174,8 +178,8 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R**
| POST | `/api/v1/deadlines` | Frist | | POST | `/api/v1/deadlines` | Frist |
| PATCH | `/api/v1/deadlines/{id}` | Titel, Datum | | PATCH | `/api/v1/deadlines/{id}` | Titel, Datum |
| POST | `/api/v1/deadlines/{id}/fulfill` | als erfüllt kennzeichnen | | POST | `/api/v1/deadlines/{id}/fulfill` | als erfüllt kennzeichnen |
| GET | `/api/v1/views/open-tasks` | offene sichtbare Aufgaben (siehe Review R-11) | | GET | `/api/v1/views/open-tasks` | offene sichtbare Aufgaben, höchstens 200. Gelesen wird in Blöcken mit Rechteprüfung je Zeile (R-11 behoben) |
| GET | `/api/v1/views/open-deadlines` | offene sichtbare Fristen (siehe Review R-11) | | GET | `/api/v1/views/open-deadlines` | offene sichtbare Fristen, wie oben |
| GET | `/api/v1/views/recent-matters` | acht zuletzt geänderte sichtbare Vorgänge | | GET | `/api/v1/views/recent-matters` | acht zuletzt geänderte sichtbare Vorgänge |
| GET | `/api/v1/placement-targets` | bearbeitbare Akten und Vorgänge als Ziele | | GET | `/api/v1/placement-targets` | bearbeitbare Akten und Vorgänge als Ziele |
| GET | `/api/v1/search?q=` | Akten, Vorgänge, Elemente (Titel und Notiztext), max. 50 | | GET | `/api/v1/search?q=` | Akten, Vorgänge, Elemente (Titel und Notiztext), max. 50 |
@@ -191,17 +195,21 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R**
| DELETE | `/api/v1/connectors/{id}` | Zugang löschen; bestehende Referenzen bleiben und werden nicht umgehängt | | DELETE | `/api/v1/connectors/{id}` | Zugang löschen; bestehende Referenzen bleiben und werden nicht umgehängt |
| POST | `/api/v1/connectors/{id}/probe` | gespeicherten Zugang prüfen; Ergebnis (auch ein Fehler) wird als Prüfstand festgehalten. Es gibt keine Prüfung mit ungespeicherten Entwürfen | | POST | `/api/v1/connectors/{id}/probe` | gespeicherten Zugang prüfen; Ergebnis (auch ein Fehler) wird als Prüfstand festgehalten. Es gibt keine Prüfung mit ungespeicherten Entwürfen |
| GET | `/api/v1/sources/capabilities` | erklärte Such-, Lese- und Editorfähigkeiten; nicht Vorhandenes bleibt falsch | | GET | `/api/v1/sources/capabilities` | erklärte Such-, Lese- und Editorfähigkeiten; nicht Vorhandenes bleibt falsch |
| GET | `/api/v1/sources/{kind}/search` | Suche der registrierten Quelle im erklärten Umfang; unbekannte Art und nicht erklärte Filter antworten `422` | | GET | `/api/v1/sources/{kind}/search` | Suche der registrierten Quelle im erklärten Umfang. Außer `q`, `connector_id`, `mailbox`, `before` und `uidvalidity` ist jeder Abfrageparameter ein Filter, den das Modul in `search_params` deklariert; unbekannte Art, nicht deklarierte oder nicht erklärte Filter und unbekannte Parameter antworten `422` |
| GET | `/api/v1/sources/{kind}/navigate?connector_id=` | Fachnavigation, wenn die Quelle sie erklärt | | GET | `/api/v1/sources/{kind}/navigate?connector_id=` | Fachnavigation, wenn die Quelle sie erklärt |
| GET | `/api/v1/sources/{kind}/preview` | Vorschau vor dem Übernehmen, wenn die Quelle sie erklärt | | GET | `/api/v1/sources/{kind}/preview` | Vorschau vor dem Übernehmen, wenn die Quelle sie erklärt |
| GET | `/api/v1/sources/paperless/documents?q=&connector_id=` | Kompatibilitätshülle der Paperless-Suche; delegiert an dieselbe Suche | | GET | `/api/v1/sources/paperless/documents?q=&connector_id=` | Paperless-Suche, 20 Treffer je Seite, `cursor` ist die Seitennummer. `mode`: `title_content` (Standard, Titel und Inhalt ohne Index), `title`, `content`, `index` (Volltextindex von Paperless). Filter: `title`, `correspondent`, `document_type`, `storage_path` (jeweils Nummer oder `none` für „ohne“), `tag` (Nummern, durch Komma) mit `tag_mode` `all` (Standard), `any` oder `none`, `document_from`/`document_to` (Dokumentdatum), `added_from`/`added_to` (Hinzugefügt), `asn`, `custom` (JSON-Liste `[{field, op, value}]`, höchstens 5, alle zugleich; `field` ist die Nummer aus `facets.custom_fields`, `op` je Feldtyp `exact`, `icontains`, `istartswith`, `iendswith`, `gt`, `gte`, `lt`, `lte`, `range` oder `exists`; Dokumentverweis-Felder sind nicht filterbar; wird als `custom_field_query` an Paperless gegeben), `sort` (`created`, `added`, `title`, jeweils mit `-`; `-score` nur bei `mode=index` mit Suchtext). Ein ungültiger Wert antwortet `422`. Je Treffer: `title`, `document_on`, `added_on`, `correspondent`, `document_type`, `storage_path`, `tags` (Namen), `pages`, `asn`, `snippet` (Textauszug, reiner Text) und `open_url`; dazu `total`, `next_page`, `previous_page` |
| GET | `/api/v1/sources/{kind}/facets?connector_id=` | Auswahllisten der Instanz (Paperless: Korrespondenten, Dokumenttypen, Speicherpfade, Tags, Zusatzfelder mit Typ und Auswahlwerten). Namen und Listen werden je Instanz und Schlüssel eine Minute lang vorgehalten |
| GET | `/api/v1/sources/paperless/documents/{id}/preview?connector_id=` | Datei zum Ansehen vor dem Übernehmen |
| GET | `/api/v1/sources/{kind}/thumbnail?connector_id=&item=` | Vorschaubild eines Treffers, nur wenn das Modul `search.result_preview` erklärt und ein `thumbnail` bereitstellt, sonst `422`. Läuft über den Server der eigenen Instanz, `private, no-store`, nur Rastergrafiken (PNG, JPEG, WebP, GIF); fremde Instanz `404`, eine Antwort ohne Bild `422` |
| GET | `/api/v1/sources/imap/mailboxes?connector_id=` | IMAP-Fächer | | GET | `/api/v1/sources/imap/mailboxes?connector_id=` | IMAP-Fächer |
| GET | `/api/v1/sources/imap/messages?connector_id=&mailbox=&q=&before=` | Nachrichtenliste, 30 je Seite, Fortsetzung über `before` | | GET | `/api/v1/sources/imap/messages?connector_id=&mailbox=&q=&before=` | Nachrichtenliste, 30 je Seite, Fortsetzung über `before` |
| GET | `/api/v1/sources/imap/message?connector_id=&mailbox=&uid=` | Vorschau vor dem Übernehmen | | GET | `/api/v1/sources/imap/message?connector_id=&mailbox=&uid=` | Vorschau vor dem Übernehmen |
| GET | `/api/v1/external-refs/{id}/content` | Original der gebundenen Instanz über das registrierte Modul (`inline`) | | GET | `/api/v1/external-refs/{id}/content` | Original der gebundenen Instanz über das registrierte Modul (`inline`, mit Dateiname). Mit `save=true` zusätzlich das Download-Recht |
| GET | `/api/v1/external-refs/{id}/mail` | vollständige E-Mail als JSON, Identität per Message-ID geprüft | | GET | `/api/v1/external-refs/{id}/mail` | vollständige E-Mail als JSON, Identität per Message-ID geprüft; `html` ist bereinigt, `remote_images` zählt Bilder fremder Server |
| GET | `/api/v1/external-refs/{id}/mail/original` | EML | | GET | `/api/v1/external-refs/{id}/mail/html-with-images` | dieselbe bereinigte Mail, die Bilder fremder Server lädt der Server auf Anforderung und liefert sie eingebettet (`loaded`, `failed`) |
| GET | `/api/v1/external-refs/{id}/mail/attachments/{index}` | Anlage (`inline`, siehe Review R-03) | | GET | `/api/v1/external-refs/{id}/mail/original` | EML unter dem Betreff als Name; verlangt das Download-Recht (403 sonst) |
| GET | `/api/v1/external-refs/{id}/mail/attachments/{index}` | Anlage (`inline`, siehe Review R-03); mit `save=true` das Download-Recht |
| POST | `/api/v1/external-refs/{id}/assignment` | Owner ordnet eine mehrdeutige oder ungebundene Referenz bewusst einer eigenen Instanz zu | | POST | `/api/v1/external-refs/{id}/assignment` | Owner ordnet eine mehrdeutige oder ungebundene Referenz bewusst einer eigenen Instanz zu |
Quelleninhalte einer eigenen Referenz laufen über die gebundene Instanz. Eine freigegebene Referenz mit `use_owner_sources` nutzt den gebundenen Zugang des Owners. Zugangsdaten und die übrigen Treffer dieses Zugangs werden nicht ausgeliefert. Eine Suche ohne `connector_id` wählt bei mehreren eigenen Instanzen nicht den zuletzt angelegten Zugang (Review R-10). Quelleninhalte einer eigenen Referenz laufen über die gebundene Instanz. Eine freigegebene Referenz mit `use_owner_sources` nutzt den gebundenen Zugang des Owners. Zugangsdaten und die übrigen Treffer dieses Zugangs werden nicht ausgeliefert. Eine Suche ohne `connector_id` wählt bei mehreren eigenen Instanzen nicht den zuletzt angelegten Zugang (Review R-10).
@@ -212,7 +220,7 @@ Quelleninhalte einer eigenen Referenz laufen über die gebundene Instanz. Eine f
|---|---|---| |---|---|---|
| GET | `/api/v1/matters/{id}/export-tree` | Auswahlbaum mit `exportable` und Grund | | GET | `/api/v1/matters/{id}/export-tree` | Auswahlbaum mit `exportable` und Grund |
| POST | `/api/v1/matters/{id}/export/review` | Prüfung: `ready`, `issues`, `notices`, `entries` mit Paketpfaden | | POST | `/api/v1/matters/{id}/export/review` | Prüfung: `ready`, `issues`, `notices`, `entries` mit Paketpfaden |
| POST | `/api/v1/matters/{id}/export` | ZIP nur bei `ready`; Audit `export_created`. Knoten `attachment`, `dispatch-item` und `evidence` gehören zur Auswahl | | POST | `/api/v1/matters/{id}/export` | ZIP nur bei `ready` und mit Download-Recht am Vorgang (403 sonst); Audit `export_created`. Knoten `attachment`, `dispatch-item` und `evidence` gehören zur Auswahl |
## Anlagen und Versand ## Anlagen und Versand
+8
View File
@@ -108,3 +108,11 @@ Der Baum „Meine Ablage“ steht links auf den Listenseiten und auf den Seiten
### E-Mails als HTML ### E-Mails als HTML
Die Mail wird formatiert (Standard) oder als Text gezeigt. Der Server bereinigt das HTML mit einer Positivliste (keine Skripte, Formulare, Rahmen, `<style>`-Blöcke; Formatierung nur aus erlaubten Eigenschaften, ohne `url()`, Position oder Ausdrücke; Bilder nur eingebettet als PNG/JPEG/GIF/WebP; Links nur `http(s)` und `mailto`, mit `noopener`). Im Browser läuft die Ansicht in einem Rahmen mit leerem `sandbox` (eigene Herkunft, keine Skripte), Content-Security-Policy `default-src 'none'; img-src data:` und ohne Referrer. Fremde Bilder werden nicht geladen. Die Mail wird formatiert (Standard) oder als Text gezeigt. Der Server bereinigt das HTML mit einer Positivliste (keine Skripte, Formulare, Rahmen, `<style>`-Blöcke; Formatierung nur aus erlaubten Eigenschaften, ohne `url()`, Position oder Ausdrücke; Bilder nur eingebettet als PNG/JPEG/GIF/WebP; Links nur `http(s)` und `mailto`, mit `noopener`). Im Browser läuft die Ansicht in einem Rahmen mit leerem `sandbox` (eigene Herkunft, keine Skripte), Content-Security-Policy `default-src 'none'; img-src data:` und ohne Referrer. Fremde Bilder werden nicht geladen.
### Bilder in E-Mails nachladen
Bilder von fremden Servern bleiben standardmäßig draußen. „Bilder nachladen“ (oder die Einstellung „Automatisch nachladen“ unter „Mein Konto“, getrennt für Computer und Touch-Geräte, gespeichert im Browser; die Gerätegruppe folgt `pointer: coarse`) ruft `GET /external-refs/{id}/mail/html-with-images` auf. Der Server lädt die Bilder selbst und liefert sie eingebettet zurück, das Gerät kontaktiert den Absender nicht. Schutz (`app/remoteimages.py`): nur `https` auf Port 443, keine Zugangsdaten in der Adresse, Auflösung der Adresse einmal und Verbindung genau dorthin (kein DNS-Rebinding), alle Ergebnisse müssen öffentliche Adressen sein (kein Loopback, privat, Link-Local, Metadaten), höchstens drei Weiterleitungen mit gleicher Prüfung, 5 Sekunden, 1,5 MB je Bild, 12 Bilder und 6 MB je Nachricht, Prüfung des Inhalts an der Dateikennung (PNG, JPEG, GIF, WebP; kein SVG). Restrisiko: Der Absender sieht den Abruf und weiß damit, dass die Nachricht geöffnet wurde.
### Handy
PDFs werden auf iPhone und iPad seitenweise mit pdf.js (Legacy-Build) gezeichnet, weil Safari PDFs in einem Rahmen nicht zuverlässig anzeigt (`pages/DocumentView.jsx`); die PDF-Vorschau in der Liste ist dort weiterhin eingeschränkt, die Leseansicht funktioniert. Der Rahmen der Ablage begrenzt die Höhe, damit die Liste in Akte und Vorgang scrollt, und der Kopf der Leseansicht ist am Handy kompakt.
+4 -1
View File
@@ -1,6 +1,6 @@
# Tsumugu – Datenmodell (Ist-Stand) # Tsumugu – Datenmodell (Ist-Stand)
**Stand:** 19.10.2026, nach den Migrationen in `backend/migrations/` (letzte: `20261022T090000Z_account_kinds.sql`). Entstanden im Code-Review vom 18.10.2026 und für die Connector-Grundlage nachgezogen. Die Arten `connectors.kind` und `external_references.source` prüft die Registrierung. Die früheren Prüflisten in der Datenbank sind entfernt. `url` bleibt eine Referenz ohne Connector. **Stand:** 19.10.2026, nach den Migrationen in `backend/migrations/` (letzte: `20261025T090000Z_account_preferences.sql`). Entstanden im Code-Review vom 18.10.2026 und für die Connector-Grundlage nachgezogen. Die Arten `connectors.kind` und `external_references.source` prüft die Registrierung. Die früheren Prüflisten in der Datenbank sind entfernt. `url` bleibt eine Referenz ohne Connector.
Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 4](TSUMUGU_TECHNICAL_SPEC.md). Diese Datei beschreibt, was tatsächlich angelegt ist. Bei jeder neuen Migration nachziehen. Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 4](TSUMUGU_TECHNICAL_SPEC.md). Diese Datei beschreibt, was tatsächlich angelegt ist. Bei jeder neuen Migration nachziehen.
@@ -21,6 +21,9 @@ Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 4](TSUMUGU_TE
| `auth_invitations` | `email`, `room_id`, `token_hash`, `invited_by`, `expires_at`, `accepted_at`, `role`, `revoked_at` | 7 Tage gültig | | `auth_invitations` | `email`, `room_id`, `token_hash`, `invited_by`, `expires_at`, `accepted_at`, `role`, `revoked_at` | 7 Tage gültig |
| `data_rooms` | `id`, `kind` (`personal`/`organization`), `name`, `status` (`active`/`archived`), `created_by` | persönlicher Raum: `id = account_id` | | `data_rooms` | `id`, `kind` (`personal`/`organization`), `name`, `status` (`active`/`archived`), `created_by` | persönlicher Raum: `id = account_id` |
| `room_memberships` | PK (`room_id`, `subject_id`), `role` (`member`/`manager`/`reader`), `status` (`active`/`removed`) | – | | `room_memberships` | PK (`room_id`, `subject_id`), `role` (`member`/`manager`/`reader`), `status` (`active`/`removed`) | – |
| `auth_accounts` (Spalte seit 23.10.2026) | `download_defaults` (jsonb: `manager`, `member`, `reader`, `direct`; Voreinstellung für das Herunterladen von Originalen, Standard überall erlaubt) | Neue Räume übernehmen die drei Rollenwerte |
| `account_preferences` (seit 25.10.2026) | PK (`account_id`, `key`), `value` (jsonb, `NULL` = erben), `team_mode` (`free`/`locked`, nur beim Hauptnutzer ausgewertet), `updated_at` | Ein Mitarbeiter erbt den Wert seines Hauptnutzers (`sponsor_id`) und kann ihn ändern. Ist die Einstellung für das Team verbindlich, gilt der Wert des Hauptnutzers; ein eigener Wert bleibt gespeichert und gilt wieder, sobald die Bindung endet. Erlaubte Werte und Standard je Schlüssel stehen in `app/preferences.py` |
| `data_rooms` (Spalte seit 23.10.2026) | `download_roles` (jsonb: `manager`, `member`, `reader`) | Regel je Raum, nur der Eigentümer ändert sie |
| `data_rooms` (Spalten seit 21.10.2026) | `owner_id` (Eigentümer; für persönliche Räume das Konto), `default_visibility` (`private`/`room`, Voreinstellung für neue Akten und Vorgänge) | Bestehende Räume `private`, Eigentümer aus `created_by` | | `data_rooms` (Spalten seit 21.10.2026) | `owner_id` (Eigentümer; für persönliche Räume das Konto), `default_visibility` (`private`/`room`, Voreinstellung für neue Akten und Vorgänge) | Bestehende Räume `private`, Eigentümer aus `created_by` |
| `room_shares` | `resource_kind` (`case`/`matter`), `resource_id`, `room_id`, `access` (`read`/`edit`), `created_by` | Freigabe einer Akte oder eines Vorgangs an einen Datenraum (Team); eindeutig je Eintrag und Raum. Endet, wenn der Freigebende den Raum verlässt oder Leser wird | | `room_shares` | `resource_kind` (`case`/`matter`), `resource_id`, `room_id`, `access` (`read`/`edit`), `created_by` | Freigabe einer Akte oder eines Vorgangs an einen Datenraum (Team); eindeutig je Eintrag und Raum. Endet, wenn der Freigebende den Raum verlässt oder Leser wird |
| `auth_login_attempts` | `bucket` (SHA-256 aus Adresse und E-Mail), `attempted_at` | zählt Fehlversuche; erfolgreicher Login löscht den Bucket | | `auth_login_attempts` | `bucket` (SHA-256 aus Adresse und E-Mail), `attempted_at` | zählt Fehlversuche; erfolgreicher Login löscht den Bucket |
+2 -2
View File
@@ -14,7 +14,7 @@ Der Datenraum bestimmt, **wo neue Objekte entstehen**. Er ist nicht der Rechtepf
## 2. Rechte und Antwortcodes ## 2. Rechte und Antwortcodes
Ein Ergebnis `Rights(read, edit, share, delete)` entsteht je Objekt und Handlungskontext. `_require` übersetzt: Ein Ergebnis `Rights(read, edit, share, delete, download)` entsteht je Objekt und Handlungskontext. `download` (Original speichern oder öffnen, Übergabepaket) hat der Owner immer; über einen Raum folgt es `data_rooms.download_roles` der Rolle, bei direkten Grants dem Wert `direct` im Konto des Owners. Es setzt Leserecht voraus, und eine Ausnahme, die das Lesen entzieht, entzieht auch das Herunterladen. Verweigert wird mit 403 und einem erklärenden Text; Lesen in der App bleibt möglich. `_require` übersetzt:
- `read` fehlt → **404** (das Objekt existiert für diese Person nicht) - `read` fehlt → **404** (das Objekt existiert für diese Person nicht)
- `read` vorhanden, verlangte Fähigkeit fehlt → **403** - `read` vorhanden, verlangte Fähigkeit fehlt → **403**
@@ -100,4 +100,4 @@ Danach Abzug der jeweiligen Ausnahme.
## 6. Bekannte Abweichungen ## 6. Bekannte Abweichungen
Siehe [Review-Befunde](../review/06_BEFUNDE.md): R-01 (Behälter löschen), R-02 (Detach/Anmerkung), R-07 (Vorgangsausnahme), R-11 (Übersicht/Suche), R-12 (Teilvorgangstiefe), R-15 (Fristen im Elternkontext). Siehe [Review-Befunde](../review/06_BEFUNDE.md): R-01 (Behälter löschen), R-02 (Detach/Anmerkung), R-07 (Vorgangsausnahme), R-12 (Teilvorgangstiefe), R-15 (Fristen im Elternkontext).
@@ -99,3 +99,20 @@ Die Konfiguration einer Instanz gehört dem Modul. Ein Modul deklariert `config_
Neuer Connector: Modul mit `config_fields` in `register_source` eintragen, UI-Modul mit `registerSourceModule` registrieren. Kein Eingriff in `ConnectorIn`, `Sources.jsx` oder die Fachseiten. Neuer Connector: Modul mit `config_fields` in `register_source` eintragen, UI-Modul mit `registerSourceModule` registrieren. Kein Eingriff in `ConnectorIn`, `Sources.jsx` oder die Fachseiten.
Nachweis und Grenzen: `backend/tests/test_connector_config.py` (Manifest, Geheimnis, Instanzerhalt, Fremdzugriff, Prüfstand) und `frontend/src/sources/config-core.test.mjs` (6 Node-Tests bestanden), Frontend-Build erfolgreich. Die Backend-Tests wurden in dieser Umgebung **nicht ausgeführt** (kein Python, keine PostgreSQL-Instanz). Keine Bedienprüfung im Browser. Zugangsdaten bleiben im Klartext (R-05); Archivmodus und DataPolicy sind nicht Teil dieses Schnitts. Nachweis und Grenzen: `backend/tests/test_connector_config.py` (Manifest, Geheimnis, Instanzerhalt, Fremdzugriff, Prüfstand) und `frontend/src/sources/config-core.test.mjs` (6 Node-Tests bestanden), Frontend-Build erfolgreich. Die Backend-Tests wurden in dieser Umgebung **nicht ausgeführt** (kein Python, keine PostgreSQL-Instanz). Keine Bedienprüfung im Browser. Zugangsdaten bleiben im Klartext (R-05); Archivmodus und DataPolicy sind nicht Teil dieses Schnitts.
## Paperless-Suche, Erweiterung am 24.10.2026
Auf Wunsch des Nutzers erweitert über den Umfang von Abschnitt C des Nacharbeitsauftrags hinaus: Suchart (Titel und Inhalt als Standard ohne Index, nur Titel, nur Inhalt, Volltextindex), Speicherpfad, Tag-Verknüpfung (alle, mindestens eines, keines), „ohne“ bei Korrespondent, Typ und Pfad, Hinzugefügt-Datum, Archivnummer und Zusatzfelder. Dazu eine Trefferkarte mit Vorschaubild, Textauszug, Tags, Typ und Seitenzahl sowie ein großer Dialog mit Filtern links und Treffern rechts. Zusatzfeld-Abfragen waren im Auftrag ausdrücklich kein Pflichtumfang und sind hier auf Nutzerwunsch dazugekommen.
**Verantwortung im Modul:** Die Filternamen eines Moduls deklariert `SourceModule.search_params` (Parametername auf Fähigkeitsflag). Die gemeinsame Suchroute kennt keinen Filternamen mehr. Sie reicht alle übrigen Abfrageparameter weiter und weist jeden ab, den das Modul nicht deklariert oder dessen Fähigkeit nicht wahr ist. Das gilt auch für unbekannte Parameter. Das Vorschaubild ist eine optionale Modulfähigkeit (`SourceModule.thumbnail`, Route `/api/v1/sources/{kind}/thumbnail`, Flag `search.result_preview`). Ein Modul ohne Deklaration lehnt es mit `422` ab.
**Verbleibende Quellkenntnis im allgemeinen Code (zulässig):** Die Kompatibilitätsrouten `/api/v1/sources/paperless/documents` und `.../preview` in `backend/app/api/sources.py` delegieren nur und tragen keine eigene Protokoll- oder Rechtelogik. Die Fähigkeiten von Paperless und IMAP stehen jetzt in `backend/app/sources.py` beim jeweiligen Modul; `backend/app/sourceport.py` enthält nur noch die Editorfähigkeiten und die Nutzlasten. Die unbenutzte Liste `UNSUPPORTED_SEARCH` und `unsupported_filters` sind entfernt. In der Oberfläche liegt der Paperless-Auswahldialog in `frontend/src/pages/Sources.jsx` und wird über `registerSourceModule` (`large: true`) eingehängt; Fachseiten und Modal kennen nur das allgemeine Flag.
| Nachweis | Ergebnis |
|---|---|
| Backend-Suite gegen `tsumugu_local_test` (eingebettete PostgreSQL, Wegwerf-Umgebung) | **164 Tests bestanden** |
| `test_connector_followup.py`: Parameterübertragung je Filter, Standardsuche, Textauszug, Namensauflösung, Zusatzfelder samt Ablehnung ungültiger Eingaben, fremder Filter und unbekannter Parameter `422`, Vorschaubild nur für Raster, fremde Instanz `404` | bestanden, gegen Attrappe |
| Bedienung im Browser (Chrome headless, Desktop 1440×900 und 390×844) gegen die echte Oberfläche, die echte API und einen nachgebildeten Paperless-HTTP-Server: Dialoggröße, Suche, Vorschaubilder, Textauszug, Ansicht in der Karte, unvollständige Zusatzfeld-Zeile meldet statt zu senden, Zusatzfelder gehen als `custom_field_query` hinaus, Blättern erhält die Auswahl, Einfügen, keine Konsolenfehler | 38 von 38 Prüfungen bestanden. Dabei ist ein Überlappen von Filtern und Treffern auf 390 px gefunden und behoben worden. |
**Grenze:** Keine der Prüfungen lief gegen die echte Paperless-Instanz. Die Parameternamen (`title_content`, `tags__id__none`, `storage_path__id`, `added__date__gte`, `archive_serial_number`, `custom_field_query`) stammen aus der Kenntnis der Paperless-ngx-API und stehen am Stellvertreter, nicht an der Nutzerinstanz belegt. Die Flags `custom_fields` und `result_preview` sind damit nur durch Code und Attrappe ausführbar belegt. Offen sind insbesondere der Wert eines Betragsfelds (reine Zahl ohne Währung), der Textauszug im Volltextindex und die Mindestversion für Zusatzfeld-Abfragen. Eine Prüfung an der echten Instanz braucht eine ausdrückliche Freigabe.
+4 -2
View File
@@ -16,7 +16,7 @@ Schweregrade siehe [01](01_METHODIK_UND_QUELLEN.md#schweregrade-der-befunde). Fu
| R-08 | Hoch | Auth | Session- und Login-Härtung unter dem Blueprint-Vertrag | | R-08 | Hoch | Auth | Session- und Login-Härtung unter dem Blueprint-Vertrag |
| R-09 | Mittel | Datenbank | Gemeinsame Zugangsdaten für Runtime und Migration, Migrator ohne Lock | | R-09 | Mittel | Datenbank | Gemeinsame Zugangsdaten für Runtime und Migration, Migrator ohne Lock |
| R-10 | Mittel | Quellen | Quelleninhalte werden über den Zugang des Betrachters geladen | | R-10 | Mittel | Quellen | Quelleninhalte werden über den Zugang des Betrachters geladen |
| R-11 | Hoch | Korrektheit | `LIMIT` vor der Rechteprüfung in Übersicht und Suche | | R-11 | Hoch | Korrektheit | `LIMIT` vor der Rechteprüfung in Übersicht und Suche (behoben am 23.10.2026) |
| R-12 | Mittel | Korrektheit | Verschieben erzeugt eine zweite Teilvorgangsebene | | R-12 | Mittel | Korrektheit | Verschieben erzeugt eine zweite Teilvorgangsebene |
| R-13 | Mittel | Export | Gelöschte oder unsichtbare Bezugsziele blockieren das Übergabepaket | | R-13 | Mittel | Export | Gelöschte oder unsichtbare Bezugsziele blockieren das Übergabepaket |
| R-14 | Mittel | Daten | Gelöschte Akte oder gelöschter Vorgang lässt Inhalte unauffindbar zurück | | R-14 | Mittel | Daten | Gelöschte Akte oder gelöschter Vorgang lässt Inhalte unauffindbar zurück |
@@ -133,7 +133,9 @@ Die Paperless-Vorschau lädt mit dem zuletzt angelegten Paperless-Zugang **der a
**Empfehlung:** Zugang fest an die Referenz binden (`connector_id` als Spalte, wie im Connectoren-Befund vorgeschlagen). Den Abruf über den Zugang des Element-Owners führen, nachdem das Leserecht im Kontext geprüft ist. Bis dahin die Grenze in der Oberfläche benennen. **Empfehlung:** Zugang fest an die Referenz binden (`connector_id` als Spalte, wie im Connectoren-Befund vorgeschlagen). Den Abruf über den Zugang des Element-Owners führen, nachdem das Leserecht im Kontext geprüft ist. Bis dahin die Grenze in der Oberfläche benennen.
## R-11 – `LIMIT` vor der Rechteprüfung (Hoch) ## R-11 – `LIMIT` vor der Rechteprüfung (Hoch) – behoben am 23.10.2026
**Stand:** Übersicht (offene Aufgaben und Fristen) und Suche lesen die Kandidaten jetzt in Blöcken von 200 in der Reihenfolge der Abfrage, prüfen je Zeile die Rechte und hören auf, sobald genug sichtbare Einträge da sind (`_scan` in `routes.py`). Fremde Zeilen verdrängen eigene nicht mehr. Die Arbeit ist bei sehr großen Beständen auf 20 000 gelesene Zeilen begrenzt; eine Vorfilterung im SQL und Cursor-Seiten bleiben eine spätere Optimierung. Test: `tests/test_views_scan.py`. Der Befund unten beschreibt den früheren Zustand.
**Fundstelle:** [routes.py:1328–1359](../../backend/app/api/routes.py#L1328-L1359), [routes.py:1412–1423](../../backend/app/api/routes.py#L1412-L1423) **Fundstelle:** [routes.py:1328–1359](../../backend/app/api/routes.py#L1328-L1359), [routes.py:1412–1423](../../backend/app/api/routes.py#L1412-L1423)
+1 -1
View File
@@ -52,7 +52,7 @@ Nichts davon verlangt einen Umbau. Die meisten Punkte lassen sich gezielt im Bes
| R-02 | Hoch | `DELETE /bindings/{id}` prüft die Zugehörigkeit der Einbindung zum Handlungskontext nicht | | R-02 | Hoch | `DELETE /bindings/{id}` prüft die Zugehörigkeit der Einbindung zum Handlungskontext nicht |
| R-03 | Hoch | E-Mail-Anlagen und Paperless-Bilder (auch SVG) werden `inline` aus dem App-Origin ausgeliefert | | R-03 | Hoch | E-Mail-Anlagen und Paperless-Bilder (auch SVG) werden `inline` aus dem App-Origin ausgeliefert |
| R-04 | Hoch | Betreiberkonto über eine öffentliche Web-Route, ohne MFA (Abweichung von ADR-05 und Auth-Blueprint 11.3) | | R-04 | Hoch | Betreiberkonto über eine öffentliche Web-Route, ohne MFA (Abweichung von ADR-05 und Auth-Blueprint 11.3) |
| R-11 | Hoch | Offene Aufgaben, Fristen und Suche: `LIMIT` vor der Rechteprüfung. Sichtbare Treffer fallen weg | | R-11 | Hoch | **Behoben am 23.10.2026.** Offene Aufgaben, Fristen und Suche lesen in Blöcken und prüfen die Rechte vor der Begrenzung (Vorfilterung im SQL bleibt eine spätere Optimierung) |
| R-08 | Hoch | Sessions 14 Tage ohne Leerlaufablauf und Rotation, Login-Begrenzung nur im Prozessspeicher, kein Passwortwechsel | | R-08 | Hoch | Sessions 14 Tage ohne Leerlaufablauf und Rotation, Login-Begrenzung nur im Prozessspeicher, kein Passwortwechsel |
| R-31 | Hoch | Hochgeladene Fotos behalten EXIF- und GPS-Metadaten, es gibt keine Normalisierung | | R-31 | Hoch | Hochgeladene Fotos behalten EXIF- und GPS-Metadaten, es gibt keine Normalisierung |
+28
View File
@@ -4,9 +4,35 @@ import { SessionContext } from "../session.js";
import { ErrorNote } from "../pages/widgets.jsx"; import { ErrorNote } from "../pages/widgets.jsx";
import { formatWhen } from "./admin-core.js"; import { formatWhen } from "./admin-core.js";
import { DownloadPolicy } from "./DownloadPolicy.jsx"; import { DownloadPolicy } from "./DownloadPolicy.jsx";
import { DuplicatePreference } from "./DuplicatePreference.jsx";
import { deviceClass, readImageChoice, writeImageChoice } from "../mail-images.js";
import { isEmployee } from "./admin-core.js"; import { isEmployee } from "./admin-core.js";
import { PasswordForm } from "./widgets.jsx"; import { PasswordForm } from "./widgets.jsx";
function MailImages() {
const [choice, setChoice] = useState(() => ({ desktop: readImageChoice("desktop"), touch: readImageChoice("touch") }));
const here = deviceClass();
const rows = [["desktop", "Am Computer"], ["touch", "Auf Handy und Tablet"]];
return (
<section className="card stack" aria-label="Bilder in E-Mails">
<h2>Bilder in E-Mails</h2>
<p className="muted">
Bilder liegen beim Absender. Beim Nachladen holt unser Server sie für dich (nur öffentliche https-Adressen, nur echte Bilddateien), dein Gerät nimmt keinen Kontakt auf.
Der Absender kann trotzdem erkennen, dass die Nachricht geöffnet wurde. Die Wahl gilt nur für diesen Browser; dieses Gerät zählt als „{here === "touch" ? "Handy und Tablet" : "Computer"}“.
</p>
{rows.map(([key, label]) => (
<label key={key}>
{label}
<select value={choice[key]} onChange={(event) => setChoice((current) => ({ ...current, [key]: writeImageChoice(key, event.target.value) }))}>
<option value="ask">Nur auf Klick nachladen</option>
<option value="auto">Automatisch nachladen</option>
</select>
</label>
))}
</section>
);
}
export function Account() { export function Account() {
const { session, logout } = useContext(SessionContext); const { session, logout } = useContext(SessionContext);
const [sessions, setSessions] = useState([]); const [sessions, setSessions] = useState([]);
@@ -56,6 +82,8 @@ export function Account() {
</header> </header>
<ErrorNote error={error} /> <ErrorNote error={error} />
{notice ? <p className="notice" role="status">{notice}</p> : null} {notice ? <p className="notice" role="status">{notice}</p> : null}
<MailImages />
<DuplicatePreference />
{isEmployee(session) ? null : ( {isEmployee(session) ? null : (
<section className="card stack" aria-label="Herunterladen"> <section className="card stack" aria-label="Herunterladen">
<h2>Herunterladen von Originalen</h2> <h2>Herunterladen von Originalen</h2>
@@ -0,0 +1,77 @@
import { useEffect, useState } from "react";
import { api } from "../api.js";
import { ErrorNote } from "../pages/widgets.jsx";
const PATH = "/api/v1/auth/preferences/duplicate_check";
export const DUPLICATE_LEVELS = {
off: "Keine Prüfung",
here: "Nur hier (direkt in diesem Vorgang oder dieser Akte)",
inside: "Hier und was darin angezeigt wird (Teilvorgänge, zugeordnete Vorgänge)",
around: "Zusätzlich dort, wo es liegt (übergeordneter Vorgang, zugeordnete Akten)",
};
// Where a document that is inserted again counts as already present. Only a warning, never a block.
// An employee inherits the value of the main user and may change it unless the main user made it binding.
export function DuplicatePreference() {
const [state, setState] = useState(null);
const [error, setError] = useState("");
const [notice, setNotice] = useState("");
useEffect(() => {
let alive = true;
api(PATH).then((data) => alive && setState(data)).catch((err) => alive && setError(err.message));
return () => {
alive = false;
};
}, []);
async function save(body) {
setError("");
setNotice("");
try {
setState(await api(PATH, { method: "PUT", body }));
setNotice("Die Einstellung ist gespeichert.");
} catch (err) {
setError(err.message);
}
}
if (!state) return error ? <ErrorNote error={error} /> : null;
const ownChoice = state.can_lock ? state.own || state.default : state.own || "";
return (
<section className="card stack" aria-label="Doppelte Dokumente">
<h2>Doppelte Dokumente</h2>
<p className="muted">
Beim Einfügen eines Dokuments aus einer Quelle warnt Tsumugu, wenn es an der gewählten Stelle schon vorhanden ist. Du kannst es trotzdem
einfügen oder das vorhandene Element zusätzlich einbinden. Dasselbe Dokument in verschiedenen Vorgängen bleibt immer erlaubt.
</p>
<label>
Prüfen in
<select
value={ownChoice}
disabled={state.locked}
onChange={(event) => save({ value: event.target.value || null })}
>
{state.can_lock ? null : <option value="">Wie beim Hauptnutzer ({DUPLICATE_LEVELS[state.inherited]})</option>}
{state.allowed.map((level) => (
<option key={level} value={level}>{DUPLICATE_LEVELS[level]}</option>
))}
</select>
</label>
{state.locked ? <p className="muted">Der Hauptnutzer hat diese Einstellung für das Team vorgegeben: {DUPLICATE_LEVELS[state.value]}.</p> : null}
{state.can_lock ? (
<label className="check">
<input
type="checkbox"
checked={state.team_mode === "locked"}
onChange={(event) => save({ value: state.own, team_mode: event.target.checked ? "locked" : "free" })}
/>
Für Mitarbeiter verbindlich
</label>
) : null}
<ErrorNote error={error} />
{notice ? <p className="notice" role="status">{notice}</p> : null}
</section>
);
}
+12
View File
@@ -153,3 +153,15 @@ body.is-resizing .library__resizer { background: color-mix(in srgb, var(--accent
.mail-frame { display: block; width: 100%; height: 60vh; min-height: 260px; border: 1px solid var(--border); border-radius: 8px; background: #fff; } .mail-frame { display: block; width: 100%; height: 60vh; min-height: 260px; border: 1px solid var(--border); border-radius: 8px; background: #fff; }
.mail-reading .segmented { display: inline-flex; gap: 6px; margin-bottom: 8px; } .mail-reading .segmented { display: inline-flex; gap: 6px; margin-bottom: 8px; }
/* Reader header on the phone: back link and tools on one line, a modest title below */
@media (max-width: 759px) {
.reader-pane__bar { padding: 8px 12px 4px; gap: 2px 8px; flex-wrap: wrap; }
.reader-pane__bar .reader-back { order: 1; }
.reader-pane__bar .reader-pane__tools { order: 2; margin-left: auto; }
.reader-pane__bar .reader-pane__heading { order: 3; flex: 1 1 100%; }
.reader-pane__heading h2 { font-size: 17px; line-height: 1.25; display: -webkit-box; -webkit-line-clamp: 2; -webkit-box-orient: vertical; overflow: hidden; }
.reader-pane__heading .direction-column__icon { width: 18px; height: 18px; }
.reader-pane__heading .reader-facts { margin: 2px 0 0; font-size: 12px; }
}
.mail-images { display: grid; gap: 4px; margin: 4px 0; }
+30
View File
@@ -0,0 +1,30 @@
// Pictures in a mail come from the sender's server. The app loads them on request, through its own server.
// The choice lives in the browser: one value for pointer devices (computer) and one for touch devices (phone, tablet).
export const IMAGE_KEYS = { desktop: "tsumugu-mail-images-desktop", touch: "tsumugu-mail-images-touch" };
export function deviceClass(match = typeof window === "undefined" ? null : window.matchMedia) {
try {
return match && match.call(typeof window === "undefined" ? undefined : window, "(pointer: coarse)").matches ? "touch" : "desktop";
} catch {
return "desktop";
}
}
export function readImageChoice(device, storage = typeof window === "undefined" ? null : window.localStorage) {
try {
return storage?.getItem(IMAGE_KEYS[device]) === "auto" ? "auto" : "ask";
} catch {
return "ask";
}
}
export function writeImageChoice(device, choice, storage = typeof window === "undefined" ? null : window.localStorage) {
const value = choice === "auto" ? "auto" : "ask";
try {
storage?.setItem(IMAGE_KEYS[device], value);
} catch {
/* The choice then only lasts for this visit. */
}
return value;
}
+28
View File
@@ -0,0 +1,28 @@
import assert from "node:assert/strict";
import test from "node:test";
import { IMAGE_KEYS, deviceClass, readImageChoice, writeImageChoice } from "./mail-images.js";
function memory() {
const data = {};
return { getItem: (key) => (key in data ? data[key] : null), setItem: (key, value) => { data[key] = value; } };
}
test("pictures are asked for by default and the choice is kept per device kind", () => {
const store = memory();
assert.equal(readImageChoice("touch", store), "ask");
assert.equal(writeImageChoice("desktop", "auto", store), "auto");
assert.equal(readImageChoice("desktop", store), "auto");
assert.equal(readImageChoice("touch", store), "ask");
assert.equal(writeImageChoice("touch", "anything", store), "ask");
assert.ok(IMAGE_KEYS.desktop !== IMAGE_KEYS.touch);
});
test("a blocked store and a missing media query fall back to the safe choice", () => {
const broken = { getItem() { throw new Error("blocked"); }, setItem() { throw new Error("blocked"); } };
assert.equal(readImageChoice("desktop", broken), "ask");
assert.equal(writeImageChoice("desktop", "auto", broken), "auto");
assert.equal(readImageChoice("desktop", null), "ask");
assert.equal(deviceClass(null), "desktop");
assert.equal(deviceClass(function () { return { matches: true }; }), "touch");
assert.equal(deviceClass(function () { throw new Error("x"); }), "desktop");
});
+36 -4
View File
@@ -1,4 +1,4 @@
import { useState } from "react"; import { useEffect, useState } from "react";
// The cleaned HTML of a mail is shown in a frame without scripts and without any reach into the app: // The cleaned HTML of a mail is shown in a frame without scripts and without any reach into the app:
// an empty `sandbox` gives it its own origin, and the policy inside allows no loads except inline images. // an empty `sandbox` gives it its own origin, and the policy inside allows no loads except inline images.
@@ -12,9 +12,33 @@ export function mailDocument(html) {
+ `</head><body>${html}</body></html>`; + `</head><body>${html}</body></html>`;
} }
export function MailBody({ html, text, truncated, notices = [] }) { // `remote` describes the pictures of other servers: { key, count, auto, load }. `load` asks the app's server for them.
export function MailBody({ html, text, truncated, notices = [], remote = null }) {
const [view, setView] = useState(html ? "html" : "text"); const [view, setView] = useState(html ? "html" : "text");
const [withImages, setWithImages] = useState(null);
const [busy, setBusy] = useState(false);
const [problem, setProblem] = useState("");
const mode = html ? view : "text"; const mode = html ? view : "text";
const shownHtml = withImages ? withImages.html : html;
async function loadImages() {
setBusy(true);
setProblem("");
try {
setWithImages(await remote.load());
} catch (err) {
setProblem(err.message || "Die Bilder konnten nicht geladen werden.");
} finally {
setBusy(false);
}
}
useEffect(() => {
setWithImages(null);
setProblem("");
if (remote?.auto && remote.count > 0) loadImages();
}, [remote?.key]);
return ( return (
<div className="mail-reading"> <div className="mail-reading">
{html ? ( {html ? (
@@ -25,8 +49,16 @@ export function MailBody({ html, text, truncated, notices = [] }) {
) : null} ) : null}
{mode === "html" ? ( {mode === "html" ? (
<> <>
<iframe className="mail-frame" title="Nachricht" sandbox="allow-popups allow-popups-to-escape-sandbox" referrerPolicy="no-referrer" srcDoc={mailDocument(html)} /> {remote && remote.count > 0 && !withImages ? (
<p className="muted">Aus Sicherheitsgründen sind Skripte, eingebettete Formulare und Bilder von fremden Servern entfernt. Links öffnen externe Seiten: nur bekannten Absendern vertrauen.</p> <p className="mail-images">
<button type="button" className="quiet" disabled={busy} onClick={loadImages}>{busy ? "Bilder werden geladen …" : `Bilder nachladen (${remote.count})`}</button>
<span className="muted">Unsere Server laden die Bilder für dich, der Absender sieht deine Adresse nicht. Er kann aber erkennen, dass die Nachricht geöffnet wurde.</span>
</p>
) : null}
{withImages ? <p className="muted">{withImages.loaded} Bild(er) geladen{withImages.failed ? `, ${withImages.failed} nicht verfügbar oder nicht sicher` : ""}.</p> : null}
{problem ? <p className="error">{problem}</p> : null}
<iframe className="mail-frame" title="Nachricht" sandbox="allow-popups allow-popups-to-escape-sandbox" referrerPolicy="no-referrer" srcDoc={mailDocument(shownHtml)} />
<p className="muted">Aus Sicherheitsgründen sind Skripte und eingebettete Formulare entfernt, Bilder von fremden Servern erst nach deiner Freigabe sichtbar. Links öffnen externe Seiten: nur bekannten Absendern vertrauen.</p>
</> </>
) : ( ) : (
<> <>
+23 -18
View File
@@ -15,6 +15,7 @@ import { Workspace, rememberScroll, restoreScroll, useWorkspaceLayout } from "./
import { useLibraryFocus } from "../library/LibraryLayout.jsx"; import { useLibraryFocus } from "../library/LibraryLayout.jsx";
import { outlineSignature } from "../library/library-core.js"; import { outlineSignature } from "../library/library-core.js";
import { NavIcon } from "./icons.jsx"; import { NavIcon } from "./icons.jsx";
import { useExternalPlacement } from "../sources/duplicates.jsx";
function matterOrigin(group) { function matterOrigin(group) {
const kind = group.kind_label === "Teilvorgang" ? "Teilvorgang" : "Vorgang"; const kind = group.kind_label === "Teilvorgang" ? "Teilvorgang" : "Vorgang";
@@ -389,7 +390,7 @@ export function CaseDetail() {
<Modal open={modal === "file"} title="Datei" onClose={() => setModal("")}> <Modal open={modal === "file"} title="Datei" onClose={() => setModal("")}>
<FileForm context={context} chapters={record.chapters} initialChapter={addChapterId} targetLabel={targetLabel} onChange={(finished = true) => { load(); if (finished) setModal(""); }} /> <FileForm context={context} chapters={record.chapters} initialChapter={addChapterId} targetLabel={targetLabel} onChange={(finished = true) => { load(); if (finished) setModal(""); }} />
</Modal> </Modal>
<Modal open={modal.startsWith("source:")} title={moduleFor(modal.slice(7))?.title || "Quelle"} wide={Boolean(moduleFor(modal.slice(7))?.wide)} split={Boolean(moduleFor(modal.slice(7))?.wide)} onClose={() => setModal("")}> <Modal open={modal.startsWith("source:")} title={moduleFor(modal.slice(7))?.title || "Quelle"} wide={Boolean(moduleFor(modal.slice(7))?.wide)} split={Boolean(moduleFor(modal.slice(7))?.wide)} large={Boolean(moduleFor(modal.slice(7))?.large)} onClose={() => setModal("")}>
<SourcePicker sourceKind={modal.slice(7)} context={context} chapterId={addChapterId} chapterTitle={addChapterId ? chapterPathLabel(record.chapters, addChapterId) : ""} targetLabel={targetLabel} onInserted={load} onDone={() => { setModal(""); load(); }} /> <SourcePicker sourceKind={modal.slice(7)} context={context} chapterId={addChapterId} chapterTitle={addChapterId ? chapterPathLabel(record.chapters, addChapterId) : ""} targetLabel={targetLabel} onInserted={load} onDone={() => { setModal(""); load(); }} />
</Modal> </Modal>
<Modal open={modal === "subcase"} title="Unterakte anlegen" onClose={() => setModal("")}> <Modal open={modal === "subcase"} title="Unterakte anlegen" onClose={() => setModal("")}>
@@ -905,6 +906,15 @@ function ExternalForm({ context, placement, chapters = [], initialChapter = "",
const [sentOn, setSentOn] = useState(""); const [sentOn, setSentOn] = useState("");
const [receivedOn, setReceivedOn] = useState(""); const [receivedOn, setReceivedOn] = useState("");
const [error, setError] = useState(""); const [error, setError] = useState("");
const { place, panel } = useExternalPlacement({
context,
placement,
onPlaced: () => {
setTitle("");
setExternalId("");
onChange();
},
});
useEffect(() => { setChapterId(initialChapter || ""); }, [initialChapter]); useEffect(() => { setChapterId(initialChapter || ""); }, [initialChapter]);
@@ -912,25 +922,19 @@ function ExternalForm({ context, placement, chapters = [], initialChapter = "",
event.preventDefault(); event.preventDefault();
setError(""); setError("");
try { try {
await api("/api/v1/external-refs", { await place({
method: "POST", title,
body: { source,
title, external_id: externalId,
source, context,
external_id: externalId, ...(placement ? { placement } : {}),
context, ...(chapterId ? { chapter_id: chapterId } : {}),
...(placement ? { placement } : {}), dates: {
...(chapterId ? { chapter_id: chapterId } : {}), document_on: documentOn || null,
dates: { sent_on: sentOn || null,
document_on: documentOn || null, received_on: receivedOn || null,
sent_on: sentOn || null,
received_on: receivedOn || null,
},
}, },
}); });
setTitle("");
setExternalId("");
onChange();
} catch (err) { } catch (err) {
setError(err.message); setError(err.message);
} }
@@ -951,6 +955,7 @@ function ExternalForm({ context, placement, chapters = [], initialChapter = "",
{chapterSelect(chapters, chapterId, setChapterId)} {chapterSelect(chapters, chapterId, setChapterId)}
<button type="submit">Referenz erfassen</button> <button type="submit">Referenz erfassen</button>
<ErrorNote error={error} /> <ErrorNote error={error} />
{panel}
</form> </form>
); );
} }
+1 -1
View File
@@ -413,7 +413,7 @@ export function MatterDetail() {
<Modal open={modal === "file"} title="Datei" onClose={() => setModal("")}> <Modal open={modal === "file"} title="Datei" onClose={() => setModal("")}>
<FileForm context={rightsContext} placement={placement} chapters={record.chapters} initialChapter={addChapterId} targetLabel={targetLabel} onChange={(finished = true) => { load(); if (finished) setModal(""); }} /> <FileForm context={rightsContext} placement={placement} chapters={record.chapters} initialChapter={addChapterId} targetLabel={targetLabel} onChange={(finished = true) => { load(); if (finished) setModal(""); }} />
</Modal> </Modal>
<Modal open={modal.startsWith("source:")} title={moduleFor(modal.slice(7))?.title || "Quelle"} wide={Boolean(moduleFor(modal.slice(7))?.wide)} split={Boolean(moduleFor(modal.slice(7))?.wide)} onClose={() => setModal("")}> <Modal open={modal.startsWith("source:")} title={moduleFor(modal.slice(7))?.title || "Quelle"} wide={Boolean(moduleFor(modal.slice(7))?.wide)} split={Boolean(moduleFor(modal.slice(7))?.wide)} large={Boolean(moduleFor(modal.slice(7))?.large)} onClose={() => setModal("")}>
<SourcePicker sourceKind={modal.slice(7)} context={rightsContext} placement={placement} chapterId={addChapterId} chapterTitle={addChapterId ? chapterPathLabel(record.chapters, addChapterId) : ""} targetLabel={targetLabel} onInserted={load} onDone={() => { setModal(""); load(); }} /> <SourcePicker sourceKind={modal.slice(7)} context={rightsContext} placement={placement} chapterId={addChapterId} chapterTitle={addChapterId ? chapterPathLabel(record.chapters, addChapterId) : ""} targetLabel={targetLabel} onInserted={load} onDone={() => { setModal(""); load(); }} />
</Modal> </Modal>
<Modal open={modal === "submatter"} title="Teilvorgang anlegen" onClose={() => setModal("")}> <Modal open={modal === "submatter"} title="Teilvorgang anlegen" onClose={() => setModal("")}>
+2 -2
View File
@@ -1,7 +1,7 @@
import { useEffect, useRef } from "react"; import { useEffect, useRef } from "react";
import { isDirty, markDirty } from "../api.js"; import { isDirty, markDirty } from "../api.js";
export function Modal({ open, title, onClose, wide = false, split = false, children }) { export function Modal({ open, title, onClose, wide = false, split = false, large = false, children }) {
const dialogRef = useRef(null); const dialogRef = useRef(null);
const restoreRef = useRef(null); const restoreRef = useRef(null);
@@ -33,7 +33,7 @@ export function Modal({ open, title, onClose, wide = false, split = false, child
if (!open) return null; if (!open) return null;
return ( return (
<div className="modal-overlay" onClick={(event) => { if (event.target === event.currentTarget) requestClose(); }}> <div className="modal-overlay" onClick={(event) => { if (event.target === event.currentTarget) requestClose(); }}>
<div ref={dialogRef} className={`modal${wide ? " modal--wide" : ""}${split ? " modal--split" : ""}`} role="dialog" aria-modal="true" aria-label={title} tabIndex={-1}> <div ref={dialogRef} className={`modal${wide ? " modal--wide" : ""}${split ? " modal--split" : ""}${large ? " modal--large" : ""}`} role="dialog" aria-modal="true" aria-label={title} tabIndex={-1}>
<header className="modal__bar"> <header className="modal__bar">
<h2>{title}</h2> <h2>{title}</h2>
<button type="button" className="quiet" onClick={requestClose}>Schließen</button> <button type="button" className="quiet" onClick={requestClose}>Schließen</button>
+609 -118
View File
@@ -5,9 +5,10 @@ import { api, fetchBlob } from "../api.js";
import { MailBody } from "../mail-view.jsx"; import { MailBody } from "../mail-view.jsx";
import { NewConnector, useSourceModules } from "../sources/connector-config.jsx"; import { NewConnector, useSourceModules } from "../sources/connector-config.jsx";
import { SourceFrame, registerSourceModule } from "../sources/registry.jsx"; import { SourceFrame, registerSourceModule } from "../sources/registry.jsx";
import { PresentNote, bindable, describeMatch, lookupPresent, presentKey, useExternalPlacement } from "../sources/duplicates.jsx";
const SEARCH_LIMITS = { const SEARCH_LIMITS = {
paperless: "Suchtext gilt für Titel und Dokumentinhalt. Zusätzlich gibt es Titel, Korrespondent, Dokumenttyp, Tags, Dokumentdatum und Sortierung.", paperless: "Die Suche läuft in Paperless selbst, über Titel und Inhalt, den Titel allein, den Inhalt allein oder den Volltextindex. Filter nach Korrespondent, Dokumenttyp, Speicherpfad, Tags, Dokumentdatum, Hinzugefügt-Datum und Archivnummer grenzen sie ein.",
imap: "Die Suche gilt für das ganze gewählte Fach, auch über die erste Seite hinaus. Betreff, Absender und Nachrichtentext werden auf dem Server geprüft.", imap: "Die Suche gilt für das ganze gewählte Fach, auch über die erste Seite hinaus. Betreff, Absender und Nachrichtentext werden auf dem Server geprüft.",
}; };
import { formatDate } from "./content.jsx"; import { formatDate } from "./content.jsx";
@@ -69,28 +70,250 @@ function paperlessLabel(item) {
return dated !== "Ohne Datum" ? `${dated} · ${item.title}` : item.title; return dated !== "Ohne Datum" ? `${dated} · ${item.title}` : item.title;
} }
function paperlessFacts(item) {
return [
formatDate(item.document_on) !== "Ohne Datum" ? formatDate(item.document_on) : "",
item.correspondent,
item.document_type,
item.pages ? `${item.pages} ${item.pages === 1 ? "Seite" : "Seiten"}` : "",
item.asn ? `ASN ${item.asn}` : "",
].filter(Boolean);
}
const PAPERLESS_MODES = [
["title_content", "Titel und Inhalt"],
["title", "Nur Titel"],
["content", "Nur Inhalt"],
["index", "Volltextindex"],
];
const PAPERLESS_MODE_HINTS = {
title_content: "Sucht den Text im Titel und im erkannten Inhalt, wie „Titel & Inhalt“ in Paperless.",
title: "Sucht den Text nur im Titel.",
content: "Sucht den Text nur im erkannten Inhalt der Dokumente.",
index: "Der Volltextindex von Paperless findet ganze Wörter, auch mit AND, OR und Anführungszeichen, und ordnet nach Relevanz. Ein neues Dokument erscheint erst, wenn Paperless es indexiert hat.",
};
const NO_FACETS = { correspondents: [], document_types: [], storage_paths: [], tags: [], custom_fields: [] };
const EMPTY_FILTERS = { const EMPTY_FILTERS = {
query: "", query: "",
mode: "title_content",
title: "", title: "",
correspondent: "", correspondent: "",
documentType: "", documentType: "",
storagePath: "",
tags: [], tags: [],
tagMode: "all",
documentFrom: "", documentFrom: "",
documentTo: "", documentTo: "",
addedFrom: "",
addedTo: "",
asn: "",
custom: [],
sort: "-created", sort: "-created",
}; };
function activeFilterCount(filters) {
return [
filters.title.trim(),
filters.correspondent,
filters.documentType,
filters.storagePath,
filters.tags.length ? "tags" : "",
filters.documentFrom,
filters.documentTo,
filters.addedFrom,
filters.addedTo,
filters.asn.trim(),
].filter(Boolean).length + customPayload(filters.custom).length;
}
const CUSTOM_KINDS = {
string: "text",
url: "text",
longtext: "text",
integer: "number",
float: "number",
monetary: "number",
date: "date",
boolean: "boolean",
select: "select",
};
const CUSTOM_OPERATORS = {
text: [["icontains", "enthält"], ["exact", "ist gleich"], ["istartswith", "beginnt mit"], ["iendswith", "endet mit"]],
number: [["exact", "gleich"], ["gte", "mindestens"], ["lte", "höchstens"], ["gt", "größer als"], ["lt", "kleiner als"], ["range", "zwischen"]],
date: [["exact", "am"], ["gte", "ab"], ["lte", "bis"], ["gt", "nach"], ["lt", "vor"], ["range", "zwischen"]],
boolean: [["exact", "ist"]],
select: [["exact", "ist"]],
};
const CUSTOM_PRESENCE = [["set", "hat einen Wert"], ["unset", "ist leer"]];
const CUSTOM_LIMIT = 5;
function customKind(field) {
return CUSTOM_KINDS[field?.type] || "text";
}
// A row that is not complete is reported, not left out of the search.
function customProblem(rows) {
for (const row of rows) {
const name = row.field ? "Eine Zusatzfeld-Bedingung" : "Eine Zusatzfeld-Bedingung ohne Feld";
if (!row.field) return `${name} ist unvollständig. Wähle ein Feld oder entferne die Zeile.`;
const presence = row.op === "set" || row.op === "unset";
if (presence || row.kind === "boolean") continue;
if (row.op === "range" ? !(row.value && row.value2) : !String(row.value).trim()) {
return `${name} ist unvollständig. Gib den Wert an oder entferne die Zeile.`;
}
}
return "";
}
function customPayload(rows) {
const payload = [];
for (const row of rows) {
if (!row.field) continue;
if (row.op === "set" || row.op === "unset") {
payload.push({ field: row.field, op: "exists", value: row.op === "set" });
} else if (row.op === "range") {
if (row.value && row.value2) payload.push({ field: row.field, op: "range", value: [row.value, row.value2] });
} else if (row.kind === "boolean") {
payload.push({ field: row.field, op: "exact", value: row.value !== "false" });
} else if (String(row.value).trim()) {
payload.push({ field: row.field, op: row.op, value: String(row.value).trim() });
}
}
return payload;
}
function CustomFieldFilters({ fields, rows, onChange }) {
if (!fields.length) return null;
const byId = new Map(fields.map((item) => [item.id, item]));
function update(key, patch) {
onChange(rows.map((row) => (row.key === key ? { ...row, ...patch } : row)));
}
function choose(key, id) {
const field = byId.get(id);
const kind = customKind(field);
const first = field ? CUSTOM_OPERATORS[kind][0][0] : "";
update(key, { field: id, kind, op: first, value: kind === "boolean" ? "true" : "", value2: "" });
}
function add() {
onChange([...rows, { key: `${Date.now()}-${rows.length}`, field: "", kind: "text", op: "", value: "", value2: "" }]);
}
return (
<fieldset className="paperless-custom">
<legend>Zusatzfelder, alle Bedingungen zugleich</legend>
{rows.map((row) => {
const field = byId.get(row.field);
const kind = customKind(field);
const value = (name, label) => {
if (kind === "boolean") {
return (
<select aria-label={label} value={row.value} onChange={(event) => update(row.key, { [name]: event.target.value })}>
<option value="true">Ja</option>
<option value="false">Nein</option>
</select>
);
}
if (kind === "select") {
return (
<select aria-label={label} value={row.value} onChange={(event) => update(row.key, { [name]: event.target.value })}>
<option value="">Wert wählen</option>
{(field.options || []).map((option) => <option key={option.id} value={option.id}>{option.name}</option>)}
</select>
);
}
return (
<input
aria-label={label}
type={kind === "date" ? "date" : "text"}
inputMode={kind === "number" ? "decimal" : undefined}
value={name === "value" ? row.value : row.value2}
onChange={(event) => update(row.key, { [name]: event.target.value })}
/>
);
};
const presence = row.op === "set" || row.op === "unset";
return (
<div key={row.key} className="paperless-custom__row">
<select aria-label="Zusatzfeld" value={row.field} onChange={(event) => choose(row.key, event.target.value)}>
<option value="">Feld wählen</option>
{fields.map((item) => <option key={item.id} value={item.id}>{item.name}</option>)}
</select>
{field ? (
<select aria-label="Bedingung" value={row.op} onChange={(event) => update(row.key, { op: event.target.value })}>
{CUSTOM_OPERATORS[kind].map(([op, label]) => <option key={op} value={op}>{label}</option>)}
{CUSTOM_PRESENCE.map(([op, label]) => <option key={op} value={op}>{label}</option>)}
</select>
) : null}
{field && !presence ? value("value", row.op === "range" ? "Von" : "Wert") : null}
{field && row.op === "range" ? value("value2", "Bis") : null}
<button type="button" className="quiet" onClick={() => onChange(rows.filter((item) => item.key !== row.key))}>Entfernen</button>
</div>
);
})}
{rows.length < CUSTOM_LIMIT ? <button type="button" className="quiet" onClick={add}>Zusatzfeld-Bedingung hinzufügen</button> : null}
</fieldset>
);
}
// The thumbnail comes through the server like every other Paperless call, so the key stays there.
function PaperlessThumb({ connectorId, id }) {
const [url, setUrl] = useState("");
const [failed, setFailed] = useState("");
useEffect(() => {
let alive = true;
let made = "";
setUrl("");
setFailed("");
fetchBlob(`/api/v1/sources/paperless/thumbnail?connector_id=${connectorId}&item=${id}`)
.then((blob) => {
if (!alive) return;
made = URL.createObjectURL(blob);
setUrl(made);
})
.catch((err) => {
if (alive) setFailed(err.message || "Das Vorschaubild ist nicht verfügbar.");
});
return () => {
alive = false;
if (made) URL.revokeObjectURL(made);
};
}, [connectorId, id]);
if (url) return <img src={url} alt="" className="paperless-thumb__image" />;
return <span className="paperless-thumb__empty" title={failed || undefined}>{failed ? "Kein Bild" : ""}</span>;
}
export function PaperlessPicker({ context, placement, chapterId = "", chapterTitle = "", targetLabel = "", onDone, onInserted }) { export function PaperlessPicker({ context, placement, chapterId = "", chapterTitle = "", targetLabel = "", onDone, onInserted }) {
const [filters, setFilters] = useState(EMPTY_FILTERS); const [filters, setFilters] = useState(EMPTY_FILTERS);
const [items, setItems] = useState([]); const [items, setItems] = useState([]);
const [connectors, setConnectors] = useState([]); const [connectors, setConnectors] = useState([]);
const [connectorId, setConnectorId] = useState(""); const [connectorId, setConnectorId] = useState("");
const [facets, setFacets] = useState({ correspondents: [], document_types: [], tags: [] }); const [facets, setFacets] = useState(NO_FACETS);
const [page, setPage] = useState(1);
const [nextPage, setNextPage] = useState(null); const [nextPage, setNextPage] = useState(null);
const [previousPage, setPreviousPage] = useState(null);
const [total, setTotal] = useState(null);
const [searched, setSearched] = useState(false);
const [picked, setPicked] = useState(() => new Map()); const [picked, setPicked] = useState(() => new Map());
// Documents that are already present around the target, by presentKey. `review` holds the choices while the user decides.
const [present, setPresent] = useState(() => new Map());
const [review, setReview] = useState(null);
const [previewKey, setPreviewKey] = useState("");
const [previewUrl, setPreviewUrl] = useState(""); const [previewUrl, setPreviewUrl] = useState("");
const [pending, setPending] = useState(false); const [pending, setPending] = useState(false);
const [error, setError] = useState(""); const [error, setError] = useState("");
// The filters sit beside the hits on a wide screen. On a phone they start folded away.
const [tagSearch, setTagSearch] = useState("");
const [moreOpen, setMoreOpen] = useState(() => typeof window === "undefined" || !window.matchMedia || window.matchMedia("(min-width: 900px)").matches);
const searchRef = useRef(0); const searchRef = useRef(0);
const previewRef = useRef(0); const previewRef = useRef(0);
const facetRef = useRef(0); const facetRef = useRef(0);
@@ -107,7 +330,7 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
useEffect(() => { useEffect(() => {
if (!connectorId) { if (!connectorId) {
setFacets({ correspondents: [], document_types: [], tags: [] }); setFacets(NO_FACETS);
return undefined; return undefined;
} }
const token = facetRef.current + 1; const token = facetRef.current + 1;
@@ -115,7 +338,7 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
let alive = true; let alive = true;
api(`/api/v1/sources/paperless/facets?connector_id=${connectorId}`) api(`/api/v1/sources/paperless/facets?connector_id=${connectorId}`)
.then((data) => { .then((data) => {
if (alive && token === facetRef.current) setFacets(data); if (alive && token === facetRef.current) setFacets({ ...NO_FACETS, ...data });
}) })
.catch((err) => { .catch((err) => {
if (alive && token === facetRef.current) setError(err.message); if (alive && token === facetRef.current) setError(err.message);
@@ -129,57 +352,112 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
if (previewUrl) URL.revokeObjectURL(previewUrl); if (previewUrl) URL.revokeObjectURL(previewUrl);
}, [previewUrl]); }, [previewUrl]);
function paramsFor(page) { function paramsFor(pageNumber) {
const params = new URLSearchParams(); const params = new URLSearchParams();
const text = filters.query.trim();
if (connectorId) params.set("connector_id", connectorId); if (connectorId) params.set("connector_id", connectorId);
if (filters.query.trim()) params.set("q", filters.query.trim()); if (text) params.set("q", text);
if (text && filters.mode !== "title_content") params.set("mode", filters.mode);
if (filters.title.trim()) params.set("title", filters.title.trim()); if (filters.title.trim()) params.set("title", filters.title.trim());
if (filters.correspondent) params.set("correspondent", filters.correspondent); if (filters.correspondent) params.set("correspondent", filters.correspondent);
if (filters.documentType) params.set("document_type", filters.documentType); if (filters.documentType) params.set("document_type", filters.documentType);
if (filters.tags.length) params.set("tag", filters.tags.join(",")); if (filters.storagePath) params.set("storage_path", filters.storagePath);
if (filters.tags.length) {
params.set("tag", filters.tags.join(","));
if (filters.tagMode !== "all") params.set("tag_mode", filters.tagMode);
}
if (filters.documentFrom) params.set("document_from", filters.documentFrom); if (filters.documentFrom) params.set("document_from", filters.documentFrom);
if (filters.documentTo) params.set("document_to", filters.documentTo); if (filters.documentTo) params.set("document_to", filters.documentTo);
if (filters.addedFrom) params.set("added_from", filters.addedFrom);
if (filters.addedTo) params.set("added_to", filters.addedTo);
if (filters.asn.trim()) params.set("asn", filters.asn.trim());
const custom = customPayload(filters.custom);
if (custom.length) params.set("custom", JSON.stringify(custom));
if (filters.sort === "-score") { if (filters.sort === "-score") {
if (filters.query.trim()) params.set("sort", "-score"); if (text && filters.mode === "index") params.set("sort", "-score");
} else if (filters.sort) { } else if (filters.sort) {
params.set("sort", filters.sort); params.set("sort", filters.sort);
} }
if (page) params.set("cursor", String(page)); if (pageNumber) params.set("cursor", String(pageNumber));
return params; return params;
} }
function closePreview() {
previewRef.current += 1;
if (previewUrl) URL.revokeObjectURL(previewUrl);
setPreviewUrl("");
setPreviewKey("");
}
function invalidate() { function invalidate() {
searchRef.current += 1; searchRef.current += 1;
previewRef.current += 1; previewRef.current += 1;
facetRef.current += 1; facetRef.current += 1;
} }
async function runSearch(event, page = "") { async function runSearch(event, wanted = "") {
event?.preventDefault(); event?.preventDefault();
setError(""); setError("");
if (connectors.length > 1 && !connectorId) { if (connectors.length > 1 && !connectorId) {
setError("Wähle zuerst den Paperless-Zugang."); setError("Wähle zuerst den Paperless-Zugang.");
return; return;
} }
const problem = customProblem(filters.custom);
if (problem) {
setError(problem);
return;
}
const token = searchRef.current + 1; const token = searchRef.current + 1;
searchRef.current = token; searchRef.current = token;
const asked = connectorId; const asked = connectorId;
try { try {
const data = await api(`/api/v1/sources/paperless/documents?${paramsFor(page)}`); const data = await api(`/api/v1/sources/paperless/documents?${paramsFor(wanted)}`);
if (token !== searchRef.current) return; if (token !== searchRef.current) return;
closePreview();
setItems(data.items || []); setItems(data.items || []);
setPage(Number(wanted) || 1);
setNextPage(data.next_page || data.continuation || null); setNextPage(data.next_page || data.continuation || null);
setPreviousPage(data.previous_page || null);
setTotal(typeof data.total === "number" ? data.total : null);
setSearched(true);
if (!asked && data.connector_id) setConnectorId(data.connector_id); if (!asked && data.connector_id) setConnectorId(data.connector_id);
refreshPresent(data.items || [], asked || data.connector_id || "");
} catch (err) { } catch (err) {
if (token !== searchRef.current) return; if (token !== searchRef.current) return;
setError(err.message); setError(err.message);
} }
} }
function presentEntry(item, bound = connectorId) {
return { source: "paperless", external_id: String(item.id), connector_id: item.connector_id || bound };
}
function matchesOf(item) {
const entry = presentEntry(item);
return present.get(presentKey(entry.source, entry.connector_id, entry.external_id)) || [];
}
async function refreshPresent(list, bound) {
const entries = list.map((item) => presentEntry(item, bound));
const found = await lookupPresent(context, placement, entries);
setPresent((current) => {
const next = new Map(current);
entries.forEach((entry) => next.delete(presentKey(entry.source, entry.connector_id, entry.external_id)));
found.forEach((value, key) => next.set(key, value));
return next;
});
}
function resetFilters() { function resetFilters() {
invalidate();
closePreview();
setFilters(EMPTY_FILTERS); setFilters(EMPTY_FILTERS);
setItems([]); setItems([]);
setNextPage(null); setNextPage(null);
setPreviousPage(null);
setTotal(null);
setSearched(false);
setPage(1);
} }
function toggle(item) { function toggle(item) {
@@ -195,73 +473,123 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
async function preview(item) { async function preview(item) {
setError(""); setError("");
if (previewUrl) URL.revokeObjectURL(previewUrl); const bound = item.connector_id || connectorId;
setPreviewUrl(""); const key = `${bound}:${item.id}`;
if (key === previewKey) {
closePreview();
return;
}
closePreview();
const token = previewRef.current + 1; const token = previewRef.current + 1;
previewRef.current = token; previewRef.current = token;
const bound = item.connector_id || connectorId; setPreviewKey(key);
try { try {
const blob = await fetchBlob(`/api/v1/sources/paperless/documents/${item.id}/preview?connector_id=${bound}`); const blob = await fetchBlob(`/api/v1/sources/paperless/documents/${item.id}/preview?connector_id=${bound}`);
if (token !== previewRef.current) return; if (token !== previewRef.current) return;
setPreviewUrl(URL.createObjectURL(blob)); setPreviewUrl(URL.createObjectURL(blob));
} catch (err) { } catch (err) {
if (token !== previewRef.current) return; if (token !== previewRef.current) return;
setPreviewKey("");
setError(err.message); setError(err.message);
} }
} }
async function linkMany() { async function linkMany(choices = null) {
const chosen = [...picked.values()]; const chosen = [...picked.values()];
if (!chosen.length) return; if (!chosen.length) return;
setError(""); setError("");
const keyOf = (item) => `${item.connector_id || connectorId}:${item.id}`;
if (!choices) {
const known = chosen.filter((item) => matchesOf(item).length);
if (known.length) {
setReview(new Map(known.map((item) => [keyOf(item), "skip"])));
return;
}
}
setReview(null);
setPending(true); setPending(true);
const inserted = []; const inserted = [];
const skipped = [];
const conflicts = [];
try { try {
for (const item of chosen) { for (const item of chosen) {
await api("/api/v1/external-refs", { const key = keyOf(item);
method: "POST", const choice = choices?.get(key);
body: { if (choice === "skip") {
title: item.title, skipped.push(key);
source: "paperless", continue;
external_id: String(item.id), }
context, const existing = choice === "bind" ? bindable(matchesOf(item)) : null;
...(placement ? { placement } : {}), try {
...(chapterId ? { chapter_id: chapterId } : {}), await api("/api/v1/external-refs", {
...(item.document_on || item.correspondent || item.open_url || connectorId method: "POST",
? { body: {
dates: { document_on: item.document_on || null, sent_on: null, received_on: null }, title: item.title,
detail: { source: "paperless",
...(item.connector_id || connectorId ? { connector_id: item.connector_id || connectorId } : {}), external_id: String(item.id),
...(item.correspondent ? { correspondent: item.correspondent } : {}), context,
...(item.open_url ? { open_url: item.open_url } : {}), ...(placement ? { placement } : {}),
}, ...(chapterId ? { chapter_id: chapterId } : {}),
} ...(choice === "insert" ? { duplicate: "insert" } : {}),
: {}), ...(existing ? { bind_element_id: existing.element_id } : {}),
}, ...(item.document_on || item.correspondent || item.open_url || connectorId
}); ? {
inserted.push(`${item.connector_id || connectorId}:${item.id}`); dates: { document_on: item.document_on || null, sent_on: null, received_on: null },
detail: {
...(item.connector_id || connectorId ? { connector_id: item.connector_id || connectorId } : {}),
...(item.correspondent ? { correspondent: item.correspondent } : {}),
...(item.open_url ? { open_url: item.open_url } : {}),
},
}
: {}),
},
});
inserted.push(key);
} catch (err) {
// Inserted by someone else in the meantime: ask again instead of failing.
if (err.code !== "already_present") throw err;
conflicts.push(item);
}
} }
} catch (err) { } catch (err) {
setError(err.message); setError(err.message);
} finally { } finally {
setPending(false); setPending(false);
} }
if (conflicts.length) {
await refreshPresent(conflicts, connectorId);
setReview(new Map(conflicts.map((item) => [keyOf(item), "skip"])));
}
if (inserted.length) { if (inserted.length) {
setItems((current) => current.filter((item) => !inserted.includes(`${connectorId}:${item.id}`))); setItems((current) => current.filter((item) => !inserted.includes(`${connectorId}:${item.id}`)));
setPicked((current) => {
const next = new Map(current);
inserted.forEach((id) => next.delete(id));
return next;
});
onInserted?.(); onInserted?.();
} }
if (inserted.length === chosen.length) onDone(); setPicked((current) => {
const next = new Map(current);
[...inserted, ...skipped].forEach((id) => next.delete(id));
return next;
});
if (inserted.length && inserted.length + skipped.length === chosen.length) onDone();
} }
function setFilter(name, value) { function setFilter(name, value) {
setFilters((current) => ({ ...current, [name]: value })); setFilters((current) => ({ ...current, [name]: value }));
} }
function setMode(mode) {
setFilters((current) => ({ ...current, mode, sort: current.sort === "-score" && mode !== "index" ? "-created" : current.sort }));
}
const visibleTags = facets.tags.filter((item) => item.name.toLocaleLowerCase().includes(tagSearch.trim().toLocaleLowerCase()));
function tagName(id) {
return facets.tags.find((item) => item.id === id)?.name || id;
}
function selectTags(ids) {
setFilters((current) => ({ ...current, tags: [...new Set([...current.tags, ...ids])] }));
}
function toggleTag(id) { function toggleTag(id) {
setFilters((current) => { setFilters((current) => {
const tags = current.tags.includes(id) ? current.tags.filter((item) => item !== id) : [...current.tags, id]; const tags = current.tags.includes(id) ? current.tags.filter((item) => item !== id) : [...current.tags, id];
@@ -270,21 +598,38 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
} }
const held = [...picked.values()].filter((item) => !items.some((entry) => `${connectorId}:${entry.id}` === `${item.connector_id}:${item.id}`)); const held = [...picked.values()].filter((item) => !items.some((entry) => `${connectorId}:${entry.id}` === `${item.connector_id}:${item.id}`));
const active = activeFilterCount(filters);
const relevance = filters.mode === "index" && filters.query.trim() !== "";
function choiceSelect(name, value, entries, empty) {
return (
<select value={value} onChange={(event) => setFilter(name, event.target.value)}>
<option value="">Alle</option>
<option value="none">{empty}</option>
{entries.map((item) => <option key={item.id} value={item.id}>{item.name}</option>)}
</select>
);
}
return ( return (
<SourceFrame targetLabel={targetLabel} chapterId={chapterId} chapterTitle={chapterTitle}> <SourceFrame targetLabel={targetLabel} chapterId={chapterId} chapterTitle={chapterTitle}>
<div className="paperless-layout">
<aside className="paperless-layout__filters" aria-label="Suche und Filter">
{connectors.length > 1 ? ( {connectors.length > 1 ? (
<label> <label>
Paperless-Zugang Paperless-Zugang
<select value={connectorId} onChange={(event) => { <select value={connectorId} onChange={(event) => {
invalidate(); invalidate();
if (previewUrl) URL.revokeObjectURL(previewUrl); closePreview();
setPreviewUrl("");
setConnectorId(event.target.value); setConnectorId(event.target.value);
setFilters(EMPTY_FILTERS); setFilters(EMPTY_FILTERS);
setItems([]); setItems([]);
setNextPage(null); setNextPage(null);
setFacets({ correspondents: [], document_types: [], tags: [] }); setPreviousPage(null);
setTotal(null);
setSearched(false);
setPage(1);
setFacets(NO_FACETS);
}}> }}>
<option value="">Zugang wählen</option> <option value="">Zugang wählen</option>
{connectors.map((item) => ( {connectors.map((item) => (
@@ -294,69 +639,169 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
</label> </label>
) : null} ) : null}
<form onSubmit={(event) => runSearch(event)} className="stack source-filters"> <form onSubmit={(event) => runSearch(event)} className="stack source-filters">
<label> <div className="paperless-search">
Suchtext in Titel und Inhalt <label>
<input value={filters.query} onChange={(event) => setFilter("query", event.target.value)} /> Suche in
</label> <select value={filters.mode} onChange={(event) => setMode(event.target.value)}>
<label> {PAPERLESS_MODES.map(([value, label]) => <option key={value} value={value}>{label}</option>)}
Titel </select>
<input value={filters.title} onChange={(event) => setFilter("title", event.target.value)} /> </label>
</label> <label className="paperless-search__text">
<label> Suchtext
Korrespondent <input value={filters.query} onChange={(event) => setFilter("query", event.target.value)} />
<select value={filters.correspondent} onChange={(event) => setFilter("correspondent", event.target.value)}> </label>
<option value="">Alle</option> <button type="submit">Suchen</button>
{facets.correspondents.map((item) => <option key={item.id} value={item.id}>{item.name}</option>)} </div>
</select> <p className="muted">{PAPERLESS_MODE_HINTS[filters.mode]}</p>
</label> <details className="paperless-more" open={moreOpen} onToggle={(event) => setMoreOpen(event.currentTarget.open)}>
<label> <summary>{active ? `Weitere Filter (${active} aktiv)` : "Weitere Filter"}</summary>
Dokumenttyp <div className="source-filters">
<select value={filters.documentType} onChange={(event) => setFilter("documentType", event.target.value)}> <label>
<option value="">Alle</option> Titel enthält
{facets.document_types.map((item) => <option key={item.id} value={item.id}>{item.name}</option>)} <input value={filters.title} onChange={(event) => setFilter("title", event.target.value)} />
</select>
</label>
<fieldset>
<legend>Tags, alle gewählten</legend>
{facets.tags.length ? facets.tags.map((item) => (
<label key={item.id} className="paperless-pick">
<input type="checkbox" checked={filters.tags.includes(item.id)} onChange={() => toggleTag(item.id)} />
<span>{item.name}</span>
</label> </label>
)) : <p className="muted">Keine Tags an diesem Zugang.</p>} <label>
</fieldset> Korrespondent
<label> {choiceSelect("correspondent", filters.correspondent, facets.correspondents, "Ohne Korrespondent")}
Dokumentdatum von </label>
<input type="date" value={filters.documentFrom} onChange={(event) => setFilter("documentFrom", event.target.value)} /> <label>
</label> Dokumenttyp
<label> {choiceSelect("documentType", filters.documentType, facets.document_types, "Ohne Dokumenttyp")}
Dokumentdatum bis </label>
<input type="date" value={filters.documentTo} onChange={(event) => setFilter("documentTo", event.target.value)} /> <label>
</label> Speicherpfad
{choiceSelect("storagePath", filters.storagePath, facets.storage_paths, "Ohne Speicherpfad")}
</label>
<fieldset>
<legend>Tags</legend>
<label>
Verknüpfung
<select value={filters.tagMode} onChange={(event) => setFilter("tagMode", event.target.value)}>
<option value="all">Alle gewählten Tags</option>
<option value="any">Mindestens eines der gewählten Tags</option>
<option value="none">Keines der gewählten Tags</option>
</select>
</label>
{filters.tags.length ? (
<ul className="paperless-chips" aria-label="Gewählte Tags">
{filters.tags.map((id) => (
<li key={id}>
<button type="button" className="paperless-chip" onClick={() => toggleTag(id)} aria-label={`Tag ${tagName(id)} entfernen`}>
{tagName(id)} <span aria-hidden="true">×</span>
</button>
</li>
))}
</ul>
) : null}
{facets.tags.length ? (
<>
<input type="search" value={tagSearch} onChange={(event) => setTagSearch(event.target.value)} placeholder="Tags suchen" aria-label="Tags suchen" />
<div className="paperless-tags__actions">
<button type="button" className="quiet" onClick={() => selectTags(visibleTags.map((item) => item.id))} disabled={!visibleTags.length}>
{tagSearch.trim() ? "Treffer wählen" : "Alle wählen"}
</button>
<button type="button" className="quiet" onClick={() => setFilter("tags", [])} disabled={!filters.tags.length}>
Alle abwählen
</button>
</div>
</>
) : null}
<div className="paperless-tags">
{visibleTags.map((item) => (
<label key={item.id} className="paperless-pick">
<input type="checkbox" checked={filters.tags.includes(item.id)} onChange={() => toggleTag(item.id)} />
<span>{item.name}</span>
</label>
))}
{!facets.tags.length ? <p className="muted">Keine Tags an diesem Zugang.</p> : null}
{facets.tags.length && !visibleTags.length ? <p className="muted">Kein Tag passt zur Suche.</p> : null}
</div>
</fieldset>
<label>
Dokumentdatum von
<input type="date" value={filters.documentFrom} onChange={(event) => setFilter("documentFrom", event.target.value)} />
</label>
<label>
Dokumentdatum bis
<input type="date" value={filters.documentTo} onChange={(event) => setFilter("documentTo", event.target.value)} />
</label>
<label>
Hinzugefügt von
<input type="date" value={filters.addedFrom} onChange={(event) => setFilter("addedFrom", event.target.value)} />
</label>
<label>
Hinzugefügt bis
<input type="date" value={filters.addedTo} onChange={(event) => setFilter("addedTo", event.target.value)} />
</label>
<label>
Archivnummer (ASN)
<input inputMode="numeric" value={filters.asn} onChange={(event) => setFilter("asn", event.target.value)} />
</label>
<CustomFieldFilters fields={facets.custom_fields || []} rows={filters.custom} onChange={(rows) => setFilter("custom", rows)} />
</div>
</details>
<label> <label>
Sortierung Sortierung
<select value={filters.sort} onChange={(event) => setFilter("sort", event.target.value)}> <select value={filters.sort} onChange={(event) => setFilter("sort", event.target.value)}>
<option value="-created">Dokumentdatum, neueste zuerst</option> <option value="-created">Dokumentdatum, neueste zuerst</option>
<option value="created">Dokumentdatum, älteste zuerst</option> <option value="created">Dokumentdatum, älteste zuerst</option>
<option value="-added">Hinzugefügt, neueste zuerst</option>
<option value="added">Hinzugefügt, älteste zuerst</option>
<option value="title">Titel aufsteigend</option> <option value="title">Titel aufsteigend</option>
<option value="-title">Titel absteigend</option> <option value="-title">Titel absteigend</option>
<option value="-score" disabled={!filters.query.trim()}>Relevanz zum Suchtext</option> <option value="-score" disabled={!relevance}>Relevanz (nur Volltextindex mit Suchtext)</option>
</select> </select>
</label> </label>
<div className="row"> <div className="row">
<button type="submit">Suchen</button>
<button type="button" className="quiet" onClick={resetFilters}>Filter zurücksetzen</button> <button type="button" className="quiet" onClick={resetFilters}>Filter zurücksetzen</button>
</div> </div>
</form> </form>
</aside>
<section className="paperless-layout__results" aria-label="Treffer">
<ErrorNote error={error} /> <ErrorNote error={error} />
<p className="muted">{SEARCH_LIMITS.paperless}</p> <p className="muted">{SEARCH_LIMITS.paperless}</p>
<p className="muted">Die Verbindung richtest du unter <Link to="/sources">Quellen</Link> ein. Ausgewählte Belege bleiben erhalten, wenn du weiterblätterst oder neu suchst.</p> <p className="muted">Die Verbindung richtest du unter <Link to="/sources">Quellen</Link> ein. Ausgewählte Belege bleiben erhalten, wenn du weiterblätterst oder neu suchst.</p>
{review ? (
<div className="card stack duplicate-panel" role="alert">
<strong>Diese Belege sind schon vorhanden. Was soll geschehen?</strong>
<ul className="duplicate-list">
{[...picked.values()].filter((item) => review.has(`${item.connector_id || connectorId}:${item.id}`)).map((item) => {
const key = `${item.connector_id || connectorId}:${item.id}`;
const matches = matchesOf(item);
return (
<li key={key}>
<span>{item.title}: {matches.length ? describeMatch(matches[0]) : "schon vorhanden"}</span>
<select
value={review.get(key)}
aria-label={`Entscheidung für ${item.title}`}
onChange={(event) => setReview((current) => new Map(current).set(key, event.target.value))}
>
<option value="skip">Nicht einfügen</option>
<option value="insert">Trotzdem erneut einfügen</option>
{bindable(matches) ? <option value="bind">Vorhandenes zusätzlich einbinden</option> : null}
</select>
</li>
);
})}
</ul>
<div className="row">
<button type="button" className="quiet" disabled={pending} onClick={() => setReview(null)}>Zurück zur Auswahl</button>
<button type="button" disabled={pending} onClick={() => linkMany(review)}>Entscheidung ausführen</button>
</div>
</div>
) : null}
{items.length || picked.size ? ( {items.length || picked.size ? (
<button type="button" disabled={pending || picked.size === 0} onClick={linkMany}> <button type="button" disabled={pending || picked.size === 0 || Boolean(review)} onClick={() => linkMany()}>
{picked.size ? `${picked.size} Belege einfügen` : "Belege auswählen"} {picked.size ? `${picked.size} Belege einfügen` : "Belege auswählen"}
</button> </button>
) : null} ) : null}
{previewUrl ? <DocumentView title="Paperless-Vorschau" url={previewUrl} /> : null} {searched ? (
<p className="muted" role="status">
{items.length
? `${total !== null ? `${total} Treffer` : "Treffer"}, Seite ${page}`
: "Dazu gibt es in Paperless keinen Treffer."}
</p>
) : null}
<div className="stack"> <div className="stack">
{held.map((item) => ( {held.map((item) => (
<div key={`held-${item.connector_id}:${item.id}`} className="paperless-pick"> <div key={`held-${item.connector_id}:${item.id}`} className="paperless-pick">
@@ -366,17 +811,65 @@ export function PaperlessPicker({ context, placement, chapterId = "", chapterTit
</label> </label>
</div> </div>
))} ))}
{items.map((item) => ( {items.map((item) => {
<div key={`${connectorId}:${item.id}`} className="paperless-pick"> const key = `${connectorId}:${item.id}`;
<label> const open = previewKey === key;
<input type="checkbox" checked={picked.has(`${connectorId}:${item.id}`)} disabled={pending} onChange={() => toggle(item)} /> const facts = paperlessFacts(item);
<span>{paperlessLabel(item)}</span> return (
</label> <article key={key} className={open ? "paperless-card is-open" : "paperless-card"}>
<button type="button" className="quiet" onClick={() => preview(item)}>Ansehen</button> <input
</div> type="checkbox"
))} checked={picked.has(key)}
disabled={pending}
onChange={() => toggle(item)}
aria-label={`${item.title} auswählen`}
/>
<button type="button" className="paperless-thumb" onClick={() => preview(item)} aria-label={`${item.title} ansehen`}>
<PaperlessThumb connectorId={connectorId} id={item.id} />
</button>
<div className="paperless-card__body">
<strong>{item.title}</strong>
<PresentNote matches={matchesOf(item)} />
{facts.length ? <span className="paperless-card__facts">{facts.join(" · ")}</span> : null}
{item.tags?.length ? (
<ul className="paperless-card__tags" aria-label="Tags">
{item.tags.map((tag) => <li key={tag}>{tag}</li>)}
</ul>
) : null}
{item.custom_fields?.length ? (
<dl className="paperless-card__fields" aria-label="Zusatzfelder">
{item.custom_fields.map((field) => (
<div key={field.name}>
<dt>{field.name}</dt>
<dd>{field.type === "date" && formatDate(field.value) !== "Ohne Datum" ? formatDate(field.value) : field.value}</dd>
</div>
))}
</dl>
) : null}
{item.snippet ? <p className="paperless-card__snippet">{item.snippet}</p> : null}
<div className="row">
<button type="button" className="quiet" onClick={() => preview(item)} aria-expanded={open}>
{open ? "Ansicht schließen" : "Ansehen"}
</button>
</div>
</div>
{open ? (
<div className="paperless-card__view">
{previewUrl ? <DocumentView title={`Paperless-Vorschau: ${item.title}`} url={previewUrl} /> : <p className="muted">Vorschau wird geladen …</p>}
</div>
) : null}
</article>
);
})}
</div>
{previousPage || nextPage ? (
<div className="row">
{previousPage ? <button type="button" className="quiet" onClick={(event) => runSearch(event, previousPage)}>Vorherige Treffer</button> : null}
{nextPage ? <button type="button" onClick={(event) => runSearch(event, nextPage)}>Weitere Treffer</button> : null}
</div>
) : null}
</section>
</div> </div>
{nextPage ? <button type="button" onClick={(event) => runSearch(event, nextPage)}>Weitere Treffer</button> : null}
</SourceFrame> </SourceFrame>
); );
} }
@@ -493,6 +986,7 @@ export function MailPicker({ context, placement, chapterId = "", chapterTitle =
const [preview, setPreview] = useState(null); const [preview, setPreview] = useState(null);
const [previewError, setPreviewError] = useState(""); const [previewError, setPreviewError] = useState("");
const [adopting, setAdopting] = useState(false); const [adopting, setAdopting] = useState(false);
const { place, panel } = useExternalPlacement({ context, placement, onPlaced: () => onDone() });
const [loading, setLoading] = useState(""); const [loading, setLoading] = useState("");
const [error, setError] = useState(""); const [error, setError] = useState("");
const requestRef = useRef(0); const requestRef = useRef(0);
@@ -675,26 +1169,22 @@ export function MailPicker({ context, placement, chapterId = "", chapterTitle =
if (cc) detail.message_cc = cc; if (cc) detail.message_cc = cc;
if (attachments.length) detail.attachments = attachments; if (attachments.length) detail.attachments = attachments;
try { try {
await api("/api/v1/external-refs", { await place({
method: "POST", title: item.title,
body: { source: "imap",
title: item.title, external_id: item.id,
source: "imap", context,
external_id: item.id, ...(placement ? { placement } : {}),
context, ...(chapterId ? { chapter_id: chapterId } : {}),
...(placement ? { placement } : {}), ...(item.document_on ? {
...(chapterId ? { chapter_id: chapterId } : {}), dates: {
...(item.document_on ? { document_on: item.document_on,
dates: { sent_on: direction === "out" ? item.document_on : null,
document_on: item.document_on, received_on: direction === "in" ? item.document_on : null,
sent_on: direction === "out" ? item.document_on : null, },
received_on: direction === "in" ? item.document_on : null, } : {}),
}, detail,
} : {}),
detail,
},
}); });
onDone();
} catch (err) { } catch (err) {
setError(err.message); setError(err.message);
} finally { } finally {
@@ -775,6 +1265,7 @@ export function MailPicker({ context, placement, chapterId = "", chapterTitle =
{selected && !preview && !previewError ? <p className="muted">Vorschau wird geladen …</p> : null} {selected && !preview && !previewError ? <p className="muted">Vorschau wird geladen …</p> : null}
{preview ? <MailBody html={preview.html} text={preview.text} truncated={preview.text_truncated} notices={preview.notices} /> : null} {preview ? <MailBody html={preview.html} text={preview.text} truncated={preview.text_truncated} notices={preview.notices} /> : null}
<ErrorNote error={error} /> <ErrorNote error={error} />
{panel}
<button type="button" onClick={() => link(selected)} disabled={!selected.uid || adopting}>{adopting ? "Wird übernommen …" : "Übernehmen"}</button> <button type="button" onClick={() => link(selected)} disabled={!selected.uid || adopting}>{adopting ? "Wird übernommen …" : "Übernehmen"}</button>
</> </>
)} )}
@@ -784,5 +1275,5 @@ export function MailPicker({ context, placement, chapterId = "", chapterTitle =
); );
} }
registerSourceModule({ kind: "paperless", title: "Paperless", addLabel: "Paperless", manual: true, searchHint: SEARCH_LIMITS.paperless, Picker: PaperlessPicker }); registerSourceModule({ kind: "paperless", title: "Paperless", addLabel: "Paperless", manual: true, large: true, searchHint: SEARCH_LIMITS.paperless, Picker: PaperlessPicker });
registerSourceModule({ kind: "imap", title: "E-Mail-Konto", addLabel: "E-Mail", wide: true, searchHint: SEARCH_LIMITS.imap, Picker: MailPicker }); registerSourceModule({ kind: "imap", title: "E-Mail-Konto", addLabel: "E-Mail", wide: true, searchHint: SEARCH_LIMITS.imap, Picker: MailPicker });
+44 -66
View File
@@ -1,11 +1,11 @@
import { DocumentView } from "../pages/DocumentView.jsx";
import { useEffect, useRef, useState } from "react"; import { useEffect, useRef, useState } from "react";
import { api, documentToText, downloadMailPart, fetchExternal, fetchMailPart, markDirty, textToDocument } from "../api.js"; import { api, documentToText, downloadMailPart, fetchExternal, fetchMailPart, markDirty, textToDocument } from "../api.js";
import { MailAttachments, MailBody } from "../mail-view.jsx"; import { MailAttachments, MailBody } from "../mail-view.jsx";
import { deviceClass, readImageChoice } from "../mail-images.js";
import { MarkdownView } from "../markdown.jsx"; import { MarkdownView } from "../markdown.jsx";
import { NoteEditor } from "../notes.jsx"; import { NoteEditor } from "../notes.jsx";
import { DirectionColumn, directionWord } from "../pages/content.jsx"; import { DirectionColumn, directionWord } from "../pages/content.jsx";
import workerSrc from "pdfjs-dist/legacy/build/pdf.worker.min.mjs?url"; import { DocumentView } from "../pages/DocumentView.jsx";
import { editors, registerStage } from "./registry.jsx"; import { editors, registerStage } from "./registry.jsx";
function previewKind(type, filename) { function previewKind(type, filename) {
@@ -16,6 +16,11 @@ function previewKind(type, filename) {
return "other"; return "other";
} }
function isPdfBytes(buffer) {
const sample = new TextDecoder("iso-8859-1").decode(buffer.slice(0, 1024)).trim();
return sample.startsWith("%PDF");
}
export function NoteStage({ element, context, binding, onReload }) { export function NoteStage({ element, context, binding, onReload }) {
const editor = editors.note; const editor = editors.note;
const [text, setText] = useState(() => documentToText(element.document)); const [text, setText] = useState(() => documentToText(element.document));
@@ -131,10 +136,6 @@ export function MailStage({ element, context, binding, shownDirection }) {
const [phase, setPhase] = useState("idle"); const [phase, setPhase] = useState("idle");
const [media, setMedia] = useState(""); const [media, setMedia] = useState("");
const [blobUrl, setBlobUrl] = useState(""); const [blobUrl, setBlobUrl] = useState("");
const [pdf, setPdf] = useState(null);
const [page, setPage] = useState(1);
const [pages, setPages] = useState(0);
const canvasRef = useRef(null);
const objectUrl = useRef(""); const objectUrl = useRef("");
const shownIndex = shownFor?.elementId === element?.id ? shownFor.index : null; const shownIndex = shownFor?.elementId === element?.id ? shownFor.index : null;
const shownFile = (letter?.attachments || []).find((item) => item.index === shownIndex) || null; const shownFile = (letter?.attachments || []).find((item) => item.index === shownIndex) || null;
@@ -161,49 +162,41 @@ export function MailStage({ element, context, binding, shownDirection }) {
}, [element.id, element.external?.source, context?.kind, context?.id]); }, [element.id, element.external?.source, context?.kind, context?.id]);
useEffect(() => { useEffect(() => {
if (shownIndex === null) return undefined; if (shownIndex === null) {
setMedia("");
setPhase("idle");
if (objectUrl.current) {
URL.revokeObjectURL(objectUrl.current);
objectUrl.current = "";
}
setBlobUrl("");
return undefined;
}
const file = (letter?.attachments || []).find((item) => item.index === shownIndex); const file = (letter?.attachments || []).find((item) => item.index === shownIndex);
if (!file) return undefined; if (!file) return undefined;
let alive = true; let alive = true;
setPhase("loading"); setPhase("loading");
setMedia("");
fetchMailPart(element.id, context, `attachments/${file.index}`).then(async (blob) => { fetchMailPart(element.id, context, `attachments/${file.index}`).then(async (blob) => {
if (!alive) return; if (!alive) return;
if (objectUrl.current) URL.revokeObjectURL(objectUrl.current); const buffer = await blob.arrayBuffer();
const url = URL.createObjectURL(blob); if (!alive) return;
const declared = previewKind(blob.type || file.content_type, file.filename);
const kind = isPdfBytes(buffer) ? "pdf" : declared;
if (objectUrl.current) {
URL.revokeObjectURL(objectUrl.current);
objectUrl.current = "";
}
const type = kind === "pdf" ? "application/pdf" : (blob.type || file.content_type || "application/octet-stream");
const url = URL.createObjectURL(new Blob([buffer], { type }));
objectUrl.current = url; objectUrl.current = url;
setBlobUrl(url); setBlobUrl(url);
const kind = previewKind(blob.type || file.content_type, file.filename); setMedia(kind === "pdf" ? "pdf" : kind);
if (kind === "image") {
setMedia("image");
setPhase("ready");
return;
}
if (kind === "pdf") {
setMedia("pdf");
try {
const bytes = await blob.arrayBuffer();
const pdfjs = await import("pdfjs-dist/legacy/build/pdf.mjs");
pdfjs.GlobalWorkerOptions.workerSrc = workerSrc;
const doc = await pdfjs.getDocument({ data: bytes }).promise;
if (!alive) {
doc.destroy();
return;
}
setPdf(doc);
setPages(doc.numPages);
setPhase("ready");
} catch (err) {
if (!alive) return;
setMedia("pdf-frame");
setPhase("ready");
}
return;
}
setMedia("other");
setPhase("ready"); setPhase("ready");
}).catch((err) => { }).catch((err) => {
if (!alive) return; if (!alive) return;
setPhase("ready"); setPhase("ready");
setMedia("");
setError(err.message || "Die Anlage ist nicht verfügbar."); setError(err.message || "Die Anlage ist nicht verfügbar.");
}); });
return () => { return () => {
@@ -212,26 +205,8 @@ export function MailStage({ element, context, binding, shownDirection }) {
}, [element.id, shownIndex, letter, context?.kind, context?.id]); }, [element.id, shownIndex, letter, context?.kind, context?.id]);
useEffect(() => () => { useEffect(() => () => {
pdf?.destroy?.();
if (objectUrl.current) URL.revokeObjectURL(objectUrl.current); if (objectUrl.current) URL.revokeObjectURL(objectUrl.current);
}, [pdf]); }, []);
useEffect(() => {
if (!pdf || !canvasRef.current) return undefined;
let alive = true;
pdf.getPage(page).then((pdfPage) => {
if (!alive || !canvasRef.current) return;
const canvas = canvasRef.current;
const context2d = canvas.getContext("2d");
const viewport = pdfPage.getViewport({ scale: 1 });
canvas.width = viewport.width;
canvas.height = viewport.height;
return pdfPage.render({ canvasContext: context2d, viewport }).promise;
}).catch(() => {});
return () => {
alive = false;
};
}, [pdf, page]);
async function openOriginal() { async function openOriginal() {
setError(""); setError("");
@@ -260,17 +235,20 @@ export function MailStage({ element, context, binding, shownDirection }) {
{shownFile ? <p className="mail-attachment__caption">Anlage: {shownFile.filename}</p> : null} {shownFile ? <p className="mail-attachment__caption">Anlage: {shownFile.filename}</p> : null}
{shownFile && phase === "loading" ? <p className="muted">Die Anlage wird geladen …</p> : null} {shownFile && phase === "loading" ? <p className="muted">Die Anlage wird geladen …</p> : null}
{media === "image" && blobUrl ? <img className="reader-image" alt={shownFile?.filename || element.title} src={blobUrl} /> : null} {media === "image" && blobUrl ? <img className="reader-image" alt={shownFile?.filename || element.title} src={blobUrl} /> : null}
{media === "pdf" && pdf ? <canvas ref={canvasRef} className="reader-canvas" /> : null} {media === "pdf" && blobUrl ? <DocumentView title={shownFile?.filename || element.title} url={blobUrl} /> : null}
{media === "pdf-frame" && blobUrl ? <DocumentView title={shownFile?.filename || element.title} url={blobUrl} /> : null}
{media === "other" && blobUrl && binding?.rights?.download ? <a href={blobUrl} target="_blank" rel="noreferrer">Anlage öffnen</a> : null} {media === "other" && blobUrl && binding?.rights?.download ? <a href={blobUrl} target="_blank" rel="noreferrer">Anlage öffnen</a> : null}
{pages > 1 ? ( <MailBody
<div className="reader-pages"> html={letter.html}
<button type="button" disabled={page <= 1} onClick={() => setPage((current) => Math.max(1, current - 1))}>Vorherige Seite</button> text={letter.text}
<span>{page} / {pages}</span> truncated={letter.text_truncated}
<button type="button" disabled={page >= pages} onClick={() => setPage((current) => Math.min(pages, current + 1))}>Nächste Seite</button> notices={letter.notices}
</div> remote={{
) : null} key: element.id,
<MailBody html={letter.html} text={letter.text} truncated={letter.text_truncated} notices={letter.notices} /> count: letter.remote_images || 0,
auto: readImageChoice(deviceClass()) === "auto",
load: () => api(`/api/v1/external-refs/${element.id}/mail/html-with-images${context?.kind && context?.id ? `?context_kind=${context.kind}&context_id=${context.id}` : ""}`),
}}
/>
{binding?.rights?.download ? <button type="button" onClick={openOriginal}>Vollständige E-Mail herunterladen</button> : null} {binding?.rights?.download ? <button type="button" onClick={openOriginal}>Vollständige E-Mail herunterladen</button> : null}
</> </>
) : ( ) : (
@@ -35,13 +35,20 @@ const OPERATIONS = [
const SEARCH_FILTERS = [ const SEARCH_FILTERS = [
["query", "Suchtext"], ["query", "Suchtext"],
["mode", "Suchart (Titel und Inhalt, Titel, Inhalt, Volltextindex)"],
["title", "Titel"], ["title", "Titel"],
["correspondent", "Korrespondent"], ["correspondent", "Korrespondent"],
["document_type", "Dokumenttyp"], ["document_type", "Dokumenttyp"],
["storage_path", "Speicherpfad"],
["tags", "Tags"], ["tags", "Tags"],
["tag_mode", "Tag-Verknüpfung (alle, mindestens eines, keines)"],
["document_range", "Dokumentdatum"], ["document_range", "Dokumentdatum"],
["added_range", "Hinzugefügt am"],
["asn", "Archivnummer"],
["custom_fields", "Zusatzfelder (benutzerdefinierte Felder)"],
["sort", "Sortierung"], ["sort", "Sortierung"],
["continuation", "Weitere Seiten"], ["continuation", "Weitere Seiten"],
["result_preview", "Vorschaubild und Textauszug je Treffer"],
]; ];
function Availability({ items, source }) { function Availability({ items, source }) {
+121
View File
@@ -0,0 +1,121 @@
import { useState } from "react";
import { api } from "../api.js";
// A document that is already present around the target. The server decides; this only asks and explains.
export function presentKey(source, connectorId, externalId) {
return `${source}:${connectorId || ""}:${externalId}`;
}
export function describeMatch(match) {
const title = match.container?.title;
const maybe = match.certain ? "" : "möglicherweise ";
if (match.where === "here") return `${maybe}schon hier${match.chapter ? ` (Kapitel „${match.chapter}“)` : ""}`;
if (match.where === "inside") return `${maybe}schon ${title ? `in „${title}“` : "in einem zugehörigen Teil"}`;
const kind = match.container?.kind === "case" ? "in der Akte" : "im Vorgang";
return `${maybe}schon ${kind}${title ? ` „${title}“` : ""}`;
}
export function bindable(matches) {
return (matches || []).find((item) => item.certain && item.can_bind && item.where !== "here") || null;
}
// entries: [{ source, external_id, connector_id }]. Answers a Map of presentKey -> matches, only for documents that are present.
// A failed lookup answers nothing: the warning is a help, the server still protects on insert.
export async function lookupPresent(context, placement, entries) {
const found = new Map();
if (!entries.length) return found;
try {
const data = await api("/api/v1/external-refs/existing", {
method: "POST",
body: {
context,
...(placement ? { placement } : {}),
items: entries.map((item) => ({ source: item.source, external_id: item.external_id, ...(item.connector_id ? { connector_id: item.connector_id } : {}) })),
},
});
for (const entry of data.items || []) {
if (entry.matches?.length) found.set(presentKey(entry.source, entry.connector_id, entry.external_id), entry.matches);
}
} catch {
return found;
}
return found;
}
export function PresentNote({ matches }) {
if (!matches?.length) return null;
return <p className="duplicate-note" role="status">Schon vorhanden: {matches.map(describeMatch).join(", ")}.</p>;
}
// Choices when a document is already present: do not insert, insert again, or bind the existing element as well.
export function DuplicatePanel({ matches, busy, onCancel, onInsert, onBind }) {
const existing = bindable(matches);
return (
<div className="card stack duplicate-panel" role="alert">
<strong>Dieses Dokument ist schon vorhanden.</strong>
<ul>
{matches.map((item) => <li key={item.element_id}>{item.title}: {describeMatch(item)}</li>)}
</ul>
<div className="row">
<button type="button" className="quiet" disabled={busy} onClick={onCancel}>Nicht einfügen</button>
<button type="button" className="quiet" disabled={busy} onClick={onInsert}>Trotzdem erneut einfügen</button>
{existing ? <button type="button" disabled={busy} onClick={() => onBind(existing.element_id)}>Vorhandenes zusätzlich einbinden</button> : null}
</div>
</div>
);
}
// One document at a time. `place(body)` posts it. If the server finds it already present, `panel` offers the choices
// and `onPlaced` runs once the document is in, whichever way.
export function useExternalPlacement({ context, placement, onPlaced }) {
const [conflict, setConflict] = useState(null);
const [busy, setBusy] = useState(false);
const [error, setError] = useState("");
async function send(body) {
await api("/api/v1/external-refs", { method: "POST", body });
setConflict(null);
await onPlaced?.();
}
async function place(body) {
setError("");
try {
await send(body);
} catch (err) {
if (err.code !== "already_present") throw err;
const entry = { source: body.source, external_id: body.external_id, connector_id: body.detail?.connector_id };
const found = await lookupPresent(context, placement, [entry]);
const matches = found.get(presentKey(entry.source, entry.connector_id, entry.external_id)) || [];
if (!matches.length) throw err;
setConflict({ body, matches });
}
}
async function resolve(extra) {
if (!conflict) return;
setBusy(true);
setError("");
try {
await send({ ...conflict.body, ...extra });
} catch (err) {
setError(err.message);
} finally {
setBusy(false);
}
}
const panel = conflict ? (
<>
<DuplicatePanel
matches={conflict.matches}
busy={busy}
onCancel={() => setConflict(null)}
onInsert={() => resolve({ duplicate: "insert" })}
onBind={(elementId) => resolve({ bind_element_id: elementId })}
/>
{error ? <p className="error" role="alert">{error}</p> : null}
</>
) : null;
return { place, panel };
}
+11 -13
View File
@@ -2,6 +2,7 @@ import { useEffect, useRef, useState } from "react";
import { api } from "../api.js"; import { api } from "../api.js";
import { TargetLine } from "../pages/content.jsx"; import { TargetLine } from "../pages/content.jsx";
import { ErrorNote } from "../pages/widgets.jsx"; import { ErrorNote } from "../pages/widgets.jsx";
import { useExternalPlacement } from "./duplicates.jsx";
import { defineSourceModules, moduleFor, registerSourceModule, sourceModules, unregisterSourceModule } from "./registry-core.js"; import { defineSourceModules, moduleFor, registerSourceModule, sourceModules, unregisterSourceModule } from "./registry-core.js";
export { defineSourceModules, moduleFor, registerSourceModule, sourceModules, unregisterSourceModule }; export { defineSourceModules, moduleFor, registerSourceModule, sourceModules, unregisterSourceModule };
@@ -71,6 +72,7 @@ export function StandardSourcePicker({ sourceKind, context, placement, chapterId
const [error, setError] = useState(""); const [error, setError] = useState("");
const [pending, setPending] = useState(false); const [pending, setPending] = useState(false);
const searchRef = useRef(0); const searchRef = useRef(0);
const { place, panel } = useExternalPlacement({ context, placement, onPlaced: () => { onInserted?.(); onDone?.(); } });
const hint = module?.searchHint || "Suche im heute unterstützten Umfang dieser Quelle."; const hint = module?.searchHint || "Suche im heute unterstützten Umfang dieser Quelle.";
useEffect(() => { useEffect(() => {
@@ -111,20 +113,15 @@ export function StandardSourcePicker({ sourceKind, context, placement, chapterId
setPending(true); setPending(true);
setError(""); setError("");
try { try {
await api("/api/v1/external-refs", { await place({
method: "POST", title: item.title,
body: { source: sourceKind,
title: item.title, external_id: String(item.id),
source: sourceKind, context,
external_id: String(item.id), ...(placement ? { placement } : {}),
context, ...(chapterId ? { chapter_id: chapterId } : {}),
...(placement ? { placement } : {}), ...(connectorId ? { detail: { connector_id: connectorId } } : {}),
...(chapterId ? { chapter_id: chapterId } : {}),
...(connectorId ? { detail: { connector_id: connectorId } } : {}),
},
}); });
onInserted?.();
onDone?.();
} catch (err) { } catch (err) {
setError(err.message); setError(err.message);
} finally { } finally {
@@ -149,6 +146,7 @@ export function StandardSourcePicker({ sourceKind, context, placement, chapterId
<button type="submit">Suchen</button> <button type="submit">Suchen</button>
</form> </form>
<ErrorNote error={error} /> <ErrorNote error={error} />
{panel}
<p className="muted">{hint}</p> <p className="muted">{hint}</p>
<div className="stack"> <div className="stack">
{items.map((item) => ( {items.map((item) => (
+89
View File
@@ -424,6 +424,16 @@ button.nav-item:active { transform: none; }
padding: 18px; padding: 18px;
box-shadow: 0 16px 40px rgba(0, 0, 0, 0.18); box-shadow: 0 16px 40px rgba(0, 0, 0, 0.18);
} }
.modal-overlay:has(.modal--large) { padding: 12px; }
.modal.modal--large {
width: min(1800px, 100%);
height: min(96vh, 1400px);
max-height: none;
overflow: hidden;
display: flex;
flex-direction: column;
}
.modal.modal--large > .stack { flex: 1; min-height: 0; display: flex; flex-direction: column; overflow: hidden; }
.modal__bar { display: flex; justify-content: space-between; gap: 12px; align-items: center; margin-bottom: 14px; } .modal__bar { display: flex; justify-content: space-between; gap: 12px; align-items: center; margin-bottom: 14px; }
.modal.modal--wide { width: min(960px, 100%); } .modal.modal--wide { width: min(960px, 100%); }
.modal.modal--wide.modal--split { width: min(1120px, 100%); } .modal.modal--wide.modal--split { width: min(1120px, 100%); }
@@ -1718,6 +1728,78 @@ button.pick-tree__twist {
.note-source__bar { display: flex; flex-wrap: wrap; gap: 6px; } .note-source__bar { display: flex; flex-wrap: wrap; gap: 6px; }
.note-source textarea { width: 100%; min-height: 8rem; } .note-source textarea { width: 100%; min-height: 8rem; }
.source-filters { display: grid; gap: 8px; } .source-filters { display: grid; gap: 8px; }
.paperless-layout { display: grid; gap: 16px; min-height: 0; }
.paperless-layout__filters, .paperless-layout__results { display: grid; gap: 8px; align-content: start; min-width: 0; }
.modal--large .paperless-layout { flex: 1; grid-template-columns: minmax(280px, 360px) minmax(0, 1fr); overflow: hidden; }
.modal--large .paperless-layout__filters, .modal--large .paperless-layout__results { overflow: auto; min-height: 0; padding-right: 6px; }
.modal--large .paperless-layout__filters { border-right: 1px solid var(--border); }
.modal--large .paperless-layout__results > button:first-of-type { position: sticky; top: 0; z-index: 1; }
.modal--large .paperless-search { grid-template-columns: minmax(0, 1fr); }
.paperless-custom { display: grid; gap: 6px; }
.paperless-custom__row { display: grid; gap: 4px; padding: 6px; border: 1px solid var(--border); border-radius: 6px; }
.paperless-search { display: grid; grid-template-columns: minmax(9rem, 12rem) minmax(0, 1fr) auto; gap: 8px; align-items: end; }
.paperless-more > summary { cursor: pointer; font-weight: 600; min-height: 32px; }
.paperless-tags { display: grid; gap: 2px; max-height: 12rem; overflow: auto; }
.paperless-tags__actions { display: flex; flex-wrap: wrap; gap: 6px; }
.paperless-chips { display: flex; flex-wrap: wrap; gap: 4px; margin: 0; padding: 0; list-style: none; }
.paperless-chip { padding: 2px 10px; border: 1px solid var(--border); border-radius: 999px; background: var(--surface2); color: var(--text); font-size: 13px; cursor: pointer; }
.paperless-card {
display: grid;
grid-template-columns: auto 5rem minmax(0, 1fr);
gap: 4px 10px;
align-items: start;
padding: 10px 2px;
border-top: 1px solid var(--border);
}
.paperless-card > input { width: 20px; min-width: 20px; min-height: 20px; margin-top: 2px; }
.paperless-card.is-open { background: var(--accent-light); }
.paperless-card__body { display: grid; gap: 4px; min-width: 0; }
.paperless-card__body strong { overflow-wrap: anywhere; }
.paperless-card__facts { color: var(--text2); font-size: 13px; }
.paperless-card__snippet {
display: -webkit-box;
margin: 0;
overflow: hidden;
color: var(--text2);
font-size: 13px;
line-height: 1.4;
overflow-wrap: anywhere;
-webkit-line-clamp: 3;
-webkit-box-orient: vertical;
}
.paperless-card__tags { display: flex; flex-wrap: wrap; gap: 4px; margin: 0; padding: 0; list-style: none; }
.paperless-card__tags li { padding: 0 8px; border-radius: 999px; background: var(--surface2); color: var(--text2); font-size: 12px; }
.paperless-card__fields { display: flex; flex-wrap: wrap; gap: 2px 14px; margin: 0; font-size: 13px; }
.paperless-card__fields div { display: flex; gap: 4px; min-width: 0; }
.paperless-card__fields dt { color: var(--text3); }
.paperless-card__fields dt::after { content: ":"; }
.paperless-card__fields dd { margin: 0; color: var(--text1); font-weight: 600; overflow-wrap: anywhere; }
.paperless-card__view { grid-column: 1 / -1; }
.paperless-thumb {
display: grid;
place-items: center;
width: 5rem;
min-width: 5rem;
min-height: 6.5rem;
padding: 0;
overflow: hidden;
border: 1px solid var(--border);
border-radius: 4px;
background: var(--surface2);
}
.paperless-thumb__image { display: block; width: 100%; height: 100%; object-fit: cover; object-position: top; }
.paperless-thumb__empty { color: var(--text3); font-size: 11px; }
@media (max-width: 899px) {
.modal.modal--large { overflow: auto; display: block; height: min(96vh, 1400px); }
.modal.modal--large > .stack { display: grid; flex: none; overflow: visible; }
.modal--large .paperless-layout { flex: none; grid-template-columns: minmax(0, 1fr); grid-auto-rows: max-content; overflow: visible; }
.modal--large .paperless-layout__filters, .modal--large .paperless-layout__results { min-height: auto; overflow: visible; border-right: 0; padding-right: 0; }
}
@media (max-width: 600px) {
.paperless-search { grid-template-columns: minmax(0, 1fr); }
.paperless-card { grid-template-columns: auto 4rem minmax(0, 1fr); }
.paperless-thumb { width: 4rem; min-width: 4rem; min-height: 5.5rem; }
}
.markdown pre, .markdown blockquote { margin: 0.4em 0; } .markdown pre, .markdown blockquote { margin: 0.4em 0; }
.markdown blockquote { border-left: 3px solid var(--border2); padding-left: 10px; } .markdown blockquote { border-left: 3px solid var(--border2); padding-left: 10px; }
.markdown pre { overflow: auto; padding: 8px; background: var(--surface2); border-radius: 6px; } .markdown pre { overflow: auto; padding: 8px; background: var(--surface2); border-radius: 6px; }
@@ -1755,3 +1837,10 @@ button.pick-tree__twist {
.mini-icon--link { color: var(--kind-room); } .mini-icon--link { color: var(--kind-room); }
.structure__links { margin-top: 12px; padding-top: 8px; border-top: 1px solid var(--border); } .structure__links { margin-top: 12px; padding-top: 8px; border-top: 1px solid var(--border); }
.structure__links h3 { font-size: 11px; letter-spacing: 0.06em; text-transform: uppercase; color: var(--text2); margin: 0 8px 4px; } .structure__links h3 { font-size: 11px; letter-spacing: 0.06em; text-transform: uppercase; color: var(--text2); margin: 0 8px 4px; }
.duplicate-note { margin: 0; color: var(--text); font-size: 13px; font-weight: 600; }
.duplicate-panel { border: 1px solid var(--border); background: var(--surface2); }
.duplicate-panel ul { margin: 0; padding-left: 18px; }
.duplicate-list { list-style: none; padding-left: 0 !important; display: grid; gap: 8px; }
.duplicate-list li { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; justify-content: space-between; }
.duplicate-list select { max-width: 100%; }