diff --git a/README.md b/README.md index c38523e..773f56e 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ Remote: `http://192.168.2.144:3000/Lars/tsumugu.git` ## Stand -Die erste benutzbare Lieferung organisiert und verfolgt: Anmeldung, Datenräume, Akten und Vorgänge, Notizen, Dateien, manuelle Paperless-Referenzen, Chronologie, Aufgaben und unabhängige Fristen. Finanzfunktionen, automatische Paperless-Synchronisation, KI und externer Anwaltszugang folgen später. +Die erste benutzbare Lieferung organisiert und verfolgt: Anmeldung, Datenräume, Akten und Vorgänge, Notizen, Dateien, manuelle Paperless-Referenzen, Chronologie, Aufgaben und unabhängige Fristen. Seitdem hinzugekommen: Paperless- und IMAP-Zugänge mit Auswahl und Vorschau, Antwortbezüge zwischen Unterlagen, Kommunikationsrichtung je Einbindung, Unterkapitel und das Übergabepaket als ZIP. Finanzfunktionen, automatische Paperless-Synchronisation, virtuelle Zusammenstellungen, KI und laufender Anwaltszugang folgen später. -Ein Push auf `develop` baut und startet Development, ein Merge von `develop` nach `main` die Produktion. Der Gitea-Runner auf dem Raspberry Pi (`192.168.2.49`) übernimmt Build und Deploy. Beim Start führt das Backend die Migrationen aus. In Development legt es synthetische Beispieldaten an, in Production nicht. +Ein Push auf `develop` baut und startet Development, ein Merge von `develop` nach `main` die Produktion. Der Gitea-Runner auf dem Raspberry Pi (`192.168.2.49`) übernimmt Build und Deploy. Beim Start führt das Backend die Migrationen aus und entfernt Konten mit Herkunft `fixture`. Synthetische Beispieldaten entstehen in keiner Umgebung automatisch, nur in der Test-Suite auf einer Datenbank `*_test`. | | Development | Production | |---|---|---| @@ -19,11 +19,12 @@ Ein Push auf `develop` baut und startet Development, ein Merge von `develop` nac Details, Ports und der Synology-Reverse-Proxy: `docs/DEPLOYMENT.md`. -Development ist nach dem Deploy unter https://dev.tsumugu.jinkendo.de oder http://192.168.2.49:3094 erreichbar. Die Beispielkonten stehen auf der Anmeldeseite: +Development ist nach dem Deploy unter https://dev.tsumugu.jinkendo.de oder http://192.168.2.49:3094 erreichbar. Auf einer leeren Datenbank bietet die Anmeldeseite einmalig das Anlegen des Betreiberkontos an; der Betreiber legt danach weitere Konten unter „Verwaltung → Nutzer“ an. -| | E-Mail | Passwort | -|---|---|---| -| Ada Beispiel | ada@example.test | Beispiel-Ada-2026 | -| Ben Beispiel | ben@example.test | Beispiel-Ben-2026 | +Die synthetischen Beispielkonten „Ada Beispiel“ (`ada@example.test`) und „Ben Beispiel“ (`ben@example.test`) mit dem Mietstreit Musterstraße legt nur die Test-Suite an (`backend/app/seed.py`). Die Anmeldeseite zeigt sie ausschließlich in Development mit einer Datenbank `*_test` und `SEED_DEMO=true`. Auf `tsumugu_dev` und `tsumugu` gibt es sie nicht; der Start entfernt sie dort. -Ada führt den Mietstreit Musterstraße. Ben darf diesen Kontext bearbeiten und sieht die private Verhandlungsstrategie nicht. Vorläufige Rechte- und Löschannahmen: `docs/MVP_ANNAHMEN.md`. `/api/health` prüft weiterhin nur, ob die Anwendung und die Datenbank antworten. +Vorläufige Rechte- und Löschannahmen: `docs/MVP_ANNAHMEN.md`. `/api/health` prüft weiterhin nur, ob die Anwendung und die Datenbank antworten. + +## Dokumentation + +Übersicht aller Dokumente: [docs/README.md](docs/README.md). Lokaler Start und Tests: [docs/ENTWICKLUNG_UND_TESTS.md](docs/ENTWICKLUNG_UND_TESTS.md). Letztes Code-Review: [docs/review/](docs/review/README.md). diff --git a/backend/app/access/limits.py b/backend/app/access/limits.py new file mode 100644 index 0000000..73d4cb1 --- /dev/null +++ b/backend/app/access/limits.py @@ -0,0 +1,26 @@ +"""The one place where subscription limits will be checked. + +Today there are no limits. The checks are called where a limit will matter, so that a plan +can be added without touching the callers. + +Target model (see the entitlements blueprint): the quota subject is the owner of a room, a main +user, like the club at Shinkan. A plan, set by the operator, may later cap the number of active +employees per main user and per room. The count comes from the database, never from the client. +""" + +from __future__ import annotations + +from uuid import UUID + + +def ensure_can_create_room(conn, owner_id: UUID) -> None: + """Called before a main user creates a room.""" + del conn, owner_id + + +def ensure_can_add_member(conn, owner_id: UUID, room_id: UUID) -> None: + """Called before a person becomes an active member of a room or is invited to it. + + ``owner_id`` is the owner of the room, the future quota subject. + """ + del conn, owner_id, room_id diff --git a/backend/app/access/policy.py b/backend/app/access/policy.py index b2deb9c..1ce5c36 100644 --- a/backend/app/access/policy.py +++ b/backend/app/access/policy.py @@ -24,30 +24,52 @@ class Rights: edit: bool = False share: bool = False delete: bool = False + # Saving or opening the original outside of Tsumugu. Reading in the app does not need it. + download: bool = False @classmethod def full(cls) -> Rights: - return cls(True, True, True, True) + return cls(True, True, True, True, True) @classmethod - def from_row(cls, row: dict | None) -> Rights: + def from_row(cls, row: dict | None, download: bool = False) -> Rights: if row is None: return cls() - return cls(bool(row["can_read"]), bool(row["can_edit"]), bool(row["can_share"]), bool(row["can_delete"])) + return cls(bool(row["can_read"]), bool(row["can_edit"]), bool(row["can_share"]), bool(row["can_delete"]), download and bool(row["can_read"])) + + def union(self, other: Rights) -> Rights: + return Rights( + self.read or other.read, + self.edit or other.edit, + self.share or other.share, + self.delete or other.delete, + self.download or other.download, + ) def without(self, blocked: Rights) -> Rights: + # An exception that takes away reading takes away downloading too. return Rights( self.read and not blocked.read, self.edit and not blocked.edit, self.share and not blocked.share, self.delete and not blocked.delete, + self.download and not blocked.read, ) def allows(self, need: str) -> bool: return bool(getattr(self, need)) def as_dict(self) -> dict[str, bool]: - return {"read": self.read, "edit": self.edit, "share": self.share, "delete": self.delete} + return {"read": self.read, "edit": self.edit, "share": self.share, "delete": self.delete, "download": self.read and self.download} + + +# The room share: what a room role may do with a case or matter that is shared with the room. +# Sharing, deleting and ownership never come from a room role. An unknown role gets nothing. +ROOM_RIGHTS = { + "reader": Rights(read=True), + "member": Rights(read=True, edit=True), + "manager": Rights(read=True, edit=True), +} @dataclass(frozen=True) @@ -189,7 +211,25 @@ def _direct_grant(conn, actor: UUID, kind: str, resource_id: UUID) -> Rights: """, (actor, kind, resource_id), ).fetchone() - return Rights.from_row(row) + if row is None: + return Rights() + return Rights.from_row(row, _direct_download(conn, kind, resource_id)) + + +def _direct_download(conn, kind: str, resource_id: UUID) -> bool: + """Direct recipients follow the download setting `direct` of the owner of the entry.""" + table = {"case": "cases", "matter": "matters", "element": "elements"}.get(kind) + if table is None: + return False + row = conn.execute( + f""" + SELECT a.download_defaults + FROM {table} r JOIN auth_accounts a ON a.id = r.owner_subject_id + WHERE r.id = %s + """, + (resource_id,), + ).fetchone() + return bool(row and row["download_defaults"].get("direct", False)) def _exceptions(conn, actor: UUID, kind: str, resource_id: UUID, context: Ctx | None) -> Rights: @@ -223,6 +263,37 @@ def _exceptions(conn, actor: UUID, kind: str, resource_id: UUID, context: Ctx | return blocked +def room_default(conn, actor: UUID, kind: str, resource_id: UUID) -> Rights: + """Rights of the current members of the rooms a case or matter is shared with. + + A room is a team. The rights of one share are the role of the person in that room, + capped by the share (`read` or `edit`). Several shares add up. Only an active room, + an active membership and an active account count. Anything unclear gives no rights. + """ + rows = conn.execute( + """ + SELECT m.role, s.access, r.download_roles + FROM room_shares s + JOIN room_memberships m ON m.room_id = s.room_id AND m.subject_id = %s AND m.status = 'active' + JOIN data_rooms r ON r.id = s.room_id AND r.kind = 'organization' AND r.status = 'active' + JOIN auth_accounts a ON a.id = m.subject_id AND a.status = 'active' + WHERE s.resource_kind = %s AND s.resource_id = %s + """, + (actor, kind, resource_id), + ).fetchall() + result = Rights() + for row in rows: + rights = ROOM_RIGHTS.get(row["role"], Rights()) + if row["access"] == "read": + rights = Rights(read=rights.read) + elif row["access"] != "edit": + rights = Rights() + if rights.read and row["download_roles"].get(row["role"], False): + rights = Rights(rights.read, rights.edit, rights.share, rights.delete, True) + result = result.union(rights) + return result + + def case_rights(conn, actor: UUID, case: dict) -> Rights: if case["parent_case_id"] is not None and case["rights_mode"] == "inherit_parent": parent = get_case(conn, case["parent_case_id"]) @@ -230,7 +301,7 @@ def case_rights(conn, actor: UUID, case: dict) -> Rights: elif case["owner_subject_id"] == actor: base = Rights.full() else: - base = _direct_grant(conn, actor, "case", case["id"]) + base = _direct_grant(conn, actor, "case", case["id"]).union(room_default(conn, actor, "case", case["id"])) return base.without(_exceptions(conn, actor, "case", case["id"], None)) @@ -244,7 +315,10 @@ def _linked(conn, matter_id: UUID, case_id: UUID) -> bool: def matter_rights(conn, actor: UUID, matter: dict, context: Ctx | None) -> Rights: if context is None: - base = Rights.full() if matter["owner_subject_id"] == actor else _direct_grant(conn, actor, "matter", matter["id"]) + if matter["owner_subject_id"] == actor: + base = Rights.full() + else: + base = _direct_grant(conn, actor, "matter", matter["id"]).union(room_default(conn, actor, "matter", matter["id"])) return base.without(_exceptions(conn, actor, "matter", matter["id"], None)) if context.kind == "case": @@ -571,6 +645,12 @@ def list_cases(conn, actor: UUID) -> list[dict]: AND g.grantee_subject_id = %(actor)s AND NOT g.is_exception AND g.can_read AND g.context_id IS NULL ) + OR (EXISTS ( + SELECT 1 FROM room_shares sh + JOIN room_memberships sm ON sm.room_id = sh.room_id AND sm.subject_id = %(actor)s AND sm.status = 'active' + JOIN data_rooms sr ON sr.id = sh.room_id AND sr.kind = 'organization' AND sr.status = 'active' + WHERE sh.resource_kind = 'case' AND sh.resource_id = c.id + )) OR ( c.rights_mode = 'inherit_parent' AND c.parent_case_id IS NOT NULL AND EXISTS ( SELECT 1 FROM cases p @@ -583,6 +663,12 @@ def list_cases(conn, actor: UUID) -> list[dict]: AND g.grantee_subject_id = %(actor)s AND NOT g.is_exception AND g.can_read AND g.context_id IS NULL ) + OR (EXISTS ( + SELECT 1 FROM room_shares sh + JOIN room_memberships sm ON sm.room_id = sh.room_id AND sm.subject_id = %(actor)s AND sm.status = 'active' + JOIN data_rooms sr ON sr.id = sh.room_id AND sr.kind = 'organization' AND sr.status = 'active' + WHERE sh.resource_kind = 'case' AND sh.resource_id = p.id + )) ) ) ) @@ -613,6 +699,12 @@ def list_matters(conn, actor: UUID) -> list[tuple[dict, list[dict]]]: AND g.grantee_subject_id = %(actor)s AND NOT g.is_exception AND g.can_read AND g.context_id IS NULL ) + OR (EXISTS ( + SELECT 1 FROM room_shares sh + JOIN room_memberships sm ON sm.room_id = sh.room_id AND sm.subject_id = %(actor)s AND sm.status = 'active' + JOIN data_rooms sr ON sr.id = sh.room_id AND sr.kind = 'organization' AND sr.status = 'active' + WHERE sh.resource_kind = 'matter' AND sh.resource_id = m.id + )) OR EXISTS ( SELECT 1 FROM matter_case_links l JOIN cases c ON c.id = l.case_id AND c.deleted_at IS NULL @@ -625,6 +717,12 @@ def list_matters(conn, actor: UUID) -> list[tuple[dict, list[dict]]]: AND g.grantee_subject_id = %(actor)s AND NOT g.is_exception AND g.can_read AND g.context_id IS NULL ) + OR (EXISTS ( + SELECT 1 FROM room_shares sh + JOIN room_memberships sm ON sm.room_id = sh.room_id AND sm.subject_id = %(actor)s AND sm.status = 'active' + JOIN data_rooms sr ON sr.id = sh.room_id AND sr.kind = 'organization' AND sr.status = 'active' + WHERE sh.resource_kind = 'case' AND sh.resource_id = c.id + )) ) ) ) diff --git a/backend/app/access/roles.py b/backend/app/access/roles.py new file mode 100644 index 0000000..5bd8aa5 --- /dev/null +++ b/backend/app/access/roles.py @@ -0,0 +1,106 @@ +"""Fixed role bundles. Rights are read from the current database state, never from the session. + +Platform: operator or user. Data room: member, manager or reader. There is no role editor. +An operator has no read right on content. Unknown role or status is refused. +""" + +from __future__ import annotations + +from uuid import UUID + +from app.core.errors import forbidden + +ROOM_ROLES = ("member", "manager", "reader") + +ROOM_ROLE_LABELS = { + "member": "Mitglied", + "manager": "Verwalter", + "reader": "Leser", +} + + +def account_kind(conn, account_id: UUID) -> str | None: + row = conn.execute("SELECT account_kind FROM auth_accounts WHERE id = %s AND status = 'active'", (account_id,)).fetchone() + return row["account_kind"] if row else None + + +def require_main_user(conn, account_id: UUID) -> None: + """Employees work in the rooms they were invited to. They create no rooms of their own.""" + if account_kind(conn, account_id) != "main": + raise forbidden("Als Mitarbeiter arbeitest du in den Räumen, in die du eingeladen wurdest.") + + +def is_operator(conn, account_id: UUID) -> bool: + row = conn.execute( + "SELECT is_superadmin FROM auth_accounts WHERE id = %s AND status = 'active'", + (account_id,), + ).fetchone() + return bool(row and row["is_superadmin"]) + + +def require_operator(conn, account_id: UUID) -> None: + if not is_operator(conn, account_id): + raise forbidden("Das darf nur der Betreiber.") + + +def room_role(conn, account_id: UUID, room_id: UUID) -> str | None: + """The active role of an account in an organization room, or None.""" + row = conn.execute( + """ + SELECT m.role + FROM room_memberships m + JOIN data_rooms r ON r.id = m.room_id + WHERE m.room_id = %s AND m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' + """, + (room_id, account_id), + ).fetchone() + if row is None or row["role"] not in ROOM_ROLES: + return None + return row["role"] + + +def require_room_manager(conn, account_id: UUID, room_id: UUID) -> None: + if room_role(conn, account_id, room_id) != "manager": + raise forbidden("Das darf nur ein Verwalter dieses Datenraums.") + + +def require_room_write(conn, account_id: UUID, room: dict) -> None: + """Creating content needs the own personal room or a member or manager role.""" + if room["kind"] == "personal": + if room["id"] != account_id: + raise forbidden("Der persönliche Datenraum gehört zu einem anderen Konto.") + require_main_user(conn, account_id) + return + role = room_role(conn, account_id, room["id"]) + if role in {"member", "manager"}: + return + if role == "reader": + raise forbidden("Als Leser kannst du in diesem Datenraum nichts anlegen.") + raise forbidden("Du bist in diesem Datenraum nicht Mitglied.") + + +def count_active_operators(conn, *, lock: bool = False) -> int: + suffix = " FOR UPDATE" if lock else "" + rows = conn.execute(f"SELECT id FROM auth_accounts WHERE is_superadmin AND status = 'active'{suffix}").fetchall() + return len(rows) + + +def count_active_managers(conn, room_id: UUID, *, lock: bool = False) -> int: + suffix = " FOR UPDATE" if lock else "" + rows = conn.execute( + f""" + SELECT m.subject_id FROM room_memberships m + JOIN auth_accounts a ON a.id = m.subject_id + WHERE m.room_id = %s AND m.role = 'manager' AND m.status = 'active' AND a.status = 'active'{suffix} + """, + (room_id,), + ).fetchall() + return len(rows) + + +def ensure_can_create(conn, account_id: UUID, scope_id: UUID) -> None: + """Gate for every operation that creates content with this home room.""" + room = conn.execute("SELECT id, kind FROM data_rooms WHERE id = %s", (scope_id,)).fetchone() + if room is None: + raise forbidden("Der gewählte Datenraum ist nicht verfügbar.") + require_room_write(conn, account_id, room) diff --git a/backend/app/api/accounts.py b/backend/app/api/accounts.py new file mode 100644 index 0000000..c2035f7 --- /dev/null +++ b/backend/app/api/accounts.py @@ -0,0 +1,273 @@ +"""Account administration for the operator and the own account (password, sessions).""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import APIRouter, Depends, Response +from fastapi.responses import JSONResponse +from psycopg import Connection + +from app.access.policy import audit +from app.access.roles import require_operator +from app.api.deps import Principal, get_conn, get_principal, get_session +from app.api.schemas import AccountIn, AccountPatchIn, PasswordChangeIn +from app.auth.accounts import ( + promote_to_main, + change_own_password, + create_account, + generate_password, + reset_password, + revoke_sessions, + set_display_name, + set_operator, + set_status, +) +from app.core.errors import not_found, unprocessable +from app.core.timeutil import dump_dt + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} + + +def _account_item(row) -> dict: + return { + "id": str(row["id"]), + "email": row["email"], + "display_name": row["display_name"], + "is_superadmin": row["is_superadmin"], + "status": row["status"], + "must_change_password": row["must_change_password"], + "created_at": dump_dt(row["created_at"]), + "last_login_at": dump_dt(row["last_login_at"]), + "account_kind": row["account_kind"], + "sponsor_id": str(row["sponsor_id"]) if row["sponsor_id"] else None, + "sponsor_name": row.get("sponsor_name"), + } + + +@router.get("/api/v1/accounts") +def accounts( + status: str | None = None, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_operator(conn, principal.account_id) + if status not in {None, "active", "disabled"}: + raise unprocessable("Der Statusfilter ist ungültig.") + rows = conn.execute( + """ + SELECT id, email, display_name, is_superadmin, status, must_change_password, created_at, last_login_at, + account_kind, sponsor_id, + (SELECT s.display_name FROM auth_accounts s WHERE s.id = auth_accounts.sponsor_id) AS sponsor_name + FROM auth_accounts + WHERE origin = 'operator' AND (%s::text IS NULL OR status = %s::text) + ORDER BY status, display_name, email + """, + (status, status), + ).fetchall() + return {"items": [_account_item(row) for row in rows], "next_cursor": None} + + +@router.get("/api/v1/accounts/{account_id}") +def account_detail(account_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_operator(conn, principal.account_id) + row = conn.execute( + """ + SELECT id, email, display_name, is_superadmin, status, must_change_password, created_at, last_login_at, + account_kind, sponsor_id, + (SELECT s.display_name FROM auth_accounts s WHERE s.id = auth_accounts.sponsor_id) AS sponsor_name + FROM auth_accounts WHERE id = %s AND origin = 'operator' + """, + (account_id,), + ).fetchone() + if row is None: + raise not_found() + rooms = conn.execute( + """ + SELECT r.id, r.name, r.status, m.role + FROM room_memberships m + JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' + ORDER BY r.name, r.id + """, + (account_id,), + ).fetchall() + sessions = conn.execute( + """ + SELECT count(*) AS n FROM auth_sessions + WHERE account_id = %s AND revoked_at IS NULL AND expires_at > now() + """, + (account_id,), + ).fetchone() + body = _account_item(row) + body["rooms"] = [ + {"id": str(room["id"]), "name": room["name"], "status": room["status"], "role": room["role"]} for room in rooms + ] + body["active_sessions"] = sessions["n"] + employees = conn.execute( + """ + SELECT id, display_name, email, status FROM auth_accounts + WHERE sponsor_id = %s ORDER BY display_name, id + """, + (account_id,), + ).fetchall() + body["employees"] = [ + {"id": str(e["id"]), "display_name": e["display_name"], "email": e["email"], "status": e["status"]} for e in employees + ] + return JSONResponse(body, headers=_NO_STORE) + + +@router.post("/api/v1/accounts") +def post_account(body: AccountIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_operator(conn, principal.account_id) + password = body.password or generate_password() + account = create_account( + conn, + body.email, + body.display_name, + password, + is_superadmin=body.is_operator, + must_change_password=True, + ) + audit(conn, principal.account_id, None, "account_created", "account", account["id"]) + if body.is_operator: + audit(conn, principal.account_id, None, "operator_granted", "account", account["id"]) + payload = { + "id": str(account["id"]), + "email": account["email"], + "display_name": account["display_name"], + "is_superadmin": account["is_superadmin"], + "status": account["status"], + "must_change_password": True, + "account_kind": account["account_kind"], + } + if not body.password: + payload["temporary_password"] = password + return JSONResponse( + status_code=201, + content=payload, + headers={"Location": f"/api/v1/accounts/{account['id']}", **_NO_STORE}, + ) + + +@router.patch("/api/v1/accounts/{account_id}") +def patch_account( + account_id: UUID, + body: AccountPatchIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_operator(conn, principal.account_id) + exists = conn.execute("SELECT 1 FROM auth_accounts WHERE id = %s AND origin = 'operator'", (account_id,)).fetchone() + if exists is None: + raise not_found() + if body.display_name is not None: + set_display_name(conn, principal.account_id, account_id, body.display_name) + if body.account_kind == "main": + promote_to_main(conn, principal.account_id, account_id) + if body.is_operator is not None: + set_operator(conn, principal.account_id, account_id, body.is_operator) + if body.status is not None: + set_status(conn, principal.account_id, account_id, body.status) + row = conn.execute( + """ + SELECT id, email, display_name, is_superadmin, status, must_change_password, created_at, last_login_at, + account_kind, sponsor_id, + (SELECT s.display_name FROM auth_accounts s WHERE s.id = auth_accounts.sponsor_id) AS sponsor_name + FROM auth_accounts WHERE id = %s + """, + (account_id,), + ).fetchone() + return _account_item(row) + + +@router.post("/api/v1/accounts/{account_id}/password-reset") +def post_password_reset( + account_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_operator(conn, principal.account_id) + password = reset_password(conn, principal.account_id, account_id) + return JSONResponse({"temporary_password": password}, headers=_NO_STORE) + + +@router.post("/api/v1/accounts/{account_id}/sessions/revoke") +def post_revoke_sessions( + account_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_operator(conn, principal.account_id) + if conn.execute("SELECT 1 FROM auth_accounts WHERE id = %s", (account_id,)).fetchone() is None: + raise not_found() + count = revoke_sessions(conn, account_id) + audit(conn, principal.account_id, None, "sessions_revoked", "account", account_id, detail={"count": count}) + return {"revoked": count} + + +@router.post("/api/v1/auth/password", status_code=204) +def post_own_password( + body: PasswordChangeIn, + session: dict = Depends(get_session), + conn: Connection = Depends(get_conn), +): + change_own_password(conn, session["account_id"], session["id"], body.current_password, body.new_password) + return Response(status_code=204) + + +@router.get("/api/v1/auth/sessions") +def own_sessions(session: dict = Depends(get_session), conn: Connection = Depends(get_conn)): + rows = conn.execute( + """ + SELECT id, label, created_at, last_seen_at, expires_at + FROM auth_sessions + WHERE account_id = %s AND revoked_at IS NULL AND expires_at > now() + ORDER BY created_at DESC, id + """, + (session["account_id"],), + ).fetchall() + return JSONResponse( + { + "items": [ + { + "id": str(row["id"]), + "label": row["label"] or "Browser", + "created_at": dump_dt(row["created_at"]), + "last_seen_at": dump_dt(row["last_seen_at"]), + "expires_at": dump_dt(row["expires_at"]), + "current": row["id"] == session["id"], + } + for row in rows + ] + }, + headers=_NO_STORE, + ) + + +@router.delete("/api/v1/auth/sessions/{session_id}", status_code=204) +def end_own_session(session_id: UUID, session: dict = Depends(get_session), conn: Connection = Depends(get_conn)): + if session_id == session["id"]: + raise unprocessable("Diese Sitzung beendest du mit Abmelden.") + row = conn.execute( + """ + UPDATE auth_sessions SET revoked_at = now() + WHERE id = %s AND account_id = %s AND revoked_at IS NULL + RETURNING id + """, + (session_id, session["account_id"]), + ).fetchone() + if row is None: + raise not_found() + audit(conn, session["account_id"], None, "session_ended", "session", session_id) + return Response(status_code=204) + + +@router.post("/api/v1/auth/sessions/revoke-others") +def end_other_sessions(session: dict = Depends(get_session), conn: Connection = Depends(get_conn)): + count = revoke_sessions(conn, session["account_id"], keep=session["id"]) + audit(conn, session["account_id"], None, "sessions_revoked", "account", session["account_id"], detail={"count": count}) + return {"revoked": count} + diff --git a/backend/app/api/deps.py b/backend/app/api/deps.py index dbf6482..567b386 100644 --- a/backend/app/api/deps.py +++ b/backend/app/api/deps.py @@ -2,8 +2,18 @@ from __future__ import annotations +import secrets from dataclasses import dataclass +from datetime import timedelta from pathlib import Path + + +def discard_stored(item) -> None: + discard = getattr(item, "discard", None) + if callable(discard): + discard() + return + Path(item).unlink(missing_ok=True) from uuid import UUID from fastapi import Depends, Request @@ -11,10 +21,19 @@ from psycopg import Connection from app.access.policy import ensure_member_or_personal from app.auth.security import hash_secret, read_session_token -from app.core.errors import forbidden, unauthorized +from app.core.errors import forbidden, password_change_required, scope_changed, unauthorized from app.core.timeutil import utcnow +# While a one-time password is set, only these routes answer. +PASSWORD_CHANGE_PATHS = frozenset({ + "/api/v1/auth/session", + "/api/v1/auth/password", + "/api/v1/auth/logout", +}) +SEEN_INTERVAL = timedelta(minutes=5) + + @dataclass(frozen=True) class Principal: account_id: UUID @@ -35,11 +54,11 @@ def get_conn(request: Request): yield conn conn.commit() for path in getattr(request.state, "delete_after_commit", []): - Path(path).unlink(missing_ok=True) + discard_stored(path) except Exception: conn.rollback() for path in getattr(request.state, "cleanup_files", []): - Path(path).unlink(missing_ok=True) + discard_stored(path) raise @@ -50,7 +69,8 @@ def _session_row(conn: Connection, request: Request) -> dict: row = conn.execute( """ SELECT s.id, s.account_id, s.csrf_token, s.scope_kind, s.scope_id, s.epoch, - s.expires_at, s.revoked_at, a.email, a.display_name, a.status AS account_status + s.expires_at, s.revoked_at, s.last_seen_at, a.email, a.display_name, + a.status AS account_status, a.must_change_password FROM auth_sessions s JOIN auth_accounts a ON a.id = s.account_id WHERE s.secret_hash = %s @@ -61,10 +81,15 @@ def _session_row(conn: Connection, request: Request) -> dict: raise unauthorized() if row["account_status"] != "active": raise unauthorized() + if row["must_change_password"] and request.url.path not in PASSWORD_CHANGE_PATHS: + raise password_change_required() if request.method not in {"GET", "HEAD", "OPTIONS"}: sent = request.headers.get("x-csrf-token") or "" - if not sent or sent != row["csrf_token"]: + if not sent or not secrets.compare_digest(sent, row["csrf_token"]): raise forbidden("Die Anfrage konnte der Sitzung nicht zugeordnet werden.") + seen = row["last_seen_at"] + if seen is None or utcnow() - seen > SEEN_INTERVAL: + conn.execute("UPDATE auth_sessions SET last_seen_at = now() WHERE id = %s", (row["id"],)) return row @@ -83,7 +108,7 @@ def get_principal(request: Request, conn: Connection = Depends(get_conn)) -> Pri except ValueError as exc: raise forbidden("Der Datenraum ist nicht gewählt.") from exc if kind != row["scope_kind"] or scope_id != row["scope_id"]: - raise forbidden("Der gewählte Datenraum entspricht nicht mehr der Sitzung.") + raise scope_changed() room = ensure_member_or_personal(conn, row["account_id"], kind, scope_id) return Principal( account_id=row["account_id"], diff --git a/backend/app/api/downloads.py b/backend/app/api/downloads.py new file mode 100644 index 0000000..ff85f3a --- /dev/null +++ b/backend/app/api/downloads.py @@ -0,0 +1,82 @@ +"""Download policy: which room role may save the original of a document or mail. + +Reading inside the app stays possible. The policy is a rule of use, not a copy protection: +what the screen shows has to reach the browser. +""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import APIRouter, Depends +from fastapi.responses import JSONResponse +from psycopg import Connection +from psycopg.types.json import Json + +from app.access.policy import audit +from app.access.roles import require_main_user, room_role +from app.api.deps import Principal, get_conn, get_principal +from app.api.schemas import DownloadPolicyIn +from app.core.errors import forbidden, not_found + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} +ROOM_KEYS = ("manager", "member", "reader") + + +def _policy(raw: dict, keys: tuple[str, ...]) -> dict[str, bool]: + return {key: bool(raw.get(key, True)) for key in keys} + + +def _room_of_member(conn, account_id: UUID, room_id: UUID) -> dict: + # A stranger and an unknown room answer alike. + room = conn.execute("SELECT * FROM data_rooms WHERE id = %s AND kind = 'organization'", (room_id,)).fetchone() + if room is None or room_role(conn, account_id, room_id) is None: + raise not_found() + return room + + +@router.get("/api/v1/rooms/{room_id}/download-policy") +def get_room_policy(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + room = _room_of_member(conn, principal.account_id, room_id) + return JSONResponse( + {"roles": _policy(room["download_roles"], ROOM_KEYS), "can_change": room["owner_id"] == principal.account_id}, + headers=_NO_STORE, + ) + + +@router.put("/api/v1/rooms/{room_id}/download-policy") +def put_room_policy( + room_id: UUID, + body: DownloadPolicyIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + room = _room_of_member(conn, principal.account_id, room_id) + if room["owner_id"] != principal.account_id: + raise forbidden("Das darf nur der Eigentümer des Datenraums.") + roles = {"manager": body.manager, "member": body.member, "reader": body.reader} + conn.execute("UPDATE data_rooms SET download_roles = %s WHERE id = %s", (Json(roles), room_id)) + audit(conn, principal.account_id, room_id, "room_download_policy_changed", "room", room_id, detail=roles) + return JSONResponse({"roles": roles, "can_change": True}, headers=_NO_STORE) + + +@router.get("/api/v1/auth/download-defaults") +def get_defaults(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_main_user(conn, principal.account_id) + row = conn.execute("SELECT download_defaults FROM auth_accounts WHERE id = %s", (principal.account_id,)).fetchone() + return JSONResponse({"roles": _policy(row["download_defaults"], (*ROOM_KEYS, "direct"))}, headers=_NO_STORE) + + +@router.put("/api/v1/auth/download-defaults") +def put_defaults( + body: DownloadPolicyIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_main_user(conn, principal.account_id) + roles = {"manager": body.manager, "member": body.member, "reader": body.reader, "direct": True if body.direct is None else body.direct} + conn.execute("UPDATE auth_accounts SET download_defaults = %s WHERE id = %s", (Json(roles), principal.account_id)) + audit(conn, principal.account_id, None, "download_defaults_changed", "account", principal.account_id, detail=roles) + return JSONResponse({"roles": roles}, headers=_NO_STORE) diff --git a/backend/app/api/export.py b/backend/app/api/export.py index 591a17f..b48db32 100644 --- a/backend/app/api/export.py +++ b/backend/app/api/export.py @@ -8,10 +8,10 @@ from uuid import UUID from fastapi import APIRouter, Depends, Request, Response from psycopg import Connection -from app.access.policy import Ctx, parse_query_context +from app.access.policy import Ctx, get_matter, matter_rights, parse_query_context from app.api.deps import Principal, get_conn, get_principal from app.api.schemas import ExportIn -from app.core.errors import unprocessable +from app.core.errors import DOWNLOAD_REFUSED, forbidden, unprocessable from app.exporting import build_export, export_tree, review_export router = APIRouter() @@ -79,6 +79,12 @@ def post_export( conn: Connection = Depends(get_conn), ): selected, assignments = _selection(body) + matter = get_matter(conn, matter_id) + if matter is not None: + rights = matter_rights(conn, principal.account_id, matter, _context(body.context)) + # Without any right the export itself answers as before. With read but no download right: refuse. + if rights.read and not rights.download: + raise forbidden(DOWNLOAD_REFUSED) filename, payload = build_export( conn, principal.account_id, diff --git a/backend/app/api/library.py b/backend/app/api/library.py new file mode 100644 index 0000000..800d025 --- /dev/null +++ b/backend/app/api/library.py @@ -0,0 +1,119 @@ +"""Library endpoints: rooms as tiles, what lies in a room, and sharing several entries at once.""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import APIRouter, Depends +from fastapi.responses import JSONResponse +from psycopg import Connection + +from app.access.policy import parse_query_context, require_case, require_matter +from app.access.roles import room_role +from app.api.deps import Principal, get_conn, get_principal +from app.api.schemas import BatchShareIn +from app.core.errors import forbidden +from app.library import case_items, case_outline, matter_items, matter_outline, room_contents, room_effect +from app.ops import share_with_room + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} + + +@router.get("/api/v1/rooms/overview") +def rooms_overview(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + """The team rooms of the person as tiles. Names of members are only shown to managers.""" + actor = principal.account_id + rooms = conn.execute( + """ + SELECT r.id, r.name, r.owner_id, r.default_visibility, m.role, + (SELECT count(*) FROM room_memberships x WHERE x.room_id = r.id AND x.status = 'active') AS members + FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' AND r.status = 'active' + ORDER BY r.name, r.id + """, + (actor,), + ).fetchall() + items = [] + for room in rooms: + cases, matters = room_contents(conn, actor, room["id"]) + people = [] + if room["role"] == "manager": + people = [ + row["display_name"] + for row in conn.execute( + """ + SELECT a.display_name FROM room_memberships x JOIN auth_accounts a ON a.id = x.subject_id + WHERE x.room_id = %s AND x.status = 'active' + ORDER BY a.display_name, a.id LIMIT 6 + """, + (room["id"],), + ).fetchall() + ] + items.append( + { + "id": str(room["id"]), + "name": room["name"], + "role": room["role"], + "is_owner": room["owner_id"] == actor, + "private": room["members"] <= 1, + "default_visibility": room["default_visibility"], + "member_count": room["members"], + "members": people, + "case_count": sum(1 for case, _ in cases if not case["parent_case_id"]), + "matter_count": sum(1 for matter, _ in matters if not matter["parent_matter_id"]), + } + ) + return JSONResponse({"items": items}, headers=_NO_STORE) + + +@router.get("/api/v1/rooms/{room_id}/contents") +def contents(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + """Cases and matters shared with the room that the person may read.""" + if room_role(conn, principal.account_id, room_id) is None: + raise forbidden() + cases, matters = room_contents(conn, principal.account_id, room_id) + return JSONResponse( + { + "cases": case_items(conn, principal.account_id, cases), + "matters": matter_items(conn, principal.account_id, matters), + }, + headers=_NO_STORE, + ) + + +@router.post("/api/v1/rooms/{room_id}/shares") +def share_many( + room_id: UUID, + body: BatchShareIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """Share several own entries with the room. All or none: one failure rolls the request back.""" + for item in body.items: + share_with_room(conn, principal.account_id, item.kind, item.id, room_id, body.access) + return { + "shared": len(body.items), + "access": body.access, + "effect": room_effect(conn, room_id, principal.account_id, body.access), + } + + +@router.get("/api/v1/cases/{case_id}/outline") +def case_outline_view(case_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + case, _rights = require_case(conn, principal.account_id, case_id, "read") + return JSONResponse(case_outline(conn, principal.account_id, case), headers=_NO_STORE) + + +@router.get("/api/v1/matters/{matter_id}/outline") +def matter_outline_view( + matter_id: UUID, + context_kind: str | None = None, + context_id: str | None = None, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + context = parse_query_context(context_kind, context_id) + matter, _rights = require_matter(conn, principal.account_id, matter_id, context, "read") + return JSONResponse(matter_outline(conn, principal.account_id, matter, context), headers=_NO_STORE) diff --git a/backend/app/api/rooms.py b/backend/app/api/rooms.py new file mode 100644 index 0000000..cf4b0d5 --- /dev/null +++ b/backend/app/api/rooms.py @@ -0,0 +1,711 @@ +"""Data rooms: members, roles, invitations, and the operator's room overview and audit.""" + +from __future__ import annotations + +from datetime import timedelta +from uuid import UUID, uuid4 + +from fastapi import APIRouter, Depends, Response +from fastapi.responses import JSONResponse +from psycopg import Connection + +from app.access.policy import account_by_email, audit +from app.access.limits import ensure_can_add_member, ensure_can_create_room +from app.access.roles import ( + ROOM_ROLES, + account_kind, + count_active_managers, + require_main_user, + require_operator, + require_room_manager, + room_role, +) +from app.api.deps import Principal, get_conn, get_principal +from app.api.schemas import ( + InvitationIn, + MemberIn, + MemberRoleIn, + OperatorManagerIn, + OperatorOwnerIn, + OwnerIn, + RoomIn, + RoomPatchIn, + VisibilityIn, +) +from app.auth.accounts import normalize_email +from app.auth.security import hash_secret, new_secret +from app.core.errors import forbidden, not_found, unprocessable +from app.core.timeutil import dump_dt, utcnow +from app.present import LABELS + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} +INVITATION_DAYS = 7 + + +def _created(path: str, payload: dict) -> JSONResponse: + return JSONResponse(status_code=201, content=payload, headers={"Location": path, **_NO_STORE}) + + +def _require_owner(conn, account_id: UUID, room_id: UUID) -> dict: + # An unknown room and a room owned by someone else answer alike: no way to probe ids. + room = conn.execute( + "SELECT * FROM data_rooms WHERE id = %s AND kind = 'organization'", (room_id,) + ).fetchone() + if room is None or room["owner_id"] != account_id: + raise forbidden("Das darf nur der Eigentümer des Datenraums.") + return room + + +def _protect_owner(room: dict, subject_id: UUID) -> None: + if room["owner_id"] == subject_id: + raise unprocessable("Der Eigentümer bleibt Verwalter. Übertrage den Datenraum zuerst.") + + +def _room(conn, room_id: UUID) -> dict: + row = conn.execute("SELECT * FROM data_rooms WHERE id = %s AND kind = 'organization'", (room_id,)).fetchone() + if row is None: + raise not_found() + return row + + +def _audit_membership(conn, actor: UUID, room_id: UUID, subject_id: UUID, action: str, role: str | None = None) -> None: + audit( + conn, + actor, + room_id, + action, + "membership", + subject_id, + detail={"room_id": str(room_id), **({"role": role} if role else {})}, + ) + + +def _drop_edit_rights_in_room(conn, subject_id: UUID, room_id: UUID) -> None: + """A reader neither edits nor shares objects of this room. Read grants stay.""" + conn.execute( + """ + UPDATE access_grants g + SET can_edit = false, can_share = false + WHERE g.grantee_subject_id = %s AND NOT g.is_exception AND (g.can_edit OR g.can_share) + AND ( + (g.resource_kind = 'case' AND g.resource_id IN (SELECT id FROM cases WHERE home_scope_id = %s)) + OR (g.resource_kind = 'matter' AND g.resource_id IN (SELECT id FROM matters WHERE home_scope_id = %s)) + OR (g.resource_kind = 'element' AND g.resource_id IN (SELECT id FROM elements WHERE home_scope_id = %s)) + OR (g.resource_kind = 'task' AND g.resource_id IN (SELECT id FROM tasks WHERE home_scope_id = %s)) + OR (g.resource_kind = 'deadline' AND g.resource_id IN (SELECT id FROM deadlines WHERE home_scope_id = %s)) + ) + """, + (subject_id, room_id, room_id, room_id, room_id, room_id), + ) + + +def _drop_shares_of(conn, room_id: UUID, subject_id: UUID) -> None: + """Whoever leaves a team, or may no longer write in it, takes the shares of their own entries with them.""" + conn.execute( + """ + DELETE FROM room_shares s + WHERE s.room_id = %s + AND ( + (s.resource_kind = 'case' AND s.resource_id IN (SELECT id FROM cases WHERE owner_subject_id = %s)) + OR (s.resource_kind = 'matter' AND s.resource_id IN (SELECT id FROM matters WHERE owner_subject_id = %s)) + ) + """, + (room_id, subject_id, subject_id), + ) + + +def _set_role(conn, actor: UUID, room_id: UUID, subject_id: UUID, role: str, *, added: bool) -> None: + if role not in ROOM_ROLES: + raise unprocessable("Diese Raumrolle gibt es nicht.") + conn.execute( + """ + INSERT INTO room_memberships (room_id, subject_id, role, status) + VALUES (%s, %s, %s, 'active') + ON CONFLICT (room_id, subject_id) DO UPDATE SET role = EXCLUDED.role, status = 'active' + """, + (room_id, subject_id, role), + ) + if role == "reader": + _drop_edit_rights_in_room(conn, subject_id, room_id) + _drop_shares_of(conn, room_id, subject_id) + _audit_membership(conn, actor, room_id, subject_id, "membership_added" if added else "membership_role_changed", role) + + +def _membership(conn, room_id: UUID, subject_id: UUID, *, lock: bool = False) -> dict | None: + suffix = " FOR UPDATE" if lock else "" + return conn.execute( + f"SELECT role, status FROM room_memberships WHERE room_id = %s AND subject_id = %s{suffix}", + (room_id, subject_id), + ).fetchone() + + +@router.get("/api/v1/rooms") +def rooms( + include_archived: bool = False, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + from app.auth.accounts import list_rooms + + items = list_rooms(conn, principal.account_id) + if include_archived: + archived = conn.execute( + """ + SELECT r.id, r.name, m.role, r.owner_id + FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' AND r.status = 'archived' + ORDER BY r.name, r.id + """, + (principal.account_id,), + ).fetchall() + items += [ + { + "id": str(row["id"]), + "kind": "organization", + "name": row["name"], + "role": row["role"], + "status": "archived", + "is_owner": row["owner_id"] == principal.account_id, + } + for row in archived + ] + return {"items": items, "next_cursor": None} + + +@router.post("/api/v1/rooms") +def create_room(body: RoomIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_main_user(conn, principal.account_id) + ensure_can_create_room(conn, principal.account_id) + room_id = uuid4() + name = body.name.strip() + conn.execute( + """ + INSERT INTO data_rooms (id, kind, name, created_by, owner_id, default_visibility, download_roles) + SELECT %s, 'organization', %s, %s, %s, %s, + jsonb_build_object( + 'manager', COALESCE((download_defaults ->> 'manager')::boolean, true), + 'member', COALESCE((download_defaults ->> 'member')::boolean, true), + 'reader', COALESCE((download_defaults ->> 'reader')::boolean, true) + ) + FROM auth_accounts WHERE id = %s + """, + (room_id, name, principal.account_id, principal.account_id, body.default_visibility, principal.account_id), + ) + conn.execute( + "INSERT INTO room_memberships (room_id, subject_id, role) VALUES (%s, %s, 'manager')", + (room_id, principal.account_id), + ) + audit(conn, principal.account_id, room_id, "room_created", "room", room_id) + return _created( + f"/api/v1/rooms/{room_id}", + {"id": str(room_id), "kind": "organization", "name": name, "default_visibility": body.default_visibility}, + ) + + +@router.patch("/api/v1/rooms/{room_id}") +def patch_room( + room_id: UUID, + body: RoomPatchIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_room_manager(conn, principal.account_id, room_id) + room = _room(conn, room_id) + if body.name is not None and body.name.strip() != room["name"]: + conn.execute("UPDATE data_rooms SET name = %s WHERE id = %s", (body.name.strip(), room_id)) + audit(conn, principal.account_id, room_id, "room_renamed", "room", room_id) + if body.default_visibility is not None and body.default_visibility != room["default_visibility"]: + _require_owner(conn, principal.account_id, room_id) + conn.execute("UPDATE data_rooms SET default_visibility = %s WHERE id = %s", (body.default_visibility, room_id)) + audit( + conn, + principal.account_id, + room_id, + "room_default_changed", + "room", + room_id, + detail={"visibility": body.default_visibility}, + ) + if body.status is not None and body.status != room["status"]: + _require_owner(conn, principal.account_id, room_id) + conn.execute("UPDATE data_rooms SET status = %s WHERE id = %s", (body.status, room_id)) + if body.status == "archived": + # Sessions working in the room return to their personal room. + conn.execute( + """ + UPDATE auth_sessions + SET scope_kind = 'personal', scope_id = account_id, epoch = epoch + 1 + WHERE scope_id = %s AND revoked_at IS NULL + """, + (room_id,), + ) + audit( + conn, + principal.account_id, + room_id, + "room_archived" if body.status == "archived" else "room_restored", + "room", + room_id, + ) + row = _room(conn, room_id) + return { + "id": str(row["id"]), + "name": row["name"], + "status": row["status"], + "default_visibility": row["default_visibility"], + } + + +def _set_owner(conn, actor: UUID, room_id: UUID, new_owner: UUID, previous: UUID | None) -> None: + conn.execute("UPDATE data_rooms SET owner_id = %s WHERE id = %s", (new_owner, room_id)) + conn.execute( + """ + INSERT INTO room_memberships (room_id, subject_id, role, status) + VALUES (%s, %s, 'manager', 'active') + ON CONFLICT (room_id, subject_id) DO UPDATE SET role = 'manager', status = 'active' + """, + (room_id, new_owner), + ) + audit( + conn, + actor, + room_id, + "room_owner_changed", + "room", + room_id, + detail={"from": str(previous) if previous else None, "to": str(new_owner)}, + ) + + +@router.post("/api/v1/rooms/{room_id}/owner") +def transfer_owner( + room_id: UUID, + body: OwnerIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """The owner hands the room to an active member. The new owner becomes manager.""" + room = _require_owner(conn, principal.account_id, room_id) + target = _membership(conn, room_id, body.subject_id, lock=True) + account = conn.execute("SELECT status FROM auth_accounts WHERE id = %s", (body.subject_id,)).fetchone() + if target is None or target["status"] != "active" or account is None or account["status"] != "active": + raise unprocessable("Der Datenraum geht nur an ein aktives Mitglied.") + if account_kind(conn, body.subject_id) != "main": + raise unprocessable("Ein Mitarbeiter-Konto kann keinen Datenraum besitzen.") + if body.subject_id == room["owner_id"]: + raise unprocessable("Diese Person ist bereits Eigentümer.") + _set_owner(conn, principal.account_id, room_id, body.subject_id, room["owner_id"]) + return {"owner_id": str(body.subject_id)} + + +@router.get("/api/v1/rooms/{room_id}/exposure") +def exposure(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + """Own cases and matters: shared with this room (`room`) or filed here but not shared (`private`).""" + if room_role(conn, principal.account_id, room_id) is None: + raise forbidden() + result: dict = {} + for table, kind in (("cases", "case"), ("matters", "matter")): + row = conn.execute( + f""" + SELECT + count(*) FILTER (WHERE EXISTS ( + SELECT 1 FROM room_shares s + WHERE s.resource_kind = %s AND s.resource_id = t.id AND s.room_id = %s)) AS shared, + count(*) FILTER (WHERE t.home_scope_id = %s AND NOT EXISTS ( + SELECT 1 FROM room_shares s + WHERE s.resource_kind = %s AND s.resource_id = t.id AND s.room_id = %s)) AS filed + FROM {table} t + WHERE t.owner_subject_id = %s AND t.deleted_at IS NULL + """, + (kind, room_id, room_id, kind, room_id, principal.account_id), + ).fetchone() + result[table] = {"room": row["shared"], "private": row["filed"]} + return JSONResponse(result, headers=_NO_STORE) + + +@router.post("/api/v1/rooms/{room_id}/my-content/visibility") +def set_my_content( + room_id: UUID, + body: VisibilityIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """`room`: share every own entry filed in this room with it. `private`: end all own shares to it.""" + role = room_role(conn, principal.account_id, room_id) + if role is None: + raise forbidden() + changed = 0 + if body.visibility == "room": + if role not in {"member", "manager"}: + raise forbidden("Als Leser kannst du nichts mit dem Datenraum teilen.") + for table, kind in (("cases", "case"), ("matters", "matter")): + rows = conn.execute( + f""" + INSERT INTO room_shares (resource_kind, resource_id, room_id, access, created_by) + SELECT %s, t.id, %s, 'edit', %s FROM {table} t + WHERE t.home_scope_id = %s AND t.owner_subject_id = %s AND t.deleted_at IS NULL + ON CONFLICT DO NOTHING + RETURNING id + """, + (kind, room_id, principal.account_id, room_id, principal.account_id), + ).fetchall() + changed += len(rows) + else: + for table, kind in (("cases", "case"), ("matters", "matter")): + rows = conn.execute( + f""" + DELETE FROM room_shares s + WHERE s.room_id = %s AND s.resource_kind = %s + AND s.resource_id IN (SELECT id FROM {table} WHERE owner_subject_id = %s) + RETURNING s.id + """, + (room_id, kind, principal.account_id), + ).fetchall() + changed += len(rows) + if changed: + audit( + conn, + principal.account_id, + room_id, + "visibility_changed", + "room", + room_id, + detail={"visibility": body.visibility, "count": changed}, + ) + return {"changed": changed} + + +@router.get("/api/v1/rooms/{room_id}/members") +def members(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_room_manager(conn, principal.account_id, room_id) + owner_id = _room(conn, room_id)["owner_id"] + rows = conn.execute( + """ + SELECT m.subject_id, m.role, a.email, a.display_name, a.status + FROM room_memberships m + JOIN auth_accounts a ON a.id = m.subject_id + WHERE m.room_id = %s AND m.status = 'active' + ORDER BY a.display_name, a.id + """, + (room_id,), + ).fetchall() + return { + "items": [ + { + "subject_id": str(row["subject_id"]), + "email": row["email"], + "display_name": row["display_name"], + "role": row["role"], + "account_status": row["status"], + "is_owner": row["subject_id"] == owner_id, + } + for row in rows + ] + } + + +@router.post("/api/v1/rooms/{room_id}/members") +def add_member( + room_id: UUID, + body: MemberIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_room_manager(conn, principal.account_id, room_id) + account = account_by_email(conn, body.email) + if account is None or account["status"] != "active": + raise not_found() + existing = _membership(conn, room_id, account["id"], lock=True) + if body.role != "manager": + _protect_owner(_room(conn, room_id), account["id"]) + if existing and existing["status"] == "active" and existing["role"] == "manager" and body.role != "manager": + if count_active_managers(conn, room_id, lock=True) <= 1: + raise unprocessable("Der letzte Verwalter bleibt Verwalter.") + was_active = bool(existing and existing["status"] == "active") + if not was_active: + ensure_can_add_member(conn, _room(conn, room_id)["owner_id"], room_id) + _set_role(conn, principal.account_id, room_id, account["id"], body.role, added=not was_active) + return {"subject_id": str(account["id"]), "role": body.role} + + +@router.patch("/api/v1/rooms/{room_id}/members/{subject_id}") +def change_member_role( + room_id: UUID, + subject_id: UUID, + body: MemberRoleIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_room_manager(conn, principal.account_id, room_id) + count_active_managers(conn, room_id, lock=True) + current = _membership(conn, room_id, subject_id, lock=True) + if current is None or current["status"] != "active": + raise not_found() + if current["role"] == body.role: + return {"subject_id": str(subject_id), "role": body.role} + _protect_owner(_room(conn, room_id), subject_id) + if current["role"] == "manager" and count_active_managers(conn, room_id) <= 1: + raise unprocessable("Der letzte Verwalter bleibt Verwalter.") + _set_role(conn, principal.account_id, room_id, subject_id, body.role, added=False) + return {"subject_id": str(subject_id), "role": body.role} + + +@router.delete("/api/v1/rooms/{room_id}/members/{subject_id}", status_code=204) +def remove_member( + room_id: UUID, + subject_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_room_manager(conn, principal.account_id, room_id) + count_active_managers(conn, room_id, lock=True) + current = _membership(conn, room_id, subject_id, lock=True) + if current is None or current["status"] != "active": + raise not_found() + _protect_owner(_room(conn, room_id), subject_id) + if current["role"] == "manager" and count_active_managers(conn, room_id) <= 1: + raise unprocessable("Der letzte Verwalter kann den Datenraum nicht verlassen.") + conn.execute( + "UPDATE room_memberships SET status = 'removed' WHERE room_id = %s AND subject_id = %s", + (room_id, subject_id), + ) + _drop_shares_of(conn, room_id, subject_id) + # The removed person works in the personal room again; own entries stay visible there. + conn.execute( + """ + UPDATE auth_sessions + SET scope_kind = 'personal', scope_id = account_id, epoch = epoch + 1 + WHERE account_id = %s AND scope_id = %s AND revoked_at IS NULL + """, + (subject_id, room_id), + ) + _audit_membership(conn, principal.account_id, room_id, subject_id, "membership_removed") + return Response(status_code=204) + + +@router.post("/api/v1/invitations") +def invite( + body: InvitationIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + # Only the owner, a main user, brings new people in; they become employee accounts. + room = _require_owner(conn, principal.account_id, body.room_id) + require_main_user(conn, principal.account_id) + ensure_can_add_member(conn, room["owner_id"], room["id"]) + email = normalize_email(body.email) + if account_by_email(conn, email): + raise unprocessable("Es gibt bereits ein Konto. Füge die Person als Mitglied hinzu.") + token = new_secret() + row = conn.execute( + """ + INSERT INTO auth_invitations (email, room_id, token_hash, invited_by, expires_at, role) + VALUES (%s, %s, %s, %s, %s, %s) + RETURNING id, expires_at + """, + (email, room["id"], hash_secret(token), principal.account_id, utcnow() + timedelta(days=INVITATION_DAYS), body.role), + ).fetchone() + audit(conn, principal.account_id, room["id"], "invitation_created", "invitation", row["id"], detail={"role": body.role}) + return _created( + f"/api/v1/invitations/{row['id']}", + {"id": str(row["id"]), "email": email, "role": body.role, "expires_at": dump_dt(row["expires_at"]), "token": token}, + ) + + +@router.get("/api/v1/rooms/{room_id}/invitations") +def invitations(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_room_manager(conn, principal.account_id, room_id) + rows = conn.execute( + """ + SELECT id, email, role, expires_at, created_at + FROM auth_invitations + WHERE room_id = %s AND accepted_at IS NULL AND revoked_at IS NULL AND expires_at > now() + ORDER BY created_at DESC, id + """, + (room_id,), + ).fetchall() + return JSONResponse( + { + "items": [ + { + "id": str(row["id"]), + "email": row["email"], + "role": row["role"], + "expires_at": dump_dt(row["expires_at"]), + "created_at": dump_dt(row["created_at"]), + } + for row in rows + ] + }, + headers=_NO_STORE, + ) + + +@router.delete("/api/v1/rooms/{room_id}/invitations/{invitation_id}", status_code=204) +def revoke_invitation( + room_id: UUID, + invitation_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_room_manager(conn, principal.account_id, room_id) + row = conn.execute( + """ + UPDATE auth_invitations SET revoked_at = now() + WHERE id = %s AND room_id = %s AND accepted_at IS NULL AND revoked_at IS NULL + RETURNING id + """, + (invitation_id, room_id), + ).fetchone() + if row is None: + raise not_found() + audit(conn, principal.account_id, room_id, "invitation_revoked", "invitation", invitation_id) + return Response(status_code=204) + + +@router.get("/api/v1/admin/rooms") +def admin_rooms(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + """Metadata only: names, state and who manages. No content.""" + require_operator(conn, principal.account_id) + rows = conn.execute( + """ + SELECT r.id, r.name, r.status, r.created_at, + (SELECT o.display_name FROM auth_accounts o WHERE o.id = r.owner_id) AS owner_name, + EXISTS ( + SELECT 1 FROM auth_accounts o + JOIN room_memberships om ON om.subject_id = o.id AND om.room_id = r.id AND om.status = 'active' + WHERE o.id = r.owner_id AND o.status = 'active' + ) AS owner_active, + (SELECT count(*) FROM room_memberships m WHERE m.room_id = r.id AND m.status = 'active') AS members, + COALESCE(( + SELECT json_agg(a.display_name ORDER BY a.display_name) + FROM room_memberships m JOIN auth_accounts a ON a.id = m.subject_id + WHERE m.room_id = r.id AND m.status = 'active' AND m.role = 'manager' AND a.status = 'active' + ), '[]'::json) AS managers + FROM data_rooms r + WHERE r.kind = 'organization' + ORDER BY r.name, r.id + """ + ).fetchall() + return { + "items": [ + { + "id": str(row["id"]), + "name": row["name"], + "status": row["status"], + "members": row["members"], + "owner": row["owner_name"], + "owner_active": row["owner_active"], + "managers": row["managers"], + "has_manager": bool(row["managers"]), + "created_at": dump_dt(row["created_at"]), + } + for row in rows + ], + "next_cursor": None, + } + + +@router.post("/api/v1/admin/rooms/{room_id}/managers") +def admin_assign_manager( + room_id: UUID, + body: OperatorManagerIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """Only for a room without an active manager, so it does not stay orphaned.""" + require_operator(conn, principal.account_id) + _room(conn, room_id) + if count_active_managers(conn, room_id, lock=True) > 0: + raise forbidden("Dieser Datenraum hat bereits einen Verwalter.") + account = account_by_email(conn, body.email) + if account is None or account["status"] != "active": + raise not_found() + existing = _membership(conn, room_id, account["id"], lock=True) + _set_role( + conn, + principal.account_id, + room_id, + account["id"], + "manager", + added=not (existing and existing["status"] == "active"), + ) + return {"subject_id": str(account["id"]), "role": "manager"} + + +@router.post("/api/v1/admin/rooms/{room_id}/owner") +def admin_set_owner( + room_id: UUID, + body: OperatorOwnerIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """The operator steps in only when the owner can no longer act: disabled or not a member any more.""" + require_operator(conn, principal.account_id) + room = _room(conn, room_id) + owner = conn.execute("SELECT status FROM auth_accounts WHERE id = %s", (room["owner_id"],)).fetchone() + active_owner = bool(owner and owner["status"] == "active" and room_role(conn, room["owner_id"], room_id)) + if active_owner: + raise forbidden("Der Eigentümer ist aktiv. Er überträgt den Datenraum selbst.") + account = account_by_email(conn, body.email) + if account is None or account["status"] != "active": + raise not_found() + if account_kind(conn, account["id"]) != "main": + raise unprocessable("Ein Mitarbeiter-Konto kann keinen Datenraum besitzen.") + _set_owner(conn, principal.account_id, room_id, account["id"], room["owner_id"]) + return {"owner_id": str(account["id"])} + + +_ADMIN_KINDS = ("account", "session", "room", "membership", "invitation") + + +@router.get("/api/v1/admin/events") +def admin_events( + cursor: int = 0, + limit: int = 50, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_operator(conn, principal.account_id) + limit = max(1, min(limit, 100)) + cursor = max(0, cursor) + rows = conn.execute( + """ + SELECT e.id, e.action, e.resource_kind, e.resource_id, e.occurred_at, e.result, e.scope_id, + actor.display_name AS actor_name, + subject.display_name AS subject_name, + room.name AS room_name + FROM domain_events e + LEFT JOIN auth_accounts actor ON actor.id = e.actor_subject_id + LEFT JOIN auth_accounts subject ON subject.id = e.resource_id AND e.resource_kind IN ('account', 'membership') + LEFT JOIN data_rooms room ON room.id = COALESCE(e.scope_id, CASE WHEN e.resource_kind = 'room' THEN e.resource_id END) + AND room.kind = 'organization' + WHERE e.resource_kind = ANY(%s) + ORDER BY e.occurred_at DESC, e.id DESC + OFFSET %s LIMIT %s + """, + (list(_ADMIN_KINDS), cursor, limit + 1), + ).fetchall() + page = rows[:limit] + return JSONResponse( + { + "items": [ + { + "id": str(row["id"]), + "action": row["action"], + "label": LABELS.get(row["action"], "Ereignis"), + "resource_kind": row["resource_kind"], + "actor": row["actor_name"], + "subject": row["subject_name"], + "room": row["room_name"], + "occurred_at": dump_dt(row["occurred_at"]), + "result": row["result"], + } + for row in page + ], + "next_cursor": cursor + limit if len(rows) > limit else None, + }, + headers=_NO_STORE, + ) diff --git a/backend/app/api/roomshare.py b/backend/app/api/roomshare.py new file mode 100644 index 0000000..c77c026 --- /dev/null +++ b/backend/app/api/roomshare.py @@ -0,0 +1,139 @@ +"""Room shares of a case or matter. A room is a team: the owner shares an entry with any of the +rooms he is a member of, independent of where the entry is filed.""" + +from __future__ import annotations + +from uuid import UUID + +from fastapi import APIRouter, Depends, Response +from fastapi.responses import JSONResponse +from psycopg import Connection + +from app.access.policy import ROOM_RIGHTS, Rights +from app.api.deps import Principal, get_conn, get_principal +from app.api.schemas import ExclusionIn, RoomShareIn +from app.library import room_effect +from app.ops import _shareable, set_person_exclusion, share_with_room, unshare_from_room + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} + + +def _capped(role: str, access: str) -> Rights: + rights = ROOM_RIGHTS.get(role, Rights()) + return Rights(read=rights.read) if access == "read" else rights + + +def _view(conn, actor: UUID, kind: str, resource_id: UUID) -> dict: + _shareable(conn, actor, kind, resource_id) + shares = conn.execute( + """ + SELECT s.room_id, s.access, r.name, r.status + FROM room_shares s JOIN data_rooms r ON r.id = s.room_id + WHERE s.resource_kind = %s AND s.resource_id = %s + ORDER BY r.name, r.id + """, + (kind, resource_id), + ).fetchall() + result = [] + for share in shares: + members = conn.execute( + """ + SELECT m.subject_id, m.role, a.display_name, + EXISTS ( + SELECT 1 FROM access_grants g + WHERE g.is_exception AND g.grantee_subject_id = m.subject_id + AND g.resource_kind = %s AND g.resource_id = %s AND g.context_kind IS NULL + ) AS excluded + FROM room_memberships m + JOIN auth_accounts a ON a.id = m.subject_id + WHERE m.room_id = %s AND m.status = 'active' AND m.subject_id <> %s + ORDER BY a.display_name, a.id + """, + (kind, resource_id, share["room_id"], actor), + ).fetchall() + result.append( + { + "room_id": str(share["room_id"]), + "room_name": share["name"], + "archived": share["status"] != "active", + "access": share["access"], + "members": [ + { + "subject_id": str(row["subject_id"]), + "display_name": row["display_name"], + "role": row["role"], + "excluded": row["excluded"], + "rights": { + "read": _capped(row["role"], share["access"]).read, + "edit": _capped(row["role"], share["access"]).edit, + }, + } + for row in members + ], + } + ) + shared_ids = {item["room_id"] for item in result} + candidates = conn.execute( + """ + SELECT r.id, r.name, m.role + FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND m.role IN ('member', 'manager') + AND r.kind = 'organization' AND r.status = 'active' + ORDER BY r.name, r.id + """, + (actor,), + ).fetchall() + return { + "id": str(resource_id), + "shares": result, + "available": [ + {"id": str(row["id"]), "name": row["name"]} for row in candidates if str(row["id"]) not in shared_ids + ], + } + + +def _register(kind: str, plural: str) -> None: + base = f"/api/v1/{plural}/{{resource_id}}/room-share" + + @router.get(base, name=f"get_{kind}_room_share") + def get_share(resource_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + return JSONResponse(_view(conn, principal.account_id, kind, resource_id), headers=_NO_STORE) + + @router.put(base + "/rooms/{room_id}", name=f"put_{kind}_room_share") + def put_share( + resource_id: UUID, + room_id: UUID, + body: RoomShareIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), + ): + shared = share_with_room(conn, principal.account_id, kind, resource_id, room_id, body.access) + shared["effect"] = room_effect(conn, room_id, principal.account_id, body.access) + return shared + + @router.delete(base + "/rooms/{room_id}", status_code=204, name=f"delete_{kind}_room_share") + def delete_share( + resource_id: UUID, + room_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), + ): + unshare_from_room(conn, principal.account_id, kind, resource_id, room_id) + return Response(status_code=204) + + @router.put(base + "/people/{subject_id}", name=f"put_{kind}_room_share_person") + def put_person( + resource_id: UUID, + subject_id: UUID, + body: ExclusionIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), + ): + set_person_exclusion(conn, principal.account_id, kind, resource_id, subject_id, body.excluded) + return {"subject_id": str(subject_id), "excluded": body.excluded} + + +_register("case", "cases") +_register("matter", "matters") diff --git a/backend/app/api/routes.py b/backend/app/api/routes.py index 9c8018d..b40adab 100644 --- a/backend/app/api/routes.py +++ b/backend/app/api/routes.py @@ -12,6 +12,7 @@ from psycopg import Connection from app.access.policy import ( Ctx, account_by_email, + audit, case_rights, deadline_rights, element_rights, @@ -31,10 +32,10 @@ from app.access.policy import ( require_matter, task_rights, ) +from app.access.limits import ensure_can_add_member from app.api.deps import Principal, get_conn, get_principal, get_session from app.api.schemas import ( AcceptInviteIn, - AccountIn, AnnotationIn, BootstrapIn, AssigneeIn, @@ -60,18 +61,15 @@ from app.api.schemas import ( ExternalIn, FulfillIn, GrantIn, - InvitationIn, LoginIn, MatterIn, MatterPatch, - MemberIn, BindingOrderIn, ContentOrderIn, ChapterMoveIn, DatesPatch, NoteIn, NotePatch, - RoomIn, ScopeIn, TaskDeadlineIn, ElementReferenceIn, @@ -82,19 +80,31 @@ from app.api.schemas import ( TaskIn, TaskPatch, ) -from app.auth.accounts import create_account, list_rooms, normalize_email, start_session +from app.auth.accounts import ( + clear_login_attempts, + create_account, + initial_scope, + list_rooms, + login_blocked, + normalize_email, + record_failed_login, + start_session, +) from app.auth.security import ( clear_session_cookie, + device_label, hash_secret, + login_bucket, new_secret, session_expiry, set_session_cookie, verify_password, + verify_unknown_account, ) -from app.core.errors import Problem, forbidden, not_found, too_many, unprocessable +from app.core.errors import DOWNLOAD_REFUSED, Problem, forbidden, not_found, too_many, unprocessable from app.core.timeutil import dump_date, dump_dt, utcnow -from app.media.storage import content_disposition, read_bytes -from app.media.storage import asset_path +from app.media.port import read_clear_bytes +from app.media.storage import content_disposition from app.ops import ( add_annotation, assign_task, @@ -157,10 +167,11 @@ from app.present import ( note_update_payload, task_payload, ) +from app.library import case_items, matter_items +from app.ops import share_with_room as _share_with_room from app.seed import demo_public router = APIRouter() -_login_attempts: dict[str, list[datetime]] = {} def _chosen_home(conn, principal: Principal, requested: UUID | None) -> UUID: @@ -192,7 +203,7 @@ def _chapter_payload(row) -> dict: def _session_body(conn, account_id: UUID, scope_kind: str, scope_id: UUID, csrf: str, epoch: int) -> dict: account = conn.execute( - "SELECT id, email, display_name, is_superadmin FROM auth_accounts WHERE id = %s", + "SELECT id, email, display_name, is_superadmin, must_change_password, account_kind FROM auth_accounts WHERE id = %s", (account_id,), ).fetchone() room = conn.execute("SELECT id, kind, name FROM data_rooms WHERE id = %s", (scope_id,)).fetchone() @@ -202,6 +213,8 @@ def _session_body(conn, account_id: UUID, scope_kind: str, scope_id: UUID, csrf: "email": account["email"], "display_name": account["display_name"], "is_superadmin": account["is_superadmin"], + "must_change_password": account["must_change_password"], + "account_kind": account["account_kind"], }, "scope": {"kind": scope_kind, "id": str(room["id"]), "name": room["name"]}, "rooms": list_rooms(conn, account_id), @@ -210,29 +223,30 @@ def _session_body(conn, account_id: UUID, scope_kind: str, scope_id: UUID, csrf: } -def _limit_login(key: str) -> None: - now = utcnow() - window = [item for item in _login_attempts.get(key, []) if now - item < timedelta(minutes=10)] - if len(window) >= 8: - _login_attempts[key] = window - raise too_many() - window.append(now) - _login_attempts[key] = window - - @router.post("/api/v1/auth/login") def login(body: LoginIn, request: Request, conn: Connection = Depends(get_conn)): email = normalize_email(body.email) - _limit_login(f"{request.client.host if request.client else 'local'}:{email}") + bucket = login_bucket(request.client.host if request.client else "local", email) + if login_blocked(conn, bucket): + raise too_many() account = conn.execute( "SELECT * FROM auth_accounts WHERE email = %s", (email,), ).fetchone() - if account is None or account["status"] != "active" or not verify_password(body.password, account["password_hash"]): + if account is None or account["status"] != "active": + # Same time and same answer as a wrong password. + verify_unknown_account(body.password) + valid = False + else: + valid = verify_password(body.password, account["password_hash"]) + if not valid: + record_failed_login(conn, bucket) raise Problem(401, "Anmeldung fehlgeschlagen", "E-Mail oder Passwort ist nicht richtig.", "unauthorized") + clear_login_attempts(conn, bucket) settings = request.app.state.settings - token, csrf = start_session(conn, settings, account["id"]) - payload = _session_body(conn, account["id"], "personal", account["id"], csrf, 1) + token, csrf = start_session(conn, settings, account["id"], device_label(request.headers.get("user-agent"))) + scope_kind, scope_id = initial_scope(conn, account["id"]) + payload = _session_body(conn, account["id"], scope_kind, scope_id, csrf, 1) response = JSONResponse(payload) set_session_cookie(response, request, settings, token) return response @@ -241,6 +255,7 @@ def login(body: LoginIn, request: Request, conn: Connection = Depends(get_conn)) @router.post("/api/v1/auth/logout", status_code=204) def logout(request: Request, session: dict = Depends(get_session), conn: Connection = Depends(get_conn)): conn.execute("UPDATE auth_sessions SET revoked_at = now() WHERE id = %s", (session["id"],)) + audit(conn, session["account_id"], None, "session_ended", "session", session["id"]) response = Response(status_code=204) clear_session_cookie(response, request, request.app.state.settings) return response @@ -303,99 +318,13 @@ def bootstrap(body: BootstrapIn, request: Request, conn: Connection = Depends(ge raise not_found() account = create_account(conn, body.email, body.display_name, body.password, is_superadmin=True) settings = request.app.state.settings - token, csrf = start_session(conn, settings, account["id"]) + token, csrf = start_session(conn, settings, account["id"], device_label(request.headers.get("user-agent"))) payload = _session_body(conn, account["id"], "personal", account["id"], csrf, 1) response = JSONResponse(status_code=201, content=payload) set_session_cookie(response, request, settings, token) return response -def _require_superadmin(conn: Connection, principal: Principal) -> None: - row = conn.execute( - "SELECT is_superadmin FROM auth_accounts WHERE id = %s", - (principal.account_id,), - ).fetchone() - if row is None or not row["is_superadmin"]: - raise forbidden("Nur der Betreiber kann Nutzer anlegen.") - - -@router.get("/api/v1/accounts") -def accounts(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - _require_superadmin(conn, principal) - rows = conn.execute( - """ - SELECT id, email, display_name, is_superadmin, created_at - FROM auth_accounts - WHERE origin = 'operator' AND status = 'active' - ORDER BY display_name, email - """ - ).fetchall() - return { - "items": [ - { - "id": str(row["id"]), - "email": row["email"], - "display_name": row["display_name"], - "is_superadmin": row["is_superadmin"], - "created_at": dump_dt(row["created_at"]), - } - for row in rows - ], - "next_cursor": None, - } - - -@router.post("/api/v1/accounts") -def post_account(body: AccountIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - _require_superadmin(conn, principal) - account = create_account(conn, body.email, body.display_name, body.password) - return _created( - f"/api/v1/accounts/{account['id']}", - { - "id": str(account["id"]), - "email": account["email"], - "display_name": account["display_name"], - "is_superadmin": account["is_superadmin"], - }, - ) - - -@router.post("/api/v1/invitations") -def invite( - body: InvitationIn, - principal: Principal = Depends(get_principal), - conn: Connection = Depends(get_conn), -): - room = conn.execute("SELECT * FROM data_rooms WHERE id = %s", (body.room_id,)).fetchone() - if room is None or room["kind"] != "organization": - raise not_found() - membership = conn.execute( - """ - SELECT role FROM room_memberships - WHERE room_id = %s AND subject_id = %s AND status = 'active' - """, - (room["id"], principal.account_id), - ).fetchone() - if membership is None or membership["role"] != "manager": - raise forbidden() - email = normalize_email(body.email) - if account_by_email(conn, email): - raise unprocessable("Es gibt bereits ein Konto. Füge die Person als Mitglied hinzu.") - token = new_secret() - row = conn.execute( - """ - INSERT INTO auth_invitations (email, room_id, token_hash, invited_by, expires_at) - VALUES (%s, %s, %s, %s, %s) - RETURNING id, expires_at - """, - (email, room["id"], hash_secret(token), principal.account_id, utcnow() + timedelta(days=7)), - ).fetchone() - return _created( - f"/api/v1/invitations/{row['id']}", - {"id": str(row["id"]), "email": email, "expires_at": row["expires_at"].isoformat(), "token": token}, - ) - - @router.post("/api/v1/invitations/{invitation_id}/accept") def accept_invite( invitation_id: UUID, @@ -404,143 +333,46 @@ def accept_invite( conn: Connection = Depends(get_conn), ): row = conn.execute("SELECT * FROM auth_invitations WHERE id = %s", (invitation_id,)).fetchone() - if row is None or row["accepted_at"] is not None or row["expires_at"] <= utcnow(): + if ( + row is None + or row["accepted_at"] is not None + or row["revoked_at"] is not None + or row["expires_at"] <= utcnow() + ): raise not_found() if row["token_hash"] != hash_secret(body.token): raise not_found() - account = create_account(conn, row["email"], body.display_name, body.password) + room = conn.execute("SELECT owner_id FROM data_rooms WHERE id = %s AND status = 'active'", (row["room_id"],)).fetchone() + if room is None: + raise not_found() + ensure_can_add_member(conn, room["owner_id"], row["room_id"]) + # An invitation creates an employee account. The owner of the room is the sponsor. + account = create_account( + conn, + row["email"], + body.display_name, + body.password, + account_kind="employee", + sponsor_id=room["owner_id"], + ) conn.execute( """ INSERT INTO room_memberships (room_id, subject_id, role) - VALUES (%s, %s, 'member') + VALUES (%s, %s, %s) """, - (row["room_id"], account["id"]), + (row["room_id"], account["id"], row["role"]), ) conn.execute("UPDATE auth_invitations SET accepted_at = now() WHERE id = %s", (invitation_id,)) + audit(conn, account["id"], row["room_id"], "invitation_accepted", "invitation", invitation_id) settings = request.app.state.settings - token, csrf = start_session(conn, settings, account["id"]) - payload = _session_body(conn, account["id"], "personal", account["id"], csrf, 1) + token, csrf = start_session(conn, settings, account["id"], device_label(request.headers.get("user-agent"))) + scope_kind, scope_id = initial_scope(conn, account["id"]) + payload = _session_body(conn, account["id"], scope_kind, scope_id, csrf, 1) response = JSONResponse(status_code=201, content=payload) set_session_cookie(response, request, settings, token) return response -@router.get("/api/v1/rooms") -def rooms(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - return {"items": list_rooms(conn, principal.account_id), "next_cursor": None} - - -@router.post("/api/v1/rooms") -def create_room(body: RoomIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - from uuid import uuid4 - - room_id = uuid4() - conn.execute( - """ - INSERT INTO data_rooms (id, kind, name, created_by) - VALUES (%s, 'organization', %s, %s) - """, - (room_id, body.name.strip(), principal.account_id), - ) - conn.execute( - """ - INSERT INTO room_memberships (room_id, subject_id, role) - VALUES (%s, %s, 'manager') - """, - (room_id, principal.account_id), - ) - return _created(f"/api/v1/rooms/{room_id}", {"id": str(room_id), "kind": "organization", "name": body.name.strip()}) - - -@router.get("/api/v1/rooms/{room_id}/members") -def members(room_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - _require_manager(conn, principal, room_id) - rows = conn.execute( - """ - SELECT m.subject_id, m.role, a.email, a.display_name - FROM room_memberships m - JOIN auth_accounts a ON a.id = m.subject_id - WHERE m.room_id = %s AND m.status = 'active' - ORDER BY a.display_name, a.id - """, - (room_id,), - ).fetchall() - return { - "items": [ - { - "subject_id": str(row["subject_id"]), - "email": row["email"], - "display_name": row["display_name"], - "role": row["role"], - } - for row in rows - ] - } - - -@router.post("/api/v1/rooms/{room_id}/members") -def add_member( - room_id: UUID, - body: MemberIn, - principal: Principal = Depends(get_principal), - conn: Connection = Depends(get_conn), -): - _require_manager(conn, principal, room_id) - account = account_by_email(conn, body.email) - if account is None or account["status"] != "active": - raise not_found() - conn.execute( - """ - INSERT INTO room_memberships (room_id, subject_id, role, status) - VALUES (%s, %s, %s, 'active') - ON CONFLICT (room_id, subject_id) DO UPDATE SET role = EXCLUDED.role, status = 'active' - """, - (room_id, account["id"], body.role), - ) - return {"subject_id": str(account["id"]), "role": body.role} - - -@router.delete("/api/v1/rooms/{room_id}/members/{subject_id}", status_code=204) -def remove_member( - room_id: UUID, - subject_id: UUID, - principal: Principal = Depends(get_principal), - conn: Connection = Depends(get_conn), -): - _require_manager(conn, principal, room_id) - if subject_id == principal.account_id: - managers = conn.execute( - """ - SELECT count(*) AS n FROM room_memberships - WHERE room_id = %s AND role = 'manager' AND status = 'active' - """, - (room_id,), - ).fetchone() - if managers["n"] <= 1: - raise unprocessable("Der letzte Verwalter kann den Datenraum nicht verlassen.") - conn.execute( - """ - UPDATE room_memberships SET status = 'removed' - WHERE room_id = %s AND subject_id = %s - """, - (room_id, subject_id), - ) - return Response(status_code=204) - - -def _require_manager(conn, principal: Principal, room_id: UUID) -> None: - membership = conn.execute( - """ - SELECT role FROM room_memberships - WHERE room_id = %s AND subject_id = %s AND status = 'active' - """, - (room_id, principal.account_id), - ).fetchone() - room = conn.execute("SELECT kind FROM data_rooms WHERE id = %s", (room_id,)).fetchone() - if room is None or room["kind"] != "organization" or membership is None or membership["role"] != "manager": - raise forbidden() - - @router.get("/api/v1/principals") def find_principal(email: str, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): account = account_by_email(conn, email) @@ -551,18 +383,7 @@ def find_principal(email: str, principal: Principal = Depends(get_principal), co @router.get("/api/v1/cases") def cases(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - items = [] - for case, rights in list_cases(conn, principal.account_id): - summary = case_summary(case, rights) - summary["home_scope_name"] = room_name(conn, case["home_scope_id"]) - summary["kind_label"] = "Unterakte" if case["parent_case_id"] else "Akte" - summary["parent_title"] = None - if case["parent_case_id"]: - parent = get_case(conn, case["parent_case_id"]) - if parent is not None and case_rights(conn, principal.account_id, parent).read: - summary["parent_title"] = parent["title"] - items.append(summary) - return {"items": items, "next_cursor": None} + return {"items": case_items(conn, principal.account_id, list_cases(conn, principal.account_id)), "next_cursor": None} @router.post("/api/v1/cases") @@ -575,6 +396,8 @@ def post_case(body: CaseIn, principal: Principal = Depends(get_principal), conn: body.parent_case_id, body.rights_mode, ) + if body.share_with_room_id is not None: + _share_with_room(conn, principal.account_id, "case", case["id"], body.share_with_room_id, "edit") rights = case_rights(conn, principal.account_id, case) return _created(f"/api/v1/cases/{case['id']}", case_summary(case, rights)) @@ -659,8 +482,7 @@ def post_case_grant( @router.get("/api/v1/matters") def matters(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - items = [matter_summary(conn, principal.account_id, matter) for matter, _contexts in list_matters(conn, principal.account_id)] - return {"items": items, "next_cursor": None} + return {"items": matter_items(conn, principal.account_id, list_matters(conn, principal.account_id)), "next_cursor": None} @router.post("/api/v1/matters") @@ -673,6 +495,8 @@ def post_matter(body: MatterIn, principal: Principal = Depends(get_principal), c body.parent_matter_id, _ctx_model(body.context), ) + if body.share_with_room_id is not None: + _share_with_room(conn, principal.account_id, "matter", matter["id"], body.share_with_room_id, "edit") return _created(f"/api/v1/matters/{matter['id']}", matter_summary(conn, principal.account_id, matter)) @@ -1108,6 +932,7 @@ def download_file( request: Request, context_kind: str | None = None, context_id: str | None = None, + save: bool = False, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn), ): @@ -1118,13 +943,27 @@ def download_file( rights = element_rights(conn, principal.account_id, element, context) if not rights.read: raise not_found() + if save and not rights.download: + raise forbidden(DOWNLOAD_REFUSED) asset = conn.execute( - "SELECT id, original_filename, content_type, status FROM media_assets WHERE element_id = %s", + """ + SELECT id, original_filename, content_type, sha256, status, + storage_instance_id, storage_generation, object_key, protection_profile + FROM media_assets WHERE element_id = %s + """, (element_id,), ).fetchone() if asset is None or asset["status"] != "active": raise not_found() - payload = read_bytes(asset_path(request.app.state.settings.media_root, asset["id"])) + payload = read_clear_bytes( + conn, + instance_id=asset["storage_instance_id"], + generation=asset["storage_generation"], + object_key=asset["object_key"] or str(asset["id"]), + media_root=request.app.state.settings.media_root, + protection_profile=asset["protection_profile"], + expected_sha256=asset["sha256"], + ) return Response( content=payload, media_type=asset["content_type"], @@ -1343,37 +1182,61 @@ def post_fulfill( return deadline_payload(conn, principal.account_id, deadline) +VIEW_LIMIT = 200 +SEARCH_LIMIT = 50 +SCAN_BATCH = 200 +SCAN_CAP = 20000 + + +def _scan(conn, sql: str, params: tuple = ()): + """Rows in the order of the query, read in batches. + + The rights are checked per row after reading. A fixed LIMIT in the query would count rows + of other people too, so the own ones could fall off the end. The reader stops once it has + enough. `SCAN_CAP` bounds the work on very large data sets. + """ + offset = 0 + while offset < SCAN_CAP: + rows = conn.execute(f"{sql} LIMIT %s OFFSET %s", (*params, SCAN_BATCH, offset)).fetchall() + yield from rows + if len(rows) < SCAN_BATCH: + return + offset += SCAN_BATCH + + @router.get("/api/v1/views/open-tasks") def open_tasks(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - rows = conn.execute( + items = [] + for task in _scan( + conn, """ SELECT * FROM tasks WHERE deleted_at IS NULL AND task_status = 'open' ORDER BY due_on NULLS LAST, created_at, id - LIMIT 200 - """ - ).fetchall() - items = [] - for task in rows: + """, + ): if _task_visible(conn, principal.account_id, task): items.append(task_payload(conn, principal.account_id, task)) + if len(items) >= VIEW_LIMIT: + break return {"items": items, "next_cursor": None} @router.get("/api/v1/views/open-deadlines") def open_deadlines(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - rows = conn.execute( + items = [] + for deadline in _scan( + conn, """ SELECT * FROM deadlines WHERE deleted_at IS NULL AND deadline_status = 'open' ORDER BY due_on, created_at, id - LIMIT 200 - """ - ).fetchall() - items = [] - for deadline in rows: + """, + ): if _deadline_visible(conn, principal.account_id, deadline): items.append(deadline_payload(conn, principal.account_id, deadline)) + if len(items) >= VIEW_LIMIT: + break return {"items": items, "next_cursor": None} @@ -1427,19 +1290,21 @@ def search(q: str, principal: Principal = Depends(get_principal), conn: Connecti ], } ) - elements = conn.execute( + elements = _scan( + conn, """ SELECT e.* FROM elements e LEFT JOIN note_bodies n ON n.element_id = e.id WHERE e.deleted_at IS NULL AND (e.title ILIKE %s ESCAPE '\\' OR COALESCE(n.document_json::text, '') ILIKE %s ESCAPE '\\') - ORDER BY e.updated_at DESC - LIMIT 100 + ORDER BY e.updated_at DESC, e.id """, (pattern, pattern), - ).fetchall() + ) for element in elements: + if len(items) >= SEARCH_LIMIT: + break from app.access.policy import visible_bindings_for_element bindings = visible_bindings_for_element(conn, principal.account_id, element) @@ -1467,7 +1332,7 @@ def search(q: str, principal: Principal = Depends(get_principal), conn: Connecti ], } ) - return {"items": items[:50], "next_cursor": None} + return {"items": items[:SEARCH_LIMIT], "next_cursor": None} @router.patch("/api/v1/cases/{case_id}/content-order") diff --git a/backend/app/api/schemas.py b/backend/app/api/schemas.py index f6e6e89..9aba820 100644 --- a/backend/app/api/schemas.py +++ b/backend/app/api/schemas.py @@ -27,9 +27,28 @@ class BootstrapIn(APIModel): class AccountIn(APIModel): + """Without a password the server generates a one-time password.""" + email: Email display_name: str = Field(min_length=1, max_length=120) - password: str = Field(min_length=10, max_length=200) + password: str | None = Field(default=None, min_length=10, max_length=200) + is_operator: bool = False + + +class AccountPatchIn(APIModel): + display_name: str | None = Field(default=None, min_length=1, max_length=120) + is_operator: bool | None = None + status: Literal["active", "disabled"] | None = None + account_kind: Literal["main"] | None = None + + +class TeamPatchIn(APIModel): + status: Literal["active", "disabled"] + + +class PasswordChangeIn(APIModel): + current_password: str = Field(min_length=1, max_length=200) + new_password: str = Field(min_length=10, max_length=200) class ScopeIn(APIModel): @@ -46,15 +65,61 @@ class AcceptInviteIn(APIModel): class InvitationIn(APIModel): email: Email room_id: UUID + role: Literal["member", "manager", "reader"] = "member" class RoomIn(APIModel): name: str = Field(min_length=1, max_length=160) + default_visibility: Literal["private", "room"] = "room" + + +class VisibilityIn(APIModel): + visibility: Literal["private", "room"] + + +class ShareItemIn(APIModel): + kind: Literal["case", "matter"] + id: UUID + + +class BatchShareIn(APIModel): + items: list[ShareItemIn] = Field(min_length=1, max_length=200) + access: Literal["read", "edit"] = "edit" + + +class RoomShareIn(APIModel): + access: Literal["read", "edit"] = "edit" + + +class ExclusionIn(APIModel): + excluded: bool + + +class OwnerIn(APIModel): + subject_id: UUID + + +class OperatorOwnerIn(APIModel): + email: Email class MemberIn(APIModel): email: Email - role: Literal["member", "manager"] + role: Literal["member", "manager", "reader"] + + +class MemberRoleIn(APIModel): + role: Literal["member", "manager", "reader"] + + +class RoomPatchIn(APIModel): + name: str | None = Field(default=None, min_length=1, max_length=160) + status: Literal["active", "archived"] | None = None + default_visibility: Literal["private", "room"] | None = None + + +class OperatorManagerIn(APIModel): + email: Email class ContextIn(APIModel): @@ -67,6 +132,7 @@ class CaseIn(APIModel): parent_case_id: UUID | None = None rights_mode: Literal["inherit_parent", "explicit"] = "inherit_parent" home_scope_id: UUID | None = None + share_with_room_id: UUID | None = None class CasePatch(APIModel): @@ -142,6 +208,7 @@ class MatterIn(APIModel): parent_matter_id: UUID | None = None context: ContextIn | None = None home_scope_id: UUID | None = None + share_with_room_id: UUID | None = None class MatterPatch(APIModel): @@ -278,7 +345,7 @@ class ExternalDetail(APIModel): class ExternalIn(APIModel): title: str = Field(min_length=1, max_length=200) - source: Literal["paperless", "url", "imap"] + source: str = Field(min_length=1, max_length=40) external_id: str = Field(min_length=1, max_length=200) context: ContextIn placement: ContextIn | None = None @@ -288,7 +355,13 @@ class ExternalIn(APIModel): class ConnectorIn(APIModel): - kind: Literal["paperless", "imap"] + """Kind is checked against the source registry. Source fields stay optional. + + Paperless and IMAP still send their existing fields. A further module + uses ``config`` and validates that bag itself. + """ + + kind: str = Field(min_length=1, max_length=40) name: str = Field(min_length=1, max_length=120) base_url: str | None = None token: str | None = None @@ -298,6 +371,11 @@ class ConnectorIn(APIModel): password: str | None = None mailbox: str | None = Field(default=None, max_length=200) security: Literal["ssl", "starttls"] | None = None + config: dict[str, str | int | None] | None = None + + +class SourceAssignmentIn(APIModel): + connector_id: UUID class DeleteElementIn(APIModel): @@ -441,3 +519,12 @@ class DispatchCorrectIn(APIModel): clear_sent_on: bool = False counterpart: str | None = Field(default=None, max_length=200) clear_counterpart: bool = False + + +class DownloadPolicyIn(APIModel): + """Who may save or open the original outside of Tsumugu. `direct` only applies to the account default.""" + + manager: bool + member: bool + reader: bool + direct: bool | None = None diff --git a/backend/app/api/sources.py b/backend/app/api/sources.py index 1a40e65..aa0d4b0 100644 --- a/backend/app/api/sources.py +++ b/backend/app/api/sources.py @@ -8,26 +8,30 @@ from fastapi import APIRouter, Depends from fastapi.responses import JSONResponse, Response from psycopg import Connection -from app.access.policy import element_rights, get_element, parse_query_context +from app.access.policy import Ctx, element_rights, get_element, parse_query_context from app.api.deps import Principal, get_conn, get_principal from app.api.routes import _created, _ctx_model -from app.api.schemas import ConnectorIn, ExternalIn -from app.core.errors import not_found, unprocessable -from app.mailpack import parse_mail +from app.api.schemas import ConnectorIn, ExternalIn, SourceAssignmentIn +from app.core.errors import DOWNLOAD_REFUSED, forbidden, not_found, unprocessable, unsupported +from app.mailpack import _inline_images, parse_mail, sanitize_html +from app.remoteimages import RemoteImages from app.media.storage import content_disposition from app.ops import create_external +from app.retrieval import RetrievalFailure +from app.sourcebind import assign_reference +from app.sourceport import capability_payload, with_capabilities +from app.sourceregistry import get_source, require_source from app.sources import ( - clean_base_url, + connector_detail, delete_connector, - imap_mailboxes, - imap_messages, imap_preview, list_connectors, - paperless_documents, + owned_connector, + paperless_content, + parse_connector, probe, - read_linked_mail, - read_linked_paperless, save_connector, + update_connector, ) _PRIVATE = { @@ -44,13 +48,38 @@ def get_connectors(principal: Principal = Depends(get_principal), conn: Connecti return {"items": list_connectors(conn, principal.account_id)} +@router.get("/api/v1/sources/capabilities") +def source_capabilities(principal: Principal = Depends(get_principal)): + del principal + return capability_payload() + + @router.post("/api/v1/connectors", status_code=201) def post_connector(body: ConnectorIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - name, settings, secret = _connector_parts(body) + name, settings, secret = _connector_parts(body, None) row = save_connector(conn, principal.account_id, body.kind, name, settings, secret) return row +@router.get("/api/v1/connectors/{connector_id}") +def get_connector(connector_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + return connector_detail(conn, principal.account_id, connector_id) + + +@router.patch("/api/v1/connectors/{connector_id}") +def patch_connector( + connector_id: UUID, + body: ConnectorIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + current = owned_connector(conn, principal.account_id, connector_id) + if current["kind"] != body.kind: + raise unprocessable("Der Zugang bleibt bei seiner Quellenart.") + name, settings, secret = _connector_parts(body, current) + return update_connector(conn, principal.account_id, connector_id, name, settings, secret) + + @router.delete("/api/v1/connectors/{connector_id}", status_code=204) def remove_connector(connector_id: UUID, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): delete_connector(conn, principal.account_id, connector_id) @@ -62,24 +91,27 @@ def probe_connector(connector_id: UUID, principal: Principal = Depends(get_princ @router.get("/api/v1/external-refs/{element_id}/content") -def view_paperless( +def view_external_content( element_id: UUID, context_kind: str | None = None, context_id: str | None = None, + save: bool = False, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn), ): element, context, ref = _opened_ref(conn, principal, element_id, context_kind, context_id) - if ref is None or ref["source"] != "paperless": - raise unprocessable("Nur ein Paperless-Dokument lässt sich hier als Datei ansehen.") - data, media_type, filename = read_linked_paperless(conn, principal.account_id, element, context, ref) - safe_name = (filename or "dokument.pdf").replace('"', "") + if save: + _require_download(conn, principal, element, context) + module = get_source(ref["source"]) if ref is not None else None + if module is None or module.open_original is None: + raise unprocessable("Für diese Quelle ist kein Dateiabruf vorgesehen.") + data, media_type, filename = module.open_original(conn, principal.account_id, element, context, ref) return Response( content=data, media_type=media_type, headers={ **_PRIVATE, - "Content-Disposition": f'inline; filename="{safe_name}"', + "Content-Disposition": content_disposition(filename or "dokument.pdf").replace("attachment;", "inline;", 1), }, ) @@ -101,14 +133,19 @@ def view_mail( "message_to": parsed.recipient, "message_cc": parsed.copy, "text": parsed.text, - "text_truncated": False, - "complete": True, + "html": parsed.html, + "remote_images": len(parsed.remote_images), + "text_truncated": parsed.text_truncated, + "complete": not parsed.missing, + "notices": parsed.notices, "attachments": [ { "index": index, - "filename": item.filename, + "filename": item.display_name or item.filename, + "archive_name": item.filename, "content_type": item.media_type, "byte_size": len(item.data), + "inline": item.inline, } for index, item in enumerate(parsed.attachments) ], @@ -117,6 +154,11 @@ def view_mail( ) +def _require_download(conn, principal: Principal, element, context) -> None: + if not element_rights(conn, principal.account_id, element, context).download: + raise forbidden(DOWNLOAD_REFUSED) + + def _opened_ref(conn, principal: Principal, element_id: UUID, context_kind: str | None, context_id: str | None): element = get_element(conn, element_id) if element is None or element["kind"] != "external_ref": @@ -126,14 +168,23 @@ def _opened_ref(conn, principal: Principal, element_id: UUID, context_kind: str if not rights.read: raise not_found() ref = conn.execute( - "SELECT source, external_id, detail FROM external_references WHERE element_id = %s", + """ + SELECT source, external_id, detail, connector_id, resolution + FROM external_references WHERE element_id = %s + """, (element_id,), ).fetchone() return element, context, ref def _complete_mail(conn, principal: Principal, element, context, ref): - raw = read_linked_mail(conn, principal.account_id, element, context, ref) + module = get_source(ref["source"]) if ref is not None else None + if module is None or module.representation != "email": + raise unprocessable("Das ist keine E-Mail.") + opened = module.open_original(conn, principal.account_id, element, context, ref) + if isinstance(opened, RetrievalFailure): + raise unprocessable(opened.message) + raw = opened[0] if isinstance(opened, tuple) else opened parsed = parse_mail(raw) if parsed.missing: names = ", ".join(parsed.missing[:5]) @@ -141,6 +192,25 @@ def _complete_mail(conn, principal: Principal, element, context, ref): return parsed, raw +@router.get("/api/v1/external-refs/{element_id}/mail/html-with-images") +def view_mail_with_images( + element_id: UUID, + context_kind: str | None = None, + context_id: str | None = None, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """The formatted mail with its pictures from other servers, loaded by the server on request.""" + element, context, ref = _opened_ref(conn, principal, element_id, context_kind, context_id) + parsed, _raw = _complete_mail(conn, principal, element, context, ref) + loader = RemoteImages() + rendered = sanitize_html(parsed.html_source, _inline_images(parsed.attachments), remote=loader) if parsed.html_source else "" + return JSONResponse( + content={"html": rendered, "loaded": loader.loaded, "failed": loader.failed}, + headers={"Cache-Control": "private, no-store", "Referrer-Policy": "no-referrer"}, + ) + + @router.get("/api/v1/external-refs/{element_id}/mail/original") def download_mail( element_id: UUID, @@ -150,8 +220,9 @@ def download_mail( conn: Connection = Depends(get_conn), ): element, context, ref = _opened_ref(conn, principal, element_id, context_kind, context_id) - _parsed, raw = _complete_mail(conn, principal, element, context, ref) - return _mail_bytes(raw, "message/rfc822", "nachricht.eml") + _require_download(conn, principal, element, context) + parsed, raw = _complete_mail(conn, principal, element, context, ref) + return _mail_bytes(raw, "message/rfc822", f"{parsed.subject or 'nachricht'}.eml") @router.get("/api/v1/external-refs/{element_id}/mail/attachments/{index}") @@ -160,21 +231,29 @@ def download_mail_attachment( index: int, context_kind: str | None = None, context_id: str | None = None, + save: bool = False, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn), ): element, context, ref = _opened_ref(conn, principal, element_id, context_kind, context_id) + if save: + _require_download(conn, principal, element, context) parsed, _raw = _complete_mail(conn, principal, element, context, ref) if index < 0 or index >= len(parsed.attachments): raise not_found() item = parsed.attachments[index] - return _mail_bytes(item.data, item.media_type or "application/octet-stream", item.filename, inline=True) + name = item.display_name or item.filename or "anlage" + media = (item.media_type or "application/octet-stream").split(";", 1)[0].strip() + if item.data[:1024].lstrip(b"\xef\xbb\xbf\r\n\t ").startswith(b"%PDF"): + media = "application/pdf" + if not name.lower().endswith(".pdf"): + leaf = name.replace("\\", "/").split("/")[-1] + name = name if "." in leaf else f"{name}.pdf" + return _mail_bytes(item.data, media or "application/octet-stream", name, inline=True) def _mail_bytes(data: bytes, media_type: str, filename: str, inline: bool = False) -> Response: - disposition = content_disposition(filename) - if inline: - disposition = disposition.replace("attachment;", "inline;", 1) + disposition = content_disposition(filename, inline=inline) return Response( content=data, media_type=media_type, @@ -186,8 +265,67 @@ def _mail_bytes(data: bytes, media_type: str, filename: str, inline: bool = Fals @router.get("/api/v1/sources/paperless/documents") -def search_paperless(q: str = "", principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - return {"items": paperless_documents(conn, principal.account_id, q.strip())} +def search_paperless( + q: str = "", + connector_id: UUID | None = None, + title: str = "", + correspondent: str = "", + document_type: str = "", + tag: str = "", + document_from: str = "", + document_to: str = "", + sort: str = "", + cursor: str = "", + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + return _run_search( + "paperless", + conn, + principal.account_id, + q.strip(), + connector_id, + "", + None, + { + "title": title, + "correspondent": correspondent, + "document_type": document_type, + "tag": tag, + "document_from": document_from, + "document_to": document_to, + "sort": sort, + "cursor": cursor, + }, + ) + + +@router.get("/api/v1/sources/{kind}/facets") +def source_facets( + kind: str, + connector_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + module = require_source(kind) + if module.facets is None: + raise unsupported("Diese Quelle hat keine Auswahllisten.") + return module.facets(conn, principal.account_id, connector_id) + + +@router.get("/api/v1/sources/paperless/documents/{document_id}/preview") +def preview_paperless_document( + document_id: str, + connector_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + data, media, name = paperless_content(conn, principal.account_id, document_id, connector_id) + return Response( + content=data, + media_type=media, + headers={**_PRIVATE, "Content-Disposition": content_disposition(name).replace("attachment;", "inline;", 1)}, + ) @router.get("/api/v1/sources/imap/mailboxes") @@ -196,7 +334,7 @@ def list_mailboxes( principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn), ): - return {"items": imap_mailboxes(conn, principal.account_id, connector_id)} + return _run_navigate("imap", conn, principal.account_id, connector_id) @router.get("/api/v1/sources/imap/messages") @@ -205,12 +343,40 @@ def search_mail( q: str = "", mailbox: str = "", before: int | None = None, + title: str = "", + correspondent: str = "", + document_type: str = "", + tag: str = "", + document_from: str = "", + document_to: str = "", + sort: str = "", + cursor: str = "", + uidvalidity: int | None = None, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn), ): if before is not None and before < 1: raise unprocessable("Die Seitenangabe ist ungültig.") - return imap_messages(conn, principal.account_id, connector_id, q.strip(), mailbox.strip(), before) + return _run_search( + "imap", + conn, + principal.account_id, + q.strip(), + connector_id, + mailbox.strip(), + before, + { + "title": title, + "correspondent": correspondent, + "document_type": document_type, + "tag": tag, + "document_from": document_from, + "document_to": document_to, + "sort": sort, + "cursor": cursor, + }, + uidvalidity, + ) @router.get("/api/v1/sources/imap/message") @@ -228,8 +394,8 @@ def read_mail( @router.post("/api/v1/sources/link", status_code=201) def link_source(body: ExternalIn, principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): - if body.source not in {"paperless", "imap"}: - raise unprocessable("Verknüpft werden Paperless-Dokumente und E-Mails.") + if get_source(body.source) is None: + raise unprocessable("Verknüpft werden registrierte Quellen.") element = create_external( conn, principal.account_id, @@ -248,20 +414,121 @@ def link_source(body: ExternalIn, principal: Principal = Depends(get_principal), return _created(f"/api/v1/elements/{element['id']}", {"id": str(element["id"]), "title": element["title"]}) -def _connector_parts(body: ConnectorIn) -> tuple[str, dict, str]: - if body.kind == "paperless": - if not body.base_url or not body.token or len(body.token.strip()) < 8: - raise unprocessable("Paperless braucht eine Adresse und einen API-Schlüssel.") - return body.name.strip(), {"base_url": clean_base_url(body.base_url)}, body.token.strip() - if not body.host or not body.username or not body.password: - raise unprocessable("Ein E-Mail-Konto braucht Server, Benutzername und Passwort.") - host = body.host.strip() - if " " in host or len(host) > 200: - raise unprocessable("Der Mailserver ist ungültig.") - return body.name.strip(), { - "host": host, - "port": body.port or 993, - "username": body.username.strip(), - "mailbox": (body.mailbox or "INBOX").strip() or "INBOX", - "security": body.security or "ssl", - }, body.password +@router.post("/api/v1/external-refs/{element_id}/assignment") +def assign_source( + element_id: UUID, + body: SourceAssignmentIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + assign_reference(conn, principal.account_id, element_id, body.connector_id) + return {"id": str(element_id), "connector_id": str(body.connector_id), "resolution": "bound"} + + +def _connector_parts(body: ConnectorIn, current: dict | None) -> tuple[str, dict, str]: + """The registered module validates. Flat fields of the first release are folded into ``config``.""" + flat = body.model_dump(exclude={"kind", "name", "config"}) + return parse_connector(body.kind, body.name, flat, body.config, current) + + +_SEARCH_FLAGS = { + "title": "title", + "correspondent": "correspondent", + "document_type": "document_type", + "tag": "tags", + "document_from": "document_range", + "document_to": "document_range", + "sort": "sort", + "cursor": "continuation", +} + + +def _run_search(kind: str, conn, owner, query: str, connector_id: UUID | None, mailbox: str, before: int | None, filters: dict, uidvalidity: int | None = None) -> dict: + module = require_source(kind) + search = module.capabilities["search"] + refused = [name for name, value in filters.items() if str(value or "").strip() and not search.get(_SEARCH_FLAGS.get(name, name))] + if before is not None and not search.get("continuation"): + refused.append("before") + if refused: + raise unsupported("Diese Suche unterstützt den Filter nicht: " + ", ".join(refused) + ".") + options = {"mailbox": mailbox, "before": before} + if uidvalidity is not None: + options["uidvalidity"] = uidvalidity + options.update({key: value for key, value in filters.items() if str(value or "").strip()}) + found = module.find(conn, owner, connector_id, query, options) + bound = str(found.get("connector_id") or connector_id or "") + return with_capabilities(kind, bound, found) + + +def _run_navigate(kind: str, conn, owner, connector_id: UUID) -> dict: + module = require_source(kind) + if not module.capabilities.get("navigate") or module.navigate is None: + raise unsupported("Diese Quelle hat keine Fachnavigation.") + return {"items": module.navigate(conn, owner, connector_id)} + + +@router.get("/api/v1/sources/{kind}/search") +def search_source( + kind: str, + q: str = "", + connector_id: UUID | None = None, + mailbox: str = "", + before: int | None = None, + title: str = "", + correspondent: str = "", + document_type: str = "", + tag: str = "", + document_from: str = "", + document_to: str = "", + sort: str = "", + cursor: str = "", + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + if before is not None and before < 1: + raise unprocessable("Die Seitenangabe ist ungültig.") + return _run_search( + kind, + conn, + principal.account_id, + q.strip(), + connector_id, + mailbox.strip(), + before, + { + "title": title, + "correspondent": correspondent, + "document_type": document_type, + "tag": tag, + "document_from": document_from, + "document_to": document_to, + "sort": sort, + "cursor": cursor, + }, + ) + + +@router.get("/api/v1/sources/{kind}/navigate") +def navigate_source( + kind: str, + connector_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + return _run_navigate(kind, conn, principal.account_id, connector_id) + + +@router.get("/api/v1/sources/{kind}/preview") +def preview_source( + kind: str, + connector_id: UUID, + mailbox: str = "", + uid: int = 0, + item: str = "", + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + module = require_source(kind) + if not module.capabilities.get("read_preview") or module.preview is None: + raise unsupported("Diese Quelle hat keine Vorschau vor dem Übernehmen.") + return module.preview(conn, principal.account_id, connector_id, {"mailbox": mailbox, "uid": uid, "item": item}) diff --git a/backend/app/api/team.py b/backend/app/api/team.py new file mode 100644 index 0000000..602d8e6 --- /dev/null +++ b/backend/app/api/team.py @@ -0,0 +1,245 @@ +"""Team of a main user and the operator's overview of all teams. + +A main user sees the people in the rooms he owns and manages the employees he invited. +The operator sees main users with their employees, rooms and roles: accounts and roles, never content. +""" + +from __future__ import annotations + +import csv +import io +from uuid import UUID + +from fastapi import APIRouter, Depends, Response +from fastapi.responses import JSONResponse +from psycopg import Connection + +from app.access.policy import audit +from app.access.roles import require_main_user, require_operator +from app.api.deps import Principal, get_conn, get_principal +from app.api.rooms import _audit_membership, _drop_shares_of +from app.api.schemas import TeamPatchIn +from app.auth.accounts import reset_password, set_status +from app.core.errors import forbidden +from app.core.timeutil import dump_dt + +router = APIRouter() + +_NO_STORE = {"Cache-Control": "no-store"} + + +def _own_employee(conn, owner_id: UUID, account_id: UUID) -> dict: + """An employee this main user invited. Unknown and foreign accounts answer alike.""" + row = conn.execute( + "SELECT id, status FROM auth_accounts WHERE id = %s AND account_kind = 'employee' AND sponsor_id = %s", + (account_id, owner_id), + ).fetchone() + if row is None: + raise forbidden("Das ist keiner deiner Mitarbeiter.") + return row + + +@router.get("/api/v1/team") +def team(principal: Principal = Depends(get_principal), conn: Connection = Depends(get_conn)): + require_main_user(conn, principal.account_id) + me = principal.account_id + people = conn.execute( + """ + SELECT DISTINCT a.id, a.display_name, a.email, a.status, a.account_kind, a.last_login_at, + (a.sponsor_id = %s) AS mine + FROM auth_accounts a + WHERE a.id <> %s AND ( + a.sponsor_id = %s + OR EXISTS ( + SELECT 1 FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = a.id AND m.status = 'active' + AND r.kind = 'organization' AND r.owner_id = %s + ) + ) + ORDER BY a.display_name, a.id + """, + (me, me, me, me), + ).fetchall() + rooms = conn.execute( + """ + SELECT m.subject_id, r.id AS room_id, r.name, r.status, m.role + FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.status = 'active' AND r.kind = 'organization' AND r.owner_id = %s AND m.subject_id <> %s + ORDER BY r.name, r.id + """, + (me, me), + ).fetchall() + by_person: dict = {} + for row in rooms: + by_person.setdefault(row["subject_id"], []).append( + {"room_id": str(row["room_id"]), "name": row["name"], "archived": row["status"] != "active", "role": row["role"]} + ) + return JSONResponse( + { + "items": [ + { + "id": str(person["id"]), + "display_name": person["display_name"], + "email": person["email"], + "status": person["status"], + "account_kind": person["account_kind"], + "last_login_at": dump_dt(person["last_login_at"]), + "mine": person["mine"], + "rooms": by_person.get(person["id"], []), + } + for person in people + ] + }, + headers=_NO_STORE, + ) + + +@router.patch("/api/v1/team/{account_id}") +def patch_team_member( + account_id: UUID, + body: TeamPatchIn, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_main_user(conn, principal.account_id) + _own_employee(conn, principal.account_id, account_id) + set_status(conn, principal.account_id, account_id, body.status) + return {"id": str(account_id), "status": body.status} + + +@router.post("/api/v1/team/{account_id}/password-reset") +def reset_team_password( + account_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + require_main_user(conn, principal.account_id) + _own_employee(conn, principal.account_id, account_id) + password = reset_password(conn, principal.account_id, account_id) + audit(conn, principal.account_id, None, "employee_password_reset", "account", account_id) + return JSONResponse({"temporary_password": password}, headers=_NO_STORE) + + +@router.delete("/api/v1/team/{account_id}/memberships", status_code=204) +def remove_from_my_rooms( + account_id: UUID, + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """End the memberships of one person in all rooms this main user owns.""" + require_main_user(conn, principal.account_id) + rows = conn.execute( + """ + SELECT m.room_id FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' AND r.owner_id = %s + FOR UPDATE OF m + """, + (account_id, principal.account_id), + ).fetchall() + if account_id == principal.account_id or not rows: + raise forbidden("Diese Person ist in keinem deiner Räume Mitglied.") + for row in rows: + room_id = row["room_id"] + conn.execute( + "UPDATE room_memberships SET status = 'removed' WHERE room_id = %s AND subject_id = %s", + (room_id, account_id), + ) + _drop_shares_of(conn, room_id, account_id) + conn.execute( + """ + UPDATE auth_sessions + SET scope_kind = 'personal', scope_id = account_id, epoch = epoch + 1 + WHERE account_id = %s AND scope_id = %s AND revoked_at IS NULL + """, + (account_id, room_id), + ) + _audit_membership(conn, principal.account_id, room_id, account_id, "membership_removed") + audit(conn, principal.account_id, None, "employee_removed", "account", account_id, detail={"rooms": len(rows)}) + return Response(status_code=204) + + +def _csv_cell(value) -> str: + text = "" if value is None else str(value) + # A cell that starts like a formula would run in a spreadsheet. + return "'" + text if text[:1] in {"=", "+", "-", "@", "\t", "\r"} else text + + +@router.get("/api/v1/admin/team") +def admin_team( + format: str = "json", + principal: Principal = Depends(get_principal), + conn: Connection = Depends(get_conn), +): + """Main users with their employees, rooms and roles. Metadata only.""" + require_operator(conn, principal.account_id) + owners = conn.execute( + """ + SELECT a.id, a.display_name, a.email, a.status + FROM auth_accounts a + WHERE a.account_kind = 'main' + AND (EXISTS (SELECT 1 FROM auth_accounts e WHERE e.sponsor_id = a.id) + OR EXISTS (SELECT 1 FROM data_rooms r WHERE r.owner_id = a.id AND r.kind = 'organization')) + ORDER BY a.display_name, a.id + """ + ).fetchall() + employees = conn.execute( + """ + SELECT e.id, e.sponsor_id, e.display_name, e.email, e.status, e.last_login_at + FROM auth_accounts e WHERE e.account_kind = 'employee' + ORDER BY e.display_name, e.id + """ + ).fetchall() + memberships = conn.execute( + """ + SELECT m.subject_id, r.id AS room_id, r.name, r.status, r.owner_id, m.role + FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.status = 'active' AND r.kind = 'organization' + ORDER BY r.name, r.id + """ + ).fetchall() + rooms_of: dict = {} + for row in memberships: + rooms_of.setdefault(row["subject_id"], []).append(row) + result = [] + for owner in owners: + mine = [e for e in employees if e["sponsor_id"] == owner["id"]] + result.append( + { + "id": str(owner["id"]), + "display_name": owner["display_name"], + "email": owner["email"], + "status": owner["status"], + "employees": [ + { + "id": str(e["id"]), + "display_name": e["display_name"], + "email": e["email"], + "status": e["status"], + "last_login_at": dump_dt(e["last_login_at"]), + "rooms": [ + {"room_id": str(r["room_id"]), "name": r["name"], "archived": r["status"] != "active", "role": r["role"]} + for r in rooms_of.get(e["id"], []) + ], + } + for e in mine + ], + } + ) + if format == "csv": + out = io.StringIO() + writer = csv.writer(out) + writer.writerow(["Hauptnutzer", "Mitarbeiter", "E-Mail", "Status", "Datenraum", "Rolle"]) + for owner in result: + for e in owner["employees"]: + if not e["rooms"]: + writer.writerow([_csv_cell(v) for v in (owner["display_name"], e["display_name"], e["email"], e["status"], "", "")]) + for room in e["rooms"]: + writer.writerow( + [_csv_cell(v) for v in (owner["display_name"], e["display_name"], e["email"], e["status"], room["name"], room["role"])] + ) + return Response( + content=out.getvalue(), + media_type="text/csv; charset=utf-8", + headers={"Content-Disposition": 'attachment; filename="team-uebersicht.csv"', **_NO_STORE}, + ) + return JSONResponse({"items": result}, headers=_NO_STORE) diff --git a/backend/app/auth/accounts.py b/backend/app/auth/accounts.py index 8e17bea..81db64e 100644 --- a/backend/app/auth/accounts.py +++ b/backend/app/auth/accounts.py @@ -1,18 +1,42 @@ -"""Accounts, personal rooms and sessions.""" +"""Accounts, personal rooms, sessions and the administrative account actions.""" from __future__ import annotations +import secrets from uuid import UUID, uuid4 -from app.auth.security import hash_password, hash_secret, new_secret, session_expiry -from app.core.errors import unprocessable +from app.access.policy import audit +from app.access.roles import count_active_operators +from app.auth.security import ( + hash_password, + hash_secret, + new_secret, + session_expiry, + verify_password, +) +from app.core.errors import not_found, unprocessable from app.core.settings import Settings +MIN_PASSWORD = 10 +MAX_PASSWORD_BYTES = 72 + def normalize_email(email: str) -> str: return email.strip().lower() +def check_password_policy(password: str) -> None: + if len(password) < MIN_PASSWORD: + raise unprocessable(f"Das Passwort braucht mindestens {MIN_PASSWORD} Zeichen.") + if len(password.encode("utf-8")) > MAX_PASSWORD_BYTES: + raise unprocessable("Das Passwort ist zu lang. Erlaubt sind höchstens 72 Byte.") + + +def generate_password() -> str: + """A one-time password for hand-over. It is shown once and must be changed at the first login.""" + return secrets.token_urlsafe(12) + + def create_account( conn, email: str, @@ -21,50 +45,255 @@ def create_account( *, is_superadmin: bool = False, origin: str = "operator", + must_change_password: bool = False, + account_kind: str = "main", + sponsor_id: UUID | None = None, ) -> dict: email = normalize_email(email) display_name = display_name.strip() if origin not in {"operator", "fixture"}: raise unprocessable("Die Kontenherkunft ist ungültig.") + if account_kind not in {"main", "employee"} or (account_kind == "employee") != (sponsor_id is not None): + raise unprocessable("Die Kontoart ist ungültig.") + check_password_policy(password) if conn.execute("SELECT 1 FROM auth_accounts WHERE email = %s", (email,)).fetchone(): raise unprocessable("Für diese E-Mail gibt es bereits ein Konto.") account_id = uuid4() password_hash = hash_password(password) conn.execute( """ - INSERT INTO auth_accounts (id, email, display_name, password_hash, is_superadmin, origin) - VALUES (%s, %s, %s, %s, %s, %s) + INSERT INTO auth_accounts ( + id, email, display_name, password_hash, is_superadmin, origin, + must_change_password, password_changed_at, account_kind, sponsor_id + ) + VALUES (%s, %s, %s, %s, %s, %s, %s, now(), %s, %s) """, - (account_id, email, display_name, password_hash, is_superadmin, origin), + ( + account_id, email, display_name, password_hash, is_superadmin, origin, + must_change_password, account_kind, sponsor_id, + ), ) conn.execute( """ - INSERT INTO data_rooms (id, kind, name, created_by) - VALUES (%s, 'personal', 'Persönlicher Raum', %s) + INSERT INTO data_rooms (id, kind, name, created_by, owner_id) + VALUES (%s, 'personal', 'Persönlicher Raum', %s, %s) """, - (account_id, account_id), + (account_id, account_id, account_id), ) return conn.execute( - "SELECT id, email, display_name, is_superadmin FROM auth_accounts WHERE id = %s", + """ + SELECT id, email, display_name, is_superadmin, status, must_change_password, account_kind + FROM auth_accounts WHERE id = %s + """, (account_id,), ).fetchone() -def start_session(conn, settings: Settings, account_id: UUID) -> tuple[str, str]: +def promote_to_main(conn, actor: UUID, account_id: UUID) -> None: + """Only the operator turns an employee into a main user.""" + account = _account(conn, account_id, lock=True) + if account["account_kind"] == "main": + return + conn.execute("UPDATE auth_accounts SET account_kind = 'main', sponsor_id = NULL WHERE id = %s", (account_id,)) + # A raised kind starts from a fresh login. + revoke_sessions(conn, account_id) + audit(conn, actor, None, "employee_promoted", "account", account_id) + + +def initial_scope(conn, account_id: UUID) -> tuple[str, UUID]: + """Where a new session starts: the personal room, or for an employee the first team room.""" + account = conn.execute("SELECT account_kind FROM auth_accounts WHERE id = %s", (account_id,)).fetchone() + if account is not None and account["account_kind"] == "employee": + room = conn.execute( + """ + SELECT r.id FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id + WHERE m.subject_id = %s AND m.status = 'active' AND r.kind = 'organization' AND r.status = 'active' + ORDER BY r.name, r.id LIMIT 1 + """, + (account_id,), + ).fetchone() + if room is not None: + return "organization", room["id"] + return "personal", account_id + + +def start_session(conn, settings: Settings, account_id: UUID, label: str | None = None) -> tuple[str, str]: token = new_secret() csrf = new_secret() - conn.execute( + scope_kind, scope_id = initial_scope(conn, account_id) + session = conn.execute( """ INSERT INTO auth_sessions ( - account_id, secret_hash, csrf_token, scope_kind, scope_id, expires_at + account_id, secret_hash, csrf_token, scope_kind, scope_id, expires_at, label, last_seen_at ) - VALUES (%s, %s, %s, 'personal', %s, %s) + VALUES (%s, %s, %s, %s, %s, %s, %s, now()) + RETURNING id """, - (account_id, hash_secret(token), csrf, account_id, session_expiry(settings)), - ) + (account_id, hash_secret(token), csrf, scope_kind, scope_id, session_expiry(settings), (label or None) and label[:80]), + ).fetchone() + conn.execute("UPDATE auth_accounts SET last_login_at = now() WHERE id = %s", (account_id,)) + audit(conn, account_id, None, "session_started", "session", session["id"]) return token, csrf +def _account(conn, account_id: UUID, *, lock: bool = False) -> dict: + suffix = " FOR UPDATE" if lock else "" + row = conn.execute(f"SELECT * FROM auth_accounts WHERE id = %s{suffix}", (account_id,)).fetchone() + if row is None: + raise not_found() + return row + + +def revoke_sessions(conn, account_id: UUID, *, keep: UUID | None = None) -> int: + rows = conn.execute( + """ + UPDATE auth_sessions SET revoked_at = now() + WHERE account_id = %s AND revoked_at IS NULL AND (%s::uuid IS NULL OR id <> %s::uuid) + RETURNING id + """, + (account_id, keep, keep), + ).fetchall() + return len(rows) + + +def set_display_name(conn, actor: UUID, account_id: UUID, display_name: str) -> None: + name = " ".join(display_name.split()) + if not name or len(name) > 120: + raise unprocessable("Der Name ist ungültig.") + _account(conn, account_id) + conn.execute("UPDATE auth_accounts SET display_name = %s WHERE id = %s", (name, account_id)) + audit(conn, actor, None, "account_renamed", "account", account_id) + + +def set_status(conn, actor: UUID, account_id: UUID, status: str) -> None: + """Disable or enable. A disabled account keeps its content, owner links and audit trail.""" + if status not in {"active", "disabled"}: + raise unprocessable("Der Kontostatus ist ungültig.") + # Lock the operators first so two parallel changes cannot remove the last one. + operators = count_active_operators(conn, lock=True) + account = _account(conn, account_id, lock=True) + if account["status"] == status: + return + if status == "disabled": + if account_id == actor: + raise unprocessable("Du kannst dein eigenes Konto nicht deaktivieren.") + owned = _owned_shared_rooms(conn, account_id) + if owned: + raise unprocessable( + f"Das Konto ist Eigentümer von {owned} Datenraum(en) mit weiteren Mitgliedern. " + "Der Eigentümer überträgt sie zuerst, oder der Betreiber setzt einen neuen Eigentümer ein." + ) + if account["is_superadmin"] and operators <= 1: + raise unprocessable("Der letzte Betreiber lässt sich nicht deaktivieren.") + conn.execute( + "UPDATE auth_accounts SET status = 'disabled', disabled_at = now(), disabled_by = %s WHERE id = %s", + (actor, account_id), + ) + revoke_sessions(conn, account_id) + audit(conn, actor, None, "account_disabled", "account", account_id) + else: + conn.execute( + "UPDATE auth_accounts SET status = 'active', disabled_at = NULL, disabled_by = NULL WHERE id = %s", + (account_id,), + ) + audit(conn, actor, None, "account_enabled", "account", account_id) + + +def set_operator(conn, actor: UUID, account_id: UUID, value: bool) -> None: + operators = count_active_operators(conn, lock=True) + account = _account(conn, account_id, lock=True) + if bool(account["is_superadmin"]) == value: + return + if value: + if account["status"] != "active": + raise unprocessable("Ein deaktiviertes Konto kann nicht Betreiber werden.") + conn.execute("UPDATE auth_accounts SET is_superadmin = true WHERE id = %s", (account_id,)) + # A raised role starts from a fresh login (Auth blueprint). + revoke_sessions(conn, account_id) + audit(conn, actor, None, "operator_granted", "account", account_id) + return + if account["status"] == "active" and operators <= 1: + raise unprocessable("Der letzte Betreiber lässt sich nicht zurückstufen.") + conn.execute("UPDATE auth_accounts SET is_superadmin = false WHERE id = %s", (account_id,)) + audit(conn, actor, None, "operator_revoked", "account", account_id) + + +def reset_password(conn, actor: UUID, account_id: UUID) -> str: + if account_id == actor: + raise unprocessable("Dein eigenes Passwort änderst du unter Mein Konto.") + account = _account(conn, account_id, lock=True) + if account["status"] != "active": + raise unprocessable("Für ein deaktiviertes Konto lässt sich kein Passwort setzen.") + password = generate_password() + conn.execute( + """ + UPDATE auth_accounts + SET password_hash = %s, must_change_password = true, password_changed_at = now() + WHERE id = %s + """, + (hash_password(password), account_id), + ) + revoke_sessions(conn, account_id) + audit(conn, actor, None, "password_reset", "account", account_id) + return password + + +def change_own_password(conn, account_id: UUID, session_id: UUID, current: str, new: str) -> None: + account = _account(conn, account_id, lock=True) + if not verify_password(current, account["password_hash"]): + raise unprocessable("Das aktuelle Passwort ist nicht richtig.") + check_password_policy(new) + if verify_password(new, account["password_hash"]): + raise unprocessable("Das neue Passwort muss sich vom bisherigen unterscheiden.") + conn.execute( + """ + UPDATE auth_accounts + SET password_hash = %s, must_change_password = false, password_changed_at = now() + WHERE id = %s + """, + (hash_password(new), account_id), + ) + revoke_sessions(conn, account_id, keep=session_id) + audit(conn, account_id, None, "password_changed", "account", account_id) + + +def login_blocked(conn, bucket: str, *, limit: int = 8, minutes: int = 10) -> bool: + row = conn.execute( + """ + SELECT count(*) AS n FROM auth_login_attempts + WHERE bucket = %s AND attempted_at > now() - make_interval(mins => %s) + """, + (bucket, minutes), + ).fetchone() + return row["n"] >= limit + + +def record_failed_login(conn, bucket: str) -> None: + """Committed at once: the failed request rolls back, the count must survive.""" + conn.execute("DELETE FROM auth_login_attempts WHERE attempted_at < now() - interval '1 day'") + conn.execute("INSERT INTO auth_login_attempts (bucket) VALUES (%s)", (bucket,)) + conn.commit() + + +def clear_login_attempts(conn, bucket: str) -> None: + conn.execute("DELETE FROM auth_login_attempts WHERE bucket = %s", (bucket,)) + + +def _owned_shared_rooms(conn, account_id: UUID) -> int: + row = conn.execute( + """ + SELECT count(*) AS n FROM data_rooms r + WHERE r.kind = 'organization' AND r.status = 'active' AND r.owner_id = %s + AND EXISTS ( + SELECT 1 FROM room_memberships m + WHERE m.room_id = r.id AND m.status = 'active' AND m.subject_id <> %s + ) + """, + (account_id, account_id), + ).fetchone() + return int(row["n"]) + + def list_rooms(conn, account_id: UUID) -> list[dict]: personal = conn.execute( "SELECT id, kind, name FROM data_rooms WHERE id = %s AND kind = 'personal'", @@ -75,7 +304,8 @@ def list_rooms(conn, account_id: UUID) -> list[dict]: rooms.append({"id": str(personal["id"]), "kind": personal["kind"], "name": personal["name"], "role": "owner"}) shared = conn.execute( """ - SELECT r.id, r.kind, r.name, m.role + SELECT r.id, r.kind, r.name, m.role, r.owner_id, r.default_visibility, + (SELECT count(*) FROM room_memberships x WHERE x.room_id = r.id AND x.status = 'active') AS members FROM room_memberships m JOIN data_rooms r ON r.id = m.room_id WHERE m.subject_id = %s AND m.status = 'active' AND r.status = 'active' AND r.kind = 'organization' @@ -84,5 +314,15 @@ def list_rooms(conn, account_id: UUID) -> list[dict]: (account_id,), ).fetchall() for room in shared: - rooms.append({"id": str(room["id"]), "kind": room["kind"], "name": room["name"], "role": room["role"]}) + rooms.append( + { + "id": str(room["id"]), + "kind": room["kind"], + "name": room["name"], + "role": room["role"], + "is_owner": room["owner_id"] == account_id, + "private": room["members"] <= 1, + "default_visibility": room["default_visibility"], + } + ) return rooms diff --git a/backend/app/auth/security.py b/backend/app/auth/security.py index c0a30be..ba8d802 100644 --- a/backend/app/auth/security.py +++ b/backend/app/auth/security.py @@ -5,6 +5,7 @@ from __future__ import annotations import hashlib import secrets from datetime import timedelta +from functools import lru_cache import bcrypt from fastapi import Request, Response @@ -27,6 +28,33 @@ def verify_password(password: str, password_hash: str) -> bool: return False +@lru_cache +def _dummy_hash() -> str: + return hash_password("kein-konto-nur-zeitausgleich") + + +def verify_unknown_account(password: str) -> None: + """Spend the same time as a real check so the answer time does not reveal an account.""" + verify_password(password, _dummy_hash()) + + +def login_bucket(host: str, email: str) -> str: + """Rate-limit key. The address is never stored in the clear.""" + return hashlib.sha256(f"{host}|{email}".encode("utf-8")).hexdigest() + + +_BROWSERS = (("Edg/", "Edge"), ("OPR/", "Opera"), ("Firefox/", "Firefox"), ("Chrome/", "Chrome"), ("Safari/", "Safari")) +_SYSTEMS = (("Windows", "Windows"), ("iPhone", "iPhone"), ("iPad", "iPad"), ("Android", "Android"), ("Mac OS", "macOS"), ("Linux", "Linux")) + + +def device_label(user_agent: str | None) -> str: + """A short, readable hint for the session list. The full user agent is not kept.""" + agent = user_agent or "" + browser = next((name for marker, name in _BROWSERS if marker in agent), "Browser") + system = next((name for marker, name in _SYSTEMS if marker in agent), "") + return f"{browser} auf {system}" if system else browser + + def new_secret() -> str: return secrets.token_urlsafe(32) diff --git a/backend/app/core/errors.py b/backend/app/core/errors.py index ce16081..3c3a747 100644 --- a/backend/app/core/errors.py +++ b/backend/app/core/errors.py @@ -33,6 +33,9 @@ def forbidden(detail: str = "Diese Aktion ist im gewählten Kontext nicht erlaub return Problem(403, "Keine Berechtigung", detail, "forbidden") +DOWNLOAD_REFUSED = "Das Herunterladen ist für dich in diesem Datenraum nicht freigegeben. Du kannst das Dokument in Tsumugu lesen." + + def not_found() -> Problem: return Problem(404, "Nicht gefunden", "Der Eintrag ist nicht vorhanden.", "not_found") @@ -50,6 +53,28 @@ def unprocessable(detail: str) -> Problem: return Problem(422, "Eingabe ungültig", detail, "unprocessable") +def unsupported(detail: str) -> Problem: + return Problem(422, "Nicht unterstützt", detail, "unsupported") + + +def scope_changed() -> Problem: + return Problem( + 403, + "Datenraum gewechselt", + "Der gewählte Datenraum entspricht nicht mehr der Sitzung. Die Ansicht wird neu geladen.", + "scope_changed", + ) + + +def password_change_required() -> Problem: + return Problem( + 403, + "Passwortwechsel erforderlich", + "Bitte lege zuerst ein eigenes Passwort fest.", + "password_change_required", + ) + + def too_many() -> Problem: return Problem(429, "Zu viele Versuche", "Bitte kurz warten und es erneut versuchen.", "too_many_requests") diff --git a/backend/app/correspondence.py b/backend/app/correspondence.py index 13a595b..3b872ba 100644 --- a/backend/app/correspondence.py +++ b/backend/app/correspondence.py @@ -26,9 +26,10 @@ from app.access.policy import ( require_case, require_matter, ) -from app.core.errors import conflict, not_found, unprocessable +from app.core.errors import Problem, conflict, not_found, unprocessable from app.core.timeutil import dump_date, dump_dt -from app.media.storage import asset_path, display_filename, read_bytes, sha256, write_bytes +from app.media.port import LEGACY_PROFILE, StoredHandle, adapter_for, instance_for_write, publish, read_clear_bytes, removal_for, stored_path +from app.media.storage import display_filename, sha256 from app.retrieval import FilePayload, MailPayload, RetrievalFailure, load_external, load_file CHANNELS = {"email": "per E-Mail", "post": "per Post", "other": "sonstiger Weg"} @@ -210,11 +211,36 @@ def record_dispatch( raise unprocessable("Dieselbe Anlage ist mehrfach im Versand genannt.") container_row, page, scope = open_container(conn, actor, context, placement, "edit") host = _bound_element(conn, actor, host_id, page, scope) - bump_version(conn, "elements", host_id, expected) known = {row["attachment_element_id"] for row in _uses(conn, scope, host_id)} if any(item not in known for item in attachment_ids): raise unprocessable("Mitversendet werden nur Anlagen dieses Schreibens.") counterpart_text = _counterpart(counterpart) + edition_plans = [] + if capture_editions: + edition_plans.append(_plan_edition(conn, actor, host, "host", 0, page, historical_gap, media_root, cleanup)) + ordered = [row for row in _uses(conn, scope, host_id) if row["attachment_element_id"] in set(attachment_ids)] + position_of = {item: index for index, item in enumerate(attachment_ids)} + ordered.sort(key=lambda row: position_of[row["attachment_element_id"]]) + for index, row in enumerate(ordered, start=1): + element = _readable(conn, actor, row["attachment_element_id"], page) + if element is None: + raise not_found() + edition_plans.append( + _plan_edition(conn, actor, element, "attachment", index, page, historical_gap, media_root, cleanup) + ) + evidence_plans = [] + for item in evidence: + element = _readable(conn, actor, item["element_id"], page) + if element is None: + raise not_found() + evidence_plans.append( + _plan_evidence( + conn, actor, element, item["role"], item.get("fold") or "keep", page, media_root, cleanup, container_row["home_scope_id"] + ) + ) + container_row, page, scope = open_container(conn, actor, context, placement, "edit") + host = _bound_element(conn, actor, host_id, page, scope) + bump_version(conn, "elements", host_id, expected) dispatch = conn.execute( """ INSERT INTO dispatches ( @@ -236,37 +262,11 @@ def record_dispatch( actor, ), ).fetchone() - states = [] - if capture_editions: - states.append(_capture_item(conn, actor, dispatch, host, "host", 0, page, historical_gap, media_root, cleanup)) - ordered = [row for row in _uses(conn, scope, host_id) if row["attachment_element_id"] in set(attachment_ids)] - position_of = {item: index for index, item in enumerate(attachment_ids)} - ordered.sort(key=lambda row: position_of[row["attachment_element_id"]]) - for index, row in enumerate(ordered, start=1): - element = _readable(conn, actor, row["attachment_element_id"], page) - if element is None: - raise not_found() - states.append( - _capture_item(conn, actor, dispatch, element, "attachment", index, page, historical_gap, media_root, cleanup) - ) - evidence_rows = [] - for item in evidence: - evidence_rows.append( - _add_evidence( - conn, - actor, - container_row, - page, - scope, - dispatch, - item["element_id"], - item["role"], - item.get("fold") or "keep", - media_root, - cleanup, - log=False, - ) - ) + states = [_write_edition(conn, dispatch, plan) for plan in edition_plans] + evidence_rows = [ + _write_evidence(conn, actor, container_row, page, scope, dispatch, plan, log=False) + for plan in evidence_plans + ] completeness = _completeness(states) if capture_editions else "secured" conn.execute( "UPDATE dispatches SET completeness = %s, updated_at = now() WHERE id = %s", @@ -291,11 +291,16 @@ def add_dispatch_evidence( ) -> dict: container_row, page, scope = open_container(conn, actor, context, placement, "edit") dispatch = _dispatch_in(conn, dispatch_id, scope) - _bump_dispatch(conn, dispatch_id, expected) - row = _add_evidence( - conn, actor, container_row, page, scope, dispatch, element_id, role, fold, media_root, cleanup, log=True + element = _readable(conn, actor, element_id, page) + if element is None: + raise not_found() + plan = _plan_evidence( + conn, actor, element, role, fold, page, media_root, cleanup, container_row["home_scope_id"], dispatch_id ) - return row + container_row, page, scope = open_container(conn, actor, context, placement, "edit") + dispatch = _dispatch_in(conn, dispatch_id, scope) + _bump_dispatch(conn, dispatch_id, expected) + return _write_evidence(conn, actor, container_row, page, scope, dispatch, plan, log=True) def detach_dispatch_evidence( @@ -415,9 +420,25 @@ def load_snapshot_file(conn, snapshot_id: UUID, media_root: str) -> FilePayload row = conn.execute("SELECT * FROM dispatch_snapshots WHERE id = %s", (snapshot_id,)).fetchone() if row is None: return RetrievalFailure("missing", "Die gesicherte Fassung ist nicht mehr vorhanden.") + profile = row["protection_profile"] or "legacy-plaintext" + if profile != "legacy-plaintext": + return RetrievalFailure( + "unsupported", + "Dieser Inhalt ist geschützt und kann hier nicht im Klartext gelesen werden.", + ) try: - data = read_bytes(_snapshot_path(media_root, row["id"])) - except Exception: + data = read_clear_bytes( + conn, + instance_id=row["storage_instance_id"], + generation=row["storage_generation"], + object_key=row["object_key"] or f"dispatch/{row['id']}", + media_root=media_root, + protection_profile=profile, + expected_sha256=row["sha256"], + ) + except Problem as exc: + if exc.code == "processing": + return RetrievalFailure("missing", "Die gesicherte Fassung lässt sich nicht unverändert lesen.") return RetrievalFailure("missing", "Die gesicherte Fassung ist nicht mehr vorhanden.") if sha256(data) != row["sha256"]: return RetrievalFailure("missing", "Die gesicherte Fassung lässt sich nicht unverändert lesen.") @@ -573,7 +594,7 @@ def purge_element_dispatch_files(conn, element_id: UUID, media_root: str) -> lis (item_id,), ).fetchone() if snap is not None: - paths.append(_snapshot_path(media_root, snap["id"])) + paths.append(_snapshot_disk(conn, snap["id"], media_root)) conn.execute("DELETE FROM dispatch_snapshots WHERE id = %s", (snap["id"],)) conn.execute( """ @@ -596,7 +617,7 @@ def purge_element_dispatch_files(conn, element_id: UUID, media_root: str) -> lis (evidence_id,), ).fetchone() if snap is not None: - paths.append(_snapshot_path(media_root, snap["id"])) + paths.append(_snapshot_disk(conn, snap["id"], media_root)) conn.execute("DELETE FROM dispatch_snapshots WHERE id = %s", (snap["id"],)) conn.execute( """ @@ -743,7 +764,7 @@ def _identify_evidence(conn, actor, dispatch, element, page, media_root, cleanup if isinstance(loaded, RetrievalFailure): return False, loaded.message, None, None, [], None if isinstance(loaded, MailPayload): - comparisons = _compare_mail(conn, actor, dispatch["id"], page, loaded) + comparisons = _compare_mail(conn, actor, dispatch["id"], page, loaded) if dispatch is not None else [] payload = FilePayload("nachweis.eml", "message/rfc822", loaded.raw, sha256(loaded.raw)) return True, None, loaded.parsed.message_id or None, payload.sha256, comparisons, payload payload = FilePayload(loaded.filename, loaded.media_type, loaded.data, sha256(loaded.data)) @@ -914,22 +935,191 @@ def _working_bytes(conn, actor, element, media_root: str): return None, "Für diesen Inhalt ist keine sicherbare Fassung vorgesehen.", None, None, None -def _store_snapshot(conn, scope_id, actor, payload: FilePayload, media_root: str, cleanup: list[Path], dispatch_item_id: UUID | None = None, evidence_id: UUID | None = None) -> UUID: +def _stage_snapshot(conn, media_root: str, data: bytes, cleanup: list) -> dict: + """Publish bytes only while the domain transaction is idle. The row comes later.""" + from app.sourceaccess import release_for_external + + instance = instance_for_write(conn, media_root) snapshot_id = uuid4() - path = _snapshot_path(media_root, snapshot_id) - write_bytes(path, payload.data) - cleanup.append(path) - digest = sha256(payload.data) + object_key = f"dispatch/{snapshot_id}" + adapter = adapter_for(instance) + release_for_external(conn) + publish(adapter, object_key, data) + cleanup.append(StoredHandle(adapter, object_key)) + return { + "id": snapshot_id, + "instance": instance, + "object_key": object_key, + "content_revision": str(uuid4()), + } + + +def _plan_edition(conn, actor, element, role, position, page, historical_gap, media_root, cleanup) -> dict: + dates = _dates(conn, element["id"]) + payload = None + gap_reason = None + note_document = None + external_source = None + external_id = None + if historical_gap: + state = "gap" + gap_reason = _HISTORICAL_GAP + else: + payload, gap_reason, note_document, external_source, external_id = _working_bytes( + conn, actor, element, media_root + ) + state = "snapshot" if payload is not None else "gap" + staged = _stage_snapshot(conn, media_root, payload.data, cleanup) if payload is not None else None + return { + "element": element, + "role": role, + "position": position, + "dates": dates, + "state": state, + "payload": payload, + "gap_reason": gap_reason, + "note_document": note_document, + "external_source": external_source, + "external_id": external_id, + "staged": staged, + } + + +def _write_edition(conn, dispatch, plan: dict) -> str: + payload = plan["payload"] + filename = payload.filename if payload is not None else None + content_type = payload.media_type if payload is not None else None + digest = sha256(payload.data) if payload is not None else None + byte_size = len(payload.data) if payload is not None else None + item = conn.execute( + """ + INSERT INTO dispatch_items ( + dispatch_id, role, position, element_id, title, document_on, filename, + content_type, byte_size, content_sha256, source_state, gap_reason, + note_document, external_source, external_id + ) + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) + RETURNING id + """, + ( + dispatch["id"], + plan["role"], + plan["position"], + plan["element"]["id"], + plan["element"]["title"], + plan["dates"]["document_on"], + filename, + content_type, + byte_size, + digest, + plan["state"], + plan["gap_reason"], + Json(plan["note_document"]) if plan["note_document"] is not None else None, + plan["external_source"], + plan["external_id"], + ), + ).fetchone() + if plan["staged"] is not None and payload is not None: + _record_snapshot(conn, dispatch["home_scope_id"], dispatch["recorded_by"], payload, plan["staged"], dispatch_item_id=item["id"]) + return plan["state"] + + +def _plan_evidence(conn, actor, element, role, fold, page, media_root, cleanup, scope_id, dispatch_id=None) -> dict: + if role not in EVIDENCE_ROLES: + raise unprocessable("Die Rolle des Nachweises ist nicht vorgesehen.") + if fold not in {"keep", "evidence"}: + raise unprocessable("Die Darstellung des Nachweises ist nicht festgelegt.") + if dispatch_id is not None and conn.execute( + "SELECT id FROM dispatch_evidence WHERE dispatch_id = %s AND element_id = %s", + (dispatch_id, element["id"]), + ).fetchone(): + raise unprocessable("Dieser Nachweis ist dem Versand bereits zugeordnet.") + dispatch = {"id": dispatch_id} if dispatch_id is not None else None + identified, gap_reason, message_id, digest, comparisons, snapshot = _identify_evidence( + conn, actor, dispatch, element, page, media_root, cleanup, scope_id + ) + staged = _stage_snapshot(conn, media_root, snapshot.data, cleanup) if snapshot is not None else None + return { + "element": element, + "role": role, + "fold": fold, + "identified": identified, + "gap_reason": gap_reason, + "message_id": message_id, + "digest": digest, + "comparisons": comparisons, + "snapshot": snapshot, + "staged": staged, + } + + +def _write_evidence(conn, actor, container_row, page, scope, dispatch, plan: dict, log: bool) -> dict: + snapshot = plan["snapshot"] + comparisons = plan["comparisons"] + if snapshot is not None and snapshot.media_type == "message/rfc822": + from app.mailpack import parse_mail + + comparisons = _compare_mail(conn, actor, dispatch["id"], page, MailPayload(snapshot.data, parse_mail(snapshot.data))) + proof = conn.execute( + """ + INSERT INTO dispatch_evidence ( + dispatch_id, element_id, role, source_state, gap_reason, message_id, + identity_sha256, comparisons, manual, created_by + ) + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, true, %s) + RETURNING * + """, + ( + dispatch["id"], + plan["element"]["id"], + plan["role"], + "identified" if plan["identified"] else "gap", + plan["gap_reason"], + plan["message_id"], + plan["digest"], + Json(comparisons), + actor, + ), + ).fetchone() + if plan["staged"] is not None and snapshot is not None: + _record_snapshot( + conn, + container_row["home_scope_id"], + actor, + snapshot, + plan["staged"], + evidence_id=proof["id"], + ) + if plan["fold"] == "evidence": + _set_binding_fold(conn, scope, plan["element"]["id"], "evidence") + if log: + _log(conn, dispatch["id"], actor, f"Nachweis ergänzt: {EVIDENCE_ROLES[plan['role']]}") + audit(conn, actor, container_row["home_scope_id"], "dispatch_evidence", "element", plan["element"]["id"], page) + return { + "id": str(proof["id"]), + "element_id": str(plan["element"]["id"]), + "role": plan["role"], + "role_label": EVIDENCE_ROLES[plan["role"]], + "source_state": proof["source_state"], + "gap_reason": plan["gap_reason"], + "manual": True, + "comparisons": comparisons, + } + + +def _record_snapshot(conn, scope_id, actor, payload: FilePayload, staged: dict, dispatch_item_id=None, evidence_id=None) -> UUID: conn.execute( """ INSERT INTO dispatch_snapshots ( id, dispatch_item_id, evidence_id, home_scope_id, owner_subject_id, - original_filename, content_type, byte_size, sha256 + original_filename, content_type, byte_size, sha256, + storage_instance_id, storage_generation, object_key, content_revision, + protection_profile, key_responsibility ) - VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s) + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, 'none') """, ( - snapshot_id, + staged["id"], dispatch_item_id, evidence_id, scope_id, @@ -937,10 +1127,20 @@ def _store_snapshot(conn, scope_id, actor, payload: FilePayload, media_root: str display_filename(payload.filename), payload.media_type or "application/octet-stream", len(payload.data), - digest, + sha256(payload.data), + staged["instance"]["id"], + staged["instance"]["generation"], + staged["object_key"], + staged["content_revision"], + LEGACY_PROFILE, ), ) - return snapshot_id + return staged["id"] + + +def _store_snapshot(conn, scope_id, actor, payload: FilePayload, media_root: str, cleanup: list[Path], dispatch_item_id: UUID | None = None, evidence_id: UUID | None = None) -> UUID: + staged = _stage_snapshot(conn, media_root, payload.data, cleanup) + return _record_snapshot(conn, scope_id, actor, payload, staged, dispatch_item_id, evidence_id) def _snapshot_file(conn, media_root: str, dispatch_item_id: UUID | None = None, evidence_id: UUID | None = None) -> FilePayload: @@ -956,20 +1156,36 @@ def _snapshot_file(conn, media_root: str, dispatch_item_id: UUID | None = None, ).fetchone() if row is None: raise not_found() - data = read_bytes(_snapshot_path(media_root, row["id"])) - if sha256(data) != row["sha256"]: - raise not_found() + profile = row["protection_profile"] or "legacy-plaintext" + data = read_clear_bytes( + conn, + instance_id=row["storage_instance_id"], + generation=row["storage_generation"], + object_key=row["object_key"] or f"dispatch/{row['id']}", + media_root=media_root, + protection_profile=profile, + expected_sha256=row["sha256"], + ) return FilePayload(row["original_filename"], row["content_type"], data, row["sha256"]) -def _snapshot_path(media_root: str, snapshot_id: UUID) -> Path: - root = Path(media_root).resolve() - folder = (root / "dispatch").resolve() - folder.mkdir(parents=True, exist_ok=True) - target = (folder / str(snapshot_id)).resolve() - if target.parent != folder: - raise unprocessable("Ungültiger Speicherpfad.") - return target +def _snapshot_disk(conn, snapshot_id: UUID, media_root: str) -> Path: + row = conn.execute( + """ + SELECT id, storage_instance_id, storage_generation, object_key + FROM dispatch_snapshots WHERE id = %s + """, + (snapshot_id,), + ).fetchone() + if row is None: + return removal_for(conn, None, "legacy-local-1", f"dispatch/{snapshot_id}", media_root) + return removal_for( + conn, + row["storage_instance_id"], + row["storage_generation"], + row["object_key"] or f"dispatch/{row['id']}", + media_root, + ) def _snapshot_id(conn, dispatch_item_id: UUID | None = None, evidence_id: UUID | None = None) -> str | None: @@ -1416,5 +1632,5 @@ def _delete_dispatch_files(conn, dispatch_id: UUID, media_root: str) -> list[Pat (dispatch_id, dispatch_id), ).fetchall() for row in rows: - paths.append(_snapshot_path(media_root, row["id"])) + paths.append(_snapshot_disk(conn, row["id"], media_root)) return paths diff --git a/backend/app/db/migrate.py b/backend/app/db/migrate.py index a2ff974..fb805b4 100644 --- a/backend/app/db/migrate.py +++ b/backend/app/db/migrate.py @@ -49,9 +49,26 @@ def migrate(settings: Settings | None = None) -> list[str]: ) applied.append(version) _ensure_runtime_role(conn, settings) + _prepare_source_storage(settings) return applied +def _prepare_source_storage(settings: Settings) -> None: + from psycopg.rows import dict_row + + from app.media.port import ensure_legacy_instance + from app.sourcebind import backfill_source_bindings + from app.sources import _install_sources + + # The process that migrates has not imported the application yet. + # Without the modules, a legacy reference would stay unresolved. + _install_sources() + with psycopg.connect(settings.dsn(), row_factory=dict_row) as conn: + backfill_source_bindings(conn) + ensure_legacy_instance(conn, settings.media_root) + conn.commit() + + def _ensure_runtime_role(conn: psycopg.Connection, settings: Settings) -> None: role = sql.Identifier(settings.db_runtime_user) database = sql.Identifier(settings.db_name) diff --git a/backend/app/exporting.py b/backend/app/exporting.py index 79e614a..e00a1ff 100644 --- a/backend/app/exporting.py +++ b/backend/app/exporting.py @@ -39,6 +39,7 @@ _KIND_LABEL = { "file": "Datei", "email": "E-Mail", "paperless": "Paperless", + "source": "Quelle", "url": "Webadresse", "annotation": "Anmerkung", "chapter": "Kapitel", @@ -233,7 +234,7 @@ def _plan(conn, actor, matter_id, context, selected, assignments, media_root) -> continue if node.content_kind == "file": result = load_file(conn, UUID(node.element_id), media_root) - elif node.content_kind in {"email", "paperless", "url"}: + elif node.content_kind in {"email", "paperless", "url", "source"}: result = load_external(conn, actor, UUID(node.element_id), assignments.get(node.id)) else: result = RetrievalFailure("capability", "Für diesen Inhalt ist kein Originalabruf vorgesehen.") @@ -461,13 +462,16 @@ def _content_kind(element: dict) -> str: if element["kind"] == "file": return "file" source = (element.get("external") or {}).get("source") - if source == "imap": - return "email" - if source == "paperless": - return "paperless" if source == "url": return "url" - return element["kind"] + from app.sourceregistry import get_source + + module = get_source(source) if source else None + if module is None: + return "source" + if module.representation == "email": + return "email" + return "source" def _chosen(root: ExportNode, selected: list[str]) -> set[str]: @@ -492,8 +496,8 @@ def _note_text(conn, element_id: UUID) -> str: parts = [] for block in document.get("blocks") or []: if isinstance(block, dict): - text = str(block.get("text") or "").strip() - if text: + text = str(block.get("text") or "") + if text.strip(): parts.append(text) return "\n\n".join(parts) @@ -691,9 +695,18 @@ def _emit_one(node, prefix, add, entries, loaded, notice_for, generated: str) -> links = [("EML-Original", eml)] try: parsed = payload.parsed if isinstance(payload, MailPayload) else parse_mail(raw) + reading = _mail_reading_html(parsed) + if reading is not None: + reading_path = add(prefix + ["lesefassung.html"], reading) + links.append(("Formatierte Lesefassung", reading_path)) + else: + notice = "\n".join(part for part in [notice, "Eine formatierte Lesefassung liegt für diese Nachricht nicht vor. Die vereinfachte Textfassung steht in der Übersicht."] if part) for attachment in parsed.attachments: + label = attachment.display_name or attachment.filename path = add(prefix + ["anlagen", attachment.filename], attachment.data) - links.append((attachment.filename, path)) + links.append((label, path)) + if parsed.notices: + notice = "\n".join(part for part in [notice, *parsed.notices] if part) except Exception: links.append(("Hinweis", eml)) entries.append(_entry(node, eml, links, notice or None)) @@ -1219,6 +1232,21 @@ def _note_markdown(node: ExportNode, body: str, generated: str) -> str: return "\n".join(lines) +def _mail_reading_html(parsed) -> bytes | None: + if not getattr(parsed, "html", ""): + return None + page = ( + "" + "" + "Lesefassung" + f"{parsed.html}" + "

Vereinfachte Textfassung und das unveränderte EML-Original stehen daneben. " + "Die Ansicht ist nicht pixelgleich zur Postfachdarstellung.

" + "" + ) + return page.encode("utf-8") + + def _mail_markdown(node, mail: MailPayload, text_path: str, eml_path: str, attachments, notice: str, generated: str) -> str: parsed = mail.parsed lines = [f"# {node.title}", "", "Lesefassung. Das unveränderte Original ist die EML-Datei.", ""] @@ -1230,7 +1258,13 @@ def _mail_markdown(node, mail: MailPayload, text_path: str, eml_path: str, attac if parsed.copy: lines.append(f"- Kopie: {parsed.copy}") lines.append(f"- [EML-Original]({_href(_relative(text_path, eml_path))})") - lines.extend(["", parsed.text or "Der Text steht im EML-Original.", ""]) + lines.extend([ + "", + "Vereinfachte Textfassung. Sie ist nicht die formatierte Ansicht und nicht das EML-Original.", + "", + parsed.text or "Der Text steht im EML-Original.", + "", + ]) if attachments: lines.append("## Anlagen") lines.append("") diff --git a/backend/app/library.py b/backend/app/library.py new file mode 100644 index 0000000..10622d6 --- /dev/null +++ b/backend/app/library.py @@ -0,0 +1,170 @@ +"""Overviews for the library: cases and matters as tiles, and what lies in a room. + +A room is a virtual folder. Cases and matters stay where they are; a share puts them into a room. +Counts only include what the person may read. Hidden entries are neither named nor counted. +""" + +from __future__ import annotations + +from collections import Counter +from uuid import UUID + +from app.access.policy import ( + ROOM_RIGHTS, + Ctx, + Rights, + case_rights, + get_case, + list_cases, + list_matters, + matter_rights, + room_name, +) +from app.present import _chapters, _matter_tree, case_summary, matter_summary + + +def shared_rooms(conn, kind: str, resource_id: UUID) -> list[dict]: + rows = conn.execute( + """ + SELECT r.id, r.name, s.access FROM room_shares s JOIN data_rooms r ON r.id = s.room_id + WHERE s.resource_kind = %s AND s.resource_id = %s + ORDER BY r.name, r.id + """, + (kind, resource_id), + ).fetchall() + return [{"id": str(row["id"]), "name": row["name"], "access": row["access"]} for row in rows] + + +def readable_matter_count(conn, actor: UUID, case: dict) -> int: + rows = conn.execute( + """ + SELECT m.* FROM matter_case_links l JOIN matters m ON m.id = l.matter_id + WHERE l.case_id = %s AND m.deleted_at IS NULL AND m.parent_matter_id IS NULL + """, + (case["id"],), + ).fetchall() + context = Ctx("case", case["id"]) + return sum(1 for matter in rows if matter_rights(conn, actor, matter, context).read) + + +def case_items(conn, actor: UUID, visible: list[tuple[dict, Rights]]) -> list[dict]: + children = Counter(case["parent_case_id"] for case, _ in visible if case["parent_case_id"]) + items = [] + for case, rights in visible: + summary = case_summary(case, rights) + owner = case["owner_subject_id"] == actor + summary["home_scope_name"] = room_name(conn, case["home_scope_id"]) + summary["kind_label"] = "Unterakte" if case["parent_case_id"] else "Akte" + summary["parent_title"] = None + summary["is_owner"] = owner + summary["shared_with"] = shared_rooms(conn, "case", case["id"]) if owner else [] + summary["child_cases"] = children.get(case["id"], 0) + summary["matter_count"] = readable_matter_count(conn, actor, case) + if case["parent_case_id"]: + parent = get_case(conn, case["parent_case_id"]) + if parent is not None and case_rights(conn, actor, parent).read: + summary["parent_title"] = parent["title"] + items.append(summary) + return items + + +def matter_items(conn, actor: UUID, visible: list[tuple[dict, list[dict]]]) -> list[dict]: + children = Counter(matter["parent_matter_id"] for matter, _ in visible if matter["parent_matter_id"]) + items = [] + for matter, _contexts in visible: + summary = matter_summary(conn, actor, matter) + summary["child_matters"] = children.get(matter["id"], 0) + items.append(summary) + return items + + +def room_contents(conn, actor: UUID, room_id: UUID) -> tuple[list[tuple[dict, Rights]], list[tuple[dict, list[dict]]]]: + """Cases and matters shared with the room that the person may read, with their readable sub-entries.""" + rows = conn.execute( + "SELECT resource_kind, resource_id FROM room_shares WHERE room_id = %s", + (room_id,), + ).fetchall() + case_ids = {row["resource_id"] for row in rows if row["resource_kind"] == "case"} + matter_ids = {row["resource_id"] for row in rows if row["resource_kind"] == "matter"} + cases = [ + (case, rights) + for case, rights in list_cases(conn, actor) + if case["id"] in case_ids + or (case["rights_mode"] == "inherit_parent" and case["parent_case_id"] in case_ids) + ] + matters = [ + (matter, contexts) + for matter, contexts in list_matters(conn, actor) + if matter["id"] in matter_ids or matter["parent_matter_id"] in matter_ids + ] + return cases, matters + + +def room_effect(conn, room_id: UUID, actor: UUID, access: str) -> list[dict]: + """Who gets which rights through a share with this room.""" + rows = conn.execute( + """ + SELECT m.subject_id, m.role, a.display_name + FROM room_memberships m JOIN auth_accounts a ON a.id = m.subject_id + WHERE m.room_id = %s AND m.status = 'active' AND a.status = 'active' AND m.subject_id <> %s + ORDER BY a.display_name, a.id + """, + (room_id, actor), + ).fetchall() + effect = [] + for row in rows: + rights = ROOM_RIGHTS.get(row["role"], Rights()) + if access == "read": + rights = Rights(read=rights.read) + effect.append( + { + "display_name": row["display_name"], + "role": row["role"], + "rights": {"read": rights.read, "edit": rights.edit}, + } + ) + return effect + + +def case_outline(conn, actor: UUID, case: dict) -> dict: + """What the navigation tree shows below a case. Only readable entries; no bindings or texts.""" + chapters = _chapters(conn, "case", case["id"], case.get("content_order") or "position", case.get("content_order_dir") or "asc") + subcases = [] + for row in conn.execute( + "SELECT * FROM cases WHERE parent_case_id = %s AND deleted_at IS NULL ORDER BY title, id", + (case["id"],), + ).fetchall(): + if case_rights(conn, actor, row).read: + subcases.append({"id": str(row["id"]), "title": row["title"], "status": row["record_status"]}) + linked = [] + for link in conn.execute( + """ + SELECT m.* FROM matter_case_links l JOIN matters m ON m.id = l.matter_id + WHERE l.case_id = %s AND m.deleted_at IS NULL ORDER BY m.title, m.id + """, + (case["id"],), + ).fetchall(): + if matter_rights(conn, actor, link, Ctx("case", case["id"])).read: + linked.append(link) + return { + "kind": "case", + "id": str(case["id"]), + "chapters": chapters, + "subcases": subcases, + "matters": _matter_tree(conn, actor, case["id"], linked), + } + + +def matter_outline(conn, actor: UUID, matter: dict, context: Ctx | None) -> dict: + """What the navigation tree shows below a matter: its chapters and its readable Teilvorgänge.""" + chapters = _chapters( + conn, "matter", matter["id"], matter.get("content_order") or "position", matter.get("content_order_dir") or "asc" + ) + children = [] + for row in conn.execute( + "SELECT * FROM matters WHERE parent_matter_id = %s AND deleted_at IS NULL ORDER BY title, id", + (matter["id"],), + ).fetchall(): + if matter_rights(conn, actor, row, context).read: + children.append({"id": str(row["id"]), "title": row["title"], "status": row["record_status"]}) + return {"kind": "matter", "id": str(matter["id"]), "chapters": chapters, "submatters": children} diff --git a/backend/app/mailpack.py b/backend/app/mailpack.py index 1c5d0bc..a487510 100644 --- a/backend/app/mailpack.py +++ b/backend/app/mailpack.py @@ -1,16 +1,46 @@ """Pure mail identity and attachment extraction for a complete export. A preview cutoff does not belong here. Duplicate attachment names stay distinct. +The same tree feeds preview, reading, attachment download and export. """ from __future__ import annotations +import base64 import hashlib +import html import re from dataclasses import dataclass, field +from email import policy +from email.header import decode_header, make_header +from email.message import EmailMessage from email.parser import BytesParser +from email.utils import parsedate_to_datetime +from html.parser import HTMLParser _UID_PREFIX = "uid:" +_POLICY = policy.default +_ALLOWED_TAGS = { + "a", "p", "br", "div", "span", "strong", "em", "b", "i", "u", + "ul", "ol", "li", "table", "thead", "tbody", "tr", "th", "td", + "blockquote", "pre", "code", "h1", "h2", "h3", "h4", "h5", "h6", "img", + "font", "center", "hr", "small", "sub", "sup", "s", "strike", "caption", "tfoot", "section", "article", +} +_VOID_TAGS = {"br", "img", "hr"} +# Formatting a mail may keep. Nothing here loads anything, positions an element or runs code. +_STYLE_PROPS = { + "color", "background-color", "font-family", "font-size", "font-weight", "font-style", "text-align", + "text-decoration", "line-height", "vertical-align", "white-space", "letter-spacing", + "margin", "margin-top", "margin-bottom", "margin-left", "margin-right", + "padding", "padding-top", "padding-bottom", "padding-left", "padding-right", + "border", "border-top", "border-bottom", "border-left", "border-right", "border-color", "border-width", + "border-style", "border-collapse", "width", "max-width", "height", "text-indent", "list-style-type", +} +_STYLE_VALUE = re.compile(r"^[#%a-z0-9 .,()\-+'\"]{1,100}$", re.IGNORECASE) +_STYLE_DENIED = re.compile(r"url|expression|var\(|attr\(|image|@|javascript|behavior|\\", re.IGNORECASE) +_COLOR = re.compile(r"^(#[0-9a-f]{3,8}|[a-z]{3,20}|rgb\(\s*\d{1,3}\s*,\s*\d{1,3}\s*,\s*\d{1,3}\s*\))$", re.IGNORECASE) +_SAFE_LINK = re.compile(r"^(https?://|mailto:)", re.IGNORECASE) +_DATA_IMAGE = re.compile(r"^data:image/(?:png|jpeg|gif|webp);base64,[a-z0-9+/=\s]+$", re.IGNORECASE) class MailIdentityError(Exception): @@ -25,6 +55,9 @@ class AttachmentBytes: media_type: str data: bytes sha256: str + display_name: str = "" + content_id: str = "" + inline: bool = False @dataclass @@ -37,10 +70,16 @@ class ParsedMail: text: str attachments: list[AttachmentBytes] = field(default_factory=list) missing: list[str] = field(default_factory=list) + html: str = "" + html_source: str = "" + remote_images: list[str] = field(default_factory=list) + notices: list[str] = field(default_factory=list) + text_truncated: bool = False + document_on: str | None = None def header_message_id(raw: bytes) -> str: - message = BytesParser().parsebytes(raw) + message = BytesParser(policy=_POLICY).parsebytes(raw) return str(message.get("Message-ID") or "").strip() @@ -69,67 +108,404 @@ def choose_message(candidate: bytes | None, alternates: list[bytes], expected: s raise MailIdentityError("Die gespeicherte Nachricht lässt sich nicht eindeutig wiederfinden.") +def header_text(value) -> str: + """Decode encoded-words. A raw MIME header is not a display name.""" + if value is None: + return "" + raw = str(value).replace("\r", " ").replace("\n", " ") + try: + text = str(make_header(decode_header(raw))) + except (UnicodeError, LookupError, ValueError): + text = raw + return re.sub(r"\s+", " ", text).strip() + + def parse_mail(raw: bytes) -> ParsedMail: - message = BytesParser().parsebytes(raw) + message = BytesParser(policy=_POLICY).parsebytes(raw) attachments: list[AttachmentBytes] = [] missing: list[str] = [] + notices: list[str] = [] used: set[str] = set() + plain: list[str] = [] + html_parts: list[str] = [] index = 0 - for part in message.walk(): - if part.is_multipart(): + for kind, part in _walk(message): + if kind == "message": + index += 1 + label = header_text(part.get_filename()) or f"nachricht-{index}.eml" + data = _message_bytes(part) + if data is None: + missing.append(label) + notices.append(f"Die angehängte Nachricht {label} ließ sich nicht vollständig lesen.") + continue + name = _unique_filename(label, used) + attachments.append(_attachment(part, name, label, data, inline=False)) continue - filename = part.get_filename() disposition = (part.get_content_disposition() or "").lower() - if not filename and disposition != "attachment": + media = part.get_content_type() + filename = part.get_filename() + content_id = _content_id(part) + body = media in {"text/plain", "text/html"} and disposition != "attachment" and not filename + if body: + text = _decoded_text(part) + if text is None: + notices.append("Ein Textteil ließ sich nicht sicher dekodieren.") + continue + if media == "text/plain": + plain.append(text) + else: + html_parts.append(text) + continue + if not filename and disposition != "attachment" and not content_id: continue index += 1 - label = _header_text(filename) if filename else f"anlage-{index}.bin" - payload = part.get_payload(decode=True) - if not isinstance(payload, (bytes, bytearray)): + label = header_text(filename) if filename else f"anlage-{index}.bin" + data = _leaf_bytes(part) + if data is None: missing.append(label or f"Anlage {index}") + notices.append(f"Die Anlage {label or index} ließ sich nicht vollständig lesen.") continue + data = _unwrap_pdf_payload(data, media, label) + if _is_pdf(data): + media = "application/pdf" + label = _pdf_filename(label) name = _unique_filename(label or f"anlage-{index}.bin", used) - data = bytes(payload) - attachments.append( - AttachmentBytes( - filename=name, - media_type=(part.get_content_type() or "application/octet-stream")[:120], - data=data, - sha256=hashlib.sha256(data).hexdigest(), - ) - ) - text = _body_text(message) + inline = disposition == "inline" or bool(content_id) + attachments.append(_attachment(part, name, label or name, data, inline=inline, media=media)) + simplified = "\n".join(chunk for chunk in plain if chunk).strip() + rendered = "" + if html_parts: + source = "\n".join(html_parts) + remote_found: list[str] = [] + rendered = sanitize_html(source, _inline_images(attachments), found=remote_found) + if not simplified: + simplified = simplified_text("\n".join(html_parts)) + if missing: + notices.append("Mindestens eine Anlage fehlt in der Aufbereitung.") return ParsedMail( message_id=str(message.get("Message-ID") or "").strip(), - subject=_header_text(message.get("Subject")) or "(ohne Betreff)", - sender=_header_text(message.get("From")), - recipient=_header_text(message.get("To")), - copy=_header_text(message.get("Cc")), - text=text, + subject=header_text(message.get("Subject")) or "(ohne Betreff)", + sender=header_text(message.get("From")), + recipient=header_text(message.get("To")), + copy=header_text(message.get("Cc")), + text=simplified, attachments=attachments, missing=missing, + html=rendered, + html_source=source if html_parts else "", + remote_images=remote_found if html_parts else [], + notices=notices, + document_on=_message_date(message.get("Date")), ) +def decode_transfer(payload: bytes, charset: str | None) -> str: + return bytes(payload).decode(charset or "utf-8", errors="replace") + + +def simplified_text(value: str) -> str: + """A recognizable plain reading. It is not the formatted view.""" + class Collector(HTMLParser): + def __init__(self) -> None: + super().__init__(convert_charrefs=True) + self.parts: list[str] = [] + self.skip = 0 + + def handle_starttag(self, tag, attrs): + name = tag.lower() + if name in {"script", "style"}: + self.skip += 1 + return + if name in {"br", "p", "div", "tr", "li", "h1", "h2", "h3", "h4", "h5", "h6", "blockquote"}: + self.parts.append("\n") + elif name in {"td", "th"}: + self.parts.append("\t") + + def handle_endtag(self, tag): + name = tag.lower() + if name in {"script", "style"} and self.skip: + self.skip -= 1 + return + if name in {"p", "div", "tr", "li", "table", "h1", "h2", "h3", "h4", "h5", "h6"}: + self.parts.append("\n") + + def handle_data(self, data): + if not self.skip: + self.parts.append(data) + + collector = Collector() + try: + collector.feed(value) + collector.close() + except Exception: + return "" + text = "".join(collector.parts) + text = text.replace("\r", "") + text = re.sub(r"[ \t]+\n", "\n", text) + text = re.sub(r"\n{3,}", "\n\n", text) + return text.strip() + + +def sanitize_html(value: str, images: dict[str, str] | None = None, *, remote=None, found: list[str] | None = None) -> str: + """Keep a formatted reading. Scripts and active content stay out. + + Pictures from other servers stay out too, unless `remote` turns an address into an inline picture. + `found` collects the addresses that were met.""" + images = images or {} + + class Cleaner(HTMLParser): + def __init__(self) -> None: + super().__init__(convert_charrefs=True) + self.parts: list[str] = [] + self.skip = 0 + + def handle_starttag(self, tag, attrs): + name = tag.lower() + if name in {"script", "style", "iframe", "object", "embed", "form", "link", "meta"}: + self.skip += 1 + return + if self.skip or name not in _ALLOWED_TAGS: + return + rendered = _clean_attrs(name, attrs, images, remote, found) + if rendered is None: + return + self.parts.append(f"<{name}{rendered}>") + + def handle_endtag(self, tag): + name = tag.lower() + if name in {"script", "style", "iframe", "object", "embed", "form", "link", "meta"}: + if self.skip: + self.skip -= 1 + return + if self.skip or name not in _ALLOWED_TAGS or name in _VOID_TAGS: + return + self.parts.append(f"") + + def handle_data(self, data): + if not self.skip: + self.parts.append(html.escape(data)) + + def handle_entityref(self, name): + if not self.skip: + self.parts.append(f"&{name};") + + def handle_charref(self, name): + if not self.skip: + self.parts.append(f"&#{name};") + + cleaner = Cleaner() + try: + cleaner.feed(value) + cleaner.close() + except Exception: + return "" + return "".join(cleaner.parts).strip() + + +def _safe_style(value: str) -> str: + kept = [] + for declaration in (value or "").split(";"): + prop, _, raw = declaration.partition(":") + prop, raw = prop.strip().lower(), raw.strip().replace("!important", "").strip() + if prop not in _STYLE_PROPS or not raw or _STYLE_DENIED.search(raw) or not _STYLE_VALUE.match(raw): + continue + kept.append(f"{prop}: {raw}") + return "; ".join(kept) + + +def _clean_attrs(tag: str, attrs, images: dict[str, str], remote=None, found: list[str] | None = None) -> str | None: + kept: list[str] = [] + for key, value in attrs: + name = (key or "").lower() + if name.startswith("on") or name == "srcset": + continue + if name == "style": + style = _safe_style(value or "") + if style: + kept.append(f' style="{html.escape(style, quote=True)}"') + continue + if name in {"bgcolor", "color"} and _COLOR.match((value or "").strip()): + kept.append(f' {name}="{html.escape(value.strip(), quote=True)}"') + continue + if name in {"width", "height"} and re.fullmatch(r"\d{1,4}%?", (value or "").strip()): + kept.append(f' {name}="{value.strip()}"') + continue + if name in {"border", "cellpadding", "cellspacing", "size"} and str(value or "").strip().isdigit(): + kept.append(f' {name}="{int(value)}"') + continue + if name in {"align", "valign"} and (value or "").strip().lower() in {"left", "right", "center", "justify", "top", "middle", "bottom"}: + kept.append(f' {name}="{value.strip().lower()}"') + continue + if tag == "a" and name == "href": + target = (value or "").strip() + if not _SAFE_LINK.match(target): + continue + kept.append(f' href="{html.escape(target, quote=True)}" title="{html.escape(target, quote=True)}"') + elif tag == "img" and name == "src": + source = (value or "").strip() + if source.lower().startswith("cid:"): + token = source[4:].strip().strip("<>") + source = images.get(token.casefold(), "") + elif re.match(r"https://", source, re.IGNORECASE): + if found is not None and source not in found: + found.append(source) + source = remote(source) if remote is not None else "" + if not _DATA_IMAGE.match(source): + continue + kept.append(f' src="{source}"') + elif tag == "img" and name == "alt": + kept.append(f' alt="{html.escape(value or "", quote=True)}"') + elif name in {"colspan", "rowspan"} and str(value or "").isdigit(): + kept.append(f" {name}=\"{int(value)}\"") + if tag == "img" and not any(item.startswith(" src=") for item in kept): + return None + if tag == "a": + kept.append(' rel="noreferrer noopener" target="_blank"') + return "".join(kept) + + +def _walk(message: EmailMessage): + if message.get_content_maintype() == "message" and message.get_content_subtype() == "rfc822": + yield "message", message + return + if message.is_multipart(): + for child in message.iter_parts(): + yield from _walk(child) + return + yield "leaf", message + + +def _decoded_text(part: EmailMessage) -> str | None: + try: + payload = part.get_payload(decode=True) + except Exception: + return None + if not isinstance(payload, (bytes, bytearray)): + raw = part.get_payload() + return raw.strip() if isinstance(raw, str) else None + charset = part.get_content_charset() or "utf-8" + try: + return decode_transfer(bytes(payload), charset).strip() + except LookupError: + return decode_transfer(bytes(payload), "utf-8").strip() + + +def _leaf_bytes(part: EmailMessage) -> bytes | None: + try: + payload = part.get_payload(decode=True) + except Exception: + return None + if isinstance(payload, (bytes, bytearray)): + return bytes(payload) + return None + + +def _message_bytes(part: EmailMessage) -> bytes | None: + payload = part.get_payload() + if isinstance(payload, list) and payload: + inner = payload[0] + if isinstance(inner, EmailMessage): + return inner.as_bytes() + data = _leaf_bytes(part) + return data or None + + +def _content_id(part: EmailMessage) -> str: + token = str(part.get("Content-ID") or "").strip().strip("<>").strip() + return token + + +def _is_pdf(data: bytes) -> bool: + return data[:1024].lstrip(b"\xef\xbb\xbf\r\n\t ").startswith(b"%PDF") + + +def _unwrap_pdf_payload(data: bytes, media: str, name: str) -> bytes: + """A PDF that arrived as undecoded Base64 is still the PDF, not the transfer text.""" + if _is_pdf(data): + return data + declared = (media or "").split(";", 1)[0].strip().lower() + hinted = "pdf" in declared or (name or "").lower().endswith(".pdf") or declared in { + "application/octet-stream", + "application/x-octet-stream", + "binary/octet-stream", + } + if not hinted: + return data + stripped = b"".join(data.split()) + if len(stripped) < 16 or re.fullmatch(rb"[A-Za-z0-9+/=]+", stripped) is None: + return data + try: + decoded = base64.b64decode(stripped, validate=True) + except Exception: + return data + return decoded if _is_pdf(decoded) else data + + +def _pdf_filename(name: str) -> str: + text = (name or "").strip() or "anlage" + if text.lower().endswith(".pdf"): + return text + if text.lower().endswith(".bin"): + text = text[:-4].rstrip(" .") or "anlage" + leaf = text.replace("\\", "/").split("/")[-1] + if "." in leaf: + return text + return f"{text}.pdf" + + +def _attachment(part: EmailMessage, filename: str, label: str, data: bytes, inline: bool, media: str | None = None) -> AttachmentBytes: + resolved = (media or part.get_content_type() or "application/octet-stream").split(";", 1)[0].strip() + return AttachmentBytes( + filename=filename, + media_type=(resolved or "application/octet-stream")[:120], + data=data, + sha256=hashlib.sha256(data).hexdigest(), + display_name=label or filename, + content_id=_content_id(part), + inline=inline, + ) + + +def _inline_images(attachments: list[AttachmentBytes]) -> dict[str, str]: + import base64 + + images: dict[str, str] = {} + for item in attachments: + if not item.content_id or not item.media_type.startswith("image/"): + continue + if len(item.data) > 1_500_000: + continue + subtype = item.media_type.split("/", 1)[1].split(";", 1)[0].lower() + if subtype == "jpg": + subtype = "jpeg" + if subtype not in {"png", "jpeg", "gif", "webp"}: + continue + encoded = base64.b64encode(item.data).decode("ascii") + images[item.content_id.casefold()] = f"data:image/{subtype};base64,{encoded}" + return images + + +def _message_date(value) -> str | None: + if not value: + return None + try: + return parsedate_to_datetime(str(value)).date().isoformat() + except (TypeError, ValueError, OverflowError): + return None + + def _bare(value: str) -> str: return value.strip().strip("<>").strip().casefold() -def _header_text(value) -> str: - if value is None: - return "" - text = str(value).replace("\r", " ").replace("\n", " ") - return re.sub(r"\s+", " ", text).strip() - - def _unique_filename(name: str, used: set[str]) -> str: cleaned = name.replace("\\", "/").split("/")[-1].strip() or "anlage.bin" - cleaned = cleaned[:180] + cleaned = re.sub(r"[\r\n\x00]", "", cleaned)[:180] or "anlage.bin" if cleaned not in used: used.add(cleaned) return cleaned stem, dot, ext = cleaned.rpartition(".") - if not dot: + if not dot or not stem: stem, ext = cleaned, "" else: ext = f".{ext}" @@ -140,36 +516,3 @@ def _unique_filename(name: str, used: set[str]) -> str: used.add(candidate) return candidate number += 1 - - -def _body_text(message) -> str: - plain: list[str] = [] - html: list[str] = [] - parts = message.walk() if message.is_multipart() else [message] - for part in parts: - if part.is_multipart() or part.get_content_disposition() == "attachment" or part.get_filename(): - continue - kind = part.get_content_type() - payload = part.get_payload(decode=True) - if isinstance(payload, (bytes, bytearray)): - charset = part.get_content_charset() or "utf-8" - text = bytes(payload).decode(charset, errors="replace").strip() - else: - raw = part.get_payload() - text = raw.strip() if isinstance(raw, str) else "" - if kind == "text/plain": - plain.append(text) - elif kind == "text/html": - html.append(_html_text(text)) - body = "\n".join(chunk for chunk in plain if chunk).strip() - if not body: - body = "\n".join(chunk for chunk in html if chunk).strip() - return body - - -def _html_text(value: str) -> str: - text = re.sub(r"(?is)<(script|style)\b.*?>.*?", " ", value) - text = re.sub(r"(?i)", "\n", text) - text = re.sub(r"(?i)

", "\n", text) - text = re.sub(r"<[^>]+>", " ", text) - return re.sub(r"[ \t]{2,}", " ", text).strip() diff --git a/backend/app/main.py b/backend/app/main.py index f69218f..30147bc 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -9,8 +9,14 @@ from fastapi import FastAPI, Request from fastapi.exceptions import RequestValidationError from fastapi.responses import JSONResponse +from app.api.accounts import router as accounts_router from app.api.correspondence import router as correspondence_router +from app.api.downloads import router as downloads_router from app.api.export import router as export_router +from app.api.library import router as library_router +from app.api.rooms import router as rooms_router +from app.api.roomshare import router as roomshare_router +from app.api.team import router as team_router from app.api.routes import router from app.api.sources import router as sources_router from app.core.errors import Problem, problem_response @@ -116,8 +122,14 @@ def create_app() -> FastAPI: ) app.include_router(router) + app.include_router(accounts_router) + app.include_router(library_router) + app.include_router(rooms_router) + app.include_router(roomshare_router) + app.include_router(team_router) app.include_router(sources_router) app.include_router(export_router) + app.include_router(downloads_router) app.include_router(correspondence_router) return app diff --git a/backend/app/media/port.py b/backend/app/media/port.py new file mode 100644 index 0000000..47fd9a8 --- /dev/null +++ b/backend/app/media/port.py @@ -0,0 +1,295 @@ +"""Opaque storage port. The adapter stores bytes and does not decide who may read them. + +Cleartext for a reader goes through require_cleartext. Locked profiles stay +bytes on disk and are not decrypted here. No new cryptography is implemented. +""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from uuid import UUID + +from app.core.errors import Problem, storage_unavailable, unprocessable +from app.media.storage import read_bytes, sha256, write_bytes + +LEGACY_GENERATION = "legacy-local-1" +LEGACY_PROFILE = "legacy-plaintext" +LOCKED_PROFILES = frozenset({"managed-encrypted", "client-encrypted"}) +PROFILES = frozenset({LEGACY_PROFILE, *LOCKED_PROFILES}) + + +def require_cleartext(profile: str | None) -> None: + name = profile or LEGACY_PROFILE + if name == LEGACY_PROFILE: + return + if name in LOCKED_PROFILES: + raise Problem( + 422, + "Geschützter Inhalt", + "Dieser Inhalt ist geschützt. Ein Klartextabruf ist in dieser Lieferung nicht möglich.", + "unsupported", + ) + raise Problem( + 422, + "Nicht unterstützt", + "Dieses Schutzprofil wird nicht unterstützt.", + "unsupported", + ) + + +def check_object_key(object_key: str) -> str: + key = (object_key or "").replace("\\", "/").strip() + parts = key.split("/") + if not key or key.startswith("/") or any(part in {"", ".", ".."} for part in parts): + raise unprocessable("Ungültiger Speicherpfad.") + try: + if len(parts) == 1: + UUID(parts[0]) + elif len(parts) == 2 and parts[0] == "dispatch": + UUID(parts[1]) + else: + raise ValueError + except ValueError: + raise unprocessable("Ungültiger Speicherpfad.") from None + return key + + +def object_path(root: Path, object_key: str) -> Path: + key = check_object_key(object_key) + base = Path(root).resolve() + base.mkdir(parents=True, exist_ok=True) + target = (base / key).resolve() + if base != target and base not in target.parents: + raise unprocessable("Ungültiger Speicherpfad.") + return target + + +class LocalAdapter: + """Files stay under one recorded root. A client path is not an address.""" + + def __init__(self, root: str | Path) -> None: + self.root = Path(root) + + def put(self, object_key: str, data: bytes) -> None: + path = object_path(self.root, object_key) + path.parent.mkdir(parents=True, exist_ok=True) + write_bytes(path, data) + + def open(self, object_key: str) -> bytes: + return read_bytes(object_path(self.root, object_key)) + + def delete(self, object_key: str) -> None: + path = object_path(self.root, object_key) + try: + path.unlink(missing_ok=True) + except OSError as exc: + raise storage_unavailable() from exc + + def describe(self, object_key: str) -> dict: + path = object_path(self.root, object_key) + try: + if not path.is_file(): + raise storage_unavailable() + return {"byte_size": path.stat().st_size, "object_key": object_key} + except OSError as exc: + raise storage_unavailable() from exc + + +class MemoryAdapter: + """In-memory stand-in for the same port. It is not a second product store.""" + + def __init__(self) -> None: + self._items: dict[str, bytes] = {} + self.fail_put = False + + def put(self, object_key: str, data: bytes) -> None: + if self.fail_put: + raise storage_unavailable() + key = check_object_key(object_key) + self._items[key] = bytes(data) + + def open(self, object_key: str) -> bytes: + key = check_object_key(object_key) + try: + return self._items[key] + except KeyError as exc: + raise storage_unavailable() from exc + + def delete(self, object_key: str) -> None: + key = check_object_key(object_key) + self._items.pop(key, None) + + def describe(self, object_key: str) -> dict: + data = self.open(object_key) + return {"byte_size": len(data), "object_key": object_key} + + +def publish(adapter, object_key: str, data: bytes) -> None: + """Make bytes readable only after a checked roundtrip. A failed put leaves nothing published.""" + adapter.put(object_key, data) + try: + opened = adapter.open(object_key) + except Problem: + adapter.delete(object_key) + raise + if opened != data: + adapter.delete(object_key) + raise storage_unavailable() + + +def ensure_legacy_instance(conn, media_root: str) -> dict: + """Bind the legacy generation to the current root once. Later roots do not retarget it.""" + root = str(Path(media_root).resolve()) + conn.execute( + """ + INSERT INTO storage_instances (kind, generation, root_path) + VALUES ('local', %s, %s) + ON CONFLICT (generation) DO NOTHING + """, + (LEGACY_GENERATION, root), + ) + conn.execute( + """ + UPDATE storage_instances + SET root_path = %s + WHERE generation = %s AND root_path IS NULL + """, + (root, LEGACY_GENERATION), + ) + row = conn.execute( + """ + SELECT id, kind, generation, root_path + FROM storage_instances + WHERE generation = %s + """, + (LEGACY_GENERATION,), + ).fetchone() + conn.execute( + """ + UPDATE media_assets + SET storage_instance_id = %s + WHERE storage_generation = %s AND storage_instance_id IS NULL + """, + (row["id"], LEGACY_GENERATION), + ) + conn.execute( + """ + UPDATE dispatch_snapshots + SET storage_instance_id = %s + WHERE storage_generation = %s AND storage_instance_id IS NULL + """, + (row["id"], LEGACY_GENERATION), + ) + return row + + +def instance_for_write(conn, media_root: str) -> dict: + legacy = ensure_legacy_instance(conn, media_root) + root = str(Path(media_root).resolve()) + if legacy["root_path"] == root: + return legacy + generation = "local-" + hashlib.sha256(root.encode("utf-8")).hexdigest()[:16] + conn.execute( + """ + INSERT INTO storage_instances (kind, generation, root_path) + VALUES ('local', %s, %s) + ON CONFLICT (generation) DO NOTHING + """, + (generation, root), + ) + return conn.execute( + """ + SELECT id, kind, generation, root_path + FROM storage_instances + WHERE generation = %s + """, + (generation,), + ).fetchone() + + +class StoredHandle: + """Rollback and delete go through the adapter that owns the bytes.""" + + def __init__(self, adapter: StoragePort, object_key: str): + self._adapter = adapter + self._key = object_key + + def discard(self) -> None: + self._adapter.delete(self._key) + + +_FACTORIES: dict[str, callable] = {} + + +def _local_factory(row: dict) -> StoragePort: + return LocalAdapter(row["root_path"]) + + +_FACTORIES["local"] = _local_factory + + +def adapter_for(row: dict) -> StoragePort: + factory = _FACTORIES.get(row["kind"]) + if factory is None or not row.get("root_path") and row["kind"] == "local": + raise storage_unavailable() + return factory(row) + + +def removal_for(conn, instance_id, generation: str | None, object_key: str, media_root: str) -> StoredHandle: + row = None + if instance_id is not None: + row = conn.execute( + "SELECT id, kind, generation, root_path FROM storage_instances WHERE id = %s", + (instance_id,), + ).fetchone() + if row is None: + root = resolve_root(conn, instance_id, generation, media_root) + return StoredHandle(LocalAdapter(root), object_key) + return StoredHandle(adapter_for(row), object_key) + + +def resolve_root(conn, instance_id, generation: str | None, media_root: str) -> str: + if instance_id is not None: + row = conn.execute( + "SELECT root_path FROM storage_instances WHERE id = %s", + (instance_id,), + ).fetchone() + if row is not None and row["root_path"]: + return row["root_path"] + if generation in {None, LEGACY_GENERATION}: + return ensure_legacy_instance(conn, media_root)["root_path"] + return str(Path(media_root).resolve()) + + +def stored_path(conn, instance_id, generation: str | None, object_key: str, media_root: str) -> Path: + return object_path(Path(resolve_root(conn, instance_id, generation, media_root)), object_key) + + +def read_clear_bytes( + conn, + *, + instance_id, + generation: str | None, + object_key: str, + media_root: str, + protection_profile: str | None, + expected_sha256: str | None, +) -> bytes: + require_cleartext(protection_profile) + row = None + if instance_id is not None: + row = conn.execute( + "SELECT id, kind, generation, root_path FROM storage_instances WHERE id = %s", + (instance_id,), + ).fetchone() + adapter = adapter_for(row) if row is not None else LocalAdapter(resolve_root(conn, instance_id, generation, media_root)) + data = adapter.open(object_key) + if expected_sha256 and sha256(data) != expected_sha256: + raise Problem( + 422, + "Inhalt nicht geprüft", + "Die gespeicherte Datei stimmt nicht mit dem erfassten Stand überein.", + "processing", + ) + return data diff --git a/backend/app/media/storage.py b/backend/app/media/storage.py index 5002fcb..1cc618e 100644 --- a/backend/app/media/storage.py +++ b/backend/app/media/storage.py @@ -4,6 +4,7 @@ from __future__ import annotations import hashlib from pathlib import Path +from urllib.parse import quote from uuid import UUID from app.core.errors import storage_unavailable, unprocessable @@ -66,6 +67,17 @@ def display_filename(name: str) -> str: return text[:240] -def content_disposition(filename: str) -> str: - safe = display_filename(filename).replace('"', "") - return f'attachment; filename="{safe}"' +def content_disposition(filename: str, *, inline: bool = False) -> str: + """The real name goes along in UTF-8 (RFC 5987), with a plain ASCII fallback for old clients.""" + name = display_filename(filename) + kind = "inline" if inline else "attachment" + suffix = "" + leaf = name.rsplit(".", 1) + if len(leaf) == 2 and leaf[1].isascii() and leaf[1].isalnum(): + suffix = f".{leaf[1]}" + fallback = name.encode("ascii", "replace").decode("ascii").replace("?", "_").replace('"', "").replace("\\", "_").replace(";", "_") + fallback = " ".join(fallback.split()) + if suffix and not fallback.lower().endswith(suffix.lower()): + fallback = f"{fallback}{suffix}".strip() + fallback = fallback or f"datei{suffix}" + return f"{kind}; filename=\"{fallback}\"; filename*=UTF-8''{quote(name, safe='')}" diff --git a/backend/app/ops.py b/backend/app/ops.py index 628a4b9..8b13ad0 100644 --- a/backend/app/ops.py +++ b/backend/app/ops.py @@ -8,6 +8,7 @@ from uuid import UUID, uuid4 from psycopg.types.json import Json +from app.access.roles import ensure_can_create, room_role from app.access.policy import ( Ctx, Rights, @@ -33,9 +34,10 @@ from app.access.policy import ( task_rights, ) from app.content.documents import clean_document -from app.core.errors import forbidden, not_found, unprocessable +from app.core.errors import forbidden, not_found, unprocessable, unsupported from app.core.timeutil import utcnow -from app.media.storage import asset_path, display_filename, sha256, sniff, write_bytes +from app.media.port import LEGACY_PROFILE, StoredHandle, adapter_for, instance_for_write, publish, removal_for +from app.media.storage import display_filename, sha256, sniff MAX_CHAPTER_DEPTH = 32 @@ -72,7 +74,135 @@ def _placement(context: Ctx, placement: Ctx | None) -> tuple[Ctx, Ctx]: return target, context if context.kind == "case" or placement is not None else None +def default_visibility(conn, scope_id: UUID) -> str: + """Whether new cases and matters are shared with their room at once. The personal room never is.""" + room = conn.execute("SELECT kind, default_visibility FROM data_rooms WHERE id = %s", (scope_id,)).fetchone() + if room is not None and room["kind"] == "organization" and room["default_visibility"] == "room": + return "room" + return "private" + + +def _auto_share(conn, actor: UUID, kind: str, resource: dict) -> None: + if default_visibility(conn, resource["home_scope_id"]) == "room": + conn.execute( + """ + INSERT INTO room_shares (resource_kind, resource_id, room_id, access, created_by) + VALUES (%s, %s, %s, 'edit', %s) + ON CONFLICT DO NOTHING + """, + (kind, resource["id"], resource["home_scope_id"], actor), + ) + + +def _shareable(conn, actor: UUID, kind: str, resource_id: UUID) -> dict: + """The own case or matter. Only the owner decides which rooms it is shared with.""" + if kind == "case": + resource = get_case(conn, resource_id) + elif kind == "matter": + resource = get_matter(conn, resource_id) + else: + raise unprocessable("Die Raumfreigabe gilt für Akten und Vorgänge.") + if resource is None or resource["owner_subject_id"] != actor: + raise not_found() + return resource + + +def _target_room(conn, actor: UUID, room_id: UUID) -> dict: + room = conn.execute("SELECT id, kind, status FROM data_rooms WHERE id = %s", (room_id,)).fetchone() + if room is None or room["kind"] != "organization" or room["status"] != "active": + raise not_found() + if room_role(conn, actor, room_id) not in {"member", "manager"}: + raise forbidden("Du kannst nur mit Datenräumen teilen, in denen du Mitglied oder Verwalter bist.") + return room + + +def share_with_room(conn, actor: UUID, kind: str, resource_id: UUID, room_id: UUID, access: str) -> dict: + if access not in {"read", "edit"}: + raise unprocessable("Die Freigabeart ist ungültig.") + resource = _shareable(conn, actor, kind, resource_id) + _target_room(conn, actor, room_id) + existing = conn.execute( + "SELECT access FROM room_shares WHERE resource_kind = %s AND resource_id = %s AND room_id = %s", + (kind, resource_id, room_id), + ).fetchone() + conn.execute( + """ + INSERT INTO room_shares (resource_kind, resource_id, room_id, access, created_by) + VALUES (%s, %s, %s, %s, %s) + ON CONFLICT (resource_kind, resource_id, room_id) DO UPDATE SET access = EXCLUDED.access + """, + (kind, resource_id, room_id, access, actor), + ) + if existing is None or existing["access"] != access: + audit( + conn, + actor, + resource["home_scope_id"], + "room_shared" if existing is None else "room_share_changed", + kind, + resource_id, + detail={"room_id": str(room_id), "access": access}, + ) + return {"room_id": str(room_id), "access": access} + + +def unshare_from_room(conn, actor: UUID, kind: str, resource_id: UUID, room_id: UUID) -> None: + resource = _shareable(conn, actor, kind, resource_id) + row = conn.execute( + "DELETE FROM room_shares WHERE resource_kind = %s AND resource_id = %s AND room_id = %s RETURNING id", + (kind, resource_id, room_id), + ).fetchone() + if row is None: + raise not_found() + audit(conn, actor, resource["home_scope_id"], "room_unshared", kind, resource_id, detail={"room_id": str(room_id)}) + + +def set_person_exclusion(conn, actor: UUID, kind: str, resource_id: UUID, subject_id: UUID, excluded: bool) -> None: + """Take one person out of the room shares of an own case or matter.""" + resource = _shareable(conn, actor, kind, resource_id) + if subject_id == actor: + raise unprocessable("Du kannst dich nicht selbst ausnehmen.") + member = conn.execute( + """ + SELECT 1 FROM room_shares s + JOIN room_memberships m ON m.room_id = s.room_id AND m.subject_id = %s AND m.status = 'active' + WHERE s.resource_kind = %s AND s.resource_id = %s + """, + (subject_id, kind, resource_id), + ).fetchone() + if member is None: + raise not_found() + conn.execute( + """ + DELETE FROM access_grants + WHERE is_exception AND grantee_subject_id = %s AND resource_kind = %s AND resource_id = %s + AND context_kind IS NULL + """, + (subject_id, kind, resource_id), + ) + if excluded: + conn.execute( + """ + INSERT INTO access_grants ( + grantee_subject_id, resource_kind, resource_id, + can_read, can_edit, can_share, can_delete, is_exception, created_by + ) + VALUES (%s, %s, %s, true, true, true, true, true, %s) + """, + (subject_id, kind, resource_id, actor), + ) + audit( + conn, + actor, + resource["home_scope_id"], + "person_excluded" if excluded else "person_included", + kind, + resource_id, + ) + + def create_case(conn, actor: UUID, scope_id: UUID, title: str, parent_case_id: UUID | None, rights_mode: str) -> dict: + ensure_can_create(conn, actor, scope_id) title = title.strip() if parent_case_id is None: row = conn.execute( @@ -103,6 +233,7 @@ def create_case(conn, actor: UUID, scope_id: UUID, title: str, parent_case_id: U """, (parent["home_scope_id"], actor, actor, title, parent["id"], rights_mode), ).fetchone() + _auto_share(conn, actor, "case", row) audit(conn, actor, row["home_scope_id"], "case_created", "case", row["id"]) return row @@ -338,6 +469,7 @@ def set_chapter_date( def create_matter(conn, actor: UUID, scope_id: UUID, title: str, parent_matter_id: UUID | None, context: Ctx | None) -> dict: + ensure_can_create(conn, actor, scope_id) title = title.strip() if parent_matter_id is None: row = conn.execute( @@ -363,6 +495,7 @@ def create_matter(conn, actor: UUID, scope_id: UUID, title: str, parent_matter_i """, (parent["home_scope_id"], actor, actor, title, parent["id"]), ).fetchone() + _auto_share(conn, actor, "matter", row) audit(conn, actor, row["home_scope_id"], "matter_created", "matter", row["id"], context) return row @@ -516,8 +649,9 @@ def move_bindings_chapter( def delete_case(conn, actor: UUID, case_id: UUID, expected: int) -> None: - case, rights = require_case(conn, actor, case_id, "edit") - if not rights.edit: + case, rights = require_case(conn, actor, case_id, "read") + # Deleting the container is the owner's right. Editing, also through the room share, is not enough. + if case["owner_subject_id"] != actor and not rights.delete: raise forbidden() children = conn.execute( "SELECT id FROM cases WHERE parent_case_id = %s AND deleted_at IS NULL", @@ -532,7 +666,9 @@ def delete_case(conn, actor: UUID, case_id: UUID, expected: int) -> None: def delete_matter(conn, actor: UUID, matter_id: UUID, expected: int, context: Ctx | None) -> None: - matter, _rights = require_matter(conn, actor, matter_id, context, "edit") + matter, rights = require_matter(conn, actor, matter_id, context, "read") + if matter["owner_subject_id"] != actor and not rights.delete: + raise forbidden() children = conn.execute( "SELECT id FROM matters WHERE parent_matter_id = %s AND deleted_at IS NULL", (matter_id,), @@ -701,6 +837,12 @@ def _actor_rights_for(conn, actor: UUID, kind: str, resource_id: UUID, context: return rights +def _is_reader_of_home(conn, account_id: UUID, home_scope_id: UUID) -> bool: + from app.access.roles import room_role + + return room_role(conn, account_id, home_scope_id) == "reader" + + def grant_access( conn, actor: UUID, @@ -740,6 +882,8 @@ def grant_access( ).fetchone() if account is None: raise not_found() + if (can_edit or can_share) and _is_reader_of_home(conn, grantee_id, resource["home_scope_id"]): + raise unprocessable("Diese Person ist in dem Datenraum der Akte nur Leser und kann nicht bearbeiten oder freigeben.") existing = conn.execute( """ SELECT id, use_owner_sources FROM access_grants @@ -908,6 +1052,7 @@ def create_note( received_on: date | None = None, communication_direction: str | None = None, ) -> dict: + ensure_can_create(conn, actor, scope_id) document = clean_document(document) target, rights_context = _placement(context, placement) require_container_edit(conn, actor, target, rights_context) @@ -929,7 +1074,7 @@ def update_note(conn, actor: UUID, element_id: UUID, expected: int, document: di document = clean_document(document) element, rights = require_element(conn, actor, element_id, context, "edit") if element["kind"] != "note": - raise not_found() + raise unsupported("Diese Fassung lässt sich nicht als Notiz bearbeiten.") if not rights.edit: raise forbidden() version = bump_version(conn, "elements", element_id, expected) @@ -998,6 +1143,11 @@ def add_annotation(conn, actor: UUID, binding_id: UUID, body: str, context: Ctx def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str, external_id: str, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, detail: dict | None = None, communication_direction: str | None = None) -> dict: + ensure_can_create(conn, actor, scope_id) + from app.sourceregistry import accepts_reference + + if not accepts_reference(source): + raise unprocessable("Diese Quelle ist nicht registriert.") target, rights_context = _placement(context, placement) require_container_edit(conn, actor, target, rights_context) element = _new_element(conn, actor, scope_id, title, "external_ref") @@ -1005,8 +1155,11 @@ def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str, "INSERT INTO external_references (element_id, source, external_id, detail) VALUES (%s, %s, %s, %s)", (element["id"], source, external_id.strip()[:200], Json(detail or {})), ) + from app.sourcebind import attach_new_reference + + attach_new_reference(conn, element["id"]) binding = insert_binding(conn, element["id"], target.kind, target.id, chapter_id, actor) - adopted = (detail or {}).get("direction") if source == "imap" else None + adopted = (detail or {}).get("direction") chosen = chosen_direction(communication_direction) or (adopted if adopted in ("in", "out") else None) store_direction(conn, binding["id"], chosen, chosen is not None) record_dates(conn, actor, scope_id, element["id"], rights_context or target, document_on, sent_on, received_on) @@ -1015,23 +1168,34 @@ def create_external(conn, actor: UUID, scope_id: UUID, title: str, source: str, def create_file(conn, actor: UUID, scope_id: UUID, title: str, filename: str, data: bytes, context: Ctx, placement: Ctx | None, chapter_id: UUID | None, document_on, sent_on, received_on, media_root: str, max_bytes: int, rollback_files: list[Path], communication_direction: str | None = None) -> dict: + ensure_can_create(conn, actor, scope_id) if not data or len(data) > max_bytes: raise unprocessable("Die Datei ist leer oder zu groß.") content_type = sniff(data) target, rights_context = _placement(context, placement) require_container_edit(conn, actor, target, rights_context) - element = _new_element(conn, actor, scope_id, title, "file") asset_id = uuid4() - path = asset_path(media_root, asset_id) - write_bytes(path, data) - rollback_files.append(path) + content_revision = str(uuid4()) + instance = instance_for_write(conn, media_root) + object_key = str(asset_id) + adapter = adapter_for(instance) + from app.sourceaccess import release_for_external + + release_for_external(conn) + publish(adapter, object_key, data) + rollback_files.append(StoredHandle(adapter, object_key)) + require_container_edit(conn, actor, target, rights_context) + element = _new_element(conn, actor, scope_id, title, "file") + digest = sha256(data) conn.execute( """ INSERT INTO media_assets ( id, element_id, owner_subject_id, home_scope_id, original_filename, - content_type, byte_size, sha256, status + content_type, byte_size, sha256, status, storage_instance_id, + storage_generation, object_key, content_revision, protection_profile, + key_responsibility ) - VALUES (%s, %s, %s, %s, %s, %s, %s, %s, 'active') + VALUES (%s, %s, %s, %s, %s, %s, %s, %s, 'active', %s, %s, %s, %s, %s, 'none') """, ( asset_id, @@ -1041,7 +1205,12 @@ def create_file(conn, actor: UUID, scope_id: UUID, title: str, filename: str, da display_filename(filename), content_type, len(data), - sha256(data), + digest, + instance["id"], + instance["generation"], + object_key, + content_revision, + LEGACY_PROFILE, ), ) conn.execute( @@ -1075,12 +1244,21 @@ def delete_element(conn, actor: UUID, element_id: UUID, expected: int, media_roo ) if asset is not None and asset["status"] == "active": conn.execute("UPDATE media_assets SET status = 'purging' WHERE id = %s", (asset["id"],)) - delete_after.append(asset_path(media_root, asset["id"])) + delete_after.append( + removal_for( + conn, + asset["storage_instance_id"], + asset["storage_generation"], + asset["object_key"] or str(asset["id"]), + media_root, + ) + ) conn.execute("UPDATE media_assets SET status = 'purged' WHERE id = %s", (asset["id"],)) audit(conn, actor, element["home_scope_id"], "element_deleted", "element", element_id) def create_task(conn, actor: UUID, scope_id: UUID, title: str, due_on: date | None, matter_id: UUID | None, context: Ctx | None) -> dict: + ensure_can_create(conn, actor, scope_id) if matter_id is not None: require_matter(conn, actor, matter_id, context, "edit") row = conn.execute( @@ -1186,6 +1364,7 @@ def assign_task(conn, actor: UUID, task_id: UUID, assignee_id: UUID | None, cont def create_deadline(conn, actor: UUID, scope_id: UUID, title: str, due_on: date, matter_id: UUID | None, context: Ctx | None) -> dict: + ensure_can_create(conn, actor, scope_id) if matter_id is not None: require_matter(conn, actor, matter_id, context, "edit") row = conn.execute( diff --git a/backend/app/present.py b/backend/app/present.py index b356096..4bb5d1d 100644 --- a/backend/app/present.py +++ b/backend/app/present.py @@ -117,10 +117,16 @@ def _preview_meta(kind: str, content_type: str | None, source: str | None) -> di return {"available": True, "media": "image", "source": "file"} if kind == "file" and media == "application/pdf": return {"available": True, "media": "pdf", "source": "file"} - if kind == "external_ref" and source == "paperless": - return {"available": True, "media": "paperless", "source": "paperless"} - if kind == "external_ref" and source == "imap": - return {"available": True, "media": "email", "source": "imap"} + if kind == "external_ref" and source: + from app.sourceregistry import get_source + + module = get_source(source) + if module is not None: + return { + "available": bool(module.capabilities.get("read_preview")), + "media": module.representation, + "source": source, + } if kind == "file": return {"available": False, "media": "file", "source": "file"} return {"available": False, "media": "none", "source": source} @@ -175,35 +181,57 @@ def _element_body(conn, actor: UUID, element: dict) -> dict: elif element["kind"] == "file": asset = conn.execute( """ - SELECT original_filename, content_type, byte_size, status + SELECT original_filename, content_type, byte_size, status, protection_profile FROM media_assets WHERE element_id = %s """, (element["id"],), ).fetchone() if asset is not None: + profile = asset["protection_profile"] or "legacy-plaintext" payload["file"] = { "filename": asset["original_filename"], "content_type": asset["content_type"], "byte_size": asset["byte_size"], "status": asset["status"], + "protection_profile": profile, } - payload["preview"] = _preview_meta("file", asset["content_type"], "file") + if profile != "legacy-plaintext": + payload["preview"] = {"available": False, "media": "locked", "source": "file"} + else: + payload["preview"] = _preview_meta("file", asset["content_type"], "file") elif element["kind"] == "external_ref": ref = conn.execute( - "SELECT source, external_id, detail FROM external_references WHERE element_id = %s", + """ + SELECT source, external_id, detail, connector_id, resolution + FROM external_references WHERE element_id = %s + """, (element["id"],), ).fetchone() if ref is not None: detail = dict(ref["detail"] or {}) - connector_owner = _connector_owner(conn, detail.get("connector_id")) + connector_owner = _connector_owner(conn, ref["connector_id"] or detail.get("connector_id")) revealed = actor == element["owner_subject_id"] or (connector_owner is not None and actor == connector_owner) if not revealed: detail = {key: value for key, value in detail.items() if key not in _HIDDEN_SOURCE_FIELDS} - external = {"source": ref["source"], "detail": detail} + resolution = ref["resolution"] or "unresolved" + external = {"source": ref["source"], "detail": detail, "resolution": resolution} if revealed: external["external_id"] = ref["external_id"] payload["external"] = external - payload["preview"] = _preview_meta("external_ref", None, ref["source"]) + from app.sourcebind import resolve_for_read + from app.sourceregistry import get_source + + module = get_source(ref["source"]) + if module is None: + payload["preview"] = _preview_meta("external_ref", None, ref["source"]) + else: + resolved = resolve_for_read(conn, element["owner_subject_id"], ref["source"], ref, None) + if resolved.code == "ok": + payload["preview"] = _preview_meta("external_ref", None, ref["source"]) + elif actor == element["owner_subject_id"]: + payload["preview"] = {"available": False, "media": "unassigned", "source": ref["source"]} + else: + payload["preview"] = {"available": False, "media": "none", "source": ref["source"]} return payload @@ -237,7 +265,7 @@ def communication_from_row(conn, binding: dict, element: dict) -> dict: "SELECT source, detail FROM external_references WHERE element_id = %s", (element["id"],), ).fetchone() - detail = (ref["detail"] or {}) if ref is not None and ref["source"] == "imap" else {} + detail = (ref["detail"] or {}) if ref is not None else {} if detail.get("direction") in ("in", "out"): source = detail["direction"] return {"direction": source, "explicit": False} @@ -719,8 +747,23 @@ def matter_contexts(conn, actor: UUID, matter: dict) -> list[dict]: return contexts +def _shared_rooms(conn, kind: str, resource_id: UUID) -> list[dict]: + rows = conn.execute( + """ + SELECT r.id, r.name, s.access FROM room_shares s JOIN data_rooms r ON r.id = s.room_id + WHERE s.resource_kind = %s AND s.resource_id = %s + ORDER BY r.name, r.id + """, + (kind, resource_id), + ).fetchall() + return [{"id": str(row["id"]), "name": row["name"], "access": row["access"]} for row in rows] + + def matter_summary(conn, actor: UUID, matter: dict) -> dict: + owner = matter["owner_subject_id"] == actor return { + "is_owner": owner, + "shared_with": _shared_rooms(conn, "matter", matter["id"]) if owner else [], "id": str(matter["id"]), "title": matter["title"], "status": matter["record_status"], @@ -1003,4 +1046,40 @@ LABELS = { "chapter_moved": "Kapitel umgehängt", "content_order": "Ordnungsart geändert", "content_reordered": "Eigene Reihenfolge geändert", + "account_created": "Konto angelegt", + "account_renamed": "Kontoname geändert", + "account_disabled": "Konto deaktiviert", + "account_enabled": "Konto aktiviert", + "operator_granted": "Betreiberrolle vergeben", + "operator_revoked": "Betreiberrolle entzogen", + "password_reset": "Passwort zurückgesetzt", + "password_changed": "Passwort geändert", + "sessions_revoked": "Sitzungen beendet", + "session_started": "Angemeldet", + "session_ended": "Abgemeldet", + "room_created": "Datenraum angelegt", + "room_renamed": "Datenraum umbenannt", + "room_archived": "Datenraum archiviert", + "room_restored": "Datenraum wieder aktiviert", + "membership_added": "Mitglied hinzugefügt", + "membership_role_changed": "Raumrolle geändert", + "membership_removed": "Mitglied entfernt", + "invitation_created": "Einladung erstellt", + "invitation_revoked": "Einladung widerrufen", + "invitation_accepted": "Einladung angenommen", + "visibility_changed": "Raumfreigabe geändert", + "employee_promoted": "Mitarbeiter zum Hauptnutzer gemacht", + "employee_disabled": "Mitarbeiter-Konto deaktiviert", + "employee_enabled": "Mitarbeiter-Konto aktiviert", + "employee_password_reset": "Passwort eines Mitarbeiters zurückgesetzt", + "employee_removed": "Mitarbeiter aus allen Räumen entfernt", + "room_shared": "Mit Datenraum geteilt", + "room_share_changed": "Freigabe an Datenraum geändert", + "room_unshared": "Freigabe an Datenraum beendet", + "person_excluded": "Person von der Raumfreigabe ausgenommen", + "person_included": "Person wieder in die Raumfreigabe aufgenommen", + "room_owner_changed": "Eigentümer des Datenraums gewechselt", + "room_default_changed": "Voreinstellung des Datenraums geändert", + "room_download_policy_changed": "Download-Regel des Datenraums geändert", + "download_defaults_changed": "Standard für Downloads geändert", } diff --git a/backend/app/remoteimages.py b/backend/app/remoteimages.py new file mode 100644 index 0000000..8a19173 --- /dev/null +++ b/backend/app/remoteimages.py @@ -0,0 +1,133 @@ +"""Fetching the pictures a mail refers to, on the server and only on request. + +The browser never contacts the sender: the server loads each picture, checks that it really is +a PNG, JPEG, GIF or WebP, and hands it over inline. This hides the reader's address and device. +It cannot hide that the mail was opened, because the sender sees the request. + +Only public https addresses are fetched. The address is resolved once, every result must be a +public one, and the connection goes to exactly that address, so a changing DNS answer cannot +lead to an internal host. Redirects are followed a few times with the same checks. +""" + +from __future__ import annotations + +import base64 +import http.client +import ipaddress +import socket +import ssl +from urllib.parse import urljoin, urlsplit + +MAX_IMAGE_BYTES = 1_500_000 +MAX_REDIRECTS = 3 +TIMEOUT_SECONDS = 5 +MAX_IMAGES = 12 +MAX_TOTAL_BYTES = 6_000_000 + +_SIGNATURES = ( + (b"\x89PNG\r\n\x1a\n", "png"), + (b"\xff\xd8\xff", "jpeg"), + (b"GIF87a", "gif"), + (b"GIF89a", "gif"), +) + + +class _Pinned(http.client.HTTPSConnection): + """An https connection that goes to a fixed, already checked address.""" + + def __init__(self, host: str, address: str, port: int) -> None: + super().__init__(host, port, timeout=TIMEOUT_SECONDS, context=ssl.create_default_context()) + self._address = address + + def connect(self) -> None: + sock = socket.create_connection((self._address, self.port), self.timeout) + self.sock = self._context.wrap_socket(sock, server_hostname=self.host) + + +def is_public(address: str) -> bool: + try: + ip = ipaddress.ip_address(address) + except ValueError: + return False + return ip.is_global and not ip.is_multicast + + +def _public_address(host: str, port: int) -> str | None: + try: + found = {item[4][0] for item in socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)} + except OSError: + return None + if not found or not all(is_public(address) for address in found): + return None + return sorted(found)[0] + + +def _kind(data: bytes) -> str | None: + for signature, kind in _SIGNATURES: + if data.startswith(signature): + return kind + if data[:4] == b"RIFF" and data[8:12] == b"WEBP": + return "webp" + return None + + +def fetch_image(url: str) -> bytes | None: + """The bytes of a picture, or None when anything about it is not safe.""" + for _ in range(MAX_REDIRECTS + 1): + parts = urlsplit(url) + if parts.scheme != "https" or not parts.hostname or parts.username or parts.password: + return None + port = parts.port or 443 + if port != 443: + return None + address = _public_address(parts.hostname, port) + if address is None: + return None + target = (parts.path or "/") + (f"?{parts.query}" if parts.query else "") + connection = _Pinned(parts.hostname, address, port) + try: + connection.request("GET", target, headers={"User-Agent": "Mozilla/5.0", "Accept": "image/png,image/jpeg,image/gif,image/webp", "Connection": "close"}) + response = connection.getresponse() + if response.status in {301, 302, 303, 307, 308}: + location = response.getheader("Location") + if not location: + return None + url = urljoin(url, location) + continue + if response.status != 200: + return None + data = response.read(MAX_IMAGE_BYTES + 1) + except (OSError, http.client.HTTPException, ssl.SSLError): + return None + finally: + connection.close() + if len(data) > MAX_IMAGE_BYTES or _kind(data) is None: + return None + return data + return None + + +class RemoteImages: + """Loads the pictures of one mail within a budget and remembers each address once.""" + + def __init__(self) -> None: + self.cache: dict[str, str] = {} + self.total = 0 + self.loaded = 0 + self.failed = 0 + + def __call__(self, url: str) -> str: + if url in self.cache: + return self.cache[url] + result = "" + if len(self.cache) < MAX_IMAGES and self.total < MAX_TOTAL_BYTES: + data = fetch_image(url) + if data is not None: + self.total += len(data) + result = f"data:image/{_kind(data)};base64,{base64.b64encode(data).decode('ascii')}" + self.cache[url] = result + if result: + self.loaded += 1 + else: + self.failed += 1 + return result diff --git a/backend/app/retrieval.py b/backend/app/retrieval.py index 33300b4..050a2db 100644 --- a/backend/app/retrieval.py +++ b/backend/app/retrieval.py @@ -10,17 +10,12 @@ from dataclasses import dataclass, field from uuid import UUID from app.core.errors import Problem -from app.mailpack import ParsedMail, identities_match, parse_mail -from app.media.storage import asset_path, read_bytes -from app.sources import COMPLETE_MAIL_LIMIT, pull_paperless_original, pull_rfc822 +from app.mailpack import ParsedMail +from app.media.port import read_clear_bytes +from app.sources import COMPLETE_MAIL_LIMIT EXPORT_BYTE_LIMIT = COMPLETE_MAIL_LIMIT -_TOO_LARGE = ( - "Die vollständige Nachricht überschreitet die Exportgrenze. " - "Eine gekürzte Vorschau wird nicht als Original übernommen." -) - @dataclass(frozen=True) class RetrievalFailure: @@ -45,132 +40,59 @@ class MailPayload: def load_external(conn, actor_id: UUID, element_id: UUID, assigned: UUID | None) -> FilePayload | MailPayload | RetrievalFailure: + from app.sourceregistry import get_source + ref = conn.execute( - "SELECT source, external_id, detail FROM external_references WHERE element_id = %s", + """ + SELECT element_id, source, external_id, detail, connector_id, resolution + FROM external_references WHERE element_id = %s + """, (element_id,), ).fetchone() if ref is None: return RetrievalFailure("missing", "Die Quellenreferenz fehlt.") - source = ref["source"] - detail = ref["detail"] or {} - if source == "paperless": - return _load_paperless(conn, actor_id, ref["external_id"], detail, assigned) - if source == "imap": - return _load_imap(conn, actor_id, ref["external_id"], detail, assigned) - if source == "url": - return RetrievalFailure("capability", "Für eine Webadresse ist kein Originalabruf vorgesehen.") - return RetrievalFailure("capability", "Für diese Quelle ist kein Originalabruf vorgesehen.") + module = get_source(ref["source"]) + if module is None or module.export_original is None: + return RetrievalFailure("capability", "Für diese Quelle ist kein Originalabruf vorgesehen.") + return module.export_original(conn, actor_id, ref["external_id"], ref["detail"] or {}, assigned, ref) def load_file(conn, element_id: UUID, media_root: str) -> FilePayload | RetrievalFailure: asset = conn.execute( """ - SELECT id, original_filename, content_type, sha256, status + SELECT id, original_filename, content_type, sha256, status, + storage_instance_id, storage_generation, object_key, protection_profile FROM media_assets WHERE element_id = %s """, (element_id,), ).fetchone() if asset is None or asset["status"] != "active": return RetrievalFailure("missing", "Die Originaldatei ist nicht mehr vorhanden.") + if (asset["protection_profile"] or "legacy-plaintext") != "legacy-plaintext": + return RetrievalFailure( + "unsupported", + "Dieser Inhalt ist geschützt und kann hier nicht im Klartext gelesen werden.", + ) try: - data = read_bytes(asset_path(media_root, asset["id"])) - except Problem: + data = read_clear_bytes( + conn, + instance_id=asset["storage_instance_id"], + generation=asset["storage_generation"], + object_key=asset["object_key"] or str(asset["id"]), + media_root=media_root, + protection_profile=asset["protection_profile"], + expected_sha256=asset["sha256"], + ) + except Problem as exc: + if exc.code == "unsupported": + return RetrievalFailure("unsupported", exc.detail) return RetrievalFailure("missing", "Die Originaldatei ist nicht mehr vorhanden.") if len(data) > EXPORT_BYTE_LIMIT: return RetrievalFailure("too_large", "Die Originaldatei ist für den Export zu groß.") return FilePayload(asset["original_filename"], asset["content_type"], data, asset["sha256"]) -def _load_paperless(conn, actor_id: UUID, external_id: str, detail: dict, assigned: UUID | None): - stored = _uuid(detail.get("connector_id")) - resolved = _resolve_connector(conn, actor_id, "paperless", stored, assigned) - if isinstance(resolved, RetrievalFailure): - return resolved - try: - data, media, name = pull_paperless_original( - resolved["settings"]["base_url"].rstrip("/"), - resolved["secret"], - str(external_id), - EXPORT_BYTE_LIMIT, - ) - except Problem as exc: - return _from_problem(exc) - if len(data) > EXPORT_BYTE_LIMIT: - return RetrievalFailure("too_large", "Die Originaldatei ist für den Export zu groß.") - return FilePayload(name or f"paperless-{external_id}", media, data, "") - - -def _load_imap(conn, actor_id: UUID, external_id: str, detail: dict, assigned: UUID | None): - stored = _uuid(detail.get("connector_id")) - resolved = _resolve_connector(conn, actor_id, "imap", stored, assigned) - if isinstance(resolved, RetrievalFailure): - return resolved - mailbox = str(detail.get("mailbox") or "").strip() - uid = detail.get("uid") - if isinstance(uid, str) and uid.isdigit(): - uid = int(uid) - if not mailbox or isinstance(uid, bool) or not isinstance(uid, int) or uid < 1: - return RetrievalFailure("identity", "Für diese E-Mail fehlt die Zuordnung zu Fach und Nachricht.") - try: - raw = pull_rfc822(resolved["settings"], resolved["secret"], mailbox, uid, str(external_id), EXPORT_BYTE_LIMIT) - except Problem as exc: - return _from_problem(exc) - if len(raw) > EXPORT_BYTE_LIMIT: - return RetrievalFailure("too_large", _TOO_LARGE) - parsed = parse_mail(raw) - if not identities_match(parsed.message_id, str(external_id)): - return RetrievalFailure("identity", "Die gefundene Nachricht ist nicht die referenzierte E-Mail.") - if parsed.missing: - names = ", ".join(parsed.missing[:5]) - return RetrievalFailure("attachment", f"Eine Anlage der E-Mail ließ sich nicht vollständig lesen: {names}.") - return MailPayload(raw, parsed) - - -def _resolve_connector(conn, actor_id: UUID, kind: str, stored: UUID | None, assigned: UUID | None): - rows = conn.execute( - """ - SELECT id, kind, name, settings, secret - FROM connectors - WHERE owner_id = %s AND kind = %s - ORDER BY name, id - """, - (actor_id, kind), - ).fetchall() - if stored is not None and assigned is None: - match = next((row for row in rows if row["id"] == stored), None) - if match is None: - return RetrievalFailure("missing", "Der gespeicherte Zugang für diese Quelle ist nicht mehr vorhanden.") - return match - chosen = assigned or stored - if chosen is not None: - match = next((row for row in rows if row["id"] == chosen), None) - if match is None: - return RetrievalFailure( - "assignment", - "Der gewählte Zugang gehört nicht zu diesem Konto oder passt nicht zur Quelle.", - ) - return match - if len(rows) == 1: - return rows[0] - if not rows: - return RetrievalFailure("missing", "Für diese Quelle ist kein Zugang vorhanden.") - return RetrievalFailure( - "ambiguous", - "Für diese Quelle ist kein eindeutiger Zugang zugeordnet. Die Zuordnung muss bewusst gewählt werden.", - tuple({"id": str(row["id"]), "name": row["name"]} for row in rows), - ) - - -def _uuid(value) -> UUID | None: - if not value: - return None - try: - return UUID(str(value)) - except ValueError: - return None - - -def _from_problem(exc: Problem) -> RetrievalFailure: +def failure_from_problem(exc: Problem) -> RetrievalFailure: text = exc.detail lowered = text.casefold() if "exportgrenze" in lowered or "zu groß" in lowered: diff --git a/backend/app/seed.py b/backend/app/seed.py index d467e86..c80c09d 100644 --- a/backend/app/seed.py +++ b/backend/app/seed.py @@ -87,11 +87,11 @@ def seed_demo(settings: Settings | None = None) -> bool: ben = create_account(conn, BEN_EMAIL, "Ben Beispiel", BEN_PASSWORD, origin="fixture") room = conn.execute( """ - INSERT INTO data_rooms (id, kind, name, created_by) - VALUES (gen_random_uuid(), 'organization', 'Haushalt Musterstraße', %s) + INSERT INTO data_rooms (id, kind, name, created_by, owner_id) + VALUES (gen_random_uuid(), 'organization', 'Haushalt Musterstraße', %s, %s) RETURNING id """, - (ada["id"],), + (ada["id"], ada["id"]), ).fetchone() conn.execute( "INSERT INTO room_memberships (room_id, subject_id, role) VALUES (%s, %s, 'manager'), (%s, %s, 'member')", diff --git a/backend/app/sourceaccess.py b/backend/app/sourceaccess.py new file mode 100644 index 0000000..a6de756 --- /dev/null +++ b/backend/app/sourceaccess.py @@ -0,0 +1,138 @@ +"""Rights stay in one place. A source module only fetches its own bytes.""" + +from __future__ import annotations + +from dataclasses import dataclass +from uuid import UUID + +import psycopg + +from app.access.policy import Ctx, audit, mediated_source_owner +from app.core.errors import Problem, not_found, unprocessable +from app.sourcebind import resolve_for_read + +ORIGINAL_UNAVAILABLE = "Das Original ist über die Freigabe nicht abrufbar." + + +@dataclass +class BorrowedOriginal: + row: dict + external_id: str + mediated: bool + actor_id: UUID + owner_id: UUID + element: dict + context: Ctx | None + kind: str + + +def release_for_external(conn) -> None: + """External calls must not sit inside an open domain transaction.""" + if conn.info.transaction_status != psycopg.pq.TransactionStatus.IDLE: + conn.commit() + + +def connector_owner(conn, connector_id: UUID | None) -> UUID | None: + if connector_id is None: + return None + row = conn.execute("SELECT owner_id FROM connectors WHERE id = %s", (connector_id,)).fetchone() + return row["owner_id"] if row else None + + +def assert_still_owned(conn, connector_id, owner_id: UUID) -> None: + row = conn.execute( + "SELECT id FROM connectors WHERE id = %s AND owner_id = %s", + (connector_id, owner_id), + ).fetchone() + if row is None: + raise not_found() + + +def audit_mediated(conn, actor_id: UUID, element: dict, context: Ctx | None, source: str, connector_id, outcome: str) -> None: + audit( + conn, + actor_id, + element["home_scope_id"], + "source_read_mediated", + "element", + element["id"], + context, + result="ok" if outcome == "ok" else "unavailable", + detail={ + "source": source, + "connector_id": str(connector_id) if connector_id else None, + "outcome": outcome, + }, + ) + if outcome != "ok": + conn.commit() + + +def _uuid(value) -> UUID | None: + if isinstance(value, UUID): + return value + try: + return UUID(str(value)) + except (TypeError, ValueError): + return None + + +def borrow_original(conn, actor_id: UUID, element: dict, context: Ctx | None, ref, kind: str) -> BorrowedOriginal: + """Resolve the owner's connector. A grantee never supplies another account's connector.""" + detail = (ref or {}).get("detail") or {} + external_id = str((ref or {}).get("external_id") or "") + stored = _uuid(detail.get("connector_id")) or _uuid((ref or {}).get("connector_id")) + stored_owner = connector_owner(conn, stored) + mediated = not (element["owner_subject_id"] == actor_id or stored_owner == actor_id) + if mediated: + owner_id = mediated_source_owner(conn, actor_id, element, context) + if owner_id is None: + raise unprocessable(ORIGINAL_UNAVAILABLE) + else: + owner_id = actor_id + result = resolve_for_read(conn, owner_id, kind, ref, None) + if result.row is None: + if mediated: + audit_mediated(conn, actor_id, element, context, kind, stored, "unavailable") + raise unprocessable(ORIGINAL_UNAVAILABLE) + raise unprocessable(result.message) + return BorrowedOriginal( + row=result.row, + external_id=external_id, + mediated=mediated, + actor_id=actor_id, + owner_id=owner_id, + element=element, + context=context, + kind=kind, + ) + + +def _recheck(conn, borrowed: BorrowedOriginal) -> None: + if borrowed.mediated: + owner_id = mediated_source_owner(conn, borrowed.actor_id, borrowed.element, borrowed.context) + if owner_id != borrowed.owner_id: + audit_mediated(conn, borrowed.actor_id, borrowed.element, borrowed.context, borrowed.kind, borrowed.row["id"], "unavailable") + raise unprocessable(ORIGINAL_UNAVAILABLE) + row = conn.execute("SELECT owner_id FROM connectors WHERE id = %s", (borrowed.row["id"],)).fetchone() + if row is None or row["owner_id"] != borrowed.owner_id: + if borrowed.mediated: + audit_mediated(conn, borrowed.actor_id, borrowed.element, borrowed.context, borrowed.kind, borrowed.row["id"], "unavailable") + raise unprocessable(ORIGINAL_UNAVAILABLE) + raise not_found() + + +def complete_borrow(conn, borrowed: BorrowedOriginal, fetch): + """Run the adapter fetch outside the open transaction, then check the grant again.""" + release_for_external(conn) + try: + value = fetch(borrowed.row) + except Problem: + if borrowed.mediated: + audit_mediated(conn, borrowed.actor_id, borrowed.element, borrowed.context, borrowed.kind, borrowed.row["id"], "unavailable") + raise unprocessable(ORIGINAL_UNAVAILABLE) from None + raise + _recheck(conn, borrowed) + if borrowed.mediated: + audit_mediated(conn, borrowed.actor_id, borrowed.element, borrowed.context, borrowed.kind, borrowed.row["id"], "ok") + return value diff --git a/backend/app/sourcebind.py b/backend/app/sourcebind.py new file mode 100644 index 0000000..a5a0ae8 --- /dev/null +++ b/backend/app/sourcebind.py @@ -0,0 +1,258 @@ +"""Bind a reference to one connector instance. + +The newest connection is never a fallback. A stored id that no longer +exists stays missing. Ambiguous stock waits for an explicit assignment. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from uuid import UUID + +from psycopg.types.json import Json + +from app.access.policy import get_element +from app.core.errors import not_found, unprocessable +from app.sourceregistry import get_source + + +@dataclass(frozen=True) +class ResolveResult: + row: dict | None + code: str + message: str + choices: tuple[dict, ...] = () + + +def _uuid(value) -> UUID | None: + if isinstance(value, UUID): + return value + if not value: + return None + try: + return UUID(str(value)) + except (TypeError, ValueError): + return None + + +def stored_id(ref) -> UUID | None: + if not ref: + return None + detail = ref["detail"] if "detail" in ref.keys() else None + detail = detail or {} + return _uuid(detail.get("connector_id")) or _uuid(ref["connector_id"] if "connector_id" in ref.keys() else None) + + +def _connectors(conn, owner_id: UUID, kind: str) -> list[dict]: + return conn.execute( + """ + SELECT id, owner_id, kind, name, settings, secret + FROM connectors + WHERE owner_id = %s AND kind = %s + ORDER BY name, id + """, + (owner_id, kind), + ).fetchall() + + +def resolve_for_read(conn, owner_id: UUID, kind: str, ref, assigned: UUID | None) -> ResolveResult: + """Resolve one access. Assignment applies only when the reference has no stored id.""" + stored = stored_id(ref) + resolution = (ref["resolution"] if ref and "resolution" in ref.keys() and ref["resolution"] else None) or "unresolved" + rows = _connectors(conn, owner_id, kind) + if stored is not None: + match = next((row for row in rows if row["id"] == stored), None) + if match is None: + return ResolveResult(None, "missing", "Der gespeicherte Zugang für diese Quelle ist nicht mehr vorhanden.") + return ResolveResult(match, "ok", "") + if assigned is not None: + match = next((row for row in rows if row["id"] == assigned), None) + if match is None: + return ResolveResult( + None, + "assignment", + "Der gewählte Zugang gehört nicht zu diesem Konto oder passt nicht zur Quelle.", + ) + return ResolveResult(match, "ok", "") + if resolution == "ambiguous" or len(rows) != 1: + if not rows: + return ResolveResult(None, "missing", "Für diese Quelle ist kein Zugang vorhanden.") + return ResolveResult( + None, + "ambiguous", + "Für diese Quelle ist kein eindeutiger Zugang zugeordnet. Die Zuordnung muss bewusst gewählt werden.", + tuple({"id": str(row["id"]), "name": row["name"]} for row in rows), + ) + return ResolveResult(rows[0], "ok", "") + + +def _set_binding(conn, element_id: UUID, connector_id: UUID | None, resolution: str, detail: dict | None = None) -> None: + if detail is None: + conn.execute( + """ + UPDATE external_references + SET connector_id = %s, resolution = %s + WHERE element_id = %s + """, + (connector_id, resolution, element_id), + ) + return + conn.execute( + """ + UPDATE external_references + SET connector_id = %s, resolution = %s, detail = %s + WHERE element_id = %s + """, + (connector_id, resolution, Json(detail), element_id), + ) + + +def attach_new_reference(conn, element_id: UUID) -> None: + row = conn.execute( + """ + SELECT er.source, er.detail, e.owner_subject_id + FROM external_references er + JOIN elements e ON e.id = er.element_id + WHERE er.element_id = %s + """, + (element_id,), + ).fetchone() + if row is None: + return + if get_source(row["source"]) is None: + _set_binding(conn, element_id, None, "none") + return + detail = dict(row["detail"] or {}) + stored = _uuid(detail.get("connector_id")) + if stored is not None: + match = conn.execute( + """ + SELECT id FROM connectors + WHERE id = %s AND owner_id = %s AND kind = %s + """, + (stored, row["owner_subject_id"], row["source"]), + ).fetchone() + _set_binding(conn, element_id, stored, "bound" if match else "unresolved") + return + rows = _connectors(conn, row["owner_subject_id"], row["source"]) + if len(rows) == 1: + detail["connector_id"] = str(rows[0]["id"]) + _set_binding(conn, element_id, rows[0]["id"], "bound", detail) + return + if len(rows) > 1: + _set_binding(conn, element_id, None, "ambiguous") + return + _set_binding(conn, element_id, None, "unresolved") + + +def backfill_source_bindings(conn) -> None: + """Idempotent. Unique stock is bound once. A stored id is never retargeted. + + Startup calls this before the application has imported the connector + modules. Register them first, otherwise every existing reference stays + unresolved and the case or matter can no longer open its originals. + """ + from app.sources import _install_sources + + _install_sources() + conn.execute( + """ + UPDATE external_references + SET resolution = 'none' + WHERE source = 'url' AND resolution = 'unresolved' + """ + ) + rows = conn.execute( + """ + SELECT er.element_id, er.source, er.detail, e.owner_subject_id + FROM external_references er + JOIN elements e ON e.id = er.element_id + WHERE er.source <> 'url' + AND er.resolution IN ('unresolved', 'ambiguous') + AND er.connector_id IS NULL + """ + ).fetchall() + for row in rows: + if get_source(row["source"]) is None: + continue + detail = dict(row["detail"] or {}) + stored = _uuid(detail.get("connector_id")) + if stored is not None: + match = conn.execute( + """ + SELECT id FROM connectors + WHERE id = %s AND owner_id = %s AND kind = %s + """, + (stored, row["owner_subject_id"], row["source"]), + ).fetchone() + _set_binding(conn, row["element_id"], stored, "bound" if match else "unresolved") + continue + found = _connectors(conn, row["owner_subject_id"], row["source"]) + if len(found) == 1: + detail["connector_id"] = str(found[0]["id"]) + _set_binding(conn, row["element_id"], found[0]["id"], "bound", detail) + elif len(found) > 1: + _set_binding(conn, row["element_id"], None, "ambiguous") + + +def assign_reference(conn, actor_id: UUID, element_id: UUID, connector_id: UUID) -> None: + element = get_element(conn, element_id) + if element is None or element["kind"] != "external_ref" or element["owner_subject_id"] != actor_id: + raise not_found() + ref = conn.execute( + "SELECT source, detail FROM external_references WHERE element_id = %s", + (element_id,), + ).fetchone() + if ref is None or get_source(ref["source"]) is None: + raise unprocessable("Diese Referenz hat keinen Quellzugang.") + connector = conn.execute( + """ + SELECT id, kind FROM connectors + WHERE id = %s AND owner_id = %s + """, + (connector_id, actor_id), + ).fetchone() + if connector is None or connector["kind"] != ref["source"]: + raise unprocessable("Der Zugang gehört nicht zu diesem Konto oder passt nicht zur Quelle.") + detail = dict(ref["detail"] or {}) + detail["connector_id"] = str(connector_id) + _set_binding(conn, element_id, connector_id, "bound", detail) + + +def remember_imap_location( + conn, + element_id: UUID, + external_id: str, + previous_mailbox: str, + previous_uid: int, + mailbox: str, + uid: int, +) -> None: + """Store the folder and UID of a message found at a new place. + + Only the previous place is replaced. The message identity, the connector + and an adopted direction stay as they were. + """ + if ( + not mailbox + or any(char in mailbox for char in "\r\n\x00") + or uid < 1 + or (mailbox == previous_mailbox and uid == previous_uid) + ): + return + conn.execute( + """ + UPDATE external_references + SET detail = jsonb_set( + jsonb_set(detail, '{mailbox}', to_jsonb(%s::text)), + '{uid}', + to_jsonb(%s::bigint) + ) + WHERE element_id = %s + AND source = 'imap' + AND external_id = %s + AND detail->>'mailbox' = %s + AND detail->>'uid' = %s + """, + (mailbox, uid, element_id, external_id, previous_mailbox, str(previous_uid)), + ) diff --git a/backend/app/sourceport.py b/backend/app/sourceport.py new file mode 100644 index 0000000..87d1428 --- /dev/null +++ b/backend/app/sourceport.py @@ -0,0 +1,111 @@ +"""Declared source and editor capabilities. + +A common contract does not invent abilities the adapter does not have. +A filter is offered only when that module implements it. +""" + +from __future__ import annotations + +PAPERLESS_CAPABILITIES = { + "kind": "paperless", + "navigate": False, + "read_original": True, + "read_preview": True, + "read_attachment": False, + "edit_content": False, + "edit_metadata": False, + "search": { + "query": True, + "title": True, + "correspondent": True, + "document_type": True, + "tags": True, + "document_range": True, + "sort": True, + "continuation": True, + }, +} + +IMAP_CAPABILITIES = { + "kind": "imap", + "navigate": True, + "read_original": True, + "read_preview": True, + "read_attachment": True, + "edit_content": False, + "edit_metadata": False, + "search": { + "query": True, + "title": False, + "correspondent": False, + "document_type": False, + "tags": False, + "document_range": False, + "sort": False, + "continuation": True, + }, +} + +CAPABILITIES = { + "paperless": PAPERLESS_CAPABILITIES, + "imap": IMAP_CAPABILITIES, +} + +EDITORS = { + "note": True, + "markdown_file": False, +} + +UNSUPPORTED_SEARCH = ( + "title", + "correspondent", + "document_type", + "tag", + "document_from", + "document_to", + "sort", + "cursor", +) + + +def capability_payload() -> dict: + from app.sourceregistry import registered_sources + + return { + "sources": [module.capabilities for module in registered_sources()], + "modules": [ + { + "kind": module.kind, + "label": module.label, + "representation": module.representation, + "config_fields": [item.public() for item in module.config_fields], + } + for module in registered_sources() + ], + "editors": EDITORS, + "protection_profiles": { + "legacy-plaintext": "cleartext", + "managed-encrypted": "locked", + "client-encrypted": "locked", + }, + } + + +def unsupported_filters(values: dict[str, str]) -> list[str]: + return [name for name in UNSUPPORTED_SEARCH if str(values.get(name) or "").strip()] + + +def with_capabilities(kind: str, connector_id: str, payload: dict) -> dict: + from app.sourceregistry import require_source + + body = dict(payload) + spec = require_source(kind).capabilities + body["connector_id"] = connector_id + body["capabilities"] = spec + body["unsupported"] = [name for name, enabled in spec["search"].items() if not enabled] + if spec["search"].get("continuation"): + body["continuation"] = body.get("next_before") if body.get("next_before") is not None else body.get("next_page") + else: + body["continuation"] = None + body["incomplete"] = bool(body.get("incomplete")) + return body diff --git a/backend/app/sourceregistry.py b/backend/app/sourceregistry.py new file mode 100644 index 0000000..6c7c386 --- /dev/null +++ b/backend/app/sourceregistry.py @@ -0,0 +1,162 @@ +"""Registered source modules. Unknown kinds are rejected, not guessed.""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Callable + +from app.core.errors import unprocessable + +FIELD_TYPES = ("text", "url", "number", "password", "select") + + +@dataclass(frozen=True) +class ConfigField: + """One configuration field a module declares for its connector instances. + + At most one field per module is a secret. Its value is stored apart from + the settings and is never handed out again. + """ + + key: str + label: str + type: str = "text" + required: bool = False + secret: bool = False + default: str | int | None = None + options: tuple[tuple[str, str], ...] = () + help: str = "" + placeholder: str = "" + minimum: int | None = None + maximum: int | None = None + max_length: int = 200 + + def public(self) -> dict: + return { + "key": self.key, + "label": self.label, + "type": self.type, + "required": self.required, + "secret": self.secret, + "default": self.default, + "options": [{"value": value, "label": label} for value, label in self.options], + "help": self.help, + "placeholder": self.placeholder, + "minimum": self.minimum, + "maximum": self.maximum, + "max_length": self.max_length, + } + + +@dataclass +class SourceModule: + kind: str + label: str + representation: str + capabilities: dict + probe: Callable + find: Callable + open_original: Callable + export_original: Callable + # A module either declares fields (generic validation) or parses the payload itself. + parse_config: Callable[[dict], tuple[str, dict, str]] | None = None + config_fields: tuple[ConfigField, ...] = () + normalize_config: Callable[[dict, str], dict] | None = None + navigate: Callable | None = None + preview: Callable | None = None + facets: Callable | None = None + + +_MODULES: dict[str, SourceModule] = {} + + +def register_source(module: SourceModule) -> None: + if module.parse_config is None and not module.config_fields: + raise ValueError(f"Quelle {module.kind} braucht Konfigurationsfelder oder parse_config.") + if sum(1 for item in module.config_fields if item.secret) > 1: + raise ValueError(f"Quelle {module.kind} darf höchstens ein Geheimnisfeld haben.") + for item in module.config_fields: + if item.type not in FIELD_TYPES: + raise ValueError(f"Feldtyp {item.type} ist unbekannt.") + _MODULES[module.kind] = module + + +def unregister_source(kind: str) -> None: + _MODULES.pop(kind, None) + + +def registered_sources() -> list[SourceModule]: + return list(_MODULES.values()) + + +def get_source(kind: str) -> SourceModule | None: + return _MODULES.get(kind) + + +def require_source(kind: str) -> SourceModule: + module = _MODULES.get(kind) + if module is None: + raise unprocessable("Diese Quelle ist nicht registriert.") + return module + + +def accepts_reference(source: str) -> bool: + return source == "url" or source in _MODULES + + +def _blank(value) -> bool: + return value is None or (isinstance(value, str) and not value.strip()) + + +def _coerce(field: ConfigField, value): + if field.type == "number": + try: + number = int(str(value).strip()) + except (TypeError, ValueError): + raise unprocessable(f"{field.label} muss eine ganze Zahl sein.") from None + if field.minimum is not None and number < field.minimum: + raise unprocessable(f"{field.label} ist zu klein.") + if field.maximum is not None and number > field.maximum: + raise unprocessable(f"{field.label} ist zu groß.") + return number + text = str(value) if field.secret else str(value).strip() + if len(text) > field.max_length: + raise unprocessable(f"{field.label} ist zu lang.") + if field.type == "select" and text not in {option for option, _label in field.options}: + raise unprocessable(f"{field.label} ist keine zulässige Auswahl.") + return text + + +def parse_fields( + fields: tuple[ConfigField, ...], + values: dict, + current_settings: dict | None = None, + current_secret: str = "", +) -> tuple[dict, str]: + """Validate declared fields. Unknown keys are refused, a blank secret keeps the stored one.""" + known = {item.key for item in fields} + extra = sorted(key for key, value in values.items() if key not in known and not _blank(value)) + if extra: + raise unprocessable("Unbekanntes Konfigurationsfeld: " + ", ".join(extra[:3]) + ".") + current_settings = current_settings or {} + settings: dict = {} + secret = current_secret + for item in fields: + raw = values.get(item.key) + if item.secret: + if _blank(raw): + if not current_secret and item.required: + raise unprocessable(f"{item.label} fehlt.") + continue + secret = _coerce(item, raw) + continue + if _blank(raw): + raw = current_settings.get(item.key) + if _blank(raw): + raw = item.default + if _blank(raw): + if item.required: + raise unprocessable(f"{item.label} fehlt.") + continue + settings[item.key] = _coerce(item, raw) + return settings, secret diff --git a/backend/app/sources.py b/backend/app/sources.py index 3bd7d2a..b302840 100644 --- a/backend/app/sources.py +++ b/backend/app/sources.py @@ -7,6 +7,7 @@ import imaplib import json import re import ssl +from dataclasses import dataclass from email.header import decode_header from email.parser import BytesParser from email.utils import parsedate_to_datetime @@ -18,45 +19,99 @@ from uuid import UUID from psycopg.types.json import Json -from app.access.policy import Ctx, audit, mediated_source_owner +from app.access.policy import Ctx from app.core.errors import Problem, not_found, unprocessable from app.core.settings import get_settings -from app.mailpack import MailIdentityError, choose_message, identities_match, header_message_id +from app.mailpack import ( + MailIdentityError, + choose_message, + decode_transfer, + header_message_id, + header_text, + identities_match, + parse_mail, + sanitize_html, + simplified_text, +) from app.media.storage import sniff +from app.sourcebind import remember_imap_location, resolve_for_read +from app.sourceaccess import ( + ORIGINAL_UNAVAILABLE, + assert_still_owned, + audit_mediated as _audit_mediated, + borrow_original, + complete_borrow, + connector_owner as _connector_owner, + release_for_external, +) +from app.sourceport import CAPABILITIES +from app.sourceregistry import ConfigField, SourceModule, get_source, parse_fields, register_source, require_source + + +_PUBLIC_COLUMNS = """ + id, kind, name, settings, created_at, + secret <> '' AS has_secret, + last_probe_at, last_probe_ok, last_probe_detail +""" + + +def parse_connector( + kind: str, + name: str, + flat: dict, + config: dict | None, + current: dict | None = None, +) -> tuple[str, dict, str]: + """Validate a connector body with the registered module. ``current`` is the stored row on change.""" + module = require_source(kind) + name = " ".join(str(name or "").split()) + values = {key: value for key, value in flat.items() if value is not None} + values.update(config or {}) + if module.config_fields: + settings, secret = parse_fields( + module.config_fields, + values, + current["settings"] if current else None, + current["secret"] if current else "", + ) + if module.normalize_config is not None: + settings = module.normalize_config(settings, secret) + return name, settings, secret + return module.parse_config({**values, "kind": kind, "name": name, "config": config or {}}) def save_connector(conn, owner_id: UUID, kind: str, name: str, settings: dict, secret: str) -> dict: - if kind == "paperless": - existing = conn.execute( - "SELECT id FROM connectors WHERE owner_id = %s AND kind = 'paperless'", - (owner_id,), - ).fetchone() - if existing: - row = conn.execute( - """ - UPDATE connectors - SET name = %s, settings = %s, secret = %s - WHERE id = %s - RETURNING id, kind, name, settings, created_at - """, - (name, Json(settings), secret, existing["id"]), - ).fetchone() - return _public(row) row = conn.execute( - """ + f""" INSERT INTO connectors (owner_id, kind, name, settings, secret) VALUES (%s, %s, %s, %s, %s) - RETURNING id, kind, name, settings, created_at + RETURNING {_PUBLIC_COLUMNS} """, (owner_id, kind, name, Json(settings), secret), ).fetchone() return _public(row) +def update_connector(conn, owner_id: UUID, connector_id: UUID, name: str, settings: dict, secret: str) -> dict: + """Credential or name change on the same instance. The id does not move.""" + current = _owned(conn, owner_id, connector_id) + row = conn.execute( + f""" + UPDATE connectors + SET name = %s, settings = %s, secret = %s, + last_probe_at = NULL, last_probe_ok = NULL, last_probe_detail = NULL + WHERE id = %s AND owner_id = %s + RETURNING {_PUBLIC_COLUMNS} + """, + (name, Json(settings), secret, current["id"], owner_id), + ).fetchone() + return _public(row) + + def list_connectors(conn, owner_id: UUID) -> list[dict]: rows = conn.execute( - """ - SELECT id, kind, name, settings, created_at + f""" + SELECT {_PUBLIC_COLUMNS} FROM connectors WHERE owner_id = %s ORDER BY kind, name """, @@ -65,6 +120,30 @@ def list_connectors(conn, owner_id: UUID) -> list[dict]: return [_public(row) for row in rows] +def connector_detail(conn, owner_id: UUID, connector_id: UUID) -> dict: + """One own instance with its use. The secret itself is never part of it.""" + row = conn.execute( + f"SELECT {_PUBLIC_COLUMNS} FROM connectors WHERE id = %s AND owner_id = %s", + (connector_id, owner_id), + ).fetchone() + if row is None: + raise not_found() + used = conn.execute( + """ + SELECT count(*) AS total + FROM external_references er + JOIN elements e ON e.id = er.element_id + WHERE (er.connector_id = %s OR er.detail ->> 'connector_id' = %s) + AND e.deleted_at IS NULL AND e.owner_subject_id = %s + """, + (connector_id, str(connector_id), owner_id), + ).fetchone() + body = _public(row) + body["references"] = int(used["total"]) + body["protection_profile"] = "legacy-plaintext" + return body + + def delete_connector(conn, owner_id: UUID, connector_id: UUID) -> None: row = conn.execute( "DELETE FROM connectors WHERE id = %s AND owner_id = %s RETURNING id", @@ -76,26 +155,131 @@ def delete_connector(conn, owner_id: UUID, connector_id: UUID) -> None: def probe(conn, owner_id: UUID, connector_id: UUID) -> dict: row = _owned(conn, owner_id, connector_id) - if row["kind"] == "paperless": - count = _paperless_count(row["settings"]["base_url"], row["secret"]) - return {"ok": True, "detail": f"Paperless antwortet. {count} Dokumente sichtbar."} - _imap_login(row["settings"], row["secret"]) - return {"ok": True, "detail": "Das Postfach ist erreichbar."} + module = get_source(row["kind"]) + if module is None: + raise unprocessable("Diese Quelle ist nicht registriert.") + release_for_external(conn) + try: + result = module.probe(row) + except Problem as exc: + assert_still_owned(conn, row["id"], owner_id) + _record_probe(conn, row["id"], False, exc.detail) + conn.commit() + raise + assert_still_owned(conn, row["id"], owner_id) + _record_probe(conn, row["id"], bool(result.get("ok", True)), str(result.get("detail") or "")) + return result -def paperless_documents(conn, owner_id: UUID, query: str) -> list[dict]: - row = conn.execute( +def _record_probe(conn, connector_id: UUID, ok: bool, detail: str) -> None: + """Display only. The detail comes from the module and must not carry a secret.""" + conn.execute( """ - SELECT settings, secret FROM connectors + UPDATE connectors + SET last_probe_at = now(), last_probe_ok = %s, last_probe_detail = %s + WHERE id = %s + """, + (ok, detail[:300] or None, connector_id), + ) + + +def _paperless_choice(value, label: str) -> str | None: + text = str(value or "").strip() + if not text: + return None + if not text.isdigit() or len(text) > 12: + raise unprocessable(f"{label} ist keine Auswahl dieser Instanz.") + return text + + +def _paperless_day(value, label: str) -> str | None: + text = str(value or "").strip() + if not text: + return None + if not re.fullmatch(r"\d{4}-\d{2}-\d{2}", text): + raise unprocessable(f"{label} ist kein Datum.") + return text + + +def _paperless_params(query: str, options: dict | None) -> dict: + """Paperless-ngx document date is the created field, not the upload or modified time.""" + options = options or {} + params = {"page_size": "20"} + text = " ".join(str(query or "").split()) + if text: + params["query"] = text + title = " ".join(str(options.get("title") or "").split()) + if title: + params["title__icontains"] = title + correspondent = _paperless_choice(options.get("correspondent"), "Der Korrespondent") + if correspondent: + params["correspondent__id"] = correspondent + document_type = _paperless_choice(options.get("document_type"), "Der Dokumenttyp") + if document_type: + params["document_type__id"] = document_type + raw_tags = options.get("tag") or options.get("tags") or "" + if isinstance(raw_tags, (list, tuple)): + parts = [str(item) for item in raw_tags] + else: + parts = str(raw_tags).split(",") + tag_ids = [chosen for chosen in (_paperless_choice(part, "Ein Tag") for part in parts) if chosen] + if tag_ids: + params["tags__id__all"] = ",".join(tag_ids) + start = _paperless_day(options.get("document_from"), "Das Dokumentdatum von") + end = _paperless_day(options.get("document_to"), "Das Dokumentdatum bis") + if start and end and start > end: + raise unprocessable("Das Dokumentdatum von liegt nach dem Dokumentdatum bis.") + if start: + params["created__date__gte"] = start + if end: + params["created__date__lte"] = end + sort = str(options.get("sort") or "").strip() + if sort: + if sort not in {"created", "-created", "title", "-title", "score", "-score"}: + raise unprocessable("Diese Sortierung gibt es nicht.") + if sort in {"score", "-score"}: + if not text: + raise unprocessable("Die Relevanz gilt zusammen mit einem Suchtext.") + # Paperless-ngx 2.20.6 has no ordering field named score. A query without + # ordering is the relevance order of the full-text index. + else: + params["ordering"] = sort + cursor = str(options.get("cursor") or "").strip() + if cursor: + if not cursor.isdigit() or not 1 <= int(cursor) <= 10000: + raise unprocessable("Die Seite ist ungültig.") + params["page"] = cursor + return params + + +def _paperless_row(conn, owner_id: UUID, connector_id: UUID | None): + if connector_id is not None: + row = _owned(conn, owner_id, connector_id) + if row["kind"] != "paperless": + raise unprocessable("Das ist kein Paperless-Zugang.") + return row + rows = conn.execute( + """ + SELECT id, settings, secret FROM connectors WHERE owner_id = %s AND kind = 'paperless' - ORDER BY created_at DESC LIMIT 1 + ORDER BY name, id """, (owner_id,), - ).fetchone() - if row is None: + ).fetchall() + if not rows: raise unprocessable("Paperless ist noch nicht verbunden. Lege die Quelle unter Quellen an.") + if len(rows) != 1: + raise unprocessable("Für Paperless ist mehr als ein Zugang vorhanden. Die Suche muss einen Zugang nennen.") + return rows[0] + + +def paperless_documents(conn, owner_id: UUID, query: str, connector_id: UUID | None = None, options: dict | None = None) -> dict: + row = _paperless_row(conn, owner_id, connector_id) base = row["settings"]["base_url"].rstrip("/") - payload = _paperless_get(base, row["secret"], "/api/documents/", {"query": query, "page_size": "20"}) + params = _paperless_params(query, options) + release_for_external(conn) + payload = _paperless_get(base, row["secret"], "/api/documents/", params) + assert_still_owned(conn, row["id"], owner_id) items = [] for doc in payload.get("results") or []: created = str(doc.get("created") or "")[:10] @@ -109,18 +293,56 @@ def paperless_documents(conn, owner_id: UUID, query: str) -> list[dict]: "open_url": f"{base}/documents/{doc.get('id')}/", } ) - return items + page = int(params.get("page") or 1) + count = payload.get("count") + if isinstance(count, int): + next_page = str(page + 1) if page * 20 < count else None + else: + next_page = str(page + 1) if len(items) == 20 else None + return {"items": items, "connector_id": str(row["id"]), "next_page": next_page, "incomplete": False} + + +def paperless_facets(conn, owner_id: UUID, connector_id: UUID) -> dict: + row = _paperless_row(conn, owner_id, connector_id) + base = row["settings"]["base_url"].rstrip("/") + token = row["secret"] + release_for_external(conn) + body = { + "correspondents": _paperless_names(base, token, "/api/correspondents/"), + "document_types": _paperless_names(base, token, "/api/document_types/"), + "tags": _paperless_names(base, token, "/api/tags/"), + "tag_mode": "all", + } + assert_still_owned(conn, row["id"], owner_id) + return body + + +def _paperless_names(base: str, token: str, path: str) -> list[dict]: + found = [] + for page in range(1, 21): + payload = _paperless_get(base, token, path, {"page": str(page), "page_size": "100"}) + rows = payload.get("results") or [] + for item in rows: + if item.get("id") is None: + continue + found.append({"id": str(item["id"]), "name": str(item.get("name") or item["id"])[:200]}) + if len(rows) < 100: + break + return found def imap_mailboxes(conn, owner_id: UUID, connector_id: UUID) -> list[dict]: row = _owned(conn, owner_id, connector_id) if row["kind"] != "imap": raise unprocessable("Das ist kein E-Mail-Konto.") + release_for_external(conn) client = _imap_login(row["settings"], row["secret"]) try: - return _imap_mailboxes(client) + items = _imap_mailboxes(client) finally: _imap_logout(client) + assert_still_owned(conn, row["id"], owner_id) + return items def imap_messages( @@ -130,6 +352,7 @@ def imap_messages( query: str, mailbox: str, before: int | None, + uidvalidity: int | None = None, ) -> dict: row = _owned(conn, owner_id, connector_id) if row["kind"] != "imap": @@ -138,11 +361,14 @@ def imap_messages( raise unprocessable("Der Fachname ist ungültig.") if len(query) > 200: raise unprocessable("Die Suche ist zu lang.") + release_for_external(conn) client = _imap_login(row["settings"], row["secret"]) try: - return _imap_page(client, mailbox, query, before) + page = _imap_page(client, mailbox, query, before, uidvalidity) finally: _imap_logout(client) + assert_still_owned(conn, row["id"], owner_id) + return page def imap_preview(conn, owner_id: UUID, connector_id: UUID, mailbox: str, uid: int) -> dict: @@ -152,12 +378,18 @@ def imap_preview(conn, owner_id: UUID, connector_id: UUID, mailbox: str, uid: in _check_mailbox(mailbox) if uid < 1: raise unprocessable("Die Nachricht ist ungültig.") - client = _imap_login(row["settings"], row["secret"]) + settings = dict(row["settings"]) + secret = row["secret"] + connector_id = row["id"] + release_for_external(conn) + client = _imap_login(settings, secret) try: _select_mailbox(client, mailbox) - return _fetch_message(client, uid, mailbox) + preview = _fetch_message(client, uid, mailbox) finally: _imap_logout(client) + assert_still_owned(conn, connector_id, owner_id) + return preview def _owned(conn, owner_id: UUID, connector_id: UUID) -> dict: @@ -170,6 +402,11 @@ def _owned(conn, owner_id: UUID, connector_id: UUID) -> dict: return row +def owned_connector(conn, owner_id: UUID, connector_id: UUID) -> dict: + """The stored row of an own instance, secret included. For server-side use only.""" + return _owned(conn, owner_id, connector_id) + + def _public(row) -> dict: return { "id": str(row["id"]), @@ -177,6 +414,15 @@ def _public(row) -> dict: "name": row["name"], "settings": row["settings"], "created_at": row["created_at"].strftime("%Y-%m-%dT%H:%M:%SZ"), + "capabilities": (get_source(row["kind"]).capabilities if get_source(row["kind"]) else None), + "has_secret": bool(row["has_secret"]), + "probe": { + "at": row["last_probe_at"].strftime("%Y-%m-%dT%H:%M:%SZ"), + "ok": bool(row["last_probe_ok"]), + "detail": row["last_probe_detail"] or "", + } + if row["last_probe_at"] + else None, } @@ -195,20 +441,14 @@ def _paperless_count(base: str, token: str) -> int: return int(payload.get("count") or 0) -def paperless_content(conn, owner_id: UUID, document_id: str) -> tuple[bytes, str, str]: +def paperless_content(conn, owner_id: UUID, document_id: str, connector_id: UUID | None = None) -> tuple[bytes, str, str]: if not document_id.isdigit() or len(document_id) > 12: raise unprocessable("Die Paperless-Nummer ist ungültig.") - row = conn.execute( - """ - SELECT settings, secret FROM connectors - WHERE owner_id = %s AND kind = 'paperless' - ORDER BY created_at DESC LIMIT 1 - """, - (owner_id,), - ).fetchone() - if row is None: - raise unprocessable("Paperless ist für dieses Konto nicht verbunden.") - return _paperless_bytes(row, document_id) + row = _paperless_row(conn, owner_id, connector_id) + release_for_external(conn) + fetched = _paperless_bytes(row, document_id) + assert_still_owned(conn, row["id"], owner_id) + return fetched def _paperless_bytes(row, document_id: str) -> tuple[bytes, str, str]: @@ -233,30 +473,18 @@ def _paperless_bytes(row, document_id: str) -> tuple[bytes, str, str]: raise unprocessable("Paperless stellt diese Datei nicht zum Ansehen bereit.") -ORIGINAL_UNAVAILABLE = "Das Original ist über die Freigabe nicht abrufbar." - - def read_linked_paperless(conn, actor_id: UUID, element: dict, context: Ctx | None, ref) -> tuple[bytes, str, str]: """The stored document. A grantee never uses a connector of their own account.""" - detail = (ref or {}).get("detail") or {} external_id = str((ref or {}).get("external_id") or "") - stored = _uuid(detail.get("connector_id")) - stored_owner = _connector_owner(conn, stored) - if element["owner_subject_id"] == actor_id or stored_owner == actor_id: - return paperless_content(conn, actor_id, external_id) - owner_id = mediated_source_owner(conn, actor_id, element, context) - if owner_id is None or not external_id.isdigit() or len(external_id) > 12: + detail = (ref or {}).get("detail") or {} + stored = _uuid(detail.get("connector_id")) or _uuid((ref or {}).get("connector_id")) + owner_path = element["owner_subject_id"] == actor_id or _connector_owner(conn, stored) == actor_id + if not external_id.isdigit() or len(external_id) > 12: + if owner_path: + raise unprocessable("Die Paperless-Nummer ist ungültig.") raise unprocessable(ORIGINAL_UNAVAILABLE) - row = _mediated_paperless(conn, owner_id, stored, element) - if row is None: - _audit_mediated(conn, actor_id, element, context, "paperless", stored, "unavailable") - raise unprocessable(ORIGINAL_UNAVAILABLE) - try: - data, media, name = _paperless_bytes(row, external_id) - except Problem: - _audit_mediated(conn, actor_id, element, context, "paperless", row["id"], "unavailable") - raise unprocessable(ORIGINAL_UNAVAILABLE) from None - _audit_mediated(conn, actor_id, element, context, "paperless", row["id"], "ok") + borrowed = borrow_original(conn, actor_id, element, context, ref, "paperless") + data, media, name = complete_borrow(conn, borrowed, lambda row: _paperless_bytes(row, external_id)) if external_id and external_id in (name or ""): name = "dokument.pdf" return data, media, name or "dokument.pdf" @@ -267,29 +495,23 @@ def read_linked_mail(conn, actor_id: UUID, element: dict, context: Ctx | None, r detail = (ref or {}).get("detail") or {} if ref is None or ref["source"] != "imap": raise unprocessable("Das ist keine E-Mail.") - connector_id = _uuid(detail.get("connector_id")) + connector_id = _uuid(detail.get("connector_id")) or _uuid(ref["connector_id"] if "connector_id" in ref.keys() else None) mailbox = str(detail.get("mailbox") or "").strip() uid = _mail_uid(detail.get("uid")) external_id = str(ref["external_id"]) - stored_owner = _connector_owner(conn, connector_id) - if element["owner_subject_id"] == actor_id or stored_owner == actor_id: - if connector_id is None or not mailbox or uid is None: + owner_path = element["owner_subject_id"] == actor_id or _connector_owner(conn, connector_id) == actor_id + if owner_path: + if not mailbox or uid is None: raise unprocessable("Für diese E-Mail fehlt die Zuordnung zu Fach und Nachricht.") - return load_imap_original(conn, actor_id, connector_id, mailbox, uid, external_id) - owner_id = mediated_source_owner(conn, actor_id, element, context) - if owner_id is None or connector_id is None or not mailbox or uid is None: + elif connector_id is None or not mailbox or uid is None: raise unprocessable(ORIGINAL_UNAVAILABLE) - row = _connector_for(conn, connector_id, owner_id, "imap") - if row is None: - _audit_mediated(conn, actor_id, element, context, "imap", connector_id, "unavailable") - raise unprocessable(ORIGINAL_UNAVAILABLE) - try: - raw = load_imap_original(conn, owner_id, connector_id, mailbox, uid, external_id) - except Problem: - _audit_mediated(conn, actor_id, element, context, "imap", connector_id, "unavailable") - raise unprocessable(ORIGINAL_UNAVAILABLE) from None - _audit_mediated(conn, actor_id, element, context, "imap", connector_id, "ok") - return raw + borrowed = borrow_original(conn, actor_id, element, context, ref, "imap") + + def fetch(row): + return pull_rfc822(row["settings"], row["secret"], mailbox, uid, external_id, COMPLETE_MAIL_LIMIT) + + found = complete_borrow(conn, borrowed, fetch) + return _keep_moved_mail(conn, element["id"], external_id, mailbox, uid, found) def _uuid(value) -> UUID | None: @@ -311,13 +533,6 @@ def _mail_uid(value) -> int | None: return None -def _connector_owner(conn, connector_id: UUID | None) -> UUID | None: - if connector_id is None: - return None - row = conn.execute("SELECT owner_id FROM connectors WHERE id = %s", (connector_id,)).fetchone() - return row["owner_id"] if row else None - - def _connector_for(conn, connector_id: UUID, owner_id: UUID, kind: str): return conn.execute( """ @@ -329,43 +544,6 @@ def _connector_for(conn, connector_id: UUID, owner_id: UUID, kind: str): ).fetchone() -def _mediated_paperless(conn, owner_id: UUID, stored: UUID | None, element: dict): - if stored is not None: - return _connector_for(conn, stored, owner_id, "paperless") - if element["owner_subject_id"] != owner_id: - return None - return conn.execute( - """ - SELECT id, owner_id, kind, settings, secret - FROM connectors - WHERE owner_id = %s AND kind = 'paperless' - ORDER BY created_at DESC - LIMIT 1 - """, - (owner_id,), - ).fetchone() - - -def _audit_mediated(conn, actor_id: UUID, element: dict, context: Ctx | None, source: str, connector_id, outcome: str) -> None: - audit( - conn, - actor_id, - element["home_scope_id"], - "source_read_mediated", - "element", - element["id"], - context, - result="ok" if outcome == "ok" else "unavailable", - detail={ - "source": source, - "connector_id": str(connector_id) if connector_id else None, - "outcome": outcome, - }, - ) - if outcome != "ok": - conn.commit() - - def _paperless_file( base: str, token: str, @@ -527,6 +705,8 @@ def _imap_login(settings: dict, secret: str) -> imaplib.IMAP4: PAGE_SIZE = 30 +UID_WINDOW = 400 +UID_ROUNDS = 8 _LIST_LINE = re.compile( r'^\((?P[^)]*)\)\s+(?P"(?:\\.|[^"])*"|NIL)\s+(?P"(?:\\.|[^"])*"|\S+)\s*$' ) @@ -558,46 +738,155 @@ def _imap_mailboxes(client: imaplib.IMAP4) -> list[dict]: return items -def _imap_page(client: imaplib.IMAP4, mailbox: str, query: str, before: int | None) -> dict: +def _untagged_int(client: imaplib.IMAP4, name: str) -> int | None: + values = (getattr(client, "untagged_responses", None) or {}).get(name) or [] + if not values: + return None + raw = values[-1] + if isinstance(raw, bytes): + raw = raw.decode("ascii", "replace") + try: + return int(str(raw).strip()) + except ValueError: + return None + + +def _uids_from(data, low: int | None, high: int | None) -> list[int]: + if data is None or isinstance(data, (bytes, str)): + raise unprocessable("Das Postfach hat eine unbrauchbare Antwort geliefert.") + chunks = [] + for part in data: + if part is None: + continue + if isinstance(part, bytes): + chunks.append(part) + elif isinstance(part, str): + chunks.append(part.encode("utf-8")) + else: + raise unprocessable("Das Postfach hat eine unbrauchbare Antwort geliefert.") + found = [] + for piece in b" ".join(chunks).split(): + if not piece.isdigit(): + continue + uid = int(piece) + if low is not None and uid < low: + continue + if high is not None and uid > high: + continue + found.append(uid) + return found + + +def _uid_call(client: imaplib.IMAP4, *args): + try: + return client.uid(*args) + except imaplib.IMAP4.abort as exc: + raise unprocessable("Die Verbindung zum Postfach ist abgebrochen.") from exc + except TimeoutError as exc: + raise unprocessable("Das Postfach hat nicht rechtzeitig geantwortet.") from exc + except imaplib.IMAP4.error as exc: + raise unprocessable("Das Postfach kann diese Suche nicht ausführen. Blättere im Fach oder kürze den Suchtext.") from exc + + +def _search_status(status) -> None: + if status in {"NO", "BAD"}: + raise unprocessable("Das Postfach hat die Suche abgelehnt.") + if status != "OK": + raise unprocessable("Das Postfach hat eine unbrauchbare Antwort geliefert.") + + +def _uid_window(client: imaplib.IMAP4, query: str, low: int, high: int) -> list[int]: + needle = " ".join(query.split()) + uid_range = f"{low}:{high}" + if not needle: + status, data = _uid_call(client, "SEARCH", None, "UID", uid_range) + else: + quoted = '"' + needle.replace("\\", "\\\\").replace('"', '\\"') + '"' + if needle.isascii(): + status, data = _uid_call( + client, "SEARCH", None, "UID", uid_range, + "OR", "OR", "SUBJECT", quoted, "FROM", quoted, "TEXT", quoted, + ) + else: + encoded = quoted.encode("utf-8") + status, data = _uid_call( + client, "SEARCH", "CHARSET", "UTF-8", "UID", uid_range, + "OR", "OR", "SUBJECT", encoded, "FROM", encoded, "TEXT", encoded, + ) + _search_status(status) + return _uids_from(data, low, high) + + +def _highest_uid(client: imaplib.IMAP4) -> int: + uidnext = _untagged_int(client, "UIDNEXT") + if uidnext is not None: + return max(0, uidnext - 1) + status, data = _uid_call(client, "SEARCH", None, "UID", "*") + _search_status(status) + found = _uids_from(data, None, None) + return max(found) if found else 0 + + +def _imap_page(client: imaplib.IMAP4, mailbox: str, query: str, before: int | None, claimed_validity: int | None = None) -> dict: folders = _imap_mailboxes(client) chosen = mailbox or "INBOX" match = next((item for item in folders if item["name"] == chosen), None) if match is None or not match["selectable"]: raise unprocessable("Dieses Fach gibt es in dem Konto nicht.") - try: - status, _ = client.select(chosen, readonly=True) - except imaplib.IMAP4.error as exc: - raise unprocessable("Dieses Fach lässt sich nicht öffnen.") from exc + status, _data = _examine(client, chosen) if status != "OK": raise unprocessable("Dieses Fach lässt sich nicht öffnen.") - status, data = _uid_search(client, query) - if status != "OK" or not data or not data[0]: - return {"mailbox": chosen, "items": [], "next_before": None} - uids = sorted(int(part) for part in data[0].split() if part.isdigit()) - if before is not None: - uids = [uid for uid in uids if uid < before] - page = uids[-PAGE_SIZE:] - next_before = page[0] if len(uids) > PAGE_SIZE else None - items = [] - for uid in reversed(page): - item = _fetch_header(client, uid, chosen) - if item is not None: - items.append(item) - return {"mailbox": chosen, "items": items, "next_before": next_before} - - -def _uid_search(client: imaplib.IMAP4, query: str): - needle = " ".join(query.split()) - if not needle: - return client.uid("SEARCH", None, "ALL") - quoted = '"' + needle.replace("\\", "\\\\").replace('"', '\\"') + '"' - try: - if needle.isascii(): - return client.uid("SEARCH", None, "OR", "SUBJECT", quoted, "FROM", quoted) - encoded = quoted.encode("utf-8") - return client.uid("SEARCH", "CHARSET", "UTF-8", "OR", "SUBJECT", encoded, "FROM", encoded) - except imaplib.IMAP4.error as exc: - raise unprocessable("Das Postfach kann diese Suche nicht ausführen. Blättere im Fach oder kürze den Suchtext.") from exc + exists = _untagged_int(client, "EXISTS") + validity = _untagged_int(client, "UIDVALIDITY") + restarted = False + if claimed_validity is not None and validity is not None and int(claimed_validity) != validity: + before = None + restarted = True + high = before - 1 if before is not None else _highest_uid(client) + collected = [] + attempts = 0 + failures = 0 + low = 1 + rounds = 0 + while high >= 1 and len(collected) < PAGE_SIZE and rounds < UID_ROUNDS: + low = max(1, high - UID_WINDOW + 1) + for uid in sorted(_uid_window(client, query, low, high), reverse=True): + if len(collected) >= PAGE_SIZE: + break + attempts += 1 + item = _fetch_header(client, uid, chosen) + if item is None: + failures += 1 + continue + collected.append(item) + rounds += 1 + if len(collected) >= PAGE_SIZE or low == 1: + break + high = low - 1 + if attempts and failures == attempts: + raise unprocessable("Die Nachrichtenköpfe dieses Fachs konnten nicht gelesen werden.") + incomplete = bool(collected) is False and low > 1 and high >= 1 and rounds >= UID_ROUNDS + if len(collected) < PAGE_SIZE and low > 1 and rounds >= UID_ROUNDS: + incomplete = True + oldest = collected[-1]["uid"] if collected else None + if len(collected) >= PAGE_SIZE and oldest and oldest > 1: + next_before = oldest + elif incomplete: + next_before = low + else: + next_before = None + query_text = " ".join(query.split()) + contradictory = bool(exists and exists > 0 and not collected and not query_text and not incomplete) + return { + "mailbox": chosen, + "items": collected, + "next_before": next_before, + "incomplete": incomplete, + "restarted": restarted, + "uidvalidity": validity, + "exists": exists, + "contradictory": contradictory, + } def _check_mailbox(mailbox: str) -> None: @@ -605,15 +894,34 @@ def _check_mailbox(mailbox: str) -> None: raise unprocessable("Der Fachname ist ungültig.") +def _mailbox_argument(mailbox: str) -> str: + """IMAP atoms stay bare. Everything else is a quoted protocol string.""" + _check_mailbox(mailbox) + protocol = _encode_utf7(mailbox) if any(ord(char) > 127 for char in mailbox) else mailbox + if re.fullmatch(r"[A-Za-z0-9!#$%&'*+\-/=?^_`{|}~.]+", protocol): + return protocol + escaped = protocol.replace("\\", "\\\\").replace('"', '\\"') + return f'"{escaped}"' + + +def _examine(client: imaplib.IMAP4, mailbox: str): + argument = _mailbox_argument(mailbox) + try: + return client.select(argument, readonly=True) + except imaplib.IMAP4.abort as exc: + raise unprocessable("Die Verbindung zum Postfach ist abgebrochen.") from exc + except TimeoutError as exc: + raise unprocessable("Das Postfach hat nicht rechtzeitig geantwortet.") from exc + except imaplib.IMAP4.error as exc: + raise unprocessable("Dieses Fach lässt sich nicht öffnen.") from exc + + def _select_mailbox(client: imaplib.IMAP4, mailbox: str) -> None: folders = _imap_mailboxes(client) match = next((item for item in folders if item["name"] == mailbox), None) if match is None or not match["selectable"]: raise unprocessable("Dieses Fach gibt es in dem Konto nicht.") - try: - status, _ = client.select(mailbox, readonly=True) - except imaplib.IMAP4.error as exc: - raise unprocessable("Dieses Fach lässt sich nicht öffnen.") from exc + status, _data = _examine(client, mailbox) if status != "OK": raise unprocessable("Dieses Fach lässt sich nicht öffnen.") @@ -653,6 +961,10 @@ def _fetch_header(client: imaplib.IMAP4, uid: int, mailbox: str) -> dict | None: } +PREVIEW_BYTE_LIMIT = 8_000_000 +PREVIEW_TEXT_LIMIT = 200_000 + + def _fetch_message(client: imaplib.IMAP4, uid: int, mailbox: str) -> dict: try: status, fetched = client.uid("FETCH", str(uid), "(BODY.PEEK[])") @@ -661,22 +973,36 @@ def _fetch_message(client: imaplib.IMAP4, uid: int, mailbox: str) -> dict: raw = _literal(fetched) if status != "OK" or raw is None: raise unprocessable("Die Nachricht lässt sich nicht lesen.") - if len(raw) > 8_000_000: + if len(raw) > PREVIEW_BYTE_LIMIT: return _oversized_message(client, uid, mailbox) - message = BytesParser().parsebytes(raw) - subject = _decode(message.get("Subject")) or "(ohne Betreff)" + parsed = parse_mail(raw) + return _preview_from_mail(uid, mailbox, parsed, limited=parsed.text_truncated) + + +def _preview_from_mail(uid: int, mailbox: str, parsed, limited: bool) -> dict: + text = parsed.text[:PREVIEW_TEXT_LIMIT] return { "uid": uid, "mailbox": mailbox, - "title": subject[:200], - "message_from": _decode(message.get("From"))[:200], - "message_to": _decode(message.get("To"))[:500], - "message_cc": _decode(message.get("Cc"))[:500], + "title": (parsed.subject or "(ohne Betreff)")[:200], + "message_from": parsed.sender[:200], + "message_to": parsed.recipient[:500], + "message_cc": parsed.copy[:500], "direction": _direction(mailbox), - "document_on": _mail_date(message.get("Date")), - "attachments": _message_attachments(message), - "text": (_text := _message_text(message))[:200_000], - "text_truncated": len(_text) > 200_000, + "document_on": parsed.document_on, + "attachments": [ + { + "filename": item.display_name or item.filename, + "archive_name": item.filename, + "content_type": item.media_type, + } + for item in parsed.attachments[:40] + ], + "text": text, + "html": parsed.html, + "text_truncated": limited or len(parsed.text) > PREVIEW_TEXT_LIMIT, + "preview_limited": limited, + "notices": parsed.notices, } @@ -703,10 +1029,37 @@ def _direction(mailbox: str) -> str | None: def _oversized_message(client: imaplib.IMAP4, uid: int, mailbox: str) -> dict: + """Read named text parts. A cut-off multipart blob is not a message.""" header = _fetch_header(client, uid, mailbox) or {} - text = _partial_text(client, uid) - if not header and not text: + structure = _body_structure(client, uid) + plain: list[str] = [] + html_parts: list[str] = [] + truncated = False + notices = [ + "Die Vorschau ist begrenzt, weil die Nachricht sehr groß ist. " + "Das Original bleibt die vollständige E-Mail." + ] + if structure is None: + notices.append("Die MIME-Struktur ließ sich nicht lesen. Rohfragmente werden nicht als Text gezeigt.") + else: + for part in _text_sections(structure): + fetched, shortened = _fetch_section(client, uid, part["section"], PREVIEW_TEXT_LIMIT) + if fetched is None: + notices.append("Ein Textteil der großen Nachricht ließ sich nicht lesen.") + continue + truncated = truncated or shortened or part["octets"] > len(fetched) + text = decode_transfer(fetched, part["charset"]).strip() + if part["media"] == "text/plain": + plain.append(text) + else: + html_parts.append(text) + simplified = "\n".join(chunk for chunk in plain if chunk).strip() + rendered = sanitize_html("\n".join(html_parts)) if html_parts else "" + if not simplified and html_parts: + simplified = simplified_text("\n".join(html_parts)) + if not header and not simplified and not rendered and structure is None: raise unprocessable("Die Nachricht lässt sich nicht lesen.") + attachments = header.get("attachments") or (_structure_names(structure) if structure is not None else []) return { "uid": uid, "mailbox": mailbox, @@ -716,31 +1069,185 @@ def _oversized_message(client: imaplib.IMAP4, uid: int, mailbox: str) -> dict: "message_cc": header.get("message_cc") or "", "direction": header.get("direction", _direction(mailbox)), "document_on": header.get("document_on"), - "attachments": header.get("attachments") or [], - "text": text, - "text_truncated": True, + "attachments": attachments, + "text": simplified[:PREVIEW_TEXT_LIMIT], + "html": rendered, + "text_truncated": True or truncated, + "preview_limited": True, + "notices": notices, } -def _partial_text(client: imaplib.IMAP4, uid: int) -> str: +def _body_structure(client: imaplib.IMAP4, uid: int): try: - status, fetched = client.uid("FETCH", str(uid), "(BODY.PEEK[TEXT]<0.200000>)") + status, fetched = client.uid("FETCH", str(uid), "(BODYSTRUCTURE)") except imaplib.IMAP4.error: - return "" + return None if status != "OK": - return "" + return None + blob = _literal(fetched) + if blob is None: + chunks: list[bytes] = [] + for part in fetched or []: + if isinstance(part, tuple): + chunks.extend(bytes(item) for item in part if isinstance(item, (bytes, bytearray))) + elif isinstance(part, (bytes, bytearray)): + chunks.append(bytes(part)) + blob = b" ".join(chunks) + text = blob.decode("utf-8", "replace") + start = text.find("(") + if start < 0: + return None + try: + node, _end = _parse_imap_list(text, start) + except ValueError: + return None + return node + + +def _parse_imap_list(text: str, index: int): + if index >= len(text) or text[index] != "(": + raise ValueError(text[index:index + 20]) + index += 1 + items = [] + while index < len(text): + char = text[index] + if char == ")": + return items, index + 1 + if char in " \t\r\n": + index += 1 + continue + if char == "(": + child, index = _parse_imap_list(text, index) + items.append(child) + continue + if char == '"': + token, index = _parse_imap_string(text, index) + items.append(token) + continue + end = index + while end < len(text) and text[end] not in " \t\r\n()": + end += 1 + token = text[index:end] + items.append(None if token.upper() == "NIL" else token) + index = end + raise ValueError("unclosed") + + +def _parse_imap_string(text: str, index: int): + index += 1 + chars: list[str] = [] + while index < len(text): + char = text[index] + if char == "\\": + if index + 1 < len(text): + chars.append(text[index + 1]) + index += 2 + continue + if char == '"': + return "".join(chars), index + 1 + chars.append(char) + index += 1 + raise ValueError("string") + + +def _text_sections(node, prefix: str = "") -> list[dict]: + if not isinstance(node, list) or not node: + return [] + if isinstance(node[0], list): + children = [] + for item in node: + if isinstance(item, list): + children.append(item) + else: + break + found = [] + for number, child in enumerate(children, start=1): + section = f"{number}" if not prefix else f"{prefix}.{number}" + found.extend(_text_sections(child, section)) + return found + media = str(node[0] or "").upper() + subtype = str(node[1] or "").upper() + if media == "MESSAGE": + return [] + if media != "TEXT" or subtype not in {"PLAIN", "HTML"}: + return [] + disposition, params = _disposition(node) + if disposition == "ATTACHMENT": + return [] + body_params = _param_map(node[2]) if len(node) > 2 else {} + octets = _imap_int(node[6]) if len(node) > 6 else 0 + return [{ + "section": prefix or "1", + "media": f"text/{subtype.lower()}", + "charset": body_params.get("charset") or "utf-8", + "octets": octets, + }] + + +def _disposition(node: list) -> tuple[str, dict]: + for item in node: + if isinstance(item, list) and item and isinstance(item[0], str) and item[0].upper() in {"ATTACHMENT", "INLINE"}: + params = _param_map(item[1]) if len(item) > 1 else {} + return item[0].upper(), params + return "", {} + + +def _param_map(value) -> dict: + if not isinstance(value, list): + return {} + pairs = {} + index = 0 + while index + 1 < len(value): + key = value[index] + raw = value[index + 1] + if isinstance(key, str): + pairs[key.casefold()] = "" if raw is None else str(raw) + index += 2 + return pairs + + +def _imap_int(value) -> int: + try: + return int(value) + except (TypeError, ValueError): + return 0 + + +def _fetch_section(client: imaplib.IMAP4, uid: int, section: str, limit: int) -> tuple[bytes | None, bool]: + spec = f"(BODY.PEEK[{section}]<0.{limit}>)" + try: + status, fetched = client.uid("FETCH", str(uid), spec) + except imaplib.IMAP4.error: + return None, False + if status != "OK": + return None, False raw = _literal(fetched) if not raw: - return "" - sample = raw[:200_000] - try: - message = BytesParser().parsebytes(b"Subject: x\r\n\r\n" + sample) - text = _message_text(message) - except Exception: - text = "" - if not text: - text = sample.decode("utf-8", errors="replace") - return text.strip()[:200_000] + return None, False + return raw[:limit], len(raw) >= limit + + +def _structure_names(node) -> list[dict]: + names = [] + + def visit(item): + if not isinstance(item, list) or not item: + return + if isinstance(item[0], list): + for child in item: + if isinstance(child, list): + visit(child) + return + _disposition_name, params = _disposition(item) + body = _param_map(item[2]) if len(item) > 2 else {} + raw = params.get("filename") or body.get("name") or "" + label = header_text(raw) + if label: + names.append({"filename": label[:200]}) + + visit(node) + return names[:40] def _literal(fetched) -> bytes | None: @@ -760,12 +1267,10 @@ def _structure_attachments(fetched) -> list[dict]: blob = b" ".join(chunks) names = re.findall(br'FILENAME"\s+"([^"\r\n]+)"', blob, flags=re.IGNORECASE) items = [] - seen: set[str] = set() for raw in names: - name = raw.decode("utf-8", errors="replace").strip()[:200] - if not name or name in seen: + name = header_text(raw.decode("utf-8", errors="replace")).strip()[:200] + if not name: continue - seen.add(name) items.append({"filename": name}) if len(items) == 40: break @@ -855,6 +1360,32 @@ def _unquote(token: str) -> str: return token +def _encode_utf7(value: str) -> str: + """Modified UTF-7 for a mailbox that is not already an ASCII protocol name.""" + pieces: list[str] = [] + pending: list[str] = [] + + def flush() -> None: + if not pending: + return + raw = "".join(pending).encode("utf-16-be") + token = base64.b64encode(raw).decode("ascii").rstrip("=").replace("/", ",") + pieces.append(f"&{token}-") + pending.clear() + + for char in value: + if char == "&": + flush() + pieces.append("&-") + elif ord(char) < 128: + flush() + pieces.append(char) + else: + pending.append(char) + flush() + return "".join(pieces) + + def _decode_utf7(value: str) -> str: if "&" not in value: return value @@ -941,11 +1472,33 @@ def load_imap_original(conn, owner_id: UUID, connector_id: UUID, mailbox: str, u row = _owned(conn, owner_id, connector_id) if row["kind"] != "imap": raise unprocessable("Das ist kein E-Mail-Konto.") - return pull_rfc822(row["settings"], row["secret"], mailbox, uid, expected_id, COMPLETE_MAIL_LIMIT) + found = pull_rfc822(row["settings"], row["secret"], mailbox, uid, expected_id, COMPLETE_MAIL_LIMIT) + return found.raw if isinstance(found, LocatedMail) else found -def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id: str, max_bytes: int) -> bytes: - """Fetch the referenced message. A different message at the old UID is not returned.""" +@dataclass(frozen=True) +class LocatedMail: + """Bytes plus the folder they were read from. `relocated` is a complete, unique hit.""" + + raw: bytes + mailbox: str + uid: int + relocated: bool = False + + +@dataclass(frozen=True) +class _MailHit: + raw: bytes + mailbox: str + uid: int + + +def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id: str, max_bytes: int) -> LocatedMail: + """Fetch the referenced message. A different message at the old UID is not returned. + + The stored folder is tried first. A single Message-ID hit elsewhere comes back + marked relocated; the caller stores that place after the access check. + """ client = _imap_login(settings, secret) try: _select_mailbox(client, mailbox) @@ -955,23 +1508,45 @@ def pull_rfc822(settings: dict, secret: str, mailbox: str, uid: int, expected_id "Die vollständige Nachricht überschreitet die Exportgrenze. " "Eine gekürzte Vorschau wird nicht als Original übernommen." ) - alternates: list[bytes] = [] - if candidate is None or not identities_match(header_message_id(candidate), expected_id): - alternates = _search_rfc822(client, expected_id, max_bytes) + if candidate is not None and identities_match(header_message_id(candidate), expected_id): + return LocatedMail(candidate, mailbox, uid, relocated=False) + hits, capped = _search_rfc822(client, expected_id, max_bytes) try: - chosen = choose_message(candidate, alternates, expected_id) + chosen = choose_message(None, [hit.raw for hit in hits], expected_id) except MailIdentityError as exc: raise unprocessable(exc.message) from exc - if len(chosen) > max_bytes: + match = next( + hit for hit in hits + if hit.raw == chosen and identities_match(header_message_id(hit.raw), expected_id) + ) + if len(match.raw) > max_bytes: raise unprocessable( "Die vollständige Nachricht überschreitet die Exportgrenze. " "Eine gekürzte Vorschau wird nicht als Original übernommen." ) - return chosen + moved = not capped and (match.mailbox != mailbox or match.uid != uid) + return LocatedMail(match.raw, match.mailbox, match.uid, relocated=moved) finally: _imap_logout(client) +def _keep_moved_mail(conn, element_id, external_id: str, mailbox: str, uid: int | None, found) -> bytes: + """Return the bytes. A unique new place is written only after the fetch has finished.""" + if not isinstance(found, LocatedMail): + return found + if found.relocated and element_id is not None and uid is not None: + remember_imap_location( + conn, + element_id, + external_id, + mailbox, + uid, + found.mailbox, + found.uid, + ) + return found.raw + + def _fetch_rfc822(client: imaplib.IMAP4, uid: int) -> bytes | None: try: status, fetched = client.uid("FETCH", str(uid), "(BODY.PEEK[])") @@ -982,11 +1557,12 @@ def _fetch_rfc822(client: imaplib.IMAP4, uid: int) -> bytes | None: return _literal(fetched) -def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> list[bytes]: +def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> tuple[list[_MailHit], bool]: + """Search every selectable folder. The second value is true when the scan stopped early.""" token = (expected_id or "").strip() if not token or token.casefold().startswith("uid:"): - return [] - found: list[bytes] = [] + return [], False + found: list[_MailHit] = [] for folder in _imap_mailboxes(client): if not folder["selectable"]: continue @@ -1010,7 +1586,238 @@ def _search_rfc822(client: imaplib.IMAP4, expected_id: str, max_bytes: int) -> l "Die vollständige Nachricht überschreitet die Exportgrenze. " "Eine gekürzte Vorschau wird nicht als Original übernommen." ) - found.append(blob) + found.append(_MailHit(blob, folder["name"], uid)) if len(found) > 5: - return found - return found + return found, True + return found, False + + +PAPERLESS_FIELDS = ( + ConfigField( + "base_url", + "Adresse", + type="url", + required=True, + placeholder="https://paperless.example", + help="Adresse deiner Paperless-Instanz ohne Pfad.", + max_length=300, + ), + ConfigField( + "token", + "API-Schlüssel", + type="password", + required=True, + secret=True, + help="Den Schlüssel erzeugst du in Paperless unter deinem Profil.", + max_length=200, + ), +) + +IMAP_FIELDS = ( + ConfigField("host", "Server", required=True, placeholder="imap.example", max_length=200), + ConfigField("port", "Port", type="number", required=True, default=993, minimum=1, maximum=65535), + ConfigField( + "security", + "Verbindung", + type="select", + required=True, + default="ssl", + options=(("ssl", "SSL"), ("starttls", "STARTTLS")), + ), + ConfigField("username", "Benutzername", required=True, max_length=200), + ConfigField("password", "Passwort", type="password", required=True, secret=True, max_length=500), + ConfigField( + "mailbox", + "Standardfach", + default="INBOX", + help="Dieses Fach öffnet die Auswahl zuerst.", + max_length=200, + ), +) + + +def _paperless_normalize(settings: dict, secret: str) -> dict: + if len(secret.strip()) < 8: + raise unprocessable("Der API-Schlüssel ist zu kurz.") + return {**settings, "base_url": clean_base_url(settings["base_url"])} + + +def _imap_normalize(settings: dict, secret: str) -> dict: + del secret + if " " in settings["host"]: + raise unprocessable("Der Mailserver ist ungültig.") + _check_mailbox(settings.get("mailbox") or "INBOX") + return settings + + +def _paperless_find(conn, owner_id: UUID, connector_id: UUID | None, query: str, options: dict) -> dict: + return paperless_documents(conn, owner_id, query, connector_id, options) + + +def _imap_find(conn, owner_id: UUID, connector_id: UUID | None, query: str, options: dict) -> dict: + if connector_id is None: + raise unprocessable("Für dieses Postfach fehlt der Zugang.") + before = options.get("before") + validity = options.get("uidvalidity") + return imap_messages( + conn, + owner_id, + connector_id, + query, + str(options.get("mailbox") or ""), + before, + int(validity) if validity else None, + ) + + +def _export_paperless(conn, actor_id: UUID, external_id: str, detail: dict, assigned: UUID | None, ref): + from app.retrieval import EXPORT_BYTE_LIMIT, FilePayload, RetrievalFailure, failure_from_problem + + del detail + resolved = resolve_for_read(conn, actor_id, "paperless", ref, assigned) + if resolved.row is None: + return RetrievalFailure(resolved.code, resolved.message, resolved.choices) + base = resolved.row["settings"]["base_url"].rstrip("/") + token = resolved.row["secret"] + connector_id = resolved.row["id"] + owner_id = resolved.row["owner_id"] + release_for_external(conn) + try: + data, media, name = pull_paperless_original(base, token, str(external_id), EXPORT_BYTE_LIMIT) + except Problem as exc: + return failure_from_problem(exc) + try: + assert_still_owned(conn, connector_id, owner_id) + except Problem as exc: + return failure_from_problem(exc) + if len(data) > EXPORT_BYTE_LIMIT: + return RetrievalFailure("too_large", "Die Originaldatei ist für den Export zu groß.") + return FilePayload(name or f"paperless-{external_id}", media, data, "") + + +def _export_imap(conn, actor_id: UUID, external_id: str, detail: dict, assigned: UUID | None, ref): + from app.mailpack import identities_match, parse_mail + from app.retrieval import EXPORT_BYTE_LIMIT, MailPayload, RetrievalFailure, failure_from_problem + + resolved = resolve_for_read(conn, actor_id, "imap", ref, assigned) + if resolved.row is None: + return RetrievalFailure(resolved.code, resolved.message, resolved.choices) + mailbox = str(detail.get("mailbox") or "").strip() + uid = detail.get("uid") + if isinstance(uid, str) and uid.isdigit(): + uid = int(uid) + if not mailbox or isinstance(uid, bool) or not isinstance(uid, int) or uid < 1: + return RetrievalFailure("identity", "Für diese E-Mail fehlt die Zuordnung zu Fach und Nachricht.") + settings = dict(resolved.row["settings"]) + secret = resolved.row["secret"] + connector_id = resolved.row["id"] + owner_id = resolved.row["owner_id"] + release_for_external(conn) + try: + found = pull_rfc822(settings, secret, mailbox, uid, str(external_id), EXPORT_BYTE_LIMIT) + except Problem as exc: + return failure_from_problem(exc) + try: + assert_still_owned(conn, connector_id, owner_id) + except Problem as exc: + return failure_from_problem(exc) + element_id = ref["element_id"] if ref is not None and "element_id" in ref.keys() else None + raw = _keep_moved_mail(conn, element_id, str(external_id), mailbox, uid, found) + if len(raw) > EXPORT_BYTE_LIMIT: + return RetrievalFailure( + "too_large", + "Die vollständige Nachricht überschreitet die Exportgrenze. " + "Eine gekürzte Vorschau wird nicht als Original übernommen.", + ) + parsed = parse_mail(raw) + if not identities_match(parsed.message_id, str(external_id)): + return RetrievalFailure("identity", "Die gefundene Nachricht ist nicht die referenzierte E-Mail.") + if parsed.missing: + names = ", ".join(parsed.missing[:5]) + return RetrievalFailure("attachment", f"Eine Anlage der E-Mail ließ sich nicht vollständig lesen: {names}.") + return MailPayload(raw, parsed) + + +def _imap_bytes(conn, actor_id: UUID, element: dict, context: Ctx | None, ref) -> tuple[bytes, str, str]: + raw = read_linked_mail(conn, actor_id, element, context, ref) + return raw, "message/rfc822", "nachricht.eml" + + +class _PaperlessAdapter: + kind = "paperless" + + def capabilities(self) -> dict: + return CAPABILITIES[self.kind] + + def probe(self, row) -> dict: + count = _paperless_count(row["settings"]["base_url"], row["secret"]) + return {"ok": True, "detail": f"Paperless antwortet. {count} Dokumente sichtbar."} + + def search(self, conn, owner_id: UUID, query: str, connector_id: UUID | None): + found = paperless_documents(conn, owner_id, query, connector_id) + return found["items"], found["connector_id"] + + def find(self, conn, owner_id: UUID, connector_id: UUID | None, query: str, options: dict) -> dict: + return _paperless_find(conn, owner_id, connector_id, query, options) + + def facets(self, conn, owner_id: UUID, connector_id: UUID) -> dict: + return paperless_facets(conn, owner_id, connector_id) + + +class _ImapAdapter: + kind = "imap" + + def capabilities(self) -> dict: + return CAPABILITIES[self.kind] + + def probe(self, row) -> dict: + _imap_login(row["settings"], row["secret"]) + return {"ok": True, "detail": "Das Postfach ist erreichbar."} + + def navigate(self, conn, owner_id: UUID, connector_id: UUID) -> list[dict]: + return imap_mailboxes(conn, owner_id, connector_id) + + def search(self, conn, owner_id: UUID, connector_id: UUID, query: str, mailbox: str, before: int | None) -> dict: + return imap_messages(conn, owner_id, connector_id, query, mailbox, before) + + def find(self, conn, owner_id: UUID, connector_id: UUID | None, query: str, options: dict) -> dict: + return _imap_find(conn, owner_id, connector_id, query, options) + + +def _install_sources() -> None: + paperless = _PaperlessAdapter() + imap = _ImapAdapter() + register_source(SourceModule( + kind="paperless", + label="Paperless", + representation="pdf", + capabilities=CAPABILITIES["paperless"], + config_fields=PAPERLESS_FIELDS, + normalize_config=_paperless_normalize, + probe=paperless.probe, + find=paperless.find, + open_original=read_linked_paperless, + export_original=_export_paperless, + navigate=None, + preview=None, + facets=paperless.facets, + )) + register_source(SourceModule( + kind="imap", + label="E-Mail", + representation="email", + capabilities=CAPABILITIES["imap"], + config_fields=IMAP_FIELDS, + normalize_config=_imap_normalize, + probe=imap.probe, + find=imap.find, + open_original=_imap_bytes, + export_original=_export_imap, + navigate=imap.navigate, + preview=lambda conn, owner, connector_id, options: imap_preview( + conn, owner, connector_id, str(options.get("mailbox") or ""), int(options.get("uid") or 0) + ), + )) + + +_install_sources() diff --git a/backend/migrations/20261019T120000Z_source_storage.sql b/backend/migrations/20261019T120000Z_source_storage.sql new file mode 100644 index 0000000..a33b796 --- /dev/null +++ b/backend/migrations/20261019T120000Z_source_storage.sql @@ -0,0 +1,66 @@ +-- Source instance binding and opaque storage generation. +-- Connector ids are not a foreign key: deleting an access must not +-- retarget or remove an existing reference. +-- The legacy root path is bound in the migrator, not in this file. + +CREATE TABLE storage_instances ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + kind text NOT NULL, + generation text NOT NULL, + root_path text, + created_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT storage_instances_kind CHECK (kind IN ('local')), + CONSTRAINT storage_instances_generation UNIQUE (generation), + CONSTRAINT storage_instances_generation_len CHECK (char_length(generation) BETWEEN 1 AND 80) +); + +ALTER TABLE media_assets + ADD COLUMN storage_instance_id uuid REFERENCES storage_instances (id), + ADD COLUMN storage_generation text, + ADD COLUMN object_key text, + ADD COLUMN content_revision text, + ADD COLUMN protection_profile text NOT NULL DEFAULT 'legacy-plaintext', + ADD COLUMN key_responsibility text NOT NULL DEFAULT 'none', + ADD CONSTRAINT media_assets_protection CHECK ( + protection_profile IN ('legacy-plaintext', 'managed-encrypted', 'client-encrypted') + ), + ADD CONSTRAINT media_assets_key_responsibility CHECK ( + key_responsibility IN ('none', 'operator', 'client') + ); + +ALTER TABLE dispatch_snapshots + ADD COLUMN storage_instance_id uuid REFERENCES storage_instances (id), + ADD COLUMN storage_generation text, + ADD COLUMN object_key text, + ADD COLUMN content_revision text, + ADD COLUMN protection_profile text NOT NULL DEFAULT 'legacy-plaintext', + ADD COLUMN key_responsibility text NOT NULL DEFAULT 'none', + ADD CONSTRAINT dispatch_snapshots_protection CHECK ( + protection_profile IN ('legacy-plaintext', 'managed-encrypted', 'client-encrypted') + ), + ADD CONSTRAINT dispatch_snapshots_key_responsibility CHECK ( + key_responsibility IN ('none', 'operator', 'client') + ); + +UPDATE media_assets +SET storage_generation = 'legacy-local-1', + object_key = id::text, + content_revision = id::text, + protection_profile = 'legacy-plaintext', + key_responsibility = 'none' +WHERE object_key IS NULL; + +UPDATE dispatch_snapshots +SET storage_generation = 'legacy-local-1', + object_key = 'dispatch/' || id::text, + content_revision = id::text, + protection_profile = 'legacy-plaintext', + key_responsibility = 'none' +WHERE object_key IS NULL; + +ALTER TABLE external_references + ADD COLUMN connector_id uuid, + ADD COLUMN resolution text NOT NULL DEFAULT 'unresolved', + ADD CONSTRAINT external_references_resolution CHECK ( + resolution IN ('bound', 'ambiguous', 'unresolved', 'none') + ); diff --git a/backend/migrations/20261019T150000Z_source_registry.sql b/backend/migrations/20261019T150000Z_source_registry.sql new file mode 100644 index 0000000..57f40c9 --- /dev/null +++ b/backend/migrations/20261019T150000Z_source_registry.sql @@ -0,0 +1,5 @@ +-- Connector kinds are validated by the registered source modules. +-- A database checklist must not grow for every additional source. + +ALTER TABLE connectors DROP CONSTRAINT IF EXISTS connectors_kind; +ALTER TABLE external_references DROP CONSTRAINT IF EXISTS external_references_source; diff --git a/backend/migrations/20261020T120000Z_connector_probe_status.sql b/backend/migrations/20261020T120000Z_connector_probe_status.sql new file mode 100644 index 0000000..8b14b76 --- /dev/null +++ b/backend/migrations/20261020T120000Z_connector_probe_status.sql @@ -0,0 +1,8 @@ +-- Last connection check per connector instance. Display only: it grants nothing +-- and carries no secret. + +ALTER TABLE connectors + ADD COLUMN last_probe_at timestamptz, + ADD COLUMN last_probe_ok boolean, + ADD COLUMN last_probe_detail text, + ADD CONSTRAINT connectors_probe_detail_len CHECK (last_probe_detail IS NULL OR char_length(last_probe_detail) <= 300); diff --git a/backend/migrations/20261020T150000Z_user_admin.sql b/backend/migrations/20261020T150000Z_user_admin.sql new file mode 100644 index 0000000..eb006bc --- /dev/null +++ b/backend/migrations/20261020T150000Z_user_admin.sql @@ -0,0 +1,31 @@ +-- Benutzerverwaltung: Kontostatus, Sitzungsübersicht, persistente Login-Begrenzung, +-- Raumrolle "Leser" und Rollen in Einladungen. Bestehende Zeilen bleiben gültig. + +ALTER TABLE auth_accounts + ADD COLUMN must_change_password boolean NOT NULL DEFAULT false, + ADD COLUMN password_changed_at timestamptz, + ADD COLUMN last_login_at timestamptz, + ADD COLUMN disabled_at timestamptz, + ADD COLUMN disabled_by uuid REFERENCES auth_accounts (id); + +ALTER TABLE auth_sessions + ADD COLUMN last_seen_at timestamptz, + ADD COLUMN label text, + ADD CONSTRAINT auth_sessions_label_len CHECK (label IS NULL OR char_length(label) <= 80); + +-- Der Bucket ist ein Hash aus Adresse und E-Mail. Die Roh-E-Mail wird nie gespeichert. +CREATE TABLE auth_login_attempts ( + bucket text NOT NULL, + attempted_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX auth_login_attempts_bucket ON auth_login_attempts (bucket, attempted_at); + +ALTER TABLE room_memberships DROP CONSTRAINT room_memberships_role; +ALTER TABLE room_memberships + ADD CONSTRAINT room_memberships_role CHECK (role IN ('member', 'manager', 'reader')); + +ALTER TABLE auth_invitations + ADD COLUMN role text NOT NULL DEFAULT 'member', + ADD COLUMN revoked_at timestamptz, + ADD CONSTRAINT auth_invitations_role CHECK (role IN ('member', 'manager', 'reader')); diff --git a/backend/migrations/20261021T090000Z_room_access.sql b/backend/migrations/20261021T090000Z_room_access.sql new file mode 100644 index 0000000..bfd2f7d --- /dev/null +++ b/backend/migrations/20261021T090000Z_room_access.sql @@ -0,0 +1,21 @@ +-- Raum-Eigentümer und Raumfreigabe. Bestehende Akten und Vorgänge bleiben privat. + +ALTER TABLE data_rooms + ADD COLUMN owner_id uuid REFERENCES auth_accounts (id), + ADD COLUMN default_visibility text NOT NULL DEFAULT 'private', + ADD CONSTRAINT data_rooms_default_visibility CHECK (default_visibility IN ('private', 'room')); + +UPDATE data_rooms SET owner_id = created_by; + +ALTER TABLE data_rooms + ADD CONSTRAINT data_rooms_owner CHECK (kind = 'personal' OR owner_id IS NOT NULL); + +CREATE INDEX data_rooms_owner ON data_rooms (owner_id) WHERE kind = 'organization'; + +ALTER TABLE cases + ADD COLUMN visibility text NOT NULL DEFAULT 'private', + ADD CONSTRAINT cases_visibility CHECK (visibility IN ('private', 'room')); + +ALTER TABLE matters + ADD COLUMN visibility text NOT NULL DEFAULT 'private', + ADD CONSTRAINT matters_visibility CHECK (visibility IN ('private', 'room')); diff --git a/backend/migrations/20261021T150000Z_room_shares.sql b/backend/migrations/20261021T150000Z_room_shares.sql new file mode 100644 index 0000000..aba6d26 --- /dev/null +++ b/backend/migrations/20261021T150000Z_room_shares.sql @@ -0,0 +1,32 @@ +-- Raum als Team: Eine Akte oder ein Vorgang wird mit beliebigen Datenräumen geteilt, +-- unabhängig vom Heimatraum. Der Heimatraum bleibt Ablage. Die Freigabe zum Heimatraum +-- aus 20261021T090000Z_room_access.sql wird übernommen, danach entfallen die Spalten. + +CREATE TABLE room_shares ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + resource_kind text NOT NULL, + resource_id uuid NOT NULL, + room_id uuid NOT NULL REFERENCES data_rooms (id), + -- Obergrenze der Freigabe: 'edit' = Rechte nach Raumrolle, 'read' = höchstens Lesen. + access text NOT NULL DEFAULT 'edit', + created_by uuid NOT NULL REFERENCES auth_accounts (id), + created_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT room_shares_kind CHECK (resource_kind IN ('case', 'matter')), + CONSTRAINT room_shares_access CHECK (access IN ('read', 'edit')), + UNIQUE (resource_kind, resource_id, room_id) +); + +CREATE INDEX room_shares_room ON room_shares (room_id); + +INSERT INTO room_shares (resource_kind, resource_id, room_id, access, created_by) +SELECT 'case', c.id, c.home_scope_id, 'edit', c.owner_subject_id +FROM cases c +WHERE c.visibility = 'room'; + +INSERT INTO room_shares (resource_kind, resource_id, room_id, access, created_by) +SELECT 'matter', m.id, m.home_scope_id, 'edit', m.owner_subject_id +FROM matters m +WHERE m.visibility = 'room'; + +ALTER TABLE cases DROP CONSTRAINT cases_visibility, DROP COLUMN visibility; +ALTER TABLE matters DROP CONSTRAINT matters_visibility, DROP COLUMN visibility; diff --git a/backend/migrations/20261022T090000Z_account_kinds.sql b/backend/migrations/20261022T090000Z_account_kinds.sql new file mode 100644 index 0000000..307c8d2 --- /dev/null +++ b/backend/migrations/20261022T090000Z_account_kinds.sql @@ -0,0 +1,13 @@ +-- Hauptnutzer und Mitarbeiter. Alle bestehenden Konten sind Hauptnutzer. +-- Ein Mitarbeiter-Konto entsteht durch eine Einladung und hat den einladenden Eigentümer als Förderer. + +ALTER TABLE auth_accounts + ADD COLUMN account_kind text NOT NULL DEFAULT 'main', + ADD COLUMN sponsor_id uuid REFERENCES auth_accounts (id), + ADD CONSTRAINT auth_accounts_kind CHECK (account_kind IN ('main', 'employee')), + ADD CONSTRAINT auth_accounts_sponsor CHECK ( + (account_kind = 'employee' AND sponsor_id IS NOT NULL) + OR (account_kind = 'main' AND sponsor_id IS NULL) + ); + +CREATE INDEX auth_accounts_sponsor ON auth_accounts (sponsor_id) WHERE sponsor_id IS NOT NULL; diff --git a/backend/migrations/20261023T090000Z_download_policy.sql b/backend/migrations/20261023T090000Z_download_policy.sql new file mode 100644 index 0000000..1353da5 --- /dev/null +++ b/backend/migrations/20261023T090000Z_download_policy.sql @@ -0,0 +1,11 @@ +-- Herunterladen des Originals: je Rolle steuerbar. Voreinstellung am Konto des Hauptnutzers, +-- Übernahme beim Anlegen eines Raums, danach je Raum änderbar. "direct" gilt für Personen, +-- die einen Eintrag direkt erhalten haben (ohne Raum). Bestehende Daten bleiben erlaubt. + +ALTER TABLE auth_accounts + ADD COLUMN download_defaults jsonb NOT NULL + DEFAULT '{"manager": true, "member": true, "reader": true, "direct": true}'::jsonb; + +ALTER TABLE data_rooms + ADD COLUMN download_roles jsonb NOT NULL + DEFAULT '{"manager": true, "member": true, "reader": true}'::jsonb; diff --git a/backend/tests/conftest.py b/backend/tests/conftest.py index 1fe3d3d..f52edf8 100644 --- a/backend/tests/conftest.py +++ b/backend/tests/conftest.py @@ -43,6 +43,7 @@ def _reset(settings) -> None: TRUNCATE TABLE media_operations, domain_events, + auth_login_attempts, auth_accounts, data_rooms RESTART IDENTITY CASCADE @@ -86,13 +87,9 @@ def client(database): @pytest.fixture(autouse=True) def clean(database, client): - from app.api import routes - - routes._login_attempts.clear() client.cookies.clear() _reset(database) yield - routes._login_attempts.clear() client.cookies.clear() _reset(database) diff --git a/backend/tests/sample_source.py b/backend/tests/sample_source.py new file mode 100644 index 0000000..337ac3e --- /dev/null +++ b/backend/tests/sample_source.py @@ -0,0 +1,165 @@ +"""Third source used only to prove that registration is enough. + +This module is not a product connector and is not imported by the application. +""" + +from __future__ import annotations + +from uuid import UUID + +from app.core.errors import not_found, unprocessable +from app.retrieval import FilePayload, RetrievalFailure +from app.sourcebind import resolve_for_read +from app.sourceregistry import ConfigField, SourceModule, register_source, unregister_source +from app.sourceaccess import borrow_original, complete_borrow + +KIND = "sample" + +CATALOG = { + "alpha": { + "brief": { + "id": "brief", + "title": "Probenbrief", + "text": "Der Probenbrief bleibt lesbar.", + }, + "notiz": { + "id": "notiz", + "title": "Randnotiz", + "text": "Die Randnotiz bleibt in der Quelle.", + }, + } +} + +CAPABILITIES = { + "kind": KIND, + "navigate": False, + "read_original": True, + "read_preview": True, + "read_attachment": False, + "edit_content": False, + "edit_metadata": False, + "search": { + "query": True, + "title": False, + "correspondent": False, + "document_type": False, + "tags": False, + "document_range": False, + "sort": False, + "continuation": False, + }, +} + + +CONFIG_FIELDS = ( + ConfigField("catalog", "Katalog", required=True, default="alpha", help="Name des Beispielkatalogs."), +) + + +def normalize_config(settings: dict, secret: str) -> dict: + del secret + if settings["catalog"] not in CATALOG: + raise unprocessable("Der Katalog dieser Beispielquelle ist unbekannt.") + return settings + + +def probe(row) -> dict: + items = _items(row) + return {"ok": True, "detail": f"Beispielquelle antwortet. {len(items)} Einträge sichtbar."} + + +def find(conn, owner_id: UUID, connector_id: UUID | None, query: str, options: dict) -> dict: + del options + row = _connector(conn, owner_id, connector_id) + needle = query.casefold() + items = list(_items(row).values()) + if needle: + items = [ + item for item in items + if needle in item["title"].casefold() or needle in item["text"].casefold() + ] + return {"items": items, "connector_id": str(row["id"])} + + +def preview(conn, owner_id: UUID, connector_id: UUID, options: dict) -> dict: + row = _connector(conn, owner_id, connector_id) + item = _items(row).get(str(options.get("item") or "")) + if item is None: + raise unprocessable("Dieser Eintrag ist in der Beispielquelle nicht vorhanden.") + return {"id": item["id"], "title": item["title"], "text": item["text"], "media": "text"} + + +def open_original(conn, actor_id: UUID, element: dict, context, ref) -> tuple[bytes, str, str]: + borrowed = borrow_original(conn, actor_id, element, context, ref, KIND) + return complete_borrow(conn, borrowed, lambda row: _bytes(row, borrowed.external_id)) + + +def export_original(conn, actor_id: UUID, external_id: str, detail: dict, assigned, ref): + del detail + result = resolve_for_read(conn, actor_id, KIND, ref, assigned) + if result.row is None: + return RetrievalFailure(result.code, result.message, result.choices) + try: + data, media, name = _bytes(result.row, str(external_id)) + except Exception as exc: + return RetrievalFailure("missing", str(getattr(exc, "detail", exc))) + return FilePayload(name, media, data, "") + + +def install_sample_source() -> None: + register_source(SourceModule( + kind=KIND, + label="Beispielquelle", + representation="text", + capabilities=CAPABILITIES, + config_fields=CONFIG_FIELDS, + normalize_config=normalize_config, + probe=probe, + find=find, + open_original=open_original, + export_original=export_original, + navigate=None, + preview=preview, + )) + + +def remove_sample_source() -> None: + unregister_source(KIND) + + +def _connector(conn, owner_id: UUID, connector_id: UUID | None): + if connector_id is not None: + row = conn.execute( + """ + SELECT id, owner_id, kind, name, settings, secret + FROM connectors WHERE id = %s AND owner_id = %s + """, + (connector_id, owner_id), + ).fetchone() + if row is None or row["kind"] != KIND: + raise not_found() + return row + rows = conn.execute( + """ + SELECT id, owner_id, kind, name, settings, secret + FROM connectors + WHERE owner_id = %s AND kind = %s + ORDER BY name, id + """, + (owner_id, KIND), + ).fetchall() + if len(rows) != 1: + raise unprocessable("Für diese Quelle ist kein eindeutiger Zugang zugeordnet.") + return rows[0] + + +def _items(row) -> dict: + catalog = (row["settings"] or {}).get("catalog") + return CATALOG.get(catalog) or {} + + +def _bytes(row, external_id: str) -> tuple[bytes, str, str]: + item = _items(row).get(external_id) + if item is None: + raise unprocessable("Dieser Eintrag ist in der Beispielquelle nicht vorhanden.") + return item["text"].encode("utf-8"), "text/plain", f"{item['id']}.txt" diff --git a/backend/tests/test_accounts.py b/backend/tests/test_accounts.py index 96c276c..cb79872 100644 --- a/backend/tests/test_accounts.py +++ b/backend/tests/test_accounts.py @@ -35,7 +35,8 @@ def test_bootstrap_once_then_superadmin_creates_users(client): status=201, ) assert created["is_superadmin"] is False - assert "password" not in created + assert created["must_change_password"] is True + assert "password" not in created and "temporary_password" not in created listed = owner.request("GET", "/api/v1/accounts") assert {item["email"] for item in listed["items"]} == {"betreiber@example.test", "mitglied@example.test"} assert all("password_hash" not in item and "password" not in item for item in listed["items"]) diff --git a/backend/tests/test_connector_basis.py b/backend/tests/test_connector_basis.py new file mode 100644 index 0000000..be1b56f --- /dev/null +++ b/backend/tests/test_connector_basis.py @@ -0,0 +1,539 @@ +"""Section A: source binding and the storage port, with today's behaviour kept.""" + +from __future__ import annotations + +import hashlib +import json +from pathlib import Path +from uuid import uuid4 + +import pytest + +from app.core.errors import Problem +from app.media.port import ( + LEGACY_GENERATION, + LocalAdapter, + MemoryAdapter, + check_object_key, + instance_for_write, + publish, + read_clear_bytes, +) +from app.sourcebind import backfill_source_bindings +from tests.support import ORIGIN, Actor, register + +PDF = b"%PDF-1.4\n1 0 obj<<>>endobj\ntrailer<<>>\n%%EOF\n" + + +def _raw(actor: Actor, path: str): + return actor.client.get( + path, + headers={ + "Origin": ORIGIN, + "X-CSRF-Token": actor.csrf, + "X-Scope-Kind": actor.scope["kind"], + "X-Scope-Id": actor.scope["id"], + }, + ) + + +def _case(actor: Actor) -> dict: + return actor.request("POST", "/api/v1/cases", json={"title": "Bestand"}, status=201) + + +def _paperless(actor: Actor, name: str, token: str) -> str: + row = actor.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": name, "base_url": "https://paper.example.test", "token": token}, + status=201, + ) + return row["id"] + + +def _link(actor: Actor, case_id: str, connector_id: str | None, external_id: str, title: str) -> str: + body = { + "title": title, + "source": "paperless", + "external_id": external_id, + "context": {"kind": "case", "id": case_id}, + } + if connector_id: + body["detail"] = {"connector_id": connector_id} + created = actor.request("POST", "/api/v1/external-refs", json=body, status=201) + return created["id"] + + +def test_storage_adapters_roundtrip_and_reject_escape(tmp_path): + payload = b"opaque-bytes" + key = str(uuid4()) + revision = "rev-1" + for adapter in (LocalAdapter(tmp_path), MemoryAdapter()): + publish(adapter, key, payload) + assert adapter.open(key) == payload + assert adapter.describe(key)["byte_size"] == len(payload) + assert adapter.open(key) == payload + assert revision == "rev-1" + memory = MemoryAdapter() + memory.fail_put = True + with pytest.raises(Problem) as failed: + publish(memory, str(uuid4()), payload) + assert failed.value.code == "storage_unavailable" + assert memory._items == {} + with pytest.raises(Problem): + check_object_key("../secret") + with pytest.raises(Problem): + check_object_key("/etc/passwd") + with pytest.raises(Problem): + LocalAdapter(tmp_path).open("dispatch/../../" + str(uuid4())) + + +def test_changed_root_does_not_retarget_legacy(db, database): + legacy = db.execute( + "SELECT id, root_path FROM storage_instances WHERE generation = %s", + (LEGACY_GENERATION,), + ).fetchone() + key = str(uuid4()) + original = b"%PDF-legacy-root" + publish(LocalAdapter(legacy["root_path"]), key, original) + other = Path(database.media_root).parent / f"other-root-{uuid4().hex}" + other.mkdir() + moved = instance_for_write(db, str(other)) + assert moved["id"] != legacy["id"] + assert moved["generation"] != LEGACY_GENERATION + publish(LocalAdapter(moved["root_path"]), key, b"%PDF-other-root") + seen = read_clear_bytes( + db, + instance_id=legacy["id"], + generation=LEGACY_GENERATION, + object_key=key, + media_root=str(other), + protection_profile="legacy-plaintext", + expected_sha256=None, + ) + assert seen == original + db.rollback() + + +def test_existing_file_keeps_id_bytes_and_revision(client, db): + register(db, "ada-file@example.test", "Ada File") + ada = Actor(client, "ada-file@example.test", "Ada File") + ada.login() + case = _case(ada) + upload = ada.request( + "POST", + "/api/v1/files", + data={"title": "Scan", "context_kind": "case", "context_id": case["id"]}, + files={"upload": ("scan.pdf", PDF, "application/pdf")}, + status=201, + ) + stored = db.execute( + """ + SELECT a.id, a.object_key, a.content_revision, a.storage_generation, a.protection_profile, + s.root_path + FROM media_assets a + JOIN storage_instances s ON s.id = a.storage_instance_id + WHERE a.element_id = %s + """, + (upload["id"],), + ).fetchone() + assert stored["protection_profile"] == "legacy-plaintext" + assert stored["object_key"] == str(stored["id"]) + assert Path(stored["root_path"]) == Path(client.app.state.settings.media_root).resolve() + assert stored["storage_generation"] + first = stored["content_revision"] + opened = _raw(ada, f"/api/v1/files/{upload['id']}/content") + assert opened.status_code == 200 + assert opened.content == PDF + again = db.execute( + "SELECT content_revision FROM media_assets WHERE element_id = %s", + (upload["id"],), + ).fetchone() + assert again["content_revision"] == first + refused = ada.client.patch( + f"/api/v1/notes/{upload['id']}", + json={"expected_version": 1, "document": {"version": 1, "blocks": [{"type": "paragraph", "text": "nein"}]}}, + headers={ + "Origin": ORIGIN, + "X-CSRF-Token": ada.csrf, + "X-Scope-Kind": ada.scope["kind"], + "X-Scope-Id": ada.scope["id"], + }, + ) + assert refused.status_code == 422 + assert refused.json()["code"] == "unsupported" + + +def test_two_paperless_instances_stay_bound_when_order_changes(client, db, monkeypatch): + register(db, "ada-two@example.test", "Ada Two") + ada = Actor(client, "ada-two@example.test", "Ada Two") + ada.login() + case = _case(ada) + older = _paperless(ada, "Zweitens", "token-older") + newer = _paperless(ada, "Erstens", "token-newer") + assert older != newer + + def fake_bytes(row, document_id): + return f"%PDF-{row['secret']}-{document_id}\n".encode(), "application/pdf", "dokument.pdf" + + monkeypatch.setattr("app.sources._paperless_bytes", fake_bytes) + left = _link(ada, case["id"], older, "15", "Altbestand") + right = _link(ada, case["id"], newer, "15", "Zweiter Bestand") + db.execute("UPDATE connectors SET created_at = now() - interval '2 days' WHERE id = %s", (newer,)) + db.execute("UPDATE connectors SET created_at = now() WHERE id = %s", (older,)) + db.commit() + assert _raw(ada, f"/api/v1/external-refs/{left}/content").content.startswith(b"%PDF-token-older-15") + assert _raw(ada, f"/api/v1/external-refs/{right}/content").content.startswith(b"%PDF-token-newer-15") + calls = [] + + def fake_get(*args, **kwargs): + calls.append(args) + return {"results": [{"id": 15, "title": "Treffer", "created": "2024-01-02"}]} + + monkeypatch.setattr("app.sources._paperless_get", fake_get) + headers = { + "Origin": ORIGIN, + "X-CSRF-Token": ada.csrf, + "X-Scope-Kind": ada.scope["kind"], + "X-Scope-Id": ada.scope["id"], + } + refused = ada.client.get("/api/v1/sources/paperless/documents", params={"q": "Treffer"}, headers=headers) + assert refused.status_code == 422 + assert calls == [] + found = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"q": "Treffer", "connector_id": older}, + ) + assert found["items"][0]["title"] == "Treffer" + assert found["connector_id"] == older + assert found["capabilities"]["search"]["correspondent"] is True + assert "correspondent" not in found["unsupported"] + filtered = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"q": "Treffer", "connector_id": older, "correspondent": "4"}, + ) + assert filtered["items"][0]["title"] == "Treffer" + assert calls[-1][3]["correspondent__id"] == "4" + assert calls[-1][3]["query"] == "Treffer" + + +def test_ambiguous_legacy_stays_until_assignment(client, db, monkeypatch): + register(db, "ada-old@example.test", "Ada Old") + ada = Actor(client, "ada-old@example.test", "Ada Old") + ada.login() + case = _case(ada) + loose = _link(ada, case["id"], None, "15", "Ohne Instanz") + _paperless(ada, "Alpha", "token-alpha") + second = _paperless(ada, "Beta", "token-beta") + db.execute( + """ + UPDATE external_references + SET connector_id = NULL, resolution = 'unresolved', detail = '{}'::jsonb + WHERE element_id = %s + """, + (loose,), + ) + backfill_source_bindings(db) + state = db.execute( + "SELECT connector_id, resolution FROM external_references WHERE element_id = %s", + (loose,), + ).fetchone() + db.commit() + assert state["resolution"] == "ambiguous" + assert state["connector_id"] is None + opened = ada.request("GET", f"/api/v1/cases/{case['id']}") + assert opened["bindings"][0]["element"]["id"] == loose + assert opened["bindings"][0]["element"]["preview"]["media"] == "unassigned" + blocked = _raw(ada, f"/api/v1/external-refs/{loose}/content") + assert blocked.status_code == 422 + assert "eindeutiger Zugang" in blocked.json()["detail"] + + def fake_bytes(row, document_id): + del document_id + return f"%PDF-{row['secret']}\n".encode(), "application/pdf", "dokument.pdf" + + monkeypatch.setattr("app.sources._paperless_bytes", fake_bytes) + ada.request( + "POST", + f"/api/v1/external-refs/{loose}/assignment", + json={"connector_id": second}, + ) + _paperless(ada, "Gamma", "token-gamma") + assert _raw(ada, f"/api/v1/external-refs/{loose}/content").content.startswith(b"%PDF-token-beta") + + +def test_legacy_stock_rebinds_and_case_and_matter_open(client, db, monkeypatch): + from app.sourceregistry import get_source, unregister_source + + register(db, "ada-keep@example.test", "Ada Keep") + ada = Actor(client, "ada-keep@example.test", "Ada Keep") + ada.login() + case = _case(ada) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Bestand"}, status=201) + kept = _paperless(ada, "Archiv", "token-kept") + case_ref = _link(ada, case["id"], None, "15", "Vertragsakte") + matter_ref = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Vertragsvorgang", + "source": "paperless", + "external_id": "16", + "context": {"kind": "matter", "id": matter["id"]}, + }, + status=201, + )["id"] + db.execute( + """ + UPDATE external_references + SET connector_id = NULL, resolution = 'unresolved', detail = '{}'::jsonb + WHERE element_id IN (%s, %s) + """, + (case_ref, matter_ref), + ) + db.commit() + unregister_source("paperless") + unregister_source("imap") + assert get_source("paperless") is None + try: + backfill_source_bindings(db) + db.commit() + for element_id in (case_ref, matter_ref): + state = db.execute( + "SELECT connector_id::text AS connector_id, resolution FROM external_references WHERE element_id = %s", + (element_id,), + ).fetchone() + assert state["resolution"] == "bound" + assert state["connector_id"] == kept + assert get_source("paperless") is not None + case_page = ada.request("GET", f"/api/v1/cases/{case['id']}") + matter_page = ada.request("GET", f"/api/v1/matters/{matter['id']}") + assert case_page["bindings"][0]["element"]["preview"]["available"] is True + assert case_page["bindings"][0]["element"]["preview"]["media"] == "pdf" + assert matter_page["bindings"][0]["element"]["preview"]["available"] is True + monkeypatch.setattr( + "app.sources._paperless_bytes", + lambda row, document_id: (f"%PDF-{row['secret']}-{document_id}\n".encode(), "application/pdf", "dokument.pdf"), + ) + assert _raw(ada, f"/api/v1/external-refs/{case_ref}/content").content.startswith(b"%PDF-token-kept-15") + assert _raw(ada, f"/api/v1/external-refs/{matter_ref}/content").content.startswith(b"%PDF-token-kept-16") + finally: + from app.sources import _install_sources + + _install_sources() + + +def test_connectors_remain_with_their_account(client, db, monkeypatch): + calls = [] + monkeypatch.setattr( + "app.sources._paperless_get", + lambda *args, **kwargs: calls.append(args) or {"results": []}, + ) + register(db, "ada-own@example.test", "Ada Own") + register(db, "ben-own@example.test", "Ben Own") + ada = Actor(client, "ada-own@example.test", "Ada Own") + ben = Actor(client, "ben-own@example.test", "Ben Own") + ada.login() + ada_paper = _paperless(ada, "Ada Archiv", "token-ada") + ben.login() + ben_paper = _paperless(ben, "Ben Archiv", "token-ben") + listed = ben.request("GET", "/api/v1/connectors") + ids = {item["id"] for item in listed["items"]} + assert ben_paper in ids + assert ada_paper not in ids + assert all(item["name"] != "Ada Archiv" for item in listed["items"]) + ben.request("POST", f"/api/v1/connectors/{ada_paper}/probe", status=404) + ben.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"q": "fremd", "connector_id": ada_paper}, + status=404, + ) + assert calls == [] + ada.login() + own = {item["id"] for item in ada.request("GET", "/api/v1/connectors")["items"]} + assert ada_paper in own + assert ben_paper not in own + + +def test_locked_profile_serves_metadata_not_cleartext(client, db): + register(db, "ada-lock@example.test", "Ada Lock") + ada = Actor(client, "ada-lock@example.test", "Ada Lock") + ada.login() + case = _case(ada) + upload = ada.request( + "POST", + "/api/v1/files", + data={"title": "Geschuetzt", "context_kind": "case", "context_id": case["id"]}, + files={"upload": ("geheim.pdf", PDF, "application/pdf")}, + status=201, + ) + dummy = b"cipher-dummy-not-clear" + asset = db.execute( + "SELECT id, object_key, storage_instance_id FROM media_assets WHERE element_id = %s", + (upload["id"],), + ).fetchone() + root = db.execute( + "SELECT root_path FROM storage_instances WHERE id = %s", + (asset["storage_instance_id"],), + ).fetchone()["root_path"] + path = Path(root) / asset["object_key"] + path.write_bytes(dummy) + db.execute( + """ + UPDATE media_assets + SET protection_profile = 'client-encrypted', + key_responsibility = 'client', + sha256 = %s + WHERE element_id = %s + """, + (hashlib.sha256(dummy).hexdigest(), upload["id"]), + ) + db.commit() + detail = ada.request("GET", f"/api/v1/cases/{case['id']}") + rendered = json.dumps(detail) + assert "client-encrypted" in rendered + assert "Geschuetzt" in rendered + assert dummy.decode() not in rendered + opened = _raw(ada, f"/api/v1/files/{upload['id']}/content") + assert opened.status_code == 422 + assert opened.json()["code"] == "unsupported" + assert path.read_bytes() == dummy + + +def test_credential_update_keeps_the_instance(client, db): + register(db, "ada-key@example.test", "Ada Key") + ada = Actor(client, "ada-key@example.test", "Ada Key") + ada.login() + first = _paperless(ada, "Archiv", "token-first-1") + second = _paperless(ada, "Anderes Archiv", "token-second-2") + renewed = ada.request( + "PATCH", + f"/api/v1/connectors/{first}", + json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-renewed"}, + ) + assert renewed["id"] == first + secrets = { + str(row["id"]): row["secret"] + for row in db.execute("SELECT id, secret FROM connectors").fetchall() + } + assert secrets[first] == "token-renewed" + assert secrets[second] == "token-second-2" + caps = ada.request("GET", "/api/v1/sources/capabilities") + assert caps["editors"]["note"] is True + assert caps["editors"]["markdown_file"] is False + assert caps["protection_profiles"]["client-encrypted"] == "locked" + paperless = next(item for item in caps["sources"] if item["kind"] == "paperless") + assert paperless["search"]["query"] is True + assert paperless["edit_content"] is False + + +def test_legacy_reference_stays_bound_and_foreign_connectors_stay_hidden(client, db, database, monkeypatch): + """A startup without loaded modules must still keep the one existing connection.""" + from app.db.migrate import _prepare_source_storage + from app.sourceregistry import unregister_source + from app.sources import _install_sources + + def fake_bytes(row, document_id): + del document_id + return f"%PDF-{row['secret']}\n".encode(), "application/pdf", "dokument.pdf" + + monkeypatch.setattr("app.sources._paperless_bytes", fake_bytes) + register(db, "ada-legacy@example.test", "Ada Legacy") + register(db, "ben-legacy@example.test", "Ben Legacy") + ada = Actor(client, "ada-legacy@example.test", "Ada Legacy") + ben = Actor(client, "ben-legacy@example.test", "Ben Legacy") + ada.login() + case = _case(ada) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Altbestand"}, status=201) + connector = _paperless(ada, "Archiv", "token-legacy") + case_ref = _link(ada, case["id"], None, "15", "Akte") + matter_ref = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Vorgang", + "source": "paperless", + "external_id": "16", + "context": {"kind": "matter", "id": matter["id"]}, + }, + status=201, + )["id"] + db.execute( + """ + UPDATE external_references + SET connector_id = NULL, resolution = 'unresolved', detail = '{}'::jsonb + WHERE element_id IN (%s, %s) + """, + (case_ref, matter_ref), + ) + db.commit() + + opened = ada.request("GET", f"/api/v1/cases/{case['id']}") + shown = next(item["element"] for item in opened["bindings"] if item["element"]["id"] == case_ref) + assert shown["preview"]["media"] == "pdf" + assert shown["preview"]["available"] is True + assert _raw(ada, f"/api/v1/external-refs/{case_ref}/content?context_kind=case&context_id={case['id']}").content.startswith( + b"%PDF-token-legacy" + ) + matter_view = ada.request("GET", f"/api/v1/matters/{matter['id']}") + matter_element = next(item["element"] for item in matter_view["bindings"] if item["element"]["id"] == matter_ref) + assert matter_element["preview"]["media"] == "pdf" + + unregister_source("paperless") + unregister_source("imap") + try: + _prepare_source_storage(database) + finally: + _install_sources() + state = db.execute( + "SELECT connector_id::text AS connector_id, resolution FROM external_references WHERE element_id = %s", + (case_ref,), + ).fetchone() + assert state["connector_id"] == connector + assert state["resolution"] == "bound" + + ben.login() + own = _paperless(ben, "Bens Archiv", "token-ben") + listed = {item["id"] for item in ben.request("GET", "/api/v1/connectors")["items"]} + assert listed == {own} + ben.request("POST", f"/api/v1/connectors/{connector}/probe", status=404) + ben.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"q": "Alt", "connector_id": connector}, + status=404, + ) + ada.login() + ada.request( + "POST", + f"/api/v1/matters/{matter['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False, "use_owner_sources": False}, + status=201, + ) + ben.login() + shared = ben.request("GET", f"/api/v1/matters/{matter['id']}") + shared_element = next(item["element"] for item in shared["bindings"] if item["element"]["id"] == matter_ref) + assert "connector_id" not in shared_element["external"]["detail"] + assert "external_id" not in shared_element["external"] + blocked = _raw( + ben, + f"/api/v1/external-refs/{matter_ref}/content?context_kind=matter&context_id={matter['id']}", + ) + assert blocked.status_code == 422 + ada.login() + ada.request( + "POST", + f"/api/v1/matters/{matter['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False, "use_owner_sources": True}, + status=201, + ) + ben.login() + allowed = _raw( + ben, + f"/api/v1/external-refs/{matter_ref}/content?context_kind=matter&context_id={matter['id']}", + ) + assert allowed.status_code == 200 + assert allowed.content.startswith(b"%PDF-token-legacy") diff --git a/backend/tests/test_connector_config.py b/backend/tests/test_connector_config.py new file mode 100644 index 0000000..4cac055 --- /dev/null +++ b/backend/tests/test_connector_config.py @@ -0,0 +1,215 @@ +"""Connector configuration: fields come from the module, secrets never leave the server.""" + +from __future__ import annotations + +import pytest + +from app.core.errors import Problem +from app.sourceregistry import ConfigField, parse_fields +from tests.sample_source import install_sample_source, remove_sample_source +from tests.support import Actor, register + +SECRET_WORDS = ("secret", "token-geheim-1234", "passwort-geheim") + +PAPERLESS = {"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-geheim-1234"} +IMAP = { + "kind": "imap", + "name": "Postfach", + "host": "imap.example.test", + "port": 993, + "username": "ada", + "password": "passwort-geheim", + "security": "ssl", +} + + +@pytest.fixture +def sample_source(): + install_sample_source() + try: + yield + finally: + remove_sample_source() + + +def _ada(client, db) -> Actor: + ada = Actor(client, "ada-config@example.test", "Ada Konfig") + register(db, ada.email, ada.name) + ada.login() + return ada + + +def _stored(db, connector_id: str) -> dict: + return db.execute("SELECT settings, secret FROM connectors WHERE id = %s", (connector_id,)).fetchone() + + +def test_parse_fields_checks_types_and_refuses_unknown_keys(): + fields = ( + ConfigField("host", "Server", required=True), + ConfigField("port", "Port", type="number", default=993, minimum=1, maximum=65535), + ConfigField("mode", "Modus", type="select", default="a", options=(("a", "A"), ("b", "B"))), + ConfigField("key", "Schlüssel", type="password", required=True, secret=True), + ) + settings, secret = parse_fields(fields, {"host": " h ", "key": "k"}) + assert settings == {"host": "h", "port": 993, "mode": "a"} + assert secret == "k" + for bad in ( + {"host": "h", "key": "k", "port": "abc"}, + {"host": "h", "key": "k", "port": 0}, + {"host": "h", "key": "k", "mode": "c"}, + {"host": "h", "key": "k", "fremd": "x"}, + {"key": "k"}, + {"host": "h"}, + ): + with pytest.raises(Problem) as caught: + parse_fields(fields, bad) + assert caught.value.status == 422 + kept_settings, kept_secret = parse_fields(fields, {"host": "neu", "key": ""}, {"host": "alt", "port": 143}, "bleibt") + assert kept_settings["host"] == "neu" and kept_settings["port"] == 143 + assert kept_secret == "bleibt" + + +def test_capabilities_list_the_fields_of_each_module(client, db): + ada = _ada(client, db) + modules = {item["kind"]: item for item in ada.request("GET", "/api/v1/sources/capabilities")["modules"]} + assert {"paperless", "imap"} <= set(modules) + paperless = {item["key"]: item for item in modules["paperless"]["config_fields"]} + assert paperless["token"]["secret"] is True and paperless["base_url"]["required"] is True + imap = {item["key"]: item for item in modules["imap"]["config_fields"]} + assert imap["password"]["secret"] is True + assert imap["port"]["type"] == "number" and imap["port"]["default"] == 993 + assert [option["value"] for option in imap["security"]["options"]] == ["ssl", "starttls"] + assert imap["mailbox"]["default"] == "INBOX" + + +def test_create_with_config_and_flat_fields_gives_the_same_stored_result(client, db): + ada = _ada(client, db) + flat = ada.request("POST", "/api/v1/connectors", json=PAPERLESS, status=201) + config = ada.request( + "POST", + "/api/v1/connectors", + json={ + "kind": "paperless", + "name": "Zweites Archiv", + "config": {"base_url": "https://paper.example.test/", "token": "token-geheim-1234"}, + }, + status=201, + ) + assert _stored(db, flat["id"]) == _stored(db, config["id"]) + assert _stored(db, flat["id"])["settings"] == {"base_url": "https://paper.example.test"} + + +def test_secrets_never_leave_the_server(client, db): + ada = _ada(client, db) + created = ada.request("POST", "/api/v1/connectors", json=IMAP, status=201) + detail = ada.request("GET", f"/api/v1/connectors/{created['id']}") + listed = ada.request("GET", "/api/v1/connectors") + for body in (created, detail, listed): + text = str(body) + assert not any(word in text for word in SECRET_WORDS if word != "secret"), text + assert "'secret'" not in text and "'password'" not in text and "'token'" not in text + assert detail["has_secret"] is True + assert detail["settings"]["mailbox"] == "INBOX" + assert detail["settings"]["port"] == 993 + assert detail["protection_profile"] == "legacy-plaintext" + assert detail["references"] == 0 + assert detail["probe"] is None + + +def test_change_without_secret_keeps_the_stored_one_and_the_instance(client, db): + ada = _ada(client, db) + created = ada.request("POST", "/api/v1/connectors", json=IMAP, status=201) + changed = ada.request( + "PATCH", + f"/api/v1/connectors/{created['id']}", + json={"kind": "imap", "name": "Umbenannt", "config": {"host": "mail.example.test", "port": "143", "security": "starttls"}}, + ) + assert changed["id"] == created["id"] + assert changed["name"] == "Umbenannt" + stored = _stored(db, created["id"]) + assert stored["secret"] == "passwort-geheim" + assert stored["settings"]["host"] == "mail.example.test" + assert stored["settings"]["port"] == 143 + assert stored["settings"]["security"] == "starttls" + assert stored["settings"]["username"] == "ada" + renewed = ada.request( + "PATCH", + f"/api/v1/connectors/{created['id']}", + json={"kind": "imap", "name": "Umbenannt", "config": {"password": "neues-passwort"}}, + ) + assert renewed["id"] == created["id"] + assert _stored(db, created["id"])["secret"] == "neues-passwort" + + +def test_module_validation_refuses_bad_input(client, db): + ada = _ada(client, db) + ada.request("POST", "/api/v1/connectors", json={**PAPERLESS, "base_url": "ftp://paper.example.test"}, status=422) + ada.request("POST", "/api/v1/connectors", json={**PAPERLESS, "token": "kurz"}, status=422) + ada.request("POST", "/api/v1/connectors", json={**IMAP, "host": "mail server"}, status=422) + ada.request("POST", "/api/v1/connectors", json={**IMAP, "port": 70000}, status=422) + ada.request("POST", "/api/v1/connectors", json={**IMAP, "password": None}, status=422) + ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": "Fremdfeld", "config": {"base_url": "https://paper.example.test", "token": "token-geheim-1234", "host": "x"}}, + status=422, + ) + created = ada.request("POST", "/api/v1/connectors", json=PAPERLESS, status=201) + ada.request( + "PATCH", + f"/api/v1/connectors/{created['id']}", + json={**PAPERLESS, "kind": "imap"}, + status=422, + ) + assert _stored(db, created["id"])["settings"] == {"base_url": "https://paper.example.test"} + + +def test_a_foreign_instance_is_not_found(client, db): + ada = _ada(client, db) + ben = Actor(client, "ben-config@example.test", "Ben Konfig") + register(db, ben.email, ben.name) + created = ada.request("POST", "/api/v1/connectors", json=PAPERLESS, status=201) + ben.login() + ben.request("GET", f"/api/v1/connectors/{created['id']}", status=404) + ben.request("PATCH", f"/api/v1/connectors/{created['id']}", json=PAPERLESS, status=404) + ben.request("DELETE", f"/api/v1/connectors/{created['id']}", status=404) + + +def test_probe_result_is_kept_for_display(client, db, sample_source): + del sample_source + ada = _ada(client, db) + created = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "sample", "name": "Katalog", "config": {"catalog": "alpha"}}, + status=201, + ) + probed = ada.request("POST", f"/api/v1/connectors/{created['id']}/probe") + detail = ada.request("GET", f"/api/v1/connectors/{created['id']}") + assert detail["probe"]["ok"] is True + assert detail["probe"]["detail"] == probed["detail"] + ada.request( + "PATCH", + f"/api/v1/connectors/{created['id']}", + json={"kind": "sample", "name": "Katalog", "config": {"catalog": "alpha"}}, + ) + assert ada.request("GET", f"/api/v1/connectors/{created['id']}")["probe"] is None + + +def test_a_module_with_fields_needs_no_central_schema(client, db, sample_source): + del sample_source + ada = _ada(client, db) + modules = {item["kind"]: item for item in ada.request("GET", "/api/v1/sources/capabilities")["modules"]} + assert [item["key"] for item in modules["sample"]["config_fields"]] == ["catalog"] + ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "sample", "name": "Ohne Wert", "config": {}}, + status=201, + ) + ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "sample", "name": "Falsch", "config": {"catalog": "kein-katalog"}}, + status=422, + ) diff --git a/backend/tests/test_connector_followup.py b/backend/tests/test_connector_followup.py new file mode 100644 index 0000000..c536e15 --- /dev/null +++ b/backend/tests/test_connector_followup.py @@ -0,0 +1,770 @@ +"""Follow-up checks for paging, filters, storage resolution and notes. + +IMAP and Paperless doubles stand in for the external servers. They do not +prove a live mailbox or a live Paperless instance. +""" + +from __future__ import annotations + +import io +import zipfile +from email.parser import BytesParser + +from app.media.port import MemoryAdapter +from app.sourceaccess import borrow_original, complete_borrow +from app.sourceregistry import SourceModule, unregister_source, register_source +from tests.support import ORIGIN, Actor, document, register + +PDF = b"%PDF-1.4\n1 0 obj<<>>endobj\ntrailer<<>>\n%%EOF\nM\n" +LETTER = ( + b"From: Ada \r\n" + b"To: Ben \r\n" + b"Subject: Brief\r\n" + b"Message-ID: \r\n" + b"MIME-Version: 1.0\r\n" + b'Content-Type: multipart/mixed; boundary="grenze"\r\n' + b"\r\n" + b"--grenze\r\n" + b"Content-Type: text/plain; charset=utf-8\r\n" + b"\r\n" + b"Hallo aus der zweiten Quelle.\r\n" + b"--grenze\r\n" + b"Content-Type: application/pdf\r\n" + b'Content-Disposition: attachment; filename="plan.pdf"\r\n' + b"\r\n" + b"%PDF-plan\r\n" + b"--grenze--\r\n" +) + + +def _headers(actor: Actor) -> dict: + return { + "Origin": ORIGIN, + "X-CSRF-Token": actor.csrf, + "X-Scope-Kind": actor.scope["kind"], + "X-Scope-Id": actor.scope["id"], + } + + +def _install_letterbox() -> None: + def parse_config(payload: dict): + name = str(payload.get("name") or "Briefkasten").strip() or "Briefkasten" + return name, {"catalog": "letters"}, "letter-secret" + + def probe(row): + return {"ok": True, "detail": "Der Briefkasten antwortet."} + + def find(conn, owner_id, connector_id, query, options): + del conn, owner_id, query, options + return {"items": [{"id": "", "title": "Brief"}], "connector_id": str(connector_id)} + + def open_original(conn, actor_id, element, context, ref): + borrowed = borrow_original(conn, actor_id, element, context, ref, "letterbox") + return complete_borrow(conn, borrowed, lambda row: (LETTER, "message/rfc822", "nachricht.eml")) + + def export_original(conn, actor_id, external_id, detail, assigned, ref): + del conn, actor_id, external_id, detail, assigned, ref + from app.retrieval import FilePayload + + return FilePayload("nachricht.eml", "message/rfc822", LETTER, "") + + register_source(SourceModule( + kind="letterbox", + label="Briefkasten", + representation="email", + capabilities={ + "kind": "letterbox", + "navigate": False, + "read_original": True, + "read_preview": False, + "read_attachment": True, + "edit_content": False, + "edit_metadata": False, + "search": {"query": True, "continuation": False}, + }, + parse_config=parse_config, + probe=probe, + find=find, + open_original=open_original, + export_original=export_original, + )) + + +class RangeIMAP: + searches: list = [] + validity = b"3" + uidnext = b"50001" + fail_headers = False + search_status = "OK" + parts = False + + def __init__(self, *args, **kwargs): + self.untagged_responses = {} + + def login(self, username, secret): + assert secret == "Postfach-Geheim" + return "OK", [b"logged"] + + def list(self): + return "OK", [b'(\\HasNoChildren) "/" "INBOX"'] + + def select(self, mailbox, readonly=False): + assert readonly is True + self.untagged_responses = {"UIDVALIDITY": [type(self).validity], "UIDNEXT": [type(self).uidnext]} + return "OK", [b"1"] + + def uid(self, command, *args): + if command == "SEARCH": + type(self).searches.append(args) + if type(self).search_status != "OK": + return type(self).search_status, [None] + if "*" in args and not any(isinstance(arg, str) and ":" in arg for arg in args): + return "OK", [b"50000"] + low, high = _range_of(args) + if type(self).parts and low <= 5 <= high: + return "OK", [b"4 5", b"3"] + needle = _needle(args) + if needle: + hit = 80 if low <= 80 <= high else None + if needle.startswith("=?") or "nderung" in needle or "Änderung" in needle: + hit = 80 if low <= 80 <= high else None + return ("OK", [str(hit).encode()]) if hit else ("OK", [b""]) + found = [str(uid).encode() for uid in range(low, high + 1)] + return "OK", [b" ".join(found)] + uid = int(args[0]) + if type(self).fail_headers: + return "NO", [None] + subject = "Seltener Treffer" if uid == 80 else f"Nachricht {uid}" + header = ( + f"From: Ada \r\nSubject: {subject}\r\n" + f"Date: Thu, 02 May 2024 08:00:00 +0000\r\n" + ).encode() + return "OK", [(b"body", header), b")"] + + def logout(self): + return "BYE", [b""] + + +def _range_of(args) -> tuple[int, int]: + for arg in args: + text = arg.decode() if isinstance(arg, bytes) else str(arg) + if ":" in text and text.replace(":", "").isdigit(): + low, high = text.split(":", 1) + return int(low), int(high) + raise AssertionError(args) + + +def _needle(args) -> str: + for arg in args: + text = arg.decode() if isinstance(arg, bytes) else str(arg) + if text.startswith('"') or "nderung" in text or "Änderung" in text: + return text + return "" + + +def _account(monkeypatch, actor: Actor, fake) -> str: + import app.sources as sources + + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", fake) + created = actor.request( + "POST", + "/api/v1/connectors", + json={ + "kind": "imap", + "name": "Kanzlei", + "host": "imap.example.test", + "port": 993, + "username": "ada@example.test", + "password": "Postfach-Geheim", + "security": "ssl", + }, + status=201, + ) + return created["id"] + + +def _spans(searches) -> list[int]: + spans = [] + for args in searches: + if any(arg == "*" for arg in args): + continue + low, high = _range_of(args) + spans.append(high - low + 1) + return spans + + +def test_large_folder_is_bounded_and_a_late_hit_is_not_empty(client, db, monkeypatch): + import app.sources as sources + + ada = Actor(client, "ada-range@example.test", "Ada Bereich") + register(db, ada.email, ada.name) + ada.login() + RangeIMAP.searches = [] + RangeIMAP.validity = b"3" + RangeIMAP.uidnext = b"50001" + RangeIMAP.fail_headers = False + RangeIMAP.search_status = "OK" + RangeIMAP.parts = False + account = _account(monkeypatch, ada, RangeIMAP) + first = ada.request("GET", "/api/v1/sources/imap/messages", params={"connector_id": account, "mailbox": "INBOX"}) + assert first["items"][0]["title"] == "Nachricht 50000" + assert len(first["items"]) == sources.PAGE_SIZE + assert first["incomplete"] is False + assert first["next_before"] is not None + assert _spans(RangeIMAP.searches) + assert max(_spans(RangeIMAP.searches)) <= sources.UID_WINDOW + assert all("ALL" not in args for args in RangeIMAP.searches) + RangeIMAP.searches = [] + page = first + seen = {item["uid"] for item in page["items"]} + hops = 0 + while 80 not in seen: + hops += 1 + assert hops < 40 + assert page["next_before"] or page["incomplete"] + page = ada.request( + "GET", + "/api/v1/sources/imap/messages", + params={"connector_id": account, "mailbox": "INBOX", "q": "Seltener", "before": page["next_before"]}, + ) + if not page["items"] and page["incomplete"]: + assert page["next_before"] + seen.update(item["uid"] for item in page["items"]) + assert any(item["title"] == "Seltener Treffer" for item in page["items"]) + assert max(_spans(RangeIMAP.searches)) <= sources.UID_WINDOW + + +def test_imap_errors_are_not_reported_as_an_empty_folder(client, db, monkeypatch): + ada = Actor(client, "ada-imap-state@example.test", "Ada Zustand") + register(db, ada.email, ada.name) + ada.login() + RangeIMAP.searches = [] + RangeIMAP.uidnext = b"1" + RangeIMAP.fail_headers = False + RangeIMAP.search_status = "OK" + RangeIMAP.parts = False + account = _account(monkeypatch, ada, RangeIMAP) + empty = ada.request("GET", "/api/v1/sources/imap/messages", params={"connector_id": account, "mailbox": "INBOX"}) + assert empty["items"] == [] + assert empty["incomplete"] is False + RangeIMAP.uidnext = b"31" + RangeIMAP.search_status = "NO" + refused = ada.request( + "GET", + "/api/v1/sources/imap/messages", + params={"connector_id": account, "mailbox": "INBOX"}, + status=422, + ) + assert "abgelehnt" in refused["detail"] + assert "leer" not in refused["detail"].casefold() + RangeIMAP.search_status = "OK" + RangeIMAP.fail_headers = True + failed = ada.request( + "GET", + "/api/v1/sources/imap/messages", + params={"connector_id": account, "mailbox": "INBOX"}, + status=422, + ) + assert "Nachrichtenköpfe" in failed["detail"] + assert "leer" not in failed["detail"].casefold() + + +def test_uidvalidity_change_restarts_and_split_responses_join(client, db, monkeypatch): + ada = Actor(client, "ada-validity@example.test", "Ada Geltung") + register(db, ada.email, ada.name) + ada.login() + RangeIMAP.searches = [] + RangeIMAP.validity = b"4" + RangeIMAP.uidnext = b"11" + RangeIMAP.fail_headers = False + RangeIMAP.search_status = "OK" + RangeIMAP.parts = False + account = _account(monkeypatch, ada, RangeIMAP) + first = ada.request("GET", "/api/v1/sources/imap/messages", params={"connector_id": account, "mailbox": "INBOX"}) + RangeIMAP.validity = b"9" + restarted = ada.request( + "GET", + "/api/v1/sources/imap/messages", + params={ + "connector_id": account, + "mailbox": "INBOX", + **({"before": first["next_before"]} if first["next_before"] else {}), + "uidvalidity": first["uidvalidity"], + }, + ) + assert restarted["restarted"] is True + assert restarted["items"][0]["title"] == "Nachricht 10" + RangeIMAP.validity = b"4" + RangeIMAP.parts = True + RangeIMAP.uidnext = b"6" + joined = ada.request("GET", "/api/v1/sources/imap/messages", params={"connector_id": account, "mailbox": "INBOX"}) + assert [item["uid"] for item in joined["items"]] == [5, 4, 3] + + +def test_paperless_filters_reach_the_api_and_cross_pages(client, db, monkeypatch): + import app.sources as sources + + calls = [] + + def fake_get(base, token, path, params): + calls.append({"path": path, "params": dict(params), "token": token}) + if path != "/api/documents/": + return {"count": 1, "results": [{"id": 4, "name": "Haus"}]} + page = int(params.get("page") or 1) + start = (page - 1) * 20 + return { + "count": 45, + "results": [ + {"id": number + 1, "title": f"Titel {number + 1}", "created": "2024-03-01"} + for number in range(start, min(start + 20, 45)) + ], + } + + def fake_bytes(row, document_id): + return f"%PDF-{row['secret']}-{document_id}\n".encode(), "application/pdf", "dokument.pdf" + + monkeypatch.setattr(sources, "_paperless_get", fake_get) + monkeypatch.setattr(sources, "_paperless_bytes", fake_bytes) + ada = Actor(client, "ada-filter@example.test", "Ada Filter") + register(db, ada.email, ada.name) + ada.login() + first = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"}, + status=201, + ) + second = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": "Zweitarchiv", "base_url": "https://paper.example.test", "token": "token-bravo"}, + status=201, + ) + facets = ada.request("GET", "/api/v1/sources/paperless/facets", params={"connector_id": first["id"]}) + assert facets["tag_mode"] == "all" + assert facets["correspondents"] == [{"id": "4", "name": "Haus"}] + ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": first["id"], "document_from": "2024-05-02", "document_to": "2024-01-01"}, + status=422, + ) + found = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={ + "connector_id": first["id"], + "q": "nurinhalt", + "title": "Vertrag", + "correspondent": "4", + "document_type": "8", + "tag": "2,5", + "document_from": "2024-01-01", + "document_to": "2024-12-31", + "sort": "-created", + "cursor": "3", + }, + ) + sent = calls[-1]["params"] + assert sent["query"] == "nurinhalt" + assert sent["title__icontains"] == "Vertrag" + assert sent["correspondent__id"] == "4" + assert sent["document_type__id"] == "8" + assert sent["tags__id__all"] == "2,5" + assert sent["created__date__gte"] == "2024-01-01" + assert sent["created__date__lte"] == "2024-12-31" + assert sent["ordering"] == "-created" + assert sent["page"] == "3" + ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": first["id"], "q": "nurinhalt", "sort": "-score"}, + ) + assert "ordering" not in calls[-1]["params"] + assert found["items"][0]["id"] == "41" + assert found["connector_id"] == first["id"] + preview = ada.client.get( + "/api/v1/sources/paperless/documents/41/preview", + params={"connector_id": first["id"]}, + headers=_headers(ada), + ) + assert preview.status_code == 200 + assert preview.content.startswith(b"%PDF-token-alpha-41") + matter = ada.request("POST", "/api/v1/matters", json={"title": "Suche"}, status=201) + linked = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Seite drei", + "source": "paperless", + "external_id": "41", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": first["id"]}, + }, + status=201, + ) + other = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Dieselbe Nummer", + "source": "paperless", + "external_id": "41", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": second["id"]}, + }, + status=201, + ) + opened = ada.client.get(f"/api/v1/external-refs/{linked['id']}/content", headers=_headers(ada)) + twin = ada.client.get(f"/api/v1/external-refs/{other['id']}/content", headers=_headers(ada)) + assert opened.content.startswith(b"%PDF-token-alpha-41") + assert twin.content.startswith(b"%PDF-token-bravo-41") + ben = Actor(client, "ben-filter@example.test", "Ben Filter") + register(db, ben.email, ben.name) + ben.login() + ben.request("GET", "/api/v1/sources/paperless/facets", params={"connector_id": first["id"]}, status=404) + ben.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": first["id"], "q": "nurinhalt"}, + status=404, + ) + + +def test_revoked_grant_hides_bytes_fetched_during_the_call(client, db, monkeypatch): + import app.sources as sources + + def fake_bytes(row, document_id): + db.execute("UPDATE access_grants SET can_read = false") + db.commit() + return b"%PDF-1.4\nsecret\n", "application/pdf", "geheim.pdf" + + monkeypatch.setattr(sources, "_paperless_bytes", fake_bytes) + ada = Actor(client, "ada-revoke@example.test", "Ada Widerruf") + ben = Actor(client, "ben-revoke@example.test", "Ben Widerruf") + register(db, ada.email, ada.name) + register(db, ben.email, ben.name) + ada.login() + paper = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"}, + status=201, + ) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Freigabe"}, status=201) + linked = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Vertrag", + "source": "paperless", + "external_id": "15", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": paper["id"]}, + }, + status=201, + ) + ada.request( + "POST", + f"/api/v1/matters/{matter['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False, "use_owner_sources": True}, + status=201, + ) + ben.login() + hidden = ben.client.get( + f"/api/v1/external-refs/{linked['id']}/content", + params={"context_kind": "matter", "context_id": matter["id"]}, + headers=_headers(ben), + ) + assert hidden.status_code == 422 + assert b"secret" not in hidden.content + + +def test_email_representation_uses_the_mail_reader_without_imap(client, db): + ada = Actor(client, "ada-letter@example.test", "Ada Brief") + register(db, ada.email, ada.name) + ada.login() + _install_letterbox() + try: + box = ada.request("POST", "/api/v1/connectors", json={"kind": "letterbox", "name": "Briefe"}, status=201) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Post"}, status=201) + linked = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Brief", + "source": "letterbox", + "external_id": "", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": box["id"]}, + }, + status=201, + ) + mail = ada.request("GET", f"/api/v1/external-refs/{linked['id']}/mail") + assert mail["attachments"][0]["filename"] == "plan.pdf" + attachment = ada.client.get( + f"/api/v1/external-refs/{linked['id']}/mail/attachments/0", + headers=_headers(ada), + ) + assert attachment.status_code == 200 + assert attachment.content.startswith(b"%PDF-plan") + raw = ada.client.get(f"/api/v1/external-refs/{linked['id']}/mail/original", headers=_headers(ada)) + parsed = BytesParser().parsebytes(raw.content) + assert parsed.get("subject") == "Brief" + finally: + unregister_source("letterbox") + + +def test_domain_storage_uses_the_resolved_adapter(client, db, monkeypatch): + import app.media.port as port + + memory = MemoryAdapter() + monkeypatch.setitem(port._FACTORIES, "local", lambda row: memory) + ada = Actor(client, "ada-store@example.test", "Ada Speicher") + register(db, ada.email, ada.name) + ada.login() + matter = ada.request("POST", "/api/v1/matters", json={"title": "Ablage"}, status=201) + ctx = {"kind": "matter", "id": matter["id"]} + note = ada.request( + "POST", + "/api/v1/notes", + json={"title": "Schreiben", "document": document("Text"), "context": ctx}, + status=201, + ) + uploaded = ada.request( + "POST", + "/api/v1/files", + data={"title": "Plan", "context_kind": "matter", "context_id": matter["id"], "document_on": "2024-01-02"}, + files={"upload": ("plan.pdf", PDF, "application/pdf")}, + status=201, + ) + detail = ada.request("GET", f"/api/v1/matters/{matter['id']}") + host = next(item for item in detail["bindings"] if item["element"]["id"] == note["id"]) + ada.request( + "POST", + f"/api/v1/elements/{note['id']}/attachments", + json={ + "expected_version": host["element"]["version"], + "context": ctx, + "attachment_element_ids": [uploaded["id"]], + "fold": "attachment", + }, + status=201, + ) + folded = ada.request("GET", f"/api/v1/matters/{matter['id']}") + host = next(item for item in folded["bindings"] if item["element"]["id"] == note["id"]) + dispatch = ada.request( + "POST", + f"/api/v1/elements/{note['id']}/dispatches", + json={ + "expected_version": host["element"]["version"], + "context": ctx, + "channel": "post", + "sent_on": "2024-02-02", + "attachment_element_ids": [uploaded["id"]], + "capture_editions": True, + }, + status=201, + ) + content = ada.client.get(f"/api/v1/files/{uploaded['id']}/content", headers=_headers(ada)) + assert content.content == PDF + item = next(entry for entry in dispatch["items"] if entry["element_id"] == uploaded["id"]) + snapshot = ada.client.get( + f"/api/v1/dispatches/{dispatch['id']}/items/{item['id']}/content", + params={"context_kind": "matter", "context_id": matter["id"]}, + headers=_headers(ada), + ) + assert snapshot.content == PDF + asset = db.execute( + "SELECT object_key, storage_generation, sha256 FROM media_assets WHERE element_id = %s", + (uploaded["id"],), + ).fetchone() + assert memory.open(asset["object_key"]) == PDF + tree = ada.request("GET", f"/api/v1/matters/{matter['id']}/export-tree")["tree"] + selected = [ + node["id"] + for node in _walk(tree) + if node.get("exportable") and node.get("kind") in {"binding", "attachment", "dispatch_item"} + ] + package = ada.client.post( + f"/api/v1/matters/{matter['id']}/export", + json={"context": ctx, "selected": selected}, + headers={**_headers(ada), "Content-Type": "application/json"}, + ) + assert package.status_code == 200, package.text + archive = zipfile.ZipFile(io.BytesIO(package.content)) + assert any(archive.read(name) == PDF for name in archive.namelist()) + version = next( + item["element"]["version"] + for item in ada.request("GET", f"/api/v1/matters/{matter['id']}")["bindings"] + if item["element"]["id"] == uploaded["id"] + ) + ada.request( + "POST", + f"/api/v1/elements/{uploaded['id']}/delete", + json={"expected_version": version}, + status=204, + ) + try: + memory.open(asset["object_key"]) + except Exception: + removed = True + else: + removed = False + assert removed is True + assert asset["storage_generation"] + + +def test_note_markdown_is_shared_exported_and_kept_on_conflict(client, db): + ada = Actor(client, "ada-note-md@example.test", "Ada Notiz") + ben = Actor(client, "ben-note-md@example.test", "Ben Notiz") + register(db, ada.email, ada.name) + register(db, ben.email, ben.name) + ada.login() + left = ada.request("POST", "/api/v1/matters", json={"title": "Links"}, status=201) + right = ada.request("POST", "/api/v1/matters", json={"title": "Rechts"}, status=201) + body = "\n".join([ + "# Überschrift", + "", + "Ein **fetter** und *kursiver* Satz.", + "", + "- erster", + "- zweiter", + "", + "> Zitat", + "", + "Siehe [Quelle](https://example.test/hinweis).", + "", + "`code`", + "", + "```", + "block", + "```", + "", + "", + "", + "![bild](https://example.test/bild.png)", + ]) + created = ada.request( + "POST", + "/api/v1/notes", + json={ + "title": "Merkzettel", + "document": {"version": 1, "blocks": [{"type": "markdown", "text": body}]}, + "context": {"kind": "matter", "id": left["id"]}, + }, + status=201, + ) + ada.request( + "POST", + "/api/v1/bindings", + json={ + "element_id": created["id"], + "source_context": {"kind": "matter", "id": left["id"]}, + "target": {"kind": "matter", "id": right["id"]}, + }, + status=201, + ) + detail = ada.request("GET", f"/api/v1/matters/{left['id']}") + element = next(item["element"] for item in detail["bindings"] if item["element"]["id"] == created["id"]) + changed = "Geändert in beiden Vorgängen.\n\n" + body + ada.request( + "PATCH", + f"/api/v1/notes/{created['id']}", + json={ + "expected_version": element["version"], + "document": {"version": 1, "blocks": [{"type": "markdown", "text": changed}]}, + "context": {"kind": "matter", "id": right["id"]}, + }, + ) + again = ada.request("GET", f"/api/v1/matters/{left['id']}") + shared = next(item["element"] for item in again["bindings"] if item["element"]["id"] == created["id"]) + assert changed in _text(shared) + ada.request( + "POST", + f"/api/v1/bindings/{next(item['id'] for item in again['bindings'] if item['element']['id'] == created['id'])}/annotations", + json={"body": "Nur links"}, + params={"context_kind": "matter", "context_id": left["id"]}, + status=201, + ) + left_detail = ada.request("GET", f"/api/v1/matters/{left['id']}") + right_detail = ada.request("GET", f"/api/v1/matters/{right['id']}") + left_binding = next(item for item in left_detail["bindings"] if item["element"]["id"] == created["id"]) + right_binding = next(item for item in right_detail["bindings"] if item["element"]["id"] == created["id"]) + assert any(note["body"] == "Nur links" for note in left_binding["annotations"]) + assert all(note["body"] != "Nur links" for note in right_binding["annotations"]) + ada.request( + "POST", + f"/api/v1/matters/{left['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False}, + status=201, + ) + ben.login() + ben.request( + "PATCH", + f"/api/v1/notes/{created['id']}", + json={ + "expected_version": shared["version"], + "document": {"version": 1, "blocks": [{"type": "markdown", "text": "fremd"}]}, + "context": {"kind": "matter", "id": left["id"]}, + }, + status=403, + ) + ada.login() + current = next( + item["element"]["version"] + for item in ada.request("GET", f"/api/v1/matters/{left['id']}")["bindings"] + if item["element"]["id"] == created["id"] + ) + conflict = ada.request( + "PATCH", + f"/api/v1/notes/{created['id']}", + json={ + "expected_version": current - 1, + "document": {"version": 1, "blocks": [{"type": "markdown", "text": "darf nicht überschreiben"}]}, + "context": {"kind": "matter", "id": left["id"]}, + }, + status=409, + ) + assert conflict["code"] == "conflict" + stored = next( + item["element"] + for item in ada.request("GET", f"/api/v1/matters/{left['id']}")["bindings"] + if item["element"]["id"] == created["id"] + ) + assert "darf nicht überschreiben" not in _text(stored) + assert "" in _text(stored) + tree = ada.request("GET", f"/api/v1/matters/{left['id']}/export-tree")["tree"] + selected = [node["id"] for node in _walk(tree) if node.get("kind") == "binding" and node.get("exportable")] + package = ada.client.post( + f"/api/v1/matters/{left['id']}/export", + json={"context": {"kind": "matter", "id": left["id"]}, "selected": selected}, + headers={**_headers(ada), "Content-Type": "application/json"}, + ) + assert package.status_code == 200, package.text + archive = zipfile.ZipFile(io.BytesIO(package.content)) + exported = "\n".join( + archive.read(name).decode("utf-8", errors="replace") + for name in archive.namelist() + if name.endswith((".md", ".txt")) + ) + assert "```" in exported + assert "" in exported + assert "https://example.test/bild.png" in exported + + +def _text(element: dict) -> str: + return "\n".join(block.get("text") or "" for block in (element.get("document") or {}).get("blocks") or []) + + +def _walk(node): + if isinstance(node, dict) and "kind" in node: + yield node + for child in node.get("children") or []: + yield from _walk(child) + elif isinstance(node, dict): + for value in node.values(): + yield from _walk(value) + elif isinstance(node, list): + for item in node: + yield from _walk(item) diff --git a/backend/tests/test_correction_k1_k7.py b/backend/tests/test_correction_k1_k7.py new file mode 100644 index 0000000..6601057 --- /dev/null +++ b/backend/tests/test_correction_k1_k7.py @@ -0,0 +1,503 @@ +"""Corrections K1–K3 and K7. Synthetic only; no live mailbox or Paperless.""" + +from __future__ import annotations + +import hashlib +import imaplib + +import psycopg +import pytest +from email.message import EmailMessage + +import app.sources as sources +from app.core.errors import unprocessable +from app.mailpack import parse_mail +from tests.support import Actor, register + + +def test_mailbox_names_are_quoted_protocol_strings(): + client = imaplib.IMAP4.__new__(imaplib.IMAP4) + client.sent = [] + client.state = "AUTH" + client._encoding = "ascii" + client.tagpre = b"T" + client.tagnum = 1 + client.tagged_commands = {} + client.untagged_responses = {} + client.is_readonly = True + client.literal = None + client.debug = 0 + client._mesg = lambda *args: None + client._log = lambda *args: None + client.send = lambda data: client.sent.append(data) + + value = "Sent Items" + client._command("EXAMINE", sources._mailbox_argument(value)) + assert client.sent[-1] == b'T1 EXAMINE "Sent Items"\r\n' + value = 'Ordner "A"\\B' + client._command("EXAMINE", sources._mailbox_argument(value)) + assert client.sent[-1] == b'T2 EXAMINE "Ordner \\"A\\"\\\\B"\r\n' + assert sources._mailbox_argument("INBOX") == "INBOX" + with pytest.raises(Exception): + sources._mailbox_argument("Sent\nItems") + + +def test_parse_mail_keeps_names_html_and_attached_messages(): + inner = EmailMessage() + inner["Subject"] = "Innen" + inner.set_content("innen bleibt drin") + inner.add_attachment(b"innen-pdf", maintype="application", subtype="pdf", filename="innen.pdf") + outer = EmailMessage() + outer["From"] = "Ada " + outer["To"] = "Ben " + outer["Subject"] = "=?utf-8?q?Aussen?=" + outer.set_content("aussen bleibt aussen") + outer.add_attachment(b"%PDF-same", maintype="application", subtype="pdf", filename="lage.pdf") + outer.add_attachment(b"%PDF-other", maintype="application", subtype="pdf", filename="lage.pdf") + outer.add_attachment(inner, subtype="rfc822", filename="weitergeleitet.eml") + html = ( + "

A & B

ZelleZwei
" + "" + "x" + ) + mixed = EmailMessage() + mixed["Subject"] = "HTML" + mixed.set_content("nur text") + mixed.add_alternative(html, subtype="html") + logo = b"\x89PNG\r\n" + mixed.add_attachment(logo, maintype="image", subtype="png", filename="logo.png", cid="") + encoded = EmailMessage() + encoded["Subject"] = "Namen" + encoded.set_content("datei") + encoded.add_attachment( + b"pdf", + maintype="application", + subtype="pdf", + filename="=?utf-8?q?Buero_Uebergabe=2Epdf?=", + ) + + parsed = parse_mail(outer.as_bytes()) + assert parsed.subject == "Aussen" + assert parsed.text == "aussen bleibt aussen" + assert "innen" not in parsed.text + names = [item.display_name for item in parsed.attachments] + assert "weitergeleitet.eml" in names + assert names.count("lage.pdf") == 2 + archives = [item.filename for item in parsed.attachments] + assert len(set(archives)) == len(archives) + attached = next(item for item in parsed.attachments if item.display_name == "weitergeleitet.eml") + assert b"innen bleibt drin" in attached.data or b"Innen" in attached.data + assert attached.sha256 == hashlib.sha256(attached.data).hexdigest() + + view = parse_mail(mixed.as_bytes()) + assert "" in view.html + assert ">\nendobj\ntrailer\n<<>>\n%%EOF\n" + raw = ( + b"Subject: Scan\r\n" + b"MIME-Version: 1.0\r\n" + b"Content-Type: multipart/mixed; boundary=bound\r\n" + b"\r\n" + b"--bound\r\n" + b"Content-Type: text/plain; charset=utf-8\r\n" + b"\r\n" + b"text\r\n" + b"--bound\r\n" + b"Content-Type: application/octet-stream\r\n" + b"Content-Transfer-Encoding: 8bit\r\n" + b"Content-Disposition: attachment; filename=\"scan-7f3a9c\"\r\n" + b"\r\n" + + base64.b64encode(pdf) + b"\r\n" + b"--bound--\r\n" + ) + item = parse_mail(raw).attachments[0] + assert item.data.startswith(b"%PDF") + assert item.media_type == "application/pdf" + assert item.display_name == "scan-7f3a9c.pdf" + header = content_disposition("\u00dcbergabe.pdf", inline=True) + assert header.startswith("inline;") + header.encode("ascii") + assert ".pdf" in header + assert "filename*=UTF-8''" in header + assert "%C3%9Cbergabe.pdf" in header + + +class QuotedIMAP: + def __init__(self, *args, **kwargs): + self.untagged_responses = {} + + def login(self, username, secret): + return "OK", [b"logged"] + + def list(self): + return "OK", [b'(\\HasNoChildren) "/" "Sent Items"'] + + def select(self, mailbox, readonly=False): + assert readonly is True + assert mailbox == '"Sent Items"' + self.untagged_responses = { + "EXISTS": [b"4200"], + "UIDVALIDITY": [b"8"], + "UIDNEXT": [b"2"], + } + return "OK", [b"4200"] + + def uid(self, command, *args): + if command == "SEARCH": + return "OK", [b""] + header = b"From: Ada \r\nSubject: Leer\r\nDate: Thu, 02 May 2024 08:00:00 +0000\r\n" + return "OK", [(b"body", header), b")"] + + def logout(self): + return "BYE", [b""] + + +def test_a_filled_folder_with_spaces_is_not_reported_empty(client, db, monkeypatch): + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", QuotedIMAP) + ada = Actor(client, "ada-sent@example.test", "Ada Gesendet") + register(db, ada.email, ada.name) + ada.login() + account = ada.request( + "POST", + "/api/v1/connectors", + json={ + "kind": "imap", + "name": "Post", + "host": "imap.example.test", + "port": 993, + "username": "ada@example.test", + "password": "Postfach-Geheim", + "security": "ssl", + }, + status=201, + ) + page = ada.request( + "GET", + "/api/v1/sources/imap/messages", + params={"connector_id": account["id"], "mailbox": "Sent Items"}, + ) + assert page["items"] == [] + assert page["contradictory"] is True + assert page["exists"] == 4200 + assert "leer" not in str(page).casefold() + + +def _multipart(text: str, name: str, payload: bytes) -> bytes: + message = EmailMessage() + message["From"] = "Ada " + message["To"] = "Ben " + message["Subject"] = "Grenze" + message["Message-ID"] = "" + message.set_content(text) + message.add_attachment(payload, maintype="application", subtype="pdf", filename=name) + return message.as_bytes() + + +class LimitIMAP: + raw = b"" + + def __init__(self, *args, **kwargs): + self.untagged_responses = {} + + def login(self, username, secret): + return "OK", [b"ok"] + + def list(self): + return "OK", [b'(\\HasNoChildren) "/" "INBOX"'] + + def select(self, mailbox, readonly=False): + self.untagged_responses = {"UIDVALIDITY": [b"1"], "UIDNEXT": [b"3"], "EXISTS": [b"1"]} + return "OK", [b"1"] + + def uid(self, command, *args): + spec = " ".join(str(part) for part in args[1:]) + if "BODY.PEEK[]" in spec and "<" not in spec: + return "OK", [(b"rfc822", type(self).raw)] + if "BODYSTRUCTURE" in spec: + structure = ( + b'(("TEXT" "PLAIN" ("CHARSET" "utf-8") NIL NIL "7BIT" 12 1 NIL NIL NIL NIL)' + b'("APPLICATION" "PDF" NIL NIL NIL "BASE64" 20 NIL ' + b'("ATTACHMENT" ("FILENAME" "=?utf-8?q?Buero_Uebergabe=2Epdf?=")) NIL NIL)' + b' "MIXED" ("BOUNDARY" "b") NIL NIL NIL)' + ) + return "OK", [(b"body", structure)] + if "BODY.PEEK[1]" in spec: + return "OK", [(b"part", b"gleicher text")] + if "BODY.PEEK[2]" in spec: + return "OK", [(b"part", b"nicht text")] + header = ( + b"From: Ada \r\nTo: Ben \r\n" + b"Subject: Grenze\r\nDate: Thu, 02 May 2024 08:00:00 +0000\r\n" + ) + return "OK", [(b"body", header), b")"] + + def logout(self): + return "BYE", [b""] + + +def test_preview_below_and_above_the_limit_uses_the_same_parts(client, db, monkeypatch): + payload = b"%PDF-plan" + small = _multipart("gleicher text", "Buero Uebergabe.pdf", payload) + LimitIMAP.raw = small + b"X" * 50 + monkeypatch.setattr(sources, "PREVIEW_BYTE_LIMIT", len(small)) + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", LimitIMAP) + ada = Actor(client, "ada-limit@example.test", "Ada Grenze") + register(db, ada.email, ada.name) + ada.login() + account = ada.request( + "POST", + "/api/v1/connectors", + json={ + "kind": "imap", + "name": "Post", + "host": "imap.example.test", + "port": 993, + "username": "ada@example.test", + "password": "Postfach-Geheim", + "security": "ssl", + }, + status=201, + ) + under = ada.request( + "GET", + "/api/v1/sources/imap/message", + params={"connector_id": account["id"], "mailbox": "INBOX", "uid": 1}, + ) + LimitIMAP.raw = small + b"Y" + over = ada.request( + "GET", + "/api/v1/sources/imap/message", + params={"connector_id": account["id"], "mailbox": "INBOX", "uid": 1}, + ) + assert "gleicher text" in under["text"] + assert "gleicher text" in over["text"] + assert under["attachments"][0]["filename"] == "Buero Uebergabe.pdf" + assert over["attachments"][0]["filename"] == "Buero Uebergabe.pdf" + assert "Content-Transfer-Encoding" not in over["text"] + assert "BASE64" not in over["text"].upper() + assert over["preview_limited"] is True + + +def test_upload_and_mail_fetch_leave_the_domain_transaction(client, db, monkeypatch): + seen = [] + + import app.api.routes as routes + import app.ops as ops + from app.media.port import publish as real_publish + + def watching(adapter, key, data): + seen.append(("publish", ops_conn[0].info.transaction_status)) + return real_publish(adapter, key, data) + + ops_conn = [None] + real_create = ops.create_file + + def wrapped_create(conn, *args, **kwargs): + ops_conn[0] = conn + return real_create(conn, *args, **kwargs) + + monkeypatch.setattr(ops, "publish", watching) + monkeypatch.setattr(routes, "create_file", wrapped_create) + ada = Actor(client, "ada-idle@example.test", "Ada Leerlauf") + register(db, ada.email, ada.name) + ada.login() + case = ada.request("POST", "/api/v1/cases", json={"title": "Ablage"}, status=201) + ada.request( + "POST", + "/api/v1/files", + data={"title": "Scan", "context_kind": "case", "context_id": case["id"]}, + files={"upload": ("scan.pdf", b"%PDF-1.4\n", "application/pdf")}, + status=201, + ) + assert seen[0][1] == psycopg.pq.TransactionStatus.IDLE + + import app.sourceaccess as sourceaccess + + phase = {"external": False, "status": None} + real_release = sourceaccess.release_for_external + + def spy_release(conn): + real_release(conn) + phase["external"] = True + phase["status"] = conn.info.transaction_status + + def spy_owned(*args, **kwargs): + if phase["external"]: + raise AssertionError("Datenbankzugriff im externen Abschnitt") + return real_owned(*args, **kwargs) + + def spy_pull(*args, **kwargs): + assert phase["status"] == psycopg.pq.TransactionStatus.IDLE + return _multipart("brief", "a.pdf", b"%PDF") + + real_owned = sources._owned + monkeypatch.setattr(sourceaccess, "release_for_external", spy_release) + monkeypatch.setattr(sources, "_owned", spy_owned) + monkeypatch.setattr(sources, "pull_rfc822", spy_pull) + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", QuotedIMAP) + account = ada.request( + "POST", + "/api/v1/connectors", + json={ + "kind": "imap", + "name": "Post", + "host": "imap.example.test", + "port": 993, + "username": "ada@example.test", + "password": "Postfach-Geheim", + "security": "ssl", + }, + status=201, + ) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Post"}, status=201) + linked = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Brief", + "source": "imap", + "external_id": "", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": account["id"], "mailbox": "Sent Items", "uid": 4}, + }, + status=201, + ) + opened = ada.request("GET", f"/api/v1/external-refs/{linked['id']}/mail") + assert opened["text"] + assert phase["status"] == psycopg.pq.TransactionStatus.IDLE + + +def test_failed_conclusion_keeps_no_partial_file(client, db, monkeypatch): + import app.ops as ops + + def fail(*args, **kwargs): + raise unprocessable("Abschluss fehlgeschlagen") + + monkeypatch.setattr(ops, "_new_element", fail) + ada = Actor(client, "ada-fail@example.test", "Ada Abschluss") + register(db, ada.email, ada.name) + ada.login() + case = ada.request("POST", "/api/v1/cases", json={"title": "Ablage"}, status=201) + ada.request( + "POST", + "/api/v1/files", + data={"title": "Scan", "context_kind": "case", "context_id": case["id"]}, + files={"upload": ("scan.pdf", b"%PDF-FAIL-CONCLUSION\n", "application/pdf")}, + status=422, + ) + stored = db.execute("SELECT count(*) AS n FROM elements WHERE title = 'Scan'").fetchone() + assert stored["n"] == 0 + from pathlib import Path + from app.core.settings import get_settings + + root = Path(get_settings().media_root) + left = [path for path in root.rglob("*") if path.is_file() and b"PDF-FAIL-CONCLUSION" in path.read_bytes()] + assert left == [] + + +def test_paperless_filters_change_the_hits(client, db, monkeypatch): + catalog = [] + for number in range(1, 46): + catalog.append({ + "id": number, + "title": "Vertrag" if number % 5 == 0 else f"Schrift {number}", + "content": "nurinhalt" if number == 41 else f"text {number}", + "correspondent": 4 if number % 2 == 0 else 9, + "document_type": 8 if number < 30 else 1, + "tags": [2, 5] if number % 3 == 0 else [2], + "created": "2024-06-01" if number > 20 else "2023-01-01", + }) + + def fake_get(base, token, path, params): + if path != "/api/documents/": + return {"count": 1, "results": [{"id": 4, "name": "Haus"}]} + matched = [] + for item in catalog: + query = params.get("query") or "" + title = params.get("title__icontains") or "" + if query and query.casefold() not in f"{item['title']} {item['content']}".casefold(): + continue + if title and title.casefold() not in item["title"].casefold(): + continue + if params.get("correspondent__id") and str(item["correspondent"]) != params["correspondent__id"]: + continue + if params.get("document_type__id") and str(item["document_type"]) != params["document_type__id"]: + continue + if params.get("tags__id__all"): + needed = {int(part) for part in params["tags__id__all"].split(",") if part} + if not needed.issubset(set(item["tags"])): + continue + start = params.get("created__date__gte") + end = params.get("created__date__lte") + if start and item["created"] < start: + continue + if end and item["created"] > end: + continue + matched.append(item) + page = int(params.get("page") or 1) + start_at = (page - 1) * 20 + rows = matched[start_at:start_at + 20] + return { + "count": len(matched), + "results": [{"id": item["id"], "title": item["title"], "created": item["created"]} for item in rows], + } + + monkeypatch.setattr(sources, "_paperless_get", fake_get) + ada = Actor(client, "ada-sense@example.test", "Ada Sinn") + register(db, ada.email, ada.name) + ada.login() + account = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "paperless", "name": "Archiv", "base_url": "https://paper.example.test", "token": "token-alpha"}, + status=201, + ) + content = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": account["id"], "q": "nurinhalt"}, + ) + assert [item["id"] for item in content["items"]] == ["41"] + title = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": account["id"], "title": "Vertrag"}, + ) + assert title["items"] + assert all("Vertrag" in item["title"] for item in title["items"]) + assert all(item["id"] != "41" for item in title["items"]) + combined = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={ + "connector_id": account["id"], + "correspondent": "4", + "document_type": "8", + "tag": "2,5", + "document_from": "2023-01-01", + "document_to": "2024-12-31", + }, + ) + plain = ada.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"connector_id": account["id"], "cursor": "2"}, + ) + assert [item["id"] for item in combined["items"]] == ["6", "12", "18", "24"] + assert [item["id"] for item in plain["items"]][0] == "21" diff --git a/backend/tests/test_correspondence.py b/backend/tests/test_correspondence.py index d364681..7e61a4c 100644 --- a/backend/tests/test_correspondence.py +++ b/backend/tests/test_correspondence.py @@ -230,7 +230,7 @@ def test_dispatch_keeps_the_secured_stand_and_distinct_mail_parts(client, db, mo "Fachlicher Zusatz: die Frist bleibt der 1. Juni.", [("rechnung.pdf", PDF_A), ("lage.pdf", PDF_B), ("konto.pdf", PDF_C), ("lage.pdf", PDF_D)], ) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: raw) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: raw) ada = Actor(client, "ada-versand@example.test", "Ada Versand") register(db, ada.email, ada.name) ada.login() diff --git a/backend/tests/test_export.py b/backend/tests/test_export.py index b695687..4f50ce7 100644 --- a/backend/tests/test_export.py +++ b/backend/tests/test_export.py @@ -141,9 +141,9 @@ def test_handover_package_follows_the_confirmed_selection(client, db, monkeypatc "Der vollständige Brieftext bleibt erhalten.", [("lage.pdf", PDF_A), ("lage.pdf", PDF_B)], ) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: raw) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: raw) monkeypatch.setattr( - "app.retrieval.pull_paperless_original", + "app.sources.pull_paperless_original", lambda *args, **kwargs: (PDF_B, "application/pdf", "archiv.pdf"), ) ada = Actor(client, "ada-export@example.test", "Ada Export") @@ -491,7 +491,7 @@ def test_export_names_gaps_and_hides_unreadable_counterparts(client, db, monkeyp calls.append(args) return _eml("", "darf nicht ersatzweise kommen", []) - monkeypatch.setattr("app.retrieval.pull_rfc822", pull) + monkeypatch.setattr("app.sources.pull_rfc822", pull) context = {"kind": "case", "id": case["id"]} tree = ada.request( "GET", @@ -547,7 +547,7 @@ def test_export_names_gaps_and_hides_unreadable_counterparts(client, db, monkeyp assert {item["name"] for item in pending["issues"][0]["choices"]} == {"Neu", "Weiter"} monkeypatch.setattr("app.retrieval.EXPORT_BYTE_LIMIT", 40) huge = _eml("", "GEKÜRZT-VERBOTEN " * 20, []) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: huge) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: huge) assigned = ada.request( "POST", f"/api/v1/matters/{matter['id']}/export/review", @@ -557,7 +557,7 @@ def test_export_names_gaps_and_hides_unreadable_counterparts(client, db, monkeyp assert "GEKÜRZT-VERBOTEN" not in json.dumps(assigned) wrong = _eml("", "FALSCHE-NACHRICHT", []) monkeypatch.setattr("app.retrieval.EXPORT_BYTE_LIMIT", 80 * 1024 * 1024) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: wrong) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: wrong) identity = ada.request( "POST", f"/api/v1/matters/{matter['id']}/export/review", @@ -609,7 +609,7 @@ def test_nested_chapter_export_keeps_path_and_complete_mail(client, db, monkeypa "Die Antwort bleibt vollständig.", [("lage.pdf", PDF_A), ("konto.pdf", PDF_B)], ) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: raw) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: raw) ada = Actor(client, "ada-nested-export@example.test", "Ada Verschachtelt") register(db, ada.email, ada.name) ada.login() @@ -844,7 +844,7 @@ def test_chapter_entries_share_one_numbering_sequence(client, db, monkeypatch): files={"upload": ("drittes.pdf", PDF_C, "application/pdf")}, status=201, ) - monkeypatch.setattr("app.retrieval.pull_rfc822", lambda *args, **kwargs: raw) + monkeypatch.setattr("app.sources.pull_rfc822", lambda *args, **kwargs: raw) tree = ada.request("GET", f"/api/v1/matters/{matter['id']}/export-tree")["tree"] exportable = {"binding", "annotation", "attachment", "dispatch_item", "evidence"} selected = [node["id"] for node in _walk(tree) if node["exportable"] and node["kind"] in exportable] diff --git a/backend/tests/test_library.py b/backend/tests/test_library.py new file mode 100644 index 0000000..7e16bad --- /dev/null +++ b/backend/tests/test_library.py @@ -0,0 +1,273 @@ +"""Bibliothek: Zähler nur aus Lesbarem, Rauminhalt, Raumübersicht und Teilen im Stapel.""" + +from __future__ import annotations + +import pytest +from fastapi.testclient import TestClient + +from app.auth.accounts import create_account +from tests.support import PASSWORD, Actor, matter_of + + +def _second_client(): + from app.main import app + + return TestClient(app) + + +def _actor(client, db, key: str, *, main: bool = False) -> Actor: + actor = Actor(client if main else _second_client(), f"{key}-bib@example.test", key.title()) + create_account(db, actor.email, actor.name, PASSWORD) + db.commit() + actor.login() + return actor + + +@pytest.fixture +def world(client, db): + ada = _actor(client, db, "ada", main=True) + ben = _actor(client, db, "ben") + cem = _actor(client, db, "cem") + dan = _actor(client, db, "dan") + room = ada.request("POST", "/api/v1/rooms", json={"name": "Werkstatt", "default_visibility": "private"}, status=201)["id"] + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "member"}) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": cem.email, "role": "reader"}) + return {"ada": ada, "ben": ben, "cem": cem, "dan": dan, "room": room} + + +def _share(actor, plural, resource_id, room, status=200): + return actor.request("PUT", f"/api/v1/{plural}/{resource_id}/room-share/rooms/{room}", json={"access": "edit"}, status=status) + + +def _link(actor, matter, case): + actor.request("POST", f"/api/v1/matters/{matter}/case-links", json={"case_id": case}, status=201) + + +def test_counts_include_only_what_the_person_may_read(world): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Vermietung"}, status=201) + child = ada.request( + "POST", "/api/v1/cases", + json={"title": "Nebenkosten", "parent_case_id": case["id"], "rights_mode": "inherit_parent"}, status=201, + ) + one = matter_of(ada, "Mietminderung") + two = matter_of(ada, "Eigenbedarf intern") + _link(ada, one["id"], case["id"]) + _link(ada, two["id"], case["id"]) + _share(ada, "cases", case["id"], room) + + mine = next(i for i in ada.request("GET", "/api/v1/cases")["items"] if i["id"] == case["id"]) + assert (mine["child_cases"], mine["matter_count"]) == (1, 2) + assert [s["name"] for s in mine["shared_with"]] == ["Werkstatt"] + + # One matter is withheld from the room members in this case. + ada.request( + "PUT", "/api/v1/exceptions", + json={"resource_kind": "matter", "resource_id": two["id"], "context": {"kind": "case", "id": case["id"]}, "remove_read": True}, + ) + seen = next(i for i in ben.request("GET", "/api/v1/cases")["items"] if i["id"] == case["id"]) + assert seen["matter_count"] == 1 and seen["shared_with"] == [] and seen["is_owner"] is False + assert child["title"] == "Nebenkosten" + + +def test_a_room_shows_its_contents_to_members_only(world): + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Im Raum"}, status=201) + sub = ada.request( + "POST", "/api/v1/cases", + json={"title": "Unter der Akte", "parent_case_id": case["id"], "rights_mode": "inherit_parent"}, status=201, + ) + ada.request("POST", "/api/v1/cases", json={"title": "Bleibt privat"}, status=201) + matter = matter_of(ada, "Vorgang im Raum") + _share(ada, "cases", case["id"], room) + _share(ada, "matters", matter["id"], room) + + for member in (ada, ben, cem): + found = member.request("GET", f"/api/v1/rooms/{room}/contents") + assert {c["title"] for c in found["cases"]} == {"Im Raum", "Unter der Akte"} + assert [m["title"] for m in found["matters"]] == ["Vorgang im Raum"] + dan.request("GET", f"/api/v1/rooms/{room}/contents", status=403) + # A person who is excluded does not see the entry in the room either. + ada.request("PUT", f"/api/v1/cases/{case['id']}/room-share/people/{_id(ada, ben)}", json={"excluded": True}) + titles = {c["title"] for c in ben.request("GET", f"/api/v1/rooms/{room}/contents")["cases"]} + assert "Im Raum" not in titles + assert sub["title"] == "Unter der Akte" + + +def test_the_room_overview_names_members_only_to_managers(world): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Gezählt"}, status=201) + _share(ada, "cases", case["id"], room) + mine = next(r for r in ada.request("GET", "/api/v1/rooms/overview")["items"] if r["id"] == room) + assert mine["is_owner"] is True and mine["role"] == "manager" and mine["member_count"] == 3 + assert mine["case_count"] == 1 and mine["matter_count"] == 0 + assert set(mine["members"]) == {"Ada", "Ben", "Cem"} + theirs = next(r for r in ben.request("GET", "/api/v1/rooms/overview")["items"] if r["id"] == room) + assert theirs["members"] == [] and theirs["member_count"] == 3 and theirs["is_owner"] is False + assert world["dan"].request("GET", "/api/v1/rooms/overview")["items"] == [] + + +def test_several_entries_are_shared_at_once_or_not_at_all(world): + ada, ben, room = world["ada"], world["ben"], world["room"] + a = ada.request("POST", "/api/v1/cases", json={"title": "Eins"}, status=201) + b = ada.request("POST", "/api/v1/cases", json={"title": "Zwei"}, status=201) + m = matter_of(ada, "Drei") + done = ada.request( + "POST", f"/api/v1/rooms/{room}/shares", + json={"items": [{"kind": "case", "id": a["id"]}, {"kind": "case", "id": b["id"]}, {"kind": "matter", "id": m["id"]}], "access": "read"}, + ) + assert done["shared"] == 3 + assert {(e["display_name"], e["role"], e["rights"]["edit"]) for e in done["effect"]} == {("Ben", "member", False), ("Cem", "reader", False)} + assert ben.request("GET", f"/api/v1/cases/{a['id']}")["rights"]["edit"] is False + + foreign = ben.request("POST", "/api/v1/cases", json={"title": "Von Ben"}, status=201) + c = ada.request("POST", "/api/v1/cases", json={"title": "Vier"}, status=201) + ada.request( + "POST", f"/api/v1/rooms/{room}/shares", + json={"items": [{"kind": "case", "id": c["id"]}, {"kind": "case", "id": foreign["id"]}]}, status=404, + ) + assert room not in [s["room_id"] for s in ada.request("GET", f"/api/v1/cases/{c['id']}/room-share")["shares"]] + # An outsider cannot put his own entry into the room, and nobody shares an entry of someone else. + own = world["dan"].request("POST", "/api/v1/cases", json={"title": "Von Dan"}, status=201) + world["dan"].request("POST", f"/api/v1/rooms/{room}/shares", json={"items": [{"kind": "case", "id": own["id"]}]}, status=403) + world["cem"].request("POST", f"/api/v1/rooms/{room}/shares", json={"items": [{"kind": "case", "id": c["id"]}]}, status=404) + + +def test_creating_an_entry_can_put_it_into_a_room_and_reports_the_effect(world): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Gleich im Raum", "share_with_room_id": room}, status=201) + matter = ada.request("POST", "/api/v1/matters", json={"title": "Vorgang im Raum", "share_with_room_id": room}, status=201) + ben.request("GET", f"/api/v1/cases/{case['id']}") + ben.request("GET", f"/api/v1/matters/{matter['id']}") + other = ada.request("POST", "/api/v1/cases", json={"title": "Ohne Raum"}, status=201) + ben.request("GET", f"/api/v1/cases/{other['id']}", status=404) + answer = _share(ada, "cases", other["id"], room) + assert {e["display_name"] for e in answer["effect"]} == {"Ben", "Cem"} + world["dan"].request("POST", "/api/v1/cases", json={"title": "Fremd", "share_with_room_id": room}, status=403) + + +def _id(owner: Actor, person: Actor) -> str: + from app.core.settings import get_settings + import psycopg + + with psycopg.connect(get_settings().dsn(), row_factory=psycopg.rows.dict_row) as conn: + return str(conn.execute("SELECT id FROM auth_accounts WHERE email = %s", (person.email,)).fetchone()["id"]) + + +def test_the_outline_lists_only_what_the_person_may_read(world): + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Gliederung"}, status=201) + ada.request("POST", f"/api/v1/cases/{case['id']}/chapters", json={"title": "Schriftverkehr"}, status=201) + sub = ada.request( + "POST", "/api/v1/cases", + json={"title": "Unterakte", "parent_case_id": case["id"], "rights_mode": "inherit_parent"}, status=201, + ) + open_matter = matter_of(ada, "Offener Vorgang") + hidden_matter = matter_of(ada, "Verborgener Vorgang") + child = ada.request("POST", "/api/v1/matters", json={"title": "Teilvorgang", "parent_matter_id": open_matter["id"]}, status=201) + ada.request("POST", f"/api/v1/matters/{open_matter['id']}/chapters", json={"title": "Belege"}, status=201) + _link(ada, open_matter["id"], case["id"]) + _link(ada, hidden_matter["id"], case["id"]) + _share(ada, "cases", case["id"], room) + ada.request( + "PUT", "/api/v1/exceptions", + json={"resource_kind": "matter", "resource_id": hidden_matter["id"], "context": {"kind": "case", "id": case["id"]}, "remove_read": True}, + ) + + mine = ada.request("GET", f"/api/v1/cases/{case['id']}/outline") + assert [c["title"] for c in mine["chapters"]] == ["Schriftverkehr"] + assert [c["title"] for c in mine["subcases"]] == ["Unterakte"] + # An exception in the case context hides the matter there for everyone, the owner included. + assert {m["title"] for m in mine["matters"]} == {"Offener Vorgang"} + seen = ben.request("GET", f"/api/v1/cases/{case['id']}/outline") + assert {m["title"] for m in seen["matters"]} == {"Offener Vorgang"} + assert [c["title"] for c in seen["matters"][0]["children"]] == ["Teilvorgang"] + assert sub["id"] in [c["id"] for c in seen["subcases"]] + + matter = ada.request("GET", f"/api/v1/matters/{open_matter['id']}/outline") + assert [c["title"] for c in matter["chapters"]] == ["Belege"] + assert [c["title"] for c in matter["submatters"]] == ["Teilvorgang"] + in_case = ben.request( + "GET", f"/api/v1/matters/{open_matter['id']}/outline", + params={"context_kind": "case", "context_id": case["id"]}, + ) + assert [c["title"] for c in in_case["submatters"]] == ["Teilvorgang"] + ben.request("GET", f"/api/v1/matters/{open_matter['id']}/outline", status=404) + assert child["title"] == "Teilvorgang" + + unknown = "00000000-0000-4000-8000-000000000000" + seen_status = dan.request("GET", f"/api/v1/cases/{case['id']}/outline", status=404) + missing = dan.request("GET", f"/api/v1/cases/{unknown}/outline", status=404) + assert seen_status["detail"] == missing["detail"] + dan.request("GET", f"/api/v1/matters/{open_matter['id']}/outline", status=404) + assert cem.request("GET", f"/api/v1/cases/{case['id']}/outline")["id"] == case["id"] + + +def test_unlinking_keeps_the_matter_and_its_other_cases(world): + ada, ben = world["ada"], world["ben"] + first = ada.request("POST", "/api/v1/cases", json={"title": "Vermietung"}, status=201)["id"] + second = ada.request("POST", "/api/v1/cases", json={"title": "Steuer"}, status=201)["id"] + matter = matter_of(ada, "Nebenkosten")["id"] + ada.request("POST", f"/api/v1/matters/{matter}/case-links", json={"case_id": first}, status=201) + ada.request("POST", f"/api/v1/matters/{matter}/case-links", json={"case_id": second}, status=201) + links = ada.request("GET", f"/api/v1/matters/{matter}")["case_links"] + assert {item["case_id"] for item in links} == {first, second} + link = next(item for item in links if item["case_id"] == first) + + # A stranger cannot detach it and learns nothing about it. + ben.request("DELETE", f"/api/v1/matter-case-links/{link['id']}", status=404) + + ada.request("DELETE", f"/api/v1/matter-case-links/{link['id']}", status=204) + opened = ada.request("GET", f"/api/v1/matters/{matter}") + assert [item["case_id"] for item in opened["case_links"]] == [second] + assert opened["title"] == "Nebenkosten" + assert matter in {item["id"] for item in ada.request("GET", "/api/v1/matters")["items"]} + ada.request("DELETE", f"/api/v1/matter-case-links/{link['id']}", status=404) + + +def test_download_policy_follows_the_room_role(world): + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + matter = matter_of(ada, "Nebenkosten")["id"] + file = ada.request( + "POST", + "/api/v1/files", + data={"title": "Vertrag", "context_kind": "matter", "context_id": matter}, + files={"upload": ("vertrag.pdf", b"%PDF-1.4\n1 0 obj<<>>endobj\ntrailer<<>>\n%%EOF\n", "application/pdf")}, + status=201, + )["id"] + _share(ada, "matters", matter, room) + path = f"/api/v1/files/{file}/content" + query = {"context_kind": "matter", "context_id": matter} + + def fetch(person, **extra): + from tests.support import ORIGIN + + headers = {"Origin": ORIGIN, "X-CSRF-Token": person.csrf} + if person.scope is not None: + headers["X-Scope-Kind"], headers["X-Scope-Id"] = person.scope["kind"], person.scope["id"] + return person.client.get(path, params={**query, **extra}, headers=headers) + + # By default every role may save the original. + for person in (ben, cem): + assert fetch(person, save="true").status_code == 200 + seen = ben.request("GET", f"/api/v1/matters/{matter}") + assert seen["rights"]["download"] is True + + # The owner turns it off for readers. Reading stays possible, saving does not. + assert cem.request("GET", f"/api/v1/rooms/{room}/download-policy")["can_change"] is False + cem.request("PUT", f"/api/v1/rooms/{room}/download-policy", json={"manager": True, "member": True, "reader": False}, status=403) + ada.request("PUT", f"/api/v1/rooms/{room}/download-policy", json={"manager": True, "member": True, "reader": False}) + assert fetch(cem).status_code == 200 + refused = fetch(cem, save="true") + assert refused.status_code == 403 + assert "freigegeben" in refused.json()["detail"] + assert cem.request("GET", f"/api/v1/matters/{matter}")["rights"]["download"] is False + assert fetch(ben, save="true").status_code == 200 + # A stranger learns nothing about the policy of the room. + dan.request("GET", f"/api/v1/rooms/{room}/download-policy", status=404) + + # The owner's default is copied into a new room. + ada.request("PUT", "/api/v1/auth/download-defaults", json={"manager": True, "member": False, "reader": False, "direct": True}) + fresh = ada.request("POST", "/api/v1/rooms", json={"name": "Neu", "default_visibility": "private"}, status=201)["id"] + assert ada.request("GET", f"/api/v1/rooms/{fresh}/download-policy")["roles"] == {"manager": True, "member": False, "reader": False} + assert ada.request("GET", f"/api/v1/rooms/{room}/download-policy")["roles"]["member"] is True diff --git a/backend/tests/test_mail_html.py b/backend/tests/test_mail_html.py new file mode 100644 index 0000000..95c7efe --- /dev/null +++ b/backend/tests/test_mail_html.py @@ -0,0 +1,81 @@ +"""HTML mail: formatting stays, everything active or loading from outside is removed.""" + +from __future__ import annotations + +from app.mailpack import sanitize_html + + +def test_formatting_stays_and_active_content_goes(): + cleaned = sanitize_html( + '
Hallo' + '
x

' + '
' + ) + assert 'style="color: #333; font-size: 14px"' in cleaned + assert '' in cleaned + assert 'bgcolor="#eee"' in cleaned and "
" in cleaned + for forbidden in ("onclick", "script", "iframe", "Text

' + ) + assert "url" not in cleaned and "position" not in cleaned and "expression" not in cleaned and "behavior" not in cleaned + assert "color: red" in cleaned and "margin: 4px" in cleaned + + +def test_links_and_images_are_limited(): + cleaned = sanitize_html( + 'abc' + 'logo' + ) + assert "javascript" not in cleaned and "data:text" not in cleaned and "track" not in cleaned and "svg" not in cleaned + assert 'href="https://ok.example/x"' in cleaned + assert 'rel="noreferrer noopener"' in cleaned and 'target="_blank"' in cleaned + assert 'src="data:image/png;base64,AAAA"' in cleaned + + +def test_remote_pictures_stay_out_unless_the_server_loads_them(): + source = 'a

Text

' + found: list[str] = [] + assert "") is None + assert module._kind(b"RIFF\x00\x00\x00\x00WEBPVP8 ") == "webp" diff --git a/backend/tests/test_mailboxes.py b/backend/tests/test_mailboxes.py index fa70fb5..51205be 100644 --- a/backend/tests/test_mailboxes.py +++ b/backend/tests/test_mailboxes.py @@ -180,9 +180,12 @@ def test_foreign_mailbox_stays_hidden(client, db, monkeypatch): class HugeIMAP(FakeIMAP): def uid(self, command, *args): spec = " ".join(str(part) for part in args[1:]) if len(args) > 1 else "" - if command == "FETCH" and "BODY.PEEK[]" in spec: + if command == "FETCH" and "BODY.PEEK[]" in spec and "<" not in spec: return "OK", [(b"rfc822", b"x" * 8_000_001)] - if "BODY.PEEK[TEXT]" in spec: + if "BODYSTRUCTURE" in spec: + structure = b'("TEXT" "PLAIN" ("CHARSET" "utf-8") NIL NIL "7BIT" 40 1 NIL NIL NIL NIL)' + return "OK", [(b"body", structure)] + if "BODY.PEEK[1]" in spec: return "OK", [(b"text", "Kurzer Auszug der grossen Nachricht.".encode())] return super().uid(command, *args) @@ -282,3 +285,195 @@ def test_an_adopted_message_loads_in_full_with_its_attachment(client, db, monkey ada.request("GET", f"/api/v1/external-refs/{wrong['element']['id']}/mail", status=422) ben.login() ben.request("GET", f"/api/v1/external-refs/{created['id']}/mail", status=404) + + +def _letter(message_id: str, body: str) -> bytes: + return ( + f"From: Ada Beispiel \r\n" + f"To: Gegenseite \r\n" + f"Subject: Verschoben\r\n" + f"Date: Thu, 02 May 2024 08:00:00 +0000\r\n" + f"Message-ID: {message_id}\r\n" + f"MIME-Version: 1.0\r\n" + f"Content-Type: text/plain; charset=utf-8\r\n" + f"\r\n" + f"{body}\r\n" + ).encode() + + +class RelocatingIMAP: + """The saved UID holds another message. The Message-ID lives in Archiv.""" + + log: list[tuple] = [] + copies: dict[tuple[str, int], bytes] = {} + header_uids: dict[str, bytes] | None = None + + def __init__(self, *args, **kwargs): + self.selected = "" + + def login(self, username, secret): + assert secret == "Postfach-Geheim" + return "OK", [b"logged"] + + def list(self): + return "OK", [ + b'(\\HasNoChildren) "/" "INBOX"', + b'(\\HasNoChildren) "/" "Archiv"', + ] + + def select(self, mailbox, readonly=False): + assert readonly is True + self.selected = mailbox + RelocatingIMAP.log.append(("select", mailbox)) + return "OK", [b"2"] + + def uid(self, command, *args): + if command == "SEARCH": + if args and args[0] == "HEADER": + RelocatingIMAP.log.append(("header", self.selected)) + override = (RelocatingIMAP.header_uids or {}).get(self.selected) + if override is not None: + return "OK", [override] + found = [ + str(uid) + for (folder, uid), raw in RelocatingIMAP.copies.items() + if folder == self.selected and b"Message-ID: " in raw + ] + return "OK", [" ".join(found).encode()] + return "OK", [b""] + uid = int(args[0]) + spec = " ".join(str(part) for part in args[1:]) + if "BODY.PEEK[]" in spec: + RelocatingIMAP.log.append(("fetch", self.selected, uid)) + raw = RelocatingIMAP.copies.get((self.selected, uid)) + if raw is None: + return "OK", [None] + return "OK", [(b"rfc822", raw)] + return "OK", [None] + + def logout(self): + return "BYE", [b""] + + +def _stored_mail(db, element_id: str) -> dict: + row = db.execute( + "SELECT detail FROM external_references WHERE element_id = %s", + (element_id,), + ).fetchone() + return dict(row["detail"]) + + +def _linked_mail(actor: Actor, account: str, matter_id: str) -> dict: + return actor.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Verschoben", + "source": "imap", + "external_id": "", + "context": {"kind": "matter", "id": matter_id}, + "detail": { + "connector_id": account, + "mailbox": "INBOX", + "uid": 2, + "direction": "in", + }, + }, + status=201, + ) + + +def test_a_moved_message_remembers_its_new_folder(client, db, monkeypatch): + from uuid import UUID + + from app.retrieval import MailPayload, load_external + + ada = Actor(client, "ada-move@example.test", "Ada Umzug") + account_row = register(db, ada.email, ada.name) + ada.login() + RelocatingIMAP.log = [] + RelocatingIMAP.header_uids = None + RelocatingIMAP.copies = { + ("INBOX", 2): _letter("", "Eine andere Nachricht."), + ("Archiv", 9): _letter("", "Der Brief liegt jetzt im Archiv."), + } + account = _account(monkeypatch, ada) + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", RelocatingIMAP) + matter = matter_of(ada, "Ablage") + created = _linked_mail(ada, account, matter["id"]) + + opened = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail") + assert "Archiv" in opened["text"] + stored = _stored_mail(db, created["id"]) + assert stored["mailbox"] == "Archiv" + assert stored["uid"] == 9 + assert stored["direction"] == "in" + assert stored["connector_id"] == account + assert any(item[0] == "header" for item in RelocatingIMAP.log) + + RelocatingIMAP.log = [] + again = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail") + assert again["text"] == opened["text"] + assert ("fetch", "Archiv", 9) in RelocatingIMAP.log + assert not any(item[0] == "header" for item in RelocatingIMAP.log) + assert _stored_mail(db, created["id"])["uid"] == 9 + + db.execute( + """ + UPDATE external_references + SET detail = jsonb_set( + jsonb_set(detail, '{mailbox}', to_jsonb('INBOX'::text)), + '{uid}', + to_jsonb(2) + ) + WHERE element_id = %s + """, + (created["id"],), + ) + db.commit() + RelocatingIMAP.log = [] + loaded = load_external(db, account_row["id"], UUID(created["id"]), None) + assert isinstance(loaded, MailPayload) + assert "Archiv" in loaded.parsed.text + exported = _stored_mail(db, created["id"]) + assert exported["mailbox"] == "Archiv" + assert exported["uid"] == 9 + assert exported["direction"] == "in" + assert any(item[0] == "header" for item in RelocatingIMAP.log) + + +def test_an_ambiguous_or_truncated_move_does_not_replace_the_stored_place(client, db, monkeypatch): + ada = Actor(client, "ada-ambiguous@example.test", "Ada Zweifach") + register(db, ada.email, ada.name) + ada.login() + RelocatingIMAP.header_uids = None + RelocatingIMAP.copies = { + ("INBOX", 2): _letter("", "Altbestand."), + ("INBOX", 4): _letter("", "Erste Kopie."), + ("Archiv", 9): _letter("", "Zweite Kopie."), + } + account = _account(monkeypatch, ada) + monkeypatch.setattr(sources.imaplib, "IMAP4_SSL", RelocatingIMAP) + matter = matter_of(ada, "Zweifach") + created = _linked_mail(ada, account, matter["id"]) + ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail", status=422) + assert _stored_mail(db, created["id"])["mailbox"] == "INBOX" + assert _stored_mail(db, created["id"])["uid"] == 2 + + RelocatingIMAP.copies = { + ("INBOX", 2): _letter("", "Altbestand."), + ("INBOX", 11): _letter("", "Treffer in der abgebrochenen Suche."), + ("INBOX", 12): _letter("", "a"), + ("INBOX", 13): _letter("", "b"), + ("INBOX", 14): _letter("", "c"), + ("INBOX", 15): _letter("", "d"), + ("INBOX", 16): _letter("", "e"), + ("Archiv", 9): _letter("", "Eigentlich auch hier."), + } + RelocatingIMAP.header_uids = {"INBOX": b"11 12 13 14 15 16"} + RelocatingIMAP.log = [] + opened = ada.request("GET", f"/api/v1/external-refs/{created['id']}/mail") + assert "abgebrochenen Suche" in opened["text"] + assert not any(item == ("header", "Archiv") for item in RelocatingIMAP.log) + assert _stored_mail(db, created["id"])["mailbox"] == "INBOX" + assert _stored_mail(db, created["id"])["uid"] == 2 diff --git a/backend/tests/test_owner_sources.py b/backend/tests/test_owner_sources.py index 1f37837..3333195 100644 --- a/backend/tests/test_owner_sources.py +++ b/backend/tests/test_owner_sources.py @@ -294,6 +294,16 @@ def test_grantee_reads_linked_originals_only_through_the_owner(client, db, monke assert opened.headers["referrer-policy"] == "no-referrer" assert calls == [{"token": OWNER_TOKEN, "path": "/api/documents/15/preview/"}] assert GRANTEE_TOKEN not in json.dumps(calls) + own_connectors = {item["id"] for item in ben.request("GET", "/api/v1/connectors")["items"]} + assert paper not in own_connectors + assert mailbox not in own_connectors + ben.request("POST", f"/api/v1/connectors/{paper}/probe", status=404) + ben.request( + "GET", + "/api/v1/sources/paperless/documents", + params={"q": "Geheim", "connector_id": paper}, + status=404, + ) search = ben.request("GET", "/api/v1/sources/paperless/documents", params={"q": "Geheim"}) assert "Geheimes Archivdokument" not in json.dumps(search) assert all(item["token"] != OWNER_TOKEN for item in calls if "query" in item) diff --git a/backend/tests/test_rights.py b/backend/tests/test_rights.py index 844c273..6eff1c2 100644 --- a/backend/tests/test_rights.py +++ b/backend/tests/test_rights.py @@ -14,7 +14,7 @@ def test_manager_does_not_read_a_personal_case(client, db): ada = Actor(client, "ada@example.test", "Ada") ada.login() private = case_of(ada, "Persoenliche Akte") - room = ada.request("POST", "/api/v1/rooms", json={"name": "Haushalt"}, status=201) + room = ada.request("POST", "/api/v1/rooms", json={"name": "Haushalt", "default_visibility": "private"}, status=201) ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": "ben@example.test", "role": "manager"}) ben = Actor(client, "ben@example.test", "Ben") ben.login() diff --git a/backend/tests/test_room_access.py b/backend/tests/test_room_access.py new file mode 100644 index 0000000..9b3934d --- /dev/null +++ b/backend/tests/test_room_access.py @@ -0,0 +1,377 @@ +"""Raum als Team: Akten und Vorgänge werden mit Datenräumen geteilt, unabhängig vom Heimatraum.""" + +from __future__ import annotations + +import pytest +from fastapi.testclient import TestClient + +from app.auth.accounts import create_account +from tests.support import PASSWORD, Actor, document, matter_of, note_on + + +def _second_client(): + from app.main import app + + return TestClient(app) + + +def _make(db, email: str, name: str): + account = create_account(db, email, name, PASSWORD) + db.commit() + return account + + +def _actor(client, db, key: str, email: str, *, main: bool = False) -> Actor: + actor = Actor(client if main else _second_client(), email, key.title()) + _make(db, actor.email, actor.name) + actor.login() + return actor + + +def _id(db, email: str) -> str: + row = db.execute("SELECT id FROM auth_accounts WHERE email = %s", (email,)).fetchone() + db.rollback() + return str(row["id"]) + + +def _share(actor: Actor, plural: str, resource_id: str, room: str, access: str = "edit", status: int = 200): + return actor.request("PUT", f"/api/v1/{plural}/{resource_id}/room-share/rooms/{room}", json={"access": access}, status=status) + + +def _unshare(actor: Actor, plural: str, resource_id: str, room: str, status: int = 204): + return actor.request("DELETE", f"/api/v1/{plural}/{resource_id}/room-share/rooms/{room}", status=status) + + +@pytest.fixture +def world(client, db): + """Ada owns the room; Ben is a member, Cem a reader, Dan is outside.""" + actors = { + "ada": _actor(client, db, "ada", "ada-raum@example.test", main=True), + "ben": _actor(client, db, "ben", "ben-raum@example.test"), + "cem": _actor(client, db, "cem", "cem-raum@example.test"), + "dan": _actor(client, db, "dan", "dan-raum@example.test"), + } + ada = actors["ada"] + room = ada.request("POST", "/api/v1/rooms", json={"name": "Werkstatt"}, status=201) + assert room["default_visibility"] == "room" + ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": actors["ben"].email, "role": "member"}) + ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": actors["cem"].email, "role": "reader"}) + ada.use_room(room["id"]) + actors["room"] = room["id"] + return actors + + +def test_a_new_entry_follows_the_room_preset_and_rights_follow_the_role(world, db): + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + shared = ada.request("POST", "/api/v1/cases", json={"title": "Gemeinsame Akte"}, status=201) + view = ada.request("GET", f"/api/v1/cases/{shared['id']}/room-share") + assert [(s["room_name"], s["access"]) for s in view["shares"]] == [("Werkstatt", "edit")] + + assert ben.request("GET", f"/api/v1/cases/{shared['id']}")["rights"] == { + "read": True, "edit": True, "share": False, "delete": False, "download": True, + } + assert cem.request("GET", f"/api/v1/cases/{shared['id']}")["rights"] == { + "read": True, "edit": False, "share": False, "delete": False, "download": True, + } + dan.request("GET", f"/api/v1/cases/{shared['id']}", status=404) + assert "Gemeinsame Akte" in {item["title"] for item in ben.request("GET", "/api/v1/cases")["items"]} + assert dan.request("GET", "/api/v1/cases")["items"] == [] + + version = ben.request("GET", f"/api/v1/cases/{shared['id']}")["version"] + ben.request("PATCH", f"/api/v1/cases/{shared['id']}", json={"expected_version": version, "title": "Von Ben geändert"}) + cem.request("PATCH", f"/api/v1/cases/{shared['id']}", json={"expected_version": version + 1, "title": "Nein"}, status=403) + ben.request("POST", f"/api/v1/cases/{shared['id']}/delete", json={"expected_version": version + 1}, status=403) + ben.request("POST", f"/api/v1/cases/{shared['id']}/grants", json={"email": dan.email, "can_read": True}, status=403) + + +def test_unsharing_and_leaving_take_the_rights_back_at_once(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Zeitweise sichtbar"}, status=201) + ben.request("GET", f"/api/v1/cases/{case['id']}") + _unshare(ada, "cases", case["id"], room) + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + _share(ada, "cases", case["id"], room) + ben.request("GET", f"/api/v1/cases/{case['id']}") + ada.request("DELETE", f"/api/v1/rooms/{room}/members/{_id(db, ben.email)}", status=204) + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + assert ben.request("GET", "/api/v1/cases")["items"] == [] + + +def test_an_archived_room_shares_nothing_until_it_is_restored(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Archivtest"}, status=201) + ben.request("GET", f"/api/v1/cases/{case['id']}") + db.execute("UPDATE data_rooms SET status = 'archived' WHERE id = %s", (room,)) + db.commit() + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + db.execute("UPDATE data_rooms SET status = 'active' WHERE id = %s", (room,)) + db.commit() + ben.request("GET", f"/api/v1/cases/{case['id']}") + + +def test_the_owner_sees_all_own_entries_whatever_happens_to_the_room(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + mine = ada.request("POST", "/api/v1/cases", json={"title": "Meine Akte im Raum"}, status=201) + personal = Actor(ada.client, ada.email, ada.name) + personal_body = personal.login() + personal.request("POST", "/api/v1/cases", json={"title": "Meine persönliche Akte"}, status=201) + # Ben owns an entry filed in the room; Ada, the owner of the room, does not see it. + ben.use_room(room) + bens = ben.request("POST", "/api/v1/cases", json={"title": "Bens Akte"}, status=201) + + def titles(actor): + return {item["title"] for item in actor.request("GET", "/api/v1/cases")["items"]} + + assert {"Meine Akte im Raum", "Meine persönliche Akte"} <= titles(personal) + assert "Bens Akte" in titles(personal) # shared with the room by the preset + _unshare(ben, "cases", bens["id"], room) + assert "Bens Akte" not in titles(personal) + assert "Bens Akte" in titles(ben) + + # Ben leaves the room, the room is archived: he still sees and edits what he owns. + personal.request("DELETE", f"/api/v1/rooms/{room}/members/{_id(db, ben.email)}", status=204) + # Ben's old room headers no longer fit his session; he is told to reload, then lists everything he owns. + assert ben.request("GET", "/api/v1/cases", status=403)["code"] == "scope_changed" + body = ben.request("GET", "/api/v1/auth/session") + assert body["scope"]["kind"] == "personal" + ben.scope, ben.csrf = body["scope"], body["csrf_token"] + assert "Bens Akte" in titles(ben) + ben.request("GET", f"/api/v1/cases/{bens['id']}") + db.execute("UPDATE data_rooms SET status = 'archived' WHERE id = %s", (room,)) + db.commit() + assert {"Meine Akte im Raum", "Meine persönliche Akte"} <= titles(personal) + personal.request("GET", f"/api/v1/cases/{mine['id']}") + assert personal_body["account"]["email"] == ada.email + + +def test_one_member_can_be_excluded(world, db): + ada, ben, cem, dan = world["ada"], world["ben"], world["cem"], world["dan"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Ohne Ben"}, status=201) + view = ada.request("GET", f"/api/v1/cases/{case['id']}/room-share") + assert {(m["display_name"], m["role"]) for m in view["shares"][0]["members"]} == {("Ben", "member"), ("Cem", "reader")} + base = f"/api/v1/cases/{case['id']}/room-share/people" + ada.request("PUT", f"{base}/{_id(db, ben.email)}", json={"excluded": True}) + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + cem.request("GET", f"/api/v1/cases/{case['id']}") + assert any(m["excluded"] for m in ada.request("GET", f"/api/v1/cases/{case['id']}/room-share")["shares"][0]["members"]) + ada.request("PUT", f"{base}/{_id(db, ben.email)}", json={"excluded": False}) + ben.request("GET", f"/api/v1/cases/{case['id']}") + ada.request("PUT", f"{base}/{_id(db, ada.email)}", json={"excluded": True}, status=422) + ada.request("PUT", f"{base}/{_id(db, dan.email)}", json={"excluded": True}, status=404) + ben.request("PUT", f"{base}/{_id(db, cem.email)}", json={"excluded": True}, status=404) + + +def test_contents_follow_the_room_role(world, db): + ada, ben, cem, dan = world["ada"], world["ben"], world["cem"], world["dan"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Akte mit Notiz"}, status=201) + note = note_on(ada, "Notiz in der Akte", "Raumtext", case["id"]) + detail = ben.request("GET", f"/api/v1/cases/{case['id']}") + assert any(b["element"]["title"] == "Notiz in der Akte" for b in detail["bindings"]) + context = {"kind": "case", "id": case["id"]} + ben.request("PATCH", f"/api/v1/notes/{note['id']}", json={"expected_version": note["version"], "document": document("Von Ben ergänzt"), "context": context}) + cem.request("PATCH", f"/api/v1/notes/{note['id']}", json={"expected_version": note["version"] + 1, "document": document("Nein"), "context": context}, status=403) + dan.request("GET", f"/api/v1/cases/{case['id']}", status=404) + assert dan.request("GET", "/api/v1/search", params={"q": "Notiz in der Akte"})["items"] == [] + assert any(i["title"] == "Notiz in der Akte" for i in cem.request("GET", "/api/v1/search", params={"q": "Notiz in der Akte"})["items"]) + + +def test_a_standalone_matter_is_shared_like_a_case(world, db): + ada, ben, cem, room = world["ada"], world["ben"], world["cem"], world["room"] + matter = matter_of(ada, "Eigener Vorgang") + assert ben.request("GET", f"/api/v1/matters/{matter['id']}")["rights"]["edit"] is True + assert cem.request("GET", f"/api/v1/matters/{matter['id']}")["rights"] == { + "read": True, "edit": False, "share": False, "delete": False, "download": True, + } + ben.request("POST", f"/api/v1/matters/{matter['id']}/delete", json={"expected_version": matter["version"]}, status=403) + _unshare(ada, "matters", matter["id"], room) + ben.request("GET", f"/api/v1/matters/{matter['id']}", status=404) + assert "Eigener Vorgang" not in {item["title"] for item in ben.request("GET", "/api/v1/matters")["items"]} + + +def test_subcases_follow_their_parent(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + parent = ada.request("POST", "/api/v1/cases", json={"title": "Eltern"}, status=201) + child = ada.request( + "POST", "/api/v1/cases", + json={"title": "Kind", "parent_case_id": parent["id"], "rights_mode": "inherit_parent"}, status=201, + ) + ben.request("GET", f"/api/v1/cases/{child['id']}") + _unshare(ada, "cases", parent["id"], room) + _unshare(ada, "cases", child["id"], room) + ben.request("GET", f"/api/v1/cases/{child['id']}", status=404) + + +def test_an_entry_in_the_personal_room_can_be_shared_with_a_team(world, db): + ada, ben, dan, room = world["ada"], world["ben"], world["dan"], world["room"] + personal = Actor(ada.client, ada.email, ada.name) + personal.login() + case = personal.request("POST", "/api/v1/cases", json={"title": "Aus dem persönlichen Raum"}, status=201) + assert personal.request("GET", f"/api/v1/cases/{case['id']}/room-share")["shares"] == [] + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + shared = personal.request("GET", f"/api/v1/cases/{case['id']}/room-share") + assert [item["name"] for item in shared["available"]] == ["Werkstatt"] + _share(personal, "cases", case["id"], room) + ben.request("GET", f"/api/v1/cases/{case['id']}") + dan.request("GET", f"/api/v1/cases/{case['id']}", status=404) + + +def test_one_matter_goes_to_the_lawyer_and_the_rest_of_the_case_stays_hidden(client, db): + ada = _actor(client, db, "ada", "ada-anwalt@example.test", main=True) + lawyer = _actor(client, db, "anwalt", "anwalt@example.test") + team = ada.request("POST", "/api/v1/rooms", json={"name": "Anwalt Müller", "default_visibility": "private"}, status=201) + ada.request("POST", f"/api/v1/rooms/{team['id']}/members", json={"email": lawyer.email, "role": "member"}) + case = ada.request("POST", "/api/v1/cases", json={"title": "Vermietung"}, status=201) + shared = ada.request("POST", "/api/v1/matters", json={"title": "Mietminderung"}, status=201) + hidden = ada.request("POST", "/api/v1/matters", json={"title": "Eigenbedarf intern"}, status=201) + for matter in (shared, hidden): + ada.request("POST", f"/api/v1/matters/{matter['id']}/case-links", json={"case_id": case["id"]}, status=201) + note_on(ada, "Schreiben des Mieters", "Minderung", case["id"]) + _share(ada, "matters", shared["id"], team["id"], access="edit") + + lawyer.request("GET", f"/api/v1/cases/{case['id']}", status=404) + assert lawyer.request("GET", "/api/v1/cases")["items"] == [] + listed = {item["title"] for item in lawyer.request("GET", "/api/v1/matters")["items"]} + assert listed == {"Mietminderung"} + lawyer.request("GET", f"/api/v1/matters/{hidden['id']}", status=404) + assert lawyer.request("GET", f"/api/v1/matters/{shared['id']}")["rights"]["edit"] is True + + +def test_the_share_caps_the_role_and_several_rooms_add_up(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + other = ada.request("POST", "/api/v1/rooms", json={"name": "Zweites Team", "default_visibility": "private"}, status=201) + ada.request("POST", f"/api/v1/rooms/{other['id']}/members", json={"email": ben.email, "role": "member"}) + case = ada.request("POST", "/api/v1/cases", json={"title": "Begrenzt"}, status=201) + _share(ada, "cases", case["id"], room, access="read") + assert ben.request("GET", f"/api/v1/cases/{case['id']}")["rights"]["edit"] is False + _share(ada, "cases", case["id"], other["id"], access="edit") + assert ben.request("GET", f"/api/v1/cases/{case['id']}")["rights"]["edit"] is True + _unshare(ada, "cases", case["id"], other["id"]) + assert ben.request("GET", f"/api/v1/cases/{case['id']}")["rights"]["edit"] is False + _unshare(ada, "cases", case["id"], other["id"], status=404) + + +def test_only_the_owner_shares_and_only_into_rooms_where_he_writes(world, db): + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Nur Ada teilt"}, status=201) + ben.request("PUT", f"/api/v1/cases/{case['id']}/room-share/rooms/{room}", json={"access": "edit"}, status=404) + ben.request("DELETE", f"/api/v1/cases/{case['id']}/room-share/rooms/{room}", status=404) + bens = ben.request("GET", "/api/v1/auth/session") # Ben may share his own entries into rooms he writes in + assert any(r["id"] == room for r in bens["rooms"]) + ben.use_room(room) + own = ben.request("POST", "/api/v1/cases", json={"title": "Bens"}, status=201) + # A reader may not share into the room; an outsider cannot even see it. + cem.use_room(room) + cem_own = Actor(cem.client, cem.email, cem.name) + cem_own.login() + mine = cem_own.request("POST", "/api/v1/cases", json={"title": "Von Cem"}, status=201) + cem_own.request("PUT", f"/api/v1/cases/{mine['id']}/room-share/rooms/{room}", json={"access": "edit"}, status=403) + dan_own = Actor(dan.client, dan.email, dan.name) + dan_own.login() + theirs = dan_own.request("POST", "/api/v1/cases", json={"title": "Von Dan"}, status=201) + dan_own.request("PUT", f"/api/v1/cases/{theirs['id']}/room-share/rooms/{room}", json={"access": "edit"}, status=403) + assert own["title"] == "Bens" + + +def test_leaving_the_team_takes_the_own_shares_along(world, db): + ada, ben, cem, room = world["ada"], world["ben"], world["cem"], world["room"] + ben.use_room(room) + bens = ben.request("POST", "/api/v1/cases", json={"title": "Bens Freigabe"}, status=201) + cem.request("GET", f"/api/v1/cases/{bens['id']}") + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{_id(db, ben.email)}", json={"role": "reader"}) + cem.request("GET", f"/api/v1/cases/{bens['id']}", status=404) + ben.request("GET", f"/api/v1/cases/{bens['id']}") + + +def test_the_bulk_action_and_the_exposure_count_own_entries(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + old = ada.request("POST", "/api/v1/cases", json={"title": "Bestand"}, status=201) + db.execute("DELETE FROM room_shares WHERE resource_id = %s", (old["id"],)) + db.commit() + ben.request("GET", f"/api/v1/cases/{old['id']}", status=404) + assert ada.request("GET", f"/api/v1/rooms/{room}/exposure")["cases"] == {"room": 0, "private": 1} + assert ada.request("POST", f"/api/v1/rooms/{room}/my-content/visibility", json={"visibility": "room"}) == {"changed": 1} + ben.request("GET", f"/api/v1/cases/{old['id']}") + assert ada.request("GET", f"/api/v1/rooms/{room}/exposure")["cases"] == {"room": 1, "private": 0} + ben.use_room(room) + theirs = ben.request("POST", "/api/v1/cases", json={"title": "Bens"}, status=201) + ada.request("POST", f"/api/v1/rooms/{room}/my-content/visibility", json={"visibility": "private"}) + ben.request("GET", f"/api/v1/cases/{old['id']}", status=404) + ben.request("GET", f"/api/v1/cases/{theirs['id']}") + world["dan"].request("GET", f"/api/v1/rooms/{room}/exposure", status=403) + + +def test_a_room_can_start_without_sharing(client, db): + ada = _actor(client, db, "ada", "ada-start@example.test", main=True) + ben = _actor(client, db, "ben", "ben-start@example.test") + room = ada.request("POST", "/api/v1/rooms", json={"name": "Privat", "default_visibility": "private"}, status=201) + ada.use_room(room["id"]) + case = ada.request("POST", "/api/v1/cases", json={"title": "Nur ich"}, status=201) + ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": ben.email, "role": "member"}) + ben.request("GET", f"/api/v1/cases/{case['id']}", status=404) + ada.request("PATCH", f"/api/v1/rooms/{room['id']}", json={"default_visibility": "room"}) + other = ada.request("POST", "/api/v1/cases", json={"title": "Jetzt geteilt"}, status=201) + ben.request("GET", f"/api/v1/cases/{other['id']}") + ben.request("PATCH", f"/api/v1/rooms/{room['id']}", json={"default_visibility": "private"}, status=403) + + +def test_lists_tell_the_owner_what_is_shared_and_rooms_are_managed_from_any_room(world, db): + ada, ben, room = world["ada"], world["ben"], world["room"] + case = ada.request("POST", "/api/v1/cases", json={"title": "Liste"}, status=201) + item = next(i for i in ada.request("GET", "/api/v1/cases")["items"] if i["id"] == case["id"]) + assert item["is_owner"] is True and [s["name"] for s in item["shared_with"]] == ["Werkstatt"] + seen = next(i for i in ben.request("GET", "/api/v1/cases")["items"] if i["id"] == case["id"]) + assert seen["is_owner"] is False and seen["shared_with"] == [] + matter = matter_of(ada, "Listenvorgang") + row = next(i for i in ada.request("GET", "/api/v1/matters")["items"] if i["id"] == matter["id"]) + assert row["is_owner"] is True and [s["name"] for s in row["shared_with"]] == ["Werkstatt"] + + # The owner manages the team while working in the personal room, without switching. + personal = Actor(ada.client, ada.email, ada.name) + body = personal.login() + assert body["scope"]["kind"] == "personal" + members = personal.request("GET", f"/api/v1/rooms/{room}/members")["items"] + assert {m["role"] for m in members} == {"manager", "member", "reader"} + personal.request("PATCH", f"/api/v1/rooms/{room}/members/{_id(db, ben.email)}", json={"role": "reader"}) + personal.request("PATCH", f"/api/v1/rooms/{room}", json={"name": "Werkstatt neu"}) + assert [r["name"] for r in personal.request("GET", "/api/v1/rooms")["items"] if r["id"] == room] == ["Werkstatt neu"] + + +def test_room_endpoints_are_closed_to_outsiders_and_do_not_reveal_which_ids_exist(world, db): + import uuid + + ada, ben, cem, dan, room = world["ada"], world["ben"], world["cem"], world["dan"], world["room"] + foreign = str(uuid.uuid4()) + dan_id = _id(db, dan.email) + ben_id = _id(db, ben.email) + attempts = [ + ("GET", "/members", None), + ("POST", "/members", {"email": dan.email, "role": "manager"}), + ("PATCH", f"/members/{ben_id}", {"role": "manager"}), + ("DELETE", f"/members/{ben_id}", None), + ("GET", "/invitations", None), + ("PATCH", "", {"name": "Übernommen"}), + ("POST", "/owner", {"subject_id": dan_id}), + ("GET", "/exposure", None), + ("POST", "/my-content/visibility", {"visibility": "room"}), + ] + for method, suffix, body in attempts: + # Dan is no member, Cem is a reader, Ben a plain member: none of them administers the room. + for actor in (dan, cem, ben): + if method == "GET" and suffix == "/exposure" and actor in (cem, ben): + continue # members may read their own exposure counts + if method == "POST" and suffix == "/my-content/visibility" and actor is ben: + continue # members may share their own entries + actor.request(method, f"/api/v1/rooms/{room}{suffix}", json=body, status=403) + # A room that does not exist answers like a room that is closed to the caller. + dan.request(method, f"/api/v1/rooms/{foreign}{suffix.replace(ben_id, dan_id)}", json=body, status=403) + dan.request("POST", "/api/v1/invitations", json={"email": "x@example.test", "room_id": room}, status=403) + dan.request("POST", "/api/v1/invitations", json={"email": "x@example.test", "room_id": foreign}, status=403) + dan.request("GET", "/api/v1/admin/rooms", status=403) + dan.request("POST", f"/api/v1/admin/rooms/{room}/owner", json={"email": dan.email}, status=403) + dan.request("GET", "/api/v1/admin/events", status=403) + # Nothing changed. + members = ada.request("GET", f"/api/v1/rooms/{room}/members")["items"] + assert {m["email"]: m["role"] for m in members}[ben.email] == "member" + assert dan.email not in {m["email"] for m in members} + assert db.execute("SELECT name FROM data_rooms WHERE id = %s", (room,)).fetchone()["name"] == "Werkstatt" + db.rollback() diff --git a/backend/tests/test_room_roles.py b/backend/tests/test_room_roles.py new file mode 100644 index 0000000..08c7743 --- /dev/null +++ b/backend/tests/test_room_roles.py @@ -0,0 +1,249 @@ +"""Datenraumrollen (Mitglied, Verwalter, Leser), Einladungen, Betreiberübersicht.""" + +from __future__ import annotations + +from fastapi.testclient import TestClient + +from tests.support import ORIGIN, PASSWORD, Actor, register + + +def _second_client(): + from app.main import app + + return TestClient(app) + + +def _setup(client, db): + ada = Actor(client, "ada-rolle@example.test", "Ada Rolle") + ben = Actor(_second_client(), "ben-rolle@example.test", "Ben Rolle") + cem = Actor(_second_client(), "cem-rolle@example.test", "Cem Rolle") + for actor in (ada, ben, cem): + register(db, actor.email, actor.name) + actor.login() + room = ada.request("POST", "/api/v1/rooms", json={"name": "Verein"}, status=201) + ada.use_room(room["id"]) + return ada, ben, cem, room["id"] + + +def test_manager_sets_roles_and_a_reader_creates_nothing(client, db): + ada, ben, cem, room = _setup(client, db) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "reader"}) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": cem.email, "role": "member"}) + members = {item["email"]: item["role"] for item in ada.request("GET", f"/api/v1/rooms/{room}/members")["items"]} + assert members == {ada.email: "manager", ben.email: "reader", cem.email: "member"} + + ben.use_room(room) + cem.use_room(room) + refused = ben.request("POST", "/api/v1/cases", json={"title": "Nicht erlaubt"}, status=403) + assert "Leser" in refused["detail"] + ben.request("POST", "/api/v1/matters", json={"title": "Auch nicht"}, status=403) + cem.request("POST", "/api/v1/cases", json={"title": "Erlaubt"}, status=201) + + # The role is read from the database at every request. + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{_subject(ada, room, ben.email)}", json={"role": "member"}) + ben.request("POST", "/api/v1/cases", json={"title": "Jetzt erlaubt"}, status=201) + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{_subject(ada, room, ben.email)}", json={"role": "reader"}) + ben.request("POST", "/api/v1/cases", json={"title": "Wieder nicht"}, status=403) + + +def test_a_reader_gets_no_edit_grant_and_loses_existing_ones(client, db): + ada, ben, cem, room = _setup(client, db) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "member"}) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": cem.email, "role": "reader"}) + case = ada.request("POST", "/api/v1/cases", json={"title": "Gemeinsam"}, status=201) + refused = ada.request( + "POST", + f"/api/v1/cases/{case['id']}/grants", + json={"email": cem.email, "can_read": True, "can_edit": True, "can_share": False}, + status=422, + ) + assert "Leser" in refused["detail"] + ada.request( + "POST", + f"/api/v1/cases/{case['id']}/grants", + json={"email": cem.email, "can_read": True, "can_edit": False, "can_share": False}, + status=201, + ) + ada.request( + "POST", + f"/api/v1/cases/{case['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": True, "can_share": True}, + status=201, + ) + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{_subject(ada, room, ben.email)}", json={"role": "reader"}) + grant = db.execute( + "SELECT can_read, can_edit, can_share FROM access_grants WHERE grantee_subject_id = (SELECT id FROM auth_accounts WHERE email = %s)", + (ben.email,), + ).fetchone() + db.rollback() + assert grant == {"can_read": True, "can_edit": False, "can_share": False} + + +def test_the_last_manager_stays(client, db): + ada, ben, _cem, room = _setup(client, db) + me = _subject(ada, room, ada.email) + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{me}", json={"role": "member"}, status=422) + ada.request("DELETE", f"/api/v1/rooms/{room}/members/{me}", status=422) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ada.email, "role": "reader"}, status=422) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "manager"}) + # Ada owns the room and stays manager until she hands it over. + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{me}", json={"role": "member"}, status=422) + ada.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": _subject(ada, room, ben.email)}) + ada.request("PATCH", f"/api/v1/rooms/{room}/members/{me}", json={"role": "member"}) + # Ben is owner and the only manager now. + ben.use_room(room) + ben.request("DELETE", f"/api/v1/rooms/{room}/members/{_subject(ben, room, ben.email)}", status=422) + ada.request("GET", f"/api/v1/rooms/{room}/members", status=403) + + +def test_owner_transfer_and_operator_step_in(client, db): + ada, ben, cem, room = _setup(client, db) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "member"}) + ben.use_room(room) + ben_id = _subject(ben, room, ben.email) + ben.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": ben_id}, status=403) + ada.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": _subject(ada, room, cem.email)}, status=422) + ada.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": _subject(ada, room, ada.email)}, status=422) + # Ada cannot leave or be demoted while she owns the room, and cannot be disabled while members depend on it. + ada_id = _subject(ada, room, ada.email) + db.execute("UPDATE auth_accounts SET origin = 'operator' WHERE id = %s", (ada_id,)) + db.commit() + operator = Actor(_second_client(), "betreiber-eigner@example.test", "Betreiber Eigner") + register(db, operator.email, operator.name, superadmin=True) + operator.login() + operator.request("PATCH", f"/api/v1/accounts/{ada_id}", json={"status": "disabled"}, status=422) + operator.request("POST", f"/api/v1/admin/rooms/{room}/owner", json={"email": ben.email}, status=403) + + ada.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": ben_id}) + ben.request("GET", f"/api/v1/rooms/{room}/members") + owners = [r for r in ben.request("GET", "/api/v1/auth/session")["rooms"] if r["id"] == room] + assert owners[0]["is_owner"] is True + ada.request("PATCH", f"/api/v1/rooms/{room}", json={"status": "archived"}, status=403) + ben.request("PATCH", f"/api/v1/rooms/{room}", json={"default_visibility": "private"}) + + # The old owner may be disabled now; the room keeps its owner. A disabled owner is replaced by the operator. + operator.request("PATCH", f"/api/v1/accounts/{ada_id}", json={"status": "disabled"}) + db.execute("UPDATE data_rooms SET owner_id = %s WHERE id = %s", (ada_id, room)) + db.commit() + stepped = operator.request("POST", f"/api/v1/admin/rooms/{room}/owner", json={"email": ben.email}) + assert stepped["owner_id"] == ben_id + events = operator.request("GET", "/api/v1/admin/events", params={"limit": 100})["items"] + assert "room_owner_changed" in [item["action"] for item in events] + + +def test_a_removed_member_loses_the_room_at_once(client, db): + ada, ben, _cem, room = _setup(client, db) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "member"}) + ben.use_room(room) + ben.request("GET", "/api/v1/cases") + ada.request("DELETE", f"/api/v1/rooms/{room}/members/{_subject(ada, room, ben.email)}", status=204) + ben.request("GET", "/api/v1/cases", status=403) + + +def test_invitations_carry_a_role_and_can_be_revoked(client, db): + ada, ben, _cem, room = _setup(client, db) + invited = ada.request( + "POST", + "/api/v1/invitations", + json={"email": "neu-rolle@example.test", "room_id": room, "role": "reader"}, + status=201, + ) + assert invited["role"] == "reader" and invited["token"] + ada.request("POST", "/api/v1/invitations", json={"email": ben.email, "room_id": room}, status=422) + open_items = ada.request("GET", f"/api/v1/rooms/{room}/invitations")["items"] + assert [item["email"] for item in open_items] == ["neu-rolle@example.test"] + assert "token" not in open_items[0] + ben.request("GET", f"/api/v1/rooms/{room}/invitations", status=403) + + guest = _second_client() + accepted = guest.post( + f"/api/v1/invitations/{invited['id']}/accept", + json={"token": invited["token"], "display_name": "Neu Gast", "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert accepted.status_code == 201, accepted.text + assert [(r["name"], r["role"]) for r in accepted.json()["rooms"] if r["kind"] == "organization"] == [("Verein", "reader")] + again = _second_client().post( + f"/api/v1/invitations/{invited['id']}/accept", + json={"token": invited["token"], "display_name": "Zweiter", "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert again.status_code == 404 + + second = ada.request("POST", "/api/v1/invitations", json={"email": "widerruf@example.test", "room_id": room}, status=201) + ada.request("DELETE", f"/api/v1/rooms/{room}/invitations/{second['id']}", status=204) + refused = _second_client().post( + f"/api/v1/invitations/{second['id']}/accept", + json={"token": second["token"], "display_name": "Widerruf", "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert refused.status_code == 404 + ada.request("DELETE", f"/api/v1/rooms/{room}/invitations/{second['id']}", status=404) + + +def test_rooms_are_renamed_archived_and_restored_by_a_manager(client, db): + ada, ben, _cem, room = _setup(client, db) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "member"}) + ben.request("PATCH", f"/api/v1/rooms/{room}", json={"name": "Fremd"}, status=403) + renamed = ada.request("PATCH", f"/api/v1/rooms/{room}", json={"name": "Verein e.V."}) + assert renamed["name"] == "Verein e.V." + ada.request("PATCH", f"/api/v1/rooms/{room}", json={"status": "archived"}) + # The session of the archiving manager moved back to the personal room. + ada.request("GET", "/api/v1/rooms", status=403) + body = ada.request("GET", "/api/v1/auth/session") + assert body["scope"]["kind"] == "personal" + ada.scope, ada.csrf = body["scope"], body["csrf_token"] + listed = ada.request("GET", "/api/v1/rooms", params={"include_archived": "true"})["items"] + assert any(item["id"] == room and item.get("status") == "archived" for item in listed) + ada.request("POST", "/api/v1/auth/session/scope", json={"kind": "organization", "id": room}, status=403) + ada.request("PATCH", f"/api/v1/rooms/{room}", json={"status": "active"}) + ada.use_room(room) + + +def test_operator_sees_rooms_without_content_and_fills_an_orphaned_room(client, db): + ada, ben, _cem, room = _setup(client, db) + operator = Actor(_second_client(), "betreiber-rolle@example.test", "Betreiber Rolle") + register(db, operator.email, operator.name, superadmin=True) + operator.login() + ada.request("POST", "/api/v1/cases", json={"title": "Nur für Ada"}, status=201) + overview = operator.request("GET", "/api/v1/admin/rooms")["items"] + row = next(item for item in overview if item["id"] == room) + assert row["managers"] == ["Ada Rolle"] and row["members"] == 1 and row["has_manager"] is True + assert row["owner"] == "Ada Rolle" and row["owner_active"] is True + assert "Nur für Ada" not in str(overview) + operator.request("GET", f"/api/v1/rooms/{room}/members", status=403) + ben.request("GET", "/api/v1/admin/rooms", status=403) + operator.request("POST", f"/api/v1/admin/rooms/{room}/managers", json={"email": ben.email}, status=403) + + db.execute("UPDATE auth_accounts SET status = 'disabled' WHERE email = %s", (ada.email,)) + db.commit() + assigned = operator.request("POST", f"/api/v1/admin/rooms/{room}/managers", json={"email": ben.email}) + assert assigned["role"] == "manager" + ben.use_room(room) + assert {item["email"] for item in ben.request("GET", f"/api/v1/rooms/{room}/members")["items"]} >= {ben.email} + + +def test_room_events_reach_the_operator_audit_without_secrets(client, db): + ada, ben, _cem, room = _setup(client, db) + operator = Actor(_second_client(), "audit-rolle@example.test", "Audit Rolle") + register(db, operator.email, operator.name, superadmin=True) + operator.login() + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "reader"}) + invited = ada.request("POST", "/api/v1/invitations", json={"email": "audit-gast@example.test", "room_id": room}, status=201) + events = operator.request("GET", "/api/v1/admin/events", params={"limit": 50}) + actions = [item["action"] for item in events["items"]] + assert {"room_created", "membership_added", "invitation_created"} <= set(actions) + text = str(events) + assert "audit-gast@example.test" not in text and invited["token"] not in text + ben.request("GET", "/api/v1/admin/events", status=403) + page = operator.request("GET", "/api/v1/admin/events", params={"limit": 1}) + assert len(page["items"]) == 1 and page["next_cursor"] == 1 + + +def _subject(actor: Actor, room: str, email: str) -> str: + # A manager lists the members; Ada manages in these tests. Fall back to the account table. + from app.core.settings import get_settings + import psycopg + + with psycopg.connect(get_settings().dsn(), row_factory=psycopg.rows.dict_row) as conn: + return str(conn.execute("SELECT id FROM auth_accounts WHERE email = %s", (email,)).fetchone()["id"]) diff --git a/backend/tests/test_sample_source.py b/backend/tests/test_sample_source.py new file mode 100644 index 0000000..bb969d0 --- /dev/null +++ b/backend/tests/test_sample_source.py @@ -0,0 +1,226 @@ +"""A third source appears only by registering its module.""" + +from __future__ import annotations + +import io +import zipfile +from pathlib import Path + +import pytest + +from tests.sample_source import install_sample_source, remove_sample_source +from tests.support import ORIGIN, Actor, document, register + +ROOT = Path(__file__).resolve().parents[2] +TEXT = "Der Probenbrief bleibt lesbar." + + +@pytest.fixture +def sample_source(): + install_sample_source() + try: + yield + finally: + remove_sample_source() + + +def test_registered_sample_source_links_reads_and_exports(client, db, sample_source): + del sample_source + ada = Actor(client, "ada-sample@example.test", "Ada Probe") + ben = Actor(client, "ben-sample@example.test", "Ben Probe") + register(db, ada.email, ada.name) + register(db, ben.email, ben.name) + ada.login() + refused = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "missing", "name": "Unbekannt"}, + status=422, + ) + assert refused["code"] == "unprocessable" + ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "sample", "name": "Ohne Katalog", "config": {"catalog": "kein-katalog"}}, + status=422, + ) + connector_id = ada.request( + "POST", + "/api/v1/connectors", + json={"kind": "sample", "name": "Katalog Alpha", "config": {"catalog": "alpha"}}, + status=201, + )["id"] + probed = ada.request("POST", f"/api/v1/connectors/{connector_id}/probe") + assert probed["ok"] is True + assert "2 Einträge" in probed["detail"] + blocked = ada.request( + "GET", + "/api/v1/sources/sample/search", + params={"q": "Probe", "correspondent": "geheim-wert"}, + status=422, + ) + assert blocked["code"] == "unsupported" + assert "geheim-wert" not in blocked["detail"] + ada.request("GET", "/api/v1/sources/sample/navigate", params={"connector_id": connector_id}, status=422) + found = ada.request("GET", "/api/v1/sources/sample/search", params={"q": "Proben", "connector_id": connector_id}) + assert [item["id"] for item in found["items"]] == ["brief"] + preview = ada.request( + "GET", + "/api/v1/sources/sample/preview", + params={"connector_id": connector_id, "item": "brief"}, + ) + assert preview["text"] == TEXT + matter = ada.request("POST", "/api/v1/matters", json={"title": "Probenvorgang"}, status=201) + linked = ada.request( + "POST", + "/api/v1/external-refs", + json={ + "title": "Probenbrief", + "source": "sample", + "external_id": "brief", + "context": {"kind": "matter", "id": matter["id"]}, + "detail": {"connector_id": connector_id}, + }, + status=201, + ) + opened = _raw(ada, f"/api/v1/external-refs/{linked['id']}/content?context_kind=matter&context_id={matter['id']}") + assert opened.status_code == 200 + assert opened.content == TEXT.encode("utf-8") + assert opened.headers["content-type"].startswith("text/plain") + detail = ada.request("GET", f"/api/v1/matters/{matter['id']}") + element = next(item["element"] for item in detail["bindings"] if item["element"]["id"] == linked["id"]) + assert element["preview"]["media"] == "text" + written = ada.request( + "PATCH", + f"/api/v1/notes/{linked['id']}", + json={ + "expected_version": element["version"], + "document": document("Das darf nicht geschrieben werden."), + "context": {"kind": "matter", "id": matter["id"]}, + }, + status=422, + ) + assert written["code"] == "unsupported" + tree = ada.request("GET", f"/api/v1/matters/{matter['id']}/export-tree")["tree"] + selected = [node["id"] for node in _walk(tree) if node["kind"] == "binding" and node.get("exportable")] + body = {"context": {"kind": "matter", "id": matter["id"]}, "selected": selected} + review = ada.request("POST", f"/api/v1/matters/{matter['id']}/export/review", json=body) + assert review["ready"] is True, review + package = _post_zip(ada, matter["id"], body) + assert package.status_code == 200, package.text + archive = zipfile.ZipFile(io.BytesIO(package.content)) + assert any(archive.read(name) == TEXT.encode("utf-8") for name in archive.namelist()) + + ben.login() + ben.request( + "GET", + "/api/v1/sources/sample/search", + params={"connector_id": connector_id, "q": "Proben"}, + status=404, + ) + hidden = _raw(ben, f"/api/v1/external-refs/{linked['id']}/content?context_kind=matter&context_id={matter['id']}") + assert hidden.status_code == 404 + ada.login() + ada.request( + "POST", + f"/api/v1/matters/{matter['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False, "use_owner_sources": False}, + status=201, + ) + ben.login() + denied = _raw(ben, f"/api/v1/external-refs/{linked['id']}/content?context_kind=matter&context_id={matter['id']}") + assert denied.status_code == 422 + assert denied.json()["detail"] == "Das Original ist über die Freigabe nicht abrufbar." + ada.login() + grant_id = next(item["id"] for item in ada.request("GET", f"/api/v1/matters/{matter['id']}")["grants"]) + ada.request("DELETE", f"/api/v1/grants/{grant_id}", status=204) + ada.request( + "POST", + f"/api/v1/matters/{matter['id']}/grants", + json={"email": ben.email, "can_read": True, "can_edit": False, "can_share": False, "use_owner_sources": True}, + status=201, + ) + ben.login() + shared = _raw(ben, f"/api/v1/external-refs/{linked['id']}/content?context_kind=matter&context_id={matter['id']}") + assert shared.status_code == 200 + assert shared.content == TEXT.encode("utf-8") + shown = ben.request("GET", f"/api/v1/matters/{matter['id']}") + visible = next(item["element"] for item in shown["bindings"] if item["element"]["id"] == linked["id"]) + assert "connector_id" not in (visible["external"].get("detail") or {}) + ada.login() + grant_id = next(item["id"] for item in ada.request("GET", f"/api/v1/matters/{matter['id']}")["grants"]) + ada.request("DELETE", f"/api/v1/grants/{grant_id}", status=204) + ben.login() + revoked = _raw(ben, f"/api/v1/external-refs/{linked['id']}/content?context_kind=matter&context_id={matter['id']}") + assert revoked.status_code == 404 + + +def test_sample_registration_does_not_edit_generic_modules(): + forbidden = [ + "backend/app/api/schemas.py", + "backend/app/exporting.py", + "backend/app/present.py", + "frontend/src/pages/Cases.jsx", + "frontend/src/pages/Matters.jsx", + "frontend/src/pages/ReadingPane.jsx", + "frontend/src/pages/ExportPackage.jsx", + "frontend/src/App.jsx", + "frontend/src/pages/Sources.jsx", + ] + for relative in forbidden: + text = (ROOT / relative).read_text(encoding="utf-8") + assert '"sample"' not in text and "'sample'" not in text, relative + # The backend image that runs this suite has no Node. The sample module is + # small enough to check against the registry operations it calls. + sample = (ROOT / "frontend/src/sources/sample-source.js").read_text(encoding="utf-8") + registry = (ROOT / "frontend/src/sources/registry-core.js").read_text(encoding="utf-8") + assert "registerSourceModule" in sample and "unregisterSourceModule" in sample + assert 'kind: "sample"' in sample and 'representation: "text"' in sample + assert 'unregisterSourceModule("sample")' in sample + assert "modules.set(module.kind, module)" in registry + assert "modules.delete(kind)" in registry + modules: dict[str, dict] = {} + + def register(module: dict) -> dict: + modules[module["kind"]] = module + return module + + def unregister(kind: str) -> None: + modules.pop(kind, None) + + register({"kind": "sample", "representation": "text"}) + assert any(item["kind"] == "sample" and item["representation"] == "text" for item in modules.values()) + unregister("sample") + assert not any(item["kind"] == "sample" for item in modules.values()) + + +def _raw(actor: Actor, path: str): + return actor.client.get( + path, + headers={ + "Origin": ORIGIN, + "X-CSRF-Token": actor.csrf, + "X-Scope-Kind": actor.scope["kind"], + "X-Scope-Id": actor.scope["id"], + }, + ) + + +def _post_zip(actor: Actor, matter_id: str, body: dict): + return actor.client.post( + f"/api/v1/matters/{matter_id}/export", + json=body, + headers={ + "Origin": ORIGIN, + "X-CSRF-Token": actor.csrf, + "Content-Type": "application/json", + "X-Scope-Kind": actor.scope["kind"], + "X-Scope-Id": actor.scope["id"], + }, + ) + + +def _walk(node): + yield node + for child in node.get("children") or []: + yield from _walk(child) diff --git a/backend/tests/test_team.py b/backend/tests/test_team.py new file mode 100644 index 0000000..0b373f7 --- /dev/null +++ b/backend/tests/test_team.py @@ -0,0 +1,220 @@ +"""Hauptnutzer, Mitarbeiter und Teamverwaltung.""" + +from __future__ import annotations + +import uuid + +import pytest +from fastapi.testclient import TestClient + +from app.auth.accounts import create_account +from app.core.errors import unprocessable +from tests.support import ORIGIN, PASSWORD, Actor + + +def _second_client(): + from app.main import app + + return TestClient(app) + + +def _main(client, db, key: str, *, main: bool = False, superadmin: bool = False) -> Actor: + actor = Actor(client if main else _second_client(), f"{key}-team@example.test", key.title()) + create_account(db, actor.email, actor.name, PASSWORD, is_superadmin=superadmin) + db.commit() + actor.login() + return actor + + +def _invite(owner: Actor, room: str, email: str, role: str = "member", name: str = "Mitarbeiter") -> Actor: + invited = owner.request("POST", "/api/v1/invitations", json={"email": email, "room_id": room, "role": role}, status=201) + client = _second_client() + response = client.post( + f"/api/v1/invitations/{invited['id']}/accept", + json={"token": invited["token"], "display_name": name, "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert response.status_code == 201, response.text + employee = Actor(client, email, name) + employee.login() + return employee + + +def _id(db, email: str) -> str: + row = db.execute("SELECT id FROM auth_accounts WHERE email = %s", (email,)).fetchone() + db.rollback() + return str(row["id"]) + + +@pytest.fixture +def world(client, db): + ada = _main(client, db, "ada", main=True) + room = ada.request("POST", "/api/v1/rooms", json={"name": "Kanzlei"}, status=201)["id"] + ada.use_room(room) + employee = _invite(ada, room, "emp-team@example.test", name="Eva Mitarbeiterin") + return {"ada": ada, "room": room, "emp": employee} + + +def test_an_invitation_creates_an_employee_account_sponsored_by_the_owner(world, db): + row = db.execute("SELECT account_kind, sponsor_id FROM auth_accounts WHERE email = 'emp-team@example.test'").fetchone() + db.rollback() + assert row["account_kind"] == "employee" + assert str(row["sponsor_id"]) == _id(db, world["ada"].email) + assert world["emp"].request("GET", "/api/v1/auth/session")["account"]["account_kind"] == "employee" + assert world["ada"].request("GET", "/api/v1/auth/session")["account"]["account_kind"] == "main" + + +def test_an_employee_works_only_in_the_rooms_he_was_invited_to(world, db): + emp, room = world["emp"], world["room"] + emp.request("POST", "/api/v1/rooms", json={"name": "Eigener Raum"}, status=403) + # The session starts in the team room, not in a personal room. + assert emp.scope["kind"] == "organization" and emp.scope["id"] == room + # In the personal room nothing is created. + personal = emp.request("POST", "/api/v1/auth/session/scope", json={"kind": "personal", "id": _id(db, emp.email)}) + emp.scope, emp.csrf = personal["scope"], personal["csrf_token"] + emp.request("POST", "/api/v1/cases", json={"title": "Privat"}, status=403) + emp.request("POST", "/api/v1/matters", json={"title": "Privat"}, status=403) + emp.use_room(room) + case = emp.request("POST", "/api/v1/cases", json={"title": "Im Team"}, status=201) + world["ada"].request("GET", f"/api/v1/cases/{case['id']}") + # Rights follow the role: a reader only reads. + world["ada"].request("PATCH", f"/api/v1/rooms/{room}/members/{_id(db, emp.email)}", json={"role": "reader"}) + emp.request("POST", "/api/v1/cases", json={"title": "Nicht mehr"}, status=403) + emp.request("GET", f"/api/v1/cases/{case['id']}") + + +def test_only_the_owner_invites_and_an_employee_never_does(world, client, db): + ada, emp, room = world["ada"], world["emp"], world["room"] + ben = _main(client, db, "ben") + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": ben.email, "role": "manager"}) + ben.use_room(room) + # A manager who is not the owner adds existing accounts but creates no new ones. + ben.request("POST", "/api/v1/invitations", json={"email": "neu-team@example.test", "room_id": room}, status=403) + ben.request("POST", f"/api/v1/rooms/{room}/members", json={"email": emp.email, "role": "reader"}) + emp.request("POST", "/api/v1/invitations", json={"email": "neu-team@example.test", "room_id": room}, status=403) + + +def test_an_employee_cannot_own_a_room(world, db): + ada, emp, room = world["ada"], world["emp"], world["room"] + ada.request("POST", f"/api/v1/rooms/{room}/owner", json={"subject_id": _id(db, emp.email)}, status=422) + operator = _main(world["ada"].client.__class__ and _second_client(), db, "betreiber", superadmin=True) + db.execute("UPDATE auth_accounts SET status = 'disabled' WHERE id = %s", (_id(db, ada.email),)) + db.commit() + operator.request("POST", f"/api/v1/admin/rooms/{room}/owner", json={"email": emp.email}, status=422) + + +def test_my_team_lists_only_people_in_my_rooms_and_acts_only_on_my_employees(world, client, db): + ada, emp = world["ada"], world["emp"] + cara = _main(client, db, "cara") + emp_id = _id(db, emp.email) + listed = ada.request("GET", "/api/v1/team")["items"] + assert [(p["email"], p["mine"], p["rooms"][0]["role"]) for p in listed] == [(emp.email, True, "member")] + assert cara.request("GET", "/api/v1/team")["items"] == [] + emp.request("GET", "/api/v1/team", status=403) + unknown = str(uuid.uuid4()) + for actor in (cara,): + foreign = actor.request("PATCH", f"/api/v1/team/{emp_id}", json={"status": "disabled"}, status=403) + missing = actor.request("PATCH", f"/api/v1/team/{unknown}", json={"status": "disabled"}, status=403) + assert foreign["detail"] == missing["detail"] + actor.request("POST", f"/api/v1/team/{emp_id}/password-reset", status=403) + actor.request("DELETE", f"/api/v1/team/{emp_id}/memberships", status=403) + # A main user is not a sponsored employee: no team action on him, not even by the owner. + cara_id = _id(db, cara.email) + ada.request("PATCH", f"/api/v1/team/{cara_id}", json={"status": "disabled"}, status=403) + ada.request("POST", f"/api/v1/team/{cara_id}/password-reset", status=403) + + +def test_the_owner_disables_resets_and_removes_an_employee(world, db): + ada, emp, room = world["ada"], world["emp"], world["room"] + emp_id = _id(db, emp.email) + ada.request("PATCH", f"/api/v1/team/{emp_id}", json={"status": "disabled"}) + emp.request("GET", "/api/v1/auth/session", status=401) + login = emp.client.post("/api/v1/auth/login", json={"email": emp.email, "password": PASSWORD}, headers={"Origin": ORIGIN}) + assert login.status_code == 401 + ada.request("PATCH", f"/api/v1/team/{emp_id}", json={"status": "active"}) + reset = ada.request("POST", f"/api/v1/team/{emp_id}/password-reset") + emp.password = reset["temporary_password"] + assert emp.login()["account"]["must_change_password"] is True + emp.request("GET", "/api/v1/cases", status=403) + + ada.request("DELETE", f"/api/v1/team/{emp_id}/memberships", status=204) + assert ada.request("GET", "/api/v1/team")["items"][0]["rooms"] == [] + # Without a room the employee lands in the personal room, where he can do nothing but change his password. + assert emp.login()["scope"]["kind"] == "personal" + ada.request("DELETE", f"/api/v1/team/{emp_id}/memberships", status=403) + ada.request("DELETE", f"/api/v1/team/{_id(db, ada.email)}/memberships", status=403) + assert room + + +def test_the_operator_sees_the_teams_as_metadata_and_exports_them_safely(world, client, db): + ada, room = world["ada"], world["room"] + _invite(ada, room, "formel-team@example.test", role="reader", name="=SUM(1+1)") + operator = _main(_second_client(), db, "betreiber", superadmin=True) + note = ada.request("POST", "/api/v1/cases", json={"title": "Geheimer Inhalt"}, status=201) + overview = operator.request("GET", "/api/v1/admin/team")["items"] + owner = next(item for item in overview if item["email"] == ada.email) + assert {e["email"] for e in owner["employees"]} == {"emp-team@example.test", "formel-team@example.test"} + assert all(r["name"] == "Kanzlei" for e in owner["employees"] for r in e["rooms"]) + assert "Geheimer Inhalt" not in str(overview) and note["title"] == "Geheimer Inhalt" + response = operator.client.get( + "/api/v1/admin/team", + params={"format": "csv"}, + headers={"Origin": ORIGIN, "X-Scope-Kind": operator.scope["kind"], "X-Scope-Id": operator.scope["id"]}, + ) + assert response.status_code == 200 and response.headers["content-type"].startswith("text/csv") + lines = response.text.splitlines() + assert lines[0] == "Hauptnutzer,Mitarbeiter,E-Mail,Status,Datenraum,Rolle" + assert any("'=SUM(1+1)" in line for line in lines) + assert not any(line.split(",")[1].startswith("=") for line in lines[1:]) + world["emp"].request("GET", "/api/v1/admin/team", status=403) + ada.request("GET", "/api/v1/admin/team", status=403) + + +def test_only_the_operator_promotes_an_employee(world, client, db): + ada, emp, room = world["ada"], world["emp"], world["room"] + emp_id = _id(db, emp.email) + ada.request("PATCH", f"/api/v1/accounts/{emp_id}", json={"account_kind": "main"}, status=403) + operator = _main(_second_client(), db, "betreiber", superadmin=True) + listed = {a["email"]: a for a in operator.request("GET", "/api/v1/accounts")["items"]} + assert listed[emp.email]["account_kind"] == "employee" and listed[emp.email]["sponsor_name"] == "Ada" + assert operator.request("GET", f"/api/v1/accounts/{_id(db, ada.email)}")["employees"][0]["email"] == emp.email + operator.request("PATCH", f"/api/v1/accounts/{emp_id}", json={"account_kind": "main"}) + emp.request("GET", "/api/v1/cases", status=401) + fresh = Actor(emp.client, emp.email, emp.name) + assert fresh.login()["account"]["account_kind"] == "main" + fresh.request("POST", "/api/v1/rooms", json={"name": "Eigener Raum"}, status=201) + assert room + + +def test_limits_have_one_place_and_are_checked_where_they_matter(world, client, db, monkeypatch): + ada, room = world["ada"], world["room"] + seen = [] + + def block_room(conn, owner_id): + seen.append(("room", owner_id)) + raise unprocessable("Grenze erreicht.") + + def block_member(conn, owner_id, room_id): + seen.append(("member", owner_id, room_id)) + raise unprocessable("Grenze erreicht.") + + cem = _main(client, db, "cem") + monkeypatch.setattr("app.api.rooms.ensure_can_create_room", block_room) + ada.request("POST", "/api/v1/rooms", json={"name": "Noch ein Raum"}, status=422) + monkeypatch.setattr("app.api.rooms.ensure_can_add_member", block_member) + ada.request("POST", f"/api/v1/rooms/{room}/members", json={"email": cem.email, "role": "member"}, status=422) + ada.request("POST", "/api/v1/invitations", json={"email": "grenze-team@example.test", "room_id": room}, status=422) + assert [entry[0] for entry in seen] == ["room", "member", "member"] + assert all(entry[1] == uuid.UUID(_id(db, ada.email)) for entry in seen) + # Changing the role of an existing member needs no check. + monkeypatch.undo() + invited = ada.request("POST", "/api/v1/invitations", json={"email": "spaet-team@example.test", "room_id": room}, status=201) + monkeypatch.setattr("app.api.routes.ensure_can_add_member", block_member) + refused = _second_client().post( + f"/api/v1/invitations/{invited['id']}/accept", + json={"token": invited["token"], "display_name": "Spät", "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert refused.status_code == 422 + assert db.execute("SELECT 1 FROM auth_accounts WHERE email = 'spaet-team@example.test'").fetchone() is None + db.rollback() diff --git a/backend/tests/test_tracking.py b/backend/tests/test_tracking.py index a4de25e..4fcd4f7 100644 --- a/backend/tests/test_tracking.py +++ b/backend/tests/test_tracking.py @@ -81,7 +81,7 @@ def test_t_anchor_assignee_sees_the_task_but_not_the_hidden_matter(client, db): ada = Actor(client, "ada@example.test", "Ada") ada.login() matter = ada.request("POST", "/api/v1/matters", json={"title": "Geheimer Vorgang Anker"}, status=201) - room = ada.request("POST", "/api/v1/rooms", json={"name": "Gleicher Raum"}, status=201) + room = ada.request("POST", "/api/v1/rooms", json={"name": "Gleicher Raum", "default_visibility": "private"}, status=201) ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": "ben@example.test", "role": "member"}) ada.request("POST", f"/api/v1/rooms/{room['id']}/members", json={"email": "cara@example.test", "role": "member"}) ada.use_room(room["id"]) diff --git a/backend/tests/test_user_admin.py b/backend/tests/test_user_admin.py new file mode 100644 index 0000000..29ce585 --- /dev/null +++ b/backend/tests/test_user_admin.py @@ -0,0 +1,261 @@ +"""Kontoverwaltung: Einmal-Passwort, Deaktivieren, Betreiberrolle, Sitzungen, Login-Begrenzung.""" + +from __future__ import annotations + +import pytest +from fastapi.testclient import TestClient + +from app.auth.accounts import create_account +from tests.support import ORIGIN, PASSWORD, Actor + +NEW_PASSWORD = "Neues-Passwort-2026" + + +def register(conn, email: str, name: str, *, superadmin: bool = False): + """Accounts of the operator kind, because the account list hides fixture accounts.""" + account = create_account(conn, email, name, PASSWORD, is_superadmin=superadmin) + conn.commit() + return account + + +@pytest.fixture +def operator(client, db) -> Actor: + actor = Actor(client, "betreiber-admin@example.test", "Betreiber Admin") + register(db, actor.email, actor.name, superadmin=True) + actor.login() + return actor + + +def _user(client, db, email: str = "nutzer-admin@example.test", name: str = "Nutzer Admin") -> Actor: + actor = Actor(client, email, name) + register(db, actor.email, actor.name) + return actor + + +def _second_client(): + from app.main import app + + return TestClient(app) + + +def test_operator_creates_an_account_with_a_one_time_password(client, operator): + created = operator.request( + "POST", + "/api/v1/accounts", + json={"email": "neu@example.test", "display_name": "Neu"}, + status=201, + ) + temporary = created["temporary_password"] + assert len(temporary) >= 10 + assert created["must_change_password"] is True + + fresh = Actor(client, "neu@example.test", "Neu") + fresh.password = temporary + body = fresh.login() + assert body["account"]["must_change_password"] is True + refused = fresh.request("GET", "/api/v1/cases", status=403) + assert refused["code"] == "password_change_required" + fresh.request("GET", "/api/v1/auth/session") + + fresh.request( + "POST", + "/api/v1/auth/password", + json={"current_password": "falsch-falsch-1", "new_password": NEW_PASSWORD}, + status=422, + ) + fresh.request("POST", "/api/v1/auth/password", json={"current_password": temporary, "new_password": temporary}, status=422) + fresh.request("POST", "/api/v1/auth/password", json={"current_password": temporary, "new_password": "kurz"}, status=422) + fresh.request( + "POST", + "/api/v1/auth/password", + json={"current_password": temporary, "new_password": NEW_PASSWORD}, + status=204, + ) + fresh.request("GET", "/api/v1/cases") + assert fresh.request("GET", "/api/v1/auth/session")["account"]["must_change_password"] is False + + again = Actor(client, "neu@example.test", "Neu") + again.password = NEW_PASSWORD + again.login() + old = client.post( + "/api/v1/auth/login", + json={"email": "neu@example.test", "password": temporary}, + headers={"Origin": ORIGIN}, + ) + assert old.status_code == 401 + + +def test_only_the_operator_manages_accounts(client, db, operator): + user = _user(client, db) + target = operator.request("GET", "/api/v1/accounts")["items"][0]["id"] + user.login() + user.request("GET", "/api/v1/accounts", status=403) + user.request("GET", f"/api/v1/accounts/{target}", status=403) + user.request("PATCH", f"/api/v1/accounts/{target}", json={"status": "disabled"}, status=403) + user.request("POST", f"/api/v1/accounts/{target}/password-reset", status=403) + user.request("POST", f"/api/v1/accounts/{target}/sessions/revoke", status=403) + user.request("POST", "/api/v1/accounts", json={"email": "x@example.test", "display_name": "X"}, status=403) + + +def test_a_disabled_account_cannot_sign_in_and_its_sessions_end(client, db, operator): + user = _user(client, db) + other = _second_client() + victim = Actor(other, user.email, user.name) + victim.login() + victim.request("GET", "/api/v1/cases") + + operator.request("PATCH", f"/api/v1/accounts/{_id(operator, user.email)}", json={"status": "disabled"}) + victim.request("GET", "/api/v1/cases", status=401) + refused = client.post( + "/api/v1/auth/login", + json={"email": user.email, "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert refused.status_code == 401 + + listed = operator.request("GET", "/api/v1/accounts", params={"status": "disabled"}) + assert [item["email"] for item in listed["items"]] == [user.email] + operator.request("PATCH", f"/api/v1/accounts/{_id(operator, user.email)}", json={"status": "active"}) + Actor(client, user.email, user.name).login() + + +def test_the_last_operator_and_the_own_account_are_protected(client, db, operator): + me = _id(operator, operator.email) + operator.request("PATCH", f"/api/v1/accounts/{me}", json={"status": "disabled"}, status=422) + operator.request("PATCH", f"/api/v1/accounts/{me}", json={"is_operator": False}, status=422) + user = _user(client, db) + other = _id(operator, user.email) + operator.request("PATCH", f"/api/v1/accounts/{other}", json={"is_operator": True}) + # A second operator exists now, so the first may step down. + operator.request("PATCH", f"/api/v1/accounts/{me}", json={"is_operator": False}) + promoted = Actor(client, user.email, user.name) + promoted.login() + promoted.request("PATCH", f"/api/v1/accounts/{promoted_id(promoted)}", json={"status": "disabled"}, status=422) + promoted.request("PATCH", f"/api/v1/accounts/{promoted_id(promoted)}", json={"is_operator": False}, status=422) + + +def test_granting_the_operator_role_ends_the_target_sessions(client, db, operator): + user = _user(client, db) + other = _second_client() + target = Actor(other, user.email, user.name) + target.login() + target.request("GET", "/api/v1/cases") + operator.request("PATCH", f"/api/v1/accounts/{_id(operator, user.email)}", json={"is_operator": True}) + target.request("GET", "/api/v1/cases", status=401) + fresh = Actor(other, user.email, user.name) + assert fresh.login()["account"]["is_superadmin"] is True + + +def test_password_reset_sets_a_one_time_password_and_ends_sessions(client, db, operator): + user = _user(client, db) + other = _second_client() + target = Actor(other, user.email, user.name) + target.login() + account_id = _id(operator, user.email) + operator.request("POST", f"/api/v1/accounts/{operator_id(operator)}/password-reset", status=422) + reset = operator.request("POST", f"/api/v1/accounts/{account_id}/password-reset") + target.request("GET", "/api/v1/cases", status=401) + renewed = Actor(other, user.email, user.name) + renewed.password = reset["temporary_password"] + assert renewed.login()["account"]["must_change_password"] is True + renewed.request("GET", "/api/v1/cases", status=403) + detail = operator.request("GET", f"/api/v1/accounts/{account_id}") + assert detail["must_change_password"] is True + assert detail["active_sessions"] == 1 + revoked = operator.request("POST", f"/api/v1/accounts/{account_id}/sessions/revoke") + assert revoked["revoked"] == 1 + renewed.request("GET", "/api/v1/auth/session", status=401) + + +def test_own_sessions_can_be_listed_and_ended(client, db): + user = _user(client, db) + user.login() + other = _second_client() + second = Actor(other, user.email, user.name) + second.login() + items = user.request("GET", "/api/v1/auth/sessions")["items"] + assert len(items) == 2 + assert sum(1 for item in items if item["current"]) == 1 + current = next(item for item in items if item["current"]) + foreign = next(item for item in items if not item["current"]) + user.request("DELETE", f"/api/v1/auth/sessions/{current['id']}", status=422) + user.request("DELETE", f"/api/v1/auth/sessions/{foreign['id']}", status=204) + second.request("GET", "/api/v1/cases", status=401) + third = Actor(other, user.email, user.name) + third.login() + assert user.request("POST", "/api/v1/auth/sessions/revoke-others")["revoked"] == 1 + third.request("GET", "/api/v1/cases", status=401) + user.request("GET", "/api/v1/cases") + + +def test_login_attempts_are_counted_in_the_database_without_the_address(client, db): + user = _user(client, db) + for _ in range(8): + response = client.post( + "/api/v1/auth/login", + json={"email": user.email, "password": "falsch-falsch-1"}, + headers={"Origin": ORIGIN}, + ) + assert response.status_code == 401 + blocked = client.post( + "/api/v1/auth/login", + json={"email": user.email, "password": PASSWORD}, + headers={"Origin": ORIGIN}, + ) + assert blocked.status_code == 429 + rows = db.execute("SELECT bucket FROM auth_login_attempts").fetchall() + assert len(rows) == 8 + assert all(user.email not in row["bucket"] and len(row["bucket"]) == 64 for row in rows) + db.rollback() + # Another address is a separate bucket and still works. + other = _user(client, db, "andere-adresse@example.test", "Andere") + Actor(client, other.email, other.name).login() + + +def test_unknown_and_wrong_password_answer_alike(client, db): + user = _user(client, db) + wrong = client.post( + "/api/v1/auth/login", + json={"email": user.email, "password": "falsch-falsch-1"}, + headers={"Origin": ORIGIN}, + ) + unknown = client.post( + "/api/v1/auth/login", + json={"email": "gibt-es-nicht@example.test", "password": "falsch-falsch-1"}, + headers={"Origin": ORIGIN}, + ) + assert wrong.status_code == unknown.status_code == 401 + assert wrong.json()["detail"] == unknown.json()["detail"] + + +def test_admin_actions_are_audited_without_secrets(client, db, operator): + created = operator.request( + "POST", + "/api/v1/accounts", + json={"email": "audit-ziel@example.test", "display_name": "Audit Ziel"}, + status=201, + ) + account_id = created["id"] + operator.request("POST", f"/api/v1/accounts/{account_id}/password-reset") + operator.request("PATCH", f"/api/v1/accounts/{account_id}", json={"status": "disabled"}) + rows = db.execute( + "SELECT action, detail FROM domain_events WHERE resource_kind = 'account' AND resource_id = %s ORDER BY occurred_at", + (account_id,), + ).fetchall() + assert [row["action"] for row in rows] == ["account_created", "password_reset", "account_disabled"] + text = str(rows) + str(db.execute("SELECT * FROM domain_events").fetchall()) + assert "audit-ziel@example.test" not in text + assert created["temporary_password"] not in text + + +def _id(actor: Actor, email: str) -> str: + items = actor.request("GET", "/api/v1/accounts")["items"] + return next(item["id"] for item in items if item["email"] == email) + + +def operator_id(actor: Actor) -> str: + return _id(actor, actor.email) + + +def promoted_id(actor: Actor) -> str: + return _id(actor, actor.email) diff --git a/backend/tests/test_views_scan.py b/backend/tests/test_views_scan.py new file mode 100644 index 0000000..36d777c --- /dev/null +++ b/backend/tests/test_views_scan.py @@ -0,0 +1,46 @@ +"""R-11: other people's rows must not push your own out of the overview and the search.""" + +from __future__ import annotations + +from tests.support import Actor, case_of, note_on, register + + +def test_own_entries_are_found_behind_many_foreign_ones(client, db, monkeypatch): + from fastapi.testclient import TestClient + + from app.api import routes + from app.main import app + + # A tiny batch size makes the paging visible with few rows. + monkeypatch.setattr(routes, "SCAN_BATCH", 2) + register(db, "ada-scan@example.test", "Ada") + register(db, "ben-scan@example.test", "Ben") + ada = Actor(client, "ada-scan@example.test", "Ada") + ben = Actor(TestClient(app), "ben-scan@example.test", "Ben") + ada.login() + ben.login() + + ada_case = case_of(ada, "Adas Akte") + for index in range(7): + matter = ada.request("POST", "/api/v1/matters", json={"title": f"Adas Vorgang {index}"}, status=201) + ada.request("POST", "/api/v1/tasks", json={"title": f"Frühe Aufgabe {index}", "due_on": "2026-01-0%d" % (index + 1), "matter_id": matter["id"]}, status=201) + ada.request("POST", "/api/v1/deadlines", json={"title": f"Frühe Frist {index}", "due_on": "2026-01-0%d" % (index + 1), "matter_id": matter["id"]}, status=201) + note_on(ada, f"Gemeinsames Stichwort {index}", "Zebrastreifen", ada_case["id"]) + + ben_matter = ben.request("POST", "/api/v1/matters", json={"title": "Bens Vorgang"}, status=201) + ben.request("POST", "/api/v1/tasks", json={"title": "Bens späte Aufgabe", "due_on": "2027-01-01", "matter_id": ben_matter["id"]}, status=201) + ben.request("POST", "/api/v1/deadlines", json={"title": "Bens späte Frist", "due_on": "2027-01-01", "matter_id": ben_matter["id"]}, status=201) + ben_case = case_of(ben, "Bens Akte") + note_on(ben, "Bens Zebrastreifen-Notiz", "Zebrastreifen", ben_case["id"]) + + assert [item["title"] for item in ben.request("GET", "/api/v1/views/open-tasks")["items"]] == ["Bens späte Aufgabe"] + assert [item["title"] for item in ben.request("GET", "/api/v1/views/open-deadlines")["items"]] == ["Bens späte Frist"] + found = ben.request("GET", "/api/v1/search", params={"q": "Zebrastreifen"})["items"] + assert [item["title"] for item in found] == ["Bens Zebrastreifen-Notiz"] + + # The own entries still come in order and the cap on the list holds. + monkeypatch.setattr(routes, "VIEW_LIMIT", 3) + first = ada.request("GET", "/api/v1/views/open-tasks")["items"] + assert [item["title"] for item in first] == ["Frühe Aufgabe 0", "Frühe Aufgabe 1", "Frühe Aufgabe 2"] + monkeypatch.setattr(routes, "SEARCH_LIMIT", 4) + assert len(ada.request("GET", "/api/v1/search", params={"q": "Zebrastreifen"})["items"]) == 4 diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 61c30f1..a6eef2b 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -42,6 +42,22 @@ Die Pflicht-Suite ist der eigene Workflow `test.yml`, nach dem Vorbild von Kansh --- +## Bekannte Betriebsgrenzen (Stand Code-Review 18.10.2026) + +Diese Punkte sind im [Code-Review](review/06_BEFUNDE.md) beschrieben. Sie stehen hier, damit sie beim Betrieb sichtbar bleiben. + +| Thema | Ist-Stand | Folge für den Betrieb | Review | +|---|---|---|---| +| Betreiberkonto | Auf einer leeren Datenbank legt die öffentliche Anmeldeseite das Betreiberkonto an. Keine MFA | Nach jedem Neuaufsetzen einer Umgebung sofort selbst anmelden, bevor die Adresse öffentlich erreichbar ist. Alternativ vorher `docker compose exec backend python -m app.cli create-account --email … --name … --password …` ausführen | R-04 | +| Sicherung | Es gibt kein Backup von Datenbank und Volume `tsumugu-media`/`dev-tsumugu-media` und keinen geprobten Restore | Laut technischer Spezifikation 8.2 nur synthetische Daten in Produktion, bis Paket P7 umgesetzt ist | R-21 | +| Release | `deploy-prod.yml` baut bei jedem Push auf `main` neu. Tests laufen nur auf `develop` nach dem Dev-Deploy | Vor dem Merge nach `main` den grünen `test.yml`-Lauf zum selben Stand prüfen | R-21, R-27 | +| Datenbankzugang | Das Backend migriert beim Start mit dem Owner-Zugang. Die Runtime-Rolle `tsumugu_runtime` erhält dasselbe Passwort wie `DB_USER` | Ein Wechsel von `DB_PASSWORD` ändert beim nächsten Start beide Rollen | R-09 | +| Quellen-Geheimnisse | Paperless-Token und IMAP-Passwörter stehen unverschlüsselt in der Tabelle `connectors` | Datenbank-Dumps wie Zugangsdaten schützen | R-05 | +| Sessions | 14 Tage gültig, keine Leerlaufabmeldung, kein Passwortwechsel in der Anwendung | Passwort nur über die Datenbank oder ein neues Konto änderbar | R-08 | +| Testläufe | `test.yml` startet je Push zweimal auf derselben Datenbank `tsumugu_test` | Ein roter Lauf kann vom parallelen Lauf verursacht sein; Ergebnis des Einzellaufs prüfen | R-27 | + +--- + ## Erreichbarkeit von außen TLS endet wie bei Kanshō auf dem Synology-NAS `192.168.2.63` (DSM Reverse Proxy), nicht auf dem Pi. diff --git a/docs/ENTWICKLUNG_UND_TESTS.md b/docs/ENTWICKLUNG_UND_TESTS.md new file mode 100644 index 0000000..456cb5a --- /dev/null +++ b/docs/ENTWICKLUNG_UND_TESTS.md @@ -0,0 +1,97 @@ +# Entwicklung und Tests – lokal + +**Stand:** 18.10.2026. Erstellt im Code-Review aus Code und Workflows. Deployment auf dem Raspberry Pi: [DEPLOYMENT.md](DEPLOYMENT.md). + +## Voraussetzungen + +- Python 3.12, Node.js (das Build-Image nutzt Node 22; lokal funktioniert auch Node 24) +- PostgreSQL 16. Der Benutzer aus `DB_USER` muss Tabellen anlegen und die Rolle `tsumugu_runtime` erstellen dürfen (`CREATEROLE`). Im Compose-Stack ist das der Superuser des Postgres-Containers. + +## Konfiguration + +Das Backend liest ausschließlich die Prozessumgebung. Eine `.env`-Datei wird nicht automatisch geladen. + +| Variable | Pflicht | Default | Bedeutung | +|---|---|---|---| +| `TSUMUGU_ENV` | ja | – | `development` oder `production`; ein anderer Wert verhindert den Start | +| `DB_NAME`, `DB_USER`, `DB_PASSWORD` | ja | – | Owner-Zugang; Migration und Runtime-Rolle | +| `DB_HOST`, `DB_PORT` | nein | `localhost`, `5432` | – | +| `DB_RUNTIME_USER` | nein | `tsumugu_runtime` | Rolle des laufenden Pools; erhält dasselbe Passwort wie `DB_USER` | +| `ALLOWED_ORIGINS` | nein | `http://localhost:5173,http://localhost:3094` | erlaubte `Origin`-Werte für schreibende Aufrufe | +| `APP_URL` | nein | `http://localhost:5173` | öffentliche Adresse | +| `MEDIA_ROOT` | nein | `data/media` | Ablage eigener Dateien | +| `PAPERLESS_MEDIA_ROOT` | nein | leer | lesender Paperless-Ablageordner als Rückfallweg | +| `SEED_DEMO` | nein | `false` | wirkt nur in Development mit Datenbank `*_test`; in Production verhindert `true` den Start | +| `SESSION_TTL_HOURS` | nein | `336` | Laufzeit einer Session | +| `UPLOAD_MAX_BYTES` | nein | `20971520` | Upload-Grenze (Nginx erlaubt 21 MiB) | + +## Backend starten + +```bash +cd backend +python -m venv .venv && . .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -r requirements-dev.txt +export TSUMUGU_ENV=development DB_NAME=tsumugu_local DB_USER=postgres DB_PASSWORD=… +python -m app.cli migrate +python -m app.cli create-account --email ich@example.test --name "Ich" --password "mindestens-zehn" +uvicorn main:app --reload --port 8000 +``` + +Das erste per CLI angelegte Konto wird Betreiber. Alternativ zeigt die Anmeldeseite auf einer leeren Datenbank das Formular „Betreiberkonto“ (siehe Review R-04). `python -m app.entrypoint` entspricht dem Containerstart: Migration, Entfernen von Fixture-Konten, danach Uvicorn auf Port 8000. + +## Frontend starten + +```bash +cd frontend +npm ci +npm run dev # http://localhost:5173, Proxy für /api und /health auf 127.0.0.1:8000 +``` + +Ein anderes Backend erreicht der Proxy über `VITE_API_PROXY`. Über HTTP heißt das Session-Cookie `jinkendo-session`. Die Kennzeichnung „Entwicklungsumgebung“ erscheint im Vite-Server automatisch, im Image über `VITE_TSUMUGU_ENV=development`. + +Produktionsbuild: `npm run build`. Für Development-Images ergänzt `mark-environment.mjs` danach Titel, Manifest und Farbe. + +## Tests + +### Backend + +Die Suite läuft gegen echtes PostgreSQL. Sie verweigert jede Datenbank, deren Name nicht auf `_test` endet, sowie `tsumugu`, `tsumugu_dev`, `postgres` und die Templates. Vor und nach jedem Test werden alle Tabellen geleert. + +```bash +cd backend +createdb tsumugu_local_test +export DB_NAME=tsumugu_local_test DB_USER=… DB_PASSWORD=… DB_HOST=localhost +python -m pytest tests -ra --basetemp=.pytest-tmp +``` + +- `conftest.py` setzt `TSUMUGU_ENV=development`, `ALLOWED_ORIGINS=http://testserver`, `SEED_DEMO=false` und ein temporäres `MEDIA_ROOT`. +- Unter Windows kann das Standard-Temp-Verzeichnis von pytest gesperrt sein. `--basetemp=.pytest-tmp` umgeht das; der Ordner gehört nicht ins Repository. +- Zwei Läufe dürfen nicht gleichzeitig dieselbe Testdatenbank verwenden. +- Paperless und IMAP werden in den Tests durch lokale Attrappen ersetzt. Es entsteht kein Netzzugriff. + +### Frontend + +```bash +cd frontend +node --test src/list-meta.test.mjs +``` + +Weitere Frontend-Tests gibt es nicht. Ein `npm test`-Skript ist nicht registriert. + +### In CI + +`.gitea/workflows/test.yml` erzeugt auf der Dev-Postgres die Datenbank `tsumugu_test` neu und führt die Suite im Backend-Image aus. Danach folgen ein lesender Health-Smoke der laufenden Dev-App und ein Frontend-Build. Bekannte Schwächen: [Review 07](review/07_TESTS_UND_CI.md). + +## Neue Migration + +1. Datei `backend/migrations/YYYYMMDDTHHMMSSZ_kurzname.sql` anlegen, Zeitstempel in UTC, lexikalisch nach der letzten. +2. Keine eigene Transaktionssteuerung in der Datei; der Runner kapselt jede Datei. +3. Angewandte Dateien nie ändern: Die Prüfsumme im Ledger verhindert danach den Start. +4. [DATENMODELL_IST.md](architecture/DATENMODELL_IST.md) nachziehen. + +## Neue Route + +1. Route in `api/routes.py`, `api/sources.py` oder `api/export.py` mit `Depends(get_principal)` und `Depends(get_conn)` anlegen. +2. Rechte über die Funktionen in `access/policy.py` prüfen, nie nur im Frontend. +3. Änderung samt `audit(...)` in derselben Verbindung. +4. [API_REFERENZ.md](architecture/API_REFERENZ.md) nachziehen. diff --git a/docs/MVP_ANNAHMEN.md b/docs/MVP_ANNAHMEN.md index 697cb65..cafad8d 100644 --- a/docs/MVP_ANNAHMEN.md +++ b/docs/MVP_ANNAHMEN.md @@ -7,6 +7,9 @@ Diese Regeln sind die Arbeitsgrundlage der ersten Lieferung. Sie sind keine endg 3. **Einbindung und Löschen.** „Aus diesem Kontext entfernen“ löscht die Einbindung und ihre Anmerkungen. Das Element bleibt bei der Eigentümerin oder dem Eigentümer, auch ohne Einbindung. „Überall löschen“ darf nur diese Person. Paperless-Originale werden nicht gelöscht. 4. **Aktenvererbung.** Die Zuordnung eines Vorgangs gibt die Rechte dieser Akte an den Vorgang und an Inhalte ohne ausdrückliche Ausnahme weiter. Was die zuordnende Person nicht weitergeben darf, bleibt in diesem Aktenkontext unsichtbar, ohne Titel und ohne Anzahl. Querverweise vererben nichts. Die Ownership bleibt unverändert. +5. **Herunterladen von Originalen.** Das Recht „Herunterladen“ ist getrennt vom Lesen, standardmäßig überall erlaubt und je Raum und Rolle abschaltbar (Voreinstellung am Konto des Hauptnutzers). Es ist eine Nutzungsregel, kein Kopierschutz: Lesen in der App bleibt möglich, Abfotografieren und Entwicklerwerkzeuge verhindert die Regel nicht. Ob später eine strengere Rechtegrundlage nötig ist, entscheiden Datenschutz und Recht. +6. **Bilder in E-Mails.** Bilder fremder Server werden nicht automatisch geladen. Auf Wunsch lädt der Server sie für die Person nach, das Gerät kontaktiert den Absender nicht. Dass die Nachricht geöffnet wurde, kann der Absender erkennen. + Der gewählte Datenraum legt fest, wo neue Objekte entstehen. Er ist nicht selbst der Rechtepfad. Eine gültige Sitzung muss einen Datenraum gewählt haben; der Server prüft danach den mitgeschickten Handlungskontext. Technische Anpassung: Das Session-Cookie heißt in Production `__Host-jinkendo-session` und ist `Secure`. Development nutzt diesen Namen nur bei HTTPS. Über die dokumentierten HTTP-Adressen im LAN heißt es `jinkendo-session`, weil ein `__Host-`-Cookie dort nicht gespeichert werden kann. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..009eb8b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,99 @@ +# Tsumugu – Dokumentation + +Übersicht aller Dokumente im Repository. Verbindlichkeit laut [AGENTS.md](../AGENTS.md): Fachlich führend ist die Produktspezifikation, technisch der Entwurf in `architecture/`. Vorläufige Rechte- und Löschregeln stehen in den MVP-Annahmen. + +## Einstieg + +| Ich will … | Dokument | +|---|---| +| wissen, was Tsumugu ist und wo es läuft | [../README.md](../README.md) | +| lokal entwickeln und testen | [ENTWICKLUNG_UND_TESTS.md](ENTWICKLUNG_UND_TESTS.md) | +| deployen und Ports nachschlagen | [DEPLOYMENT.md](DEPLOYMENT.md) | +| den fachlichen Stand und offene Entscheidungen kennen | [product/TSUMUGU_PRODUCT_SPEC.md](product/TSUMUGU_PRODUCT_SPEC.md) | +| den aktuellen Qualitätsstand und offene Mängel sehen | [review/README.md](review/README.md) | + +## Produkt (`product/`) + +| Dokument | Rolle | Stand | +|---|---|---| +| [TSUMUGU_PRODUCT_SPEC.md](product/TSUMUGU_PRODUCT_SPEC.md) | führende Fachspezifikation, alle bestätigten Entscheidungen F-01 bis F-08, S-01 | 0.37 | +| [TSUMUGU_UX_MVP_ERGAENZUNG.md](product/TSUMUGU_UX_MVP_ERGAENZUNG.md) | Bedienung, Orientierung, Abnahmekriterien UX-01 bis UX-21 | 0.8 | +| [TSUMUGU_KOMPAKTE_INHALTSLISTE_AUFTRAG.md](product/TSUMUGU_KOMPAKTE_INHALTSLISTE_AUFTRAG.md) | Auftrag F-06f, hat Vorrang vor älteren Darstellungsvorgaben | 1.0 | +| [TSUMUGU_KOMMUNIKATIONSRICHTUNG_AUFTRAG.md](product/TSUMUGU_KOMMUNIKATIONSRICHTUNG_AUFTRAG.md) | Auftrag F-06e | 1.2 | +| [TSUMUGU_SYMBOLS_LOGOFRAME_AUFTRAG.md](product/TSUMUGU_SYMBOLS_LOGOFRAME_AUFTRAG.md) | Auftrag F-06c, Typ-Symbole und Logo-Rahmen | 17.10.2026 | +| [TSUMUGU_ENSO_ANIMATION_AUFTRAG.md](product/TSUMUGU_ENSO_ANIMATION_AUFTRAG.md) | Einbindung der Ensō-Animation | 17.10.2026 | +| [TSUMUGU_EXPORT_NAECHSTE_LIEFERUNG_2026-10-17.md](product/TSUMUGU_EXPORT_NAECHSTE_LIEFERUNG_2026-10-17.md) | Auftrag F-07a–c, Übergabepaket | 17.10.2026 | +| [TSUMUGU_EXPORT_PRUEFBERICHT_2026-10-17.md](product/TSUMUGU_EXPORT_PRUEFBERICHT_2026-10-17.md) | Prüfbericht der Exportlieferung | 17.10.2026 | +| [TSUMUGU_UNTERKAPITEL_AUFTRAG_2026-10-17.md](product/TSUMUGU_UNTERKAPITEL_AUFTRAG_2026-10-17.md) | Auftrag F-08a | 17.10.2026 | +| [TSUMUGU_UNTERKAPITEL_PRUEFBERICHT_2026-10-17.md](product/TSUMUGU_UNTERKAPITEL_PRUEFBERICHT_2026-10-17.md) | Prüfbericht Unterkapitel | 17.10.2026 | +| [TSUMUGU_CONNECTOREN_BEFUND_2026-10-17.md](product/TSUMUGU_CONNECTOREN_BEFUND_2026-10-17.md) | Bestandsanalyse Paperless/IMAP, Umbauschätzung | 17.10.2026 | +| [TSUMUGU_CONNECTOR_UMBAU_AUFTRAG_2026-10-19.md](product/TSUMUGU_CONNECTOR_UMBAU_AUFTRAG_2026-10-19.md) | begrenzter Umsetzungsauftrag, Abschnitt A beauftragt, Abschnitt B der nächste Ausbau | 19.10.2026 | +| [TSUMUGU_CONNECTOR_ABSCHLUSSKRITERIEN_2026-10-19.md](product/TSUMUGU_CONNECTOR_ABSCHLUSSKRITERIEN_2026-10-19.md) | Abschlusskriterien der Kapselung; Vorrang vor einem teilweise integrierten Zwischenstand | 19.10.2026 | +| [TSUMUGU_CONNECTOR_ABSCHLUSSMATRIX_2026-10-19.md](product/TSUMUGU_CONNECTOR_ABSCHLUSSMATRIX_2026-10-19.md) | Kriterium, Nachweis und Ergebnis der Kapselung | 19.10.2026 | +| [TSUMUGU_CONNECTOR_NACHARBEIT_IMAP_PAPERLESS_MARKDOWN_AUFTRAG_2026-10-19.md](product/TSUMUGU_CONNECTOR_NACHARBEIT_IMAP_PAPERLESS_MARKDOWN_AUFTRAG_2026-10-19.md) | Nacharbeit: IMAP-Seiten, Paperless-Suche, Markdown-Notiz | 19.10.2026 | +| [TSUMUGU_KORREKTURAUFTRAG_CONNECTOREN_MAIL_MARKDOWN_2026-10-20.md](product/TSUMUGU_KORREKTURAUFTRAG_CONNECTOREN_MAIL_MARKDOWN_2026-10-20.md) | Korrektur K1–K7: IMAP-Ordner, MIME, Transaktion, Markdown, Leser, Paperless-Antworten | 20.10.2026 | +| [TSUMUGU_IMAP_SENT_ITEMS_MIME_BEFUND_UND_FIXAUFTRAG_2026-10-20.md](product/TSUMUGU_IMAP_SENT_ITEMS_MIME_BEFUND_UND_FIXAUFTRAG_2026-10-20.md) | Befund zu Sent Items und MIME, unverändert übernommen | 20.10.2026 | +| [TSUMUGU_RELEASE_GATE_BACKUP_AUFTRAG_2026-10-22.md](product/TSUMUGU_RELEASE_GATE_BACKUP_AUFTRAG_2026-10-22.md) | Auftrag P7: Release-Gate, Backup und Restore vor Produktion (R-21), für eine eigene Sitzung | 22.10.2026 | +| [TSUMUGU_CONNECTOR_NACHARBEIT_REVIEW_2026-10-20.md](product/TSUMUGU_CONNECTOR_NACHARBEIT_REVIEW_2026-10-20.md) | Review der Nacharbeit, unverändert übernommen | 20.10.2026 | +| [TSUMUGU_CONNECTOR_BASIS_ERGEBNIS_2026-10-19.md](product/TSUMUGU_CONNECTOR_BASIS_ERGEBNIS_2026-10-19.md) | früherer Zwischenstand von Abschnitt A vor den Abschlusskriterien | 19.10.2026 | +| [TSUMUGU_WEITERPLANUNG_UEBERGABE_2026-10-17.md](product/TSUMUGU_WEITERPLANUNG_UEBERGABE_2026-10-17.md) | Einstieg für die fachliche Weiterplanung | 17.10.2026 | +| [TECHNICAL_SPEC_HANDOFF.md](product/TECHNICAL_SPEC_HANDOFF.md) | historischer Auftrag zur technischen Spezifikation | 16.10.2026 | +| [Tsumugu_Product_Recovery_Baseline_v0.1.md](product/Tsumugu_Product_Recovery_Baseline_v0.1.md) | unveränderte historische Ausgangsbasis | 0.1 | + +## Architektur (`architecture/`) + +| Dokument | Rolle | Stand | +|---|---|---| +| [TSUMUGU_TECHNICAL_SPEC.md](architecture/TSUMUGU_TECHNICAL_SPEC.md) | technischer Entwurf (Soll): Pakete, Datenmodell, Rechte, API, Nachweise | 0.3 mit Ergänzungen | +| [API_REFERENZ.md](architecture/API_REFERENZ.md) | tatsächlich registrierte Endpunkte (Ist) | 19.10.2026 | +| [DATENMODELL_IST.md](architecture/DATENMODELL_IST.md) | Schema nach allen Migrationen (Ist) | 19.10.2026 | +| [RECHTEMODELL_IST.md](architecture/RECHTEMODELL_IST.md) | implementierte Rechteauswertung (Ist) | 19.10.2026 | +| [BENUTZERVERWALTUNG.md](architecture/BENUTZERVERWALTUNG.md) | Konten, Rollen, Einmal-Passwort, Einladungen, Audit (Ist) und Abgrenzung zum Sicherheitskonzept | 20.10.2026 | +| [ARCHITECTURE_DISCOVERY.md](architecture/ARCHITECTURE_DISCOVERY.md) | Ist-Aufnahme der Jinkendo-Familie vor Tsumugu, keine Beschlüsse | 15.10.2026 | +| [TSUMUGU_CONNECTOR_STORAGE_VERTRAG_v0.1_2026-10-19.md](architecture/TSUMUGU_CONNECTOR_STORAGE_VERTRAG_v0.1_2026-10-19.md) | Umsetzungsvorschlag Source- und Storage-Vertrag | 19.10.2026 | +| [Tsumugu_Connector_Suite_Architektur_v0.2.md](architecture/Tsumugu_Connector_Suite_Architektur_v0.2.md) | Zielarchitektur der Connector Suite, keine pauschale Beschlussübernahme | 19.10.2026 | +| [TSUMUGU_STORAGE_SCHLUESSEL_PRUEFAUFTRAG_2026-10-19.md](architecture/TSUMUGU_STORAGE_SCHLUESSEL_PRUEFAUFTRAG_2026-10-19.md) | vorbereiteter Prüfauftrag an die Architektur-Session, nicht versandt | 19.10.2026 | + +## Design (`design/`) + +| Dokument | Rolle | Stand | +|---|---|---| +| [LIEFERUNG_2026-10-17.md](design/LIEFERUNG_2026-10-17.md) | Lieferbericht der Arbeitsfläche, bewusste Abweichungen | 17.10.2026, historisch | + +Das ausgewählte Designkonzept und die Assetpakete liegen außerhalb des Repositorys unter `C:/Dev/jinkendo-app-design-2026-10-17/`. Die übernommenen Assets stehen in `frontend/public/brand/` und `frontend/public/ui/`. + +## Referenz (`reference/`) + +Familienweite Vorgaben der Jinkendo-Anwendungen. Verbindliche Verträge und anpassbare Defaults sind dort jeweils getrennt. + +| Dokument | Geltung für Tsumugu | +|---|---| +| [A – Technische Referenzarchitektur v0.3](reference/jinkendo-phase3-2-technical-reference-architecture-v0.3-2026-10-16/A_Jinkendo_Technical_Reference_Architecture.md) | verbindlich | +| [B – Implementation Blueprint Foundation v0.3](reference/jinkendo-phase3-2-technical-reference-architecture-v0.3-2026-10-16/B_Implementation_Blueprint_Application_Foundation.md) | verbindlich | +| [C – Technische Entscheidungen v0.3](reference/jinkendo-phase3-2-technical-reference-architecture-v0.3-2026-10-16/C_Technical_Decisions_Open_Issues.md) | verbindlich | +| [Auth, Session & Scope](reference/Jinkendo_Auth_Session_Scope_Blueprint.md) | verbindlich | +| [Medien, Dateien & sicheres Rendering](reference/Jinkendo_Media_Files_Safe_Rendering_Blueprint.md) | verbindlich | +| [Release, Betrieb & Recovery](reference/Jinkendo_Release_Betrieb_Recovery_Blueprint.md) | verbindlich vor echtem Betrieb (Paket P7) | +| [KI-Gateway](reference/Jinkendo_AI_Gateway_Provider_Privacy_Blueprint.md), [Prompt/Workflow](reference/Jinkendo_Prompt_Placeholder_Workflow_Blueprint.md), [Entitlements](reference/Jinkendo_Entitlements_Usage_Quotas_Blueprint.md) | nur als Grenze; keine MVP-Pflicht | + +Die Links innerhalb der Referenzdokumente zeigen auf die ursprünglichen Ablageorte unter `C:/Dev/…` und funktionieren im Repository nicht. + +## Betrieb und Regeln + +| Dokument | Rolle | +|---|---| +| [DEPLOYMENT.md](DEPLOYMENT.md) | Umgebungen, Ports, Workflows, Proxy, bekannte Betriebsgrenzen | +| [MVP_ANNAHMEN.md](MVP_ANNAHMEN.md) | vorläufige Rechte- und Löschregeln, Cookie-Abweichung in Development | +| [ENTWICKLUNG_UND_TESTS.md](ENTWICKLUNG_UND_TESTS.md) | Konfiguration, lokaler Start, Tests, neue Migration und Route | + +## Review (`review/`) + +| Dokument | Inhalt | +|---|---| +| [review/README.md](review/README.md) | Code-Review vom 18.10.2026: Gesamtbewertung, Anforderungsabdeckung, Befunde, Maßnahmenplan | + +## Pflege + +- Neue Fachentscheidung → Produktspezifikation (Abschnitt 5 und Fortschreibung). +- Neue Route, Migration oder Rechteregel → die passende Ist-Datei in `architecture/` im selben Commit. +- Neuer Auftrag oder Prüfbericht → `product/`, hier in der Tabelle eintragen. diff --git a/docs/architecture/API_REFERENZ.md b/docs/architecture/API_REFERENZ.md index 73db9d2..4d67450 100644 --- a/docs/architecture/API_REFERENZ.md +++ b/docs/architecture/API_REFERENZ.md @@ -1,6 +1,6 @@ # Tsumugu – API-Referenz (Ist-Stand) -**Stand:** 18.10.2026. Erstellt im Code-Review aus `backend/app/main.py`, `api/routes.py`, `api/sources.py`, `api/export.py` und `api/correspondence.py`. +**Stand:** 18.10.2026, ergänzt am 19.10.2026 um Quelleninstanzen, Fähigkeiten und Zuordnung und am 23.10.2026 um Download-Regeln, HTML-Mail mit Bildern, Übersicht und Suche. Erstellt im Code-Review aus `backend/app/main.py`, `api/routes.py`, `api/sources.py`, `api/export.py` und `api/correspondence.py`. Diese Datei beschreibt die tatsächlich registrierten Endpunkte. Die Sollfassung steht in der [technischen Spezifikation, Abschnitt 6.2](TSUMUGU_TECHNICAL_SPEC.md). OpenAPI ist in allen Umgebungen abgeschaltet (`docs_url=None`, `openapi_url=None`). Diese Datei ist deshalb die einzige Übersicht. Sie wird bei jeder Routenänderung mitgepflegt. @@ -31,28 +31,61 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R** | Methode | Pfad | Auth | Zweck | |---|---|---|---| -| POST | `/api/v1/auth/login` | – | E-Mail, Passwort → Session-Sicht und Cookie. 8 Versuche je IP+E-Mail in 10 min (Prozessspeicher) | +| POST | `/api/v1/auth/login` | – | E-Mail, Passwort → Session-Sicht und Cookie. 8 Fehlversuche je Bucket (Hash aus IP und E-Mail) in 10 min, gezählt in `auth_login_attempts` | | POST | `/api/v1/auth/logout` | S | Session widerrufen, Cookie löschen, 204 | +| GET / PUT | `/api/v1/auth/download-defaults` | S+R, Hauptnutzer | Voreinstellung je Rolle und `direct` (direkt freigegebene Personen); neue Räume übernehmen die drei Rollenwerte. Audit `download_defaults_changed` | | GET | `/api/v1/auth/session` | S | Konto, bestätigter Datenraum, Räume, CSRF-Token, Epoche | | POST | `/api/v1/auth/session/scope` | S | Datenraum wechseln (`kind`, `id`); erhöht die Epoche | | GET | `/api/v1/auth/bootstrap` | – | `{available: true}`, solange kein Betreiber existiert, sonst 404 | | POST | `/api/v1/auth/bootstrap` | – | erstes Konto als Betreiber anlegen, nur ohne vorhandenen Betreiber | | GET | `/api/v1/auth/demo` | – | Beispielkonten; nur Development mit Datenbank `*_test` und `SEED_DEMO=true`, sonst 404 | -| GET | `/api/v1/accounts` | S+R, B | aktive Betreiberkonten (`origin='operator'`) | -| POST | `/api/v1/accounts` | S+R, B | Konto mit Passwort anlegen, erhält persönlichen Raum | +| GET | `/api/v1/accounts` | S+R, B | Betreiberkonten (`origin='operator'`) mit Status, Rolle, letzter Anmeldung; Filter `status=active\|disabled` | +| GET | `/api/v1/accounts/{id}` | S+R, B | Detail mit Raummitgliedschaften und Zahl aktiver Sitzungen | +| POST | `/api/v1/accounts` | S+R, B | Konto anlegen, persönlicher Raum, `must_change_password`. Ohne `password` erzeugt der Server ein Einmal-Passwort und liefert es einmal als `temporary_password`. `is_operator` optional | +| PATCH | `/api/v1/accounts/{id}` | S+R, B | `display_name`, `is_operator`, `status` (`active`/`disabled`), `account_kind: main` (Mitarbeiter zum Hauptnutzer machen). Letzter Betreiber und eigenes Konto sind geschützt | +| POST | `/api/v1/accounts/{id}/password-reset` | S+R, B | neues Einmal-Passwort, beendet alle Sitzungen; nicht für das eigene Konto | +| POST | `/api/v1/accounts/{id}/sessions/revoke` | S+R, B | alle Sitzungen des Kontos beenden | +| POST | `/api/v1/auth/password` | S | eigenes Passwort ändern (`current_password`, `new_password`); beendet die anderen Sitzungen; 204. Erlaubt auch bei `must_change_password` | +| GET | `/api/v1/auth/sessions` | S | eigene aktive Sitzungen (Gerätehinweis, zuletzt aktiv) | +| DELETE | `/api/v1/auth/sessions/{id}` | S | eigene andere Sitzung beenden; die aktuelle über Abmelden | +| POST | `/api/v1/auth/sessions/revoke-others` | S | überall sonst abmelden | | GET | `/api/v1/principals?email=` | S+R | aktives Konto per E-Mail auflösen (ID, Anzeigename) | -| POST | `/api/v1/invitations` | S+R, M | Einladung in einen gemeinsamen Raum; Antwort enthält das Token einmalig. Keine Oberfläche | -| POST | `/api/v1/invitations/{id}/accept` | – | Token, Anzeigename, Passwort → Konto, Mitgliedschaft, Session | +| POST | `/api/v1/invitations` | S+R, M | Einladung in einen gemeinsamen Raum mit `role` (`member`/`manager`/`reader`); Antwort enthält das Token einmalig | +| POST | `/api/v1/invitations/{id}/accept` | – | Token, Anzeigename, Passwort → Konto, Mitgliedschaft mit der Rolle der Einladung, Session. Widerrufene, benutzte oder abgelaufene Einladung: 404 | ## Datenräume | Methode | Pfad | Auth | Zweck | |---|---|---|---| -| GET | `/api/v1/rooms` | S+R | wählbare Räume | +| GET | `/api/v1/rooms` | S+R | wählbare Räume; `include_archived=true` ergänzt die archivierten Räume der Person | | POST | `/api/v1/rooms` | S+R | gemeinsamen Raum anlegen, Anlegende Person wird Verwalter | -| GET | `/api/v1/rooms/{id}/members` | S+R, M | Mitglieder | -| POST | `/api/v1/rooms/{id}/members` | S+R, M | bestehendes Konto als `member` oder `manager` aufnehmen | -| DELETE | `/api/v1/rooms/{id}/members/{subject_id}` | S+R, M | Mitgliedschaft beenden; letzter Verwalter bleibt. Keine Oberfläche | +| PATCH | `/api/v1/rooms/{id}` | S+R, M / E | `name` (Verwalter); `status` und `default_visibility` nur der Eigentümer (E). Archivieren setzt Sitzungen im Raum auf den persönlichen Raum zurück | +| GET / PUT | `/api/v1/rooms/{id}/download-policy` | S+R, Mitglied / E | je Rolle (`manager`, `member`, `reader`), ob das Original gespeichert oder geöffnet werden darf; `can_change` nennt, ob die Person ändern darf. Fremde und unbekannte Räume: 404. Audit `room_download_policy_changed` | +| GET | `/api/v1/rooms/{id}/members` | S+R, M | Mitglieder mit Rolle und Kontostatus | +| POST | `/api/v1/rooms/{id}/members` | S+R, M | bestehendes Konto als `member`, `manager` oder `reader` aufnehmen oder dessen Rolle setzen | +| PATCH | `/api/v1/rooms/{id}/members/{subject_id}` | S+R, M | Rolle ändern; letzter Verwalter bleibt. Rückstufung auf `reader` entzieht Bearbeiten- und Freigeben-Grants auf Objekte des Raums | +| DELETE | `/api/v1/rooms/{id}/members/{subject_id}` | S+R, M | Mitgliedschaft beenden; letzter Verwalter bleibt | +| GET | `/api/v1/rooms/{id}/invitations` | S+R, M | offene Einladungen ohne Token | +| DELETE | `/api/v1/rooms/{id}/invitations/{invitation_id}` | S+R, M | Einladung widerrufen | +| POST | `/api/v1/rooms/{id}/owner` | S+R, E | Raum an ein aktives Mitglied übertragen (Eigentümer); der neue Eigentümer wird Verwalter | +| GET | `/api/v1/rooms/{id}/exposure` | S+R, Mitglied | eigene Akten und Vorgänge: mit dem Raum geteilt (`room`) oder dort abgelegt, aber nicht geteilt (`private`) | +| POST | `/api/v1/rooms/{id}/my-content/visibility` | S+R, Mitglied | `room`: alle eigenen im Raum abgelegten Akten und Vorgänge mit dem Raum teilen; `private`: alle eigenen Freigaben an diesen Raum beenden | +| GET | `/api/v1/cases/{id}/room-share`, `/api/v1/matters/{id}/room-share` | S+R, Owner | Freigaben des Eintrags an Datenräume (mit Mitgliedern, Rolle, Ausnahme) und die Räume, mit denen er noch teilen kann | +| PUT / DELETE | `/api/v1/cases/{id}/room-share/rooms/{room_id}` (PUT antwortet zusätzlich mit `effect`), `/api/v1/matters/{id}/room-share/rooms/{room_id}` | S+R, Owner | mit dem Raum teilen (`access`: `edit` = Rechte nach Rolle, `read` = höchstens Lesen) oder die Freigabe beenden; nur in Räumen, in denen der Owner Mitglied oder Verwalter ist | +| PUT | `/api/v1/cases/{id}/room-share/people/{subject_id}`, `/api/v1/matters/{id}/room-share/people/{subject_id}` | S+R, Owner | eine Person von der Raumfreigabe ausnehmen (`excluded`) | +| POST | `/api/v1/admin/rooms/{id}/owner` | S+R, B | Betreiber setzt einen Eigentümer ein, nur wenn der bisherige nicht mehr handeln kann (Konto deaktiviert oder nicht mehr Mitglied) | +| GET | `/api/v1/rooms/overview` | S+R | Räume der Person als Kacheln: Rolle, Eigentümer, Zahl der Mitglieder und der lesbaren Akten und Vorgänge; Mitgliedernamen nur für Verwalter | +| GET | `/api/v1/rooms/{id}/contents` | S+R, Mitglied | Akten und Vorgänge, die mit dem Raum geteilt und für die Person lesbar sind, mit Zählern | +| POST | `/api/v1/rooms/{id}/shares` | S+R, Mitglied | mehrere eigene Einträge auf einmal mit dem Raum teilen (alle oder keiner); Antwort nennt die Wirkung je Mitglied | +| GET | `/api/v1/cases/{id}/outline`, `/api/v1/matters/{id}/outline` | S+R | Struktur für den Navigationsbaum: Kapitel, Unterakten und Vorgänge bzw. Teilvorgänge, nur aus Lesbarem; für Vorgänge optional `context_kind=case&context_id=` | +| GET | `/api/v1/team` | S+R, Hauptnutzer | Personen in den eigenen Räumen mit Rolle je Raum; `mine` = selbst eingeladen | +| PATCH | `/api/v1/team/{id}` | S+R, Hauptnutzer | `status` eines selbst eingeladenen Mitarbeiters; fremde und unbekannte ID: 403 | +| POST | `/api/v1/team/{id}/password-reset` | S+R, Hauptnutzer | Einmal-Passwort für einen selbst eingeladenen Mitarbeiter | +| DELETE | `/api/v1/team/{id}/memberships` | S+R, Hauptnutzer | Person aus allen eigenen Räumen entfernen | +| GET | `/api/v1/admin/team` | S+R, B | Hauptnutzer mit Mitarbeitern, Räumen und Rollen; `format=csv` für den Export | +| GET | `/api/v1/admin/rooms` | S+R, B | alle gemeinsamen Räume mit Zustand, Mitgliederzahl, Verwaltern; keine Inhalte | +| POST | `/api/v1/admin/rooms/{id}/managers` | S+R, B | Verwalter einsetzen, nur wenn der Raum keinen aktiven Verwalter hat | +| GET | `/api/v1/admin/events` | S+R, B | Verwaltungsprotokoll (`cursor`, `limit`) für Konto, Sitzung, Raum, Mitgliedschaft, Einladung | ## Akten @@ -111,7 +144,7 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R** | POST | `/api/v1/notes` | Notiz mit Einbindung (`context`, optional `placement`, `chapter_id`, `dates`, `communication_direction`) | | PATCH | `/api/v1/notes/{id}` | gemeinsamen Text ändern; Antwort mit `visible_bindings` und `notice` | | POST | `/api/v1/files` | Datei hochladen (Multipart: `title`, `context_kind`, `context_id`, `upload`, optional Platzierung, Kapitel, Daten, Richtung); PDF, PNG, JPEG, WebP bis 20 MiB | -| GET | `/api/v1/files/{id}/content` | Datei als `attachment`, `no-store` | +| GET | `/api/v1/files/{id}/content` | Datei als `attachment` mit Dateiname (UTF-8, `filename*`), `no-store`. Mit `save=true` zusätzlich das Download-Recht, sonst 403 | | POST | `/api/v1/external-refs` | externe Referenz (`paperless`, `imap`, `url`) mit `detail` | | POST | `/api/v1/sources/link` | wie oben, nur `paperless`/`imap` (doppelter Weg, siehe Connectoren-Befund) | | GET | `/api/v1/elements` | eigene, nicht eingebundene Elemente (max. 100) | @@ -143,8 +176,8 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R** | POST | `/api/v1/deadlines` | Frist | | PATCH | `/api/v1/deadlines/{id}` | Titel, Datum | | POST | `/api/v1/deadlines/{id}/fulfill` | als erfüllt kennzeichnen | -| GET | `/api/v1/views/open-tasks` | offene sichtbare Aufgaben (siehe Review R-11) | -| GET | `/api/v1/views/open-deadlines` | offene sichtbare Fristen (siehe Review R-11) | +| GET | `/api/v1/views/open-tasks` | offene sichtbare Aufgaben, höchstens 200. Gelesen wird in Blöcken mit Rechteprüfung je Zeile (R-11 behoben) | +| GET | `/api/v1/views/open-deadlines` | offene sichtbare Fristen, wie oben | | GET | `/api/v1/views/recent-matters` | acht zuletzt geänderte sichtbare Vorgänge | | GET | `/api/v1/placement-targets` | bearbeitbare Akten und Vorgänge als Ziele | | GET | `/api/v1/search?q=` | Akten, Vorgänge, Elemente (Titel und Notiztext), max. 50 | @@ -153,20 +186,28 @@ Legende Spalte „Auth“: **–** öffentlich, **S** gültige Session, **S+R** | Methode | Pfad | Zweck | |---|---|---| -| GET | `/api/v1/connectors` | eigene Zugänge ohne Secret | -| POST | `/api/v1/connectors` | Paperless (überschreibt den vorhandenen) oder IMAP anlegen | -| DELETE | `/api/v1/connectors/{id}` | Zugang löschen | -| POST | `/api/v1/connectors/{id}/probe` | Zugang prüfen | -| GET | `/api/v1/sources/paperless/documents?q=` | Paperless-Suche (letzter eigener Zugang) | +| GET | `/api/v1/connectors` | eigene Zugänge ohne Secret, mit Fähigkeiten der registrierten Quelle, `has_secret` und letztem Prüfstand (`probe`) | +| GET | `/api/v1/connectors/{id}` | Detail eines eigenen Zugangs: nicht geheime Einstellungen, `has_secret`, Fähigkeiten, Schutzprofil, Prüfstand und Zahl der eigenen Verknüpfungen. Nie das Secret. Fremde Instanz: 404 | +| POST | `/api/v1/connectors` | neue Instanz einer registrierten Quelle; Body `{kind, name, config}`. `config` wird gegen das Feldmanifest des Moduls geprüft (`modules[].config_fields` aus `/api/v1/sources/capabilities`); unbekannte Art oder unbekanntes Feld wird abgewiesen; ersetzt keinen vorhandenen Zugang. Die flachen Felder der ersten Lieferung (`base_url`, `token`, `host`, …) gelten nur als Kompatibilität und werden in `config` übersetzt | +| PATCH | `/api/v1/connectors/{id}` | Name, Einstellungen und Zugangsdaten derselben Instanz. Ein leeres Geheimnisfeld behält das gespeicherte Geheimnis; der letzte Prüfstand wird zurückgesetzt | +| DELETE | `/api/v1/connectors/{id}` | Zugang löschen; bestehende Referenzen bleiben und werden nicht umgehängt | +| POST | `/api/v1/connectors/{id}/probe` | gespeicherten Zugang prüfen; Ergebnis (auch ein Fehler) wird als Prüfstand festgehalten. Es gibt keine Prüfung mit ungespeicherten Entwürfen | +| GET | `/api/v1/sources/capabilities` | erklärte Such-, Lese- und Editorfähigkeiten; nicht Vorhandenes bleibt falsch | +| GET | `/api/v1/sources/{kind}/search` | Suche der registrierten Quelle im erklärten Umfang; unbekannte Art und nicht erklärte Filter antworten `422` | +| GET | `/api/v1/sources/{kind}/navigate?connector_id=` | Fachnavigation, wenn die Quelle sie erklärt | +| GET | `/api/v1/sources/{kind}/preview` | Vorschau vor dem Übernehmen, wenn die Quelle sie erklärt | +| GET | `/api/v1/sources/paperless/documents?q=&connector_id=` | Kompatibilitätshülle der Paperless-Suche; delegiert an dieselbe Suche | | GET | `/api/v1/sources/imap/mailboxes?connector_id=` | IMAP-Fächer | -| GET | `/api/v1/sources/imap/messages?connector_id=&mailbox=&q=&before=` | Nachrichtenliste, 30 je Seite | +| GET | `/api/v1/sources/imap/messages?connector_id=&mailbox=&q=&before=` | Nachrichtenliste, 30 je Seite, Fortsetzung über `before` | | GET | `/api/v1/sources/imap/message?connector_id=&mailbox=&uid=` | Vorschau vor dem Übernehmen | -| GET | `/api/v1/external-refs/{id}/content` | Paperless-Datei zum Ansehen (`inline`) | -| GET | `/api/v1/external-refs/{id}/mail` | vollständige E-Mail als JSON, Identität per Message-ID geprüft | -| GET | `/api/v1/external-refs/{id}/mail/original` | EML | -| GET | `/api/v1/external-refs/{id}/mail/attachments/{index}` | Anlage (`inline`, siehe Review R-03) | +| GET | `/api/v1/external-refs/{id}/content` | Original der gebundenen Instanz über das registrierte Modul (`inline`, mit Dateiname). Mit `save=true` zusätzlich das Download-Recht | +| GET | `/api/v1/external-refs/{id}/mail` | vollständige E-Mail als JSON, Identität per Message-ID geprüft; `html` ist bereinigt, `remote_images` zählt Bilder fremder Server | +| GET | `/api/v1/external-refs/{id}/mail/html-with-images` | dieselbe bereinigte Mail, die Bilder fremder Server lädt der Server auf Anforderung und liefert sie eingebettet (`loaded`, `failed`) | +| GET | `/api/v1/external-refs/{id}/mail/original` | EML unter dem Betreff als Name; verlangt das Download-Recht (403 sonst) | +| GET | `/api/v1/external-refs/{id}/mail/attachments/{index}` | Anlage (`inline`, siehe Review R-03); mit `save=true` das Download-Recht | +| POST | `/api/v1/external-refs/{id}/assignment` | Owner ordnet eine mehrdeutige oder ungebundene Referenz bewusst einer eigenen Instanz zu | -Quelleninhalte werden mit den Zugängen der **anfragenden** Person geladen (Review R-10). +Quelleninhalte einer eigenen Referenz laufen über die gebundene Instanz. Eine freigegebene Referenz mit `use_owner_sources` nutzt den gebundenen Zugang des Owners. Zugangsdaten und die übrigen Treffer dieses Zugangs werden nicht ausgeliefert. Eine Suche ohne `connector_id` wählt bei mehreren eigenen Instanzen nicht den zuletzt angelegten Zugang (Review R-10). ## Übergabepaket @@ -174,7 +215,7 @@ Quelleninhalte werden mit den Zugängen der **anfragenden** Person geladen (Revi |---|---|---| | GET | `/api/v1/matters/{id}/export-tree` | Auswahlbaum mit `exportable` und Grund | | POST | `/api/v1/matters/{id}/export/review` | Prüfung: `ready`, `issues`, `notices`, `entries` mit Paketpfaden | -| POST | `/api/v1/matters/{id}/export` | ZIP nur bei `ready`; Audit `export_created`. Knoten `attachment`, `dispatch-item` und `evidence` gehören zur Auswahl | +| POST | `/api/v1/matters/{id}/export` | ZIP nur bei `ready` und mit Download-Recht am Vorgang (403 sonst); Audit `export_created`. Knoten `attachment`, `dispatch-item` und `evidence` gehören zur Auswahl | ## Anlagen und Versand diff --git a/docs/architecture/BENUTZERVERWALTUNG.md b/docs/architecture/BENUTZERVERWALTUNG.md new file mode 100644 index 0000000..bdf9ca7 --- /dev/null +++ b/docs/architecture/BENUTZERVERWALTUNG.md @@ -0,0 +1,118 @@ +# Benutzerverwaltung, Rollen und Rechte (Ist, 20.10.2026) + +Erster Schnitt der Administration. Er baut auf den vorhandenen Mechanismen auf (Kontext-Grants, Datenräume, Audit über `domain_events`) und führt kein neues Fachmodell ein. Die Auswertung der Objektrechte bleibt unverändert, siehe [RECHTEMODELL_IST.md](RECHTEMODELL_IST.md). + +## Rollen + +Feste Bündel, kein Rolleneditor (Technische Spezifikation 5.2). Rollen werden bei jeder Anfrage aus der Datenbank gelesen, nicht in der Session gehalten (`backend/app/access/roles.py`). + +| Ebene | Rolle | Darf | +|---|---|---| +| Plattform | Betreiber (`is_superadmin`, mehrere möglich) | Konten anlegen, deaktivieren und aktivieren, Passwort zurücksetzen, Sitzungen beenden, Betreiberrolle vergeben und entziehen, Räume mit Verwaltern sehen, Verwalter in einen Raum ohne Verwalter einsetzen, Verwaltungsprotokoll lesen. **Kein Leserecht auf Inhalte.** | +| Plattform | Nutzer | eigenes Passwort ändern, eigene Sitzungen ansehen und beenden | +| Datenraum | Verwalter (`manager`) | Mitglieder und Rollen verwalten, Einladungen, Raum umbenennen, archivieren und wieder aktivieren | +| Datenraum | Mitglied (`member`) | Inhalte im Raum anlegen. Objektrechte weiter nur per Grant | +| Datenraum | Leser (`reader`) | Raum wählen und freigegebene Inhalte lesen. Legt im Raum nichts an. Erhält für Objekte dieses Raums keine Bearbeiten- oder Freigeben-Grants | + +### Schutzregeln + +- Der letzte aktive Betreiber lässt sich weder deaktivieren noch zurückstufen. Niemand deaktiviert das eigene Konto. +- Der letzte aktive Verwalter eines Raums bleibt Verwalter. Er kann weder zurückgestuft noch entfernt werden. +- Wer zum Betreiber befördert wird, verliert alle Sitzungen und meldet sich neu an. +- Konten werden **deaktiviert, nicht gelöscht**. Inhalte, Owner-Bezüge und Protokoll bleiben. Löschen oder Anonymisieren ist ein eigener späterer Schnitt. + +### Wirkung der Leser-Rolle + +Einzelfreigaben (Grants) hängen am Objekt, nicht am Raum. Der Leser begrenzt daher zusätzlich zur Raumfreigabe (siehe unten, dort liest er nur) drei Stellen: + +1. Anlegen von Akte, Vorgang, Notiz, Datei, Referenz, Aufgabe und Frist mit diesem Heimatraum (`ensure_can_create` in `ops.py`). +2. Vergabe neuer Bearbeiten- oder Freigeben-Grants an einen Leser für Objekte dieses Raums (`grant_access`). +3. Beim Zurückstufen auf Leser werden dessen vorhandene Bearbeiten- und Freigeben-Grants auf Objekte dieses Raums in derselben Transaktion entzogen. Lese-Grants bleiben. + +Grants auf Objekte anderer Räume bleiben unberührt. + +## Datenräume, Eigentümer und Raumfreigabe (seit 21.10.2026) + +- **Mehrere Räume:** Jede Person kann beliebig viele gemeinsame Datenräume anlegen. Ein Raum mit nur dem Eigentümer als Mitglied ist **privat**, mit weiteren Mitgliedern **Mehrbenutzer**. Der persönliche Raum bleibt unverändert. +- **Eigentümer** (`data_rooms.owner_id`): der Ersteller. Er ist stets Verwalter, lässt sich nicht entfernen oder zurückstufen und überträgt den Raum an ein aktives Mitglied. Nur er archiviert und stellt wieder her und ändert die Voreinstellung für neue Einträge. Ein Konto, das einen Raum mit weiteren Mitgliedern besitzt, lässt sich erst nach Übertragung deaktivieren. Ist der Eigentümer nicht mehr handlungsfähig, setzt der Betreiber einen neuen ein (protokolliert). Die Eigentümerrolle verleiht kein Leserecht. +- **Raumfreigabe:** Ein Raum ist ein Team. Der Owner teilt eine Akte, Unterakte oder einen Vorgang mit einem oder mehreren seiner Räume, unabhängig davon, wo der Eintrag abgelegt ist. Mitglieder erhalten Rechte nach Rolle (Leser lesen; Mitglied und Verwalter bearbeiten; Weitergeben und Löschen nie), die Freigabe kann auf „nur lesen“ begrenzt werden, einzelne Personen lassen sich ausnehmen. Ein einzelner Vorgang lässt sich teilen, ohne die Akte offenzulegen (Beispiel: nur die Vorgänge, die der Anwalt sehen soll). Die Voreinstellung eines Raums teilt neu angelegte Einträge, die in diesem Raum entstehen, automatisch mit ihm. Der Owner sieht seine Einträge unabhängig von Raum und Mitgliedschaft. Wer ein Team verlässt oder Leser wird, nimmt seine Freigaben an dieses Team mit; wer aus einem Raum entfernt wird, arbeitet danach im persönlichen Raum. Beim ersten weiteren Mitglied zeigt die Oberfläche, was bereits mit dem Raum geteilt ist. Bestehende Freigaben wurden übernommen, nicht geteilte Einträge blieben privat. +- **Quellzugänge:** Die Raumfreigabe gibt keine Paperless- oder Mail-Zugänge des Owners frei. Dafür bleibt die ausdrückliche Freigabe mit der Option nötig. +- **Löschen:** Akte und Vorgang löscht nur der Owner (oder wer ein Löschrecht hat), nicht jeder Bearbeiter. + +## Zugang + +- **Einmal-Passwort:** Der Betreiber legt ein Konto an oder setzt das Passwort zurück. Tsumugu erzeugt ein Passwort und zeigt es einmal an. Das Konto steht auf `must_change_password`, bis die Person ein eigenes festgelegt hat. Bis dahin antworten nur `auth/session`, `auth/password` und `auth/logout`, alles andere mit 403 `password_change_required`. +- **Einladungslink:** Ein Verwalter erstellt eine Einladung mit Rolle. Der Link enthält das Token im URL-Fragment (`/invite/{id}#{token}`), es erreicht also keinen Server-Log. Der Link gilt 7 Tage und einmal und lässt sich widerrufen. Es gibt keinen E-Mail-Versand. +- **Passwortregel:** mindestens 10 Zeichen, höchstens 72 UTF-8-Byte (bcrypt). Beim Ändern des eigenen Passworts enden alle anderen Sitzungen. + +## Anmeldung + +- Die Versuchsbegrenzung steht in der Tabelle `auth_login_attempts` (8 Fehlversuche in 10 Minuten je Bucket). Der Bucket ist ein SHA-256 aus Adresse und E-Mail, die Roh-E-Mail wird nicht gespeichert. Ein erfolgreicher Login löscht den Bucket. +- Unbekannte und deaktivierte Konten laufen durch einen bcrypt-Vergleich und erhalten dieselbe Antwort wie ein falsches Passwort. +- Der CSRF-Vergleich ist konstantzeitlich. `last_seen_at` wird höchstens alle 5 Minuten geschrieben. + +## Audit + +Alle schreibenden Aktionen schreiben in `domain_events`, ohne E-Mail, Passwort und Token. Ressourcenarten: `account`, `session`, `room`, `membership`, `invitation`. Der Betreiber liest sie unter `GET /api/v1/admin/events`. Nicht protokolliert werden fehlgeschlagene Anmeldungen (die Begrenzung zählt sie in `auth_login_attempts`). + +## Bewusst nicht enthalten (Sicherheitskonzept) + +MFA (später optional für alle Konten), Leerlaufablauf und Rotation der Sitzung, Änderung der Sitzungsdauer von 14 Tagen, Härtung des Web-Bootstraps für das erste Betreiberkonto (R-04), Mindestlänge 15, Einschränkung von `GET /api/v1/principals`, E-Mail-Versand, Löschen und Anonymisieren von Konten, Übertragung von Ownership. + +## Bedienung (21.10.2026) + +- **Eine Stelle für Datenräume:** Verwaltung → Datenräume listet alle eigenen Räume mit Zustand, Rolle und Eigentümer. Ein Klick öffnet die Verwaltung des Raums (`/rooms/{id}`): Mitglieder, Rollen, Einladungen, Umbenennen, Voreinstellung, Übertragung, Archivieren. Dafür ist kein Wechsel in den Raum nötig. Der Betreiber sieht darunter zusätzlich alle Räume mit Eigentümer und Verwalter. +- **Nutzer ist etwas anderes:** Verwaltung → Nutzer (nur Betreiber) verwaltet Konten, Verwaltungsprotokoll das Audit. Mein Konto betrifft das eigene Passwort und die Geräte. +- **Ablage mit Ordnerbaum und Kacheln** (Akten, Vorgänge, Räume): Links ein Ordnerbaum („Meine Ablage“ mit Akten und Vorgängen, darunter „Räume“), rechts Kacheln: Ordnersymbol für Akte und Unterakte, Dokumentstapel für Vorgang, Teamordner für Räume. Umschaltbar auf eine kompakte Liste. Ein Klick öffnet den Eintrag, „Inhalt ›“ wechselt in den Ordner (Unterakten und Vorgänge der Akte). Am Handy liegt der Baum hinter „Ordner anzeigen“. +- **Räume sind virtuelle Ordner:** Ein Eintrag kann in mehreren Räumen liegen, ohne verschoben zu werden. „In Raum legen“ und „Aus Raum nehmen“ ändern nur die Zuordnung. Auf jeder Kachel zeigen Marken, in welchen Räumen der Eintrag liegt (nur dem Owner). +- **Vier Wege zum Zuordnen:** (1) im Raum „Hinzufügen“ mit Auswahl eigener Akten und Vorgänge, (2) eine Kachel auf einen Raum im Baum oder auf eine Raumkachel ziehen, (3) Knopf „In Raum legen“ auf der Kachel (auch am Handy), (4) die Marke auf der Kachel öffnen und „Aus Raum nehmen“. Jede Aktion nennt danach, wer Zugriff hat (zum Beispiel „Ben bearbeitet, Cem liest“). +- **Neu anlegen** fragt nicht mehr nach dem Heimatraum. In der Raumansicht entsteht ein Eintrag gleich mit diesem Raum geteilt. + +## Betreiber, Hauptnutzer und Mitarbeiter (seit 22.10.2026) + +| Ebene | Kontoart | Aufgabe | +|---|---|---| +| Betreiber | Hauptnutzer mit `is_superadmin` | SaaS-Betreiber: verwaltet Hauptnutzer, sieht Räume und Team-Übersicht (Metadaten), keine Inhalte | +| Hauptnutzer | `account_kind = main` | privater Raum und eigene Team-Räume; Eigentümer; lädt Mitarbeiter ein und verwaltet sie | +| Mitarbeiter | `account_kind = employee` | arbeitet nur in eingeladenen Räumen nach Raumrolle; legt keine Räume an; kein privater Arbeitsbereich | + +- **Einladung:** nur der Eigentümer des Raums (als Hauptnutzer). Das neue Konto ist ein Mitarbeiter; `sponsor_id` ist der Eigentümer. Ein Verwalter ohne Eigentum nimmt nur bestehende Konten auf. +- **Mitarbeiter-Sitzung:** beginnt im ersten Team-Raum, in dem die Person Mitglied ist. Im persönlichen Raum legt ein Mitarbeiter nichts an. Ohne Raum kann er sich anmelden, aber nichts tun. +- **Eigentümer:** nur ein Hauptnutzer kann Raum-Eigentümer sein, auch bei Übertragung und beim Einsetzen durch den Betreiber. +- **Mein Team** (`/team`): zeigt die Personen in den eigenen Räumen mit Rolle je Raum. Für selbst eingeladene Mitarbeiter: deaktivieren/aktivieren, Einmal-Passwort, Entfernen aus allen eigenen Räumen. Fremde und unbekannte Konten antworten gleich mit 403. +- **Betreiber:** `/admin/team` zeigt Hauptnutzer mit ihren Mitarbeitern, Räumen und Rollen (auch als CSV, mit Schutz vor Formeleingabe). Nur der Betreiber macht einen Mitarbeiter zum Hauptnutzer; die Person meldet sich dann neu an. + +### Vorbereitung Subscription + +`backend/app/access/limits.py` ist die einzige Stelle für Grenzen. `ensure_can_create_room(conn, owner_id)` läuft beim Anlegen eines Raums, `ensure_can_add_member(conn, owner_id, room_id)` beim Aufnehmen, Einladen und Annehmen einer Einladung. Heute lassen beide alles zu. Das Kontingentsubjekt ist der Eigentümer des Raums, ein Hauptnutzer, wie der Verein bei Shinkan (Entitlements-Blueprint). Vorgesehene Messgrößen: aktive Mitarbeiter je Hauptnutzer und je Raum, aus der Datenbank gezählt. Der Betreiber setzt den Plan. Zahlungsdienst und Tarife gehören nicht in diese Stufe. Offene Entscheidung: Ein Mitarbeiter, den ein zweiter Hauptnutzer als bestehendes Konto aufnimmt, zählt beim Raum-Eigentümer; die Zuständigkeit bei Grenzen klärt das Tarifkonzept. + +### Ein Navigationsbaum für Ablage, Akten und Vorgänge (seit 23.10.2026) + +Der Baum „Meine Ablage“ steht links auf den Listenseiten und auf den Seiten von Akte und Vorgang. Die frühere Spalte „Struktur“ ist darin aufgegangen. + +- **Aufklappen mit Fokus:** Ein Klick auf eine Akte oder einen Vorgang öffnet ihn und klappt seine Struktur auf. Der Pfad dorthin bleibt offen, die aktuelle Zeile (auch ein Kapitel) ist hervorgehoben. Nach dem Neuladen der Seite stimmt der Fokus wieder. +- **Unter einer Akte:** Überblick, „Direkt in dieser Akte“, Kapitel mit Unterkapiteln, Unterakten und die Vorgänge der Akte. **Unter einem Vorgang:** „Alle Inhalte“, Kapitel, Teilvorgänge. „Neues Kapitel“ und die Pfeile zum Verschieben stehen im Baum, solange der Eintrag offen ist. +- **Direkt von Vorgang zu Vorgang:** Ein Vorgang erscheint unter jeder Akte, in der er liegt, und unter „Vorgänge“. Ein Klick unter einer Akte öffnet ihn in diesem Aktenkontext, unter „Vorgänge“ ohne Akte. Die frühere Auswahl des Handlungskontexts entfällt. Querverweise und weitere Akten stehen unter dem Titel des Vorgangs. +- **Nachladen:** Die Wurzeln kommen aus `/cases` und `/matters`, die Struktur eines Knotens aus `GET /cases/{id}/outline` und `GET /matters/{id}/outline`, einmal beim Aufklappen. Die Antworten enthalten nur Lesbares (keine Inhalte). Ändert sich der offene Eintrag (Kapitel, Unterakte, Vorgang), lädt der Baum seine Struktur neu. +- **Breite und Dichte:** Der Baum lässt sich an seiner rechten Kante mit der Maus ziehen (200 bis 560 Pixel, Pfeiltasten auf dem Griff, Doppelklick setzt zurück). Die Breite wird im Browser gemerkt (`localStorage`, Schlüssel `tsumugu-tree-width`); ohne Speicher gilt die Standardbreite. Zeilen und Einrückung sind kompakt, damit tiefe Strukturen hineinpassen. +- **Vorgang aus einer Akte lösen:** Im Menü des Vorgangs steht „Aus Akte lösen“, wenn er in Akten liegt. Gelöst wird nur die Zuordnung zur gewählten Akte (`DELETE /matter-case-links/{id}`); der Vorgang, seine Inhalte und seine anderen Akten bleiben. Erlaubt ist es mit Bearbeitungsrecht in der Akte oder mit Teilen-Recht am Vorgang. + +### Herunterladen von Originalen (seit 23.10.2026) + +- **Regel je Rolle:** Das Recht „Herunterladen“ ist neben Lesen, Bearbeiten, Teilen und Löschen ein eigenes Recht. Der Eigentümer hat es immer. Über einen Raum folgt es der Rolle (Verwalter, Mitglied, Leser) laut Regel des Raums; direkt freigegebene Personen folgen dem Wert „direkt“ des Eigentümers. Ohne Leserecht gibt es kein Herunterladen. +- **Wo einstellen:** Voreinstellung am Konto des Hauptnutzers („Mein Konto“, wird beim Anlegen eines Raums übernommen), danach je Raum in der Raumverwaltung (nur Eigentümer). Bestehende Daten bleiben erlaubt (Migration `20261023T090000Z_download_policy.sql`). +- **Was geprüft wird (serverseitig):** Speichern oder Öffnen des Originals (`?save=true` bei Dateien, Paperless-Dokumenten und Mail-Anlagen), die vollständige E-Mail (`.eml`) und das Übergabepaket. Lesen in Tsumugu bleibt möglich; die Regel ist ein Nutzungsrecht, kein Kopierschutz (Abfotografieren oder Entwicklerwerkzeuge verhindert sie nicht). +- **Dateiname:** Gespeichert wird unter dem echten Namen (UTF-8, `filename*`), die E-Mail unter ihrem Betreff. +- **Endpunkte:** `GET/PUT /rooms/{id}/download-policy`, `GET/PUT /auth/download-defaults`. + +### E-Mails als HTML + +Die Mail wird formatiert (Standard) oder als Text gezeigt. Der Server bereinigt das HTML mit einer Positivliste (keine Skripte, Formulare, Rahmen, `" + + `${html}`; +} + +// `remote` describes the pictures of other servers: { key, count, auto, load }. `load` asks the app's server for them. +export function MailBody({ html, text, truncated, notices = [], remote = null }) { + const [view, setView] = useState(html ? "html" : "text"); + const [withImages, setWithImages] = useState(null); + const [busy, setBusy] = useState(false); + const [problem, setProblem] = useState(""); + const mode = html ? view : "text"; + const shownHtml = withImages ? withImages.html : html; + + async function loadImages() { + setBusy(true); + setProblem(""); + try { + setWithImages(await remote.load()); + } catch (err) { + setProblem(err.message || "Die Bilder konnten nicht geladen werden."); + } finally { + setBusy(false); + } + } + + useEffect(() => { + setWithImages(null); + setProblem(""); + if (remote?.auto && remote.count > 0) loadImages(); + }, [remote?.key]); + + return ( +
+ {html ? ( +
+ + +
+ ) : null} + {mode === "html" ? ( + <> + {remote && remote.count > 0 && !withImages ? ( +

+ + Unsere Server laden die Bilder für dich, der Absender sieht deine Adresse nicht. Er kann aber erkennen, dass die Nachricht geöffnet wurde. +

+ ) : null} + {withImages ?

{withImages.loaded} Bild(er) geladen{withImages.failed ? `, ${withImages.failed} nicht verfügbar oder nicht sicher` : ""}.

: null} + {problem ?

{problem}

: null} +