Secure FileIngest Pipeline

The document domain owns upload, scanning, storage, and retrieval as one bounded context. Its architecture reflects a specific security stance: the storage write is not the first step, it's the last.

An upload arrives at the gateway, is streamed to the document domain, and is held before it reaches durable storage. An antivirus engine — deployed into the cluster as its own Helm release rather than called as an external API — scans the payload. Only a clean result produces a write to MinIO and a file.uploaded event on NATS. An infected file never becomes a retrievable object.

Storage is S3-compatible via MinIO running in-cluster, with AWS S3 used elsewhere in the platform. That choice keeps sensitive compliance evidence inside the cluster boundary while leaving the S3 API surface unchanged — the same client code works against either, so the decision is deployment-time rather than baked into the application.

The file.uploaded event lets the rest of the platform react without coupling to this service: the analytics domain counts it, ops logs it, the core domain attaches it to the relevant control. None of them needs to know how storage works.

For an ISO 27001 platform this is table stakes — an auditor will ask how uploaded evidence is protected, and "we scan every file before it's stored, and infected files never reach storage at all" is an answer auditors accept. Running the scanner inside the cluster also means document contents never leave the customer's infrastructure boundary, which matters when the documents are the security evidence.

A virus scan takes real time, and holding an HTTP connection open through it means upload latency is bounded by the slowest scan — with timeouts on large files and no ability to scale scanning independently of ingest.

The event-driven design keeps this tractable: the scan boundary is a decision, not a request/response call. Storage commits and downstream notification are decoupled from the client's connection through NATS, so the user-facing acknowledgement and the fan-out to the analytics, ops and core domains happen on different timelines. Scanning capacity scales as its own deployment.

The ordering exists for one failure mode. If a file is written to storage before the scan result is known, then a crash between write and verdict leaves an unscanned object in a bucket the platform trusts — indistinguishable from a clean one. Ordering the pipeline so the storage write is gated on a clean verdict makes the dangerous state unrepresentable rather than merely unlikely.

The ordering is the sharpest trade-off. Writing to storage first would make uploads feel instant — and a crash between write and verdict would leave an unscanned file in a trusted bucket. For compliance evidence, slower uploads are the right price. Security properties that depend on cleanup running correctly are not security properties.