System Topology

Acts as the single entry point for the frontend. It strictly handles HTTP routing, payload validation, and interacts directly with the Auth Service to validate JWTs before accepting any documents for upload or queries for search. It immediately acknowledges uploads, publishing an event to RabbitMQ, ensuring a non-blocking UX.

Provides asynchronous decoupling. If 500 documents are uploaded simultaneously, the Gateway does not wait for OCR. RabbitMQ queues the tasks. Worker nodes consume these tasks at their own processing capacity. This ensures the system absorbs traffic spikes without memory exhaustion.

Python worker nodes execute OCR on binary blobs. Extracted text is normalized, chunked into overlapping windows, and passed to a local embedding model. The resulting dense vectors are indexed into a Milvus Vector Database, appended with metadata (Department ID, Access Level) for hybrid search capabilities.

Search queries are intercepted here. A user's query is vectorized and sent to Milvus, but a mandatory pre-filter is applied using the user's Department ID from their validated JWT. Every query, and the resulting documents accessed, are logged to a PostgreSQL audit table for compliance.