System Architecture
Core Features
Data Management
Frontend Components
Extensibility
The following files were used as context for generating this wiki page:
The Knowledge Base module provides a robust system for managing organizational documents and manual answers, integrating file storage, database persistence, and asynchronous processing capabilities. It allows users to upload, list, retrieve, and delete documents, as well as manage manual answers, with support for vector database integration for intelligent search and retrieval. The module leverages Amazon S3 for secure file storage and Trigger.dev for orchestrating background processing tasks.
This module is designed to handle various aspects of knowledge management, from initial document ingestion to their eventual processing and deletion, ensuring data consistency across storage, database, and vector store components.
The Knowledge Base module follows a standard NestJS architecture, with a Controller handling incoming API requests and a Service encapsulating the business logic. It interacts with external services such as S3 for file storage and Trigger.dev for asynchronous task orchestration, and persists metadata in a database.
} }Sources: apps/api/src/knowledge-base/knowledge-base.module.ts, apps/api/src/knowledge-base/knowledge-base.controller.ts, apps/api/src/knowledge-base/knowledge-base.service.ts
The KnowledgeBaseController exposes a set of RESTful API endpoints for interacting with knowledge base documents and manual answers.
| Method | Endpoint | Description | DTO / Parameters The Knowledge Base module is a core component that manages the storage, processing, and retrieval of documents and manual answers for an organization. It integrates with Snbsp;S3 for file storage and Trigger.dev for asynchronous processing, providing a robust system for knowledge management. Sources: apps/api/src/knowledge-base/knowledge-base.module.ts, apps/api/src/knowledge-base/knowledge-base.service.ts
The Knowledge Base module follows a standard NestJS architecture, with a Controller handling incoming API requests and a Service encapsulating the business logic. It interacts with external services such as S3 for file storage and Trigger.dev for asynchronous task orchestration, and persists metadata in a database.
Sources: apps/api/src/knowledge-base/knowledge-base.module.ts, apps/api/src/knowledge-base/knowledge-base.controller.ts, apps/api/src/knowledge-base/knowledge-base.service.ts
The KnowledgeBaseController exposes a set of RESTful API endpoints for interacting with knowledge base documents and manual answers.
| Method | Endpoint | Description | DTO / Parameters Sources: apps/api/src/knowledge-base/knowledge-base.controller.ts, apps/api/src/knowledge-base/knowledge-base.service.ts, apps/api/src/knowledge-base/utils/s3-operations.ts
The listDocuments endpoint retrieves all knowledge base documents associated with a given organizationId. It fetches document metadata from the database, including id, name, description, s3Key, fileType, fileSize, processingStatus, createdAt, and updatedAt.
async listDocuments(organizationId: string) {
return db.knowledgeBaseDocument.findMany({
where: { organizationId },
select: {
id: true,
name: true,
description: true,
s3Key: true,
fileType: true,
fileSize: true,
processingStatus: true,
createdAt: true,
updatedAt: true,
},
orderBy: { createdAt: 'desc' },
});
}Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:18-32, apps/api/src/knowledge-base/knowledge-base.controller.ts:40-59
The module provides functionality to generate time-limited, signed URLs for both downloading and viewing documents directly in a browser.
The getDownloadUrl method generates a signed URL that forces the browser to download the file.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:51-64, apps/api/src/knowledge-base/knowledge-base.controller.ts:76-92
The getViewUrl method generates a signed URL intended for inline viewing in the browser. It also indicates whether the file type is generally viewable in a browser based on predefined MIME types.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:66-82, apps/api/src/knowledge-base/knowledge-base.controller.ts:94-115, apps/api/src/knowledge-base/utils/constants.ts:30-32
Deleting a document involves multiple steps to ensure consistency across S3, the database, and any associated vector store embeddings. This process is partially asynchronous to avoid blocking the API response.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:84-122, apps/api/src/knowledge-base/knowledge-base.controller.ts:117-142, apps/api/src/knowledge-base/utils/s3-operations.ts:130-143
The processDocuments endpoint initiates the asynchronous processing of one or more knowledge base documents. This typically involves extracting content and generating vector embeddings for search. The service intelligently dispatches either a single document processing task or an orchestrator task for multiple documents.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:124-154, apps/api/src/knowledge-base/knowledge-base.controller.ts:144-162, apps/api/src/knowledge-base/dto/process-documents.dto.ts
Document processing tasks are handled asynchronously by Trigger.dev. The API immediately returns a runId and a publicAccessToken which can be used to monitor the status of the background job.
The module also supports the management of "manual answers," which are likely pre-defined responses or knowledge entries.
The deleteManualAnswer method removes a specific manual answer from the system. It triggers an asynchronous task to delete associated vector embeddings before removing the record from the main database.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:175-199, apps/api/src/knowledge-base/knowledge-base.controller.ts:182-196
The deleteAllManualAnswers method allows for the bulk deletion of all manual answers belonging to a specific organization. It first identifies all relevant manual answer IDs and then triggers an orchestrator task for batch vector deletion before removing all records from the database.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:201-229, apps/api/src/knowledge-base/knowledge-base.controller.ts:198-207
The s3-operations.ts utility file provides core functions for interacting with Amazon S3, specifically for the knowledge base bucket.
| Function Name | Description ```mermaid erDiagram knowledgeBaseDocument { string id string name string description string s3Key string fileType number fileSize string processingStatus string createdAt string updatedAt string organizationId
securityQuestionnaireManualAnswer { string id string organizationId
organization ||--o{ knowledgeBaseDocument : "manages" organization ||--o{ securityQuestionnaireManualAnswer : "manages"
Sources: [apps/api/src/knowledge-base/knowledge-base.service.ts:18-32](https://github.com/blade47/comp/blob/main/apps/api/src/knowledge-base/knowledge-base.service.ts#L18-L32), [apps/api/src/knowledge-base/knowledge-base.service.ts:175-181](https://github.com/blade47/comp/blob/main/apps/api/src/knowledge-base/knowledge-base.service.ts#L175-L181)
## Constants and Utilities
The `constants.ts` file defines various configuration parameters and helper functions used across the Knowledge Base module.
| Constant/Function Name | Description
Sources: [apps/api/src/knowledge-base/knowledge-base.controller.ts](https://github.com/blade47/comp/blob/main/apps/api/src/knowledge-base/knowledge-base.controller.ts), [apps/api/src/knowledge-base/knowledge-base.service.ts](https://github.com/blade47/comp/blob/main/apps/api/src/knowledge-base/knowledge-base.service.ts), [apps/api/src/knowledge-base/utils/s3-operations.ts](https://github.com/blade47/comp/blob/main/apps/api/src/knowledge-base/utils/s3-operations.ts)
### Listing Documents
The `listDocuments` endpoint retrieves all knowledge base documents associated with a given `organizationId`. It fetches document metadata from the database, including `id`, `name`, `description`, `s3Key`, `fileType`, `fileSize`, `processingStatus`, `createdAt`, and `updatedAt`.
```typescript
async listDocuments(organizationId: string) {
return db.knowledgeBaseDocument.findMany({
where: { organizationId },
select: {
id: true,
name: true,
description: true,
s3Key: true,
fileType: true,
fileSize: true,
processingStatus: true,
createdAt: true,
updatedAt: true,
},
orderBy: { createdAt: 'desc' },
});
}
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:18-32, apps/api/src/knowledge-base/knowledge-base.controller.ts:40-59
The module provides functionality to generate time-limited, signed URLs for both downloading and viewing documents directly in a browser.
The getDownloadUrl method generates a signed URL that forces the browser to download the file.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:51-64, apps/api/src/knowledge-base/knowledge-base.controller.ts:76-92
The getViewUrl method generates a signed URL intended for inline viewing in the browser. It also indicates whether the file type is generally viewable in a browser based on predefined MIME types.
Sources: apps/api/src/knowledge-base/knowledge-base.service.ts:66-82, apps/api/src/knowledge-base/knowledge-base.controller.ts:94-115, apps/api/src/knowledge-base/utils/constants.ts:30-32
Deleting a document involves multiple