Relevant source files
The following files were used as context for generating this wiki page:
This document outlines the build pipelines for the api application, covering both the traditional CodeBuild approach and a more streamlined multi-stage Docker build process. It also details the local development setup using Docker Compose and the project's dependency management strategy. The primary goal of these pipelines is to build the api service, package it into a Docker image, and deploy it to an Amazon ECS cluster.
The project utilizes two distinct CodeBuild specifications for the api service: buildspec.yml for a standard, step-by-step build within the CodeBuild environment, and buildspec.multistage.yml which offloads most of the build logic to a multi-stage Dockerfile. For local development and testing, docker-compose.yml provides a convenient way to run the api service using the same multi-stage Docker build approach. Dependency consistency across the monorepo is enforced using syncpack as configured in .syncpackrc.json.
Standard CodeBuild Pipeline (buildspec.yml)
The apps/api/buildspec.yml file defines a comprehensive CodeBuild pipeline for the api application. This pipeline is responsible for fetching dependencies, building the application, creating a Docker image, pushing it to Amazon ECR, and finally updating the Amazon ECS service.
Pipeline Phases
The build process is divided into three main phases: pre_build, build, and post_build.
Pre-Build Phase
This phase focuses on initial setup, including logging into Amazon ECR and preparing environment variables for the build process. It also installs
bun, the JavaScript runtime and package manager used in the project.Key Actions:
- Log in to Amazon ECR using AWS CLI.
- Define
REPOSITORY_URIbased on$ECR_REPOSITORY_URI. - Derive
COMMIT_HASHfrom$CODEBUILD_RESOLVED_SOURCE_VERSIONand setIMAGE_TAG. - Install
bunusing its official installation script.
Build Phase
This is the core of the pipeline, where the application is built and packaged into a Docker image. It involves environment configuration, validation, dependency installation, workspace package building, the main API application build, and meticulous preparation of build artifacts for Docker.
Key Actions:
- Environment Setup: Sets critical environment variables like
PATH,PGSSLMODE,NODE_ENV,NEXT_TELEMETRY_DISABLED,UV_THREADPOOL_SIZE, andNODE_OPTIONS. - Environment Variable Validation: Ensures essential environment variables (e.g.,
DATABASE_URL,BASE_URL, AWS credentials) are set, failing the build if any are missing. - Dependency Installation: Installs only the
apiworkspace dependencies usingbun install --filter=@comp/api. - Workspace Package Building: Builds shared workspace packages (
packages/db,packages/integration-platform). - NestJS Application Build: Navigates to
apps/apiand builds the NestJS application usingbun run build. - Build Output Verification: Checks for the presence of
main.jsin the build output. - Artifact Preparation:
- Creates a
docker-builddirectory. - Copies the built
apiapplication artifacts (handling bothdist/apps/api/srcanddist/srcoutput structures). - Copies the
prismadirectory. - Copies the root
node_modulesdirectory. - Replaces workspace symlinks for
@trycompai/utils,@trycompai/db, and@comp/integration-platformwith their actual built output andpackage.jsonfiles. - Copies the
Dockerfileto thedocker-builddirectory. - Modifies
package.jsonto remove the workspace dependency for@comp/integration-platformbefore copying. - Copies
bun.lock.
- Creates a
- Docker Image Build: Builds the Docker image using the prepared
docker-buildcontext and tags it with the commit hash andlatest.
- Environment Setup: Sets critical environment variables like
Post-Build Phase
This phase handles the deployment of the newly built Docker image.
Key Actions:
- Push the Docker image to Amazon ECR with both the specific
IMAGE_TAGandlatest. - Update the Amazon ECS service to use the new image, forcing a new deployment.
- Generate
imagedefinitions.jsonfor ECS deployment.
- Push the Docker image to Amazon ECR with both the specific
The build process critically depends on several environment variables, which are validated during the build phase. These include database connection strings, base URLs for various services, and AWS credentials for S3 access.
Essential Environment Variables
| Variable Name | Description |
|---|---|
DATABASE_URL | Connection string for the database. |
BASE_URL | Base URL for the application. |
BETTER_AUTH_URL | URL for the authentication service. |
TRUST_APP_URL | URL for the trusted application. |
APP_AWS_BUCKET_NAME | AWS S3 bucket name for application assets. |
APP_AWS_ACCESS_KEY_ID | AWS access key ID for S3. |
APP_AWS_SECRET_ACCESS_KEY | AWS secret access key for S3. |
Build Flowchart
The following flowchart illustrates the detailed steps within the build phase of the standard CodeBuild pipeline.
Sources: apps/api/buildspec.yml:1-105
Multi-stage Docker Build Pipeline (buildspec.multistage.yml)
This alternative pipeline simplifies the CodeBuild process by leveraging a multi-stage Dockerfile (Dockerfile.multistage) to handle the entire build process within a Docker container. CodeBuild's role is reduced to orchestrating the Docker build, pushing the image, and updating the ECS service.
Pipeline Phases
Pre-Build Phase
Similar to the standard pipeline, this phase handles ECR login and image tagging.
Key Actions:
- Log in to Amazon ECR.
- Derive
COMMIT_HASHand setIMAGE_TAG.
Build Phase
The core difference lies here: the actual application build is performed by Docker.
Key Actions:
- Change directory to
apps/api. - Execute
docker buildusingDockerfile.multistagewith theproductiontarget. The build context is set to the monorepo root (../..), allowing the Dockerfile to access all necessary files.
- Change directory to
Post-Build Phase
This phase is identical to the standard pipeline, handling deployment.
Key Actions:
- Push the Docker image to Amazon ECR.
- Update the Amazon ECS service.
- Generate
imagedefinitions.json.
Build Flowchart (Multi-stage)
Sources: apps/api/buildspec.multistage.yml:1-32
Local Development and Testing (docker-compose.yml)
The apps/api/docker-compose.yml file is designed for local development and testing of the api service. It utilizes the same multi-stage Dockerfile (Dockerfile.multistage) as the simplified CodeBuild pipeline to ensure consistency between local and production builds.
Service Configuration
The api service is configured with the following properties:
| Property | Value | Description |
|---|---|---|
build.context | ../.. | The build context is the monorepo root. |
build.dockerfile | apps/api/Dockerfile.multistage | Specifies the multi-stage Dockerfile for building. |
build.target | production | Builds the production stage of the Dockerfile. |
container_name | comp-api-test | Assigns a specific name to the container. |
ports | "3333:3333" | Maps container port 3333 to host port 3333. |
environment | NODE_ENV=production, PORT=3333 | Sets environment variables within the container. |
env_file | .env | Loads environment variables from a local .env file. |
healthcheck | CMD wget ... http://localhost:3333/v1/health | Defines a health check that pings the /v1/health endpoint. |
restart | unless-stopped | Restarts the container automatically unless explicitly stopped. |
This docker-compose.yml is explicitly marked for local testing only. It should not be used for production deployments.
Healthcheck Flow
The healthcheck defined in docker-compose.yml ensures that the api service is running and responsive before marking the container as healthy.
Sources: apps/api/docker-compose.yml:1-24
Dependency Management (.syncpackrc.json)
The .syncpackrc.json file configures syncpack, a tool used to maintain consistency in package dependencies across the monorepo. This ensures that all packages using a particular dependency (e.g., React, Next.js) are on the same version, and that internal workspace packages are correctly referenced.
Configuration Elements
source: Specifies the locations ofpackage.jsonfiles to be managed. This includes the rootpackage.jsonand those withinapps/*/package.jsonandpackages/*/package.json.dependencyTypes: Defines which types of dependenciessyncpackshould manage:prod(dependencies),dev(devDependencies), andpeer(peerDependencies).semverGroups: Defines rules for semantic versioning ranges.- A specific rule ensures that internal packages (prefixed with
@comp/) always useworkspace:*for their version range, indicating they are part of the monorepo.
- A specific rule ensures that internal packages (prefixed with
versionGroups: Defines groups of packages that must have consistent versions across allpackage.jsonfiles in the monorepo. This is crucial for avoiding dependency hell and ensuring a stable build environment.lintRules: Defines rules for forbidden dependencies. For example, it prevents direct dependencies on Node.js built-in modules likecrypto,fs, orpath, which might indicate a misunderstanding of module bundling or environment.
Version Groups Examples
The following table lists some of the key dependency groups enforced by syncpack to maintain consistency:
| Label | Dependencies |
|---|---|
| Use exact versions for internal packages | @comp/** |
| Ensure React is consistent | react, react-dom, @types/react, @types/react-dom, react-is |
| Ensure Next.js is consistent | next |
| Ensure TypeScript is consistent | typescript |
| Ensure common build tools are consistent | postcss, tailwindcss, @tailwindcss/**, autoprefixer |
| Ensure testing tools are consistent | @types/node, prettier, turbo |
| Ensure ESLint is consistent | eslint, eslint-config-next |
Sources: .syncpackrc.json:1-55
On This Page
Standard CodeBuild Pipeline (buildspec.yml)Pipeline PhasesPre-Build PhaseBuild PhasePost-Build PhaseBuild FlowchartMulti-stage Docker Build Pipeline (buildspec.multistage.yml)Pipeline PhasesPre-Build PhaseBuild PhasePost-Build PhaseBuild Flowchart (Multi-stage)Local Development and Testing (docker-compose.yml)Service ConfigurationHealthcheck FlowDependency Management (.syncpackrc.json)Configuration ElementsVersion Groups Examples