Project Name
Auto-Generating Docker-Compose Configs at Runtime Using Node.js Engine
Our client is a multi-tenant SaaS platform provider managing containerised workloads for dozens of customer environments, each requiring a slightly different service topology. Their operations team was maintaining a growing library of hand-written Docker Compose files, one per environment configuration, leading to configuration drift, deployment errors from copy-paste mistakes, and a maintenance burden that scaled linearly with customer count.
Some customers required a Redis cache, others a dedicated message queue, others a combination that no existing static file covered. They required an automated approach that could generate valid, tested Compose files programmatically from a single source of truth, with support for environment-specific overrides, dynamic port allocation, runtime secret injection, and service dependency management.
Dozens of hand-maintained Compose files, configuration drift causing deployment failures, sensitive credentials committed to version control, and a provisioning process that consumed up to 90 minutes of manual effort per new customer environment.
- Configuration Drift at Scale: Dozens of manually maintained Compose files led to inconsistencies in service versions, environment variables, and network settings, resulting in deployment failures that became harder to troubleshoot as environments grew.
- Dynamic Service Topology: Each customer required a different mix of services, making static Compose files impractical and template duplication increasingly error-prone.
- Port Conflict Management: Manual port assignments caused collisions during parallel deployments, requiring automated dynamic port allocation to ensure uniqueness.
- Environment Variable Injection: Credentials, database connections, and feature flags needed to be injected securely at runtime instead of being stored in version-controlled Compose files.
- Dependency Graph Validation: Generated files had to enforce correct service dependencies and health checks to prevent startup race conditions common in manually created configurations.
- Compose Schema Compliance: Every generated YAML file needed automated validation against the Docker Compose v3 schema to eliminate deployment failures caused by invalid configurations.
Ksolves, an AI-first DevOps consulting services company, developed a Node.js-based Docker Compose generation engine that transforms structured JSON configurations into validated Docker Compose YAML files through a REST API. Built on a configuration-as-code approach, the solution delivers consistent, version-controlled, and deployment-ready configurations for CI/CD pipelines.
- JSON-Driven Configuration Schema: A structured JSON schema validates service images, ports, environment variables, volumes, resource limits, and health checks before generating a Compose file.
- Node.js Template Rendering Engine: A modular template engine converts validated JSON into consistent Docker Compose YAML while automatically resolving service dependencies.
- Dynamic Port Allocation: An automated port registry assigns unique host ports across concurrent deployments, eliminating manual conflicts.
- Secrets Manager Integration: Sensitive environment variables are securely retrieved from AWS Secrets Manager at generation time instead of being stored in source code.
- Dependency & Health Check Automation: The engine automatically injects depends_on rules and health checks, ensuring services start in the correct order.
- YAML Schema Validation: Every generated Compose file is validated against the Docker Compose v3 schema before deployment, preventing configuration errors from reaching production.
Technology Stack
| Category | Technology |
|---|---|
| Runtime | Node.js |
| Containerisation | Docker / Docker Compose v3 |
| Secrets Management | AWS Secrets Manager |
| Schema Validation | AJV (JSON Schema Validator) |
| Infrastructure | AWS Lambda + API Gateway |
The solution streamlined deployment workflows, improved consistency, and strengthened operational reliability across customer environments.
- Configuration Drift Eliminated: Standardized generation from a single versioned schema removes configuration drift across environments.
- Provisioning Time Cut from 90 Minutes to Under 10 Seconds: New Docker Compose files are generated through an API in seconds, replacing lengthy manual creation.
- 100% Removal of Credentials from Version Control: Sensitive values are securely retrieved from AWS Secrets Manager instead of being stored in Compose files.
- Near-Zero Cold-Start Failures: Automated dependency resolution and health checks eliminate startup race conditions and improve deployment reliability.
By replacing manually maintained Docker Compose files with a Node.js-based, schema-driven generation engine, Ksolves transformed infrastructure provisioning into a fast, secure, and automated process. The solution eliminated configuration drift, strengthened credential security, accelerated environment provisioning, and improved deployment reliability. More importantly, it established a scalable foundation that can be extended to support Kubernetes manifests, Helm charts, and future infrastructure automation initiatives.
Is the Number of Compose Files You Maintain Growing as Fast as Your Customer Count?
