Node Backend Template

Production-ready Node.js/Express.js backend template for building scalable, secure APIs with standardized patterns and best practices.

Maintained by: Codebucket Solutions Pvt. Ltd.

📚 Documentation: See docs/ for detailed usage guides. Workflow engine guide: service/v1/workflow-engine/README.md

🤖 Codex / Agent-Ready Additions

This template now includes an additive repository operating layer for Codex and other coding agents while preserving the existing backend template structure.

Added capabilities

• AGENTS.md for repo-level operating guidance
• .codex/config.toml for trusted project-scoped Codex defaults
• docs/ARCHITECTURE.md, docs/WORKFLOW.md, docs/QUALITY.md, docs/SECURITY.md, and docs/RELIABILITY.md
• docs/OBSERVABILITY.md for local logs, metrics, traces, and query workflows
• execution plans under docs/exec-plans/
• deterministic worktree bootstrap script
• repository verification, smoke, and quality scoring scripts
• local developer stack in docker-compose.dev.yml
• CI workflow for baseline enforcement
• Codex app setup guide in docs/CODEX_APP_SETUP.md

Recommended commands

npm run worktree:bootstrap
npm run dev:stack:up
npm run claim:check
npm run verify
npm run plan:new -- --slug=my-task --title="My Task"

---

🚀 Quick Start

Prerequisites

• Node.js 16+ (Node 20+ recommended)
• MySQL 5.7+
• Redis (for production rate limiting)

Installation

# Clone the repository
git clone https://github.com/Codebucket-Solutions/node-template.git
cd node-template

# Install dependencies
npm install

# Set up environment
cp env.development.example .env.development
npm run worktree:bootstrap

# Optional: start MySQL, Redis, and the local observability stack
npm run dev:stack:up

# Run in development mode
npm run start:dev

The server will start on the PORT defined in .env.worktree or .env.development. npm run worktree:bootstrap creates a deterministic .env.worktree with worktree-local stack ports and a compose project name if it does not exist.

---

📁 Project Structure

node-template/
├── bin/
│   └── www                    # Server entry point with clustering
├── config/
│   ├── db.js                  # Database configuration
│   └── migrator.js            # Umzug migration runner
├── controllers/               # Request handlers (versioned)
│   └── v1/
├── helper/                    # Helper utilities
├── instrumentation/           # Optional runtime bootstrap hooks (OpenTelemetry)
├── middleware/                # Express middleware
│   ├── auth.js                # JWT authentication
│   ├── validator.js           # Request validation
│   ├── dispatcher.js          # Permission-based routing
│   ├── rate-limiter.js        # Rate limiting
│   ├── logger.js              # Request logging
│   └── handle-error.js        # Error handling
├── models/                    # Sequelize models
├── migrations/                # Timestamped schema changes
├── routes/                    # API routes (versioned)
│   └── v1/
├── service/                   # Business logic layer
│   └── v1/                    # Includes workflow/assignment orchestration services
├── utils/                     # Utility functions
├── plop-templates/            # Code generation templates
├── app.js                     # Express app configuration
└── package.json

---

🏗️ Architecture

Layered Architecture

Request → Routes → Middleware → Controllers → Services → Models → Database

Middleware Chain:

CORS & Helmet (Security)
   ↓
Global Rate Limiter
   ↓
Body Parser
   ↓
Logger (Pino)
   ↓
Routes
   ↓
Authentication & Authorization (Casbin)
   ↓
Validation (Joi)
   ↓
Controller
   ↓
Service Layer
   ↓
Database (Sequelize ORM)

Versioned API Design

APIs are organized by version for backward compatibility:

/v1/general/endpoint
/v2/general/endpoint

---

✨ Key Features

1. Rate Limiting 🛡️

Environment-aware rate limiting with automatic store selection:

• Development: In-memory store (no setup required)
• Production: Redis store (distributed rate limiting)

Available Limiters:

Limiter	Limit	Window	Use Case
globalLimiter	100 req	15 min	All routes (applied globally)
authLimiter	5 req	15 min	Login/auth endpoints
apiLimiter	30 req	1 min	High-traffic endpoints
uploadLimiter	10 req	1 hour	File uploads

📖 View detailed usage guide →

2. Authentication & Authorization 🔐

• JWT-based authentication with token validation middleware
• Casbin RBAC for fine-grained permission control
• Dispatcher pattern for centralized permission checking

3. Request Validation ✅

Joi-based schema validation middleware for request body, query, and params validation.

4. Logging 📝

• Pino for high-performance structured logging
• Custom log server transport (@codebucket/logserver-transport)
• Request/response logging with correlation IDs

4.1 Observability Harness 📡

• Disabled-by-default OpenTelemetry bootstrap in instrumentation/opentelemetry.js
• Local developer stack for MySQL, Redis, Grafana, Loki, Tempo, and OTLP ingestion in docker-compose.dev.yml
• Terminal query helper for logs, metrics, and traces in scripts/query-observability.js

📖 View observability guide →

5. Database Management 🗄️

• Sequelize ORM with MySQL support
• Transaction support using CLS (Continuation Local Storage)
• Model auto-initialization with migration support
• Umzug-backed migrations with generator and runner scripts
• Master startup schema bootstrap and migration application before worker boot
• Startup DB guardrails that allow automatic bootstrap only against the worktree-local DB stack or an explicit deployment override
• Connection pooling (max 1000 connections)

📖 View migration guide →

5.1 Pagination Queries 📄

• pagi-help/v2 for offset and cursor pagination query generation
• Shared helper exported from utils/pagination.js
• Service-layer usage keeps SQL pagination logic out of routes and controllers

📖 View pagination guide →

5.2 Canonical Integrations 🔌

• @codebucket/files for uploads and file delivery
• @codebucket/sms for SMS provider abstraction
• @codebucket/mail-transport for gateway email transport
• @codebucket/puppet-master for HTML-to-PDF rendering
• Example request flows under routes/v1/examples/, controllers/v1/examples/, and service/v1/examples/

📖 View integration guide →

6. Security 🔒

• Helmet.js - Security headers
• CORS - Cross-origin resource sharing
• Bcrypt - Password hashing
• Input trimming - Automatic whitespace removal
• ClamAV virus scanning (optional)
• Rate limiting - DDoS protection

7. Code Quality 📐

• ESLint - Code linting and quality checks
• Prettier - Automatic code formatting
• EditorConfig - Consistent editor settings across IDEs
• Pre-commit hooks - Automatic linting before commits
• IDE integration - VSCode, WebStorm, and others
• Health and smoke verification - Runtime boot checks in npm run verify

📖 View code quality guide →

8. Code Generation ⚡

Built-in Plop.js generators for scaffolding:

npm run plop

Generates:

• Versioned route files
• Controller boilerplate
• Service layer logic

9. File Upload 📤

• Multer for multipart form handling
• @codebucket/files for filesystem and S3-compatible storage
• Virus scanning support (ClamAV)
• Upload rate limiting

10. Workflow & Assignment Engine 🔄

• Sequelize-backed assignment rules for auto-assignment and reassignment
• Workflow action timeline for application-level state transitions
• Escalation matrix for stage/action-driven routing and auto escalation
• Resolver-based assignee lookup for context-sensitive assignment flows

📖 View workflow engine guide →

10. Email Notifications 📧

• Nodemailer with Zoho SMTP or @codebucket/mail-transport
• OTP templates (sys_otp_template model)
• Template-based email rendering

11. PDF Rendering 🧾

• @codebucket/puppet-master for HTML-to-PDF rendering through the shared Puppet Master service

12. Clustering 🔄

Multi-core support using Node.js cluster module for improved performance and scalability.

---

🛠️ Scripts

# Development (with auto-reload)
npm run start:dev

# Production (Node 20+)
npm start

# Production (Node 16)
npm run start:16

# Code generation
npm run plop

# Local developer stack
npm run dev:stack:up

# Schema migrations
npm run migration:new -- --name=create-example-table
npm run migrate
npm run migrate:status

# Testing
npm run test
npm run test:smoke

# Full verification
npm run verify

---

🌍 Environment Variables

Create .env.development or .env.production. npm run worktree:bootstrap also creates .env.worktree with deterministic local overrides:

# Database
DB_HOST=127.0.0.1
DB_USER=root
DB_PASSWORD=your_password
DB_NAME=your_database

# Server
PORT=9000
NODE_ENV=development

# JWT
JWT_PRIVATE_KEY=your_secret_key

# Email
EMAIL_TRANSPORT=SMTP
EMAIL_HOST=smtp.zoho.in
EMAIL_PORT=465
EMAIL_USER=your_email@example.com
EMAIL_PASSWORD=your_email_password

# Mail server transport (used when EMAIL_TRANSPORT is not SMTP)
MAILSERVER_URL=
MAILSERVER_SENDERID=
MAILSERVER_ACCESS_TOKEN=
# Backward-compatible alias still accepted:
MAILSERVER_ACCESSTOKEN=

# SMS
SMS_TRANSPORT=GOVT
# SMS server transport
SMS_SERVER_URL=
SMS_SENDER_ID=
SMS_ACCESS_TOKEN=

# Mgov transport (canonical names)
MGOV_URL=
MGOV_USERNAME=
MGOV_PASSWORD=
MGOV_SENDER_ID=
MGOV_SECURE_KEY=
# Backward-compatible aliases still accepted:
ENDPOINT=
USERNAMESMS=
PASSWORDSMS=
SENDERID=
KEY=

# Msg91 transport
MSG91_AUTH_KEY=
MSG91_SENDER_ID=
MSG91_ROUTE=
MSG91_COUNTRY=91

# SmartSol transport
SMARTSOL_API_KEY=
SMARTSOL_MASK=
SMARTSOL_PEID=
SMARTSOL_USE_GET=false

# PDF rendering
PUPPET_MASTER_BASE_URL=
PUPPET_MASTER_API_KEY=

# Redis (Production Rate Limiting)
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
REDIS_PASSWORD=

# Optional Features
VIRUS_CHECKER=OFF
BASEURL=http://localhost:9000/

---

📖 Usage Examples

Creating a New API Endpoint

1. Generate files:

npm run plop
# Select: route, controller, service
# Enter: version=v1, group=products, name=create

2. Define model (models/products.js):

module.exports = function (sequelize, DataTypes) {
	return sequelize.define("products", {
		id: { type: DataTypes.INTEGER, primaryKey: true, autoIncrement: true },
		name: { type: DataTypes.STRING, allowNull: false },
		price: { type: DataTypes.DECIMAL(10, 2), allowNull: false },
	});
};

3. Implement service (service/v1/products.js):

const { products } = require("../../models");

const createProduct = async data => {
	return await products.create(data);
};

module.exports = { createProduct };

4. Implement controller (controllers/v1/products.js):

const { createProduct } = require("../../service/v1/products");

const create = async (req, res) => {
	const product = await createProduct(req.body);
	return res.json({ success: true, data: product });
};

module.exports = { create };

5. Define route (routes/v1/products/create.js):

const express = require("express");
const router = express.Router();
const { dispatcher, validateToken, apiLimiter } = require("../../../middleware");
const { create } = require("../../../controllers/v1/products");
const { PERMS, RESOURCES } = require("../../../utils/constant");

router.post("/", validateToken, apiLimiter, (req, res, next) =>
	dispatcher(req, res, next, create, RESOURCES.PRODUCTS, PERMS.ADD),
);

module.exports = router;

---

🐳 Docker Deployment

Basic Deployment

docker build -t node-template .
docker run -p 3000:3000 --env-file .env.production node-template

Docker Compose

docker-compose up -d

Production Setup:

• Uses internal registry: harbor.internal.codebuckets.in
• NPM registry: https://npm.internal.codebuckets.in

---

🧪 Testing Rate Limits

# Make multiple requests to test rate limiting
for i in {1..101}; do
  curl http://localhost:9000/v1/endpoint
done

# Check rate limit headers
curl -i http://localhost:9000/v1/endpoint

Response Headers:

RateLimit-Limit: 100
RateLimit-Remaining: 99
RateLimit-Reset: 900

Rate Limit Exceeded (429):

{
	"success": false,
	"message": "Too many requests from this IP, please try again later."
}

---

🔧 Configuration Tips

Adjusting Rate Limits

Edit middleware/rate-limiter.js:

const customLimiter = createLimiter({
	windowMs: 10 * 60 * 1000, // 10 minutes
	max: 50, // 50 requests
	message: { success: false, message: "Custom message" },
	prefix: "rl:custom:",
});

User-Based Rate Limiting

const userLimiter = createLimiter({
	windowMs: 15 * 60 * 1000,
	max: 200,
	keyGenerator: req => req.user?.id || req.ip,
});

Increasing Worker Processes

Edit bin/www:

const cpu = require("os").cpus().length;
for (let i = 0; i < cpu; i++) {
	// Use all CPU cores
	cluster.fork();
}

---

📚 System Models

Included System Tables

• sys_tables - Dynamic table configuration
• sys_dropdown_list - Dropdown options
• sys_otp_template - OTP email templates
• casbin_rule - RBAC policies

---

🎯 Best Practices

✅ Always use transactions for multi-step database operations
✅ Apply appropriate rate limiters to different endpoint types
✅ Validate all inputs using Joi schemas
✅ Use Casbin dispatcher for permission checks
✅ Log important events with proper context
✅ Handle errors centrally using error middleware
✅ Use environment variables for configuration
✅ Version your APIs for backward compatibility
✅ Test thoroughly before production deployment

---

🤝 Contributing

This is an internal template maintained by Codebucket Solutions. For issues or suggestions, please contact the development team.

---

📄 License

ISC License - Copyright © Codebucket Solutions Pvt. Ltd.

---

🆘 Troubleshooting

Database Connection Refused

# Check MySQL is running
mysql.server status

# Verify credentials in .env
DB_HOST=127.0.0.1
DB_USER=root
DB_PASSWORD=your_password

Redis Connection Failed (Production)

# Check Redis is running
redis-cli ping  # Should return PONG

# System will fallback to memory store with warning

Rate Limiter Not Working

• Verify middleware is imported in app.js
• Check environment configuration
• Review console logs for initialization messages

Port Already in Use

# Find process using port 9000
lsof -ti:9000 | xargs kill -9

# Or change PORT in .env
PORT=9001

---

📞 Support

For internal support, contact: team@thecodebucket.com

Repository: https://github.com/Codebucket-Solutions/node-template

Symphony support

This template now includes a root WORKFLOW.md, scripts/symphony-bootstrap.sh, and docs/SYMPHONY_SETUP.md so it can be used with OpenAI Symphony in addition to Codex worktree workflows.