It's my birthday! 🎂. To celebrate, I want to teach you about creating SERVERLESS MCPS 💪!

In lieu of a present, I'll accept connections on LinkedIn

Here's what this blog post, when deployed, will look like:

Find the complete code on GitHub.

The Problem: AI Assistants Need Better Access to Your World

Picture this: You're using Claude to help with your daily work, but it needs access to your company's internal APIs, databases, or custom tools. Traditional approaches involve:

• Hard-coding integrations that break with every update

• Managing always-on servers that sit idle 99% of the time

• Complex authentication flows that are either insecure or user-hostile

• Paying for infrastructure whether it's used or not

Enter the Model Context Protocol (MCP) - Anthropic's game-changing open standard that's revolutionizing how AI assistants interact with the digital world. Learn more at modelcontextprotocol.io

Why Serverless MCP Changes Everything

Here's the killer combination: MCP + Serverless = Magic

MCP servers are naturally request/response based - they wake up when Claude needs something, do their job, and go back to sleep. This is exactly what serverless excels at. You're not paying for idle servers waiting for the occasional request from an AI assistant. You're paying for actual usage - pennies per thousands of requests.

But here's where it gets really interesting: The serverless advantage compounds when you consider the MCP ecosystem. Imagine thousands of specialized MCP servers, each handling different tools and data sources:

• Your CRM integration MCP server

• Your analytics dashboard MCP server

• Your code deployment MCP server

• Your customer support MCP server

In a traditional architecture, you'd need infrastructure for each. With serverless? They all scale to zero when not in use. You could have 100 MCP servers and pay nothing when they're idle.

What We're Building: A Production Blueprint

This isn't another "Hello World" tutorial. We're building a production-ready, secure, scalable MCP server that demonstrates enterprise-grade patterns you can use immediately. Our example - a Dog Facts server - is intentionally simple so we can focus on the architecture that matters.

The Three-Pillar Architecture

We've designed a modular system with three distinct components, each handling a critical piece of the puzzle:

┌─────────────────────┐    ┌─────────────────────┐    ┌─────────────────────┐
│  McpAuthConstruct   │───▶│ McpLambdaConstruct  │───▶│McpApiGatewayConstruct│
│   "The Gatekeeper"  │    │    "The Worker"     │    │   "The Gateway"     │
│                     │    │                     │    │                     │
│ • OAuth 2.0 + PKCE  │    │ • Your Logic Here   │    │ • Auto-discovery    │
│ • Self-registration │    │ • Pay-per-request   │    │ • RFC 9728 Support  │
│ • Enterprise SSO    │    │ • Scales to zero    │    │ • Custom domains    │
└─────────────────────┘    └─────────────────────┘    └─────────────────────┘

The Authentication Revolution: Custom Dynamic Client Registration

Here's something most developers don't know exists: Dynamic Client Registration (DCR). It's OAuth 2.0's best-kept secret and it's perfect for the MCP ecosystem.

Instead of manually configuring OAuth clients for every tool that wants to connect to your MCP server, DCR allows clients to register themselves programmatically. Claude Code (an MCP client in Anthropic's tooling) can literally say "Hi, I'd like to access this MCP server" and get its own OAuth credentials automatically.

Critical Implementation Detail: AWS Cognito doesn't natively implement RFC 7591 Dynamic Client Registration. What we're building is a custom DCR-compatible endpoint that uses API Gateway and VTL templates to call Cognito's CreateUserPoolClient API. This gives us the DCR workflow while working within AWS's authentication infrastructure, but with limitations: our endpoint is an API Gateway facade over CreateUserPoolClient, not full RFC 7591 registration semantics.

Why This Matters for MCP

Traditional OAuth flow:

1. Developer manually creates OAuth client in console

2. Copies client ID and secret

3. Configures application

4. Hopes nothing breaks

DCR-enabled MCP flow:

1. Claude Code discovers your MCP server

2. Registers itself automatically

3. Starts using your tools immediately

This is the difference between minutes and days of setup time.

The Implementation Secret

Here's a crucial implementation detail that took me longer than I'd like to admit to figure out:

// IMPORTANT: Must use CLASSIC_HOSTED_UI for DCR support
managedLoginVersion: ManagedLoginVersion.CLASSIC_HOSTED_UI,

AWS Cognito's newer managed login UI doesn't support dynamically registered clients because each client requires a theme configuration that doesn't get created automatically. This is a practical gotcha discovered during implementation. Use Classic Hosted UI for DCR compatibility until AWS guarantees branding defaults for dynamic clients. See Managed Login documentation for the newer option's requirements.

The Serverless Advantage in Action

Let's talk real numbers. A typical MCP server handling 1,000 requests per day:

Traditional Server (t3.micro):

• Monthly cost: ~$7.50*

• Always running, mostly idle

• Fixed capacity

• Maintenance overhead

Serverless MCP:

• Monthly cost: ~$0.20**

• Scales to zero

• Scales automatically within account concurrency and per-function limits

• Minimal ops overhead (soft limit defaults, request increases available via AWS Service Quotas)

* EC2 instance cost only, excluding storage, data transfer, load balancer (~$16-25/month)
** Lambda: 30,000 invocations × 100ms avg × $0.0000166667/GB-second at 2048MB = $0.67/month + $0.006 requests = $0.68/month (AWS Lambda Pricing)

But here's the real kicker: Most MCP servers will handle far fewer than 1,000 requests per day. Your internal tool integration might get 10 requests a week. With serverless, you're paying fractions of a penny.

More importantly... if you practice Domain Driven Development and end up having an MCP per knowledge-domain it ends up being ~$7.50 per knowledge domain. If you're an indie dev and want to incorporate MCPs into your side projects... that could end up being a lot OR you have to combine them / more carefully manage them. If you're practicing with Ephemeral CDK Stacks maybe that's not a big deal though.

Building Your First Serverless MCP Server

Let's build something real. Here's our complete Dog Facts MCP server:

export class DogFactsServer implements IMCPServer {
  initialize(): MCPInitializeResult {
    return {
      protocolVersion: "2025-06-18",  // Latest MCP specification
      capabilities: { tools: {} },
      serverInfo: { name: "dog-facts-server", version: "1.0.0" }
    };
  }

  listTools(): MCPToolsListResult {
    return {
      tools: [{
        name: "getDogFacts",
        description: "Get random facts about dogs",
        inputSchema: {
          type: "object",
          properties: {
            limit: {
              type: "number",
              description: "Maximum facts to return (1-10)",
              minimum: 1,
              maximum: 10,
              default: 5
            }
          }
        }
      }]
    };
  }

  async callTool(params: MCPToolCallParams): Promise<MCPToolCallResult> {
    const { name, arguments: args } = params;

    if (name === "getDogFacts") {
      const limit = Math.min(Math.max((args?.limit as number) || 5, 1), 10);
      const response = await fetch(`https://dogapi.dog/api/v2/facts?limit=${limit}`);
      const data = await response.json();
      const facts = data.data.map(fact => fact.attributes.body);

      return {
        content: [{
          type: "text",
          text: facts.map((fact, i) => `${i + 1}. ${fact}`).join('\n\n')
        }]
      };
    }

    throw new Error(`Unknown tool: ${name}`);
  }
}

This simple tool (getDogFacts) is just making a fetch request to an external service. Instead you could integrate this with DynamoDB queries, add a knowledge base, react-agents... you can pretty much do anything here.

This is all the code you need. The framework handles:

• OAuth authentication

• JSON-RPC protocol

• Error handling

• CORS

• Scaling

• Monitoring

Advanced Patterns for Production

Pattern 1: Multi-Tool Servers

Don't create separate MCP servers for related functionality or knowledge domains. Bundle them:

export class CompanyToolsServer implements IMCPServer {
  listTools() {
    return {
      tools: [
        { name: "searchEmployees", /* ... */ },
        { name: "getRoomAvailability", /* ... */ },
        { name: "submitExpenseReport", /* ... */ },
        { name: "checkDeploymentStatus", /* ... */ }
      ]
    };
  }
}

One Lambda, multiple tools, single authentication flow.

Since these are also packaged as CDK Constructs, it would be easy to create an inner-sourced construct library with MCPs and then have them as default items in your stacks... each stack could automatically get its own MCP! To be clear... I would centralize cognito, so that you only have to manage one user pool and then you'd get access to all of the MCPs in your AWS account.

Pattern 2: Async Operations with Step Functions

For long-running operations, combine MCP with Step Functions:

async callTool(params) {
  if (params.name === "generateReport") {
    // Start Step Function execution
    const executionArn = await startReportGeneration(params);

    return {
      content: [{
        type: "text",
        text: `Report generation started. Check status with execution ID: ${executionArn}`
      }]
    };
  }
}

The Enterprise Story

Imagine you're a Fortune 500 company with hundreds of internal tools. Traditional approach:

• Months of integration work per tool

• Expensive API gateway infrastructure

• Complex authentication federation

• Ongoing maintenance nightmare

With serverless MCP:

1. Week 1: Deploy MCP framework

2. Week 2: First 10 tools integrated

3. Month 1: 50 tools available to Claude

4. Month 2: Entire organization using AI-powered tools

Cost for 50 MCP servers handling 100,000 requests/month total: ~$67/month*

Cost for traditional infrastructure: ~$375/month plus maintenance

* 2M Lambda invocations × 150ms × $0.0000166667/GB-second × 2GB = $50/month + $0.40 requests + API Gateway ~$17/month (AWS Lambda Pricing)

Security: Enterprise-Grade by Default

Our implementation includes:

• OAuth 2.0 with PKCE: No client secrets, perfect for public clients. Note: Refresh tokens for public clients have shorter lifespans and require proper rotation handling

• Cognito User Pools: Enterprise SSO ready

• Scope-based authorization: Fine-grained access control

• API Gateway throttling: Rate limiting and AWS Shield Standard reduce volumetric risk (organizations should add WAF and rate plans for comprehensive protection)

• CloudWatch integration: Full audit trail

• VPC options: Private connectivity when needed

Deploy Your Own in ~10 Minutes*

# Clone the repository
git clone https://github.com/martzmakes/mcp-cdk-lambda-cognito
cd mcp-cdk-lambda-cognito

# Install dependencies
npm install

# Deploy to AWS (custom domain required for .well-known endpoints)
npx cdk deploy

# Done! Your MCP server URL will be in the outputs

* Custom domain is required because Claude Code and other MCP clients use the base domain for .well-known paths, which bypasses API Gateway's stage path. ACM DNS validation adds 2-10 minutes for certificate creation.

What You Can Build: Real-World Examples

Customer Support MCP

tools: [
  "searchTickets",
  "updateTicketStatus", 
  "getCustomerHistory",
  "escalateToManager"
]

Let Claude handle tier-1 support with access to your ticketing system.

DevOps MCP

tools: [
  "checkDeploymentStatus",
  "rollbackRelease",
  "queryMetrics",
  "pageOnCall"
]

Claude becomes your intelligent ops assistant.

Analytics MCP

tools: [
  "runSQLQuery",
  "generateReport",
  "exportDashboard",
  "scheduleAlert"
]

Natural language to insights, instantly.

The Future is Serverless MCP

We're in the midst of a revolution. With major platforms like OpenAI, Google DeepMind, and Microsoft adopting MCP in 2025, the serverless advantage becomes overwhelming:

• Ecosystem explosion: Thousands of MCP servers, all scaling independently

• Universal adoption: OpenAI adoption March 26, 2025, Google DeepMind April 2025, Microsoft Copilot Studio GA May 29, 2025

• Cost approaching zero: Pay only for actual AI assistance

• Instant integration: Tools become AI-ready in minutes, not months

• Standardization wins: The MCP specification 2025-06-18 is now the widely adopted open standard

Conclusion: Why This Matters

MCP has fundamentally changed how we think about AI integration. With 2025's universal adoption across all major AI platforms, serverless makes it economically viable at any scale. Together, they're democratizing AI-powered automation.

The architecture we've built here isn't just another example - it's a production blueprint you can use today to start building the AI-integrated future. Whether you're a startup looking to give Claude access to your tools or an enterprise wanting to modernize your AI strategy, serverless MCP is your answer.

With OpenAI, Google, and Microsoft all standardizing on MCP, the question isn't whether to build MCP servers. It's how many you'll build this month.

Next Steps

1. Deploy the example: Get hands-on with the code

2. Build your first custom server: Start with one internal tool

3. Share with the community: The MCP ecosystem grows with every contribution

Ready to dive deeper into serverless patterns? Check out my post on securing serverless apps with Cognito's managed login pages.

Find the complete code on GitHub.

Technical Deep Dive: Implementation Details

The Three-Construct Architecture: Deep Dive

Our modular approach separates concerns into three specialized constructs, each with a clean interface and specific responsibility. Here's how intermediate CDK developers can leverage this pattern:

McpAuthConstruct: The OAuth Foundation

Full implementation: lib/constructs/mcp-auth-construct.ts

export interface McpAuthConstructProps {
  serverName: string;  // Only required input
}

export interface McpAuthResult {
  userPool: UserPool;
  resourceServer: UserPoolResourceServer;
  oauthScopes: OAuthScope[];
  clientId: string;
  authUrl: string;
  tokenUrl: string;
  oauthScope: string;
  signInUrl: string;
}

The Critical DCR Implementation Detail: Here's the configuration that enables Dynamic Client Registration:

// IMPORTANT: Must use CLASSIC_HOSTED_UI for DCR support
const userPoolDomain = userPool.addDomain("McpAuthUserPoolDomain", {
  cognitoDomain: {
    domainPrefix: `mcp-${serverName}-${domainHash}`,
  },
  managedLoginVersion: ManagedLoginVersion.CLASSIC_HOSTED_UI, // 🔑 KEY!
});

Source: mcp-auth-construct.ts:65-67

Why this matters: AWS Cognito's newer managed login UI doesn't support dynamically registered clients because each client requires a theme configuration that doesn't get created automatically. This single line is the difference between DCR working and spending hours debugging OAuth flows.

Domain Collision Prevention: Notice the domainHash pattern:

const domainHash = this.node.addr.substring(0, 8);
const domainPrefix = `mcp-${serverName}-${domainHash}`;

Source: mcp-auth-construct.ts:60-63

This uses CDK's internal node addressing to create unique domain prefixes, preventing conflicts when multiple developers deploy the same stack.

McpLambdaConstruct: Optimized for Performance

Full implementation: lib/constructs/mcp-lambda-construct.ts

this.lambdaFunction = new NodejsFunction(this, "function", {
  entry: path.join(__dirname, "../lambda/mcp.ts"),
  functionName: `mcp-server-${serverName}`,
  memorySize: 2048,                    // Sweet spot for CPU allocation
  timeout: Duration.seconds(29),       // Just under API Gateway's 30s limit*
  architecture: Architecture.ARM_64,   // 20% better price/performance
  runtime: Runtime.NODEJS_22_X,       // Latest runtime for better cold starts
  environment: {
    LOG_LEVEL: logLevel,
  },
  bundling: {
    format: OutputFormat.ESM,          // Modern modules, better tree-shaking
    mainFields: ["module", "main"],    // Prioritize ES modules
  },
});

Source: mcp-lambda-construct.ts:38-53

Performance Optimization Notes:

• ARM64: Graviton2/3 processors offer significantly better price/performance

• 2048MB: This memory allocation provides optimal CPU-to-memory ratio for most workloads

• ESM Format: Modern bundling reduces cold start times and bundle size

• 29-second timeout: Keeps us safely under API Gateway's 30-second limit*

* Since mid-2024, some REST API timeouts can be increased via quota requests, but 29s remains the safe default for user-facing requests

McpApiGatewayConstruct: The Complex Integration Layer

Full implementation: lib/constructs/mcp-api-gateway-construct.ts

This is where the CDK complexity really shows its value. Here's the DCR endpoint implementation:

private createDcrEndpoint(api: RestApi, userPool: UserPool, serverName: string) {
  // Create IAM role for API Gateway to call Cognito
  const cognitoIntegrationRole = new Role(this, "CognitoIntegrationRole", {
    assumedBy: new ServicePrincipal("apigateway.amazonaws.com"),
    inlinePolicies: {
      CognitoAccess: new PolicyDocument({
        statements: [
          new PolicyStatement({
            effect: Effect.ALLOW,
            actions: ["cognito-idp:CreateUserPoolClient"],
            resources: [userPool.userPoolArn],
          }),
        ],
      }),
    },
  });

  // AWS Integration directly with Cognito API
  // Note: This is a custom DCR implementation, not native RFC 7591 support
  const dcrIntegration = new AwsIntegration({
    service: "cognito-idp",
    action: "CreateUserPoolClient",
    options: {
      credentialsRole: cognitoIntegrationRole,
      requestTemplates: {
        "application/json": `#set($rawName = $input.path('$.client_name'))
#if(!$rawName || $rawName == "")
  #set($rawName = "client")
#end
#set($name1 = $rawName.trim())
#set($name2 = $name1.replaceAll("[^\\\\w\\\\s+=,.@-]", ""))
#set($safeName = $util.escapeJavaScript($name2))
#if($safeName.length() > 128)
  #set($safeName = $safeName.substring(0,128))
#end

#set($cb = $input.json('$.redirect_uris'))
#if(!$cb) #set($cb = '[]') #end
{
  "UserPoolId": "${userPool.userPoolId}",
  "ClientName": "$safeName",
  "CallbackURLs": $cb,
  "AllowedOAuthFlows": ["code"],
  "AllowedOAuthFlowsUserPoolClient": true,
  "AllowedOAuthScopes": ["mcp-${serverName}/${serverName}", "openid", "email", "profile"],
  "SupportedIdentityProviders": ["COGNITO"],
  "GenerateSecret": false  // 🔑 This enables PKCE!
}`
      }
    }
  });
}

Source: mcp-api-gateway-construct.ts:315-398

VTL Template Deep Dive: This Velocity Template Language (VTL) template is doing critical work:

1. Input Sanitization: Removes dangerous characters from client names

2. Length Validation: Ensures client names don't exceed Cognito's 128-char limit

3. PKCE Configuration: "GenerateSecret": false is what makes PKCE work

4. Scope Assignment: Automatically assigns the correct MCP scope

The Response Transformation:

responseTemplates: {
  "application/json": `{
    "client_id": $input.json('$.UserPoolClient.ClientId'),
    "client_name": $input.json('$.UserPoolClient.ClientName'),
    "redirect_uris": $input.json('$.UserPoolClient.CallbackURLs'),
    "response_types": ["code"],
    "grant_types": ["authorization_code"],
    "token_endpoint_auth_method": "none",  // PKCE indicator
    "scope": "mcp-${serverName}/${serverName} openid email profile"
  }`
}

This transforms Cognito's API response into RFC 7591-compliant DCR response format.

OAuth Metadata Endpoints: RFC Compliance Made Easy

OAuth Protected Resource Metadata (RFC 9728, published April 2025):

const metadataIntegration = new MockIntegration({
  integrationResponses: [{
    statusCode: "200",
    responseTemplates: {
      "application/json": JSON.stringify({
        resource_name: `${serverName} MCP Server`,
        resource: finalApiUrl,
        authorization_servers: [
          `https://${customDomain.customDomainName}/.well-known/oauth-authorization-server`
        ],
        scopes_supported: [`mcp-${serverName}/${serverName}`],
        bearer_methods_supported: ["header"],
      }, null, 2)
    }
  }]
});

RFC 8414 Authorization Server Metadata:

const authServerMetadataIntegration = new MockIntegration({
  integrationResponses: [{
    responseTemplates: {
      "application/json": JSON.stringify({
        issuer: oauthConfig.authUrl.split("/oauth2/authorize")[0],
        authorization_endpoint: oauthConfig.authUrl,
        token_endpoint: oauthConfig.tokenUrl,
        registration_endpoint: `https://${customDomain.customDomainName}/connect/register`,
        response_types_supported: ["code"],
        grant_types_supported: ["authorization_code", "client_credentials"],
        code_challenge_methods_supported: ["S256"],
        token_endpoint_auth_methods_supported: ["client_secret_post", "client_secret_basic", "none"]
      }, null, 2)
    }
  }]
});

Using MockIntegration here is a CDK pattern that lets us return static JSON without invoking Lambda, keeping costs near zero for metadata requests.

Critical Architecture Note: Custom domains are required for MCP servers because clients like Claude Code make .well-known requests to the base domain (e.g., https://example.com/.well-known/oauth-protected-resource), which bypasses API Gateway's stage path entirely. Without a custom domain, these discovery endpoints would return 404s.

Stack Orchestration: The 78-Line Marvel

Full implementation: lib/mcp-cdk-lambda-cognito-stack.ts

export class McpCdkLambdaCognitoStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: McpCdkLambdaCognitoProps) {
    super(scope, id, props);

    const serverName = "dog-facts";
    const customDomainName = `mcp-dogfacts.martzmakes.com`;

    // Certificate and DNS setup
    const hostedZone = HostedZone.fromLookup(this, "HostedZone", {
      domainName: "martzmakes.com",
    });

    const certificate = new Certificate(this, "Certificate", {
      domainName: customDomainName,
      validation: CertificateValidation.fromDns(hostedZone),
    });

    // Three-construct composition
    const authConstruct = new McpAuthConstruct(this, "Auth", { serverName });
    const lambdaConstruct = new McpLambdaConstruct(this, "Lambda", { serverName });
    const apiGatewayConstruct = new McpApiGatewayConstruct(this, "ApiGateway", {
      serverName,
      lambdaFunction: lambdaConstruct.lambdaFunction,
      userPool: authConstruct.result.userPool,
      resourceServer: authConstruct.result.resourceServer,
      oauthScopes: authConstruct.result.oauthScopes,
      oauthConfig: {
        clientId: authConstruct.result.clientId,
        authUrl: authConstruct.result.authUrl,
        tokenUrl: authConstruct.result.tokenUrl,
        scope: authConstruct.result.oauthScope,
      },
      customDomain: { customDomainName, certificate, hostedZone },
    });
  }
}

Pattern Highlights for CDK Users:

1. Construct Dependency Flow: Auth → Lambda → API Gateway, with clean interfaces

2. Custom Domain Integration: ACM certificate with DNS validation

3. Result Object Pattern: Each construct exposes a clean result interface

4. Resource Naming: Consistent serverName-based naming throughout

Advanced CDK Patterns in Action

Gateway Responses for OAuth Compliance:

new GatewayResponse(this, "UnauthorizedResponse", {
  restApi: api,
  type: ResponseType.UNAUTHORIZED,
  statusCode: "401",
  responseHeaders: {
    "WWW-Authenticate": "'Bearer realm=\"MCP Server\", error=\"invalid_request\"'",
  },
});

This ensures proper HTTP error responses that comply with OAuth 2.0 Bearer Token specification.

CORS Configuration for MCP Clients:

defaultCorsPreflightOptions: {
  allowOrigins: Cors.ALL_ORIGINS,
  allowMethods: Cors.ALL_METHODS,
  allowHeaders: Cors.DEFAULT_HEADERS.concat(["Authorization"]),
}

Essential for browser-based MCP clients that need to make cross-origin requests.

The Type System That Saves You

Our construct interfaces enforce compile-time correctness:

export interface McpApiGatewayConstructProps {
  serverName: string;
  lambdaFunction: NodejsFunction;
  userPool: UserPool;
  resourceServer: UserPoolResourceServer;
  oauthScopes: OAuthScope[];
  oauthConfig: {
    clientId: string;
    authUrl: string;
    tokenUrl: string;
    scope: string;
  };
  customDomain: {
    customDomainName: string;
    certificate: Certificate;
    hostedZone: IHostedZone;
  };
}

You literally cannot wire constructs incorrectly. TypeScript prevents entire categories of deployment failures.

(dog tax from my late pup Astro)

Let's build the future of AI integration together. One serverless function at a time.

If you’ve ever stitched together a backend client and an LLM agent and wondered why you’ve got two subtly divergent ways of calling your API, this one's for you.

The code for this post is here: https://github.com/martzmakes/cdk-zod-agent

Why AWS, CDK, and Serverless?

As an AWS Community Builder, I’m passionate about leveraging AWS’s powerful cloud-native services to build scalable, reliable, and cost-effective solutions. This project is built from the ground up using AWS CDK (Cloud Development Kit) to define infrastructure as code, and it’s fully serverless—every API endpoint is powered by AWS Lambda, with DynamoDB as the persistent data store.

AWS CDK: Infrastructure as Code, Supercharged

The AWS CDK lets you define your entire cloud architecture in TypeScript, making it easy to version, review, and evolve your infrastructure alongside your application code. In this project, the lib/constructs/ directory contains reusable CDK constructs for DynamoDB tables, Lambda functions, and internal API routing. The stack is defined in lib/cdk-zod-agent-stack.ts, and you can deploy the whole system with a single command.

If you’d like to learn more about using AWS CDK, I put together a crash course of it for freeCodeCamp: https://www.youtube.com/watch?v=T-H4nJQyMig

Why Zod?

Zod lets you define schemas for your data and validate at runtime, giving you the confidence of TypeScript types with the safety net of runtime checks. In this project, every API endpoint is defined with Zod schemas for both requests and responses. That means whether you’re adding a new hero or logging a daring rescue, you know your data is always in tip-top shape.

Here’s the core insight: define your endpoints and schemas once with Zod, and you get two clients for the price of one—your human backend code and your ReAct agent both use the same strongly-typed, runtime-validated API client.

No discrepancies. No parallel validation stacks. No translation layers. Just one clean, unified source of truth.

The API Client Framework: Your Hero Utility Belt

Inside the package/ folder, you’ll find the core of the framework:

• Endpoint Definitions: Each endpoint is defined with defineZodEndpoint, specifying the path, HTTP method, and Zod schemas for request/response.

• Client Generator: createApiClient takes these definitions and generates a fully-typed client. Each method validates input/output with Zod.

• Path Parameter Magic: The framework auto-generates Zod schemas for path parameters.

The real magic happens in helpers/endpoints.ts:

• TypeScript Path Parameter Extraction: Recursively parses a path like /heroes/{hero}/rescues/{rescueId}into { hero: string; rescueId: string }.

• Zod Path Parameter Schemas: Extracts {param} segments from paths and builds runtime validation schemas.

• Endpoint Definition: Validates both request and response payloads at runtime and compile time.

Example:

addHero: defineZodEndpoint({
  path: "/heroes",
  method: "POST",
  schemas: {
    request: AddHeroRequestSchema,
    response: AddHeroResponseSchema,
  },
  description: "Add a new hero to the system."
})

You get full request/response validation, path parameter type inference—everything you’d expect for an internal client. But—crucially—those very same schemas and definitions are directly leveraged to generate callable, Zod-validated “tools” for your ReAct agent.

How Zod Parsing Works with the API Client

When using createApiClient:

• Validates the request body before API call.

• Validates the response after API call.

• Validates path parameters.

The same client instance can be:

• Imported into Lambda handlers and backend services.

• Wrapped into agent tools for LLM workflows.

One source of truth. One validation story.

Lambda Handlers: Fortress of Validation (and Serverless Power)

Lambda handlers use the same Zod schemas for validation. initApiHandler wraps your handler to:

• Parse and validate the incoming event body.

• Validate the outgoing response.

Example:

export const handler = initApiHandler({
  apiHandler,
  inputSchema: AddHeroRequestSchema,
  outputSchema: AddHeroResponseSchema,
});

While creating the apiHandler for the addHero endpoint, you get a pre-typed body:

Similarly… the listHeroRescues endpoint’s path parameters are also typed:

When the LLM chooses an action, it’s calling the same code-path as your backend logic. This makes logs, errors, and even side effects far more predictable when debugging.

Deep Dive: The ReAct Agent in Action

1. Auto-Generating Tools from the API Client

Every endpoint becomes a tool validated with Zod:

const generateTools = <T extends Record<string, any>>(client: T) => {
  return Object.keys(client)
    .map((key) => { /* generates validated tools */ })
    .filter((tool) => tool !== null);
};

These tools are directly generated from the same client you use in your backend.

2. Making Tools Available to the Agent

const tools = [...generateTools(heroClient(process.env.API_ID!))];

As soon as a new endpoint is defined, it's available to the agent—no custom adapter required.

3. Customizing the Agent's Prompt

const prompt = (state, config) => {
  const userName = config.configurable?.userName || "Human";
  return [{ role: "system", content: `You are a helpful assistant. Address the user as ${userName}.` }, ...state.messages];
};

4. Running the Agent

const result = await agent.invoke({
  messages: [new HumanMessage(`What can you do?`)],
}, {
  configurable: { userName: "Matt", userId: "9efe72ed-b182-46b1-bc96-f125b7042599" }
});

5. Why This Approach is Super

• Safety: Zod validation prevents malformed requests.

• Flexibility: New endpoints become instantly usable by both humans and agents.

• Observability: Unified logs and error handling across both human and LLM calls.

How the ReAct Agent Knows Exactly How to Format API Calls

Every tool generated from the API client is backed by a Zod schema that defines the expected structure for inputs (path parameters and body) and outputs.

When a ReAct agent is reasoning about which tool to call, it uses the tool's schema to automatically format requests correctly — including nesting, field names, required parameters, and types — without needing any extra code or custom adapters.

Because the tool's input schema is a real Zod object, the agent (through LangChain) can:

• Auto-suggest the correct fields during planning.

• Validate its proposed actions before sending them.

• Receive clear, structured error messages if something goes wrong.

In short: the Zod schema acts like an instruction manual and a safety net at the same time — making sure the agent speaks the API’s language natively.

Example from generateTools:

const combinedSchema = z.object({
  pathParameters: pathParametersSchema || z.object({}),
  body: requestSchema || z.null(),
});

This means every tool knows exactly what shape of data to expect, and the agent never has to guess.

Practical Takeaways

• Share definition and validation logic between humans and agents as a first principle.

• If it isn’t easy for a human to call, it’s probably a minefield for the LLM.

• Avoid translation adapters and parallel validation stacks wherever possible.

• Build one honest client and let both humans and AI use it safely.

Up, Up, and Away!

By combining AWS, CDK, Serverless, Zod, TypeScript, and LangGraph, you get a framework that’s type-safe, runtime-safe, and seamlessly integrated with ReAct agents. Whether you’re building for superheroes or just want to avoid villainous bugs, this approach keeps your API and clients in perfect harmony—for both humans and machines.

Check out the code, try it out, and let me know how you’re using AWS and Zod to save your own day!

The code for this blog post is located at: https://github.com/martzmakes/discord-lambda

The CDK Construct library I used for this project is located at: https://github.com/martzmakes/constructs

The Power of Serverless: Why AWS Lambda Is Your Best Bet

AWS Lambda presents a compelling case for bot creators:

• Cost Efficiency: Pay only for computing time used—no need to maintain costly idle servers.

• Scalability on Demand: Effortlessly manage traffic surges with Lambda’s auto-scaling capabilities.

• Simplicity: Focus on building features while AWS manages the infrastructure.

Unlike an ECS-based bot that could rack up recurring costs due to constant server activity, Lambda charges only during actual code execution, saving you money when demand is low.

Getting Started: Setting Up Your Discord Bot

First, lay the groundwork for your Discord bot:

1. Create a new app in the [Discord Developer Portal](https://discord.com/developers/applications).

2. Build your bot under the Bot section and note down your bot token and application ID.

3. Store these credentials securely in AWS Secrets Manager:

{
  "BOT_TOKEN": "<tokenhere>",
  "APPLICATION_ID": "<application id>",
  "DISCORD_PUBLIC_KEY": "<public key>"
}

4. Add the secretArn to the project’s bin file.

5. Deploy your CDK Project… continue these steps after the project is deployed, you may need to skip ahead.

6. Add your interactions endpoint to your bot’s configuration

   

7. Add the permissions you need

   

8. Use the OAuth2 URL from the Developer Portal to add your bot to the server.

   

Understanding the CDK Structure

Your project begins with setting up the entry point (discord-lambda.ts):

#!/usr/bin/env node
import { App } from 'aws-cdk-lib';
import { DiscordLambdaStack } from '../lib/discord-lambda-stack';

const app = new App();
new DiscordLambdaStack(app, 'DiscordLambdaStack', {
  env: {
    account: process.env.CDK_DEFAULT_ACCOUNT,
    region: process.env.CDK_DEFAULT_REGION,
  },
  envName: 'main',
  eventSource: 'discordLambda',
  discordSecretArn: 'put your discord secret arn here',
  domainName: 'martzmakes.com',
});

This initializes a DiscordLambdaStack for organizing and managing resources, leveraging constructs for seamless AWS integration.

Leveraging @martzmakes/constructs

Explore the potency of these constructs:

• Discord Construct: Simplifies setup involving API Gateway and command registration.

• Lambda Construct: Ensures optimal Lambda configuration with added flexibility.

The Build Process: From Stack to Lambda Setup

Dive deeper into resource configuration in the stack (discord-lambda-stack.ts):

import { MMStackProps } from "@martzmakes/constructs/cdk/interfaces/MMStackProps";
import { MMStack } from "@martzmakes/constructs/cdk/stacks/MMStack";
import { Discord } from "@martzmakes/constructs/cdk/constructs/discord";
import { Lambda } from "@martzmakes/constructs/cdk/constructs/lambda";
import { join } from "path";

export interface DiscordLambdaStackProps extends MMStackProps {
  discordSecretArn: string;
  domainName: string;
}

export class DiscordLambdaStack extends MMStack {
  constructor(scope: Construct, id: string, props: DiscordLambdaStackProps) {
    super(scope, id, props);

    const registerCommandLambda = new Lambda(this, 'register', {
      entry: join(__dirname, `./fns/register-command.ts`)
    });

    const interactionsLambda = new Lambda(this, 'interactions', {
      entry: join(__dirname, `./fns/interactions.ts`)
    });

    new Discord(this, 'Discord', {
      discordSecretArn: props.discordSecretArn,
      domainName: props.domainName,
      interactionsLambda,
      registerCommandLambda,
    });
  }
}

Within this stack, you set up Lambda functions to handle command registrations and Discord interactions. The Discord construct takes care of API Gateway integration and domain configuration.

Mastering Command Registration and Interactions

Command Registration Simplified

Commands are registered using the Discord API through a custom resource utilizing the bot’s credentials securely stored in Secrets Manager:

import { initHandler } from "@martzmakes/constructs/lambda/handlers/initHandler";
import { registerCommands } from "./utils/registerCommands";

const main = async (event: any) => {
  await registerCommands({});
  return {
    PhysicalResourceId: event.PhysicalResourceId,
  }
};

export const handler = initHandler({ handler: main });

Interaction Handling Magic

In the interactions.ts, manage how your bot will respond to commands:

import {
  DiscordInteractionHandler,
  initDiscordInteractionHandler,
} from "@martzmakes/constructs/lambda/handlers/initDiscordInteractionsHandler";
import { roll } from "./utils/roll";

const slashHandler: DiscordInteractionHandler<any, any> = async ({
  body,
}) => {
  switch (body.data.name) {
    case "roll":
      return {
        statusCode: 200,
        data: {
          type: 4,
          data: {
            content: await roll({ input: body.data.options[0].value }),
          },
        },
      };
    default:
      return {
        statusCode: 200,
        data: {
          content: "Unknown command",
        },
      };
  }
};

export const handler = initDiscordInteractionHandler({
  slashHandler,
});

Discord interactions span several categories, but our focus here is on handling application commands.

Example Use Cases

Want your bot to stand out? Here are some ideas:

• Gaming Assistance: Command your bot to manage scores or deliver game statistics.

• Community Manager: Automate server moderation tasks with slash commands. Kick off processes to archive a channel's contents or perform LLM analysis.

• Content Curation: Use advanced AI to summarize key discussion topics and insights from the server.

Dungeons and Dragons Bot Idea

Imagine a bot dedicated to Dungeons and Dragons that can help settle arguments between players (or players and the DM)… an impartial Rule Judge, if you will. I created this using event-driven architecture and langchain to resolve game queries fairly and efficiently. If you’d like to learn more about RuleJudge, let me know.

Limitations

While this approach is powerful, it has its boundaries:

• Limited Interaction: Bots can't be directly DM'd or mentioned with @; command use is via slash commands, buttons, and modals only.

• Latency Possibilities: Cold starts and network delays can affect response times.

It's actually really disappointing that @ mentioning the bot isn't considered an interaction. By contrast, Slack's API they do consider @ mentions as an interaction.

Conclusion

By adopting this serverless approach, you open up a multitude of possibilities for your Discord bot. Imagine seamlessly handling various slash command requests, orchestrating server activities, or even integrating AI features—all while enjoying the benefits of reduced costs and easy scalability that AWS Lambda offers. Your bot could become a versatile tool for communities, provide insightful server analytics, or even automate routine tasks for server admins. The combination of serverless architecture and Discord's interaction model empowers you to build a bot that is not only functional but also transformative in its capabilities. Now, it's time for you to transform your Discord ideas into reality. With this guide, you have the blueprint to create a sophisticated, serverless bot ready to meet and exceed the needs of any community. Start experimenting, share your creations, and see how your bot can make a difference in the Discord ecosystem. Let's see what you build!

In this post, we’ll explore how to set up AWS Cognito’s managed login pages using AWS CDK to host a lambda-rendered site. We’ll also dive into cookie-based authorization using a Lambda Authorizer, allowing you to protect specific routes without exposing any backend code. If you’re looking for a hands-on example, you can check out the code here: https://github.com/martzmakes/cognito-hosted.

Hosting the Website with Lambda Rendering and API Proxying

One unique part of my setup is that the website is Lambda-rendered, with all non-/api paths proxied. This means that all API requests happen on the same domain name via the /api resource, allowing the front-end to make relative calls to the backend without needing to define different API domains. This simplifies deployment and makes for easy blog examples that don’t require any knowledge of frontend frameworks.

The non /api routes on the apigateway trigger lambda(s) that return vanilla HTML with everything needed to render the site.

I am NOT recommending you do this in practice… but for frameworks like Next.js that support server-side rendering with lambda… you might be able to make use of this. The further ability of being able to protect certain paths with the cookie-validating lambda could add additional protection to your site.

While I wouldn’t recommend this for large-scale production apps, it’s a great way to prototype serverless apps without needing a separate frontend stack. Plus, if you’re working with a framework like Next.js, Lambda rendering can help you keep everything under the same API Gateway domain for a more seamless deployment.

Why Use Cognito Managed Login Pages?

AWS Cognito offers a fully managed user authentication and authorization service. By leveraging its hosted UI, you get:

• Ease of Setup: No need to build your own login pages or OAuth flows.

• Security: AWS handles security best practices out of the box.

• Customization: Tailor the look and feel to match your application’s branding.

Initial Setup with CDK

When I was first creating the code for this blog post, I was using the regular CognitoUserPoolsAuthorizer construct. This construct expects a header with the Bearer id token that you get back from Cognito. By default it goes in the Authorization header, but you can change it with properties if you want. I started out with this code which is largely pulled from the documentation:

const userPool = new UserPool(this, "UserPool");
const domainPrefix = "martzmakes-example";
const domain = userPool.addDomain("CognitoDomainWithBlandingDesignManagedLogin",
  {
    cognitoDomain: { domainPrefix },
    managedLoginVersion: ManagedLoginVersion.NEWER_MANAGED_LOGIN,
  }
);

const homeUrl = `https://${clientBaseDomain}/home`;
const client = new UserPoolClient(this, "Client", {
  userPool,
  oAuth: {
    flows: {
      implicitCodeGrant: true,
    },
    callbackUrls: [`https://${clientBaseDomain}`, homeUrl],
  },
});

domain.signInUrl(client, { redirectUri: homeUrl });

I create the userPool, add a cognitoDomain to it with ManagedLoginVersion.NEWER_MANAGED_LOGIN (which is the newer managed login page). cognitoDomain means that the domain will be hosted on an AWS URL and not your own. Then we create the UserPoolClient that the domain attaches to.

Out-of-the-box this seems like it should work… but it DOES NOT. If you deploy this as-is… when you open the login page you’ll get a non-descript error along the lines of there being a system error and to contact the administrator… which is rich since you ARE the admin. 🤦‍♂️

At first, I thought AWS was just messing with me. Everything should have been working. But after some furious clicking through the console, I stumbled upon the ‘Styles’ section of the App Client. Turns out, AWS simply refuses to render the login pages unless you define a branding resource—because why not add one more undocumented requirement? 😅

I manually created a style (just to see if this worked) and that was indeed what was missing. After troubleshooting, I found that the issue stemmed from the absence of CfnManagedLoginBranding. This resource is essential because it defines the branding for Cognito’s managed login UI—without it, AWS simply refuses to render the login pages. While my initial setup wasn’t based on Sebastian Sturm's post, his blog helped me identify this missing component, which ultimately resolved my issue.

// all I was missing was this...
new CfnManagedLoginBranding(this, "ManagedLoginBranding", {
  userPoolId: userPool.userPoolId,
  clientId: client.userPoolClientId,
  returnMergedResources: true,
  useCognitoProvidedValues: true,
});

With the default styles in place I have a pretty nice looking login UI:

Next, we can create the authorizer and attach it to a RestApi:

const restApi = new RestApi(this, `Api`, {
  defaultCorsPreflightOptions: {
    allowOrigins: Cors.ALL_ORIGINS,
  },
  endpointConfiguration: {
    types: [EndpointType.REGIONAL],
  },
});
restApi.addDomainName("domain", {
  domainName: clientBaseDomain,
  certificate,
});

new ARecord(this, "ARecord", { zone: hostedZone, recordName: clientBaseDomain, target: RecordTarget.fromAlias(new ApiGateway(restApi)) });

const apiFn = new NodejsFunction(this, "api", {
  entry: join(__dirname, "fns/api.ts"),
  runtime: Runtime.NODEJS_LATEST,
  logGroup: new LogGroup(this, `/${id}ApiLogs`, { logGroupName: `/${id}-api`, removalPolicy: RemovalPolicy.DESTROY }),
  architecture: Architecture.ARM_64,
});

const authorizer = new CognitoUserPoolsAuthorizer(this, `${id}UserPoolAuthorizer`, { cognitoUserPools: [userPool], });

const apiResource = restApi.root.addResource("api");
apiResource.addProxy({
  anyMethod: true,
  defaultIntegration: new LambdaIntegration(apiFn, { proxy: true }),
  defaultMethodOptions: {
    authorizer,
    authorizationType: AuthorizationType.COGNITO,
  },
});

const fn = new NodejsFunction(this, "site", {
  entry: join(__dirname, "fns/site.ts"),
  runtime: Runtime.NODEJS_LATEST,
  logGroup: new LogGroup(this, `/${id}SiteLogs`, { logGroupName: `/${id}-site`, removalPolicy: RemovalPolicy.DESTROY }),
  architecture: Architecture.ARM_64,
  environment: {
    AUTH_PREFIX: domainPrefix,
    BASE_DOMAIN: clientBaseDomain,
    USER_POOL_CLIENT_ID: client.userPoolClientId,
    USER_POOL_ID: userPool.userPoolId,
  }
});

const siteProxy = restApi.root.addProxy({
  anyMethod: false, // Disables automatic handling of all methods
});
siteProxy.addMethod("GET", new LambdaIntegration(fn));

Key parts in the above code are that initially I used the CognitoUserPoolsAuthorizer and I create a proxy on the api resource that uses it. That means that all requests going to /api on the RestApi require that Authorization header with the cognito id token. Finally, we create the lambda-rendered site proxy which handles non-/api paths. That means requests to / and /home for example, will invoke that site lamdba. Later in this blog post we’ll carve out a /protected path which requires authentication in order to display.

💡 Why the id token and not the access token? The Cognito User Pool Authorizer in API Gateway expects the ID token instead of the access token because it is designed primarily for authenticating users, NOT authorizing API access. You can get more fine grained with authorization by using a lambda authorizer.

Lambda-based Site Rendering

My site lambda in this case is pretty simple. It always returns HTML in the response. The HTML includes hard-coded javascript (via script tags) that check to see if a cookie has been set. If it hasn’t it redirects to the Managed Auth page. When a user successfully signs in via the Managed Auth page… it redirects to the /home path and includes the Cognito id token in query string parameters. We take those, and store them in the cookie for use with the fetch request. It sounds like a lot but it’s fairly simple javascript.

export const handler = async () => {
  return {
    statusCode: 200,
    headers: {
      "Content-Type": "text/html", // required for proper browser rendering
      "Access-Control-Allow-Origin": "*", // Required for CORS support to work
    },
    body: `<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Lambda Page</title>
    <!-- styles removed for brevity -->
    <script type="module">
        function setCookie(name, value, days) {
            let expires = "";
            if (days) {
                const date = new Date();
                date.setTime(date.getTime() + (days * 24 * 60 * 60 * 1000));
                expires = "; expires=" + date.toUTCString();
            }
            document.cookie = name + "=" + value + "; path=/" + expires;
        }

        function getCookie(name) {
            const match = document.cookie.match(new RegExp('(^| )' + name + '=([^;]+)'));
            return match ? match[2] : null;
        }

        function parseHashParams() {
            const hash = window.location.hash.substring(1);
            const params = new URLSearchParams(hash);
            const idToken = params.get("id_token");
            const expiresIn = params.get("expires_in");

            if (idToken) {
                setCookie("CognitoIdToken", idToken, expiresIn / 86400);
                window.location.hash = "";
            }
        }

        function updateUI() {
            const idToken = getCookie("CognitoIdToken");
            document.getElementById("signIn").classList.toggle("hidden", !!idToken);
            document.getElementById("signOut").classList.toggle("hidden", !idToken);
        }

        async function fetchAPIData() {
            const idToken = getCookie("CognitoIdToken");
            if (!idToken) return;
            try {
                const response = await fetch('/api/hello', {
                    headers: { 'Authorization': \`Bearer \${idToken}\` }
                });
                const data = await response.json();
                document.getElementById("api-response").textContent = JSON.stringify(data, null, 2);
            } catch (error) {
                document.getElementById("api-response").textContent = "Error fetching API.";
            }
        }

        window.onload = () => {
            parseHashParams();
            updateUI();
            fetchAPIData();
        }

        window.signIn = function () {
            window.location.href = "https://${process.env.AUTH_PREFIX}.auth.us-east-1.amazoncognito.com/login?client_id=${process.env.USER_POOL_CLIENT_ID}&response_type=token&scope=aws.cognito.signin.user.admin+email+openid+phone+profile&redirect_uri=https%3A%2F%2F${process.env.BASE_DOMAIN}%2Fhome";
        }

        window.signOut = function () {
            document.cookie = "CognitoIdToken=; expires=Thu, 01 Jan 1970 00:00:00 UTC; path=/;";
            updateUI();
            window.location.href = "https://${process.env.AUTH_PREFIX}.auth.us-east-1.amazoncognito.com/logout?client_id=${process.env.USER_POOL_CLIENT_ID}&response_type=token&scope=aws.cognito.signin.user.admin+email+openid+phone+profile&redirect_uri=https%3A%2F%2F${process.env.BASE_DOMAIN}";
        }
    </script>
</head>
<body>
    <div class="container">
        <h1>Welcome to a Lambda-Powered Page</h1>
        <button id="signIn" class="button sign-in" onclick="signIn()">Sign In</button>
        <button id="signOut" class="button sign-out hidden" onclick="signOut()">Sign Out</button>
        <h2>API Response</h2>
        <pre id="api-response">No data yet</pre>
    </div>
</body>
</html>`,
  };
};

Note: In fetchAPIData we retrieve the cookie from the document and include that as the Authorization header in the request.

Moving to Lambda-based Authorization

While the Cognito User Pool Authorizer works well for API authentication, it relies on ID tokens in headers—which doesn’t help when protecting UI routes. Browsers don’t automatically send Authorization headers in GET requests, but they do send cookies. That’s why switching to a Lambda Authorizer with cookie-based authentication makes for a much smoother experience.

By switching to a Lambda-based Authorizer with cookie-based authentication, I could:

• Protect sub-routes across multiple Lambda-rendered pages.

• Use the same cookies for seamless user experience.

• Avoid exposing any backend code directly.

Here’s an example of how the Lambda Authorizer is set up:

const authFn = new NodejsFunction(this, "auth", {
  entry: join(__dirname, "fns/auth.ts"),
  runtime: Runtime.NODEJS_LATEST,
  logGroup: new LogGroup(this, `/${id}authLogs`, { logGroupName: `/${id}-auth`, removalPolicy: RemovalPolicy.DESTROY }),
  architecture: Architecture.ARM_64,
  environment: {
    AUTH_PREFIX: domainPrefix,
    BASE_DOMAIN: clientBaseDomain,
    USER_POOL_CLIENT_ID: client.userPoolClientId,
    USER_POOL_ID: userPool.userPoolId,
  },
});
const authorizerFn = new RequestAuthorizer(this, "Authorizer", {
  handler: authFn,
  identitySources: ["method.request.header.Cookie"],
});

The Lambda function verifies the cookie and extracts the user's identity, allowing for seamless authentication without exposing tokens.

The JWT validation is similar to what I did in my old blog post on https://martzmakes.com/creating-verifiable-json-web-tokens-jwts-with-aws-cdk … If you’re more of a visual person, I gave a talk on this at CDK Day 2021 here (yeah… I know it’s 2025… but it holds up). I’m not going to go into the specifics, but the code for the authorizer function is here: https://github.com/martzmakes/cognito-hosted/blob/main/lib/fns/auth.ts

Since we’re already storing the id token as a cookie we can modify the fetchAPIData function in the site’s script to not include the header:

async function fetchAPIData() {
    try {
        const response = await fetch('/api/hello');
        const data = await response.json();
        document.getElementById("api-response").textContent = JSON.stringify(data, null, 2);
    } catch (error) {
        document.getElementById("api-response").textContent = "Error fetching API.";
    }
}

Furthermore we can add a /protected route on the API gateway for a lambda-rendered page that requires a user to be logged in.

const protectedFn = new NodejsFunction(this, "protected", {
  entry: join(__dirname, "fns/protected.ts"),
  runtime: Runtime.NODEJS_LATEST,
  logGroup: new LogGroup(this, `/${id}ProtectedLogs`, { logGroupName: `/${id}-protected`, removalPolicy: RemovalPolicy.DESTROY }),
  architecture: Architecture.ARM_64,
  environment: {
    AUTH_PREFIX: domainPrefix,
    BASE_DOMAIN: clientBaseDomain,
    USER_POOL_CLIENT_ID: client.userPoolClientId,
    USER_POOL_ID: userPool.userPoolId,
  },
});
restApi.root
  .addResource("protected")
  .addMethod("GET", new LambdaIntegration(protectedFn), {
    authorizer: authorizerFn,
  });

Demo

With all of that in place, we can open up the site and see that the API Response is “Unauthorized” because we haven’t logged in.

If we go to the PROTECTED link, we see that we get an Unauthorized response.

After signing in, we get redirected to the home route where the Cookie is stored, and the API request returns successful:

And finally… if we click the PROTECTED link, we see the users only page because that lambda-rendered route is protected with the cookie authorizer:

Wrapping It Up

AWS Cognito’s managed login pages offer a powerful, no-fuss way to authenticate users while offloading the complexity of OAuth flows and security best practices to AWS. But by combining Cognito with Lambda-based site rendering and a custom Lambda Authorizer, you get fine-grained control over authentication, seamless cookie-based authorization, and a flexible way to protect both API and UI routes—without exposing backend logic.

This setup isn’t just a cool experiment; it’s a practical approach for serverless applications that need secure, scalable authentication without introducing unnecessary client-side complexity. Whether you’re looking to lock down API routes, protect UI pages, or simply avoid dealing with OAuth headaches, this method offers a clean, effective solution.

Want to dive deeper? Grab the full implementation and try it yourself on GitHub: martzmakes/cognito-hosted. Got questions or ideas? Let’s chat in the comments or on LinkedIn! 🚀

In this post, I’ll show you how I built a serverless solution using AWS Lambda, Puppeteer, and CDK to do just that. The solution consists of two Lambda functions working in tandem to render PNG images and serve them up via pre-signed S3 URLs. Along the way, we’ll talk about the infrastructure, the code, and some of the lessons learned.

The code for this project is available on GitHub.

The Problem and The Solution

Let’s set the stage. Imagine you’re working on a project that needs to dynamically render diagrams—whether from Mermaid markdown or Draw.io XML. Once rendered, these diagrams must be stored securely as PNG images in S3 for easy sharing and integration, with access provided via pre-signed URLs.

Sounds simple enough, but if you’ve ever tried rendering diagrams programmatically, you know it’s a bit like herding cats—except these cats are diagrams, and they bite. That’s where the power of AWS Lambda, Puppeteer, and the AWS CDK comes in. Together, they tame the chaos and streamline the process.

Here’s how the solution works:

1. Mermaid Renderer Lambda: Accepts Mermaid markdown as input, uses Puppeteer to generate the PNG, and uploads the result to S3.

2. Draw.io Renderer Lambda: Does the same, but for Draw.io XML files.

3. S3 Bucket with Pre-Signed URLs: A secure storage and sharing mechanism for the rendered diagrams.

4. Infrastructure with CDK: Custom constructs tie everything together in a clean and reusable way.

Why Puppeteer?

Puppeteer, a Node.js library, provides a high-level API to control Chromium or Chrome browsers, making it perfect for rendering web-based content like Mermaid and Draw.io diagrams. However, running Puppeteer in Lambda isn’t plug-and-play. Lambda’s execution environment requires headless browser support, which can be tricky to set up.

To solve this, I used @sparticuz/chromium-min, a prebuilt Chromium layer optimized for AWS Lambda. It ensures that Puppeteer runs seamlessly within Lambda's constraints, handling the rendering process efficiently and reliably.

Infrastructure: Laying the Groundwork with CDK and Custom Constructs

To power our diagram-rendering solution, we need a robust and flexible infrastructure. Enter AWS CDK, with a sprinkle of customization courtesy of my @martzmakes/constructs library—a set of open-source constructs designed to simplify and enhance AWS projects. Let’s break down the infrastructure code for this project.

Buckets, Lambdas, and Puppeteer, Oh My!

The heart of the infrastructure is the LambdaDiagramStack class. It uses CDK constructs and custom opinions from @martzmakes/constructs to define two Lambda functions (Mermaid and Draw.io) for the rendering work.

Here’s the full code for the stack:

export class LambdaDiagramStack extends MMStack {
  constructor(scope: Construct, id: string, props: MMStackProps) {
    super(scope, id, props);
    const diagramBucket = new Bucket(this, "DiagramBucket", {
      blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
      removalPolicy: RemovalPolicy.DESTROY,
      autoDeleteObjects: true,
      eventBridgeEnabled: true,
      objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED,
      lifecycleRules: [
        {
          expiration: Duration.days(1),
          enabled: true,
        },
      ],
    });

    new Lambda(this, "MermaidLambda", {
      entry: join(__dirname, `./fns/mermaid.ts`),
      eventPattern: {
        source: [this.eventSource],
        detailType: ["mermaid"],
      },
      name: "mermaid",
      architecture: Architecture.X86_64, // puppeteer needs x86_64
      bundling: {
        externalModules: [],
      },
      memorySize: 10240,
      buckets: {
        BUCKET_NAME: { bucket: diagramBucket, access: "rw" },
      },
    });

    new Lambda(this, "DrawIOLambda", {
      entry: join(__dirname, `./fns/drawio.ts`),
      eventPattern: {
        source: [this.eventSource],
        detailType: ["drawio"],
      },
      bundling: {
        externalModules: [],
      },
      name: "drawio",
      architecture: Architecture.X86_64, // puppeteer needs x86_64
      memorySize: 10240,
      buckets: {
        BUCKET_NAME: { bucket: diagramBucket, access: "rw" },
      },
    });
  }
}

Key Components

1. The S3 Bucket:

   • Block Public Access: Ensures the bucket is private and secure.

   • Lifecycle Rules: Automatically cleans up old files after one day, keeping things tidy.

   • Ownership Enforced: Simplifies managing access policies.

2. Custom Lambda Constructs:

   • MermaidLambda and DrawIOLambda are created using the Lambda construct from @martzmakes/constructs.

   • Both Lambdas are configured with:

     • X86_64 architecture: Puppeteer requires this architecture to work correctly.

     • Generous memory allocation: 10 GB ensures Puppeteer runs smoothly.

     • Bucket access: Enables each Lambda to read from and write to the S3 bucket.

3. CDK Opinions:

   • The @martzmakes/constructs library simplifies common patterns while enabling observability. For instance, the Lambda construct extends CDK’s NodejsFunction to handle bundling, memory configuration, and event patterns.

Puppeteer and the X86_64 Architecture

A critical note for Puppeteer users: it won’t run on ARM-based architectures like Graviton. By explicitly setting the Lambdas to use Architecture.X86_64, we sidestep potential compatibility issues. While this may sacrifice some cost or performance benefits of ARM, it ensures Puppeteer operates reliably.

Lambda Functions: Rendering Diagrams with Puppeteer and S3 Integration

Now that we’ve set up the infrastructure, let’s dive into how the Lambda functions actually work. Both the Mermaid and Draw.io rendering Lambdas follow a similar pattern, leveraging Puppeteer for rendering and S3 for storing the output. They’re wrapped with helper methods from @martzmakes/constructs, ensuring observability and seamless integration.

Common Patterns in Both Lambdas

1. Initialization with initEventHandler: Each Lambda uses initEventHandler from @martzmakes/constructs to:

   • Set up AWS X-Ray tracing for better observability using Lambda Powertools.

   • Emit an architecture event to infer system architecture for future observability enhancements.

2. Chromium Setup with @sparticuz/chromium-min: The @sparticuz/chromium-min library provides a prebuilt Chromium binary tailored for AWS Lambda. It’s essential for rendering web-based diagrams within Puppeteer.

3. S3 Upload with Pre-Signed URL: The rendered diagrams are uploaded to S3 using uploadImageToS3AndGetPresignedUrl from @martzmakes/constructs. This helper function simplifies the process of generating a pre-signed URL after uploading an image.

Mermaid Lambda

Here’s how the Mermaid Lambda works:

• Input: Takes a title and a mermaidCode (the diagram definition in Mermaid markdown).

• Process:

  1. Generates an HTML template embedding the Mermaid diagram.

  2. Launches Puppeteer using the prebuilt Chromium binary.

  3. Sets the HTML content and waits for the diagram to render.

  4. Captures the SVG element as a PNG image buffer.

• Output: Uploads the buffer to S3 and returns a pre-signed URL.

Here’s the core implementation:

const mermaidToImageBuffer = async ({
  title,
  mermaidCode,
}: {
  title: string;
  mermaidCode: string;
}): Promise<Buffer> => {
  // HTML template to render the Mermaid diagram
  const htmlTemplate = `
      <!DOCTYPE html>
      <html lang="en">
      <head>
          <meta charset="UTF-8">
          <title>${title}</title>
          <script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
          <style>
              body {
                  margin: 0;
                  display: flex;
                  justify-content: center;
                  align-items: center;
                  height: 100vh;
                  background: white;
              }
          </style>
      </head>
      <body>
          <div id="mermaidContainer">
              <div class="mermaid">${mermaidCode}</div>
          </div>
          <script>
              mermaid.initialize({ startOnLoad: true });
          </script>
      </body>
      </html>
  `;

  // Launch Puppeteer
  chromium.setHeadlessMode = true;
  chromium.setGraphicsMode = true;
  const browser = await puppeteer.launch({
    args: [
      ...chromium.args,
    ],
    defaultViewport: {
      width: 1920,
      height: 1080,
      deviceScaleFactor: 3,
    },
    executablePath: await chromium.executablePath(
      "https://github.com/Sparticuz/chromium/releases/download/v119.0.2/chromium-v119.0.2-pack.tar"
    ),
    headless: false,
  });
  const page = await browser.newPage();

  // Set content
  await page.setContent(htmlTemplate, { waitUntil: "networkidle0" });

  // Wait for the diagram to render
  await page.waitForSelector(".mermaid > svg");

  // Select the SVG element and capture as an image
  const element = await page.$(".mermaid > svg");
  if (!element) {
    throw new Error("Failed to render Mermaid diagram");
  }

  // Scale bounding box dimensions by the deviceScaleFactor
  const buffer = await element.screenshot({
    type: "png",
  });

  await browser.close();
  return Buffer.from(buffer);
};

const eventHandler: EventHandler<{
  title: string;
  mermaidCode: string;
}> = async ({ data }) => {
  const { title, mermaidCode } = data;
  const buffer = await mermaidToImageBuffer({
    title,
    mermaidCode,
  });

  const key = `${Date.now()}-${title}.png`;
  const presignedUrl = await uploadImageToS3AndGetPresignedUrl({ key, buffer });
  console.log(JSON.stringify({ presignedUrl }));
};

export const handler = initEventHandler({ eventHandler });

Draw.io Lambda

The Draw.io Lambda follows a similar flow but uses a Draw.io-specific rendering process:

• Input: Takes a title and a drawioXml (the diagram definition in Draw.io XML).

• Process:

  1. Navigates Puppeteer to the Draw.io export URL.

  2. Uses page scripts to render the Draw.io XML into an SVG.

  3. Captures the SVG as a PNG image buffer.

• Output: Same as Mermaid Lambda—uploads to S3 and generates a pre-signed URL.

Here’s the core implementation:

const drawioToImageBuffer = async ({
  drawioXml,
}: {
  drawioXml: string;
}): Promise<Buffer> => {
  // Launch Puppeteer
  chromium.setHeadlessMode = true;
  chromium.setGraphicsMode = true;
  const width = 2 * 1920;
  const height = 2 * 1080;
  const browser = await puppeteer.launch({
    args: [...chromium.args],
    defaultViewport: {
      width,
      height,
      deviceScaleFactor: 3,
    },
    executablePath: await chromium.executablePath(
      "https://github.com/Sparticuz/chromium/releases/download/v119.0.2/chromium-v119.0.2-pack.tar"
    ),
    headless: false,
  });
  const page = await browser.newPage();
  await page.goto("https://www.draw.io/export3.html", {
    waitUntil: "networkidle0",
  });

  await page.evaluate(
    (obj) => {
      return (window as any).render({
        h: obj.height,
        w: obj.width,
        xml: obj.drawioXml,
      });
    },
    { drawioXml, width, height }
  );
  await page.waitForSelector("#LoadingComplete");
  console.log("Loading complete");

  // Select the SVG element and capture as an image
  const element = await page.$("#graph > svg");
  if (!element) {
    throw new Error("Failed to render Mermaid diagram");
  }

  // Scale bounding box dimensions by the deviceScaleFactor
  const buffer = await element.screenshot({
    type: "png",
    fullPage: true,
  });

  await browser.close();
  return Buffer.from(buffer);
};

const eventHandler: EventHandler<{
  title: string;
  drawioXml: string;
}> = async ({ data }) => {
  const { title, drawioXml } = data;
  const buffer = await drawioToImageBuffer({
    drawioXml,
  });

  const key = `${Date.now()}-${title}.png`;
  const presignedUrl = await uploadImageToS3AndGetPresignedUrl({ key, buffer });
  console.log(JSON.stringify({ presignedUrl }));
};

export const handler = initEventHandler({ eventHandler });

Observability and Future Enhancements

Both Lambdas emit architecture events for observability—a feature I’ll explore in a future post. This ensures we can visualize architecture changes and interactions using the rendered diagrams. For now, the event data helps lay the groundwork for a deeper understanding of your system.

Putting It All Together: Seamless Serverless Diagram Rendering

Here’s how the system operates end-to-end:

1. Trigger:

   • A client sends an event to the EventBridge source configured for either the Mermaid or Draw.io Lambda.

   • The event payload includes the diagram’s title and its respective code or XML.

2. Processing:

   • The appropriate Lambda function is triggered based on the event type (mermaid or drawio).

   • The Lambda:

     • Processes the input using Puppeteer to render the diagram.

     • Uploads the generated PNG to the S3 bucket.

     • Retrieves a pre-signed URL for accessing the image.

3. Response:

   • The Lambda returns the pre-signed URL to the client, making the image available for secure, time-bound access.

4. Lifecycle Management:

   • A lifecycle policy on the S3 bucket ensures that old diagrams are automatically deleted after one day, maintaining a clean and cost-effective storage environment.

By leveraging @martzmakes/constructs, the system can provide insights:

• Tracing with Lambda Powertools:

  • End-to-end tracing via AWS X-Ray gives a detailed view of the rendering process.

  • Helps identify bottlenecks, such as rendering delays or S3 upload issues.

• Architecture Events:

  • Emitted by the initEventHandler, these events are invaluable for mapping how diagrams relate to system architecture.

  • In future posts, we’ll use this data to visualize system interactions and changes.

What’s Next?

In future posts, I’ll show you how to use this solution for:

• Visualizing advanced multi-node LLM agents and their decisions using LangGraph and LangChain with Mermaid.

• Observing architectural changes in your projects with Draw.io.

• Generating visuals of upstream/downstream error propagation and sending them to Slack.

Stay tuned for those deep dives, but in the meantime, feel free to tinker with the code, experiment with new diagramming formats, or even extend the solution for other use cases.

Serverless rendering of diagrams might not save the world, but it can definitely save your sanity when managing complex systems. By combining the power of AWS Lambda, Puppeteer, and S3, we’ve created a flexible and scalable solution for rendering and sharing diagrams.

What’s the most creative way you can think of to use this? Let me know in the comments!

Enter the Backend for Frontend (BFF) pattern—the unsung hero in this story. In today’s post, we’ll look at why BFFs work so well in tandem with Domain-Driven Design (DDD), and why they should be an integral part of your application architecture. For a little bonus, we’ll sprinkle in some practical examples with AWS CDK (including a neat workaround for attaching usage plans and API keys via CDK Aspects).

Why BFF and DDD Form a Winning Partnership

As your system grows—more domains, more APIs, more everything—the complexities on the frontend will increase accordingly. Domain services might be neatly separated at the backend, but the burden of aggregating and juggling their data will inevitably fall on the frontend developers. It's a bit unfair—and terribly inefficient. That's where BFFs come in, acting as a dedicated middle layer, transforming backend complexity into frontend simplicity.

Let's break it down.

1. Streamlining Frontend Requests

The Problem Without BFF: When your frontend directly interacts with various domain services, you quickly end up with a mess of HTTP calls, each dealing with its own service. Imagine the complexity of interacting with a dozen APIs from a simple React app—the requests, the retry logic, the sequence of events.

With BFF: The frontend doesn’t need to know or care about the multitude of services lurking in the depths of your backend kingdom. The BFF abstracts all that, providing a single API that interacts with the necessary services and returns a nicely structured response.

Here’s a simple example of how BFF comes to the rescue using AWS API Gateway and AWS Lambda. Your frontend can gracefully call a single endpoint to retrieve detailed information about, say, a user or a product. The BFF takes on the dirty work—fetching data from multiple services like Users, Inventory, and Recommendations.

2. Keeping Domain Responsibilities Where They Belong

The Problem Without BFF: Let’s say you’re building the frontend for an e-commerce platform. You need product details, which means calling the Pricing, Inventory, and Reviews domain services, among others.

What happens? Your frontend is now responsible for making all these calls, sequencing them appropriately, aggregating the results, and dealing with all sorts of edge cases—timeouts, failed calls, out-of-sync responses. Complexity creeps in fast, and your frontend is now shouldering work it was never meant to handle.

With BFF: The BFF can take responsibility for orchestrating these calls. It doesn't merely proxy the requests—it performs the necessary API calls to gather the data, handles errors when they arise, and packages everything as an elegant response for the frontend.

A classic BFF endpoint would look something like /product/{productId}/details. The API call behind the scenes automatically handles pricing, availability, reviews, and product images. The frontend, blissfully unaware of the chaos happening below the surface, receives a neatly compiled JSON response.

3. Performance and Security Benefits

Without BFF: More direct API calls from the frontend mean more round-trips, more latency, and the potential for increased attack surfaces. You’re also pushing API key management onto the frontend, and that’s just bad practice.

With BFF: The BFF optimizes how backend calls are handled. It can batch and parallelize requests, reducing latency by minimizing unnecessary round-trips. By serving as a single point of contact for all frontend calls, the BFF reduces the risk of exposing sensitive backend services directly to the public.

And yes, in AWS, you can manage API Gateway’s keys and usage plans from the BFF side, ensuring that only appropriate calls are routed, secured, and handled efficiently.

Putting It All Together: A Practical Example with AWS CDK

Building a BFF can seem daunting, but thankfully the AWS Cloud Development Kit (CDK) comes to the rescue. Let’s walk through how you can deploy a BFF using API Gateway, Lambda, and some CloudFormation magic.

Setting Up The Basics

You’ll need the usual suspects:

• API Gateway to expose your BFF.

• AWS Lambda to process the backend logic and communicate with other domain services.

• API Keys and Usage Plans, because we do care about rate-limiting and access control.

Here’s where things get a bit tricky—out of the box, CDK’s API Gateway constructs don’t play nicely when trying to link specific APIs to usage plans when the APIs are coming from other projects. Thankfully, you can use CDK Aspects to work around this.

class AddApisToUsagePlanAspect implements IAspect {
  constructor(private readonly externalApiIds: string[]) {}

  public visit(node: IConstruct): void {
    if (node instanceof UsagePlan) {
      const usagePlan = node as UsagePlan;
      const cfnUsagePlan = usagePlan.node.defaultChild as CfnUsagePlan;
      cfnUsagePlan.apiStages = this.externalApiIds.map(apiId => ({
        apiId, stage: "prod",
      })) as any;
    }
  }
}

Let’s break this down so it doesn't feel like we’re just throwing jargon your way and calling it a day:

1. Aspect Class: We created an Aspect, which is just a bit of code that inspects each construct in your CDK stack. In our case, we looked for the UsagePlan constructs.

2. CloudFormation Hackery: We dive into the underlying CloudFormation representation of the usage plan (i.e., CfnUsagePlan), pull out the existing apiStages (if any), and append our shiny new list of external API IDs to this array. This way, AWS knows that these external APIs should be attached and governed by the same usage plan.

3. Attach External APIs: CDK didn’t natively allow us to link multiple external APIs under a single usage plan the way we needed. Thanks to CDK Aspects, we were able to surgically inject the correct API IDs into the API Gateway definition, ensuring all external API calls route through the correct usage plan and API key management without manually editing the generated CloudFormation.

Want to Go Deeper into CDK Aspects?

If this introduced you to the wondrous world of CDK Aspects but left you wanting more, you're in luck! I go waaaaay deeper into the theory and practical use cases of Aspects in my other blog post, Breaking Bad Practices with CDK Aspects Whether you're struggling with modifying CDK constructs mid-flight, dealing with compliance checks, or just looking for creative ways to improve your CDK game, that post has got you covered. Go peek at it! I promise you’ll walk away inspired.

With the CDK Aspect out of the way…

Here’s a simplified example of what your BFF Stack might look like:

export class BlogBffStack extends Stack {
  constructor(scope: Construct, id: string, props: BlogBffStackProps) {
    super(scope, id, props);

    const apiKey = new ApiKey(this, 'ApiKey', { apiKeyName: 'BFFApiKey' });

    const api = new RestApi(this, 'BFFApi', {
      description: 'Backend for Frontend API',
      endpointConfiguration: { types: [EndpointType.REGIONAL] },
    });

    const usagePlan = new UsagePlan(this, 'UsagePlan');
    usagePlan.addApiKey(apiKey);

    Aspects.of(this).add(new AddApisToUsagePlanAspect([api.restApiId, ...Object.values(props.externalApis)]));

    Object.entries(props.externalApis).forEach(([apiName, apiId]) => {
      const integration = new HttpIntegration(`https://${apiId}.execute-api.us-east-1.amazonaws.com/prod/{proxy}`);
      api.root.addResource(apiName.toLowerCase()).addProxy({ anyMethod: true, defaultIntegration: integration });
    });
  }
}

This will give you a properly configured BFF API, safely hidden behind an API Gateway, and all access controlled via API keys. The frontend, in turn, gets a consistent and reliable interface without ever knowing what’s happening behind the scenes.

Advanced Orchestration in the BFF

Note: My GitHub repo doesn't actually cover this portion directly, but this is how a BFF endpoint could work in practice to merge multiple sources of information on the backend.

Your BFF doesn’t need to merely pass requests through—it can take on roles like service aggregation and orchestration. Here’s a sample Lambda function that aggregates product data from various domain services:

import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';

export const handler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
  const productId = event.pathParameters?.productId;
  const externalApis = JSON.parse(process.env.EXTERNAL_APIS!);

  try {
    const [priceResp, inventoryResp, reviewsResp] = await Promise.all([
      fetch(`https://${externalApis["Pricing"]}.execute-api.us-east-1.amazonaws.com/prod/price/${productId}`),
      fetch(`https://${externalApis["Inventory"]}.execute-api.us-east-1.amazonaws.com/prod/inventory/${productId}`),
      fetch(`https://${externalApis["Reviews"]}.execute-api.us-east-1.amazonaws.com/prod/reviews/${productId}`)
    ]);

    const [priceData, inventoryData, reviewsData] = await Promise.all([
      priceResp.json(),
      inventoryResp.json(),
      reviewsResp.json()
    ]);

    const responseData = {
      productId,
      pricing: priceData,
      inventory: inventoryData,
      reviews: reviewsData,
    };

    return {
      statusCode: 200,
      body: JSON.stringify(responseData),
    };
  } catch (error) {
    return {
      statusCode: 500,
      body: JSON.stringify({ message: 'Failed to retrieve product details.' }),
    };
  }
};

This Lambda function executes parallel requests to Pricing, Inventory, and Review services, compiles the data, and returns it as a single response, reducing latency and simplifying frontend calls.

Full Code Available on GitHub 💾

All code featured in this article—including the BFF stack, the proxy configuration, the AddApisToUsagePlanAspect, and Lambda functions—is available at this GitHub repository:

https://github.com/martzcodes/blog-bff

So feel free to clone it, test it, and even poke around to see how it all works together in glorious AWS harmony. 🎶

Wrapping Up: Why You Need a BFF

Using a BFF in conjunction with Domain-Driven Design isn't just a nice-to-have; it’s essential. You not only reduce the burden on your frontend but also streamline backend complexity into manageable, secure, and high-performing APIs.

Here’s what you gain:

1. Reduced complexity for your frontend with a single call to a BFF that handles all orchestration.

2. Optimized performance via parallel tasks, batching, and reduced round-trips.

3. Enhanced security by hiding your backend services behind one simplified API surface controlled through API keys.

And with powerful tools like AWS CDK, building and deploying a BFF is no longer a headache—you can set up everything automatically while ensuring you have tight monitoring, access control, and scalability.

In short, your frontend will breathe a sigh of relief, and your backend will perform smoothly without bogging down the user experience. Really, what's not to love?

May your backend architecture remain clean, and your frontend blissfully unaware.

Happy coding—and may your BFF always have your frontend’s back! 👊

The updated Deployer Bot is not just smarter; it's designed to be more responsive and informative by utilizing an event-driven architecture that streamlines notifications and summaries. By tapping into the power of generative AI, the bot now offers concise, human-readable summaries of commits and release notes, making it easier for teams to grasp the impact of their work at a glance.

As an example... When I added this to my team's internal Slack Bot (based on the previous post's work). Bedrock provided this commit summary: "The commit enables the bot to summarize code changes and releases using AI via AWS Bedrock, including analyzing commits for risks and generating release prep recommendations between environments.". My commit message was only "bedrock... not handling brext w/ios though". 🤯

Commits trigger webhooks that flow through a series of AWS Lambda functions, orchestrating the process from commit tracking to AI-powered summarization, culminating in neatly packaged per-environment releases communicated via Slack. As part of the commit analysis and summarization, we get results like this on a completed deployment:

In this article, we'll explore the rationale behind each architectural decision, the process of incorporating Amazon Bedrock into our bot, and the benefits that an event-driven model brings to our CI/CD pipelines. For the DevOps enthusiasts and the code-curious alike, the complete codebase is accessible on GitHub at https://github.com/martzcodes/blog-cicd-bedrock-releases.

A quick note on costs: Amazon Bedrock and the models they run are NOT free. In my case, I've restricted the analysis to a limit of 10000 "tokens" so the max cost of processing a large commit or release for my case is about $0.10 - $0.20. Claude's upper token limit is 100k tokens which would have a max cost of $1-2 per model invocation.

Revamping the Architecture: Embracing Event-Driven Design

The transformation of the Deployer Bot’s architecture to an event-driven model marks a significant enhancement from its original design. This section will explore the rationale behind adopting an event-driven approach, the benefits it offers, and how it is implemented within the context of the Deployer Bot integrated with Amazon Bedrock for AI-powered commit summarization and release management.

Understanding Event-Driven Architecture (EDA)

Event-Driven Architecture (EDA) is a design paradigm centered around the production, detection, consumption, and reaction to events. An event is any significant state change that is of interest to a system or component. EDA allows for highly reactive systems that are more flexible, scalable, and capable of handling complex workflows. It is particularly well-suited for asynchronous data flow and microservices patterns, often found in cloud-native environments.

Why Event-Driven?

The original Deployer Bot followed a more traditional request/response model, where actions were triggered by direct requests. While functional, this approach had limitations in terms of scalability and real-time responsiveness. The integration of Amazon Bedrock and the necessity to process and summarize commit data presented an opportunity to redesign the architecture to be more reactive and efficient.

Benefits of Event-Driven Architecture

1. Scalability: EDA allows each component to operate independently, scaling up or down as needed without impacting the entire system.

2. Resilience: The decoupled nature of services in EDA results in a system that is less prone to failures. If one service goes down, the rest can continue to operate.

3. Real-Time Processing: Events can be processed as soon as they occur, providing immediate feedback and actions, which is crucial for CI/CD workflows.

4. Flexibility: New event consumers can be added to the architecture without impacting existing workflows, allowing for easier updates and enhancements.

Implementing EDA in Deployer Bot

The integration of EDA into the Deployer Bot involves several key components working in tandem:

1. Event Sources: These are the triggers for the workflow, such as GitHub webhooks for commits and deployments that initiate the process.

2. Event Bus: AWS services like Amazon EventBridge can serve as the backbone of EDA, routing events to the appropriate services.

3. Lambda Functions: Serverless functions respond to events, such as fetching, processing, and summarizing commit data, and orchestrating the workflow.

4. DynamoDB: Acts as the storage mechanism, logging events, and maintaining state where necessary.

5. Amazon Bedrock: Provides AI-powered summarization of commits and releases.

In the original architecture (above) we did everything synchronously driven by webhooks. A GitHub Action CI/CD deployment would occur that would lead to a message posted in Slack with a button. If the approve button was clicked it would create the next environment's deployment in GitHub. There was no tracking of commits (or even what was in a release) and as a user of this system for several months, it was often hard to link the Slack messages back to the actual code that was being deployed (despite the commit SHA's being there). There was a lot of mental overhead. 🥵

In the new architecture (below) we expand on this by isolating responsibilities between lambdas, making them simpler (do less) and tracking new information asynchronously.

Two parallel/asynchronous paths happen as part of the deployments in GitHub. The first path relates to the commit and the second relates to the deployment.

Some notes on the architecture diagram:

• The red lines are for visibility only (to help highlight the paths when lines cross each other).

• We're only using the Default Event Bus

• Lambdas F, B and 2 are API Driven and in the Nested API Stack

• The rest of the Lambdas are Event Driven and in the Nested Event Stack

For the first path with the GitHub Commit Webhook...

1. GitHub's commit webhook sends the push event to the /github/commit endpoint

2. The lambda makes sure that it was a commit to the main branch for a project we care about. It forwards an event to the event bus to process the commit message with the minimum information we need. It quickly responds back to GitHub with an OK status. (If we waited for the commit fetching/analysis via bedrock sometimes the lambda wouldn't respond quick enough and GitHub would think it failed).

3. Asynchronously the process-commit lambda fetches the actual FULL commit from GitHub which includes the patches made in this commit.

4. The commit w/patches are sent to Bedrock where we use Anthropic's Claude v2 LLM to summarize the commit into 1-2 sentences for a target audience of developers or product managers.

5. The commit (without the patches) and summary are then stored in DynamoDB for later release-querying.

The Commit path takes on the order of 30 seconds (frequently less) to complete. Meanwhile, since a commit occurred on the main branch and triggered GitHub Actions... the pipeline should be testing/deploying in the background. Once the GitHub Actions pipeline is complete it will send a deployment webhook:

A. The GitHub Actions pipeline completes sending a deployment webhook to the API

B. The Lambda that receives this webhook stores the deployment information in DynamoDB and emits two EventBridge events. One to send a message to the deployment channel in Slack and another to summarize the release.

C. The track-release lambda fetches all of the commits that occurred in the environment since the last release. Here a release is considered a group of commits that were newly deployed in an environment. The dev environment releases are (usually) single-commits. Ideally test and prod would follow this pattern but frequently there's some lag and the test/prod releases end up being larger. Note: this lambda also fetches the NEXT higher environment's commits (a sort of "draft" release) and those also get summarized. I should have spun this out into a separate lambda, but I'll leave that for future-me.

D. With the release commits fetched, they're all sent together to Bedrock to be summarized. For a larger release, it ends up summarizing multiple commit summaries.

E. With the release summary and next-env summary the track-release lambda stores the release notes in DynamoDB and sends an event to update the deployment's message with this new information.

F. (Arguably this could be a third path... 😅) Once the user clicks the approve button, the /slack/interactive endpoint emits an event to deploy the next environment.

G. A lambda receives that event and triggers the GitHub Actions pipeline for the next environment.

Code Structure

I'm not going to do a complete walkthrough of the code... because there is a lot of it. Instead I will highlight particular files of interest at a high level. Feel free to reach out on socials or in the comments if you'd like something explained more in-depth.

• blog-cicd-bedrock-releases-stack.ts - This stack creates the DynamoDB table and two Nested Stacks

• nested-api-stack.ts - Creates a RestAPI backed by Lambdas for the Webhook Endpoints

• routes/webhooks.ts - Defines the Endpoint structure for the lambdas and what permissions they should have via a common interface (nested-api-stack uses this to build the lambdas)

• constructs/api.ts - Creates the actual API and Lambdas w/their permissions and paths based on the routes/webhooks.ts file

• nested-event-stack.ts - Creates Event-Driven Lambdas and their Rules

• routes/events.ts - Similar to routes/webhooks.ts this defines the Lambdas with their corresponding Rules and Permissions.

• lambda/ - This folder contains all of the lambda runtime eligible code. Files from inside of here should not be making imports outside of this file structure (other than external npm libraries). This is to help isolate the code and ensure we aren't accidentally bundling things into our lambdas we don't need. I've seen this a lot in teams that make heavy use of index.ts files for imports 🤢

• lambda/common/bedrock.ts - The Bedrock Helper file which has the functions to do the various summaries.

😍 I've been using a similar pattern at work using Nested API and Nested EventBridge Stacks and am loving it. If you'd be interested in a dedicated post on that let me know in the comments!

Prompt-Engineering for Commit and Release Summaries

The integration of generative AI into the Deployer Bot's operations involved precise prompt engineering to ensure that commit and release summaries are informative and accessible. The focus was on creating concise yet comprehensive summaries tailored to the needs of both developers and product managers. The following discussion dives into how the code facilitates this process.

The below prompts are in the lambda helper methods at https://github.com/martzcodes/blog-cicd-bedrock-releases/blob/main/lib/lambda/common/bedrock.ts

Commit Summaries

For commit summaries, I designed a prompt to guide the AI to provide succinct summaries that highlight the purpose and potential impact of the changes, particularly emphasizing backward compatibility and flagging possible breaking changes. The following TypeScript excerpt outlines this process:

// Function to generate a summary for a single commit
export const summarizeCommit = async (commit: string): Promise<string> => {
  ...
  const prompt = `...Provide a 1-2 sentence summary of the commit that would be useful for developers and product managers...`;
  ...
};

In the function summarizeCommit, the prompt specifically instructs the AI to focus on a summary that is relevant to both technical stakeholders and decision-makers. This helps ensure that any non-backwards compatible changes are prominently reported, which is crucial for maintaining the integrity of the API.

Release Summaries

The task of summarizing releases brings together multiple commits into a narrative that outlines the key developments and their implications. The summarizeRelease function employs a carefully designed prompt to distill this information:

// Function to create a summary for a release
export const summarizeRelease = async (release: string): Promise<string> => {
  ...
  const prompt = `...You will create a 1-4 sentence summary of the release below...`;
  ...
};

Here, the prompt emphasizes not only the inclusion of changes but also highlights the importance of metrics, contributions, and cadence—all of which are critical for assessing the release's impact.

Environment Comparison Summaries

When preparing to promote changes from one environment to another, it's vital to understand the differences. The prepRelease function encapsulates this through its prompt, which is structured to provide a recommendation based on the commits analyzed:

// Function to summarize differences between environments and provide a release recommendation
export const prepRelease = async ({
  ...
}): Promise<string> => {
  ...
  const prompt = `...Make a recommendation for whether to promote or not...`;
  ...
};

In this function, the AI is tasked not just with summarizing the technical changes but also with evaluating the suitability of promoting the release, incorporating a strategic aspect into the summary.

Utilizing Amazon Bedrock Runtime

All these prompts are then passed to the Amazon Bedrock Runtime, invoking the model through InvokeModelCommand with an input that defines the parameters of the AI's generation process, including token limits and stop sequences. These configurations are essential for controlling costs and ensuring the responses are concise:

const command = new InvokeModelCommand(input);
const response = await client.send(command);
...

This snippet is a crucial part of the process, as it executes the command and handles the response from the Bedrock AI, translating it into a usable summary.

ONE IMPORTANT NOTE: At the time of this writing (11/6/2023) the AWS Lambda Runtime for NodeJS (18) does NOT include the @aws-sdk/client-bedrock bundled into that. As an added "bonus" the CDK's NodejsFunction construct (which uses esbuild) by default makes @aws-sdk/* as external modules. This means that @aws-sdk/client-bedrock ends up NOT being bundled into the lambda. In order to get around this I needed to update our NodejsFunction props to bypass this. I also have to give the lambdas IAM access to invoke bedrock models. This can be done via by adding an initial policy to the lambda:

const fn = new NodejsFunction(this, `${endpoint.lambda}Fn`, {
  // ...
  bundling: {
    // Nodejs function excludes aws-sdk v3 by default because it is included in the lambda runtime
    // but bedrock is not built into the lambda runtime so we need to override the @aws-sdk/* exclusions
    externalModules: [
      "@aws-sdk/client-dynamodb",
      "@aws-sdk/client-eventbridge",
      "@aws-sdk/client-secrets-manager",
      "@aws-sdk/lib-dynamodb",
    ],
  },
  ...(endpoint.bedrock && {
    initialPolicy: [
      new PolicyStatement({
        effect: Effect.ALLOW,
        actions: ["bedrock:InvokeModel"],
        resources: ["*"],
      }),
    ],
  }),
});

Continual Evolution of Prompts

It's important to note that these prompts are not static. They are subject to continuous evaluation and iteration, ensuring that the summaries remain pertinent and value-adding as the project and AI capabilities evolve.

By embedding such targeted prompts into the Deployer Bot's workflow, the DevOps team ensures that the summaries generated are not only informative but also actionable, fostering a deeper understanding and facilitating informed decision-making throughout the development process.

Now let's try it!

In order to test this and iterate on my prompt templates, I created a CICD-example project: https://github.com/martzcodes/cicd-example

This project uses OIDC and GitHub Actions to deploy the stack. In this case, I'm just deploying the same stack with an "environment"-specific name to the same account. To reset I would stash my changes and force-push to an earlier state and re-apply the stashes.

Backwards compatibility is really important in software engineering, so one of the first things I wanted to focus on was that. I created a simple RestApi with a single endpoint pointed at /dummy. On my first attempt at prompt engineering, I included a statement like APIs must be backwards compatible, if they are not make a note of it.

I then did a deployment where I renamed the /dummy endpoint to /something (creative, I know). The response from bedrock specifically said this was backwards compatible / not a breaking change:

The release for cicd-example in prod environment contains 5 commits:
- Adds an API Gateway API with Lambda endpoint
- Defines the Lambda handler function
- Updates API path from /dummy to /example (committed twice)
- Updates API Gateway path resource from dummy to example
No breaking changes or bugs were noted. The API update is backwards compatible.

A renamed endpoint could absolutely be breaking. After a few iterations, I settled on a prompt line like: APIs must be backwards compatible which includes path changes, if they are not it should be highlighted in the summary. After that, I got much more reliable callouts for path changes*.

I also made an attempt for bedrock/Claude to detect misspellings in code. For example, I defined an endpoint called /soemthing ... I could not find a prompt combination that would identify that misspelling... and in fact, in the summaries, it actually corrected it (which is VERY BAD) 😬

The release adds a new API and Lambda function. A new /something endpoint was added to the API without breaking backwards compatibility.

After installing it at work and running it for a day I asked my colleagues for feedback on the accuracy... and the feedback was very positive but it wasn't perfect.

For example...

The XXXXXX repo in the dev environment released changes on 2023-11-06T22:26:41.517Z. It includes 1 commit which adds a new '/cognito/revoke' endpoint that could break backwards compatibility if clients are not updated. No other major changes or risks noted.

New endpoints can rarely be a breaking change. Back to the drawing board, I guess 😅

Future Directions and Conclusion

As we continue to explore the intersection of generative AI and DevOps, I can see a lot of potential for a GenerativeAI+Serverless Deployer Bot. The integration of AI-driven summaries for commits and releases is just the beginning. The future is poised for a host of innovative features that could transform CI/CD pipelines and development workflows, making them more efficient and intelligent.

Expanding AI Capabilities in DevOps

• Automated Code Review Assistance: By refining our prompts, we could extend the Deployer Bot's functionality to include automated code reviews, where the bot could provide preliminary feedback on pull requests, analyzing code for style, complexity, and even security vulnerabilities.

• Dynamic Troubleshooting Guides: Generative AI could be harnessed to create real-time troubleshooting guides based on the errors and logs encountered during builds or deployments, providing developers with immediate, context-specific solutions.

• Predictive Analytics for CI/CD: Leveraging historical data, the bot could predict potential bottlenecks and suggest optimizations in the CI/CD pipeline, leading to preemptive resource management and smoother release cycles.

• Personalized Developer Assistance: AI could be programmed to learn individual developer preferences and work patterns, offering customized tips, reminders, and resources to enhance productivity.

• Enhanced Onboarding: For new team members, the Deployer Bot could become an on-demand mentor, explaining CI/CD processes, and codebase navigation, and providing answers to common questions through an interactive AI-driven chat.

• AI-Powered Testing and Quality Assurance: Integrating AI to analyze test results could lead to quicker identification of flaky tests and provide insights on test coverage and quality, potentially predicting which parts of the code are most likely to fail.

Conclusion

The integration of generative AI into the Deployer Bot represents a significant leap forward for DevOps teams. It is a testament to the transformative potential of AI when applied with precision and creativity. The Deployer Bot, once a mere facilitator of notifications, has evolved into a sophisticated assistant that enhances decision-making and streamlines workflows. Looking ahead, I am excited about the prospect of a more proactive, AI-powered assistant that not only informs but also predicts and strategizes, becoming an indispensable ally in the fast-paced world of software development.

The current capabilities of the Deployment Deployer Bot lay the foundation for these advancements. I'm sure this will not be my last post on the matter. My recent trip to EDA Day in Nashville gave me a lot of inspiration.

• GitHub Deployments notifies our serverless app, which consists of an AWS API Gateway Rest API backed by multiple lambda functions. One of these lambdas gets invoked by the GitHub Deployments webhook to handle deployment status updates.

• Once informed, our bot relays these updates to a Slack channel.

• Upon a successful deployment, the bot prompts users with, "Do you want to deploy this to the next environment?"

• Users have the option to greenlight this deployment or reject it.

• And when it comes to the all-important production environment, only selected approvers can make the final decision. Others can voice their views through a "vote".

The best part? Although our demonstration employs CDK, this framework isn't limited to CDK deployments. You can adapt it for any project deployable via GitHub Actions, making it a flexible tool for various deployment needs.

Here's a structured flow of our approach:

1. Setting up a placeholder for secret tokens.

2. Constructing the CDK App with key components: RestApi, DynamoDB, and several lambdas.

3. Safeguarding the GitHub access token and Slack Bot token inside our placeholder.

4. Integrating the GitHub Deployment Webhook.

5. Ensuring seamless Slack interactions.

6. Incorporating GitHub Deployments within our GitHub Actions workflows.

Eager to dive into the code? Check out the project on GitHub: https://github.com/martzcodes/blog-cicd-slackbot.

Note: This project has multiple layers, and we won't be delving into each line of code. If you find any part lacking, don't be shy. Drop your questions in the comments or catch me on Mastodon at https://awscommunity.social/@martzcodes.

Create a Placeholder Secret

Why a Placeholder?: Before diving in, you might wonder why we're setting placeholders. When working with AWS CDK, there's a tendency to automate the creation of secrets. However, I've found this method can sometimes reset these secrets unintentionally. And although CloudFormation is an option, it would expose our Secret values in its YAML file. This is where manually set placeholders come in handy — they ensure our secrets remain secret.

Let's get started:

1. Head over to the AWS Console and create a new Secret.

2. Choose the "Other type of secret" option.

3. Toggle over to the Plaintext option under the Key/value pairs tab.

4. For the value, enter: {"SLACK_TOKEN":"xoxb-placeholder","GITHUB_TOKEN":"github_pat_placeholder"}.

5. Click on "Next".

6. Name your secret slackbot-deployer.

7. Proceed by clicking "Next".

8. Opt for "No rotation" and click "Next".

9. Finally, click on "Store" to save the secret.

Once saved, refreshing the secrets list will show you the secret's details. Look out for the secret ARN, which will appear something like this: arn:aws:secretsmanager:us-east-1:123456789012:secret:slackbot-deployer-XYZ123. Keep this ARN handy, as we'll integrate it into our CDK App in the subsequent steps.

Deploying the CDK App

Time to create our CDK App! If you're looking to speed things up, grab the example code from my GitHub. Alternatively, for the DIY enthusiasts, start a CDK project from the ground up. Navigate to your desired project folder and initiate the project with:

npx cdk init --language typescript

This command spins up a CDK version 2 project. If you've been working with a globally installed version 1 CDK, it's time to uninstall it and stick with npx.

Once our app is initialized, we're going to make changes to the bin/blog-cicd-slackbot.ts file to introduce necessary configurations.

const oidcs: Record<string, string> = {
  test: "arn:aws:iam::922113822777:role/GitHubOidcRole",
  prod: "arn:aws:iam::349520124959:role/GitHubOidcRole",
};
const nextEnvs: Record<string, string> = {
  dev: "test",
  test: "prod",
};

const app = new cdk.App();
new BlogCicdSlackbotStack(app, "BlogCicdSlackbotStack", {
  nextEnvs,
  oidcs,
  secretArn: "YOUR SECRET ARN HERE",
});

This configuration sets the stage for our environment flow and the OIDC Roles, enabling GitHub Actions to deploy CDK applications without saving secrets in GitHub. For a hands-on demonstration on crafting these OIDC roles with CDK, check out the Construct in lib/github-oidc.ts. While my roles are provisioned externally and are commented out, this should offer you a practical reference.

Create a RestAPI

Got the starter code? Jump ahead to "Deploy the App". If not, keep reading.

Let's tweak the lib/blog-cicd-slackbot-stack.ts file, first defining the Stack's Prop interface to resonate with the bin file. Here, we'll amplify the default StackProps interface by integrating three additional attributes:

export interface BlogCicdSlackbotStackProps extends cdk.StackProps {
  nextEnvs: Record<string, string>;
  oidcs: Record<string, string>;
  secretArn: string;
}

Then, modify the constructor like so:

constructor(scope: Construct, id: string, props: BlogCicdSlackbotStackProps) {

Our next move is forging the RestApi and carving out a slack/ resource:

const api = new RestApi(this, "BlogCicdSlackbotApi", {
  deployOptions: {
    dataTraceEnabled: true,
    tracingEnabled: true,
    metricsEnabled: true,
  },
  description: `API for BlogCicdSlackbotApi`,
  endpointConfiguration: {
    types: [EndpointType.REGIONAL],
  },
});
const slackResource = api.root.addResource("slack");

Let's establish our default lambda environment and properties:

const environment = {
  OIDCS: JSON.stringify(oidcs),
  SECRET_ARN: secret.secretArn,
  NEXT_ENVS: JSON.stringify(nextEnvs),
  TABLE_NAME: table.tableName,
};

const lambdaProps = {
  runtime: Runtime.NODEJS_18_X,
  memorySize: 1024,
  timeout: cdk.Duration.seconds(30),
  environment,
};

Remember: Not all our lambda functions will harness the secret/table and related functionalities. We're only granting access to lambdas if they genuinely require it. Yet, knowledge of the SECRET_ARN is harmless.

Wrap it up by creating the lambda and its endpoint:

const slackAction = new NodejsFunction(this, "SlackActionFn", {
  entry: "lib/lambda/api/slack-action.ts",
  ...lambdaProps,
});
slackResource.addResource("action").addMethod(
  "POST",
  new LambdaIntegration(slackAction)
);

Peek into my lambda's handler code present in lib/lambda/api/slack-action.ts:

import { APIGatewayEvent } from "aws-lambda";

export const handler = async (event: APIGatewayEvent) => {
  const body = JSON.parse(event.body || "{}");
  console.log(JSON.stringify({body}, null, 2));

  return {
    statusCode: 200,
    body: body.challenge,
  };
};

Heads Up!: It's essential to have esbuild and aws-lambda types added to your project. Here's how:

npm i --save-dev esbuild @types/aws-lambda

This ensures CDK avoids invoking a Docker container for lambda bundling.

Deploy the App

Everything's set? Deploy the app! Once done, you'll receive the API Url as an Output resembling:

Outputs:
BlogCicdSlackbotStack.BlogCicdSlackbotApiEndpointCDDA7E36 = https://xicnr82c7a.execute-api.us-east-1.amazonaws.com/prod/

Jot down this URL (inclusive of /prod/) – we'll integrate it into the Slack App's manifest soon.

Gathering the Secrets

Before diving deep, let's ensure we have all the secrets we need for this project safely tucked away in our placeholder Secret.

We'll retrieve two primary secrets:

1. GitHub Personal Access Token: Enables our application to interact with GitHub Actions.

2. Slack App Bot Token: Helps us create and edit Slack messages.

GitHub Personal Access Token

For this, we'll employ GitHub's fine-grained tokens, which offer precise control over permissions.

1. Head over to the GitHub tokens page.

2. Remember, these tokens come with an expiration which you can set for up to a year.

3. If you're working within an organization like me, set the "Resource owner" to your organization.

4. Grant access to All repositories (both public and private).

5. Then, ensure you grant the following repository-level access:

   • Actions: Read & Write

   • Deployments: Read

   • Environments: Read

   • Metadata: Read (Mandatory)

Once you've completed these steps, you can copy your access token. It'll look something like this: github_pat_BLAH. Keep this safe – we'll need it shortly.

Creating a Slack App

Time to nab the Slack App's Bot token:

1. Start by visiting the Slack API page.

2. Click on "Create New App".

3. Opt to create an app using a manifest:

{
    "display_information": {
        "name": "deploy-bot"
    },
    "features": {
        "bot_user": {
            "display_name": "deployer",
            "always_online": true
        },
        "slash_commands": [
            {
                "command": "/deployer_add_auth",
                "url": "REPLACE THIS WITH YOUR APIGW URL/slack/add-approver",
                "description": "Add an approver for Deployments",
                "usage_hint": "@user",
                "should_escape": true
            },
            {
                "command": "/deployer_list_auth",
                "url": "REPLACE THIS WITH YOUR APIGW URL/slack/list-approvers",
                "description": "Show who can approve deployments",
                "should_escape": false
            },
            {
                "command": "/deployer_remove_auth",
                "url": "REPLACE THIS WITH YOUR APIGW URL/slack/remove-approver",
                "description": "Remove someone as an approver",
                "usage_hint": "@user",
                "should_escape": true
            }
        ]
    },
    "oauth_config": {
        "scopes": {
            "user": [
                "users.profile:read"
            ],
            "bot": [
                "app_mentions:read",
                "channels:history",
                "chat:write",
                "chat:write.customize",
                "chat:write.public",
                "emoji:read",
                "groups:history",
                "groups:read",
                "groups:write",
                "im:history",
                "im:read",
                "im:write",
                "incoming-webhook",
                "pins:read",
                "pins:write",
                "reactions:read",
                "reactions:write",
                "users:read",
                "users.profile:read",
                "commands"
            ]
        }
    },
    "settings": {
        "event_subscriptions": {
            "request_url": "REPLACE THIS WITH YOUR APIGW URL/slack/action",
            "bot_events": [
                "app_mention"
            ]
        },
        "interactivity": {
            "is_enabled": true,
            "request_url": "REPLACE THIS WITH YOUR APIGW URL/slack/interaction"
        },
        "org_deploy_enabled": false,
        "socket_mode_enabled": false,
        "token_rotation_enabled": false
    }
}

1. Once the app's up and running, install it in your workspace.

2. Navigate to the OAuth & Permissions page to fetch the Bot User OAuth Token.

Update the Placeholder Secret

Armed with both tokens, revisit your placeholder Secret on the AWS Console. Here's what you do:

1. Click on Retrieve secret value.

2. Choose Edit.

3. Enter the tokens in their respective key-value fields.

4. Don't forget to hit "Save" after inputting both tokens.

Get ready, the exciting part is about to start!

Setting Up the GitHub Deployment Webhook Integration

Integrating GitHub Deployment Status webhook can be a game-changer. Not only will it ensure timely Slack notifications but also helps in maintaining a reliable history in DynamoDB.

1. Provisioning a DynamoDB Table

Firstly, we need to create a table where deployments will be recorded:

const table = new Table(this, "Table", {
  partitionKey: { name: "pk", type: AttributeType.STRING },
  sortKey: { name: "sk", type: AttributeType.STRING },
  billingMode: BillingMode.PAY_PER_REQUEST,
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  timeToLiveAttribute: "ttl",
});

This table is straightforward. Given that our traffic isn't predictable, opting for the Pay Per Request model ensures cost-effectiveness.

2. Incorporating the Secret

Before we proceed, it's essential to incorporate the secret saved from the earlier stages:

const secret = Secret.fromSecretCompleteArn(this, `BlogCicdSlackbotSecret`, secretArn);

3. Lambda & API Gateway Endpoint

As there’s only one GitHub endpoint, bundling the creation of both the Lambda function and its associated resource streamlines the process. Make sure the Lambda function can access both DynamoDB and Secrets Manager:

const githubWebhookFn = new NodejsFunction(this, "GithubWebhookFn", {
  entry: "lib/lambda/api/github-webhook.ts",
  ...lambdaProps,
});
table.grantReadWriteData(githubWebhookFn);
secret.grantRead(githubWebhookFn);
api.root
  .addResource("github")
  .addMethod("POST", new LambdaIntegration(githubWebhookFn));

Dive into lib/lambda/api/github-webhook.ts to examine the logic. While the file may seem hefty, the bulk of it centers around Slack message formatting.

- Storing Deployment Details:

We extract vital details from the event to be logged in DynamoDB:

const {
  state: status,
  environment: env,
  created_at: createdAt,
  updated_at: updatedAt,
  target_url: url,
} = body.deployment_status;
const { id: deploymentId, ref: branch, sha } = body.deployment;
const repo = body.repository.name;
const author = body.deployment_status.creator.login;
const owner = body.repository.owner.login;

- Retrieving Slack Token:

Fetch the Slack token securely from Secrets Manager:

const secret = await sm.send(
  new GetSecretValueCommand({
    SecretId: process.env.SECRET_ARN,
  })
);
const slackToken = JSON.parse(secret.SecretString || "").SLACK_TOKEN;

- Managing Deployment Statuses:

When a new deployment status hits, it's crucial to determine its relative standing. We do this by fetching the most recent deployment status and comparing the deploymentId:

const pk = `REPO#${repo}#ENV#${env}`.toUpperCase();
const sk = "LATEST";

// get deployment status from dynamodb
const ddbRes = await ddbDocClient.send(
  new GetCommand({
    TableName: process.env.TABLE_NAME,
    Key: {
      pk,
      sk,
    },
  })
);

If the deploymentId from the incoming webhook doesn't match the last "LATEST" item, it implies a newer deployment has superseded it. As a result, we should:

1. Update the previous Slack message by removing its action buttons.

2. Append a note indicating Automatic rejection by subsequent deployment.

3. Archive the last item under a unique Sort Key, ensuring it’s retrievable but not in the immediate queue.

const slackRes = await fetch("https://slack.com/api/chat.update", {
  method: "POST",
  body: JSON.stringify({
    channel: "C04KW81UAAV",
    ts: existingItem.slackTs,
    blocks: oldBlocks,
  }),
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${slackToken}`,
  },
});
await slackRes.json();
await ddbDocClient.send(
  new PutCommand({
    TableName: process.env.TABLE_NAME,
    Item: {
      ...existingItem,
      sk: `DEPLOYMENT#${existingItem.deploymentId}`.toUpperCase(),
      blocks: JSON.stringify(oldBlocks),
    },
  })
);

New deployments, on the other hand, engage the chat.postMessage Slack API, furnishing essential deployment details.

For deployments matching the "LATEST" deploymentId, it’s crucial to ensure the incoming status isn’t redundant. Successful deployments headed for another environment get approve/reject action buttons. This updated deployment, now the newest, gets stored under the "LATEST" sort key, paired with the Slack message timestamp for subsequent edits.

Lastly, successful deployments are logged in a meta item, an inventory of all triumphant deployments segregated by environment:

const envLatest = await ddbDocClient.send(
    new GetCommand({
      TableName: process.env.TABLE_NAME,
      Key: {
        pk: `LATEST`,
        sk: `${nextEnvs[env]}`,
      },
    })
  );
  const updatedRepo = {
    url: item.url,
    sha: item.sha,
    deploymentId: item.deploymentId,
    deployedAt: Date.now(),
    branch: item.branch,
    owner: item.owner,
  };
  if (!envLatest.Item) {
    await ddbDocClient.send(
      new PutCommand({
        TableName: process.env.TABLE_NAME,
        Item: {
          pk: `LATEST`,
          sk: `${nextEnvs[env]}`,
          repos: {
            [repo]: updatedRepo,
          },
        },
      })
    );
  } else {
    // Update existingItem by replacing the repo
    await ddbDocClient.send(
      new UpdateCommand({
        TableName: process.env.TABLE_NAME,
        Key: {
          pk: `LATEST`,
          sk: `${nextEnvs[env]}`,
        },
        // update the repos attribute
        UpdateExpression: "SET repos.#repo = :repo",
        ExpressionAttributeNames: {
          "#repo": repo,
        },
        ExpressionAttributeValues: {
          ":repo": updatedRepo,
        },
      })
    );
  }

The structuring of this meta item is worth noting. The deployment details nestle within a repos map in DynamoDB. Leveraging UpdateCommand helps pinpoint updates to specific repositories, ensuring that new build data doesn't accidentally overwrite unrelated data due to race conditions.

4. Integrating the GitHub Webhook

Navigate to your GitHub Organization's settings. Create a fresh webhook with the following attributes:

• Payload URL: your-apigateway-url/github

• Content type: application/json

• SSL Verification: Enabled

• Events: Specifically opt for Deployment statuses

• Status: Active

Finalize your configurations. Now, every GitHub Deployment activates the Lambda, meticulously processing deployment data in line with your design.

Integrating Slack Interactions

Let's enhance the workflow and user experience by setting up Slack interactions for our deployment notifications.

1. Establishing the Interaction Endpoint

To catch Slack interactions like button presses, we need a dedicated endpoint. Update the lib/blog-cicd-slackbot-stack.ts to introduce this Lambda function and endpoint:

const slackInteractiveFn = new NodejsFunction(this, "SlackInteractiveFn", {
  entry: "lib/lambda/api/slack-interactive.ts",
  ...lambdaProps,
});
table.grantReadWriteData(slackInteractiveFn);
secret.grantRead(slackInteractiveFn);
slackResource
  .addResource("interaction")
  .addMethod("POST", new LambdaIntegration(slackInteractiveFn));

Though the handler for this function (lib/lambda/api/slack-interactive.ts) is extensive, it's primarily devoted to processing Slack messages and extracting information from the event.

- Decoding the Slack Payload:

Given Slack's use of the x-www-form-urlencoded content type, manual decoding becomes imperative:

const decodedString = decodeURIComponent(event.body!);
const jsonString = decodedString.replace("payload=", "");
const jsonObject = JSON.parse(jsonString);
const message = jsonObject.message;
const approved = jsonObject.actions[0].value === "approved";
const repo = jsonObject.message.text.split("Repo:*\n")[1].split("+")[0];
const env = jsonObject.message.text
  .split("+deployment+to+")[1]
  .split("+by+")[0];
const authority = jsonObject.user.name; // user who did the interaction
const branch = jsonObject.message.text.split("Branch:*\n")[1].split("+")[0];

- Fetching Tokens:

This Lambda could require tokens for both GitHub and Slack. Thus, let’s retrieve them:

const secret = await sm.send(
  new GetSecretValueCommand({
    SecretId: process.env.SECRET_ARN,
  })
);
const slackToken = JSON.parse(secret.SecretString || "").SLACK_TOKEN;
const githubToken = JSON.parse(secret.SecretString || "").GITHUB_TOKEN;

- Retrieving User's Image:

To enrich our Slack messages with user details, fetch the user's profile picture via the Slack API:

const slackAuthorityRes = await fetch(
  `https://slack.com/api/users.profile.get?user=${jsonObject.user.id}`,
  {
    method: "GET",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${slackToken}`,
    },
  }
);
const slackAuthority = await slackAuthorityRes.json();
const userImg = slackAuthority.profile.image_24;

- Processing Slack Messages:

A series of operations then follows:

1. Embed the user's profile picture, corresponding to their action (approve/reject).

2. Handle vote changes to ensure clarity in responses.

3. For deployments to "prod", cross-verify the user's authority against a list of approved users.

On gaining approval:

1. Update the LATEST deployment item status

2. Retrieve the workflow list for the repository, enabling us to identify the correct workflowId

3. Initiate the deployment for the succeeding environment

await ddbDocClient.send(
  new PutCommand({ TableName: process.env.TABLE_NAME, Item: existingItem })
);
const githubListWorkflowsRes = await fetch(
  `https://api.github.com/repos/${existingItem.owner}/${repo}/actions/workflows`,
  {
    method: "GET",
    headers: {
      Accept: "application/vnd.github+json",
      "X-GitHub-Api-Version": "2022-11-28",
      Authorization: `Bearer ${githubToken}`,
    },
  }
);
const { workflows } = await githubListWorkflowsRes.json();
const workflow = workflows.find(
  (workflow: any) => workflow.name === "deploy-to-env"
);
await fetch(
  `https://api.github.com/repos/${existingItem.owner}/${repo}/actions/workflows/${workflow.id}/dispatches`,
  {
    method: "POST",
    body: JSON.stringify({
      ref: branch,
      inputs: {
        deploy_env: nextEnvs[env],
        oidc_role: oidcs[nextEnvs[env]],
      },
    }),
    headers: {
      Accept: "application/vnd.github+json",
      "X-GitHub-Api-Version": "2022-11-28",
      Authorization: `Bearer ${githubToken}`,
    },
  }
);

Concluding this segment, the Slack message is refreshed, and the updated information is stored in DynamoDB.

2. Managing Approvers

The next phase involves facilitating the management of deployment approvers via Slack slash commands. Three distinct Lambdas will handle these functionalities – addition, removal, and listing of approvers.

An initial approver sets the tone, and subsequent approvals or removals of approvers happen under their discretion. To glean details about an approver, like their name, email, or profile picture, the add-approver endpoint uses the Slack Token.

For those eager to dive into the implementation, the code is hosted in the following:

• lib/lambda/api/slack-add-approver.ts

• lib/lambda/api/slack-list-approvers.ts

• lib/lambda/api/slack-remove-approver.ts

Though we're summarizing this section, remember that having a robust system of approval is paramount, especially for production deployments.

Setting Up GitHub Deployments with GitHub Actions

To harness the power of GitHub Deployments, let's configure some environments for your project.

1. Configuring Environments:

First, head to your repository's settings page and open the Environments tab. For instance, the URL might resemble: https://github.com/martzcodes/blog-cicd-slackbot/settings/environments. Here, introduce a fresh environment for each stage you aim to monitor. Ensure their names correspond with your nextEnvs configuration object. For example, my setup included dev, test, and prod.

2. Configuring GitHub Actions:

Now, ensure your deployment-focused GitHub Actions are set to employ GitHub Deployments. Some sample workflows are available at: GitHub Sample Workflows.

The pipeline.yml file, which springs into action upon commits to the main branch, facilitates continuous deployment to the dev environment. This action sets the stage for our Slack integration. Notably, this pipeline is named Deploy - a detail the GitHub webhook Lambda verifies.

Subsequently, the deploy-to-env.yml workflow is tailored to possess matching inputs. Triggered by the pipeline.yml workflow, both workflow_call and workflow_dispatch triggers accept these inputs:

inputs:
  deploy_env:
    description: 'Environment to deploy to'
    required: true
    type: string
  oidc_role:
    description: 'OIDC Role to assume for deployment'
    required: true
    type: string

Although your deployment steps could differ, it's crucial to encapsulate your deployment within status update stages:

  - name: start deployment
    uses: bobheadxi/deployments@v1.2.0
    id: deployment
    with:
      step: start
      env: ${{ inputs.deploy_env }}
  - name: ${{ inputs.deploy_env }} deploy
    env:
      DEPLOY_ENV: ${{ inputs.deploy_env }}
    run: npx cdk deploy --ci --require-approval never --concurrency 5 -v
  - name: update deployment status
    uses: bobheadxi/deployments@v1.2.0
    with:
      step: finish
      status: ${{ job.status }}
      env: ${{ inputs.deploy_env }}
      deployment_id: ${{ steps.deployment.outputs.deployment_id }}

The central idea here is that the bobheadxi/deployments action communicates with GitHub's API to register a deployment for the relevant environment. For a clearer perspective, a live example resides here: Live GitHub Example.

Live Demonstration

Let's observe this integration in its full glory.

1 - Initialization: Using the Slack slash command /deployer_list_auth, I'll confirm our approver list starts empty:

2 - Commencing a Deployment: I'll initiate a deployment in my dev environment and after a brief wait an in_progress message surfaces:

3 - Deployment Completion: On the successful completion of the deployment, our message updates, introducing action buttons. Given our next environment, "test", no approvers are mandated:

4 - Deployment Approval: Tapping the "Approve" button results in another message transformation, indicating approval along with the removal of the action buttons:

5 - Test Environment Deployment: Shortly, the in_progress message for the test environment arrives:

6 - Rejection Attempt: After the test environment has a successful deployment and the buttons appear... As an outsider to the approver list, my "Reject" action prompts an updated message, retaining the buttons:

7 - Adding to the Approver List: I'll employ the /deployer_add_auth command to add myself to the approver list:

8 - Final Rejection: After clicking the "Reject' button again the deployment is successfully rejected, with our message updated accordingly:

Conclusion

The intertwining of continuous integration and deployment with communication tools, as we've explored in this journey, opens a myriad of possibilities. Not only does it seamlessly merge developer actions with team-wide notifications, but it also introduces an additional layer of transparency and control over our deployment processes. The adaptability of the systems we've integrated, namely GitHub and Slack, offer a rich tapestry of features to build upon, as demonstrated by our use of GitHub Deployment and its potential with GitHub Actions.

It's worth noting the original plan was to leverage GitHub Deployment Protection rules. These rules present a solid framework for controlling deployments in a more granular way. However, a significant limitation arose: their availability is restricted to either public repositories or those operating under GitHub Enterprise. This limitation led to a more creative approach, embedding the essence of what these rules offer, but in a broader context suitable for various repository types.

To conclude, technology continues to provide tools and platforms, ripe with features and functionalities, waiting to be moulded and interconnected in ways that best suit our needs. This exploration was just a glimpse into the vast world of CI/CD and team communication integrations. As you embark on your own integrative ventures, remember that while off-the-shelf solutions are great, sometimes thinking outside the box— or outside the repository, in this case— can lead to even more robust and tailor-made solutions for your team.

While AWS's initial tutorial covered basic username/password authentication... users appreciate convenience and security. To support this, we'll add "Sign in with Apple" support, which offers users a seamless and privacy-focused login option. With "Sign in with Apple," users can authenticate with their Apple ID and stay in control of their personal information (including giving them the ability to hide their email).

A successful app requires understanding user behavior and optimizing user experiences. That's where AWS Pinpoint Analytics comes into play. I'll demonstrate how to integrate AWS Pinpoint into the notes app, enabling us to collect critical user engagement data. With this newfound insight, we can analyze user interactions, monitor feature adoption, and make data-driven decisions to enhance the app's performance.

AWS Amplify is a comprehensive development platform offered by Amazon Web Services (AWS) that simplifies the process of building web and mobile applications. It provides developers with a set of tools, services, and libraries to accelerate the development of cloud-powered applications.

This tutorial assumes you are familiar with the basics of AWS Amplify, including authentication setup and working with the Amplify Storage component. If you're new to these concepts, I recommend checking out the original AWS Amplify tutorial to get up to speed.

The code for this project is located here: https://github.com/martzcodes/blog-amplifyhackathon-ios-2023

Updating the App to use "Sign in with Apple"

Amplify and Apple make it really easy to incorporate "Sign in with Apple" (SIWA) into your applications. SIWA allows you to federate users into an Amazon Cognito identity pool using the AWS Amplify Libraries for Swift. By federating a user into a Cognito identity pool, it allows us to use temporary AWS IAM credentials for the users based on the identity token that Apple provides us. In doing so, we can use those AWS IAM credentials to access other services like Amazon S3, AppSync and Pinpoint.

This article will show you how to use Sign in with Apple (SIWA) to retrieve an identity token and federate the user in an Amazon Cognito identity pool using the AWS Amplify Libraries for Swift. Federating a user in an identity pool provides them with credentials that allow them to access other services like Amazon S3 and Amazon DynamoDB.

First, we'll update amplify's auth to use Apple's provider. Below is the full list of answers when running amplify update auth:

$ amplify update auth
 What do you want to do? Walkthrough all the auth configurations
 Select the authentication/authorization services that you want to use: User Sign-Up, Sign-In, connected with AWS IAM controls (Enables per-user Storage features for images or other content, Analytics, and more)
 Allow unauthenticated logins? (Provides scoped down permissions that you can control via AWS IAM) Yes
 Do you want to enable 3rd party authentication providers in your identity pool? Yes
 Select the third party identity providers you want to configure for your identity pool: Apple

 You've opted to allow users to authenticate via Sign in with Apple. If you haven't already, you'll need to go to https://developer.apple.com/account/#/welcome and configure Sign in with Apple.

 Enter your Bundle Identifier for your identity pool:  <your app bundle id from apple>
 Do you want to add User Pool Groups? No
 Do you want to add an admin queries API? No
 Multifactor authentication (MFA) user login options: OFF
 Email based user registration/forgot password: Enabled (Requires per-user email entry at registration)
 Specify an email verification subject: Your verification code
 Specify an email verification message: Your verification code is {####}
 Do you want to override the default password policy for this User Pool? No
 Specify the app's refresh token expiration period (in days): 30
 Do you want to specify the user attributes this app can read and write? No
 Do you want to enable any of the following capabilities?
 Do you want to use an OAuth flow? No
? Do you want to configure Lambda Triggers for Cognito? Yes
? Which triggers do you want to enable for Cognito

Identity pools do NOT use Cognito User Groups... they use AWS IAM-based access. In order to make use of these for our app, we need to switch AWS AppSync to use IAM auth instead of Cognito User Groups. Let's run amplify update api. Below are the full answers for this section:

amplify update api
? Select from one of the below mentioned services: GraphQL
...
? Select a setting to edit Authorization modes
? Choose the default authorization type for the API IAM
? Configure additional auth types? No

Before we can push these changes, we need to update our schema.graphql. It was using auth based on Cognito Pools, but we need to switch it to IAM-based auth. For demo sake, we also want to enable guests (unauthenticated users) to also read notes. We'll update the schema.graphql file to:

type NoteData
@model
@auth(
    rules: [
      { allow: public, provider: iam, operations: [read] }
      { allow: private, provider: iam, operations: [read, create, update, delete] }
    ]
  ) {
    id: ID!
    name: String!
    description: String
    image: String
}

• { allow: public, provider: iam, operations: [read] } gives guests read access

• { allow: private, provider: iam, operations: [read, create, update, delete] } gives logged in users full CRUD access

With the schema.graphql updated, we can run amplify codegen models to regenerate the APIs.

With these changes, we can run amplify push and our backend will update to reflect these new changes. Next, we'll need to modify some of the swift code.

Instead of using Amplify.Auth.signInWithWebUI we need to use the SignInWithApple button and then make a call to federateToIdentityPool. SignInWithApple is a capability provided by Apple. Once signed in, it provides an identity token that is used by federateToIdentityPool

In ContentView.swift:

// add this to the top of the file
import AuthenticationServices

// In the ContentView view:
func configureRequest(_ request: ASAuthorizationAppleIDRequest) {
    request.requestedScopes = [.email]
}
func handleResult(_ result: Result<ASAuthorization, Error>) {
    switch result {
    case .success(let authorization):
        guard let credential = authorization.credential as? ASAuthorizationAppleIDCredential,
                let identityToken = credential.identityToken else {
                    return
                }
        guard let tokenString = String(data: identityToken, encoding: .utf8) else {
            return
        }
        Backend.shared.federateToIdentityPools(with: tokenString)
        self.userData.isSignedIn = true;
    case .failure(let error):
        print(error)
    }
}

// replace the original sign in button with:
SignInWithAppleButton(
    onRequest: configureRequest,
    onCompletion: handleResult
)
.frame(maxWidth: 300, maxHeight: 45)

SignInWithAppleButton triggers the SIWA capability by Apple. configureRequest ensures that the email is returned as part of the scope of the identityToken. On completion then parses the identityToken and sends it to our Backend.swift service.

In Backend.swift the Amplify.Auth plugin calls federateToIdentityPool:

func federateToIdentityPools(with tokenString: String) {
    guard
        let plugin = try? Amplify.Auth.getPlugin(for: "awsCognitoAuthPlugin") as? AWSCognitoAuthPlugin
    else { return }

    Task {
        do {
            let result = try await plugin.federateToIdentityPool(
                withProviderToken: tokenString,
                for: .apple
            )
            print("Successfully federated user to identity pool with result:", result)
        } catch {
            print("Failed to federate to identity pool with error:", error)
        }
    }
}

In this version of our code, we also have an auth listener that triggers UI updates based on session events. The auth listener in our Backend.swift file looks like this now:

public func listenAuthUpdate() async -> AsyncStream<AuthStatus> { 
    return AsyncStream { continuation in
        continuation.onTermination = { @Sendable status in
                   print("[BACKEND] streaming auth status terminated with status : \(status)")
        }

        // listen to auth events.
        // see https://github.com/aws-amplify/amplify-ios/blob/master/Amplify/Categories/Auth/Models/AuthEventName.swift
        let _  = Amplify.Hub.listen(to: .auth) { payload in            
            print(payload.eventName)
            switch payload.eventName {
            case "Auth.federatedToIdentityPool":
                print("User federated, update UI")
                continuation.yield(AuthStatus.signedIn)
                Task {
                    await self.updateUserData(withSignInStatus: true)
                }
            case "Auth.federationToIdentityPoolCleared":
                print("User unfederated, update UI")
                continuation.yield(AuthStatus.signedOut)
                Task {
                    await self.updateUserData(withSignInStatus: false)
                }
            case HubPayload.EventName.Auth.sessionExpired:
                print("Session expired, show sign in aui")
                continuation.yield(AuthStatus.sessionExpired)
                Task {
                    await self.updateUserData(withSignInStatus: false)
                }
            default:
                print("\(payload)")
                break
            }
        }
    }
}

For federation events, the events end up being Auth.federatedToIdentityPool and Auth.federationToIdentityPoolCleared.

Adding AWS Pinpoint for App Analytics

Next, we want to see what our users are doing in our Notes app and see if they're really making use of the "new" image upload feature.

First, we'll need to update amplify by running amplify add analytics:

amplify add analytics
? Select an Analytics provider Amazon Pinpoint
✔ Provide your pinpoint resource name: · amplifypushup

Then we'll run amplify push.

We'll need to add the Amplify Analytics libraries to our app by making sure the AWSPinpointAnalyticsPlugin is installed. And then we'll add the initialization in our Backend.swift file:

private init() {
  // initialize amplify
  do {
      try Amplify.add(plugin: AWSCognitoAuthPlugin())
      try Amplify.add(plugin: AWSAPIPlugin(modelRegistration: AmplifyModels()))
      try Amplify.add(plugin: AWSS3StoragePlugin())
      try Amplify.add(plugin: AWSPinpointAnalyticsPlugin()) // <-- add this
      try Amplify.configure()
      print("Initialized Amplify");
  } catch {
    print("Could not initialize Amplify: \(error)")
  }

If we want to track how popular our image upload feature is, we can add an event:

let properties: AnalyticsProperties = [
    "eventPropertyStringKey": "eventPropertyStringValue",
    "eventPropertyIntKey": 123,
    "eventPropertyDoubleKey": 12.34,
    "eventPropertyBoolKey": true
]

let event = BasicAnalyticsEvent(
    name: "imageUploaded",
    properties: properties
)

try Amplify.Analytics.record(event: event)

You can also emit events related to authentication that Pinpoint will automatically track:

• _userauth.sign_in

• _userauth.sign_up

• _userauth.auth_fail

Conclusion

In my quest to elevate the capabilities of AWS Amplify tutorials, I ventured into iOS development for the 2023 AWS Amplify + Hashnode Hackathon. Building upon the introductory iOS notes app tutorial provided by AWS Amplify, I took the app to the next level by incorporating sought-after features like "Sign in with Apple" and AWS Pinpoint Analytics.

The inclusion of "Sign in with Apple" as a federated login option significantly enhances the app's authentication experience. Users can now seamlessly log in with their Apple ID, ensuring both convenience and privacy. This advanced authentication option empowers users to stay in control of their personal information, including the ability to hide their email.

In addition to empowering users with a better way to sign-in, I integrated AWS Pinpoint Analytics into the notes app. By leveraging AWS Pinpoint's analytical capabilities, we gained valuable insights into user engagement and behavior. This data-driven approach allows us to analyze user interactions, monitor feature adoption, and make informed decisions to optimize the app's performance and user experience.

As you explore the code and implement these advanced features, I encourage you to further experiment and customize the app to suit your specific use case and preferences. Remember to refer to the code repository for detailed implementations.

Thank you for joining me on this amplified journey to enhance AWS tutorials. With "Sign in with Apple" and AWS Pinpoint Analytics at your fingertips, you're equipped to build robust and engaging mobile apps that connect users with seamless authentication and actionable insights.

Creating these diverse feeds, however, requires a Bluesky feed generator, a tool that necessitates some technical know-how. This is where AWS services come into the picture, simplifying the deployment and operation of a Bluesky feed generator.

Our proposed architecture leverages AWS Fargate, AWS Lambda, and Aurora Serverless. AWS Fargate runs a container designed to parse the Bluesky event stream, capturing only the relevant data. This data is then stored in an Aurora Serverless database. AWS Lambda is employed to deliver the stream, working in tandem with AWS Fargate to reduce operational overhead and allow us to focus on core functionality. This combination also serves as our "feed algorithm".

Deploying the Bluesky feed generator as an AWS CDK project not only streamlines the process of feed creation but also democratizes it, making it accessible to a wider audience. It facilitates easy scaling of resources, and efficient cost management, and enables us to focus more on developing unique, user-centric feeds rather than managing the underlying infrastructure.

In the upcoming sections of this blog post, we'll delve deeper into the technical aspects of deploying a Bluesky feed generator using AWS services. We'll provide a step-by-step guide to help you get started, including creating a feed of "skeets" from AWS Employees, AWS Heroes, and AWS Community Builders.

PSST... feel free to follow me on Bluesky too

The Bluesky-provided Feed Generator

Bluesky provides a basic feed generator on their GitHub, but it's a bit like having a camera without the right settings - it lacks the necessary architecture to capture the perfect shot.

At its heart, the service provided by Bluesky's repo performs three key functions:

1. It latches onto the Bluesky Websocket stream and filters events into a database. It's like adjusting the focus on your camera, ensuring only the relevant subjects are in clear view 1.

2. It features a feed endpoint that cherry-picks relevant rows from the database. Think of it as the photographer, selecting the best shots for the final album 2.

3. It includes a static endpoint that "registers" the feed. This is like the metadata of a photo, providing all the necessary information about the shot 3.

To register the feed service, a script is run that's a bit like the final editing process before the photos are published. It connects to Bluesky and registers the feed name and service URL, making sure everything is picture-perfect 4.

Now, here's where we bring in the big guns - AWS CDK. We're going to give this feed generator a major upgrade.

Firstly, we'll move the WebSocket stream connection to a Fargate service. This is akin to upgrading from a manual focus to an automatic one - it's faster, more efficient, and doesn't require as much manual effort.

Next, we'll transform the feed endpoint into an AWS Lambda function. This is like having an automated photo editor - it's more efficient, scalable, and doesn't require constant supervision.

The static endpoint will be relocated to a simple MockIntegration in the APIGateway. This is like moving your photo metadata management to a digital platform - it's more efficient, reliable, and easily accessible.

Lastly, we'll shift the feed registration script to be run by a CustomResource-invoked Lambda. This is like automating your final photo editing process - it's more reliable, efficient, and doesn't require constant attention.

In essence, we're taking the basic structure provided by Bluesky and supercharging it with the power of AWS CDK.

Crafting The Bluesky Database

First up on our agenda is the creation of the database that will serve as the meeting point for our parser and feed. This setup bears a resemblance to my recent post about Creating an Aurora MySQL Database and Setting Up a Kinesis CDC Stream with AWS CDK. However, there's a twist - we won't be needing the stream this time, and we'll be employing Aurora Serverless instead.

Our game plan involves crafting a CDK Construct that houses the database and a CustomResource-invoked Lambda that will lay the groundwork for the database schema. Let's dive into the creation of the database:

this.db = new ServerlessCluster(this, 'cluster', {
  clusterIdentifier: `bluesky`,
  credentials: Credentials.fromGeneratedSecret('admin'),
  defaultDatabaseName: dbName,
  engine: DatabaseClusterEngine.AURORA_MYSQL,
  removalPolicy: RemovalPolicy.DESTROY,
  enableDataApi: true,
  vpc,
});

Next, we'll whip up the lambda function and the custom resource:

const dbInitFn = new NodejsFunction(this, "dbInitFn", {
  functionName: "bluesky-db-init",
  entry: join(__dirname, "lambda/db-init.ts"),
  runtime: Runtime.NODEJS_18_X,
  timeout: Duration.minutes(15),
  tracing: Tracing.ACTIVE,
  environment: {
    DB_NAME: dbName,
    CLUSTER_ARN: this.db.clusterArn,
    SECRET_ARN: this.db.secret?.secretArn || '',
  },
});
this.db.grantDataApiAccess(dbInitFn);

const initProvider = new Provider(this, `init-db-provider`, {
  onEventHandler: dbInitFn,
});

new CustomResource(this, `init-db-resource`, {
  serviceToken: initProvider.serviceToken,
});

Our lambda's handler will be interacting with the database using the RDS Data API, taking advantage of the secrets that CloudFormation has created for the database:

if (event.RequestType === "Create") {
    await client.send(cmd(`select 1`));
    console.log("db-init: create tables");
    await client.send(
      cmd(
        `CREATE TABLE IF NOT EXISTS post (uri VARCHAR(255) NOT NULL, cid VARCHAR(255) NOT NULL, author VARCHAR(255) NOT NULL, replyParent VARCHAR(255), replyRoot VARCHAR(255), indexedAt DATETIME NOT NULL, PRIMARY KEY (uri));`
      )
    );
    console.log("created post table");
    await client.send(cmd(`CREATE TABLE IF NOT EXISTS sub_state (service VARCHAR(255) NOT NULL, cursor_value INT NOT NULL, PRIMARY KEY (service));`))
    console.log("created sub_state table");
}

Since we want the tables to be created only when the stack is first deployed, we filter on event.RequestType === "Create". However, to err on the side of caution, we've also included IF NOT EXISTS in the SQL commands. Better safe than sorry!

Constructing the Parser: Connecting to the Bluesky Event Stream

In the next phase of our process, we're going to construct the parser that connects to the Bluesky event stream. This component is the linchpin that connects us to the publicly available Bluesky web socket event stream. It's like the lens of our camera, capturing the events that we're interested in and focusing them into a coherent image.

Let's dive into the code and understand the key parts:

const cluster = new Cluster(this, "bluesky-feed-generator-cluster", {
  vpc,
  enableFargateCapacityProviders: true,
});

Here, we're setting up our Fargate cluster. This is akin to positioning our camera on a tripod, providing a stable platform for our operations.

const taskDefinition = new FargateTaskDefinition(
  this,
  "bluesky-feed-generator-task",
  {
    runtimePlatform: {
      cpuArchitecture: CpuArchitecture.ARM64,
    },
    memoryLimitMiB: 1024,
    cpu: 512,
  }
);
db.grantDataApiAccess(taskDefinition.taskRole);

Next, we're defining a Fargate task. This is like adjusting the camera's settings, such as aperture and shutter speed, to ensure we capture the best possible shot. We're also granting this task access to our database, much like giving our camera the ability to store the photos it captures.

const logging = new AwsLogDriver({
  logRetention: RetentionDays.ONE_DAY,
  streamPrefix: "bluesky-feed-generator",
});

We're setting up a log driver here, which is akin to the camera's viewfinder, allowing us to monitor and review our operations.

taskDefinition.addContainer("bluesky-feed-parser", {
  logging,
  image: ContainerImage.fromDockerImageAsset(
    new DockerImageAsset(this, "bluesky-feed-parser-img", {
      directory: join(__dirname, ".."),
      platform: Platform.LINUX_ARM64,
    })
  ),
  environment: {
    DB_NAME: dbName,
    CLUSTER_ARN: db.clusterArn,
    SECRET_ARN: db.secret?.secretArn || '',
  }
});

Here, we're adding a container to our task definition. This is like attaching a lens to our camera, defining what it will capture. We're also specifying the environment variables, which are akin to the camera's internal settings.

new FargateService(this, "bluesky-feed-generator", {
  cluster,
  taskDefinition,
  enableExecuteCommand: true,
  // fargate service needs to select subnets with the NAT in order to access AWS services
  vpcSubnets: {
    subnetType: SubnetType.PRIVATE_WITH_EGRESS,
  },
  securityGroups: [securityGroup]
});

Finally, we're creating a new Fargate service, which is like pressing the camera's shutter button, setting everything into motion. We're specifying that the service should be able to execute commands and that it should select subnets with NAT to access AWS services, ensuring our camera can communicate with the outside world.

The Service Code

The code for our Fargate Service is heavily inspired by the feed generator provided by Bluesky. However, we've stripped out all the unnecessary parts (like express, etc). All we need to do is connect to the WebSocket stream and save relevant events to the already-created database.

When we created the DockerImageAsset above, we pointed it to our root directory which contains a Dockerfile. This Dockerfile installs the dependencies and runs the app which creates a FirehoseSubscription.

for (const post of ops.posts.creates) {
  if (awsCommunityDids.includes(post.author)) {
    const user = awsCommunityDidsToKeys[post.author];
    console.log(`${user} posted ${post.record.text}`);
    postsToCreate.push({
      uri: post.uri,
      cid: post.cid,
      author: user,
      replyParent: post.record?.reply?.parent.uri ?? null,
      replyRoot: post.record?.reply?.root.uri ?? null,
      indexedAt: new Date().toISOString().slice(0, 19).replace("T", " "),
    });
  }
}

if (postsToCreate.length > 0) {
  console.log(JSON.stringify({ postsToCreate }));
  const insertSql = `INSERT INTO post (uri, cid, author, replyParent, replyRoot, indexedAt) VALUES ${postsToCreate
    .map(
      () => "(:uri, :cid, :author, :replyParent, :replyRoot, :indexedAt)"
    )
    .join(", ")} ON DUPLICATE KEY UPDATE uri = uri`;

  const insertParams = postsToCreate.flatMap((post) => [
    { name: "uri", value: { stringValue: post.uri } },
    { name: "cid", value: { stringValue: post.cid } },
    { name: "author", value: { stringValue: post.author } },
    {
      name: "replyParent",
      value: post.replyParent
        ? { stringValue: post.replyParent }
        : { isNull: true },
    },
    {
      name: "replyRoot",
      value: post.replyRoot
        ? { stringValue: post.replyRoot }
        : { isNull: true },
    },
    { name: "indexedAt", value: { stringValue: post.indexedAt } },
  ]);

  const insertCmd = cmd(insertSql, insertParams);
  await client.send(insertCmd);
  console.log(`Created ${postsToCreate.length} posts`);
}

This is akin to the post-processing phase in photography. We're selecting the best shots (or in this case, posts) based on our predefined criteria, and storing them in our database. We're also ensuring that we handle deletions appropriately, keeping our collection of shots up-to-date.

The Bluesky Feed

The final CDK Construct creates an API Gateway with two endpoints, and the CustomResource that registers the feed.

Creating the Feed

This section of the code is primarily concerned with setting up the API Gateway and the DNS records for the service. Here are the key parts:

const hostedzone = HostedZone.fromLookup(this, "hostedzone", {
  domainName: zoneDomain,
});
const certificate = new Certificate(this, "certificate", {
  domainName,
  validation: CertificateValidation.fromDns(hostedzone),
});

Here, we're looking up the hosted zone for our domain and creating a certificate for it. This is like setting up the address and credentials for our online photo gallery.

const api = new RestApi(this, "RestApi", {
  defaultMethodOptions: {
    methodResponses: [{ statusCode: "200" }],
  },
  deployOptions: {
    tracingEnabled: true,
    metricsEnabled: true,
    dataTraceEnabled: true,
  },
  endpointConfiguration: {
    types: [EndpointType.REGIONAL],
  },
});

Next, we're setting up the REST API. This is like setting up the interface for our online gallery, defining how users will interact with it.

Of course, let's not forget the MockIntegration:

const didIntegration = new MockIntegration(didOptions);
const didResource = api.root
  .addResource(".well-known")
  .addResource("did.json");
didResource.addMethod("GET", didIntegration, {
  methodResponses: [
    {
    statusCode: "200",
    },
  ],
});

Here, we're setting up a MockIntegration. In the context of AWS API Gateway, a MockIntegration is a type of integration that allows you to simulate API behavior without implementing any backend logic. It's like a placeholder or a dummy that returns pre-configured responses to requests.

In this case, we're using it to serve a static JSON response for the did.json endpoint under the .well-known path. This endpoint is typically used to provide a standard way to discover information about the domain, and in this case, it's providing information about the Bluesky feed generator service.

This is akin to having a static information page in our photo gallery that provides details about the gallery itself. It doesn't change or interact with the visitor but provides essential information for anyone who asks.

const feedFn = new NodejsFunction(this, "feed", {
  functionName: "bluesky-feed",
  entry: join(__dirname, "lambda/feed.ts"),
  runtime: Runtime.NODEJS_18_X,
  timeout: Duration.seconds(30),
  tracing: Tracing.ACTIVE,
  environment: {
    DB_NAME: dbName,
    CLUSTER_ARN: db.clusterArn,
    SECRET_ARN: db.secret?.secretArn || "",
  },
});
db.grantDataApiAccess(feedFn);

Here, we're defining a new Node.js function that will serve as the feed for our service. This is like setting up the mechanism that will display the photos in our gallery.

api.root
  .addResource("xrpc")
  .addResource("app.bsky.feed.getFeedSkeleton")
  .addMethod("GET", feedIntegration);

This is where we're defining the endpoint for our feed. This is like setting up the URL where users can view our photo gallery.

api.addDomainName(`Domain`, {
  domainName,
  certificate,
  securityPolicy: SecurityPolicy.TLS_1_2,
});
new ARecord(scope, `ARecord`, {
  zone: hostedzone,
  recordName: domainName,
  target: RecordTarget.fromAlias(new ApiGateway(api)),
});

Finally, we're associating our domain name with our API and creating an A record for it. This is like linking our online gallery to our chosen web address, making it accessible to the public.

Registering the feed

We create another lambda and custom resource with:

const publishSecret = Secret.fromSecretCompleteArn(this, "publish-secret", props.publishFeed.blueskySecretArn);
const publishFeedFn = new NodejsFunction(this, "publish-feed", {
  functionName: "bluesky-publish-feed",
  entry: join(__dirname, "lambda/publish-feed.ts"),
  runtime: Runtime.NODEJS_18_X,
  timeout: Duration.seconds(30),
  tracing: Tracing.ACTIVE,
  environment: {
    HANDLE: props.publishFeed.handle,
    SECRET_ARN: props.publishFeed.blueskySecretArn,
    FEEDGEN_HOSTNAME: domainName,
    FEEDS: JSON.stringify(props.publishFeed.feeds),
  },
});
publishSecret.grantRead(publishFeedFn);

const publishProvider = new Provider(this, `publish-feed-provider`, {
  onEventHandler: publishFeedFn,
});

new CustomResource(this, `publish-feed-resource`, {
  serviceToken: publishProvider.serviceToken,
  properties: {
    Version: `${Date.now()}`,
  },
});

Here we make sure that it has access to our Bluesky App password (created on our Bluesky account's settings page).

The handler code follows the same process as Bluesky's own publishFeedGen.ts script, except we get the password from the secrets manager first. This code runs with every deployment and supports registering multiple feeds pointing at the same lambda.

Tying It All Together: Deploying the Stack

Having created the individual constructs, we now need to assemble them into a cohesive stack. This is akin to putting together our camera, lens, and tripod into a complete photography setup.

const vpc = new Vpc(this, "vpc");
const securityGroup = new SecurityGroup(this, "security-group", {
  vpc,
  allowAllOutbound: true,
});

const dbName = 'bluesky';
const { db } = new BlueskyDb(this, 'bluesky-db', {
  dbName,
  vpc,
});

const domainName = 'martz.codes';

new BlueskyParser(this, 'bluesky-parser', {
  db,
  dbName,
  securityGroup,
  vpc,
});

new BlueskyFeed(this, 'bluesky-feed', {
  db,
  dbName,
  domainName,
  publishFeed,
});

In the code above, we're first setting up a Virtual Private Cloud (VPC) and a security group. This is like choosing a location for our photo shoot and setting up the necessary security measures.

Next, we're creating our Bluesky database within this VPC. This is akin to setting up our storage system for the photos we'll capture.

We then instantiate our Bluesky parser and feed constructs, passing in the necessary parameters such as the database, domain name, and security group. This is like setting up our camera and lens, ready to start capturing photos.

In the bin file, we include the required publishFeed properties:

publishFeed: {
    handle: 'martz.codes',
    blueskySecretArn: "arn:aws:secretsmanager:us-east-1:359317520455:secret:bluesky-rQXJxQ",
    feeds: {
      "aws-community": {
        displayName: "AWS Community",
        description: "This is a test feed served from an AWS Lambda. It is a list of AWS Employees, AWS Heroes and AWS Community Builders",
      }
    },
  },

This is like setting up the details of our photo gallery, including the name, description, and access credentials.

It's worth noting that this stack was built iteratively. While it should work as expected, there may be a missing dependency that could affect the deployment order for the CustomResources.

Once deployed, we can visit the feed's page on Bluesky: https://bsky.app/profile/did:plc:a62mzn6xxxxwktpdprw2lvnc/feed/aws-community

The URL structure is as follows: https://bsky.app/profile/<owner's DID>/feed/<short-name>. When this URL is loaded in Bluesky, the Bluesky service makes a call to the feed URL. The feed then replies with several DIDs of posts, which the Bluesky service hydrates. This is like visiting our online photo gallery, where the service fetches and displays the photos based on the visitor's request.

Conclusion

And there you have it! We've walked through the process of deploying a Bluesky feed generator using AWS services, specifically AWS CDK, Fargate, Lambda, and Aurora Serverless. This setup allows us to parse the Bluesky event stream, store relevant events in a database, and serve the feed using a serverless architecture. It's like setting up a fully automated photography studio that captures, stores, and displays photos based on specific criteria.

But this is just the beginning. There are countless ways you could expand on this setup to suit your specific needs or explore new possibilities. Here are a few ideas to get you started:

1. Customize Your Feed Algorithm: The feed algorithm we used in this example is relatively simple, focusing on posts from specific authors. You could expand on this by incorporating more complex criteria, such as keywords, hashtags, or even sentiment analysis. This would be like using advanced filters or editing techniques to select and enhance your photos.

2. Integrate with Other Services: You could integrate your feed generator with other AWS services or third-party APIs to add more functionality. For example, you could use AWS Comprehend to analyze the sentiment of posts, AWS Translate to support multiple languages, or AWS SNS to send notifications when new posts are added to the feed.

3. Create a User Interface: While Bluesky provides a platform to view the feeds, you could also create your own user interface to display the feeds in a unique way. This could be a web app, a mobile app, or even an Alexa skill. This would be like creating your own online gallery or photo app to showcase your photos.

4. Scale Up: Our setup is designed to be scalable, but you could take this further by implementing more advanced scaling strategies. For example, you could use AWS Auto Scaling to automatically adjust the capacity of your Fargate service based on demand, or AWS ElastiCache to improve the performance of your database.

5. Secure Your Setup: While we've implemented basic security measures, there's always more you can do to protect your data and your users. You could use AWS Shield for DDoS protection, AWS WAF for web application firewall, or AWS Macie to discover, classify, and protect sensitive data.

Remember, the sky's the limit when it comes to what you can achieve with AWS services and Bluesky. So don't be afraid to experiment, innovate, and create something truly unique.

Ephemeral stacks are temporary stacks in AWS that are designed to exist for a short period of time. This is particularly useful in development environments where you want to test something but don’t need the stack to be up indefinitely.

This article is a follow-up to two previous posts on the topic of ephemeral stacks:

• Say Goodbye to Your CDK Stacks: A Guide to Self-Destruction

• Blink and It's Gone: Embracing Ephemeral CDK Stacks for Efficient DevOps

The Value of Ephemeral Stacks

But why would you want to use ephemeral stacks?

1. Cost Savings 💰: By using resources only for the time needed, you can significantly reduce costs. You no longer have to worry about unused resources accumulating costs because the stacks self-terminate after the stipulated period.

2. Efficient Resource Allocation 🔄: In fast-paced development environments, resources are constantly being allocated and deallocated. Ephemeral stacks make this process more efficient, ensuring that resources are available when needed and are released when no longer in use.

3. Reduced Complexity 🧠: Keeping track of which resources are actively being used can be a complex task. By using ephemeral stacks, you know that any active resource is being used for a good reason. This reduces the complexity of managing your infrastructure.

4. Enhanced Security 🔒: Minimizing the lifespan of your stacks reduces the exposure window for potential security vulnerabilities. By limiting the duration a resource is up, you inherently limit the time it can be exploited.

5. Realistic Testing Environments 🧪: Ephemeral stacks are great for simulating production environments without the permanence. They allow you to conduct realistic tests and experiments, enabling you to glean insights and identify issues that might not be evident in traditional development environments.

6. Simplified Clean-Up 🧹: Forget the days of manually cleaning up resources post-testing. With the self-destruction aspect of ephemeral stacks, the clean-up is automatic. This not only saves time but also ensures that no remnants are left behind that can cause clutter or additional costs.

7. Easy Scalability for Temporary Needs ⚖️: Sometimes you need to scale resources quickly to meet a temporary need (e.g., a one-time data processing job). Ephemeral stacks allow for such scalability without the long-term commitment.

Armed with these benefits, it’s clear that ephemeral stacks are an incredibly powerful tool for optimizing AWS resource management, especially in development environments. Let's dive into how we can further improve the architecture by consolidating the destruction process.

Understanding the Key Components

We will go through the changes to the @aws-community/ephemeral npm library and demonstrate how it can be used.

The code for the @aws-community/ephemeral npm library is here: https://github.com/aws-community-projects/ephemeral

The example project that uses it is here: https://github.com/martzcodes/blog-ephemeral

The DestroyMe Stack and Construct

The DestroyMeConstruct uses the SelfDestructAspect from previous posts to ensure that all of the AWS Resources in the stack are set to a DESTROY retention policy. Additionally, it sets a STACK_LIFE tag on the stack, which indicates how long the stack should remain if there are no updates to it. This tag will be used by an external service to pick up and process the stack for destruction. Here’s the code snippet for this part:

Tags.of(Stack.of(this)).add('STACK_LIFE', duration.toSeconds().toString());
Aspects.of(Stack.of(this)).add(new SelfDestructAspect());

DestroyMeStack is a higher-level construct that simply includes DestroyMeConstruct, making it convenient to extend.

The Destroyer Stack

The DestroyerStack is the core of this enhancement. Instead of having each stack deploy a step function that will self-destroy, which could lead to conflicts or complications, we centralize the destruction process.

DestroyerStack uses AWS Service Events from CloudFormation to detect stacks that have the STACK_LIFE tag. Every time a CDK Stack deploys, it generates a CloudFormation Stack Status event. Using this event, we can fetch the stack details, including the tags, and determine if we should track the stack for deletion.

If the stack has the STACK_LIFE tag, we add an entry into a DynamoDB table with a TimeToLive (TTL) property. This TTL is the sum of the current time and the stack life duration. When DynamoDB removes the item due to TTL expiration, we trigger a Lambda function to delete the stack.

Here's how the table is created:

const tableName = 'destroyer';
const table = new Table(this, tableName, {
  tableName,
  partitionKey: {
    name: 'pk',
    type: AttributeType.STRING,
  },
  billingMode: BillingMode.PAY_PER_REQUEST,
  removalPolicy: RemovalPolicy.DESTROY,
  timeToLiveAttribute: 'ttl',
  stream: StreamViewType.NEW_AND_OLD_IMAGES,
});

In case the stack deletion fails, we can also track the DELETE FAILED status and send notifications to an SNS Topic for manual intervention.

const failTopic = new Topic(this, 'fail-topic');
new Rule(this, 'delete-failed-rule', {
  eventPattern: {
    source: ['aws.cloudformation'],
    detailType: ['CloudFormation Stack Status Change'],
    detail: {
      resourceStatus: ['DELETE_FAILED'],
    },
  },
  targets: [new SnsTopic(failTopic)],
});

CloudFormation Event Function

This Lambda function is triggered by AWS Service Events. It retrieves information from the CloudFormation service and writes to the DynamoDB table.

Here's how the Lambda function is configured:

const cloudformationFn = new NodejsFunction(this, 'fn-cloudformation', {
  runtime: Runtime.NODEJS_18_X,
  memorySize: 1024,
  timeout: Duration.minutes(5),
  entry: join(__dirname, local ? 'destroyer-stack.fn-cloudformation.ts' : 'destroyer-stack.fn-cloudformation.js'),
  initialPolicy: [
    new PolicyStatement({
      effect: Effect.ALLOW,
      actions: [
        'cloudformation:Describe*',
        'cloudformation:Get*',
        'cloudformation:List*',
      ],
      resources: ['*'],
    }),
  ],
});
table.grantReadWriteData(cloudformationFn);
cloudformationFn.addEnvironment('DESTROY_TABLE_NAME', table.tableName);

And then we trigger the lambda on those AWS Service Events:

new Rule(this, 'cloudformation-rule', {
  eventPattern: {
    source: ['aws.cloudformation'],
    detailType: ['CloudFormation Stack Status Change'],
  },
  targets: [new LambdaFunction(cloudformationFn)],
});

In our case we don't really care if the stack successfully deployed or not. We reset the ttl with every deployment (failure or not). Since a developer is actively working on the project, we don't want to delete it.

The lambda handler code simply describes the stack and if the STACK_LIFE tag exists, it puts the item into DynamoDB with the StackName as the primary key.

const StackName = event.detail['stack-id'];
const describeCommand = new DescribeStacksCommand({
StackName,
});
const stacks = await cf.send(describeCommand);
const stack = stacks.Stacks?.[0];
const stackLife = stack?.Tags?.find((tag) => tag.Key === 'STACK_LIFE')?.Value;
if (stackLife) {
    try {
      await ddbDocClient.send(
        new PutCommand({
          TableName: process.env.DESTROY_TABLE_NAME,
          Item: {
            pk: stack.StackName,
            ttl: Math.ceil(new Date().getTime() / 1000 + Number(stackLife)),
          },
        }),
      );
    } catch (e) {
      console.log(e);
    }
}

Destroy Function

The destroy function operates similarly. We make sure that it is triggered from the DynamoDB Stream and that it has access to cloudformation:DeleteStack.

destroyFn.addEventSource(
  new DynamoEventSource(table, {
    startingPosition: StartingPosition.LATEST,
  }),
);
destroyFn.addToRolePolicy(
  new PolicyStatement({
    actions: ['cloudformation:DeleteStack'],
    resources: ['*'],
    effect: Effect.ALLOW,
  }),
);

The destroy function handler code filters the dynamodb stream records to make sure that the item is being removed and the the ttl is actually expired. For safety there's an escape hatch that you can remove an item from DynamoDB if it's before the expiration, and it won't delete the stack.

const currentTimeInSeconds = new Date().getTime() / 1000;
if (item.ttl > currentTimeInSeconds) {
  // item was manually removed and not expired
  console.log('item was manually removed and not expired', currentTimeInSeconds, item.ttl);
  return [...p];
}

Then it removes the the valid expired stacks:

const client = new CloudFormationClient({});
await Promise.all(
  stacksToDestroy.map(
    async (stackName) =>
      await client.send(
        new DeleteStackCommand({
          StackName: stackName,
        }),
      ),
  ),
);

How to Use @aws-community/ephemeral

Example Code for this section is located here: https://github.com/martzcodes/blog-ephemeral

First, you need to deploy the DestroyerStack. You can do this in a separate project or by manually deploying the stack with npx cdk deply DestroyerStack

import { DestroyerStack } from '@aws-community/ephemeral';

const app = new cdk.App();
new DestroyerStack(app, 'DestroyerStack');

Once the DestroyerStack is in-place and monitoring the AWS Service Events, you can make any of your stacks ephemeral by extending the DestroyMeStack or adding the DestroyMeConstruct.

Here, we have extended the stack:

import { DestroyMeStack, DestroyMeStackProps } from '@aws-community/ephemeral';
import { Construct } from 'constructs';

export class BlogEphemeralStack extends DestroyMeStack {
  constructor(scope: Construct, id: string, props: DestroyMeStackProps) {
    super(scope, id, props);
    // your stuff here
  }
}

and then we can deploy it using npx cdk deploy EphemeralStack

new BlogEphemeralStack(app, 'EphemeralStack', {
  destroyMeEnable: true,
  destroyMeDuration: cdk.Duration.minutes(3),
});

It is important to note that the Dynamo TTL timing is NOT exact

TTL typically deletes expired items within a few days. Depending on the size and activity level of a table, the actual delete operation of an expired item can vary. Because TTL is meant to be a background process, the nature of the capacity used to expire and delete items via TTL is variable (but free of charge).

If you need to delete the stack sooner, you can manually do it or delete the item from DynamoDB after the ttl has expired.

Conclusion

In this blog post, we dove into enhancing our ephemeral stack architecture by centralizing the stack destruction process and employing a stack life tag with the MakeDestroyable aspect from the @aws-community/ephemeral npm library. This approach ensures that all of the AWS resources in the stack are set to a DESTROY retention and also sets a STACK_LIFE tag, indicating the lifetime of the stack in the absence of updates.

To summarize the key enhancements:

1. Centralized Destruction Service: The centralization of destruction using the DestroyerStack minimizes the risks of conflicts and complications, making it more efficient.

2. AWS Service Events: Utilizing AWS Service Events to detect stacks with the STACK_LIFE tag enables automation and efficiency in monitoring and managing the lifetime of resources.

3. Automated Cleanup: The architecture now has an automated cleanup mechanism, which will be triggered based on the STACK_LIFE tag, and if there's a failure in the cleanup process, you will be notified via SNS.

4. Enhanced Resource Management: With this setup, resources can be more efficiently managed, particularly during development stages where resource provisioning might be ephemeral.

This enhancement is particularly beneficial for DevOps environments, where teams frequently create and destroy resources for testing and development purposes. By automating the destruction of temporary resources, teams can ensure that only necessary resources are retained, leading to cost savings and more manageable infrastructure.

However, do remember that the timing for deletion with DynamoDB's TTL is not precise. If you require more exact timing for resource cleanup, additional manual steps may be necessary.

By integrating these enhancements into your ephemeral stack architecture, you’ll enable more streamlined, automated, and efficient resource management within your AWS environment.

This post builds upon the concepts introduced in How to Use BinLogs to Make an Aurora MySQL Event Stream. Instead of relying on a lambda to parse the BinLog periodically, we'll be leveraging the capabilities of AWS DMS. The future integration of Serverless DMS with CloudFormation promises to further enhance this system.

You can find the code for this project on GitHub.

To ensure clarity and organization, our project will be structured into two separate stacks: the DatabaseStack and the DMS Stack. The DatabaseStack includes the VPC, the Aurora MySQL database, and the lambdas responsible for initializing and seeding the database. The DMS Stack encompasses DMS, the CustomResources that manage the DMS Replication Task, and the target Kinesis Stream.

This division allows us to accommodate those who already have a VPC and Aurora MySQL database in place. If you fall into this category, you can easily integrate your existing resources into the DMS stack.

Step 1: Creating the VPC and Aurora MySQL Database

Our first step is to create the DatabaseStack, which involves setting up the Aurora MySQL database and the VPC. You can find the code for this step here.

When creating the VPC, it's crucial to ensure that it includes a NAT Gateway. This gateway allows instances in a private subnet to connect to the internet or other AWS Services while preventing inbound connections from the internet. This is essential because our resources within the VPC need to communicate with AWS Services. Fortunately, VPCs are equipped with a NAT Gateway by default.

const vpc = new Vpc(this, "vpc", {
  maxAzs: 2,
});

Next, we create the database cluster using the appropriate code.

const db = new DatabaseCluster(this, "db", {
  clusterIdentifier: `db`,
  credentials: Credentials.fromGeneratedSecret("admin"),
  defaultDatabaseName: dbName,
  engine: DatabaseClusterEngine.auroraMysql({
    version: AuroraMysqlEngineVersion.VER_3_03_0,
  }),
  iamAuthentication: true,
  instanceProps: {
    instanceType: InstanceType.of(InstanceClass.T4G, InstanceSize.MEDIUM),
    vpc,
    vpcSubnets: {
      onePerAz: true,
    },
  },
  removalPolicy: RemovalPolicy.DESTROY,
  parameters: {
    binlog_format: "ROW",
    log_bin_trust_function_creators: "1",
    // https://aws.amazon.com/blogs/database/introducing-amazon-aurora-mysql-enhanced-binary-log-binlog/
    aurora_enhanced_binlog: "1",
    binlog_backup: "0",
    binlog_replication_globaldb: "0"
  },
});
db.connections.allowDefaultPortInternally();

Now that the database is created, we need to initialize the schema. We achieve this by utilizing a CustomResource, which triggers actions during a CloudFormation stack deployment. In our case, we'll trigger a lambda function that connects to the database and creates a table. This CustomResource can also be used to create users or seed the database with data, but for now, we'll focus on creating an empty table.

The first step in this process is to create the lambda. We ensure that the lambda has access to the database's secret (which is automatically created) with db.secret?.grantRead(initFn). This secret contains the credentials that the lambda needs to connect to the database.

const initFn = new NodejsFunction(this, `db-init`, {
  ...lambdaProps,
  entry: join(__dirname, "lambda/db-init.ts"),
  environment: {
    SECRET_ARN: secret.secretArn,
    DB_NAME: dbName,
    TABLE_NAME: tableName,
  },
  vpc,
  vpcSubnets: {
    onePerAz: true,
  },
  securityGroups: db.connections.securityGroups,
});
db.secret?.grantRead(initFn);
initFn.node.addDependency(db);

The lambda handler code is responsible for creating the table, and we ensure that this action is carried out when the stack is created:

import type {
  CloudFormationCustomResourceEvent,
  CloudFormationCustomResourceFailedResponse,
  CloudFormationCustomResourceSuccessResponse,
} from 'aws-lambda';

import { getConnectionPool } from './utils/connection';

export const handler = async (
  event: CloudFormationCustomResourceEvent,
): Promise<CloudFormationCustomResourceSuccessResponse | CloudFormationCustomResourceFailedResponse> => {
  switch (event.RequestType) {
    case 'Create':
      try {
        const connection = await getConnectionPool();

        await connection.query(
          "CALL mysql.rds_set_configuration('binlog retention hours', 24);"
        );

        await connection.query(`DROP TABLE IF EXISTS ${process.env.DB_NAME}.${process.env.TABLE_NAME};`);
        await connection.query(`CREATE TABLE ${process.env.DB_NAME}.${process.env.TABLE_NAME} (id INT NOT NULL AUTO_INCREMENT, example VARCHAR(255) NOT NULL, PRIMARY KEY (id));`);

        return { ...event, PhysicalResourceId: `init-db`, Status: 'SUCCESS' };
      } catch (e) {
        console.error(`initialization failed!`, e);
        return { ...event, PhysicalResourceId: `init-db`, Reason: (e as Error).message, Status: 'FAILED' };
      }
    default:
      console.error('No op for', event.RequestType);
      return { ...event, PhysicalResourceId: 'init-db', Status: 'SUCCESS' };
  }
};

To ensure that the lambda is invoked as part of the stack deployment, we create a provider for the lambda and a CustomResource. The provider specifies the lambda to be invoked, and the CustomResource triggers the invocation when the stack is deployed. This ensures that the database initialization is fully integrated into the stack deployment process.

const initProvider = new Provider(this, `init-db-provider`, {
  onEventHandler: initFn,
});

new CustomResource(this, `init-db-resource`, {
  serviceToken: initProvider.serviceToken,
});

Step 2: Understanding CloudFormation and DMS Streams

DMS Change Data Capture replication relies on MySQL's Binlog. To enable DMS, binlog must be enabled in MySQL. When creating the database in the previous step, we included parameters that enable Aurora's enhanced binlog for improved performance. More information about this feature can be found here.

binlog_format: "ROW",
log_bin_trust_function_creators: "1",
aurora_enhanced_binlog: "1",
binlog_backup: "0",
binlog_replication_globaldb: "0"

Moving on, we can now create the stack that contains the DMS Replication Task and Kinesis stream. You can access the relevant code here.

First, we create the Kinesis stream that will serve as the target for the events.

const dbStream = new Stream(this, `db-stream`, {
  streamName: `db-stream`,
  streamMode: StreamMode.ON_DEMAND,
});

DMS requires a role called dms-vpc-role to function correctly, but it doesn't have the necessary permissions by default. Therefore, we need to manually create this role.

const dmsRole = new Role(this, `dms-role`, {
  roleName: `dms-vpc-role`, // need the name for this one
  assumedBy: new ServicePrincipal("dms.amazonaws.com"),
});
dmsRole.addManagedPolicy(
  ManagedPolicy.fromManagedPolicyArn(this, `AmazonDMSVPCManagementRole`, `arn:aws:iam::aws:policy/service-role/AmazonDMSVPCManagementRole`)
);

Next, we create the Replication Subnet Group that DMS will use to connect to the database. Since it would attempt to create the dms-vpc-role with the wrong permissions, we need to ensure that it uses the existing role we created. This requires adding a dependency between the two resources.

const dmsSubnet = new CfnReplicationSubnetGroup(this, `dms-subnet`, {
  replicationSubnetGroupDescription: "DMS Subnet",
  subnetIds: vpc.selectSubnets({
    onePerAz: true,
  }).subnetIds,
});
dmsSubnet.node.addDependency(dmsRole);

Now we can create the replication instance itself, utilizing the subnet we just created. For simplicity, we are using the smallest instance class, although ideally, we would support using Serverless DMS. Unfortunately, CloudFormation does not yet provide support for this.

const dmsRep = new CfnReplicationInstance(this, `dms-replication`, {
  replicationInstanceClass: "dms.t2.micro",
  multiAz: false,
  publiclyAccessible: false,
  replicationSubnetGroupIdentifier: dmsSubnet.ref,
  vpcSecurityGroupIds: securityGroups.map(
    (sg) => sg.securityGroupId
  ),
});

To enable DMS to connect to Aurora, we need to grant it access to the role created by the database. We accomplish this by manually creating a Role, granting it permission to read the database's secret, and providing it as the source endpoint for DMS.

const dmsSecretRole = new Role(this, `dms-secret-role`, {
  assumedBy: new ServicePrincipal(
    `dms.${Stack.of(this).region}.amazonaws.com`
  ),
});
secret.grantRead(dmsSecretRole);

const source = new CfnEndpoint(this, `dms-source-endpoint`, {
  endpointType: "source",
  engineName: "aurora",
  mySqlSettings: {
    secretsManagerAccessRoleArn: dmsSecretRole.roleArn,
    secretsManagerSecretId: secret.secretName,
  },
});

Since our target is Kinesis, we also need to create a "target" endpoint and assign a role that has access to put records on the Kinesis stream.

const streamWriterRole = new Role(this, `dms-stream-role`, {
  assumedBy: new ServicePrincipal(
    `dms.${Stack.of(this).region}.amazonaws.com`
  ),
});

streamWriterRole.addToPolicy(
  new PolicyStatement({
    resources: [dbStream.streamArn],
    actions: [
      "kinesis:DescribeStream",
      "kinesis:PutRecord",
      "kinesis:PutRecords",
    ],
  })
);

const target = new CfnEndpoint(this, `dms-target-endpoint`, {
  endpointType: "target",
  engineName: "kinesis",
  kinesisSettings: {
    messageFormat: "JSON",
    streamArn: dbStream.streamArn,
    serviceAccessRoleArn: streamWriterRole.roleArn,
  },
});

Finally, we create the replication task itself. We provide a generic table mapping that emits events for changes to any table. It's worth noting that wildcards can be used to restrict the mapping to specific tables if desired. For more information about table mappings take a look at the docs

const dmsTableMappings = {
  rules: [
    {
      "rule-type": "selection",
      "rule-id": "1",
      "rule-name": "1",
      "object-locator": {
        "schema-name": dbName,
        "table-name": "%",
        "table-type": "table",
      },
      "rule-action": "include",
      filters: [],
    },
  ],
};
const task = new CfnReplicationTask(this, `dms-stream-rep`, {
  replicationInstanceArn: dmsRep.ref,
  migrationType: "cdc",
  sourceEndpointArn: source.ref,
  targetEndpointArn: target.ref,
  tableMappings: JSON.stringify(dmsTableMappings),
  replicationTaskSettings: JSON.stringify({
    BeforeImageSettings: {
      EnableBeforeImage: true,
      FieldName: "before",
      ColumnFilter: "all",
    }
  }),
});

Additionally, we provide BeforeImageSettings in the replicationTaskSettings, which enables us to include a before image for row updates, allowing us to infer deltas on table row updates. For Change Data Capture, we are using the migration type cdc since we are not migrating existing data.

Important Considerations for CloudFormation and DMS

CloudFormation can NOT update DMS tasks that are actively running. It also does not automatically start a DMS Replication Task as part of the deployment. In order to get around this we'll set up some CustomResources and enforce their order that one will run before DMS changes, and one will run after.

The "pre" lambda will check if there are changes to DMS in the CloudFormation change set, if there are it will check if DMS is running, and if it is it will stop the task. It will also wait for the task to finish stopping before responding.

const StackName = `${process.env.STACK_NAME}`;
if (!ReplicationTaskArn) {
  ReplicationTaskArn = await getDmsTask({ cf, StackName });
}
const status = await getDmsStatus({ dms, ReplicationTaskArn });
if (status === 'running') {
  if (event.RequestType === 'Delete' || await hasDmsChanges({ cf, StackName })) {
    // pause task
    const stopCmd = new StopReplicationTaskCommand({
      ReplicationTaskArn,
    });
    await dms.send(stopCmd);
    // wait for task to be fully paused
    await waitForDmsStatus({ dms, ReplicationTaskArn, targetStatus: 'stopped' });
  }
}

On the other end, the "post" lambda will do the opposite. It will start (or resume) the DMS replication and wait for it to finish spinning up.

  const startCmd = new StartReplicationTaskCommand({
    ReplicationTaskArn,
    StartReplicationTaskType: "resume-processing",
  });
  await dms.send(startCmd);
  await waitForDmsStatus({
    dms,
    ReplicationTaskArn,
    targetStatus: "running",
  });

Additionally, we set up a lambda function that is invoked by the Kinesis stream. We add the stream as an event source for this lambda function.

const kinesisFn = new NodejsFunction(this, `stream-kinesis`, {
  ...lambdaProps,
  entry: join(__dirname, "lambda/stream-subscriber.ts"),
  tracing: Tracing.ACTIVE,
});

kinesisFn.addEventSource(
  new KinesisEventSource(dbStream, {
    batchSize: 100, // default
    startingPosition: StartingPosition.LATEST,
    filters: [
      { pattern: JSON.stringify({ partitionKey: [`${dbName}.${tableName}`] }) },
    ],
  })
);

Step 3: Testing the Event Stream

With both stacks deployed, we can now test DMS. To facilitate this process, a lambda function has been created, and its code can be accessed here. You can invoke this function using test events via the AWS console.

By logging in to the DMS console, we can observe that the replication task is already running, thanks to the CustomResources.

To view the table statistics for the task, we can see that our schema has been identified. If there were additional tables, they would also be listed here, but in our case, we only have the examples table.

Invoking our seed lambda function will insert a row into the table. After a short time, the table statistics page will reflect the insert operation.

If we invoke a lambda from the kinesis stream, it will get the following event:

{
    "record": {
        "kinesis": {
            "kinesisSchemaVersion": "1.0",
            "partitionKey": "blog.examples",
            "sequenceNumber": "49642050638404656466522708801490648817992453925189451794",
            "data": "ewoJImRhdGEiOgl7CgkJImlkIjoJMSwKCQkiZXhhbXBsZSI6CSJoZWxsbyA2MzkiCgl9LAoJIm1ldGFkYXRhIjoJewoJCSJ0aW1lc3RhbXAiOgkiMjAyMy0wNi0yNVQxNToyMToxNC4wNTUxMzdaIiwKCQkicmVjb3JkLXR5cGUiOgkiZGF0YSIsCgkJIm9wZXJhdGlvbiI6CSJpbnNlcnQiLAoJCSJwYXJ0aXRpb24ta2V5LXR5cGUiOgkic2NoZW1hLXRhYmxlIiwKCQkic2NoZW1hLW5hbWUiOgkiYmxvZyIsCgkJInRhYmxlLW5hbWUiOgkiZXhhbXBsZXMiLAoJCSJ0cmFuc2FjdGlvbi1pZCI6CTEyODg0OTAyNjA5Cgl9Cn0=",
            "approximateArrivalTimestamp": 1687706474.102
        },
        "eventSource": "aws:kinesis",
        "eventVersion": "1.0",
        "eventID": "shardId-000000000001:49642050638404656466522708801490648817992453925189451794",
        "eventName": "aws:kinesis:record",
        "invokeIdentityArn": "arn:aws:iam::359317520455:role/BlogDmsStreamStack-streamkinesisServiceRole6A79529-7U8Q9JUVULLO",
        "awsRegion": "us-east-1",
        "eventSourceARN": "arn:aws:kinesis:us-east-1:359317520455:stream/db-stream"
    }
}

It's worth noting that Kinesis Base64 encodes the data, so decoding is necessary to make it usable. By decoding the event's data we get the following:

{
    "data": {
        "id": 1,
        "example": "hello 639"
    },
    "metadata": {
        "timestamp": "2023-06-25T15:21:14.055137Z",
        "record-type": "data",
        "operation": "insert",
        "partition-key-type": "schema-table",
        "schema-name": "blog",
        "table-name": "examples",
        "transaction-id": 12884902609
    }
}

By examining the decoded event, we can determine that it was an "insert" operation, and the "data" field contains the full row. In this case, since it was an insert, there is no "before" image.

If a row is updated, the event will contain an image representing the changes.

{
    "parsed": {
        "data": {
            "id": 1,
            "example": "hello 297"
        },
        "before": {
            "id": 1,
            "example": "hello 639"
        },
        "metadata": {
            "timestamp": "2023-06-25T15:50:51.449661Z",
            "record-type": "data",
            "operation": "update",
            "partition-key-type": "schema-table",
            "schema-name": "blog",
            "table-name": "examples",
            "transaction-id": 12884903827
        }
    }
}

From there you could calculate a diff to see that the example column went from hello 639 to hello 297.

Conclusion

In conclusion, this comprehensive guide has provided you with the knowledge and steps necessary to create an Aurora MySQL Database, initialize it using Custom Resources, and set up a Change Data Capture (CDC) Stream with AWS CDK, AWS Data Migration Service (DMS), and Kinesis. By leveraging AWS DMS and event-driven architecture principles, you can unlock the full potential of real-time data replication and event streaming.

As you move forward, there are several ways you can expand on the concepts and ideas covered in this guide. Here are a few suggestions:

1. Explore Advanced CDC Stream Configurations: Dive deeper into the configuration options available with DMS CDC streams. Experiment with table mappings, filtering options, and advanced settings to tailor the replication process to your specific use cases.

2. Integrate Additional AWS Services: Consider integrating other AWS services into your event-driven architecture. For example, you could explore using AWS Lambda to process the replicated events, Amazon S3 for data storage, or AWS Glue for data cataloging and ETL operations.

3. Implement Event-Driven Microservices: Build event-driven microservices that consume the CDC stream events to trigger actions or updates across different systems. Explore how you can use services like AWS Step Functions or AWS EventBridge to orchestrate complex workflows based on the captured events.

4. Scale and Optimize: Experiment with scaling and optimizing your CDC stream setup. Explore strategies for handling high-velocity data streams, optimizing performance, and implementing fault-tolerant architectures.

5. Monitor and Analyze: Set up monitoring and analytics solutions to gain insights into your event-driven system. Utilize services like Amazon CloudWatch, AWS X-Ray, or AWS AppSync to track and analyze the performance, reliability, and usage patterns of your CDC stream and associated components.

By expanding on the ideas presented in this guide, you can harness the full potential of a Change Data Capture stream in an event-driven architecture. This approach allows you to build scalable, real-time systems that react to changes in your data and drive intelligent decision-making in your applications. The possibilities for innovation and optimization are vast, so take this foundation and continue exploring the exciting world of event-driven architectures.

This post serves as a supplement to my talk (spoilers), providing more insights into ephemeral CDK Stacks, their implementation, and best practices. If you're exploring the topic or haven't yet decided on attending the AWS Summit, I hope this post sparks your interest and encourages you to join me for an engaging discussion. See you there! 🎉

In the fast-paced DevOps world, managing and cleaning up temporary cloud resources can be challenging. Forgotten stacks from testing or PoC stages lead to resource wastage and inflated cloud bills. To address this, ephemeral AWS CDK Stacks are a game changer.

This blog post explores integrating ephemeral CDK Stacks into CI/CD pipelines and development workflows. Powered by Self-Destructing Constructs and CDK Aspects, they automate the cleanup process, ensuring a tidy AWS environment and resource savings.

Ephemeral stacks are a simple yet powerful addition to your DevOps toolkit, streamlining resource management and reducing costs. Let's dive into leveraging ephemeral stacks, saving you from manual cleanup and the perils of forgotten stacks.

Self-Destructing Construct and CDK Aspects

Ephemeral CDK Stacks are made possible by combining the concepts from two of my previous posts on Self-Destructing Constructs and CDK Aspects.

Self-Destructing Constructs: As discussed in my previous post, Say Goodbye to Your CDK Stacks: A Guide to Self-Destruction, these are unique AWS CDK constructs that employ Step Functions to automatically delete the stack after a defined duration. This construct helps ensure that unnecessary resources aren't lingering around, reducing costs and freeing up space in your AWS account.

CDK Aspects: My last post, Breaking Bad Practices with CDK Aspects, explained how CDK Aspects allow for implementing cross-cutting concerns across your stack, such as applying an aggressive removal policy to all resources. When paired with self-destructing stacks, this ensures a clean slate post-destruction, leaving no stray resources behind.

Integrating Ephemeral Stacks with CI/CD Pipelines

CI/CD pipelines, a cornerstone of modern DevOps practices, can greatly benefit from ephemeral CDK Stacks. In a typical CI/CD pipeline, each code commit triggers a process that includes building, testing, and deploying the application. This often involves deploying a stack and, once the tests are executed, cleaning up the resources.

However, sometimes stacks are left behind due to failures in the pipeline or prematurely terminated tests. These forgotten stacks lead to unnecessary costs and clutter. Ephemeral stacks resolve this by auto-deleting after a specific duration, ensuring a clean AWS environment and reducing the wastage of resources.

To illustrate this, let's compare the sequence of events in a typical CI/CD pipeline and one enhanced with ephemeral stacks.

Typical CI/CD pipeline:

In the traditional CI/CD pipeline, the stack is explicitly deleted as part of the pipeline, which could fail or be skipped, resulting in lingering stacks.

CI/CD pipeline with Ephemeral Stacks:

With ephemeral stacks, the stack deletion is automated and independent of the CI/CD pipeline, ensuring that stacks are always cleaned up.

This approach reduces the complexity of your CI/CD pipeline and ensures cleaner resource management in your AWS environment. Additionally, you can set up an aggressive removal policy using CDK Aspects for a more comprehensive cleanup of all resources associated with the stack.

It doesn't sound like a lot, but in practice, it can save hours (over the course of weeks) while ensuring AWS resources (and money) aren't being wasted. 🚀

Implementing Ephemeral Stacks in Development Workflows

Developers often deploy proof of concept (PoC) stacks to test new features, validate concepts, or debug issues. These stacks provide a sandboxed environment to experiment without affecting the production infrastructure. However, once the purpose is served, these stacks often become "forgotten" entities in the cloud environment. They no longer serve a meaningful purpose and, over time, contribute to clutter and unnecessary costs. Implementing ephemeral stacks in development workflows can be an excellent solution to this common oversight.

The Problem of Forgotten Stacks

In the energetic world of development work, the adrenaline rush of solving complex problems or moving onto the next exhilarating task can often eclipse the essential, albeit less exciting, cleanup step. This oversight, often spurred by the thrill of innovation 💡 or the satisfaction of squashing a bug 🐛, can lead to the pile-up of forgotten temporary stacks. Over time, these lingering stacks clutter your environment and inflate your cloud bill 💸.

Consider some common scenarios where these temporary stacks crop up:

• Proof of Concept Testing: During the innovation process, PoC stacks are often crafted to assess feasibility or demonstrate the practicality of a concept. Upon approval or rejection of the concept, these stacks fulfill their purpose and, ideally, should vanish 🗑️.

• Feature Testing: To ensure isolated and accurate testing, developers frequently deploy separate stacks for new features. Once validated and merged into the primary codebase, these stacks have served their purpose and become redundant 🔄.

• Debugging: In the process of troubleshooting intricate issues, developers may create replica stacks to isolate and understand the problem. After resolving the issue, these stacks lose their relevance and ought to be removed 🔎.

Let me illustrate this with a personal anecdote to drive the point home:

In my role, I once assumed the responsibility of cleaning up unused stacks in our development accounts. On this particular day, I found myself navigating through and deleting over 200 of these forgotten stacks. As a serverless-first shop, this exercise didn't incur exorbitant costs, but it did consume resources and impacted our resource quotas ⏳. This served as a stark reminder of the importance of efficient stack management, and how easily these forgotten stacks can clutter our environment 🧹.

Ephemeral stacks can be the silver bullet to these common oversights, introducing an automated cleanup process to ensure your development environment stays neat, tidy and cost-effective 💰.

The Ephemeral Stack Solution

Ephemeral stacks can serve as a fail-safe, ensuring that even if a developer forgets to delete a stack, it won't linger indefinitely. By setting a self-destruct timer at the time of stack creation, developers can rest easy knowing the stack will automatically clean itself up after a specific period.

Here's how this can be integrated into the common scenarios mentioned above:

• Proof of Concept Testing: Set the stack to auto-delete after the meeting where the PoC will be demonstrated. This way, if the concept is rejected, the stack is automatically cleaned up. If the concept is approved, the stack can be manually preserved or redeployed as a more permanent resource.

• Feature Testing: Set a short lifespan for the stack — perhaps a few hours or a day — to allow for feature validation. After that period, the stack, if not needed, will self-destruct.

• Debugging: Given the unpredictable nature of debugging, a slightly longer lifespan could be set. Once the issue is resolved, if the developer forgets to clean up, the stack will still auto-delete after the set duration.

Integrating ephemeral stacks into your development workflow can lead to more efficient resource utilization, a cleaner cloud environment, and cost savings. It's a practical way to safeguard against human error without adding an extra burden on the developers.

Best Practices and Practical Insights

Incorporating ephemeral stacks into your development workflow requires careful planning and adherence to best practices. Here are some insights to help you avoid common pitfalls and get the most out of self-destructing stacks.

Set a Reasonable Lifespan

While the primary goal is to prevent forgotten stacks from lingering, setting the self-destruction timer too short might interrupt development or testing work. Consider the typical time required for the task at hand and add some buffer to determine the optimal lifespan.

Integrate with CI/CD Pipeline

To get the most out of ephemeral stacks, they should be integrated with your CI/CD pipeline. This ensures that the stacks are created as part of your automated testing and deployment process and that they clean up after themselves when no longer needed.

Manage Permissions Carefully

Remember, self-destructing stacks need permission to delete resources. Ensure that these permissions are granted judiciously, keeping in line with the principle of least privilege. Be especially cautious when dealing with production environments.

Clearly Label Ephemeral Stacks

To avoid confusion and potential mishaps, it's important to clearly label ephemeral stacks. This could be through a naming convention or tagging. Clear labels also make it easier to locate and manage these stacks in the AWS Management Console.

Employ a Monitoring System

While the self-destruction mechanism should work reliably, it's a good idea to have a monitoring system in place. This will alert you to any stacks that didn't delete as expected or if there are any issues with the self-destruction process.

Consider Exceptions

Not every stack should be ephemeral. Some stacks may need to persist for longer periods for certain tasks, like long-running data processing or scenarios where manual intervention is required. Ensure that your system allows for such exceptions.

Educate Your Team

Finally, make sure that your development team is fully aware of the ephemeral stack concept and its implications. They should understand the purpose, the lifespan of these stacks, and what they can expect during the self-destruction process.

Implementing ephemeral stacks requires more than just technical setup — it's a shift in the development mindset. With proper planning and adherence to these best practices, you can seamlessly integrate ephemeral stacks into your development workflow, leading to more efficient resource utilization and cost savings.

Conclusion

In the realm of DevOps, resource management efficiency is critical, and ephemeral CDK Stacks provide a practical, automated solution to a common problem. By incorporating these self-destructing constructs into your CI/CD pipelines and development workflows, you can ensure the timely cleanup of temporary stacks, reducing resource wastage and maintaining a cleaner AWS environment.

Ephemeral stacks offer a powerful combination of convenience and economy, alleviating developers from the burden of manual cleanup and saving valuable time and costs. They also guard against human error and oversight, a common cause of lingering, unnecessary cloud resources.

Remember, the effective implementation of ephemeral stacks involves more than just technical setup. It also requires a shift in mindset, careful planning, and adherence to best practices. But with these in place, ephemeral stacks can become a vital part of your DevOps toolkit, promoting efficiency, tidiness, and cost-effectiveness in your cloud journey.

This article aims to dissect the intricacies of CDK Aspects, shedding light on their fundamental purpose, operation, and advanced use cases. Aspects, in many ways, embody the balance between consistency and flexibility in cloud infrastructure development, a concept that's growing increasingly important in this era of complex, scalable applications.

Whether you are a seasoned AWS CDK user or a newcomer looking to expand your cloud development toolkit, this deep dive into CDK Aspects will provide valuable insights into this powerful feature. As we peel back the layers, you'll discover how Aspects can enhance resource management, improve security protocols, and promote code efficiency. Let's 'cook' up some knowledge on CDK Aspects.

The example code for this repository is located here: https://github.com/martzcodes/blog-aspects

This article by @JannikWempe is another great resource: https://aws.hashnode.com/the-power-of-aws-cdk-aspects

Understanding CDK Aspects - The Basics

At its core, the AWS Cloud Development Kit (CDK) is a software development framework that allows developers to define cloud infrastructure in code. This is where CDK Aspects come into play. Aspects are a feature within CDK that act like intelligent filters, traversing your code, identifying specific constructs, and applying modifications to them.

Consider this simple TypeScript code snippet of an AWS S3 bucket defined using CDK:

import { Stack, StackProps } from 'aws-cdk-lib';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { Construct } from 'constructs';

export class MyCDKStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    new Bucket(this, 'MyBucket', {
      versioned: false,
    });
  }
}

Now, let's say you want to ensure that every S3 bucket in your CDK application has versioning enabled. With CDK Aspects you can do this in two ways.

1. Validation - Create a CDK Aspect to validate all Buckets in the Stack and throw an Error if it is not set

2. Modification - Create a CDK Aspect to automatically set the versioned property on all buckets

In this context, your CDK code is like a complex chemistry experiment that needs careful management to prevent unwanted reactions.

1. Validation (the Chemistry Professor): In the validation role, the Chemistry Professor is like a vigilant observer, watching you perform your experiment. If they notice you're about to mix incompatible substances or your measurements are incorrect, they intervene immediately (throw an error) to prevent a potential disaster and ensure the effectiveness of your experiment.

2. Modification (the Chemistry Assistant): In the lab, an Assistant stands by to help with the experiment, offering a slight adjustment to ensure the reactions go as planned. They don't conduct the experiment for you but provide just enough assistance to keep you on track. This is akin to the modification role of Aspects, which scan your constructs and make slight but important tweaks to ensure consistency and conformity to standards.

To draw the parallel back to our CDK Aspects, the Aspect could, like a Chemistry Assistant, automatically adjust certain aspects of your resources (like enabling versioning for all S3 buckets), ensuring your infrastructure maintains the proper 'formula' throughout its configuration.

Here's how you might define that Aspect:

// For validation
export class ValidateVersioningAspect implements IAspect {
  public visit(node: IConstruct): void {
    if (node instanceof CfnBucket) {
      if (!node.versioningConfiguration
        || (!Tokenization.isResolvable(node.versioningConfiguration)
            && node.versioningConfiguration.status !== 'Enabled')) {
              Annotations.of(node).addError('Bucket versioning is not enabled');
      }
    }
  }
}

const app = new App();
const stack = new MyCDKStack(app, 'MyStack');
Aspects.of(stack).add(new ValidateVersioningAspect());

In this code, the ValidateVersioningAspect Aspect will add an error Annotation if it finds an S3 bucket with versioning disabled, ensuring that all S3 buckets comply with the requirement for versioning to be enabled. On synth, the error would look like this:

[Error at /MyTestStack/MyBucket/Resource] Bucket versioning is not enabled

Found errors

Annotations can also be tested and have support from CDK's built-in assertions: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.assertions.Annotations.html

// For modification
export class EnableVersioningAspect implements IAspect {
  public visit(node: IConstruct): void {
    if (node instanceof CfnBucket) {
      if (!node.versioningConfiguration
        || (!Tokenization.isResolvable(node.versioningConfiguration)
            && node.versioningConfiguration.status !== 'Enabled')) {
              node.versioningConfiguration = {
                status: 'Enabled'
              };
      }
    }
  }
}

const app = new App();
const stack = new MyCDKStack(app, 'MyStack');
Aspects.of(stack).add(new EnableVersioningAspect());

In this code, the EnableVersioningAspect class defines an Aspect that will "visit" every construct in the stack. If the construct is an instance of the Bucket class, the Aspect will set its versioned property to true, effectively enabling versioning for every bucket in the stack.

Deep Dive into CDK Aspects

In this section, we'll delve deeper into the inner workings of CDK Aspects and uncover the magic behind the scenes. We'll explain the crucial role of the visit method, discuss the Aspects.of(scope).add(aspect) pattern, and illustrate how Aspects interact with the CDK's synthesis process. When using Aspects, it's crucial to be aware of how the AWS CDK uses tokenization to manage resources' properties.

The visit Method

At the heart of any CDK Aspect is the visit method. As an implementer of the IAspect interface, this method is called when the Aspect is applied to a construct. The visit method takes a single argument—IConstruct—which is the construct the Aspect is visiting. What you do inside this method is the meat of your Aspect: whether you choose to throw an error for validation, or modify a property of the construct.

The Intricacies of Tokenization in AWS CDK

Tokens are placeholders used by the CDK to represent values that are not known until deployment time. For example, if you create an S3 bucket without specifying a bucket name, CDK generates a unique name and represents it with a token in your code.

When you inspect the bucketName property during the visit method, you might expect to see an actual bucket name. However, you'll instead see a token, something like ${Token[TOKEN.12]}.

The tokenization system can lead to unexpected results when using Aspects. For instance, if you attempt to modify a property that uses a token, your Aspect might not behave as expected. This is because tokens aren't resolved until the CDK synthesizes your app into a CloudFormation template.

Here's an example:

export class TokenAwareAspect implements IAspect {
  visit(node: IConstruct): void {
    if (node instanceof Bucket) {
      console.log(`Bucket name is ${node.bucketName}`);
    }
  }
}

const app = new App();
const stack = new Stack(app, 'MyStack');
new Bucket(stack, 'MyBucket');
Aspects.of(stack).add(new TokenAwareAspect());

In the console output, you'll see a token as the bucket name, not a real bucket name. Keep this in mind when designing your Aspects!

Applying Aspects

The Aspects.of(scope).add(aspect) pattern is the standard way to apply an Aspect to a construct. In this pattern, Aspects.of(scope) returns an Aspects object associated with a construct, and add(aspect) adds an Aspect to this object. The scope here could be any construct to which you want to apply the Aspect—typically an instance of a Stack or an App.

Aspects and the Synthesis Process

CDK Aspects play a crucial role during the CDK's synthesis process. The synthesis process is a multi-stage operation where CDK translates your code into a CloudFormation template, which AWS can understand. During this process, Aspects are invoked after the construct tree has been fully initialized, but before synthesis. This allows Aspects to validate or modify constructs right before the CloudFormation templates are generated.

Just as Skyler White had to understand the sequence of money laundering, let's delve deeper into the sequence of CDK Aspects with a diagram.

sequenceDiagram
    participant User
    participant CDK App
    participant Aspect
    participant CloudFormation
    User->>CDK App: Runs CDK Synth
    CDK App->>CDK App: Initializes Construct Tree
    CDK App->>Aspect: Invokes Aspects
    Aspect-->>CDK App: Validates/Modifies Constructs
    CDK App->>CloudFormation: Generates CloudFormation Template

CDK Aspects in Action: An Architecture Diagram Generator

In this section, we'll explore a concrete example of using CDK Aspects in a real-world scenario. We'll delve into the internals of a recently published npm library, @aws-community/arch-dia, which uses a CDK Aspect to generate a pseudo-architecture diagram of a project. Not only does it visualize your AWS infrastructure, but it also tracks changes between synthesis stages, providing a visual diff.

The Architecture Diagram Aspect

The key component in @aws-community/arch-dia is the ArchitectureDiagramAspect, an implementation of the IAspect interface. This Aspect traverses the constructs in a given Stack to generate a Mermaid diagram representing the architecture of your AWS resources. Here's an overview of the code:

export class ArchitectureDiagramAspect implements IAspect {
  private readonly mermaidDiagram: string[];
  private stackName = '';

  constructor () {
    this.mermaidDiagram = [];
  }

  visit (node: IConstruct): void {
    if (node instanceof Stack) {
      this.stackName = node.stackName;
      this.traverseConstruct(node, '');
    }
  }
  ...
}

This Aspect, like all Aspects, has a visit method. It checks if the visited construct is an instance of the Stack class. If it is, it initiates a traversal of the constructs in that stack.

The traverseConstruct method iteratively visits all children of a given construct, building up a Mermaid diagram string in the process:

private traverseConstruct (construct: IConstruct, parentPath: string): void {
  ...
  construct.node.children.forEach((child) => {
    this.traverseConstruct(child, currentPath);
  });
}

Generating and Comparing Diagrams

Once all constructs have been visited, the Aspect can generate a Mermaid diagram of the entire Stack using the generateDiagram method. This method also handles comparing the newly generated diagram with the previous one, if it exists, to create a visual diff:

generateDiagram (): string {
  ...
  const addedElements = [...newElements].filter((e) => !oldElements.has(e));
  const removedElements = [...oldElements].filter((e) => !newElements.has(e));
  const added = this.mermaidDiagram.filter((line) => !old.includes(line));
  const removed = old.filter((line) => !this.mermaidDiagram.includes(line));
  ...
}

This visual diff highlights the changes between the old and new architectures, providing a clear visualization of how your resources have evolved.

By traversing the constructs in a Stack, @aws-community/arch-dia can generate a visual representation of your AWS resources and track changes over time. This not only aids in understanding and documenting your infrastructure but can also serve as a powerful tool for communicating changes to stakeholders.

Best Practices and Tips: Using CDK Aspects Effectively

Once you've got a handle on the basics of CDK Aspects, here are a few additional best practices and tips to help you use them more effectively.

1. Use Aspects for Cross-Cutting Concerns

CDK Aspects are ideal for applying changes or enforcing rules that cut across different layers or types of resources in your infrastructure. Consider using Aspects when you want to apply a consistent policy or setting across multiple resources, especially when they are of different types.

2. Be Cognizant of the Construct Tree Traversal

CDK Aspects traverse the construct tree using a depth-first approach. This means that the visit method is invoked on a construct only after it has been invoked on all of its children. In certain scenarios, you may need to be aware of this order of traversal to achieve the desired results.

3. Account for CDK Tokenization

As we discussed earlier, the CDK uses tokenization to handle values that aren't known until deployment time. Be aware of this while designing your Aspects, especially when inspecting or modifying properties that might be tokenized. If necessary, consider using the Token.isUnresolved method to check if a value is a token.

4. Avoid Making Changes Outside the Visit Method

The visit method is the only place where you should make changes to constructs when using Aspects. While it might be technically possible to modify constructs outside this method, doing so can lead to unexpected behavior and hard-to-debug issues.

5. Test Your Aspects

As with any code, you should thoroughly test your Aspects. Given that Aspects can modify constructs across your app, a small error in an Aspect can have a broad impact. Consider using the AWS CDK's built-in testing tools, like the aws-cdk-lib/assertions library, to write unit tests for your Aspects.

My example code includes tests for the aspects: https://github.com/martzcodes/blog-aspects/tree/main/test as does the @aws-community/arch-dia library

CDK Aspects offer a powerful way to enforce consistency and automate modifications across your cloud infrastructure code. With these best practices and tips, you'll be well-equipped to use Aspects effectively in your AWS CDK applications. 🚀

Wrap Up

To wrap it up, here are some final thoughts:

1. Harnessing the Power of Aspects: CDK Aspects are a powerful tool for AWS developers, offering a way to automate cross-cutting concerns and enforce consistency across an entire application. While they may seem complex at first, understanding how they work and how to use them effectively can greatly enhance your AWS CDK toolkit.

2. Exploration and Experimentation: Don't be afraid to explore and experiment with Aspects. Whether you're trying to create an architecture diagram generator or a recursive Aspect, there's a lot of potential for creative and effective solutions.

3. Caution and Diligence: Despite their power, it's important to be cautious when working with Aspects. Be aware of potential pitfalls, such as tokenization and the performance implications of using Aspects. Always test your Aspects thoroughly to avoid introducing broad-reaching errors into your application.

In the end, CDK Aspects can be your 'Heisenberg' in managing cloud infrastructure - they have the potential to be influential, powerful, and transforming. They provide a way to simplify and automate many tasks that would otherwise require manual, error-prone work.

In this guide, we'll show you how to set up self-destructing CDK Stacks and integrate them into your CI/CD pipeline. By doing so, you can reduce costs and improve the efficiency of your development process. We'll also share some best practices and tips to help you make the most out of this feature. So, if you're ready to optimize your development process, read on to learn how to implement self-destructing CDK Stacks! 🤯

Code: https://github.com/martzcodes/blog-cdk-self-destruct

What Will We Make?

We'll create a Step Function that will be executed during the deployment of a Stack and will wait for a specified period of time. Since Step Functions are charged based on state transitions, and not the duration of the run, this will not result in additional costs. Additionally, Standard Step Functions can run for up to a year, providing us with plenty of flexibility. Once the Wait period is over, the Step Function will use the AWS SDK to automatically delete the Stack. 🗑️

Creating a SelfDestruct Construct

We're going to get started by creating a new CDK Construct that can be used in any project. The only thing this Construct will need as a property input will be the Duration that we want the stack to destroy itself after.

export interface SelfDestructProps {
  duration: Duration;
}

export class SelfDestruct extends Construct {
  constructor(scope: Construct, id: string, props: SelfDestructProps) {
    super(scope, id);
    const { duration } = props;
  }
}

From here, we're going to want the Step Function to handle a few things. It should:

1. Re-execute the Step Function on every Stack deployment

2. Close out old executions on new deployments (only have one execution running at any given time)

3. Wait for a pre-defined duration

4. Delete the Stack after the Wait period

List Already Running Step Functions

First, we need to get the list of running executions of this Step Function. We can do that with the states:ListExecutions SDK Command.

const listExecutions = new CallAwsService(this, `ListExecutions`, {
  action: "listExecutions",
  iamAction: "states:ListExecutions",
  iamResources: ["*"],
  parameters: {
    "StateMachineArn.$": "$$.StateMachine.Id",
    StatusFilter: "RUNNING",
  },
  service: "sfn",
});

🏃‍♂️We pass in the StatusFilter: "RUNNING" to make sure we only get back executions that are still in the RUNNING state. Typically there should only be one of these (from the last deployment).

Stop Other Executions

Next we'll want to Map over the returned Executions. Maps are Step Function for-loops, effectively.

const executionsMap = new Map(this, `ExecutionsMap`, {
  inputPath: "$.Executions",
});

In this loop, we're going to want to make sure that the execution isn't going to kill itself (not yet at least). We do this by checking the map item's Execution Id versus the current running execution's Arn:

const stopExecution = new CallAwsService(this, `StopExecution`, {
  action: "stopExecution",
  iamAction: "states:StopExecution",
  iamResources: ["*"],
  parameters: {
    Cause: "Superceded",
    "ExecutionArn.$": "$.ExecutionArn",
  },
  service: "sfn",
});

executionsMap.iterator(
  new Choice(this, "NotSelf?")
    .when(
      Condition.not(
        Condition.stringEqualsJsonPath("$.ExecutionArn", "$$.Execution.Id")
      ),
      stopExecution
    )
    .otherwise(new Pass(this, "self"))
);

$.ExecutionArn refers to the mapped execution's item and $$.Execution.Id refers to the Step Function itself... that is $$ is an escape to "top-level".

Check and Wait to Delete

Next, we can check the State Machine to make sure this resource isn't invoking because of a Stack that is already destroying itself. If it is, we can exit. This is actually very nice because since we just killed the other executions, we're tying up loose ends from previous deployments by making sure that there won't be any executions running.

const wait = new Wait(this, "Wait", {
  time: WaitTime.duration(duration),
});
const wasDelete = new Choice(this, "WasDelete?")
  .when(
    Condition.stringEquals("$$.Execution.Input.Action", "Delete"),
    new Succeed(this, "DeleteSuccess")
  )
  .otherwise(wait);

As part of this, we end up Waiting the duration we set. This could be anywhere from seconds to days (up to 1 year).

After the Wait is over, we need to delete the stack:

const deleteStack = new CallAwsService(this, `DeleteStack`, {
  action: "deleteStack",
  iamAction: "cloudformation:DeleteStack",
  iamResources: ["*"],
  parameters: {
    "StackName.$": "$$.Execution.Input.StackName",
  },
  service: "cloudformation",
});

This is done by an AWS SDK Call cloudformation:DeleteStack.

Creating the State Machine

With all the steps created, we can tie them together to create the actual Step Function:

const finished = new Succeed(this, `Finished`);

listExecutions.next(executionsMap);
executionsMap.next(wasDelete);
wait.next(deleteStack);
deleteStack.next(finished);

const sm = new StateMachine(this, `SelfDestructMachine`, {
  definition: listExecutions,
});

Running the Step Function with Every Deployment

This construct is only useful if it is consistently run with Stack Deployments. So, let's add a Custom Resource that executes the Step Function as part of the Deployment. We can do this with an AwsCustomResource construct:

new AwsCustomResource(this, `SelfDestructCR`, {
  onCreate: {
    action: "startExecution",
    parameters: {
      input: JSON.stringify({
        Action: "Create",
        StackArn: Stack.of(this).stackId,
        StackName: Stack.of(this).stackName,
      }),
      stateMachineArn: sm.stateMachineArn,
    },
    physicalResourceId: PhysicalResourceId.of("SelfDestructCR"),
    service: "StepFunctions",
  },
  onDelete: {
    action: "startExecution",
    parameters: {
      input: JSON.stringify({
        Action: "Delete",
        StackArn: Stack.of(this).stackId,
        StackName: Stack.of(this).stackName,
      }),
      stateMachineArn: sm.stateMachineArn,
    },
    physicalResourceId: PhysicalResourceId.of("SelfDestructCR"),
    service: "StepFunctions",
  },
  onUpdate: {
    action: "startExecution",
    parameters: {
      input: JSON.stringify({
        Action: "Update",
        Version: new Date().getTime().toString(),
        StackArn: Stack.of(this).stackId,
        StackName: Stack.of(this).stackName,
      }),
      stateMachineArn: sm.stateMachineArn,
    },
    physicalResourceId: PhysicalResourceId.of("SelfDestructCR"),
    service: "StepFunctions",
  },
  policy: AwsCustomResourcePolicy.fromSdkCalls({
    resources: [sm.stateMachineArn],
  }),
});

When the Stack deploys it makes a different SDK call based on the type of Stack operation (Create, Update, Delete). Custom Resources only execute when input parameters change. onCreate and onDelete are considered "new" since the stack is being created or destroyed, but in order to make sure the onUpdate call happens we have to touch an input parameter within it. That's why we set the Version to the current time.

Tips for Self-Destruction

💡 Did you notice that the code above didn't explicitly set any IAM permissions? CDK + Step Functions handle all of that for you. By defining the action, and iamActions / services as part of the Step and AwsCustomResource constructs CDK automatically infers IAM permissions and make sure those are attached to the Resources so that they have access to perform their functions!

Creating a DeveloperStack

For a better DevEx you could create a standardized Stack template that includes the self-destruct Construct by default. For example, you could publish BlogCdkSelfDestructStack as your common stack in an npm library:

export class BlogCdkSelfDestructStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new SelfDestruct(this, `SelfDestruct`, {
      duration: Duration.minutes(3),
    });
  }
}

When teams create new projects, instead of creating a stack and basing it off of cdk.Stack .... they would base it off of BlogCdkSelfDestructStack which has self-destruction built in!

Automatically Detecting Temporary Stacks

Clearly you don't want your production stacks to delete themselves. Another tip would be to introduce a property into your Base stack that indicates whether it should self-destruct or not. You could do this by stack naming conventions, or have a developer or CI/CD property. For example:

export interface BlogCdkSelfDestructStackProps extends cdk.StackProps {
  cicd?: boolean;
  developer?: boolean;
  production?: boolean;
}
export class BlogCdkSelfDestructStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: BlogCdkSelfDestructStackProps) {
    super(scope, id, props);
    const { cicd, developer, production } = props;

    if (developer && production) {
      throw new Error("Don't use developer stacks in production");
    }

    if (!production && (developer || cicd)) {
      new SelfDestruct(this, `SelfDestruct`, {
        duration: Duration.minutes(3),
      });
    }
  }
}

And then in your bin file, you would pass in the appropriate properties (which could come from node config, environment variables / etc. CDK Best Practices recommend Configure with properties and methods, not environment variables which is why you would place these properties into your bin file.

Many CI/CD systems have pre-defined system environment variables, and those could be used to automatically detect the CI/CD for self-destruction. For example you could create a namespaced Stack that gets deployed as part of an automated PR integration check. Then succeed or fail the stack would automatiaclly clean up after itself without CI/CD having to do it.

Can I Extend the Wait Without Re-Deploying?

Absolutely! Simply re-execute the Step Function! This will reset the timer giving you more time if you need it.

Conclusion

And just like that, you're a self-destructing CDK Stack pro! You can now confidently say "adios" to stacks that are taking up too much space and draining your resources.

With this newfound knowledge, you can save on infrastructure costs and keep your AWS account looking fresh and tidy. Plus, you'll have the satisfaction of knowing that you're incorporating a little excitement and danger into your development process.

Just remember, with great power comes great responsibility. Be sure to set a reasonable Wait period and test your code thoroughly before deploying. And don't worry, we won't tell anyone if you shed a tear or two as your stacks go boom.

Allen is quite a prolific writer and publishes his articles in (at least) four places. He has a self-hosted static blog built with Amplify using Hugo, as well as using dev.to, hashnode, and medium. His self-hosted blog on his personal domain is his primary platform and dev.to, hashnode, and medium all get canonical URLs for SEO purposes. 🌟

While Allen's code is great, it does have some limitations. For instance, it's written using SAM/yaml, requires a Hugo/Amplify built blog, effectively has no optional features, and he still manually uploads image assets to S3 for all of his articles. 😱

In this article, we'll go over my fork of Allen's code, where I have:

• Converted the project to use AWS CDK

• Made Hugo/Amplify and most of the other platforms optional 🚀

• Added a direct (private) GitHub webhook integration

• Automatically parsed images committed to GitHub and uploaded them to a public S3 Bucket (and updated the content to use them)

I'm excited to share these improvements with you and hope you find them useful!

Code: https://github.com/martzcodes/blog-crossposting-automation

Converting the Project to CDK

Let's talk about converting projects from a format, like SAM, to CDK. It can be a bit tricky, but the easiest way is to focus on the architecture. Get the architecture skeleton right first, and everything else will fall into place. 💪

So, looking at Allen's project structure, we can see that he has one DynamoDB table, five lambda functions, and a step function. One lambda is triggered by an Amplify EventBridge Event. That lambda then triggers a step function where the other four lambdas are used. 🤓

To improve this, we're going to make Amplify optional and add the ability to pull images used in GitHub and re-store them in S3, bringing the S3 bucket into our CloudFormation Stack. 🚀

We can start by creating a Construct for the DynamoDB Table lib/dynamo.ts

export class DynamoDb extends Construct {
  table: Table;
  constructor(scope: Construct, id: string) {
    super(scope, id);
    this.table = new Table(this, `ActivityPubTable`, {
      partitionKey: { name: "pk", type: AttributeType.STRING },
      sortKey: { name: "sk", type: AttributeType.STRING },
      billingMode: BillingMode.PAY_PER_REQUEST,
      timeToLiveAttribute: "ttl",
      removalPolicy: RemovalPolicy.DESTROY,
    });

    this.table.addGlobalSecondaryIndex({
      indexName: "GSI1",
      partitionKey: { name: "GSI1PK", type: AttributeType.STRING },
      sortKey: { name: "GSI1SK", type: AttributeType.STRING },
      projectionType: ProjectionType.ALL,
    });
  }
}

We create the secret via CDK (and then manually put the secrets into it):

const secret = Secret.fromSecretNameV2(
  this,
  `CrosspostSecrets`,
  secretName
);

Allen had a single lambda do the data transformations for the three blog services... I opted to split that up for better traceability. My architecture will end up having somewhere between 3-7 lambdas depending on what options you turned on. The lambdas are only created if you pass in the properties. They're all created the same general way (side note... I also updated Allen's code from javascript to TypeScript #scopecreep):

const lambdaProps: NodejsFunctionProps = {
  architecture: Architecture.ARM_64,
  memorySize: 1024,
  timeout: Duration.minutes(5),
  runtime: Runtime.NODEJS_18_X,
  environment: {
    TABLE_NAME: table.tableName,
    SECRET_ID: secret.secretName,
  },
};

const sendApiRequestFn = new NodejsFunction(this, `SendApiRequestFn`, {
  ...lambdaProps,
  entry: join(__dirname, `../functions/send-api-request.ts`),
});
sendApiRequestFn.addEnvironment("DRY_RUN", dryRun ? "1" : "0");
secret.grantRead(sendApiRequestFn);

Finally, we need to create the Step Function. Step Functions are notoriously hard to code since they use Amazon States Language (ASL) to define all of the steps. I created a separate CrossPostStepFunction construct.

My step function adds the ability to pick which service creates the Canonical URL and it will first post to that service... get the canonical URL and use that in the subsequent services. There's also a lot of logic to remove things from the State Machine if properties weren't configured... which makes this very flexible.

We were able to abstract out the process for posting to a service to a State Machine Fragment. This fragment is a CDK construct that allows us to re-use a lot of the underlying logic used in the parallel paths for posting to the services. When I configured my stack to not send status emails, not use Hugo and have Hashnode be the primary blog post we get a Step Function that looks like this:

A lot of this is 1:1 with what Allen had in his ASL json file. One interesting fact is that Allen's JSON is 953 lines while my two files of TypeScript code that make up the Step Function ends up being 596 lines (431 + 165)... so almost half while adding a few additional features.

Adding a Direct GitHub Webhook Integration

For our next trick, we will use GitHub Webhook events to trigger our cross-posting, instead of Amplify Events. We can do this by adding a Function URL to the Identify Content Lambda:

const fnUrl = identifyNewContentFn.addFunctionUrl({
  authType: FunctionUrlAuthType.NONE,
  cors: {
    allowedOrigins: ["*"],
  },
});
new CfnOutput(this, `GithubWebhook`, { value: fnUrl.url });

We can then enter this lambda into our GitHub repo's webhook settings so that it will use the webhook for Push events. This enables us to skip some of the identify lambda's code... since the push event happens for every commit and pre-includes the list of files that were added.

export const handler = async (event: any) => {
  try {
    await initializeOctokit();

    let newContent: { fileName: string; commit: string }[] = [];
    if (event.body) {
      const body = JSON.parse(event.body);
      console.log(JSON.stringify({ body }, null, 2));
      if (body.commits) {
        newContent = body.commits.reduce(
          (
            p: { fileName: string; commit: string }[],
            commit: {
              id: string;
              added: string[];
              modified: string[];
              // ... there is more stuff here, but this is all we need
            }
          ) => {
            const addedFiles = commit.added.filter(
              (addedFile: string) =>
                (!blogPathDefined ||
                  addedFile.startsWith(`${process.env.BLOG_PATH}/`)) &&
                addedFile.endsWith(".md")
            );
            return [
              ...p,
              ...addedFiles.map((addedFile) => ({
                fileName: addedFile,
                commit: commit.id,
              })),
            ];
          },
          [] as { fileName: string; commit: string }[]
        );
      } else {
        const recentCommits = await getRecentCommits();
        if (recentCommits.length) {
          newContent = await getNewContent(recentCommits);
        }
      }
    }
    if (newContent.length) {
      const data = await getContentData(newContent);
      const imagesProcessed = await saveImagesToS3(data);
      await processNewContent(imagesProcessed);
    }
  } catch (err) {
    console.error(err);
  }
};

The webhook's event body includes a list of the commits. The Amplify event doesn't have this same list, so we can save some GitHub API calls here. I think this would also be compatible with Allen's code (just in case he wants to switch to that 😈).

Parse and Store Images in S3

But what do we do about images? My overall idea here (for my personal use) was to use a private GitHub repo to store these posts (to avoid SEO shenanigans) and just use relative image linking within the repo for the draft images... that way I could use VS Code's Markdown Preview or Obsidian.md to draft my posts. I asked Allen what he does and was surprised to hear that he hasn't automated this part yet... and as part of his writing he manually uploads images to S3.

So, I got a little creative with some Regular Expressions and parsed out any embedded markdown links... which are formatted with an exclamation point in front of a markdown link (ironically, I can't post an example because my RegExp would incorrectly ingest that 😅)

const contentData: {
  fileName: string;
  commit: string;
  content: string;
  sendStatusEmail: boolean;
}[] = [];
const imgRegex = /!\[(.*?)\]\((.*?)\)/g;
for (let j = 0; j < newContent.length; j++) {
  const workingContent = { ...newContent[j] };
  const imageSet = new Set<string>([]);
  let match;
  while ((match = imgRegex.exec(newContent[j].content)) !== null) {
    imageSet.add(match[2]);
  }
  const images = [...imageSet];
  if (images.length === 0) {
    // no images in the post... passthrough
    contentData.push(newContent[j]);
    continue;
  }
  const blogFile = newContent[j].fileName;
  const blogSplit = `${blogFile}`.split("/");
  blogSplit.pop();
  const blogBase = blogSplit.join("/");
  const s3Mapping: Record<string, string> = {};
  for (let k = 0; k < images.length; k++) {
    const image = images[k];
    const githubPath = join(blogBase, image);
    const imageSplit = image.split(".");
    const imageExtension = imageSplit[imageSplit.length - 1];
    const s3Path = `${blogFile}/${k}.${imageExtension}`.replace(/\ /g, "-");
    const s3Url = `https://s3.amazonaws.com/${process.env.MEDIA_BUCKET}/${s3Path}`;
    const postContent = await octokit.request(
      "GET /repos/{owner}/{repo}/contents/{path}",
      {
        owner: `${process.env.OWNER}`,
        repo: `${process.env.REPO}`,
        path: githubPath,
      }
    );

    const buffer = Buffer.from((postContent.data as any).content, "base64");

    // upload images to s3
    const putImage = new PutObjectCommand({
      Bucket: `${process.env.MEDIA_BUCKET}`,
      Key: s3Path,
      Body: buffer,
    });
    await s3.send(putImage);

    s3Mapping[image] = s3Url;
  }
  const rewriteLink = (match: string, text: string, url: string) => {
    return `![${text}](${s3Mapping[url]})`;
  }
  workingContent.content = workingContent.content.replace(imgRegex, rewriteLink);
  contentData.push(workingContent);
}
return contentData;

This code parses out the image links, fetches them from GitHub, uploads them to S3 and replaces the public S3 URLs in the blog post before proceeding. I suppose a requirement here is that the images are required to be in GitHub and if they aren't... things will break. That would be an easy thing for someone to fix/make more flexible 😉

Final Thoughts

That was quite the journey! Although this project took me longer than expected, it was still a lot of fun to work on. I had a tough time trying to limit myself from adding more and more features to it. 😂

After spending some time with it, I'm not entirely convinced that the return on investment is worth it for my specific needs. I only post to two platforms, hashnode and dev.to, it's simple enough for me to copy and paste from one to the other and add the canonical URL to the dev.to metadata. In fact, the two platforms even have an integration that might allow me to skip the copy/paste step entirely. 🤔

But even though I may not use this stack myself, I do hope that it showcases the power and flexibility of creating with CDK. In comparing SAM to CDK... the CDK code clocked in at 907 lines of code (including the Step Function + additional features) while the SAM YAML + ASL JSON came in at 1259 lines of configuration.

This project would have been much quicker to build if I already had a Hugo/Amplify setup or if I hadn't converted everything to TypeScript or added all the other features. 😅

What do you think? Have you ever worked on a project that ended up taking longer than you expected? Did you find it hard to limit yourself from adding more and more features? What do you think about the differences between SAM and CDK here? Let's chat about it! 💬

What are Core Web Vitals?

Core Web Vitals are a set of metrics defined by Google to measure the user experience on the web. They focus on the key aspects of website performance that directly impact user experience, such as loading speed, interactivity, and visual stability. The basics of Core Web Vitals are:

1. Largest Contentful Paint (LCP): measures loading performance and is calculated as the time it takes for the largest content element on the page (e.g. an image or text block) to load and become visible to the user.

2. First Input Delay (FID): measures interactivity and is calculated as the time it takes for a page to become responsive after a user first interacts with it (e.g. clicks a button).

3. Cumulative Layout Shift (CLS): measures visual stability and is calculated as the total amount of unexpected layout shifts that occur during the page load.

These metrics are considered important because they directly impact user experience and are tied to search ranking factors. Websites that score well on Core Web Vitals are likely to have better search engine rankings and provide a better user experience. In other words, they lead to:

1. Happy Users: By optimizing Core Web Vitals, your website will load faster, be more interactive, and have less visual instability. This leads to a better overall user experience, which means visitors will stay on your site longer and be more likely to return in the future.

2. Better SEO: Google uses Core Web Vitals as part of its ranking algorithm, so websites that score well on these metrics are likely to have better search engine rankings. That means more visibility, more traffic, and more potential customers!

3. Increased Conversions: A great user experience leads to higher engagement and increased conversions. By optimizing Core Web Vitals, you'll give your visitors a smooth and seamless experience that will keep them coming back for more.

4. Industry Standard: Core Web Vitals are becoming the industry standard for measuring website performance and user experience. By optimizing these metrics, you'll ensure that your website is up-to-date and providing the best possible experience for your visitors.

Real User Monitoring with CloudWatch RUM and CDK

Let's get started with Core Web Vitals by writing some reusable components: A CDK Construct and Typescript snippet so that we can add CloudWatch RUM (Real User Monitoring) to our application.

The code for this section is located at http://github.com/martzcodes/blog-cdk-rum

We're going to start with a baseline project that includes a simple website deployed to S3 and hosted by a CloudFront distribution. This is similar to the site we created in my article about how to Protect a Static Site with Auth0 Using Lambda@Edge and CloudFront

We're going to focus on the 3 most important files:

1. /lib/rum-runner-construct.ts - CDK Construct

2. /lib/rum-runner-fn.ts - Custom Resource Function

3. /ui/rum.ts - Typescript Snippet to add to our front-end code

CDK Construct

Our RumRunnerConstruct, /lib/rum-runner-construct.ts, will take in two properties to its interface:

• bucket - the UI's deployment bucket

• cloudFront - the UI's CloudFront distribution

CloudWatch RUM requires a Cognito Identity Pool to allow unauthenticated access for the CloudWatch RUM web client to publish events.

We create the Identity Pool

const cwRumIdentityPool = new CfnIdentityPool(
  this,
  "cw-rum-identity-pool",
  { allowUnauthenticatedIdentities: true }
);

A role for unauthenticated users to use:

const cwRumUnauthenticatedRole = new Role(
  this,
  "cw-rum-unauthenticated-role",
  {
    assumedBy: new FederatedPrincipal(
      "cognito-identity.amazonaws.com",
      {
        StringEquals: {
          "cognito-identity.amazonaws.com:aud": cwRumIdentityPool.ref,
        },
        "ForAnyValue:StringLike": {
          "cognito-identity.amazonaws.com:amr": "unauthenticated",
        },
      },
      "sts:AssumeRoleWithWebIdentity"
    ),
  }
);

And we make sure that role has access to put events into CloudWatch RUM:

cwRumUnauthenticatedRole.addToPolicy(
  new PolicyStatement({
    effect: Effect.ALLOW,
    actions: ["rum:PutRumEvents"],
    resources: [
      `arn:aws:rum:${Stack.of(this).region}:${
        Stack.of(this).account
      }:appmonitor/${cloudFront.distributionDomainName}`,
    ],
  })
);

Then we attach the role to the unauthenticated users:

new CfnIdentityPoolRoleAttachment(
  this,
  "cw-rum-identity-pool-role-attachment",
  {
    identityPoolId: cwRumIdentityPool.ref,
    roles: {
      unauthenticated: cwRumUnauthenticatedRole.roleArn,
    },
  }
);

Next, we need to create the app monitor, which we do by using the Level 1 CDK Construct CfnAppMonitor:

const cwRumAppMonitor = new CfnAppMonitor(this, "cw-rum-app-monitor", {
  domain: cloudFront.distributionDomainName,
  name: cloudFront.distributionDomainName,
  appMonitorConfiguration: {
    allowCookies: true,
    enableXRay: false,
    sessionSampleRate: 1,
    telemetries: ["errors", "performance", "http"],
    identityPoolId: cwRumIdentityPool.ref,
    guestRoleArn: cwRumUnauthenticatedRole.roleArn,
  },
  cwLogEnabled: true,
});

Now, to automatically include the newly created app monitor client into our app... we need to take a few extra steps in our Construct. We'll need to run a Custom Resource to fetch some metadata and store it in the UI's bucket for the UI to load. The RUM App Monitor web client needs the CloudWatch RUM App Monitor Id, the Role ARN and the Identity Pool Id.

We create the NodeJS Function:

const rumRunnerFn = new NodejsFunction(this, "rum-runner", {
  runtime: Runtime.NODEJS_18_X,
  environment: {
    BUCKET_NAME: bucket.bucketName,
    RUM_APP: cloudFront.distributionDomainName,
    GUEST_ROLE_ARN: cwRumUnauthenticatedRole.roleArn,
    IDENTITY_POOL_ID: cwRumIdentityPool.ref,
  },
  timeout: Duration.seconds(30),
  entry: join(__dirname, "./rum-runner-fn.ts"),
});

And give it the right permissions:

bucket.grantWrite(rumRunnerFn);
rumRunnerFn.addToRolePolicy(
  new PolicyStatement({
    effect: Effect.ALLOW,
    resources: [
      `arn:aws:rum:${Stack.of(this).region}:${
        Stack.of(this).account
      }:appmonitor/${cloudFront.distributionDomainName}`,
    ],
    actions: ["rum:GetAppMonitor"],
  })
);

Then we create the Custom Resource, with a dependency on the App Monitor so that it runs AFTER the App Monitor is created:

const rumRunnerProvider = new Provider(this, "rum-runner-provider", {
  onEventHandler: rumRunnerFn,
});

const customResource = new CustomResource(this, "rum-runner-resource", {
  serviceToken: rumRunnerProvider.serviceToken,
  properties: {
    // Bump to force an update
    Version: "2",
  },
});

customResource.node.addDependency(cwRumAppMonitor);

Custom Resource Function

CloudFormation Custom Resources are a way to write some provisioning logic that is executed as part of a CDK (CloudFormation) deployment. The Custom Resource will invoke our lambda (/lib/rum-runner-fn.ts ) which will use the AWS SDK to fetch the App Monitor Id after it's created and store it in S3, along with the Role ARN and Identity Pool Id.

We import and create two clients:

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { RUMClient, GetAppMonitorCommand } from "@aws-sdk/client-rum";

const s3 = new S3Client({});
const rum = new RUMClient({});

Fetch the App Monitor config using the RUMClient:

export const handler = async () => {
  const app = await rum.send(
    new GetAppMonitorCommand({ Name: `${process.env.RUM_APP}` })
  );
};

And then upload them to the UI bucket:

export const handler = async () => {
  // ...
  const command = new PutObjectCommand({
    Key: "rum.json",
    Bucket: process.env.BUCKET_NAME,
    Body: JSON.stringify({
      APPLICATION_ID: `${app?.AppMonitor?.Id}`,
      guestRoleArn: `${process.env.GUEST_ROLE_ARN}`,
      identityPoolId: `${process.env.IDENTITY_POOL_ID}`,
    }),
  });
  await s3.send(command);
};

Typescript Snippet

The final piece of the puzzle is injecting this into our front end. The front-end will need to include our front-end typescript snippet/ui/rum.ts

This snippet extends the official/boilerplate snippet, that you would download from the CloudWatch RUM Console, by including some code to fetch the required metadata we stored using the Custom Resource above.

We fetch the configuration by calling:

const res = await fetch('/rum.json');
const rum = await res.json();

And then using the fetched metadata in the configuration of the web client:

const config: AwsRumConfig = {
  sessionSampleRate: 1,
  guestRoleArn: `${rum?.guestRoleArn}`,
  identityPoolId: `${rum?.identityPoolId}`,
  endpoint: "https://dataplane.rum.us-east-1.amazonaws.com",
  telemetries: ["errors","performance","http"],
  allowCookies: true,
  enableXRay: false
};

const APPLICATION_ID: string = `${rum?.APPLICATION_ID}`;
const APPLICATION_VERSION: string = "1.0.0";
const APPLICATION_REGION: string = "us-east-1";

if (APPLICATION_ID) {
  const awsRum: AwsRum = new AwsRum(
    APPLICATION_ID,
    APPLICATION_VERSION,
    APPLICATION_REGION,
    config
  );
  console.log(awsRum);
}

Then we only need to make sure we import and run the rumRunner function at the start of our front-end code (ui/main.ts):

import { rumRunner } from "./rum";

rumRunner();

Deploying and Integrating with Other Web Applications

By deploying this code we can start to gain insights from our applications including Core Web Vitals and other performance metrics.

By packaging this as a CDK Construct and Typescript snippet... we could easily add this to other CDK projects by adding in FOUR lines of code 🤯

Two lines in your CDK App:

import { RumRunnerConstruct } from './rum-runner-construct';
new RumRunnerConstruct(this, `Rum`, { bucket, cloudFront, });

And two lines in your front-end app:

import { rumRunner } from "./rum";
rumRunner();

(and I'm being generous with the line counts)

Conclusion

Measuring the quality of user experience is key to delivering a seamless and enjoyable experience for your website visitors. Core Web Vitals play a crucial role in this measurement, providing objective metrics to assess the performance of your website. To make the most of these metrics, it's important to consider Core Web Vitals early on in the development process. One way to do this is by using reusable CDK constructs, which can help you identify areas for improvement and optimize the user experience of your website.

🛑 Not sure where to start with CDK? See my CDK Crash Course on freeCodeCamp

The architecture we'll be deploying with CDK is:

In this part we'll focus on the final bit of architecture for subscribing to EventBridge Schema Registry Events and bootstrapping them into the EventCatalog. We'll also talk about strategies for integrating this into CI/CD to make it fully automated.

💻 The code for this series is published here: https://github.com/martzcodes/blog-event-driven-documentation

🤔 If you have any architecture or post questions/feedback... feel free to hit me up on Twitter @martzcodes.

EventBridge Schema Discovery

Amazon EventBridge offers a Schema Registry and Discovery feature. This feature monitors Event traffic and creates JSON Schemas based on the events it sees. The awesome thing about this is every time it creates a new schema or updates a new one... it emits an AWS Event that we can trigger off of! We'll use these events to export the discovered event's schema and bundle them in to EventCatalog, similar to how we did with API Gateways in part 2.

⚠️ If you have inconsistent Event Schemas (schemas with "optional" fields) a new version will be created every time the optional fields appear/disappear. A best practice for Event Schemas would be to make sure the event interfaces stay consistent (no optional fields and try not to use objects with changing keys).

Enabling Schema Discovery

First, we'll create a new construct for our Account Stack:

export interface EventsConstructProps {
  bus: IEventBus;
  specBucket: Bucket;
}

export class EventsConstruct extends Construct {
  constructor(scope: Construct, id: string, props: EventsConstructProps) {
    super(scope, id);
    const { bus, specBucket } = props;

    new CfnDiscoverer(this, `Discoverer`, {
      sourceArn: bus.eventBusArn,
      description: "Schema Discoverer",
      crossAccount: false,
    });
  }
}

This construct uses CDK's level 1 construct called CfnDiscoverer. We provide it with our default bus and tell it not to track events that came from outside of the account we're currently in (that could get noisy).

🌈✨ Level 1 Constructs are 1:1 mappings with the equivalent CloudFormation(e.g. CloudFormation vs CDK L1)

Exporting Event Schemas

Creating the Infrastructure

With Schema Discovery enabled, we can create our lambda and invoke that lambda based on the AWS Service Events.

const eventsFn = new NodejsFunction(this, `eventsFn`, {
  runtime: Runtime.NODEJS_16_X,
  entry: join(__dirname, `./events-lambda.ts`),
  logRetention: RetentionDays.ONE_DAY,
  initialPolicy: [
    new PolicyStatement({
      effect: Effect.ALLOW,
      actions: ["schemas:*"],
      resources: ["*"],
    }),
  ],
});
specBucket.grantReadWrite(eventsFn);
eventsFn.addEnvironment("SPEC_BUCKET", specBucket.bucketName);
bus.grantPutEventsTo(eventsFn);

We make sure to grant the lambda the right permissions (read/write to the bucket, the bucket name as an environment variable and putEvents to the default bus).

new Rule(this, `eventsRule`, {
  eventBus: props.bus,
  eventPattern: {
    source: ["aws.schemas"],
    detailType: ["Schema Created", "Schema Version Created"],
  },
  targets: [new LambdaFunction(eventsFn)],
});

AWS Schema events offer two detail types: "Schema Created" and "Schema Version Created". You can see the contents of these on the Explore page in the EventBridge console. We invoke our lambda using these detail types.

Processing the Events

Unlike Part 2... processing these events are a lot easier because we have everything we need via the Event and only need to make one aws-sdk call. The event includes the Schema Name and Version and we use that to export the JSONSchema via the aws sdk:

const RegistryName = event.detail!.RegistryName;
const SchemaName = event.detail!.SchemaName;
const SchemaVersion = event.detail!.Version;
const SchemaDate = event.detail!.CreationDate;

const exportSchemaCommand = new ExportSchemaCommand({
  RegistryName,
  SchemaName,
  Type: "JSONSchemaDraft4",
});
const schemaResponse = await schemasClient.send(exportSchemaCommand);

From there we put it in our spec bucket:

const schema = JSON.parse(schemaResponse.Content);

const fileLoc = {
  Bucket: process.env.SPEC_BUCKET,
  Key: `events/${SchemaName}/spec.json`,
};

const putObjectCommand = new PutObjectCommand({
  ...fileLoc,
  Body: JSON.stringify(schema),
});
await s3.send(putObjectCommand);

And emit the event with our presigned URL:

const getObjectCommand = new GetObjectCommand({
  ...fileLoc,
});
const url = await getSignedUrl(s3, getObjectCommand, { expiresIn: 60 * 60 });

const eventDetail: EventSchemaEvent = {
  SchemaName,
  SchemaVersion,
  RegistryName,
  SchemaDate,
  url,
};

const putEvent = new PutEventsCommand({
  Entries: [
    {
      Source,
      DetailType: BlogDetailTypes.EVENT,
      Detail: JSON.stringify(eventDetail),
    },
  ],
});
await eb.send(putEvent);

Updating the Watcher to Copy the Schemas

In Part 2 we added a utility method to our spec construct that creates a lambda with a rule. We need to use that here to add a lambda for these Event schemas:

this.addRule({
  detailType: BlogDetailTypes.EVENT,
  lambdaName: `eventWatcher`,
});

This lambda simply copies the spec files using a certain S3 Key naming convention:

export const handler = async (
  event: EventBridgeEvent<string, EventSchemaEvent>
) => {
  const res = await fetch(event.detail.url);
  const spec = (await res.json()) as Record<string, any>;

  const fileLoc = {
    Bucket: process.env.SPEC_BUCKET,
    Key: `events/${event.account}/${event.detail.SchemaName}/${event.detail.SchemaVersion}.json`,
  };

  const putObjectCommand = new PutObjectCommand({
    ...fileLoc,
    Body: JSON.stringify(spec, null, 2),
  });
  await s3.send(putObjectCommand);
};

Bootstrapping the Markdown files for EventCatalog

Now that we have our event JSONSchemas stored in our Watcher's Spec Bucket, we can update our prepare scripts to pull the files and bootstrap them (similar to how we did the API Gateway files in Part 2). One notable difference is that EventCatalog's Event interface offers "Consumers and Producers" and Event Versioning. We're going to create a pseudo-service that represents our Account's EventBus and specify that as the Producer for these events. This is kind of a hack, but it's a useful one. We're also going to create the files needed to version our events.

The folder structure for a domain will end up looking like this:

acct-<account>/
┣ events/
┃ ┣ blog.dev.catalog@Spec.event/
┃ ┃ ┣ index.md
┃ ┃ ┗ schema.json
┃ ┣ blog.dev.catalog@Spec.openapi/
┃ ┃ ┣ versioned/
┃ ┃ ┃ ┣ 1/
┃ ┃ ┃ ┃ ┣ changelog.md
┃ ┃ ┃ ┃ ┣ index.md
┃ ┃ ┃ ┃ ┗ schema.json
┃ ┃ ┃ ┗ 2/
┃ ┃ ┃   ┣ changelog.md
┃ ┃ ┃   ┣ index.md
┃ ┃ ┃   ┗ schema.json
┃ ┃ ┣ index.md
┃ ┃ ┗ schema.json
┣ services/
┃ ┣<account>-bus/
┃ ┃ ┣ index.md
┃ ┃ ┗ openapi.json
┃ ┗ iam-backed-api/
┃   ┣ index.md
┃   ┗ openapi.json
┗ index.md

Fetch the Events

To fetch the events we use aws-sdk's ListObjectsCommand to get the files prefixed with events/.

const listBucketObjectsCommand = new ListObjectsCommand({
  Bucket,
  Prefix: "events/",
});
const bucketObjects = await s3Client.send(listBucketObjectsCommand);
const specs = bucketObjects.Contents!.reduce((p, c) => {
  const key: string = c.Key!;
  const splitKey = key.split("/");
  const account = splitKey[1];
  const schemaName = splitKey[2];
  const schemaVersion = splitKey[3].split(".")[0];
  if (!Object.keys(p).includes(`${account}-${schemaName}`)) {
    return {
      ...p,
      [`${account}-${schemaName}`]: {
        key,
        account,
        schemaName,
        schemaVersion,
        versions: [{ schemaVersion, key }],
      },
    };
  }
  p[`${account}-${schemaName}`].versions.push({ schemaVersion, key });
  return p;
}, {} as Record<string, { key: string; account: string; schemaName: string; schemaVersion: string; versions: { schemaVersion: string; key: string }[] }>);

We store these S3 keys in an object so that we can determine the latest version of each spec, and we process them by schema.

const specKeys = Object.keys(specs);
for (let j = 0; j < specKeys.length; j++) {
  const specMeta = specs[specKeys[j]];
  const versionInfo = {
    schemaVersion: 0,
    key: "",
    index: -1,
  };
  specMeta.versions.forEach((version, versionInd) => {
    if (Number(version.schemaVersion) > versionInfo.schemaVersion) {
      versionInfo.schemaVersion = Number(version.schemaVersion);
      versionInfo.key = version.key;
      versionInfo.index = versionInd;
    }
  });
  if (versionInfo.index > -1) {
    specMeta.key = versionInfo.key;
    specMeta.schemaVersion = `latest`;
    specMeta.versions.splice(versionInfo.index, 1);
  }

  const getSpecCommand = new GetObjectCommand({
    Bucket,
    Key: specMeta.key,
  });

  const specObj = await s3Client.send(getSpecCommand);
  const spec = await streamToString(specObj.Body as Readable);
  // ...
}

Ensure the Domain folder exists

In Part 2 we created a makeDomain shared method. To ensure the domain folder exists we just need to call it:

const domainPath = makeDomain(specMeta.account);

Create the Pseudo Bus Service

Next, we create the pseudo service:

const pseudoServiceName = `${specMeta.account}-bus`;
const pseudoServicePath = join(
  domainPath,
  `./services/${pseudoServiceName}`
);
mkdirSync(pseudoServicePath, { recursive: true });
const apiMd = [
  `---`,
  `name: ${pseudoServiceName}`,
  `summary: |`,
  `  This is a pseudo-service that represents the Default Event Bus in the AWS Account.  It isn't a real service.`,
  `owners:`,
  `  - martzcodes`,
  `badges:`,
  `  - content: EventBus`,
  `    backgroundColor: red`,
  `    textColor: red`,
  `---`,
];
writeFileSync(join(pseudoServicePath, `./index.md`), apiMd.join("\n"));

Create the Events

We create the latest (parent) event:

const eventPath = join(domainPath, `./events/${specMeta.schemaName}`);
mkdirSync(eventPath, { recursive: true });
writeFileSync(join(eventPath, `./schema.json`), spec);
if (!existsSync(join(eventPath, `./index.md`))) {
  const apiMd = [
    `---`,
    `name: ${specMeta.schemaName}`,
    `version: latest`,
    `summary: |`,
    `  This is the automatically stubbed documentation for the ${specMeta.schemaName} Event in the ${specMeta.account} AWS Account.`,
    `producers:`,
    `  - ${pseudoServiceName}`,
    `owners:`,
    `  - martzcodes`,
    `---`,
    ``,
    `<Schema />`,
  ];
  writeFileSync(join(eventPath, `./index.md`), apiMd.join("\n"));
}

Add Versioning

And finally the version:

for (let k = 0; k < specMeta.versions.length; k++) {
  const specMetaVersion = specMeta.versions[k];

  const getSpecVersionCommand = new GetObjectCommand({
    Bucket,
    Key: specMeta.key,
  });

  const specVersionObj = await s3Client.send(getSpecVersionCommand);
  const specVersion = await streamToString(
    specVersionObj.Body as Readable
  );

  const versionPath = join(
    eventPath,
    `./versioned/${specMetaVersion.schemaVersion}`
  );
  mkdirSync(versionPath, { recursive: true });
  writeFileSync(join(versionPath, `./schema.json`), specVersion);
  const apiMd = [
    `---`,
    `name: ${specMeta.schemaName}`,
    `version: ${specMetaVersion.schemaVersion}`,
    `summary: |`,
    `  This is the automatically stubbed documentation for the ${specMeta.schemaName} Event in the ${specMeta.account} AWS Account.  This is an old version of the spec.`,
    `producers:`,
    `  - ${pseudoServiceName}`,
    `owners:`,
    `  - martzcodes`,
    `---`,
    ``,
    `<Schema />`,
  ];
  writeFileSync(join(versionPath, `./index.md`), apiMd.join("\n"));

  const changelog = [`### Changes`];
  writeFileSync(
    join(versionPath, `./changelog.md`),
    changelog.join("\n")
  );
}

The Final Result

⚡️ You can see this in action at docs.martz.dev

CI/CD Strategies

There are a few factors when determining your own CI/CD strategy for this. Right now everything is automatically updated when we do CDK deploys... but the CDK deploys themselves aren't automated.

The big factor is how many things are you tracking. At work we monitor 30+ AWS accounts used by > 100 developers. That runs the risk of being too much to do something like having the watched events kick off a deployment pipeline. Instead we'll likely use a scheduled CI/CD build to periodically update the documentation.

For CI/CD you could:

• Create a CodeBuild/CodePipeline project to automatically deploy the EventCatalog based on "watched" events.
• Connect your normal CI/CD up to a schedule (maybe you have a lot of events from many accounts and only want to update documentation every hour or so).
• Continue manually deploying it (which is what I'll do for my personal account since I don't deploy there often)

What's Next?

AWS Service Events offer a lot of useful insights in to your applications deployed to AWS.

💡Want to see what other Service Events are available? Check out the EventBridge "Explore" page in the console

From here you could:

• You could extend the schemas by Improving EventBridge Schema Discovery
• Maybe you want to store additional information from GitHub webhooks into DynamoDB
• Track EventBridge Rule changes via CloudFormation deployments

What would you do next?

🙌 If anything wasn't clear or if you want to be notified on future posts... feel free to hit me up on Twitter @martzcodes.

🛑 Not sure where to start with CDK? See my CDK Crash Course on freeCodeCamp

The architecture we'll be deploying with CDK is:

We'll focus on creating an "Account Stack" that will subscribe to CloudFormation Events within that stack and forward them to a central stack (possibly in another account) to be included in the EventCatalog.

💻 The code for this series is published here: https://github.com/martzcodes/blog-event-driven-documentation

🤔 If you have any architecture or post questions/feedback... feel free to hit me up on Twitter @martzcodes.

Wait... Account Stack?

This architecture is designed to be modular. If you only have a single AWS account you could install all of the constructs from this series in a single stack and still get the same result.

At my work we practice Domain Driven Design. Because of this we end up having over 30 AWS Accounts in use by 10+ teams. There are plenty of cases where these domains need to interact with each other via API gateway and EventBridge events, so we wanted a single place to hold the documentation for all of them.

We do this by having a central "Watcher" stack and deploy an "Account" stack to each AWS Account. CDK can handle these with a single cdk deploy command (provided you have the right permissions / have bootstrapped the trusts). In practice, I deploy these Account Stacks in 3 waves (one for each environment type: dev -> qa -> prod).

The Account Stack is responsible for monitoring the API gateway deployments (and in Part 3... the EventBridge schemas) within its domain. When something we care about changes in an account, the Account Stack forwards it to the central AWS Account with the "Watcher" stack where it gets processed.

Creating the Account Stack

Our "Account" stack will only need two main things: A bucket to store our account artifacts and a lambda to process AWS Service events and fetch the API gateway spec. The components of this stack will live in the ./lib/account project folder.

Spec Bucket and Event Bus

💡Looking ahead... we can re-use the spec bucket for both API Gateway specs and EventBridge schemas. We'll create this on the stack itself so we can share it with the CloudFormation listener construct as well as the construct we make in part 3.

const specBucket = new Bucket(this, `AccountSpecBucket`, {
  removalPolicy: RemovalPolicy.DESTROY,
  blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
  objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED,
  lifecycleRules: [ { expiration: Duration.days(7), }, ],
  autoDeleteObjects: true,
});

The bucket follows my "standard" Bucket template with the addition of setting lifecycle rules on the objects. This will expire the objects after 7 days. This is probably unnecessary since the bucket will never grow too large in size, but it doesn't hurt to have.

We also need access to the default Event Bus that we'll be using to grant permissions to our lambda to put events with. We can import that within the stack too:

const bus = EventBus.fromEventBusName(this, `bus`, "default");

CloudFormation Listener Construct

Next, we'll create a construct. When I make constructs I like to start out with a simple template where I define the construct along with some props:

export interface CloudFormationListenerProps {
  bus: IEventBus;
  specBucket: Bucket;
}
export class CFListener extends Construct {
  constructor(
    scope: Construct,
    id: string,
    props: CloudFormationListenerProps
  ) {
    super(scope, id);
    const { bus, specBucket } = props;
  }
}

Our construct only needs to 1) create the lambda and 2) create a rule to trigger the lambda.

Our lambda will be using AWS Service events by subscribing to CloudFormation service events. Any time a CloudFormation (CDK) stack deploys with changes to an API Gateway we'll want to export that gateway's OpenAPI Spec.

😢 Unfortunately there aren't any service events for API Gateway deployments, which is why we're using CloudFormation Events.

const cfFn = new NodejsFunction(this, `cfStatusFn`, {
    runtime: Runtime.NODEJS_16_X,
    entry: join(__dirname, `./cf-listener-lambda.ts`),
    logRetention: RetentionDays.ONE_DAY,
    initialPolicy: [
      new PolicyStatement({
        effect: Effect.ALLOW,
        actions: ['cloudformation:Describe*', 'cloudformation:Get*', 'cloudformation:List*', 'apigateway:Get*'],
        resources: ['*']
      }),
    ],
});

Here we grant the lambda access to some CloudFormation actions that will be used to fetch information on the stack, and we also include apigateway:Get* since we'll use that to get the OpenAPI Spec export.

Our lambda needs to be able to read and write from our spec bucket, and to putEvents onto the event bus:

specBucket.grantReadWrite(cfFn);
cfFn.addEnvironment('SPEC_BUCKET', specBucket.bucketName);
bus.grantPutEventsTo(cfFn);

Finally we create the rule to invoke the lambda:

new Rule(this, `cfRule`, {
  eventBus: props.bus,
  eventPattern: {
    source: ["aws.cloudformation"],
    detailType: ["CloudFormation Stack Status Change"],
    detail: {
      "status-details": {
        status: ["CREATE_COMPLETE", "UPDATE_COMPLETE", "IMPORT_COMPLETE",],
      },
    },
  },
  targets: [new LambdaFunction(cfFn)]
});

⚠️ We only care about events where the stack was successfully updated. We don't want to re-export the OpenAPI spec for a failed deployment.

CloudFormation Listener Lambda

Our lambda is actually slightly complicated. Each CloudFormation SDK API has slightly different things within it (and the property names even change). For example... there's no way to get API Gateway Ids out of it. We're going to end up having to:

1. Describe the Stack
2. Get (and paginate through) the CloudFormation ChangeSet
3. Get the "processed" Stack Template
4. List all of the API Gateways and Find the API Gateways that match our Stack Id
5. Export the OpenAPI specs from the API Gateway to S3
6. Emit an Event that OpenAPI Specs were exported

We'll be using the AWS-SDK v3 clients to do all of this.

Describe the Stack

We want the "friendly" stack name as part of our output. We get this by describing the stack:

const StackName = event.detail["stack-id"];
const describeCommand = new DescribeStacksCommand({ StackName });
const stacks = await cf.send(describeCommand);
const stack = stacks.Stacks?.[0];

Get the ChangeSets

Next, we only care about these events if the ChangeSet includes API Gateways that change. We'll paginate through the ChangeSet sdk and if we see an API Gateway we'll stop:

const ChangeSetName = stack?.ChangeSetId;
const getChangeSets = async (NextToken?: string): Promise<boolean> => {
  const changeSet = await cf.send(
    new DescribeChangeSetCommand({
      StackName,
      ChangeSetName,
      NextToken,
    })
  );
  const apiChanged =
    (changeSet.Changes || []).filter((change) =>
      change.ResourceChange?.ResourceType?.startsWith("AWS::ApiGateway")
    ).length !== 0;
  if (apiChanged) {
    return true;
  }
  if (changeSet.NextToken) {
    return getChangeSets(changeSet.NextToken);
  }
  return false;
};

const apiChanged = await getChangeSets();

We use change.ResourceChange?.ResourceType?.startsWith("AWS::ApiGateway") to detect if API Gateways had any changes.

Get the Template

Next we get the "processed" CloudFormation template. "Processed" means it has the ARNs in it after everything is deployed / resolved. We need this so we can get the names of the API Gateway stages (which is unfortunately not in any other query).

const getTemplateCommand = new GetTemplateCommand({
  StackName,
  TemplateStage: TemplateStage.Processed,
});
const template = await cf.send(getTemplateCommand);
if (!template.TemplateBody) {
  return;
}
const resources: Record<string, any> = JSON.parse(
  template.TemplateBody
)?.Resources;
const apiStages = Object.values(resources).filter(
  (res: any) => res.Type === "AWS::ApiGateway::Stage"
);

Find our API Gateways

🤢 This part sucks. There's no way to query API Gateways for a particular stack... so you have to list them all and filter by the apigateway being tagged with a cloudformation stack id:

const getApis = new GetRestApisCommand({
  limit: 500,
});
const apiRes = await api.send(getApis);
if (!apiRes.items) {
  return;
}
const apis = apiRes.items.reduce((p, c) => {
  if (c.tags?.["aws:cloudformation:stack-id"] !== StackName) {
    return p;
  }
  if (c.tags?.["aws:cloudformation:logical-id"]) {
    return { ...p, [c.tags?.["aws:cloudformation:logical-id"]]: c.id! };
  }
  return p;
}, {} as Record<string, string>);

Export the OpenAPI Specs to S3

Now that we know what APIs are in our stack, we can export the OpenAPI Specs:

for (let j = 0; j < apiStages.length; j++) {
  const restApiId = apis[apiStages[j].Properties.RestApiId.Ref];
  const stageName = apiStages[j].Properties.StageName;
  const exportCommand = new GetExportCommand({
    accepts: "application/json",
    exportType: "oas30",
    restApiId,
    stageName,
  });
  const exportRes = await api.send(exportCommand);
  const oas = Buffer.from(exportRes.body!.buffer).toString();
  apiSpecs[`${restApiId}-${stageName}`] = JSON.parse(oas);
}

And upload them to our Account's Spec Bucket:

const fileLoc = {
  Bucket: process.env.SPEC_BUCKET,
  Key: `openapi/${stack.StackName}/specs.json`,
};

const putObjectCommand = new PutObjectCommand({
  ...fileLoc,
  Body: JSON.stringify(apiSpecs),
});
await s3.send(putObjectCommand);

Emit the Event

Finally, we can emit the Event to the default Event Bus. We get a pre-signed URL and send that to EventBridge with the "friendly" stack name and the the number of API Specs contained within:

const getObjectCommand = new GetObjectCommand({
  ...fileLoc,
});
const url = await getSignedUrl(s3, getObjectCommand, { expiresIn: 60 * 60 });

const eventDetail: OpenApiEvent = {
  stackName: stack.StackName!,
  apiSpecs: Object.keys(apiSpecs).length,
  url,
};

const putEvent = new PutEventsCommand({
  Entries: [
    {
      Source,
      DetailType: BlogDetailTypes.OPEN_API,
      Detail: JSON.stringify(eventDetail),
    },
  ],
});
await eb.send(putEvent);

We use pre-signed URLs because they're easy and flexible. We could also grant read access to the Watcher Stack's Account Principal. The spec files are generally too large to include in the event body.

Forward Events

⚠️ For single-account setups, this step isn't necessary. But for cross-account setups you'd need to forward your events from your account's bus to the target bus:

if (watcherAccount && watcherAccount !== Stack.of(this).account) {
  new Rule(this, `WatcherFwd`, {
    eventPattern: {
      source: [Source],
    },
    targets: [new EventBusTarget(
      EventBus.fromEventBusArn(this, `watcher-bus`, `arn:aws:events:${Stack.of(this).region}:${watcherAccount}:event-bus/default`)
    )]
  });
}

This rule forwards anything with our "shared" Source to the "Watcher" account's default bus.

Updating the Watcher Stack

Now we have events "flowing" to our "Watcher" account and we need to use them. The next part of our architecture will subscribe to these events, copy the specs to our Watcher Bucket and bootstrap the needed markdown files for EventCatalog to use them.

Watcher Spec Construct

The Watcher spec construct will grant cross-account permissions (if in a multi-account situation) and create a new bucket for the specs.

this.bus = EventBus.fromEventBusName(this, `bus`, "default");
const { watchedAccounts = [] } = props;
watchedAccounts.forEach((watchedAccount) => {
  if (watchedAccount !== Stack.of(this).account) {
    this.bus.grantPutEventsTo(new AccountPrincipal(watchedAccount));
  }
});

In a cross-account situation, buses need to grant access by ARN to allow other buses to have putEvent access (which is used by the rules). We didn't do this in the "Account" stack because the Watcher stack doesn't emit events.

And then we create the bucket.

this.specBucket = new Bucket(this, `WatcherSpecBucket`, {
  removalPolicy: RemovalPolicy.DESTROY,
  blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
  objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED,
  autoDeleteObjects: true,
});
new CfnOutput(this, `WatcherSpecBucketOutput`, {
  value: this.specBucket.bucketName,
});

Here the CfnOutput is useful because later we'll need the bucket's name for our post-processing script.

Add Rule Method

Next we need to create a lambda and invoke it via a rule. We know we're going to use a similar pattern in part 3... so we can abstract this a little bit. Both will need a lambda invoked by a rule... so we'll create an addRule method on our Construct:

const fn = new NodejsFunction(this, lambdaName, {
  runtime: Runtime.NODEJS_16_X,
  entry: join(__dirname, `./${lambdaName}.ts`),
  logRetention: RetentionDays.ONE_DAY,
});
this.specBucket.grantWrite(fn);
fn.addEnvironment("SPEC_BUCKET", this.specBucket.bucketName);

new Rule(this, `${lambdaName}Rule`, {
  eventBus: this.bus,
  eventPattern: {
    source: [Source],
    detailType: [detailType],
  },
  targets: [new LambdaFunction(fn)],
});

Copy the Spec(s)

To reduce the number of file transfers our Account Stack combined (potentially) multiple OpenAPI specs in a single file. We'll grab that file and split it up.

const res = await fetch(event.detail.url);
const specs = (await res.json()) as Record<string, any>;

const gateways = Object.keys(specs);
for (let j = 0; j < gateways.length; j++) {
  const fileLoc = {
    Bucket: process.env.SPEC_BUCKET,
    Key: `openapi/${event.account}/${event.detail.stackName}/${gateways[j]}/openapi.json`,
  };

  const putObjectCommand = new PutObjectCommand({
    ...fileLoc,
    Body: JSON.stringify(specs[gateways[j]], null, 2),
  });
  await s3.send(putObjectCommand);
}

We use the pre-signed url to fetch the file and then we store each spec within it as its own file. We're storing this with a templated path that we'll be able to extract information out of later.

Bootstrap the Spec(s) with Markdown into EventCatalog

Unfortunately with EventCatalog you can't just throw it a bunch of spec files and have everything work. It's a static site and those spec files need some markdown in a particular folder structure in order to work. It does have a handy domain concept though and we're going to leverage that to split out things by AWS Account.

documentation
├── domains
│   ├──AWS Account A
│   │     ├──index.md
│   │     ├──services
│   │     │  └──API Gateway
│   │     │     └──index.md
│   │     │     └──schema.json
│   ├──AWS Account B
│   │     ├──index.md
│   │     ├──services
│   │     │  └──API Gateway
│   │     │     └──index.md
│   │     │     └──schema.json
│   │     ├──events
├── eventcatalog.config.js
├── package.json
├── README.md
└── yarn.lock

So we need to run a script that pulls the S3 files and processes them into this structure, creating the files as-needed. We'll need to run this with an AWS Profile that has access to read from the bucket.

Locally, we're going to store this in ./lib/prepare a subfolder and we'll add a script to our package.json to run it: "prepare:catalog": "ts-node ./lib/prepare $SPEC_BUCKET",

We also use the WatcherSpecBucketOutput from the CfnOutput we made when creating the bucket and export SPEC_BUCKET=<WatcherSpecBucketOutput>.

List the files

We'll use the v3 aws-sdk client to list the objects with the openapi/ prefix.

const listBucketObjectsCommand = new ListObjectsCommand({
  Bucket,
  Prefix: "openapi/",
});
const bucketObjects = await s3Client.send(listBucketObjectsCommand);
const specs = bucketObjects.Contents!.map((content) => {
  const key = content.Key!;
  const splitKey = key.split("/");
  const account = splitKey[1];
  const stack = splitKey[2];
  const apiId = splitKey[3].split(".")[0];
  return { key, account, stack, apiId };
});

From there we'll map those objects splitting out the useful information from the key name.

Create the Domain folder

Next, we need to make sure the "Domain" folder exists and if not, create it with the markdown files.

const domain = `acct-${account}`;
const domainPath = join(__dirname, `../../catalog/domains/${domain}/`);
mkdirSync(domainPath, { recursive: true });
if (!existsSync(join(domainPath, `./index.md`))) {
  const domainMd = [
    `---`,
    `name: ${domain}`,
    `summary: |`,
    `  This is the automatically stubbed documentation. Please replace this by clicking the edit button above.`,
    `owners:`,
    `  - martzcodes`,
    `---`,
  ];
  writeFileSync(join(domainPath, `./index.md`), domainMd.join("\n"));
}

It's a very simple stub that just says it was automatically stubbed.

⚡️You can add yourself as a contributor to the ./catalog/eventcatalog.config.js file and then list yourself as an owner in the markdown, and you'll be displayed on the page.

Create the Service Folder

Next, we need to create the service folder.

const basePath = join(domainPath, `./services/${apiName}`);
mkdirSync(basePath, { recursive: true });
writeFileSync(join(basePath, `./openapi.json`), spec);
if (!existsSync(join(basePath, `./index.md`))) {
  const apiMd = [
    `---`,
    `name: ${apiName}`,
    `summary: |`,
    `  This is the automatically stubbed documentation for the ${apiName} API (${specMeta.apiId}) in the ${specMeta.stack} stack. Please replace this.`,
    `owners:`,
    `  - martzcodes`,
    `---`,
    ``,
    `<OpenAPI />`,
  ];
  writeFileSync(join(basePath, `./index.md`), apiMd.join("\n"));
}

Of note here is that we're NOT re-stubbing the file if the file already exists. The idea here is that developers would come into the catalog project and update their documentation via commit. Spec/Schema discovery will be automated and from that point it's easily extended by the devs. 💪

Update the BucketDeployment

Last (but not least)... we want to update our BucketDeployment from Part 1 to run the prepare:catalog script as part of the cdk deployment. This means that any time the watcher stack is deployed, it'll pull the files from S3 and bootstrap any that weren't already defined! The asset generation then becomes:

const bundle = Source.asset(uiPath, {
  assetHash: `${Date.now()}`,
  bundling: {
    command: ["sh", "-c", 'echo "Not Used"'],
    image: DockerImage.fromRegistry("alpine"), // required but not used
    local: {
      tryBundle(outputDir: string) {
        execSync("npm run prepare:catalog"); // <-- added this
        execSync("cd catalog && npm i");
        execSync("cd catalog && npm run build");
        copySync(uiPath, outputDir, {
          ...execOptions,
          recursive: true,
        });
        return true;
      },
    },
  },
});

See It In Action

🙈 SPOILER ALERT: You can see this in action at docs.martz.dev which will be the result of this series🤫

I went ahead and deployed my blog post Create a Cross-Account IAM Authorized APIGateway with CDK in order to get some files to work with and then I CDK deployed this project.

The domain was created:

And the Service was created with the exported OpenAPI spec!

What's Next?

Now that we have automatic API Gateway specs we're only missing the EventBridge Event Schemas... and that's what we'll be doing in Part 3.

🙌 If anything wasn't clear or if you want to be notified on when I post part 3... feel free to hit me up on Twitter @martzcodes.

🛑 Not sure where to start with CDK? See my CDK Crash Course on freeCodeCamp

The architecture we'll be deploying with CDK is:

In this post we'll be focusing on creating the "Watcher Stack" that creates the EventCatalog UI Bucket along with the CloudFront distribution. I'll be going into more detail on our target architecture in the follow-on posts.

💻 The code for this series is published here: https://github.com/martzcodes/blog-event-driven-documentation

🤔 If you have any architecture or post questions/feedback... feel free to hit me up on Twitter @martzcodes.

What is EventCatalog?

EventCatalog is an awesome Open Source project built by David Boyne that helps you document your events, services and domains. It ingests a combination of markdown files, OpenAPI specs and EventBridge event schemas (or AsyncAPI specs) to build a static documentation site.

🙈 SPOILER ALERT: You can see this in action at docs.martz.dev which will be the result of this series🤫

Deploying EventCatalog

Our EventCatalog is going to be stored in S3 and hosted via CloudFront. To do that we're going to create a Level 3 CDK Construct that will:

• Create the UI Bucket
• Create the CloudFront Distribution that hosts the contents of the UI Bucket
• Use a BucketDeployment resource to upload the EventCatalog assets to the UI Bucket

After initializing a CDK project, we can install EventCatalog using: npx @eventcatalog/create-eventcatalog@latest catalog and it will go into a catalog subfolder in our project.

Create the UI Bucket

Creating Buckets with CDK is fairly simple. We'll use the L2 Bucket Construct in our L3 Catalog construct to create the bucket... and then we'll give it some sensible defaults.

export class CatalogOne extends Construct {
  constructor(scope: Construct, id: string) {
    super(scope, id);
    const destinationBucket = new Bucket(this, `EventCatalogBucket`, {
      removalPolicy: RemovalPolicy.DESTROY,
      blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
      objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED,
      autoDeleteObjects: true,
    });
  }
}

• removalPolicy: RemovalPolicy.DESTROY will delete the Bucket if the Stack is destroyed. In order to do this, we need to make sure the bucket is empty. autoDeleteObjects: true creates a CustomResource that will empty the bucket if the Stack is destroyed.
• blockPublicAccess: BlockPublicAccess.BLOCK_ALL will prevent users from directly retrieving files from S3 (forcing them to go through CloudFront).
• objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED enforces normal IAM permissions on the bucket (instead of the hard-to-use ACL-based permissions that S3 started with long ago...)

Create the CloudFront Distribution

In order for CloudFront to access the files in the S3 bucket, we need to grant read access to this. We do this by creating an OriginAccessIdentity (emphasis on IDENTITY) and using the bucket's grantRead method to grant the access.

const originAccessIdentity = new cloudfront.OriginAccessIdentity(
  this,
  `OriginAccessIdentity`
);
destinationBucket.grantRead(originAccessIdentity);

We then create the CloudFront Distribution with an S3 Origin that uses the identity.

const distribution = new cloudfront.Distribution(
  this,
  `EventCatalogDistribution`,
  {
    defaultRootObject: "index.html",
    defaultBehavior: {
      origin: new S3Origin(destinationBucket, { originAccessIdentity }),
    },
  }
);

For convenience, we'll create a CloudFormation Output that has the Catalog's CloudFront-hosed url. This will be logged out as part of the deployment.

new CfnOutput(this, `CatalogUrl`, {
  value: `https://${distribution.distributionDomainName}`,
});

Use a BucketDeployment to Upload Assets

Finally, we need something to actually host. We can use S3 Deployment constructs to upload our EventCatalog's output and deploy it to S3.

const execOptions: ExecSyncOptions = {
  stdio: ["ignore", process.stderr, "inherit"],
};
const uiPath = join(__dirname, `../../../catalog/out`);
const bundle = Source.asset(uiPath, {
  bundling: {
    command: ["sh", "-c", 'echo "Not Used"'],
    image: DockerImage.fromRegistry("alpine"), // required but not used
    local: {
      tryBundle(outputDir: string) {
        execSync("cd catalog && npm i");
        execSync("cd catalog && npm run build");
        copySync(uiPath, outputDir, {
          ...execOptions,
          recursive: true,
        });
        return true;
      },
    },
  },
});

Source.asset can accept commands to locally bundle things. Normally it tries to use docker to do the bundling, but will accept a local override. The command and image are used as the fall-back in case the tryBundle output returns falsey. Within tryBundle we can use any commands we need to create the output.

• cd catalog && npm i changes directory into our catalog folder and makes sure the dependencies are installed
• cd catalog && npm run build run's the build script for EventCatalog
• copySync(... recursively copies the output folder of EventCatalog as an S3 Asset

Next we use that S3 Asset in a Bucket Deployment:

new BucketDeployment(this, `DeployCatalog`, {
  destinationBucket,
  distribution,
  sources: [bundle],
  prune: true,
  memoryLimit: 1024,
});

Here, we specify our UI Bucket, CloudFront distribution and S3 Asset. We include prune: true to ensure old versions of the static site's assets get removed on subsequent deployments and we bumped he memory limit of the BucketDeployment lambda so it's a little faster. In the background this BucketDeployment construct uses a lambda to do the S3 upload. By setting the memoryLimit we're setting the memory of that lambda.

💡 If you use BucketDeployments for other things and run into issues with slowness or failures... try increasing the memory. The default memory is only 128 MB.

If we deploy our EventCatalog now and go to the Catalog's output URL in the deployment log... we'll see our EventCatalog!

EventCatalog includes some Examples built-in! BUT if we reload any page, we'll get a NoSuchKey error 😱

We get this because EventCatalog is trying to access something in S3 at some direct object and doesn't know to add index.html to the end of it. For Single Page Apps that use UI frameworks like React or Angular... this is less of a problem because you can set a default routing to go to the root index.html where the framework will handle it. But this isn't a single page app. It's a static site!

Fixing the lack of CloudFront URL rewrites

We can use an edge lambda to do this rewrite for us. CloudFront has an example of how to do this. Using CDK I'll create an Edge Function to do the rewriting. First we need the lambda:

const edgeFn = new cloudfront.experimental.EdgeFunction(
  this,
  `EdgeRedirect`,
  {
    code: Code.fromInline(
      '"use strict";var n=Object.defineProperty;var u=Object.getOwnPropertyDescriptor;var c=Object.getOwnPropertyNames;var d=Object.prototype.hasOwnProperty;var a=(e,r)=>{for(var i in r)n(e,i,{get:r[i],enumerable:!0})},o=(e,r,i,s)=>{if(r&&typeof r=="object"||typeof r=="function")for(let t of c(r))!d.call(e,t)&&t!==i&&n(e,t,{get:()=>r[t],enumerable:!(s=u(r,t))||s.enumerable});return e};var f=e=>o(n({},"__esModule",{value:!0}),e);var l={};a(l,{handler:()=>h});module.exports=f(l);var h=async e=>{let r=e.Records[0].cf.request;return r.uri!=="/"&&(r.uri.endsWith("/")||r.uri.lastIndexOf(".")<r.uri.lastIndexOf("/"))&&(r.uri=r.uri.concat(`${r.uri.endsWith("/")?"":"/"}index.html`)),r};0&&(module.exports={handler});'
    ),
    handler: "index.handler",
    runtime: Runtime.NODEJS_16_X,
    logRetention: RetentionDays.ONE_DAY,
  }
);

Since Edge Lambda Functions don't support automatic bundling using esbuild like regular lambdas do we just used the Code.fromInline method to automatically upload the inline code as our Lambda source.

Next, we can update our CloudFront Distribution's props to include the edge function:

{
  defaultRootObject: "index.html",
  defaultBehavior: {
    origin: new S3Origin(destinationBucket, { originAccessIdentity }),
    viewerProtocolPolicy: ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
    edgeLambdas: [
      {
        functionVersion: edgeFn.currentVersion,
        eventType: cloudfront.LambdaEdgeEventType.VIEWER_REQUEST,
      },
    ],
  },
}

⚡️ Are you making an internal (private) documentation page? See my last post on how to Protect a Static Site with Auth0 Using Lambda@Edge and CloudFront

⚠️ When you use Lambda@Edge functions... you need to specify the region in your stack. If you don't, you'll get an error like this Error: stacks which use EdgeFunctions must have an explicitly set region when you try to deploy. Set your region like this:

new BlogDevCatalogStack(app, 'BlogDevCatalogWatcherStack', {
  env: {
    region: process.env.CDK_DEFAULT_REGION,
    account: process.env.CDK_DEFAULT_ACCOUNT
  }
});

Now when we deploy, we can refresh our pages (or go directly to pages) without having NoSuchKey errors!

Deploying to a Custom Domain

That's great but <random cloudfront url> is pretty boring. AWS hosts domains too! If you purchased a domain and have a hosted zone set up, you can have CloudFront use it.

First we look up the HostedZone by the domain name.

const domainName = `docs.${hostDomain}`;
const hostedZone = HostedZone.fromLookup(this, `UIZone`, {
  domainName: hostDomain,
});

Then we create a DNS Certificate (so we can use https):

const certificate = new DnsValidatedCertificate(this, `EventCatalogCert`, {
  domainName,
  hostedZone,
});

Then we pass these in to our CloudFront Distribution props:

{
  defaultRootObject: "index.html",
  certificate, // <--
  domainNames: [domainName], // <--
  // ...
}

And finally we create an A Record where the target is for our CloudFront Distribution:

new ARecord(this, `ARecord`, {
  zone: hostedZone,
  recordName: domainName,
  target: RecordTarget.fromAlias(new CloudFrontTarget(distribution)),
});

Putting it all together, I can host my personal documentation at docs.martz.dev!

⚠️ When using hosted zones... you need to specify the account in your stack. If you don't, you'll get an error like this Error: Cannot retrieve value from context provider hosted-zone since account/region are not specified at the stack level. when you try to deploy. Set your account like this:

new BlogDevCatalogStack(app, 'BlogDevCatalogWatcherStack', {
  env: {
    region: process.env.CDK_DEFAULT_REGION,
    account: process.env.CDK_DEFAULT_ACCOUNT
  }
});

What's Next?

Hosting a static site is great, but we haven't even scratched the surface of Event Driven Documentation yet. In parts 2 and 3 we'll automatically fetch API Gateway OpenAPI specs + EventBridge Event Schemas and bundle them into our EventCatalog!

🙌 If anything wasn't clear or if you want to be notified on when I post parts 2 and 3... feel free to hit me up on Twitter @martzcodes.