AWS Lambda Message Producer

It is an AWS Lambda function that validates email sending requests and produces them as messages in an SQS queue for subsequent processing.

Description

This Lambda function is designed to receive email sending requests through HTTP events, validate the data structure, and produce messages in an SQS queue. It is the entry point of the email sending system, where another lambda (Template consumer) will consume these messages to perform template processing and email sending.

Architecture

HTTP Request → Lambda Producer → SQS Queue → Lambda Consumer

Features

Validation of email sending requests
Batch processing of up to 10 messages per request
Message production in SQS queue using SendMessageBatch
Partial error handling
Structured JSON responses
Required field validation

Project Structure

├── message_producer.py    # Main Lambda function
├── parse_and_validate.py # Input validation and data structure
├── find_template_s3.py   # Template existence verification in S3
├── config.py             # AWS SQS/S3 configuration and constants
├── build_response.py     # Utility to build HTTP responses
└── README.md            # Project documentation

Configuration

Required Environment Variables

Variable	Description	Example
`SQS_QUEUE_URL`	URL of the SQS queue where messages will be produced	`https://sqs.<region>.amazonaws.com/<account-id>/<queue-name>`
`S3_BUCKET_NAME`	Name of the S3 bucket where templates reside	`my-templates-bucket`

Configured SQS Parameters

Message receive wait time: 15 seconds
Message Retention: 0 minutes
Visibility Timeout: 30 seconds
Processing attempts: according to message retention and visibility timeout for a maximum of 12 times

Required IAM Permissions

The Lambda function needs the following permissions:

{
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:SendMessage",
                "sqs:SendMessageBatch"
            ],
            "Resource": "arn:aws:sqs:region:account-id:queue-name"
        }
    ]
}

Input Format

Required Request Structure

For message processing to work correctly, the request must follow this specific structure that will allow integration with the email sending flow:

Content-Type: application/json

Method: POST

Body: Array of JSON objects (maximum 10 elements, total size ≤ 200KB)

Request Body

The function expects an event with the following structure:

[
  {
    "sender": "no-reply@my-domain.com",
    "subject": "This is a test from lambda",
    "user_email": "user@example.com",
    "template_id": "welcome_user_external",
    "variables": {
      "user_name": "Sofía"
    }
  },
  {
    "sender": "no-reply@my-domain.com",
    "subject": "This is a test from lambda",
    "user_email": "user@example.com",
    "template_id": "codigo_otp_internal",
    "variables": {
      "user_name": "Carlos",
      "otp_code": "123456"
    }
  }
]

Important rules:

Each object must contain the required fields: sender, subject, user_email, template_id
Maximum 10 objects per request
variables is optional and is used for dynamic content in templates

Dynamic Variables Handling

Templates stored in S3 can contain variables that are replaced dynamically. The payload can include a variables object with name/value pairs that correspond to placeholders in the template.

Example Structure with Variables:

{
  "sender": "no-reply@my-domain.com",
  "subject": "This is a test from lambda",
  "user_email": "user@example.com",
  "template_id": "codigo_otp_internal",
  "variables": {
    "user_name": "carlos",
    "otp_code": "123456"
  }
}

Considerations for Variables:

The variable name must match exactly with the field in the template
Templates are stored in S3 and processed dynamically
Examples of common variables: user_name, otp_code, verification_link, company_name

Validations

Input Validations

Event structure: Must be an object with non-empty body field
JSON format: The body must be valid JSON
Size: The body must not exceed 200KB
Data type: The body must be an array
Element limit: Maximum 10 objects in the array
Required fields: Each object must contain sender, subject, user_email, template_id
Template existence: template_id must exist as <template_id>/index.html in the configured S3 bucket
At least one valid message: There must be at least one message that passes all validations

Response Format

Successful Response (200)

{
  "statusCode": 200,
  "body": "{\"status\":\"success\",\"results\":{\"Procesados\":2,\"failed_messages\":[]}}"
}

Partial Response (200)

{
  "statusCode": 200,
  "body": "{\"status\":\"partial_success\",\"results\":{\"Procesados\":1,\"failed_messages\":[{\"message\":{\"template_id\":\"invalido\",\"reason\":\"Template ID 'invalido' does not exist in S3.\"}}]}}"
}

Error Response (400)

{
  "statusCode": 400,
  "body": "{\"status\":\"error\",\"results\":{\"Procesados\":0,\"failed_messages\":[]},\"message\":\"Invalid request body: <detail>\"}"
}

Error Response (500)

{
  "statusCode": 500,
  "body": "{\"status\":\"internal_error\",\"results\":{\"Procesados\":0,\"failed_messages\":[]},\"message\":\"Internal error while queuing messages: <detail>\"}"
}

Limitations and Considerations

Maximum size: 200KB for the body received by the Lambda
Maximum messages: 10 messages per request
Timeout: Configure appropriate timeout according to message volume

Testing with Postman

Postman Configuration

Method: POST
URL: The API Gateway endpoint
Headers:
```
Content-Type: application/json
```
Body:
- Select raw or JSON
- Use the provided example structure
Send the request

Example Usage with cURL

# Send email sending request (current structure)
curl -X POST https://your-api-gateway-url \
  -H "Content-Type: application/json" \
  -d '[
    {
      "sender": "no-reply@my-domain.com",
      "subject": "This is a test from lambda",
      "user_email": "test@example.com",
      "template_id": "welcome_user_external",
      "variables": {"user_name": "Test User"}
    }
  ]'

# Example with additional variables
curl -X POST https://your-api-gateway-url \
  -H "Content-Type: application/json" \
  -d '[
    {
      "sender": "no-reply@my-domain.com",
      "subject": "This is a test from lambda",
      "user_email": "test@example.com",
      "template_id": "codigo_otp_internal",
      "variables": {
        "otp_code": "123456",
        "verification_link": "https://example.com/verify"
      }
    }
  ]'

Error Handling

Error Types

Validation Errors: Missing fields or incorrect format
SQS Errors: Connectivity or configuration issues
Partial Errors: Some messages fail in the batch

Logging

The function logs:

Validation summary (number of valid and failed messages)
Validation errors with specific details
Partial failure warnings with message IDs
Internal SQS errors

Processing Flow

Reception: Lambda receives the HTTP event
Validation: The structure and content of the body are validated
Processing: Entries for SendMessageBatch are created
Sending: Messages are sent to SQS
Response: The processing result is returned (success or error)

Monitoring and Logs

CloudWatch Logs

Function logs are available in CloudWatch under:

/aws/lambda/Osborn-MessageProducer

Important Metrics

Invocations: Number of times the function is executed
Duration: Average execution time
Errors: Number of errors
Messages sent to SQS: Counter of successfully produced messages

Description​

Architecture​

Features​

Project Structure​

Configuration​

Required Environment Variables​

Configured SQS Parameters​

Required IAM Permissions​

Input Format​

Required Request Structure​

Request Body​

Dynamic Variables Handling​

Validations​

Input Validations​

Response Format​

Successful Response (200)​

Partial Response (200)​

Error Response (400)​

Error Response (500)​

Limitations and Considerations​

Testing with Postman​

Postman Configuration​

Example Usage with cURL​

Error Handling​

Error Types​

Logging​

Processing Flow​

Monitoring and Logs​

CloudWatch Logs​

Important Metrics​