Rizqi Mulki

Backend Developer

Deploying a Python Backend on AWS: EC2, Lambda, and Docker

—

by

Rizqi Mulki
in

Deploying Python backend applications efficiently requires understanding various hosting options and their tradeoffs. Amazon Web Services (AWS) offers multiple deployment methods, each with unique advantages for different use cases. In this post, I’ll walk through three popular approaches to deploying Python backends on AWS: traditional EC2 instances, serverless Lambda functions, and containerized deployments with Docker.

Understanding Your Deployment Options

Before diving into implementation details, let’s compare these three approaches:

EC2 (Elastic Compute Cloud):

  • Full control over the server environment
  • Suitable for complex applications with specific system requirements
  • Requires more management of the underlying infrastructure

Lambda:

  • Serverless, event-driven execution
  • Pay only for compute time used
  • Auto-scaling with zero infrastructure management
  • Limited execution time and resource constraints

Docker on AWS:

  • Consistent environments across development and production
  • Efficient resource utilization
  • Multiple orchestration options (ECS, EKS, Fargate)
  • Better isolation between applications

Let’s explore each approach with practical examples.

Deploying Python on EC2

EC2 provides virtual servers where you have complete control over the environment. This approach works well for applications requiring specific system configurations or long-running processes.

Setting Up an EC2 Instance for Python

First, let’s create an EC2 instance properly configured for a Python application:

  1. Launch an EC2 instance:
    • Choose Amazon Linux 2 or Ubuntu Server
    • Select appropriate instance size (t3.micro is good for starting)
    • Configure security groups to allow HTTP (80), HTTPS (443), and SSH (22)
  2. Connect to your instance: ssh -i your-key.pem ec2-user@your-instance-public-dns
  3. Install Python and dependencies: # For Amazon Linux 2 sudo yum update -y sudo yum install -y python3 python3-pip sudo pip3 install --upgrade pip # For Ubuntu sudo apt update sudo apt install -y python3 python3-pip sudo pip3 install --upgrade pip
  4. Set up a virtual environment: python3 -m pip install virtualenv python3 -m virtualenv ~/venv source ~/venv/bin/activate

Deploying a Flask Application on EC2

Let’s deploy a simple Flask application:

  1. Create application directory: mkdir -p ~/myapp cd ~/myapp
  2. Create a basic Flask application (app.py): from flask import Flask, jsonify app = Flask(__name__) @app.route('/api/health') def health_check(): return jsonify({"status": "healthy"}) @app.route('/api/data') def get_data(): return jsonify({ "items": [ {"id": 1, "name": "Item 1"}, {"id": 2, "name": "Item 2"}, {"id": 3, "name": "Item 3"} ] }) if __name__ == '__main__': app.run(host='0.0.0.0', port=5000)
  3. Create requirements file (requirements.txt): Flask==2.2.3 gunicorn==20.1.0
  4. Install dependencies: pip install -r requirements.txt
  5. Configure Gunicorn for production: Create a systemd service file for reliable operation: sudo nano /etc/systemd/system/flaskapp.service Add the following content: [Unit] Description=Flask Application After=network.target [Service] User=ec2-user WorkingDirectory=/home/ec2-user/myapp ExecStart=/home/ec2-user/venv/bin/gunicorn -b 0.0.0.0:5000 app:app Restart=always [Install] WantedBy=multi-user.target
  6. Enable and start the service: sudo systemctl daemon-reload sudo systemctl enable flaskapp sudo systemctl start flaskapp
  7. Set up Nginx as a reverse proxy: sudo yum install -y nginx # or apt install for Ubuntu Configure Nginx: sudo nano /etc/nginx/conf.d/flaskapp.conf Add the following: server { listen 80; server_name your_domain.com; location / { proxy_pass http://127.0.0.1:5000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } }
  8. Start Nginx and check status: sudo systemctl enable nginx sudo systemctl start nginx sudo systemctl status nginx

Your Flask application should now be accessible through your EC2 instance’s public IP address.

CI/CD for EC2 Deployments

For automated deployments to EC2, here’s a simple GitHub Actions workflow:

name: Deploy to EC2

on:
  push:
    branches: [ main ]

jobs:
  deploy:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v2
    
    - name: Configure SSH
      run: |
        mkdir -p ~/.ssh/
        echo "${{ secrets.EC2_SSH_KEY }}" > ~/.ssh/id_rsa
        chmod 600 ~/.ssh/id_rsa
        ssh-keyscan -H ${{ secrets.EC2_HOST }} >> ~/.ssh/known_hosts
    
    - name: Deploy to EC2
      run: |
        ssh -i ~/.ssh/id_rsa ${{ secrets.EC2_USER }}@${{ secrets.EC2_HOST }} <<'ENDSSH'
          cd ~/myapp
          git pull
          source ~/venv/bin/activate
          pip install -r requirements.txt
          sudo systemctl restart flaskapp
        ENDSSH

Serverless Deployment with AWS Lambda

AWS Lambda lets you run code without provisioning servers, making it ideal for event-driven workloads and APIs with variable traffic.

Creating a Python Lambda Function

Let’s create a serverless API using AWS Lambda and API Gateway:

  1. Set up your project structure: lambda-api/ ├── app.py ├── requirements.txt └── serverless.yml
  2. Create the Lambda handler (app.py): import json def health_check(event, context): return { "statusCode": 200, "body": json.dumps({"status": "healthy"}), "headers": { "Content-Type": "application/json" } } def get_data(event, context): return { "statusCode": 200, "body": json.dumps({ "items": [ {"id": 1, "name": "Item 1"}, {"id": 2, "name": "Item 2"}, {"id": 3, "name": "Item 3"} ] }), "headers": { "Content-Type": "application/json" } }
  3. Using API Gateway with Lambda: The Serverless Framework simplifies deployment. Create serverless.yml: service: python-api provider: name: aws runtime: python3.9 region: us-east-1 functions: healthCheck: handler: app.health_check events: - http: path: api/health method: get getData: handler: app.get_data events: - http: path: api/data method: get
  4. Install Serverless Framework: npm install -g serverless
  5. Deploy the service: serverless deploy

After deployment, you’ll receive endpoints for your Lambda functions that can be accessed via HTTP requests.

Using Flask with Lambda via AWS Lambda Web Adapter

For more complex applications, you might want to run Flask on Lambda:

  1. Install required packages: pip install flask aws-lambda-web-adapter
  2. Create a Flask application (app.py): from flask import Flask, jsonify from aws_lambda_web_adapter.adapters.flask_adapter import FlaskLambdaAdapter app = Flask(__name__) @app.route('/api/health') def health_check(): return jsonify({"status": "healthy"}) @app.route('/api/data') def get_data(): return jsonify({ "items": [ {"id": 1, "name": "Item 1"}, {"id": 2, "name": "Item 2"}, {"id": 3, "name": "Item 3"} ] }) # For local development if __name__ == '__main__': app.run(debug=True) else: # For Lambda deployment lambda_handler = FlaskLambdaAdapter(app).get_lambda_handler()
  3. Update the serverless.yml: service: flask-lambda-api provider: name: aws runtime: python3.9 region: us-east-1 functions: api: handler: app.lambda_handler events: - http: path: /{proxy+} method: any
  4. Package dependencies: pip install -r requirements.txt -t package/ cp app.py package/ cd package zip -r ../deployment.zip . cd ..
  5. Deploy using AWS CLI: aws lambda create-function \ --function-name flask-lambda-api \ --runtime python3.9 \ --handler app.lambda_handler \ --role <your-lambda-execution-role-arn> \ --zip-file fileb://deployment.zip

Managing Cold Starts

Lambda functions experience “cold starts” when they haven’t been used recently. To mitigate this:

  1. Use provisioned concurrency: functions: api: handler: app.lambda_handler provisionedConcurrency: 5
  2. Optimize package size:
    • Include only necessary dependencies
    • Use Lambda Layers for common libraries
    • Minimize code size and initialization time
  3. Keep the runtime warm with scheduled pings: functions: warmup: handler: warmup.handler events: - schedule: rate(5 minutes)

Containerized Deployment with Docker on AWS

Docker containers provide consistent environments and efficient resource utilization. AWS offers several services for deploying containers.

Creating a Dockerized Flask Application

First, let’s containerize our Flask application:

  1. Create a Dockerfile: FROM python:3.9-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . EXPOSE 5000 CMD ["gunicorn", "--bind", "0.0.0.0:5000", "app:app"]
  2. Build and test locally: docker build -t flask-app . docker run -p 5000:5000 flask-app

Deploying on Amazon ECR and ECS

Let’s deploy our container using Amazon Elastic Container Registry (ECR) and Elastic Container Service (ECS):

  1. Create an ECR repository: aws ecr create-repository --repository-name flask-app
  2. Authenticate Docker to ECR: aws ecr get-login-password | docker login --username AWS --password-stdin <aws-account-id>.dkr.ecr.<region>.amazonaws.com
  3. Tag and push the image: docker tag flask-app:latest <aws-account-id>.dkr.ecr.<region>.amazonaws.com/flask-app:latest docker push <aws-account-id>.dkr.ecr.<region>.amazonaws.com/flask-app:latest
  4. Create an ECS cluster: aws ecs create-cluster --cluster-name flask-cluster
  5. Create a task definition (task-definition.json): { "family": "flask-app", "networkMode": "awsvpc", "executionRoleArn": "arn:aws:iam::<account-id>:role/ecsTaskExecutionRole", "containerDefinitions": [ { "name": "flask-app", "image": "<aws-account-id>.dkr.ecr.<region>.amazonaws.com/flask-app:latest", "essential": true, "portMappings": [ { "containerPort": 5000, "hostPort": 5000, "protocol": "tcp" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/ecs/flask-app", "awslogs-region": "<region>", "awslogs-stream-prefix": "ecs" } } } ], "requiresCompatibilities": ["FARGATE"], "cpu": "256", "memory": "512" }
  6. Register the task definition: aws ecs register-task-definition --cli-input-json file://task-definition.json
  7. Create a service: aws ecs create-service \ --cluster flask-cluster \ --service-name flask-service \ --task-definition flask-app \ --desired-count 1 \ --launch-type FARGATE \ --network-configuration "awsvpcConfiguration={subnets=[<subnet-id>],securityGroups=[<security-group-id>],assignPublicIp=ENABLED}"

Using AWS App Runner for Simplified Deployments

AWS App Runner offers an even simpler way to deploy containerized applications:

  1. Create an App Runner service using the AWS Console or CLI: aws apprunner create-service \ --service-name flask-app \ --source-configuration "ImageRepository={ImageIdentifier='<aws-account-id>.dkr.ecr.<region>.amazonaws.com/flask-app:latest',ImageConfiguration={Port=5000},ImageRepositoryType='ECR'}" \ --instance-configuration "Cpu='1 vCPU',Memory='2 GB'"
  2. Set up auto-scaling (optional): aws apprunner update-service \ --service-arn <service-arn> \ --auto-scaling-configuration-arn <auto-scaling-configuration-arn>

Using Infrastructure as Code with AWS CDK

For production deployments, consider using AWS Cloud Development Kit (CDK) to define infrastructure as code:

# app.py
from aws_cdk import (
    core,
    aws_ec2 as ec2,
    aws_ecs as ecs,
    aws_ecr as ecr,
    aws_ecs_patterns as ecs_patterns,
)

class FlaskAppStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Create a VPC
        vpc = ec2.Vpc(self, "FlaskAppVPC", max_azs=2)

        # Create an ECS cluster
        cluster = ecs.Cluster(self, "FlaskAppCluster", vpc=vpc)

        # Create a Fargate service
        fargate_service = ecs_patterns.ApplicationLoadBalancedFargateService(
            self, "FlaskAppService",
            cluster=cluster,
            cpu=256,
            memory_limit_mib=512,
            desired_count=2,
            task_image_options=ecs_patterns.ApplicationLoadBalancedTaskImageOptions(
                image=ecs.ContainerImage.from_ecr_repository(
                    repository=ecr.Repository.from_repository_name(
                        self, "FlaskAppRepo", "flask-app"
                    )
                ),
                container_port=5000,
            ),
            public_load_balancer=True
        )

        # Auto-scaling
        scaling = fargate_service.service.auto_scale_task_count(
            max_capacity=10
        )
        scaling.scale_on_cpu_utilization(
            "CpuScaling",
            target_utilization_percent=70,
            scale_in_cooldown=core.Duration.seconds(60),
            scale_out_cooldown=core.Duration.seconds(60)
        )

app = core.App()
FlaskAppStack(app, "FlaskAppStack")
app.synth()

To deploy:

cdk deploy

Choosing the Right Deployment Strategy

Each deployment approach has its own strengths:

Choose EC2 when you need:

  • Complete control over the server environment
  • Long-running processes
  • Specialized system requirements
  • CPU-intensive applications

Choose Lambda when you need:

  • Cost-efficiency for variable workloads
  • Automatic scaling to zero
  • Event-driven architecture
  • Minimal operational overhead

Choose Docker containers when you need:

  • Consistent environments
  • Microservice architecture
  • Portability across environments
  • Better resource utilization

Best Practices for AWS Python Deployments

Regardless of your deployment method, follow these best practices:

  1. Environment Configuration:
    • Use AWS Parameter Store or Secrets Manager for sensitive values
    • Never hardcode credentials in your application
    import boto3 def get_config(param_name): ssm = boto3.client('ssm') response = ssm.get_parameter(Name=param_name, WithDecryption=True) return response['Parameter']['Value'] # Usage db_password = get_config('/app/production/db_password')
  2. Monitoring and Logging:
    • Integrate CloudWatch for logs and metrics
    • Set up alarms for critical thresholds
    • Use structured logging for better searchability
  3. Security:
    • Follow the principle of least privilege for IAM roles
    • Enable AWS WAF for API security
    • Regularly update dependencies
  4. Cost Optimization:
    • Right-size your resources
    • Use auto-scaling to match capacity with demand
    • Consider reserved instances for EC2 or Savings Plans for predictable workloads
  5. Performance:
    • Set up CloudFront as a CDN
    • Use ElastiCache for caching
    • Optimize database queries

Conclusion

AWS offers multiple paths for deploying Python backends, each with distinct advantages. EC2 provides maximal control, Lambda offers serverless simplicity, and Docker brings consistency and portability. By understanding these options, you can choose the best approach for your specific requirements.

Remember that the most effective deployment strategy often combines multiple approaches. For example, you might use Lambda for API endpoints, Docker containers for background processing, and EC2 for database operations.

What deployment strategy are you using for your Python backends? Share your experiences in the comments below!

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA ImageChange Image