A robust, secure, and easily deployable image resizing service that resizes, optimizes, and caches images on "the edge," on the fly, built on AWS Serverless technologies. Served by CloudFront via an Origin Access Identity. Executed on Lambda@Edge. Backed by S3. Protected by AWS WAF. Provisioned via CloudFormation. Deployed by the AWS SAM CLI.

Overview

Image Flex

A robust, secure, and easily deployable image resizing service that resizes, optimizes, and caches images on "the edge," on the fly, built on AWS Serverless technologies. Served by CloudFront via an Origin Access Identity. Executed on Lambda@Edge. Backed by S3. Protected by AWS WAF. Provisioned via CloudFormation. Built and deployed by the Serverless Application Model (SAM) CLI.

Image Flex system diagram

Resized images will be converted to WebP format if the image request includes a valid Accepts header with "webp" listed in its value.

The original inspiration for this application came from this AWS blog post I read a few years back. The article intended to provide a [semi-]working example, which was far from being suitable for a production environment.

Prerequisites

Note that this is a production-ready application, not a tutorial. This document assumes you have some working knowledge of AWS, CloudFormation and the Serverless Application Model (SAM), AWS Lambda, S3, Node.js, NPM, and JavaScript.

Requirements

  1. Node.js v12.x (this is the latest version supported by Lambda@Edge). It's recommended to use Node Version Manager, which allows one system to install and switch between multiple Node.js versions.
  2. An AWS account.
  3. The AWS CLI
  4. The AWS SAM CLI

Be sure to configure the AWS CLI:

$ aws configure

For detailed instructions on setting up the AWS CLI, read the official AWS CLI documentation.

Quickstart

Deploy the whole service in 2 commands! Run the setup and update NPM scripts, passing a name for your execution environment (see Setting the execution environment). For a detailed explanation of these commands, see the section Building and Deploying.

$ npm run setup -- dev
$ npm run update -- dev
  1. The setup NPM script will create the CloudFormation deployment bucket. You only need to run this command once per execution environment.
  2. The update NPM script will build, package, and deploy the application stack to CloudFormation using the AWS SAM CLI. When the script is finished, it will print an "Outputs" section that includes the "DistributionDomain," which is the URL for your CloudFront distribution (e.g., [Distro ID].cloudfront.net). Note this value for later, as it is how you will access the service.

These scripts take an optional argument to indicate the execution environment. If you don't set the execution environment, the default of "dev" will be used. For info on setting the execution environment, see Setting the execution environment.

Example:

$ npm run setup -- staging
$ npm run update -- staging

Usage

Using an Image Flex implementation is easy. Once the infrastructure has spun up, simply upload your raw, unoptimized images to the S3 bucket root. You can then access those files directly, or pass a w (width) query string parameter to fetch a resized and optimized copy, which also gets stored in the S3 bucket and cached in CloudFront.

Example:

Suppose that you drop a 1600x900-pixel image named myimage.png into the created S3 bucket. You can now load this exact image in the browser via the distribution domain:

1600x900 pixels

https://[Distro ID].cloudfront.net/myimage.png

Using this full-resolution, unoptimized image would have negative performance impacts.

Resizing your images

w parameter

Now suppose that you want to load that image at 400 pixels width, maintaining the aspect ratio. It's as easy as adding the ?w=400 query string parameter.

400x225 pixels

https://[Distro ID].cloudfront.net/myimage.png?w=400

This will return a resized and optimized image (WebP, if supported by the browser).

h parameter

You can also add an h query string parameter to set the height. Note that changing the aspect ratio will clip the image (like object-fit: cover in CSS), not stretch or squash the image.

400x400 pixels, clipped

https://[Distro ID].cloudfront.net/myimage.png?w=400&h=400

How It Works

The fully actioned (built, packaged, and deployed) SAM template will result in a CloudFormation stack of resources being created across numerous AWS services (see the following table).

Any named resources will have the name prepended with the name of the stack, which itself is assembled from the application (image-flex), your AWS account ID, and the execution environment ("dev" by default). Example stack name: image-flex-412342973409-prod Example S3 bucket name: image-flex-412342973409-prod-images

Resource Type Resource Name Description
AWS WAF Web ACL [Stack Name]-WebAcl Defends the application from common web exploits by enforcing various access rules. This application implements AWS's Core Rule Set.
CloudFront Distribution N/A Content Delivery Network (CDN) to cache images at locations closest to users.
Logging S3 Bucket [Stack Name]-cflogs Stores the compressed CloudFront logs.
Hosting S3 Bucket [Stack Name]-images Serves as the CloudFront origin, storing the original image assets in the root, and resized image assets within subdirectories by width.
Origin Access Identity N/A Restricts direct access to the S3 bucket content, only allowing the CloudFront distribution to read and serve the image files.
Viewer Request Lambda@Edge [Stack Name]-UriToS3Key Responds to the "viewer request" CloudFront trigger, and will reformat the requested URI into a valid S3 key expected by the S3 bucket. Example: /image.png?w=300 => /300/image.webp
Origin Response Lambda@Edge [Stack Name]-GetOrCreateImage Responds to the "origin response" CloudFront trigger, and:
  1. If the requested image in the requested size is found, return it.
  2. Otherwise, if the requested image in the requested size is not found, attempt to create an image in the requested size from the base image.
  3. Otherwise, if the base image is not found, return HTTP status 404: Not Found.

Building and Deploying

The following NPM scripts are available:

  1. setup
  2. build
  3. package
  4. deploy
  5. update

Each NPM script calls a shell script of the same name in the /bin directory.

ℹ️ Setting the execution environment

These scripts (except for build) all run within the context of an execution environment (e.g., dev, staging, prod, etc.). This will be appended to the name of your Image Flex-based application in CloudFormation.

There are 2 ways to set the execution environment. If you don't explicitly set it via one of these methods, the default environment "dev" will be used.

To set the execution environment:

  1. Via the IF_ENV environment variable.
  2. By passing the [-- env] argument when calling the build/deploy scripts.

Note that if you both set the IF_ENV environment variable and pass this argument via the command line, the command line argument will take priority.

via environment variable

You can set the execution environment for all scripts by setting the IF_ENV environment variable.

Example: For MacOS:

export IF_ENV=prod

For Windows (development is untested on Windows):

setx IF_ENV "prod"

and then run the scripts, affecting your "prod" environment

npm run setup
npm run update

via the command line

Alternately, the setup, package, deploy, and update scripts accept an optional command line argument to indicate the current execution environment (e.g., dev, staging, prod, etc.).

Examples:

  • $ npm run update -- dev
  • $ npm run update -- staging
  • $ npm run update -- prod
  • $ npm run update -- bills-test

1. Setup

$ npm run setup [-- env]

Creates the CloudFormation deployment S3 bucket. SAM/CloudFormation will upload packaged build artifacts to this bucket to later be deployed. You only need to run this command once per execution environment.

2. Update

$ npm run update [-- env]

A convenience script that runs npm run build, npm run package, and npm run deploy in order.


These are generally only called directly when debugging.

3. Build

$ npm run build

Installs and builds the dependencies for the GetOrCreateImage Lambda function using a Docker container built on the lambci/lambda:build-nodejs12.x Docker container image.

4. Package

$ npm run package [-- env]

Packages (zips) the functions and built dependencies, and uploads the artifacts to the deployment bucket.

5. Deploy

$ npm run deploy [-- env]

Deploys the application as defined by the SAM template, creating or updating the resources.

Linting

Linting is instrumented via ESLint using Standardx (JavaScript Standard Style). To execute linting, run the following:

npm run lint

Testing

Unit tests are instrumented via Jest.

npm run test

Make it fully production-ready

While these steps are in no way required, here are some recommendations for a rock-solid, production ready implementation.

1. Use a CNAME

In the SAM template, under the Distribution resource, you can uncomment the following lines to use a CNAME instead of the *.cloudfront.net distribution domain.

# Uncomment the next two lines to use a custom CNAME (must be configured in Route 53 or another DNS provider).
Aliases:
  - YOUR CNAME HERE

Be sure to replace YOUR CNAME HERE with your actual CNAME, and ensure that CNAME is created in Route 53 (or another DNS provider).

2. Add an your own SSL certificate for HTTPS

By default, this application will use the default CloudFront certificate for SSL/TLS. However, if you configure an Alias per the instructions above, it is required that you use your own certificate for SSL/TLS. In the SAM template, under the Distribution resource, make the following changes to configure the distribution to use your own certificate stored in Certificate Manager.

Change...

ViewerCertificate:
  CloudFrontDefaultCertificate: true

To...

ViewerCertificate:
  CloudFrontDefaultCertificate: false
  AcmCertificateArn: YOUR CERTIFICATE MANAGER ARN HERE
  SslSupportMethod: "sni-only"

Be sure to replace YOUR CERTIFICATE MANAGER ARN HERE with the ARN of your certificate.

Comments
  • [Question] Why there is need to save processed images to another s3 instead of keeping it at CF cache?

    [Question] Why there is need to save processed images to another s3 instead of keeping it at CF cache?

    Hi,

    I'm curios why did you decide to save processed/optimized images to another s3 instead of keeping it at CloudFront cache? What if I want to invalidate cache of the image, because content of some specific image changed without changing name/path? Or if I deployed new version of the my AcmeImageFlex with different optimization rules and want to invalidate all webp images in the project.

    So what would be benefit of using separate s3 to store processed images? At CF level I can create invalidation at least by file path.

    On separate note, great solution, I've learned that much from it and in the process of integrating in our company's production. :)

    opened by diimpp 6
  • Size Issue

    Size Issue

    Hie the solution is working great but rather I noted a huge growth in my page size after inspecting thoroughly I found out that if I upload a jpeg with 1200x800 size 88kb then try to resize it to ?w=500 the size actually grows from 88kb to 265kb when it is supposed to have gone down. I tried to change the sharp quality to less than 50 but did not change a thing. thank you bro please look into it.

    opened by yomp-admin 5
  • Error when trying to create a bucket using custom region.

    Error when trying to create a bucket using custom region.

    An error occurred (IllegalLocationConstraintException) when calling the CreateBucket operation: The unspecified location constraint is incompatible for the region specific endpoint this request was sent to.

    opened by yomp-admin 1
  • Error if image is not in the bucket root

    Error if image is not in the bucket root

    original URL: distribution/uploads/test.jpeg?w=200 Error while getting source image object "/uploadstest.jpeg": NoSuchKey: The specified key does not exist.

    If the image is in the subfolder i get the following error how to fix it.

    opened by yomp-admin 1
  • Feat/cloudfront versioning

    Feat/cloudfront versioning

    opened by esc5221 0
  • Add configurable AWS region

    Add configurable AWS region

    Addresses Issue https://github.com/HoraceShmorace/Image-Flex/pull/6

    Allow region to be either:

    1. Set to the value of the AWS_REGION environment variable.
    2. Set to the value of the IF_REGION environment variable.
    3. Passed in as a second command-line argument to the setup, deploy, update NPM scripts.
    opened by HoraceShmorace 0
  • allow switch region out of east-us-1

    allow switch region out of east-us-1

    allow switch s3 bucket out of east-us-1 region

    • go to Image-Flex-master/src/GetOrCreateImage folder to manually change the region in GetOrCreateImage.js file
    • allow bucket access on lambda role
    opened by ziling777 0
  • VipsJpeg: Insufficient memory

    VipsJpeg: Insufficient memory

    Hi, I'm facing the following error with some images that are a little bit heavy, around 5MB. Did you face this error too? Or do you know a workaround for this? Unfortunately, the memory limit to lambda@edge is 256MB :(

    {
        "errorType": "Error",
        "errorMessage": "VipsJpeg: Insufficient memory (case 4)\n",
        "stack": [
            "Error: VipsJpeg: Insufficient memory (case 4)",
            ""
        ]
    }
    
    opened by leomeneguzzi 5
  • fix: Fixed issue where accessing an inexistent file returned 503

    fix: Fixed issue where accessing an inexistent file returned 503

    When file doesn't exist and S3 returns 403 in consequence, GetOrCreate lambda returns an error (cannot find method "replace" of undefined) and it causes a 503 Bad Gateway response.

    This is because it tries to read from "sourceImage" which comes from querystring but querystring is actually undefined.

    opened by LeandroDG 0
Owner
Horace Nelson
React. Serverless/microservices. PWAs. DApps. AWS certified. Scrum certified.
Horace Nelson
awsrun 189 Jan 3, 2023
AWS Lambda & Serverless - Developer Guide with Hands-on Labs. Develop thousands line of aws lambda functions interact to aws serverless services with real-world hands-on labs

AWS Lambda & Serverless - Developer Guide with Hands-on Labs UDEMY COURSE WITH DISCOUNTED - Step by Step Development of this Repository -> https://www

awsrun 35 Dec 17, 2022
Digital Identifier is a secure, decentralized, anonymous and tampered proof way of maintaining and verifying all essential identity-based documents to create a unique digital identity of a person.

Digital Identifier ?? To design and develop a secure, decentralized, anonymous and tampered proof way of maintaining and verifying all essential ident

Mukul Kolpe 4 Dec 17, 2022
It shows how to escape cross-origin issues for web client and API server using CloudFront routing.

AWS CloudFront의 URL Routing을 이용한 Web Client 및 API Server 구현 여기서는 CliendFront의 URL Routing을 이용하여 Web Client와 API Server를 구현하고자 합니다. Web Client는 Amazon

John Park 4 Nov 20, 2022
MerLoc is a live AWS Lambda function development and debugging tool. MerLoc allows you to run AWS Lambda functions on your local while they are still part of a flow in the AWS cloud remote.

MerLoc MerLoc is a live AWS Lambda function development and debugging tool. MerLoc allows you to run AWS Lambda functions on your local while they are

Thundra 165 Dec 21, 2022
A quickstart AWS Lambda function code generator. Downloads a template function code file, test harness file, sample SAM deffiniation and appropriate file structure.

Welcome to function-stencil ?? A quickstart AWS Lambda function code generator. Downloads a template function code file, test harness file, sample SAM

Ben Smith 21 Jun 20, 2022
Student reviews for OMS courses. Built with NextJS and Typescript. Backed by Sanity CMS. Deployed on Vercel.

This is a Next.js project bootstrapped with create-next-app. Getting Started First, run the development server: npm run dev # or yarn dev Open http://

OMS Tech 27 Dec 3, 2022
Piccloud is a full-stack (Angular & Spring Boot) online image clipboard that lets you share images over the internet by generating a unique URL. Others can access the image via this URL.

Piccloud Piccloud is a full-stack application built with Angular & Spring Boot. It is an online image clipboard that lets you share images over the in

Olayinka Atobiloye 3 Dec 15, 2022
Learn Web 2.0 and Web 3.0 Development using Next.js, Typescript, AWS CDK, AWS Serverless, Ethereum and AWS Aurora Serverless

Learn Web 2.0 Cloud and Web 3.0 Development in Baby Steps In this course repo we will learn Web 2.0 cloud development using the latest state of the ar

Panacloud Multi-Cloud Internet-Scale Modern Global Apps 89 Jan 3, 2023
Webhook service built with serverless technologies

Webhook service built with serverless technologies

Luciano Pellacani Franca 10 May 13, 2022
A serverless AWS expense tracker API. AWS Lambda functions, API gateway, and Dynamodb are among the ingredients.

AWS-Serverless-API A serverless AWS expense tracker API. AWS Lambda functions API gateway Dynamodb Endpoints Create a new expense: Method: POST Body f

Ondiek Elijah Ochieng 1 Jul 16, 2022
☁ ⚡ Serverless v2/v3 plugin to add custom dependsOn to CloudFormation resouces.

serverless-custom-depends-on Serverless v2/v3 plugin to add custom dependsOn to CloudFormation resouces. What it does It helps you to add the "Depends

Alexsandro G Bezerra 5 Dec 23, 2022
Under the Sea is an official AWS workshop delivered by AWS SAs and AWS Partners to help customers and partners to learn about AIOps with serverless architectures on AWS.

Under the Sea - AIOps with Serverless Workshop Under the Sea is an exciting MMORPG developed by the famous entrepreneur behind Wild Rydes, the most po

AWS Samples 4 Nov 16, 2022
Get-A-Room example application using Domain Driven Design and Clean Architecture. Written in TypeScript and deployed to AWS with a serverless stack.

Domain Driven Microservices on AWS in Practice This project provides a Domain Driven Design & Clean Architecture-informed, multi-service event-driven

Mikael Vesavuori 5 Dec 31, 2022
Manage GitHub resources like repositories, teams, members, integrations and workflows with the AWS CDK as Custom Resources in CloudFormation.

CDK Github Manage GitHub resources like repositories, teams, members, integrations and workflows with the AWS CDK as Custom Resources in CloudFormatio

Pepperize 8 Nov 25, 2022
AWS SAM project that adds the snippets from serverlessland.com/snippets as saved queries in CloudWatch Logs Insights

cw-logs-insights-snippets Serverlessland.com/snippets hosts a growing number of community provided snippets. Many of these are useful CloudWatch Logs

Lars Jacobsson 12 Nov 8, 2022
MidJourney is an AI text-to-image service that generates images based on textual prompts. It is often used to create artistic or imaginative images, and is available on Discord and through a web interface here.

Midjourney MidJourney is an AI text-to-image service that generates images based on textual prompts. It is often used to create artistic or imaginativ

Andrew Tsegaye 8 May 1, 2023
🚀 Using top-level await in AWS Lambda with TypeScript, esbuild and Serverless Framework

?? Top-level await in AWS Lamba with TypeScript Articles https://dev.to/oieduardorabelo/top-level-await-in-aws-lamba-with-typescript-1bf0 https://medi

Eduardo Rabelo 17 Nov 23, 2022
Pfapi plugin uses local and redis caches to achieve single digit milliseconds on average api response time.

Strapi plugin pfapi Pfapi plugin provides configurable, secure and fast API services. APIs are configurable through the admin panel with components an

null 5 Sep 17, 2022