Introducing a Storage image processing extension which provides an extensive image processing API with 30+ image mutation operations from resizing and colour manipulation to overlaying text and compositing images.

The core image processing logic in the implementation is portable and therefore can be used in a variety of ways, e.g.,

• reacting to a storage event to automatically process an image (operations could be defined via a metadata key)
• adding image operations to Firestore and listening to a Firestore collection that would trigger these operations to be applied and output
• etc

but for this implementation, initially, interfacing with the image processing is provided as a URL safe HTTP GET API via a Cloud Function (and could be used for embedding e.g., <img src="<extensionFunctionUrl/process/...ops />).

Accessing GCS bucket objects is done via tokenated download urls, e.g., getDownloadUrl() on Firebase SDKs or providing a path to a public object on the bucket, e.g., /foo/bar.png. URLs & paths must be URL encoded.

Process Image URL Schema

The URL schema to apply operations is built into url parts separated by /, each segment is then in the following format;

/<operationName>~<optionName>:<optionValue>/

• A single operation and it's options are defined per url segment. Multiple options are separated by ~.
• Option values are automatically coerced into their appropriate types, e.g. '5' as a string would become 5 as a number, or '[1,2,3,4]' as a string would result in a JS array of numbers.
• Truthy boolean options can just specify the <optionName> only to have the value default to true (think flags).
• Operations that have no options or only have optional options can just specify the operation without options, e.g./negate/.
• An input operation must be the first operation specified and is required.
• An output operation must be the last operation specified and is required.

Having the URL schema as url segments vs query parameters allows us to easily have operation ordering and support using multiple operations of the same type in a chain of operations. Additionally makes options clearer that they are for a specific operation.

Example

URL:

/process/input~type:gcs~source:%2Fassets%2Fhexagons.jpeg/resize~width:500~height:500~fit:cover~position:left%20bottom/extend~all:20~background:tomato/extend~all:20~background:lightgreen/composite~top:20~left:26~tile~blend:screen~input:%2Fassets%2Finvertase.png/text~strokeWidth:4~top:75~left:75~font:95px%20tahoma~strokeColor:SeaGreen~textAlign:center~value:Hello%0DWorld~gravity:northwest~backgroundColor:pink~textColor:MediumSpringGreen~padding:15~borderWidth:5~borderColor:black/output~format:png

Output:

Supported operations

• input
• output
• affine
• blur
• clahe
• composite
  • experimental, needs some TLC around retrieving input sources
• extend
• extract
  • a.k.a crop
• flatten
  • e.g., can be used to remove transparency
• flip
• flop
• gamma
• median
• negate
• normalize
• resize
• rotate
• sharpen
• text
  • custom implementation since Sharp doesn't have a direct api for this - uses the composition API to overlay a generated SVG. This same kind of logic with overlaying SVGs could be used to do things like gradients & shapes
• trim
• convolve
• threshold
• boolean
• linear
• recomb
• modulate
• tint
• grayscale
• colorspace

Operations pending implementation

Not blocked or difficult by any means, just needs time invested to implement. These will be added in a future PR.

• [ ] removeAlpha
• [ ] ensureAlpha
• [ ] extractChannel
• [ ] joinChannel
• [ ] bandbool

TODO

• [x] Finish up composite - currently sourcing from local files for testing purposes, should source from same sources that the input operation supports.
• [x] Add remaining output operation format specific options, e.g., https://github.com/invertase/firebase-extensions/blob/storage-image-processing-api/extensions/storage-image-processing-api/functions/src/operations/output.ts#L32
• [x] Add all available operation types (Currently Prs)
• [x] Tests
• [ ] Documentation
• [ ] Input from GCS bucket path
• [ ] Investigate possible alternatives to output to a bucket rather than returning to the requesting client (maybe a POST api)

The current Api works as expected, however it is difficult to produce the query-string required to produce valid urls.

This PR looks to augment the current implementation by adding utility functions as a solution.

I have my images in subfolders on GCS / firebase storage with URL pattern /bucket/o/<folder>%2F<subfolder>%2F<file>.

When I parse my operations with encodeURIComponent it escapes the %2F correctly as %252F which is sent to the cloud function.

In the cloud function error logs I can see that the %252F is replaced by / instead of expected %2F which results in an invalid URL.

Using the variable of the final deployed function URL

If the extension is deployed twice to the same project the URL will contain the extension instance id.

Error: node_modules/@invertase/image-processing-api/dist/types/types.d.ts:2:19 - error TS2307: Cannot find module 'sharp' or its corresponding type declarations.

2 import sharp from 'sharp';