Content aware image cropping

Overview

smartcrop.js

Build Status

Smartcrop.js implements an algorithm to find good crops for images. It can be used in the browser, in node or via a CLI.

Example Image: https://www.flickr.com/photos/endogamia/5682480447/ by N. Feans

Note I'm currently working on a more advanced version of smartcrop.js based on machine learning. As part of that I'm looking for a large dataset of manually cropped images. If you know of such a dataset, please let me know.

Demos

Simple Example

// you pass in an image as well as the width & height of the crop you
// want to optimize.
smartcrop.crop(image, { width: 100, height: 100 }).then(function(result) {
  console.log(result);
});

Output:

// smartcrop will output you its best guess for a crop
// you can now use this data to crop the image.
{topCrop: {x: 300, y: 200, height: 200, width: 200}}

Download/ Installation

npm install smartcrop or just download smartcrop.js from the git repository.

Smarcrop requires support for Promises, use a polyfill for unsupported browsers or set smartcrop.Promise to your favorite promise implementation (I recommend bluebird).

Consider avoiding crops using dont-crop

If you are interested in using smartcrop.js to crop your images you should also consider to avoid cropping them by using dont-crop. Dont-crop gives you matching gradients and colors to pad and complement your images.

Example

Command Line Interface

The smartcrop-cli offers command line interface to smartcrop.js.

Node

You can use smartcrop from nodejs via either smartcrop-gm (which is using image magick via gm) or smartcrop-sharp (which is using libvips via sharp). The smartcrop-cli can be used as an example of using smartcrop from node.

Stability

While smartcrop.js is a small personal project it is currently being used on high traffic production sites. It has a basic set of automated tests and a test coverage of close to 100%. The tests run in all modern browsers thanks to saucelabs. If in any doubt the code is short enough to perform a quick review yourself.

Algorithm Overview

Smartcrop.js works using fairly dumb image processing. In short:

  1. Find edges using laplace
  2. Find regions with a color like skin
  3. Find regions high in saturation
  4. Boost regions as specified by options (for example detected faces)
  5. Generate a set of candidate crops using a sliding window
  6. Rank them using an importance function to focus the detail in the center and avoid it in the edges.
  7. Output the candidate crop with the highest rank

Face detection

The smartcrop algorithm itself is designed to be simple, relatively fast, small and generic.

In many cases it does make sense to add face detection to it to ensure faces get the priority they deserve.

There are multiple javascript libraries which can be easily integrated into smartcrop.js.

You can experiment with all of these in the smartcrop.js testbed

On the client side I would recommend using tracking.js because it's small and simple. Opencv.js is compiled from c++ and very heavy (~7.6MB of javascript + 900kb of data). jquery.facedetection has dependency on jquery and from my limited experience seems to perform worse than the others.

On the server side node-opencv can be quicker but comes with some annoying issues as well.

It's also worth noting that all of these libraries are based on the now dated viola-jones object detection framework. It would be interesting to see how more state of the art techniques could be implemented in browser friendly javascript.

Supported Module Formats

  • CommonJS
  • AMD
  • global export / window

Supported Browsers

See caniuse.com/canvas. A polyfill for Promises is recommended if you need to support old browsers.

API

smartcrop.crop(image, options)

Find the best crop for image using options.

image: anything ctx.drawImage() accepts, usually HTMLImageElement, HTMLCanvasElement or HTMLVideoElement.

Keep in mind that origin policies apply to the image source. You may not use cross-domain images without CORS clearance.

options: cropOptions

returns: A promise for a cropResult.

cropOptions

minScale: minimal scale of the crop rect, set to 1.0 to prevent smaller than necessary crops (lowers the risk of chopping things off).

width: width of the crop you want to use.

height: height of the crop you want to use.

boost: optional array of regions whose 'interestingness' you want to boost (for example faces). See boost;

ruleOfThirds: optional boolean if set to false it will turn off the rule of thirds composition weight.

debug (internal): if true, cropResults will contain a debugCanvas and the complete results array.

There are many more (for now undocumented) options available. Check the source and be advised that they might change in the future.

cropResult

Result of the promise returned by smartcrop.crop.

{
  topCrop: crop;
}

crop

An individual crop.

{
  x: 11, // pixels from the left side
  y: 20, // pixels from the top
  width: 1, // pixels
  height: 1 // pixels
}

boost

Describes a region to boost. A usage example of this is to take into account faces in the image. See smartcrop-cli for an example on how to integrate face detection.

{
  x: 11, // pixels from the left side
  y: 20, // pixels from the top
  width: 32, // pixels
  height: 32, // pixels
  weight: 1 // in the range [0, 1]
}

Note that the impact the boost has is proportional to it's weight and area.

Tests

You can run the tests using grunt test. Alternatively you can also just run grunt (the default task) and open http://localhost:8000/test/.

Benchmark

There are benchmarks for both the browser (test/benchmark.html) and node (node test/benchmark-node.js [requires node-canvas]) both powered by benchmark.js.

If you just want some rough numbers: It takes < 20 ms to find a square crop of a 640x427px picture on an i7. In other words, it's fine to run it on one image, it's suboptimal to run it on an entire gallery on page load.

Contributors

Ports, Alternatives

Version history

2.0.5

Fix TS1046: Top-level declarations in .d.ts files must start with either a 'declare' or 'export' modifier..

2.0.4

Typescript type definitions.

2.0.2

In short: It's a lot faster when calculating bigger crops. The quality of the crops should be comparable but the results are going to be different so this will be a major release.

1.1.1

Removed useless files from npm package.

1.1

Creating github releases. Added options.input which is getting passed along to iop.open.

1.0

Refactoring/cleanup to make it easier to use with node.js (dropping the node-canvas dependency) and enable support for boosts which can be used to do face detection. This is a 1.0 in the semantic meaning (denoting backwards incompatible API changes). It does not denote a finished product.

License

Copyright (c) 2018 Jonas Wagner, licensed under the MIT License (enclosed)

Comments
  • Making it smarter

    Making it smarter

    Whereto from here? Are there any efforts to make it smarter and get more accurate? It's already pretty good so far for 80% of the cases, but I would like it to reach a higher level of accuracy.

    question 
    opened by Charuru 12
  • Multiple Skin Colors

    Multiple Skin Colors

    Right now it seems that the skin color defaults to #C69170 and can only take one value. It may be useful to have multiple default values (since not everyone is white) and check all of them in skinDetect.

    opened by mynameisfiber 8
  • Cropped image dimensions by 30% is like 5% less size on memory

    Cropped image dimensions by 30% is like 5% less size on memory

    note: I know the main purpose of this, is to smart crop, and this is already great. Still I have another problem

    Context

    I want to use this amazing library for uploads in browser. I tried to grab code from testsuite and it works fine.

    Unfortunately I find in most cases:

    Problem

    cropped size in px / original size in px

    is always way less than

    cropped size in mb / original size in mb

    I wished smartcrop.js would let the image quality as is, in this case I think size on memory would drop relatively to the cropped size ( logically as images are raw data? ).

    Thanks a lot !!

    opened by bacloud23 6
  • Adds Typescript definitions

    Adds Typescript definitions

    Adds index.d.ts with Typescript typings for the library.

    index.d.ts is the default definitions locations. If the file is in the project root and is named index.d.ts, then there is no need to modify package.json.

    opened by goooseman 6
  • Add options object to iop.open call

    Add options object to iop.open call

    Some image processors need to know the type of the image in order to open them as a buffer. This is the case with lwip.

    To make this possible, we could pass the filename (or just the extension) to smartcrop as part of the options object, and then make it available to iop.open as a second argument. That way, the iop can decide how best to open the image when open() is called.

    (Does this make sense? 😅 )

    opened by eduardoboucas 5
  • Aspect ratio instead of pixel amount

    Aspect ratio instead of pixel amount

    I have a bunch of 3:2 images that I'd like to crop at 16:9 while keeping as much of the original image as possible. I'd like to do this locally using smartcrop-cli.

    Would this be possible using smartcrop.js? It would be a dream that comes true.

    EDIT: Every picture has different pixel size, but the same 3:2 aspect ratio. If that's not possible with smartcrop.js native functionality, maybe with a shell script as a wrapper? 🤔🤔

    opened by mrzool 4
  • Any way to crop the image into a circle, instead of a square?

    Any way to crop the image into a circle, instead of a square?

    I am looking to crop an image into a circle, hopefully using this library, or a combination of this (to detect a person's face), and then cropping that image into a circle (limitation of another library that doesn't allow css intervention on an image).

    opened by Bengejd 4
  • Content Security Policy

    Content Security Policy

    The crop fails when I add

    meta http-equiv="Content-Security-Policy" content="default-src 'self'"

    for security perposes regarding xss how can I be able to fix this???

    I think the cause of this is because your code creates script or img tags somewhere but I dont know where, can you help me with that please

    needs info 
    opened by Xaplink 4
  • Type Definition Error

    Type Definition Error

    Getting this in our builds:

    >>> project_name
    rm -rf dist/* && tsc
    Error: ../../common/temp/node_modules/.registry.npmjs.org/smartcrop/2.0.4/node_modules/smartcrop/index.d.ts(25,1): error TS1046: Top-level declarations in .d.ts files must start with either a 'declare' or 'export' modifier.
    

    Which seems to be coming from this:

    const smartcrop: {
      crop(image: CanvasImageSource, options: CropOptions): Promise<CropResult>;
    };
    export default smartcrop;
    

    I'm not a .d.ts expert - but looking at other examples (https://github.com/stephenmathieson/node-tlds/issues/11) this may be better done this way (based on their actual index.d.ts, not the suggestions, but I'm not sure and this is untested):

    declare const smartcrop: {
      crop(image: CanvasImageSource, options: CropOptions): Promise<CropResult>;
    };
    export = smartcrop;
    
    opened by Borvik 3
  • Base64

    Base64

    Hi! I try to use this library in React Native, but in react native i don't have HTMLImages for input. Can i pass base64 image or something else?

    Thanks!

    opened by AlanSaracho 3
  • Solution to this image?

    Solution to this image?

    Hey guys,

    I have this image: trouble image

    And this is the area smartcrop is giving me to use: croppy-buggy

    This is the first image that I'm getting this kind of result... Is there something I can do to avoid these?

    I know that if I use smartcrop + opencv I can boost the areas with faces and I can get this result: right-crop

    But I really don't want add opencv to my stack. It's too heavy I was hoping to get this fixed without using heavy artillery.

    Ideas?

    opened by igorescobar 3
  • is it possible to input raw data & get raw data as output?

    is it possible to input raw data & get raw data as output?

    I was wondering if this library supports a case of:

    I want to input containerWidth, containerHeight, as well imageWidth, imageHeight, and imageFacesOnScreen, and I would like to get back:

    imageScale, imageX, imageY (basically these 3 parameters help me to scale/reposition the image in the container) thus giving me the best "crop", in essence I am not sure how to create this algorithm, but I dont need any face detection algorithms as I already have them, and I dont need any bitmap manipulation as its all handled. I would essentially need "description" of how to position the face in a container the best way.

    opened by chr4ss12 1
  • cropping not remove empty zones and just minimize my image

    cropping not remove empty zones and just minimize my image

    I want the image to be 64x64 without transparency around and with content centered. But I only get a 64x64 minimized version of my image, without anything cropped.

    I made this reproduce repo

    I gave a try to my image in the online tool and the hot zones are correctly detected.

    Thanks for that great lib.

    opened by Fr33maan 0
  • Is it possible to force centering?

    Is it possible to force centering?

    I have an image

    01-once-you-see-what-nutella-is-made-up-of-you ll-never-want-to-eat-it-again-1024x683

    Smartcrop is shifting centering when I use more square aspect ratio

    800x600-01-once-you-see-what-nutella-is-made-up-of-you-ll-never-want-to-eat-it-again-1024x683

    Is it possible to set detection only for horizontal (top, bottom) so images doesn't shift center?

    opened by ChrisRobston 1
  • Is is possible to generate a percentage center point instead of absolute boundary?

    Is is possible to generate a percentage center point instead of absolute boundary?

    Hi, @jwagner

    This is an awesome library, thank you for creating it!

    I am considering to use the library in production environment, with responsiveness support. However, the absolute cropped boundary numbers are not very friendly to responsiveness. If the calculation result is a center point with percentage, we can use the following CSS styles to stick to that center point, then it will show good in PC view or mobile view.

    const style = {
      backgroundImage: `url("${imageURL}")`,
      backgroundPosition: `${centerPoint.x}% ${centerPoint.y}%`
    }
    

    Now, I am calculating the central point of the crop result as the center point. That is working but not really accurate.

    More ideas?

    opened by lijunle 3
  • Typoed coefficients in luminance calculation

    Typoed coefficients in luminance calculation

    The luminance calculation in smartcrop.js is currently:

    function cie(r, g, b) {
        return 0.5126 * b + 0.7152 * g + 0.0722 * r;
    }
    

    But going back to the HDTV Rec. 709 (either https://en.wikipedia.org/wiki/Rec._709 or the original https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.709-6-201506-I!!PDF-E.pdf ), these coefficients should be:

    0.2126 * r + 0.7152 * g + 0.0722 * b
    

    With two differences: first is that the r and b channels have been swapped, and second is the typo of 0.5126 for 0.2126 in the weight of the red channel. This should also be technically on linear RGB rather than sRGB (as you'll likely get from JS) but that's less of an issue.

    The current code weights blue ~7x higher than it should. I don't know whether it's worth changing smartcrop.js results at this point to fix it, though.

    bug 
    opened by pteichman 3
Releases(v2.0.5)
Jcrop - The Javascript Image Cropping Engine

Jcrop Image Cropping Plugin Jcrop is the quick and easy way to add image cropping functionality to your web application. It combines the ease-of-use o

Kelly Hallman 4.3k Dec 20, 2022
Simple Cropper is a jQuery plugin which gives you ability to add cropping functionality to your web application

Simple Cropper is a jQuery plugin which gives you ability to add cropping functionality to your web application. It uses html5 canvas to create cropped images and css3, so it only works on latest browsers.

null 1 Feb 15, 2022
optimize image & upload file to cloud as image bed with tiny image automic.

Rush! 图片压缩 & 直传图床工具 这是一个兴趣使然的项目, 希望 Rush! 能让这个世界的网络资源浪费减少一点点 下载 Downloads 获取最新发行版 功能 Features 拖拽批量压缩图片, 支持格式 jpg/png/gif Drop to optimize, jpg/png/gif

{ Chao } 3 Nov 12, 2022
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

The Apache Software Foundation 20.2k Jan 5, 2023
Context-aware smart contracts for blockchain IoT systems

Context-aware smart contracts for blockchain IoT systems It contains 2 directories: contracts: contains the ContextAwareSmartContract.sol contract wit

ibelab 6 Jun 17, 2022
An experimental framework-aware Firebase CLI

Firebase Experimental framework-aware CLI Usage $ npm i -g firebase-frameworks $ cd <MY-APP> $ firebase-frameworks init $ firebase-frameworks build $

null 146 Dec 27, 2022
🔥 Bon5R — Create static, blog-aware websites with pure MDX.

Bon5R Create static, blog-aware websites with pure MDX. Explore the docs » View Demo · Report Bug · Request Feature Table of Contents About The Projec

Manu Anish 3 Jul 5, 2022
Syntactically-aware grep for JavaScript and TypeScript

ESGrep Syntactically-aware grep for JavaScript and TypeScript Usage as a CLI Install it with npm install --global esgrep or the equivalent using pnpm/

Nino Filiu 8 Sep 28, 2022
Simple time-aware report bot for Telegram.

report bot Built using grammY. Simple time-aware report bot for Telegram. It listens for /report, /admin commands or @admin, @admins mentions in group

Dunkan 11 Oct 2, 2022
A progressively enhanced server-rendered form-aware web component counter using WebC

Eleventy WebC Number Counter Demo A progressively enhanced server-rendered form-aware web component counter using WebC. This enhances from an <input t

Eleventy 6 Nov 11, 2022
✂ An online image cropper for content creators

Introducing CropScore ✂️ Resolutions and aspect ratios should be the last things you worry about. About ?? CropScore is an online image cropper for co

Madza 49 Dec 30, 2022
🖼️ Image proxy for Next.js. Makes it possible to use dynamic domains in next/image component.

Next.js Image Proxy Image proxy for Next.js. Makes it possible to use dynamic domains in next/image component. ❔ Motivation This library makes it poss

Blazity 30 Dec 1, 2022
A Javascript library that discourages and prevents image theft/download by preventing client ability to retrieve the image.

ProtectImage.js ProtectImage.js is a Javascript library that helps prevent image theft by disabling traditional user interactions to download/copy ima

null 4 Aug 18, 2022
Simple JavaScript Library to add parallax image to background-image

Backpax Simple JavaScript Library to add parallax image to background-image Install $ npm install backpax --save Demo Demo page is here Usage If you

appleple 13 Oct 12, 2022
Piccloud is a full-stack (Angular & Spring Boot) online image clipboard that lets you share images over the internet by generating a unique URL. Others can access the image via this URL.

Piccloud Piccloud is a full-stack application built with Angular & Spring Boot. It is an online image clipboard that lets you share images over the in

Olayinka Atobiloye 3 Dec 15, 2022
Hashmat Noorani 4 Mar 21, 2023
Make drag-and-drop easier using DropPoint. Drag content without having to open side-by-side windows

Make drag-and-drop easier using DropPoint! DropPoint helps you drag content without having to open side-by-side windows Works on Windows, Linux and Ma

Sudev Suresh Sreedevi 391 Dec 29, 2022
📚 Share content for learn web development

Frontend Developer Content for learn web development Here we can learn a lot content for better a web developer Technologies HTML ✨ CSS ✨ JavaScript T

Juliâncio Carvalho 1 Dec 25, 2021