Automatically Create Image Montages with PhotoSketch

We've gotten so used to technology that there isn't that much that impresses us these days. But with so much technology and computing power at our fingertips it's getting harder to just realize what can already be done not what the future will bring. Drawing a rough sketch and then having a computer generate a photo based on that sketch sounds a little sci-fi but it's not only possible, it has actually been done. A group of computer science students in China have created PhotoSketch, a project that blends images found online according to the users' drawings.

The process, while outlined in the video below and detailed in the documentation and the source code they provided, is nothing short of magical. A user creates a rough hand-drawn image placing objects and people in a scene they choose. The user has to label all of the objects but that's all the input PhotoSketch needs. Anything goes, really, even complex or composite objects, as the software just needs a basic shape and a tag describing what it is and it will take care of the rest.

It then scours the Internet looking for the photos that match the tags and the basic shapes. After finding several options for all of the objects in the sketch and the background it proceeds to find the best matches to stitch together using a specialized algorithm. With the images acquired, the software applies some pretty advanced photo manipulation techniques to blend the different components seamlessly. The results speak for themselves. The project's website is currently down due to high demand and the team is working on an online demo for PhotoSketch as well.

“We present a system that composes a realistic picture from a simple freehand sketch annotated with text labels. The composed picture is generated by seamlessly stitching several photographs in agreement with the sketch and text labels; these are found by searching the Internet,” the team explains in the project's abstract. “We also provide a novel image blending algorithm to allow seamless image composition. Each blending result is given a numeric score, allowing us to find an optimal combination of discovered images.”