In this assignment, I implemented smooth face morphing between two images. I also calculated the mean face of the Danish, extrapolated it for a caricature, played with the perceived gender of my face, and made a smashing music video.
As pixel coordinates span a linear subspace, we may change their relationships and create "warped" images (in this case, used for morphing faces to each other.)
The first step is to select a bunch of matching image correspondance pairs. Here, we will use Bjoern and Eric as our guinea pigs. After we've selected the points and add in the corner points of the image, we can define an alpha to interpolate to a common point location (such as alpha=.5, a midpoint). We then use Delaunay triangulation on these points to get a nice mesh of triangles to manipulate: since it's the dual of Voronoi cells, it makes triangles fat to help prevent non-local warping (aka ugly results).
Once we get the triangles, we can use some linear algebra to compose their affine transformation matrix (in homogenous coordinates, which means the last row is 0s except a 1 in the right corner). As we learned in class, going from the source image to the target image might result in some sampling issues and blank pixels: instead, we will take the inverse of this transformation matrix and apply it to our desired image pixel locations, to get the pixel locations in the original image where we should sample our colors from.
Since the results of the matrix multiplication usually were not integer values like pixel locations, I just rounded to the nearest neighbor. Sadly, this occasionally produces some visible artifacts.
After we have the images warped, we can cross-fade their colors (again with some alpha) to get a combined photo! Were we in want of an animation, we can get multiple frames, and vary alpha from 0 to 1 in frame number of steps.
My point selection probably wasn't the best as Bjoern had an ear and Eric didn't, so there are some artifacts.
Below are some example animations, their original photos, and their middle frames. If you want to see all 45 frames of the Bjoern/Eric transition, check out this Dropbox folder.
I have a lot of feelings about this. Come talk to be about Hannigram.
I used the 37 faces provided in this dataset to calcualte the mean face of some Danish people. I did this by first calculating the average shape, and warping each individual person's face into that shape. You can see all 37 here, and below are 3 I found interesting. Note that the control points were only defined on the face, so there are a lot of strange forehead warps.
After that, it was a simple matter of averaging. I warped my face shape into that of the Danish, and then the average Danish person me.
Averages remove individual blemishes and present us with nice, generic symmetry.
Me with the average Danish facial structure.
The average Danish person with my facial structure.
To produce caricatures of people, I first calculated the difference between someone's face shape and the average face shape from a population. I then weighted this by an alpha and added it back onto the original face shape, and morphed to this new shape.
On the same vein as caricatures, I took the average mainlander Chinese man and Chinese woman to see what I would look like if my face weren't so deliberately gender neutral.
Male me = bushier eyebrows...
Female me = higher cheekbones...
This is probably the pinnacle of my CS career. I took Neil Cicierega's mash up of a John Lennon and Smash Mouth song and, well...just watch it. If you would like to the video in gif form, it's here.
I participated in it by putting my face in the pool.
This project was super cool, and not just because I could become Sufjan Stevens. It's incredibly beautiful how we can triangulate things relatively arbitrarily yet their boundaries will be continuous. If I had more time, I would try to make my code faster with smarter matrix masking--it took almost an hour to "render" the 55 frames for the video. Solidarity to the rendering folks out there.