1 Defining Correspondence

• To morph from Anne to Adams, we start by defining pairs of corresponding points by hand.
• I used given tools to create this correspondence, which is saved as point coordinates in a json file.
• In order to get the Delauney triangulation, I calculate the average point with average_points = (im1_points + im2_points) / 2
• Below is a display of the triangulation results on the two faces.

Anne with Delauney Triangulation

Adams with Delauney Triangulation

2 Computing the Mid-way Face

• Mid-way Face Calculation
• This the pre-step before computing the full morph sequence: we compute the mid-way face of images A and B.
• Compute the average shape by averaging each keypoint location of faces A and B.
• Warp both faces into the average shape.
• Average the colors of both images.
• Affine Warp Implementation
• Implement an affine warp for each triangle in the triangulation.
• Compute an affine transformation matrix between two triangles with this: A = computeAffine(tri1_pts,tri2_pts)
• Use a set of these transformation matrices to implement an inverse warp for all pixels.
• Generate a mask using polygons e.g., polygon in Python.
• Loop over triangles, not pixels.
• And...here is the result! Please just ignore the hair style issue, that's not an issue at all, in some sense~

Anne Original

Anne-Adams Mid-way Face

Adams Original

3 Morph Sequence!

• Just write a function to produce the morph for each frame:
• morphed_im = morph(im1, im2, im1_pts, im2_pts, tri, warp_frac, dissolve_frac)
• In total, I created 45 frames here, and each frame stops for 35 ms to create this GIF.
• <<<<<< Notice >>>>>> The GIF is 85M, which might take a long time to load, please download it directly here if you don't wanna wait: Download_GIF

From Anne to Adams

4 Mean Face of Population

• Dataset Selection
• I choose to use FEI Face Database
• In which, I choose frontalimages_spatiallynormalized_part1 and frontalshapes_manuallyannotated_46points.
• This dataset contains 50 men and 50 women's front normalized faces. For each numbered participant, a is a neutral expression face and b is a smiling face.
• Each face is annotated with 46 points following a specific order. See the image on the right for the sequence, which will be used for morphing from other faces into this dataset's faces.
• One thing to notice is that their annotations don't include the four points of the image, which would cause issue since I'm using mush that only calculate images within polygons!
• Before I manually added 4 annotated points to the image, the morphing image will only contain a polygon-shaped face, and black all around.
• Below are the typical images like in the dataset with my added points.

1a participant with triangulation

51b participant with triangulation

• Average Face
• After deviding the dataset into two types: neutral face and smiling face.
• Below is the result of average neutral, and average smiling face of this population.

Average Neutral Face

Average Smiling Face

• Morphing: dataset faces into average faces
• Morph the first few examples into neutral faces and smiling faces geometry respectively.
• Four images in each row:
• original neutral face    --->    morphed neutral face                original smiling face    --->    morphed smiling face

1a: original neutral

1a: morphed neutral

1b: original smiling

1b: morphed smiling

2a: original neutral

2a: morphed neutral

2b: original smiling

2b: morphed smiling

3a: original neutral

3a: morphed neutral

3b: original smiling

3b: morphed smiling

• Morphing: Elon face into average faces & the other way around
• This honestly looks creepy, so I'm not gonna use large images. Please see the right. --->

Elon to Average

Average to Elon

5 Caricatures: Extrapolating from the Mean

• Key formula used: (1-alpha)*mean_pts + alpha*individual_pts
• A negative alpha would emphasize the average face's feature, while a positive alpha would do the opposite.

Elon Original

\(\alpha = -0.5\)

\(\alpha = 1.5\)

6 Bells & Whistles: PCA

• Let's try to get a eigen faces basis from the FEI dataset, using PCA on all 100 neutral faces.
• Below shows the first 12 eigen faces. In total, I created 100 components (which is the maxium, and I expect near pefect reconstruction.)

• We first try to recontruct the faces, and then use a similar formula for caricature: (1+alpha) * original_coeffs where the original_coeffs is the list of coefficients of each eigen faces when projecting a target image into this Eigen space.
• We expect a negative alpha would emphasize the average face's feature, while a positive alpha would do the opposite.
• We test on 1a participant, since I am very familiar with this guy's face now...

1a Original

\(\alpha = -3\)

\(\alpha = 1.5\)

\(\alpha = 0\)

• This actually looks pretty good, since the \(\alpha = -3\) looks nothing like the guy, all the key features of 1a are almost reversed, while \(\alpha = 1.5\) clearly exagerated the key features like the shape and depth of this guy's nose and upper lip.

 

• Now let's try our beloved Elon's face. (Spoiler Alert: It's pooooor...)

Elon Original

\(\alpha = -0.5\)

\(\alpha = 1.5\)

\(\alpha = 0\)

• You should have noticed that even when alpha=0, meaning we are basically reconstructing, the image is not even half like Elon. At the same time, we've seen an almost perfect reconstruction of the 1a guy's face!
• Therefore, we know why: Elon's face does not lie in the space that these eigen spaces span, so many features of elon's face just cannot be reconstructed using merely linear combination of these eigen faces.