1 Fun Filters

1.1 FDO: Finite Difference Operator

In section 1.1, we will derive and test a naive finite difference filter to perform the classic edge detection task.
Finite difference operator is defined as:

\( \mathbf{D}_x = \begin{bmatrix} 1 & -1 \end{bmatrix}, \quad \mathbf{D}_y = \begin{bmatrix} 1 \\ -1 \end{bmatrix} \)

Then we convolve the image first with \(\mathbf{D}_x \) obtaining all vertical edges (img: grad_x), then with \(\mathbf{D}_y\) extracting all horizontal edges (img: grad_y).
Then, we compute the gradient magnitude with a simple \(\sqrt{\mathbf{D}_x^2 + \mathbf{D}_y^2}\), which gives the edge image (img: grad_mag).
Finally, choosing a threshold value of 0.28, we take the image with pixel value higher than threshold as the edge image (img: edge_image).

Original: Cameraman

Shape: (542, 540, 4)

grad_x

grad_y

grad_mag

edge_image

1.2 DoG: Derivative of Gaussian Filter

To get rid of noise, in section 2.1, we derive two ways:
Method 1:

First get a 2D Gaussian kernel using an outer product between two 1D Gaussian obtained from cv2.getGaussianKernel().
Then, convolve the image with the Gaussian kernel to get a blurred image.
Next, we compute the gradient magnitude of the blurred image using the same method as in section 1.1.
Finally, we choose a threshold to get the edge image.

blurred_image

grad_x from blurred_image

grad_y from blurred_image

grad_mag from blurred_image

edge_image

Method 2:

First, we use the Gaussian to convolve with \(\mathbf{D}_x \) and \(\mathbf{D}_y \) to obtain two blurred filters.
Second, we convolve the original image with these two blurred filters respectively to obtain (img: grad_x_DoG) and (img: grad_y_DoG)
Then, we compute the gradient magnitude as in section 1.1 (img: grad_mag_DoG).
Finally, we choose a threshold to get the edge image (img: edge_image_DoG).

DoG_x Filter

DoG_y Filter

blurred_x

grad_x_DoG

grad_y_DoG

grad_mag_DoG

edge_image_DoG

It's easily observable that edge_image from Method 1 looks almost identical to edge_image_DoG from Method 2, proving that convolution is indeed associative and commutative.

2 Fun with Frequencies!

2.1 Image "Sharpening"

How to sharpen an image? The idea is pretty simple in this implementation.
First, we blur the image using a Gaussian filter with a kernel size of 5.
Then, we subtract the blurred image from the original image to get the high frequency features of the image.
Finally, we add the high frequency features back to the original image to get the sharpened image with fake but convincing details. The intensity of the weighted high frequency image is gauged by alpha.

Sample Image: Taj

Original

alpha = 1

alpha = 2

alpha = 10

Self Selected: Gru

Original

alpha = 1

alpha = 4

alpha = 10

Self Selected: Crease Pattern of Forest Scorpion 3.0 by Kota Imai

Original

alpha = 1

alpha = 2

alpha = 10

Notice the color of blue and red seems to disappear after sharpening! This is one typical drawback of this kind of sharpening, with high frequency part still being colored, the "sharpened" part actually becomes darker and darker as alpha goes up.

Evaluation: Blurred and Resharpened Tomato

Original

Blurred

Resharpened: alpha = 2

Resharpened: alpha = 20

Observation: most details are recovered but really detailed and good features like the fluff texture on the stem of the tomato is permenantly lost.

2.2 Image Hybriding

How to make a hybrid image? The idea is also simple. (But the implementation is lot of hard work).
First, we choose img1 as the one to keep high frequency features and img2 as the one to keep low frequency features.
For img1, we blur it with a hand-picked kernel sigma, and keep the high frequency features by subtracting the blurred one from the original, similar to what we did in section 2.1.
For img2, we blur it with a hand-picked kernel sigma, and keep the low frequency directly after the low-pass filtering.
Here are the results:

Nutrek (sample images)

Derek Original

Nutmeg Original

Nutmeg_Derek (both colored)

\(\sigma_1=3, \sigma_2=5\)

Bells & Whistles: Which looks better? Colored or Gray

Nutmeg: Colored

Derek: Colored

\(\sigma_1=3, \sigma_2=5\)

Nutmeg: Colored

Derek: Gray

\(\sigma_1=7, \sigma_2=5\)

Nutmeg: Gray

Derek: Colored

\(\sigma_1=7, \sigma_2=5\)

Nutmeg: Gray

Derek: Gray

\(\sigma_1=3, \sigma_2=5\)

!Failure Case!

Nutmeg: Colored

Derek: Gray

\(\sigma_1=21, \sigma_2=5\)

Observation: Generally, when high freqeuncy retains only lines/textures (aka. Gray) and low frequency keeps all the blurry color, things look the best. The first line of 4 exapmles are not very intuitive but the fifth image with the colored cat (high-freq) having lapha=21 really pushes to the extreme case and surely looks bad.

Elonaltman (Fav!)

(Spoiler Alert: This are the images I will perform frequency analysis on.)

Elon Original

Altman Original

Elon_Altman (both colored)

\(\sigma_1=7, \sigma_2=5\)

Catiger

Cat Original

Tiger Original

Cat_Tiger (Cat: Gray, Tiger: colored)

\(\sigma_1=7, \sigma_2=5\)

I also played around using colored Cat and colored Tiger to hybrid, which is not good looking since the color of cat actually enhanced the red-ish and yellow-ish thing. Failed images are omitted.

Hamelon

Hamster Original

Watermelon Original

Hamster_Watermelon (both colored)

\(\sigma_1=11, \sigma_2=13\)

Fav Set: Elonaltman Frequency Analysis

Elon Original Frequency

Elon High Frequency

Altman Original Frequency

Altman Low Frequency

Elon_Altman Frequency

Kiwibara: A Failure Case!

Kiwi Original

Capybara Original

Kiwi_Capybara (both colored)

\(\sigma_1=1, \sigma_2=7\)

Kiwi_Capybara (both colored)

\(\sigma_1=3, \sigma_2=5\)

The key problem with this failure is that: the high frequency image (Kiwi) doesn't really have enough textures/lines for human eyes to identify! So when hybriding these two together, people can see the capybara pretty well, but the Kiwi always looks like a stain or artifact...

2.3 Gaussian and Laplacian Stacks

Below will first display a Gaussian Stack of Apple, then a Laplacian Stack of Apple.
Then, we will display a Gaussian Stack of Orange, then a Laplacian Stack of Orange.
Taking level 0, 1, 2, 4 for each image.

Gaussian Level: 0

Gaussian Level: 1

Gaussian Level: 2

Gaussian Level: 4

Laplacian Level: 0

Laplacian Level: 1

Laplacian Level: 2

Laplacian Level: 4

Gaussian Level: 0

Gaussian Level: 1

Gaussian Level: 2

Gaussian Level: 4

Laplacian Level: 0

Laplacian Level: 1

Laplacian Level: 2

Laplacian Level: 4

Here is the recreation of the classic figure:

Laplacian Level: 0

Laplacian Level: 0

Laplacian Level: 0

Laplacian Level: 1

Laplacian Level: 1

Laplacian Level: 1

Laplacian Level: 2

Laplacian Level: 2

Laplacian Level: 2

Laplacian Level: 3

Laplacian Level: 3

Laplacian Level: 3

Laplacian Level: 4

Laplacian Level: 4

Laplacian Level: 4

Reconstructed

Addition Bells & Whistles included!!! Cheat the seaming line.

However parameter tuning I tried, there is always a seam line that is somewhat visible. So, to overcome the isseu, I found a way to cheat a little bit by replacing the mask for level 0 laplacian stack! The noticeable seam line in the reconstructed image mainly comes from the 0-th level of the Laplacian stack because the mask on that level has a hard edge! So a very cheap way to overcome that is simply using a Gaussian blurred mask in the 0-th level and keeps blurring away! This cheating method turns out to be quite good. Check out the images below. First row is new cheated level 0 from three masked Laplacian stacks. Second row is the reconstructed. All other parameters remain the same, aka. kernel = 7, sigma = 3.