Project 2: Fun with filters and frequencies

Part 1: Fun with Filters

1.1: Finite difference operator

In this section, we use the finite difference operators to obtain the partial derivatives of an image with respect to the x and y directions. To do this, we convolve the image with the vector [1 -1] (for dx) and [1 1].T (for dy). Here are the results:

Original cameraman image
Original cameraman image
Cameraman D_x
Cameraman D_x
Cameraman D_y
Cameraman D_y

As part of this section, I also computed the gradient magnitude image and binarized to create an edge image. The gradient magintude image like the gradient of any other 2-dimensional function f(x,y). That is, grad = sqrt((dI/dx)^2+(dI/dy)^2). To create the gradient image, we simply calculate this value at each point of the original image. An inperfection of the binarized image is that it's very noisy because we had to pick a large threshold. We'll solve this in the following section.

Gradient image
Gradient image
Edge image
Edge image

1.2: Derivative of Gaussian (DoG) filter

Since the above image was very noisy, we will apply a gaussian filter before calculating the gradient image. That is, we first blur the image and then compute gradient magnitude:

Gradient image
Blurred cameraman image
Edge image
Blurred image gradient magnitude
Edge image
Blurred image edge image

In the case of the new edge image, we can see that it's significantly less noisy, because we were able to choose a much lower binary threshold (namely, I picked .08 instead of 0.3). Given that we blurred the image first, random noise won't make this threshold, leading to better results.

As prompted, I also carried out this procedure by calculating first the DoG filters (for x and y), and applying those analogously to part a. Indeed, we obtain the same result:

Gradient image
DoG_x
Edge image
DoG_y
Gradient image
Blurred image edge image
Edge image
Regular image with DoG

Part 2: Fun with frequencies

2.1: Image Sharpening

In this section, I explored how amplifying the higher frequencies of an image led to the image appearing "sharper". Indeed, to obtain a high pass filter, we subtract a low-pass filter from them image. Then, we add the high-filtered image to the original using some scaling factor alpha and observe the results. For this section, I picked old image that were originally not "sharp". I display the sharpened verisons for different values of alpha - my favorite is the berlin wall picture.

Original
Original image
Forest
Alpha=1
Mountains
Alpha=4
Original
Original image
Forest
Alpha=1
Mountains
Alpha=4
Original
Original image
Forest
Alpha=1
Mountains
Alpha=4
Original
Original image
Forest
Alpha=1
Mountains
Alpha=4

As we can see the optimal alpha isn't constant but rather depends on the original image. A value of alpha that is too large results in noise being amplified to the point where the image becomes worse. For example, the football squad image looks better at alpha=1 than 4.

2.2: Hybrid images

Hybrid images are images a blend of two images that appears different from up close and far away. This effect is accomplished by overlaying a high-filtered image of one object with a low-filtered one of another. The following are the results the derek nutmeg example:
Original
Aligned Derek
Forest
Aligned Nutmeg
Mountains
Derek-Nutmeg
Original
Angel (me!)
Forest
Dog
Mountains
Angel Perro
Original
Happy Tom Cruise
Forest
Serious Tom Cruise
Mountains
Mysterious Tom

For my favorite example (the Tom Cruise happy/sad), here are the log magnitudes of the fourier transforms:

Original
Happy Tom Cruise fft
Forest
Blurred Happy Tom fft
Mountains
Serious Tom fft
Mountains
Filtered Serious Tom fft
Mountains
Result fft
Mountains
Result image

Here is an example that didn't work: I tried to join Barney Stinson and Joey Tribbiani's faces into one but couldn't. Due to the relative size of their faces and the specific picture I chose, only Joey is visible at any distance:

Original
Happy Tom Cruise
Forest
Serious Tom Cruise
Mountains
Mysterious Tom

2.3: Gaussian and Laplacian Stacks

A gaussian stack is an image to which we apply a gaussian filter repeatedly. That is G_n = I*G^n. The laplacian filter is calculated by subtracting an element from the gaussian Stack from the next. That is L_n = G_n - G_(n+1). The following are the gaussian and laplacian stacks for oraple apple:

Original
Apple G_0
Forest
Apple L_0
Original
Apple G_1
Forest
Apple L_1
Original
Apple G_2
Forest
Apple L_2
Original
Apple G_3
Forest
Apple L_3
Original
Apple G_4
Forest
Apple L_4
Original
Apple G_5

Note that there is one more element in the gaussian stack (as expected given what an element in the laplacian is). Here is the laplacian and guassian for the orange:

Original
Orange G_0
Forest
Orange L_0
Original
Orange G_1
Forest
Orange L_1
Original
Orange G_2
Forest
Orange L_2
Original
Orange G_3
Forest
Orange L_3
Original
Orange G_4
Forest
Orange L_4
Original
Orange G_5

2.4: Multiresolution Blending

By following the explanations from class and the suggested paper, I learned that Multiresolution blending is achieved masking the laplacian stack of each image at different layers and then sum the results to collapse the image back. For the oraple example:

Forest
Apple Laplacian L_0 (masked)
Original
Mask Laplacian L_0
Forest
Orange Laplacian L_0 (masked)
Forest
Resulting Image at this level
Forest
Apple Laplacian L_1 (masked)
Original
Mask Laplacian L_1
Forest
Orange Laplacian L_1 (masked)
Forest
Resulting Image at this level
Forest
Apple Laplacian L_2 (masked)
Original
Mask Laplacian L_2
Forest
Orange Laplacian L_2 (masked)
Forest
Resulting Image at this level
Forest
Apple Gaussian last element (masked)
Original
Mask Gaussian last element
Forest
Orange Gaussian last element (masked)
Forest
Resulting Image at this level
Forest
Original apple
Original
Original Orange
Forest
Final result

For my own images with a regular mask, I picked to blend landscapes:

Forest
City Laplacian L_0 (masked)
Original
Mask Laplacian L_0
Forest
Forest Laplacian L_0 (masked)
Forest
Resulting Image at this level
Forest
City Laplacian L_1 (masked)
Original
Mask Laplacian L_1
Forest
Forest Laplacian L_1 (masked)
Forest
Resulting Image at this level
Forest
City Laplacian L_2 (masked)
Original
Mask Laplacian L_2
Forest
Forest Laplacian L_2 (masked)
Forest
Resulting Image at this level
Forest
City Gaussian Last Element (masked)
Original
Mask Gaussian last element
Forest
Forest Gasussian Last Element (masked)
Forest
Resulting Image at this level
Forest
Original city
Original
Original Orange
Forest
Final result

For the irregular mask example, I chose to put squidward's house in space:

Forest
Squidhouse Laplacian L_0 (masked)
Original
Mask Laplacian L_0
Forest
Space Laplacian L_0 (masked)
Forest
Resulting Image at this level
Forest
Squidhouse Laplacian L_1 (masked)
Original
Mask Laplacian L_1
Forest
Space Laplacian L_1 (masked)
Forest
Resulting Image at this level
Forest
Squid Laplacian L_2 (masked)
Original
Mask Laplacian L_2
Forest
Space Laplacian L_2 (masked)
Forest
Resulting Image at this level
Forest
Squidhouse Last Element (masked)
Original
Mask Gaussian last element
Forest
Space Gasussian Last Element (masked)
Forest
Resulting Image at this level
Forest
Original Squidhouse image
Original
Original Space Image
Forest
Final result