The goal of this project is to try to make a hybrid image, and to be more, to make a morphing gif
- Project details: First I collect some typical pictures, just like those:
Then, I need to write the filter,like the detail said, I tried to implement the Gaussian filter, and the equation would be:
so, according to this equation, I used numpy package to implement my 2D Gaussian filter, and then convolute the whole image using the filter, but then I found that, using this filter to convolute would lose some information around the edge, just like that:
you can see some of the ears and months of the dog get clipped, so I need to use some '0' to fill the edge so that the shape of the image could be divisible by the shape of the filter size, just like that:
and that's how I get the low frequency images. Then I use the impulse filter minus the Gaussian filter (which can be computed by subtracting the Gaussian-filtered image from the original) as you said in the project detail.Just like that:
and then, just hybrid it!But the prior problem is how to match up the size between those images,I have tried to use the Google API to locate the object and resize those images according to the proportion of the bounding size and the image size, but it's not worth that without a large scale of data in it, and for this project, I used some methods of data augmentation to make a bigger scale of data.Then I thought, what about rescale all the data so that the object in the pictures is all in the middle.So, at the end,I got this hybrid images:
using these two images:
another examples would be:
- original images:
and the hybrid result would be:
- original images:
and the hybrid result would be:
- original images:
and the hybrid result would be:
- original images:
and the result:
For those images, I used the function you gave to compute and display the 2D Fourier transform, and the result would be :
More images:
Original:
Hybrid:
Original:
Hybrid Image:
Original:
Hybrid:
Original:
Hybrid:
2.Bells & Whistles:
- Generate a morphing gif:
The reason I thought of the following method is that for face morphing, all we need to do is to select some specific points in the first image and then find the corresponding one in the second image.When we find those points pair, the next thing we do is to make routes from one to the other so that we can generate the gif, so the model I used in it couldn't be more appropriate!
Luckily I have done one before, here is some details:
Using dlib and opencv to detect the face. Advices on working with facial landmarks with dlib and opencv https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python/
the dlib model we used to detect face landmarks is put in the model folder
The facial landmark detector will give us the estimated location of 68 (x, y) pair coordinates that represent salient regions of the face, including eye, eyebrows, nose, mouth, and jawline.
Using inverse warping to do the morphing.Using Delaunay triangulation to organize each pair of feature points to a triangle.
the dlib model we used to detect face landmarks is put in the extra_lab/model folder and the model will automatically locate the points of each triangle corresponding.Then generate the gif!
In that way, I can see how the face morphing in a gif file!
the original two images are:
and the output mp4 would be (here gives the link): https://www.bilibili.com/video/BV1NU4y1M7n4/
The problem of this project is that the quality of the morphing is determined by the first image in the model, because it is the standard of the whole morphing procedure.
- using color to enhance the effect
First I transformed the color space from RGB to HSV, but it doesn't work well
Then I transformed the color space from RGB to LUV, the result is kind of fairy:
Then I transformed the color space from RGB to BGR, it seems like we have better color:
Using gray image for both high frequency image and low frequency image and merge:
But first time it turns out to be green:
and I found out I missed one parameter when trying to show the grayscale image of .bmp file (when using pyplot to show the image), it is cmap=plt.get_cmap('gray')
but when I tried to use just gray image to get the low frequency image and merge with original high frequency image, I get this image which is obvious wrong.
Then I found it is when I get the range of my value in my low frequencies image wrongly so the hybrid is almost the same with the high frequencies image
after fixing this, I get:
And do the opposite, use just gray image to get the high frequency image and merge with original low frequency image, then I get this:
Both are the gray images:
Another example:
low frequencies image is gray when high frequencies image is a RGB image:
high frequencies image is gray when low frequencies image is a RGB image:
Both would be gray image:
I found that the color of the submarine image is quite dark, and the same with the fish image, so when I merge them together, you can just find some edges of the submarine, that's what I should be avoid of. But on the contrary, the hybrid image with bicycle and motor would be quite great, since they have some dark and bright colors, so that when hybriding, both could be seen much more clearly
Group Member:Jingzhou Shen