## Image Blending • Image blending is the process to blend (mixing) two images.
• One of the important application of image blending is to remove the artifacts while creating the panorama or stitching the images.
• Image blending also used to add the object in background or fore ground in images/scenes.

Lets take the two images which we want to blend…..

Following are some of the image blending process :

Suppose first image is a and second image is b

1. Multiply  (darken) image blending:

F_blend (a,b) = a *b

Darken image blending is multiplication of  two images pixel by pixel. Result of blending will be darker compared to both input images as shown in above  figure .1

2. Screen or brighter image blending:

F_blend (a, b) = 1 – (1-a)* (1-b)

Screen image blending is reverse process of darken image blending its make the image much brighter as shown in figure.2 so if in case we want dark area also be brighter we can use screen image blending .

3. Overlay Image blending :

F_blend (a, b)   =          1 – 2* (1-a) * (1-b)  if  (a > .5)   else   2* a * b

Overlay blending provide the results in which darker area will become more darker and brighter area becomes brighter on the basis of base image as shown in figure 3. The image in which we check condition is called the base image here we checked the condition on image a .

4. Weightage image blending : Figure. 4 Weightage image blending.

F_blend (a,b) =  (weight * a + (1-weight)*b ) /2

Weightage blending provide the weights to images and take the average of them. Figure 4 shows the weightage blending where the weight value is 0.5. Range of weight is 0 to 1.

5. Cross fading blending:

Cross fading is one of the most important blending used to smooth panorama images. Figure. 5 Ramp function for image

Figure. 5 shows we create the ramp function for input images where ramp intensity is 0 to 1 and width of the ramp is the part of image we want to blend. figure. 6 Blending with ramp function

We take two ramp images according to figure. 5 ad add them to get the result.

We took full ramp to blend the image as shown in figure. 7 . In left most side image 1 effect is dominant and in right most side image 2 is dominant. In middle it will be average. Figure. 8  shows the image feathering by step function . Figure 8 we can apply the cross fading in  middle strip window to remove the sharp line you can see the result in Figure 9.

In some applications we create image pyramid and apply the cross fading in each gaussian and laplacian level and reconstruct the result. That is known as pyramid blending process.

%%%%%%%%%%%%%%%%%%%% CODE START %%%%%%%%%%%%%%%%%%

clc % clear screen
clear all % clear all variable
close all % close all open window

img1 = imresize(img1,[480,640]);
img2 = imresize(img2,[480,640]);
figure, imshow(img1)
title(‘input imag1’)
figure, imshow(img2)
title(‘input imag2’)

img1 = im2double(img1);
img2 = im2double(img2);

[row,col,channel] = size(img1);

%% multiply darken image blending
darken_blend = img1.*img2;

figure, imshow(darken_blend,[])
title(‘darken blending result’)

%% screen brighter image blending.

brighter_blend = 1 – ((1-img1).*(1-img2));
figure, imshow(brighter_blend,[])
title(‘brighter blending result’)

%% weightage blending. weight 1 rang is (0 to 1)

weight1 = .5;
weight2 = 1 – weight1;

weightage_blend = (weight1.*img1 + weight2.*img2)/2;

figure,
imshow(weightage_blend);
title(‘weightage blending result’)

% overlay blending

%% initialize image
overlay_blending = zeros(row,col,channel);

%% assign the value only on index where img1 < .5
overlay_blending(img1<.5) = 2.*img1(img1<.5).*img2(img1<.5);
%% assign the value only on index where img1 > .5

overlay_blending(img1>.5) = 1 – (2.*(1-img1(img1>.5)).*(1-img2(img1>.5)));

figure, imshow(overlay_blending,[])
title(‘overlay blending result’)

%% cross fading blending

%% genrate ramp function which size is widt of image ( full ramp function)
ramp1 = (0:1/col:1);
ramp2 = (1:-1/col:0);

for cols = 1: col

%%% multiply the ramp function with image to get ramp image 1 and ramp image 2 and %%add them to get result
cross_fading(:,cols,:) = (ramp2(cols).*img1(:,cols,:) + ramp1(cols).*img2(:,cols,:));

end

feathering = zeros(row,col,channel);

%% genrate step images and add them
feathering(:,1:col/2,:) = img1(:,1:col/2,:);
feathering(:,col/2:end,:) = img2(:,col/2:end,:);

imshow(feathering)
title(‘image feathering’)

ramp3 = (0:1/100:1);
ramp4 = (1:-1/100:0);

ground = zeros(1,(col/2)-50);
one = ones(1, (col/2)-50);

%% generate the ramp function for cross fading in middle of image with 100 pixel width.
function3 = [ground ramp3 one];
function4 = [one ramp4 ground];

feathering1 = zeros(row,col,channel);

for cols = 1: col

feathering1(:,cols,:) = (function4(cols).*img1(:,cols,:) + function3(cols).*img2(:,cols,:));

end

figure,
imshow(feathering1)

%%% %%%%%%%%%%%%%%%% CODE END %%%%%%%%%%%%%%%%%%%%%  Happy Learning

Cheers

lication

## Gaussian and Laplacian Pyramids Gaussian and laplacian pyramids are applying gaussian and laplacian filter  in an image in cascade order with different kernel sizes of gaussian and laplacian filter.

• Figure. 1 shows pyramid of image.
• Full image resolution is taken at level 0.
• At each step up level image resolution is down sample by 2. So if starting image size is 256 X 256 at level 0 in level 1 image size will be 128 X 128.

Gaussian Pyramid : Figure. 2 Gaussian Pyramid of level 2
• Figure. 2 shows the Gaussian pyramid block diagram of input image.
• Take input image  (in_image),  apply gaussian filter, output will be level 0 image.
• Down sample the level o image by 2, apply gaussian filter, output will be level 1 image.
• Down sample the level 1 image by 2, apply gaussian filter, output will be level 2 image.
• Similar way we can create full pyramid of gaussian.
• Every level use same Gaussin filter. (If you are down sampling the image by 2 at each level, size of gassuan filter should not be changed)
• Effect of down sampling the image by 2 is equivalent to increasing the bandwidth of gaussian filter by 2.
•  Suppose gassuain filter cutoff frequency is F1 in 0 level if we down sample image by 2 gaussian filter cut of frequency will be 2*F1 for down sampled image.
• So by down sampling the image, we are applying different sizes of gaussian in input image. That is also called multi resolution analysis.
• Gaussian pyramids are used in SIFT, Surf, Gabor filters for multi resolution features extractor Figure. 3 Gaussian pyramid results

Figure. 3 shows gaussian pyramid results. Level 2 image is much blurred as compared to level 0.

Laplacian Pyramid: Figure. 4 Laplacian pyramid of level 1

• Suppose Level 0, level 1 and level 2 is the output of gaussian pyramid as we discussed above. Figure. 4 shows the block diagram of laplacian pyramid from gaussian pyramid.
• Up sample the gaussian level 1 image  by 2  (level 1_2)and subtract the level 1_2 image from level 0 image, output will be laplacian of level 0.
• Upsample the gaussian level 2 image  by 2 (level 2_2) and subtract the level 2_2 image from level 1 gaussian image , output will be laplacian of level1.
• Laplacian pyramid try to find out the pass band frequency.
• Suppose gaussian level 0 frequency is f1 and gaussian level 1 frequency is f2, so laplacian level 0 frequency will be f2 – f1.
• Laplacian pyramid used many time in object detection pre processing steps.
• It can help to compute the optical flow for large motion vector. Figure .5 Laplacian pyramid results of level 2

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Code Start %%%%%%%%%%%%%%%

clc % clear the screen

clear all % clear all variables

close all % close all MATLAB windows

% set path to read image from system

ImagePath = ‘D:\DSUsers\uidp6927\image_processingCode\lena.jpg’;

img_RGB = imread(ImagePath); % read image from path

img_RGB = im2double(img_RGB); % Convert image to double precision

img_gray = rgb2gray(img_RGB); % convert image to gray

[row,col,channel] = size(img_gray); % get size of image
h_gauss = fspecial(‘gaussian’,11,1);

% apply gaussian filter to gray image
gauss_level_0 = imfilter(img_gray, h_gauss,’replicate’);

% down sample the gauss_level_0 image by 2 (resize function)
% Apply gaussian filter
gauss_level_1 = imfilter(imresize(gauss_level_0,[round(row/2),round(col/2)]), h_gauss,’replicate’);
% down sample the gauss_level_1 image by 2 (resize function)
% Apply gaussian filter
gauss_level_2 = imfilter(imresize(gauss_level_1,[round(row/4),round(col/4)]), h_gauss,’replicate’);
% down sample the gauss_level_2 image by 2 (resize function)
% Apply gaussian filter
gauss_level_3 = imfilter(imresize(gauss_level_2,[round(row/8),round(col/8)]), h_gauss,’replicate’);

% upsample the gauss_level_1 image by 2(resize function)
% subtract resized gauss_level_1 image from gauss_level_0 image
laplacian_level_0 = gauss_level_0 – imresize(gauss_level_1,[row,col]);
% upsample the gauss_level_2 image by 2(resize function)
% subtract resized gauss_level_2 image from gauss_level_1 image
laplacian_level_1 = (gauss_level_1 – imresize(gauss_level_2,[round(row/2),round(col/2)]));
% upsample the gauss_level_3 image by 2(resize function)
% subtract resized gauss_level_3 image from gauss_level_2 image
laplacian_level_2 = (gauss_level_2 – imresize(gauss_level_3,[round(row/4),round(col/4)]));
% draw all the images
subplot(2,2,1), imshow(img_gray,[])
title(‘input image’)
subplot(2,2,2), imshow(gauss_level_0,[])
title(‘gaussian pyramid level0’)
subplot(2,2,3), imshow(gauss_level_1,[])
title(‘gaussian pyramid level1’)
subplot(2,2,4), imshow(gauss_level_2,[])
title(‘gaussian pyramid level2’)

figure,
subplot(2,2,1), imshow(img_gray,[])
title(‘input image’)
subplot(2,2,2), imshow(laplacian_level_0,[])
title(‘laplacian pyramid level0’)
subplot(2,2,3), imshow(laplacian_level_1,[])
title(‘laplacian pyramid level1’)
subplot(2,2,4), imshow(laplacian_level_2,[])
title(‘laplacian pyramid level2’)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%Code End %%%%%%%%%%%%%%%%  Happy Learning

Cheers

## Image Enhancements • Image enhancements is manipulation of images (in term of images) that suits to our application.
• As we discussed in last post by using Gaussian filter we can able to remove noises that is also one type of image enhancement technique.

Some of image enhancement techniques are

1. Negative image
2. Log transform
3. Power law transform
4. Filtering  (Gaussian filter, Median filter)
5. Histogram equalization.
• Post will discuss negative image, power log transform and log transform of images .
• Histogram equalization is one the famous method of  image enhancement that I am not going to discuss in this post :D.

Negative Transform :

output_image =  (Negative_range – 1)  – input_image ;

• Its generate the negative image of input image.
• We can manipulate the output image with Negative_range parameter.
• Negative_range parameter can be 256 to 1.
• Value below negative_range in input image will become zero in output image.

Figure shows the negative image of input Lena image with Negative_range = 256.

Power Law Transform:

output_image = Constant * (input_image^ gama) ;

• Power low transform manipulates contrast of image with the help of gama value.
• Figure.2 shows the power law transform output with Constant =1 and gama  =3;

Log Transform:

output_image = constant * log(1 + input_image);

• Log transform changes the contrast of image in sort of reverse of power transform.
• 1 is added to input image to avoid infinity (log (0)).
• Log transform manipulate images with help of constant multiplier.
• Figure.2 shows log transform of input lena image with constant value 5.

MATLAB code:

%%%%%%%%%%%%%%%%%%%%%%% code start %%%%%%%%%%%%%%%%%%%%%%

img = imread(‘lena’,‘jpg’); % load the image

img_gray = rgb2gray(img);

img_gray = double(img_gray); % convert image to double data type

[row,col,channel] = size(img);

subplot(2,2,1), imshow(uint8(img_gray))

title(‘Lena image’)

%% negative transform

%% Subtract the input image from 255

img_gray_negative = 255 – uint8(img_gray);

% plot the negative image

subplot(2,2,2), imshow(img_gray_negative)

title(‘Negative Lena image’)

%% power transform

constant = 1;   % assign the constant parameter equal to 1

gamma = 3;      % assign gama value to 3

img_gray_power = constant * (img_gray +1).^gamma;

subplot(2,2,3), imshow(img_gray_power,[])

title(‘Power law Lena image gamma = 3’)

% log transform

C = 5;   % assign constant C value to 5

img_gray_log = C * log(1+ img_gray);

subplot(2,2,4), imshow(img_gray_log,[])

title(‘log Lena image C = 5’)

%%%%%%%%%%%%%%%%%%% code end %%%%%%%%%%%%%%%%%%%%%

Note*

These transform serve as pre processing modules in image processing pipeline and help to enhance the image significantly according to applications.  Happy Learning

Cheers

## 2 D FFT and Image Filtering In this post I will discuss the 2 D FFT in term of image processing.

Steps we are going to perform:

1. Read input images and observe FFT of images.
2. Generate noisy image with random noise.
3. Analyze the FFT of Lena clean image and Lena noisy image.
4. Use Gaussian filters in frequency domain to clean image
5. Analysis of results.

Figure.1 shows the FFT of single color image and white and gray image FFT.

• In single color image all pixel are same their is no variations in pixel color.
• Frequency of image is almost zeros so we can see small dot at center.
• That’s mean all frequency is concentrated near to zero.
• In gray and white image color is changed in vertical direction (Y direction), FFT of image shows the frequency distributions in Y direction. No frequency changed in X direction.
• If image is blur (means color variation in image is less) its FFT distribution will be concentrated. That’s help to find out amount of blur in image.

• Random noise image have random pixel variation, that shows many random frequencies are present in noise image.
• FFT of random noise image will be distributed in all direction without having any prominent frequency as shown in figure.2.

• Figure.3 shows FFT of standard lena image we can see lot of information lie on center and some high prominent frequency content also present in image that makes image sharp.
• In lena noisy image random noise is distributed in image but some information is their at center we can see some brightness at center .

• If we take the filter which take out center frequency of image we can able to rid of random noise.
• Let take the gaussian mask ( as shown in figure4)  and multiply it with image in frequency domain. (frequency domain multiplication is convolutions in special domain).
• Inverse the image to special domain which is filtered image (shown in figure. 4).

• Figure 5 shows the result of image filtering with many sizes of Gaussian mask.
• Gaussian filter sigma is 5, image is too blurred and noise is removed.
• When Gaussian sigma is 57, image is not cleaned.

So with the help of FFT we can find out the size and distribution of Gaussian filter according to our applications.

Some applications of FFT in image processing

•  Feature extractions.
• Image Compression.
• Filtering
• Find the amount of blur in image.
• Many time used to speed up the 2 d convolution, operation like erosion, dilation.

MATLAB code:

%%%%%%%%%%%%%%%%%%%%%%code start%%%%%%%%%%%%%

clc;        % clear  screen

clear all  % clear all variables

close all

img = imread(‘lena’,‘jpg’); % load the image

img_gray = rgb2gray(img);

img_gray = double(img_gray);   % convert image to double data type

[row,col,channel] = size(img);

noise = 255*rand(row,col);         % generate random noise

noisy_img = (img_gray + noise)/2 ;    % create noisy image

% calculate fft  of clean image.

fftimg = fft2(img_gray);

% use fftshift to shift the fft in center

fft_img = fftshift(fftimg);

% calculate fft  of noisy image.

fftnoisy_img = fft2(noisy_img);

fft_noisy_img = fftshift(fftnoisy_img);

% Genrate gassuian filter with size of image and sigma 27.

H = fspecial(‘gaussian’,[row, col] ,27);

% multiplication of gausaain filter with noisy image in frequency domain

fft_filter_img = H.*fft_noisy_img;

% take inverse of fftshift

fft_filterimg = ifftshift(fft_filter_img);

% calculate absolute value of inverse FFT that is filtered image.

filterimg = abs(ifft2(fft_filterimg));

%plot all the results

subplot(2,2,1), imshow(uint8(img_gray))

title(‘Lena image’)

subplot(2,2,2), pcolor((log(abs(fft_img)))); shading interp

colormap(gray), set(gca,‘Xtick’,[],‘Ytick’,[])

title(‘FFT of Lena image’)

subplot(2,2,3), imshow(uint8(noisy_img))

title(‘Lena noisy image’)

subplot(2,2,4), pcolor((log(abs(fft_noisy_img)))); shading interp

colormap(gray), set(gca,‘Xtick’,[],‘Ytick’,[])

title(‘FFT of Lena noisy image’)

figure,

subplot(2,2,1), imshow(uint8(noisy_img))

title(‘Lena noisy image’)

subplot(2,2,2), pcolor((log(abs(fft_noisy_img)))); shading interp

colormap(gray), set(gca,‘Xtick’,[],‘Ytick’,[])

title(‘FFT of Lena noisy image’)

subplot(2,2,3),imshow(H,[])

subplot(2,2,4),imshow(filterimg,[])

title(‘Filtered image’)

figure,

subplot(2,1,1),imshow(H,[])

title(‘gaussian filter mask of sigma 27’)

subplot(2,1,2),imshow(filterimg,[])

title(‘Filtered image with gaussian’)

%%%%%%%%%%%%%%%%%%%%%code end%%%%%%%%%%%%%%%%  Happy Learning

Cheers

## Basic Filtering MATLAB To start work in CV or IP we should have a input image to do some experiments on it. Go to google and type lena. You will get lena standard image like I added below. Lena image

Some words about Miss Lena:

• Lena is most famous image to perform compression, IP and CV.
• Lena digitized picture was appear on playboy in 1972.
• Lena Soderberg (ne Sjööblom) was last reported living in her native Sweden, happily married with three kids and a job. if you want to more about lena go through following article Lena .

Steps we are going to perform:

1. Read the Lina image as input.
2. Generate random noise.
3. Add noise to input image.
4. Apply average filter and gaussian filter to clean the noisy image Figure shows the mask used to filter image.

• Random noise is random in nature so if we take the average of random noise its tends to cancel out each other.
• In above average filter we assign the equal weight to all 25 pixel that is 1/25 .
• In Gaussian filter instead of providing equal weight we distribute the weights in gaussian manner so in center max weight will be assign. Result of Average filter and Gaussian filter on noisy image.

MATLAB code:

%%%%%%%%%%%%%%%%%% Code Start %%%%%%%%%%%%%%%%%%%%%%%%%%

clc    % clear the screen

clear all  % clear all variables

close all % close all MATLAB windows

% set path to read image from system

ImagePath = ‘D:\DSUsers\uidp6927\image_processingCode\lena.jpg’

img_RGB = imread(ImagePath); % read image from path

img_RGB = im2double(img_RGB);  % Convert image to double precision

img_gray = rgb2gray(img_RGB);   % convert image to gray

[row,col,channel] = size(img_gray);    % get size of image

% select noise factor

% noise can be controlled by noise factor.

noise_factor = 0.7;

% generate noise

noise = noise_factor*rand(row,col);

% Generate noisy image by adding noise to input image.

noisy_img = 1/2*(img_gray + noise);

%generate average filter with help of fspecial command.

h_ave = fspecial(‘average’,11);

%apply average filter to noisy image

avg_filter = imfilter(noisy_img,h_ave,’replicate’);

%generate gaussian filter of size 11×11 with sigma 1

h_gauss = fspecial(‘gaussian’,11,1);

% apply gaussian filter to noisy image

gauss_filter = imfilter(noisy_img, h_gauss,’replicate’);

%plot all images for results

figure,

subplot(2,1,1),

surf(h_ave);

title(‘ Average filter mask’)

subplot(2,1,2),

surf(h_gauss);

figure,

subplot(2,2,1)

imshow(img_gray,[])

title(‘Input image’);

subplot(2,2,2)

imshow(noisy_img,[]);

title(‘Noisy image’);

subplot(2,2,3)

imshow(avg_filter,[])

title(‘Average image’);

subplot(2,2,4)

imshow(gauss_filter,[])

title(‘Gaussian image’);

%%%%%%%%%%%%%%%%%%%%%%%%Code End %%%%%%%%%%%%%%%%%%%%%%

Note*

• fspecial is used to generate filter mask. Type help fspecial in Matlab command window for more detail.
• imfilter command is used to convolve 2 D filter mask with image.
• In next post will discuss filtering in frequency domain.  Happy Learning

Cheers

## Prerequisites

Computer Vision is big domain and I am afraid from where should I start Starting post will cover the image pre processing techniques and its applications, slowly will move to feature extraction techniques and further will cover tracking and machine learning techniques.

Prerequisites:

• Install MATLAB any version above then 2008 (necessary). If you don’t have MATLAB install GNU OCTAVE  (open source ) latest version with statistical, signal processing and image processing library.
• Install python 2.7 and OpenCV 2.9 (optional) it is needed for machine learning applications. You can follow  link to install OpenCv python in Linux  installation of OpenCv with python in Ubuntu .
• Some knowledge of MATLAB/ Octave because in blog I will only discuss the image processing concepts.
• Most important interest in Computer vision / image processing.

That’s ALL Happy learning

Cheers