Diadic Image Processing

Another important class of image processing operations are what we call diadic operations.
Now diadic operations have got two input arguments and one output. As we discussed for the monadic case, we have an arbitrary function; in this case it is a function of two input variables which we will call x and y. So it is f(x,y). And that function is applied to the corresponding pixels from the two input images, and the result is deposited in the corresponding element of the output image. All three images have to be exactly the same size and we apply the same function to every pixel in the two input images.

So what sort of functions are useful for diadic image processing? What is this function f of x and y?

Well we could perform arithmetic, we could add two images together or we could subtract two images, or we could multiply two images. We could also perform more complex functions like perhaps take the square root of the sum of the squares.

When it comes to implementing some of these diadic operations, MATLAB’s inbuilt vectorization capacity comes to our aid.

So if I was to add two matrices together, and I do that by using the plus operator, what MATLAB will do is add the corresponding elements of each matrix, deposit the result into the corresponding element of the output matrix.

Once again all of these matrices are of exactly the same size.

So we can use any of MATLAB’s arithmetic operators and we can also use any of MATLAB’s built-in logical operators.

If we use the logical operators what happens is that the corresponding elements of the input matrices are compared using that logic operator and the result, either true or false, is deposited into the corresponding element in the output matrix.

Now you might notice here that amongst the arithmetic operators there are a couple of unusual ones.

One is .* and that means multiply the corresponding elements of the input matrices together.
And the other one is ./ which means divide the corresponding elements of the input matrices.
In MATLAB if you use just the * operator, that is a traditional matrix multiplication which is quite different. Here we are talking about multiplying element by element.

Similarly, ./ means element wise division of corresponding elements in the input matrices, where / refers to multiplication by an inverse of a matrix, quite a different thing.

Let’s have a look at a possible example of a diadic operator and in this particular case we will look at image subtraction—one image minus another image.

So here is an image of an intersection taken from a high vantage point and looking down. And here is another frame from this same sequence—it is actually the fourth frame in the sequence.
And these images look very, very similar to each other. The traffic is not moving very quickly at all.
But if I subtract these two images, I get an image that looks like this. And what it has done is it has shown us whereabouts the two images are different. It shows us the discrepancy between the two images. And what we can see is that there are some bright pixels up here. So this shows quite clearly whereabouts the two images are different and we can see that the difference is mostly in that column of traffic, which is beginning to move downward in the frame.

Although the images look very, very similar, simply by taking the difference it really highlights where the change in the image has occurred.

But we do need to be a little bit careful when we subtract pixel values, particularly if they are of the unsigned integer eight type that we have used previously.

Consider this example. Consider a pixel in frame one has got a value of 55. The corresponding pixel in frame two has got a value of 55. The difference: clearly 0.

Now in this particular case the pixel in frame one has got a value of 60, it is a bit bigger then the value of the pixel in frame two. The difference is clearly of 5.

Now in this case when the frame one pixel has got a value of 55, the frame two pixel has got a value of 60, MATLAB will compute a difference of zero and this is because we are subtracting unsigned integers. Unsigned integers cannot be negative by definition, they are unsigned. So in this case we are going to get an erroneous result.

The way to get around it is to cast the images to a different data type; move them from being unsigned integers, perhaps to being signed integers, or perhaps to being double precision floating point values.

Let’s create a tangible example of this. I am going to create a camera, a movie camera, which is a special type of object which opens a movie file and returns the frames of that as individual grey scale images.

So I have created the movie object; it is in the workspace called cam. It is a movie object. And now I am going to grab one frame from that movie.

So I am going to use the grab method, which we have seen before and I am going to grab the first frame out of the movie. And I am going to grab the next frame out of the movie, so two sequential frames. Now I am going to display the image difference and I am going to display the image which is the result of image one minus image two.

And here we see a grey scale image and it looks a little bit unusual and we can see some interesting activity around here.

The result of im1 minus im2 is actually assigned quantity. The pixel in image one could be smaller then the pixel in image two. So it is useful to display this with a different colour map and I am going to use the option signed here. And what it is going to do is display pixels that are negative as red and pixels that are positive as blue, and the background, where it is 0, is white.

So let’s zoom in on this part of the image up here. And what we can see is some red pixels and some blue pixels and this is showing us where the pixels in the image have changed from one frame to the next.

So this is where there is motion in the scene. If we go back and look at the original movie we can see that this is where the traffic starts to move across the intersection. These are the only pixels in the scene that are changing. We can use this image difference mechanism to determine which parts of the scene have been changed.

Let’s discuss a more sophisticated image processing operation. It is what is called the green screen effect. And what we want to do is to superimpose this abstract scene of a hat, a ball and a ladder into the scene of a road in a desert.

What we want to do is to merge these two images, and we are going to do that using the information in the top image, where those pixels that are green are going to be substituted by the background pixels from the bottom image.

The green screen effect is widely used in television production and here is an image of me filming one of the introductory pieces for the MOOC. All of the pixels that are in the green background are substituted by a number of photographs and graphics and so on.

This effect is also used by newsreaders, weather men on TV. When I am doing this it is really important that I don’t wear anything that is green. If, for instance, I wore a green shirt then my body would just disappear, and it would be replaced by the background photographs.
So let’s talk about how we do the green screen effect. So here are the two images again. In the top we have our abstract image against the green background and in the bottom we have the real scene that we want to superimpose the abstract image into.

The first step is to determine which pixels are green and we do that by making two decisions and we create two logical images. The top one shows all the pixels that are not green. So white value means that the corresponding input pixel is not green. The bottom one is the opposite of that: the white pixels correspond to green in the input image. These images are logical images. The pixels have only got the value true or false. Sometimes these are called binary images because they only have two possible values. Sometimes they are called mask images because we use them to mask out particular regions of the input scene.

The next step then is to multiply the mask matrices by the original input images. And what we have in the top line is an image where all of the green pixels have been set to black, and the remaining pixels are the abstract object.

We do the same thing for the bottom image. We multiply the input image by the mask and now we have the input image with a hole in it. The black pixels correspond to the abstract object.
Notice that we have used the diadic operation twice here, in each case to multiply a masked matrix by an original colour image.

The last step then is to add these two images together. And that is yet another diadic operation, and what we are left with then is an image which contains our abstract object against the background of a desert scene.