Further topics in image processing

Previously we have looked at using mathematical morphology and the structuring in the elements that we used were either squares or rectangles and we showed how we could use morphological processing technique to select particular shapes out of an input image. We could use a square structuring element and we would select only the squares from the input image when we can eliminate rectangles and vice versa.

Let's now consider the problem where we want to find shapes that are not just squares or rectangles. Mathematical morphology allows us to do this but we need to make some small adjustments to the technique that we talked about previously.

Let's consider now the problem of trying to find the diagonal line segments in this particular image. So we want a structuring element that will select these shapes here shown in red or alternatively we might want to find all the shapes that are a diagonal line but in the opposite direction.

In the most general case, we should be able to select a line at any particular angle. In order to be able to do this, we need to change a definition of the structuring element. We are going to allow the cells within the structuring element to have the possible value which is don't care.

That means they are not taken into account when we perform the morphological processing. We consider that those cell elements can have a value of zero which means they are not considered to be part of the structuring element or they can have the value of one which means they are considered to be part of the structuring element.

Let's consider a simple example. Here we have a low resolution example image which contains two lines which slope upwards from left to right, another diagonal line which slopes downward from left to right and a vertical line. We are going to choose a three by three structuring element which will select the diagonal lines which slope upwards looked at from left to right.

I am going to show another one of the animations that we have looked at previously and it's going to show the mathematical morphology process in practice.

Once again we see the active elements in our structuring element are coloured red and as they move across the image, the result will only be true if the three red elements in the structuring element lay across pixels which are all set or have a value of true.

As the structuring element moves across we see that our output image contains line segments which slope upwards from left to right. Clearly I can flip that structuring element around and I will be able to select lines which slope downwards when looked at from left to right.

Once again we have this notion of keeping or retaining only those elements of the image which are compatible with the structuring element. Here is our original image, the input image. This is the result of the mathematical erosion process.

In this example, the strutting element was a three by three matrix but only the elements on the diagonal of that three by three matrix took part in the morphological processing. The rest of them were set to the don't care value and they did not participate.

Let's introduce a real example here. I am going to load an image of a tomato and this is an image that we have seen previously and on the way in I am going to gamma correct the image. And I am assuming that the image was gamma encoded with a gamma value of 2.2.

Let's just display the image, see what it looks like and there we have it. Now I want to find just the red pixels. I want to identify just the tomato fruit and to do that I am going to use a toolbox function, which performs a kmeans class location I will explain that in a moment.

It has three output arguments and the function colorkmeans and I pass in the image and the next argument that I pass in are the number of different colors that I want the scene classified into. And I am going to tell that there are four colors in this particular image.

This looks to me that clearly a bunch of red pixels, there is a bunch of green pixels, there is a bunch of quite dark background pixels and then there is also some lightish pixels, So I am going to tell it to do it's best effort at trying to segment those pixels into four unique classes.

The result then is our three matrices. One of the matrices is the variable CLS or class and it is a double position image just the same size as the input image. It's 238 pixels high by 318 pixels wide.

Then we can have a look at the class image. Create a new figure and I will display the matrix CLS class and here it is. And what we see is it looks quite posterized.

If I click on some values here we see that the red tomato fruit pixels have been classified with a value of three.

So all of the red pixels have been assigned to belong to class 3. This dark background has been assigned class 4, which appears quite bright actually. This background down here has been classified as belonging to class one and a lot of the green material has been classified as belonging to class two.

This algorithm has got a random initialization in it so every time you run it you get a different class assignment but in this particular run the important thing to remember is that the tomato colored pixels have all been assigned to class number 3. I can create now a binary image all pixels that belong to class 3 and I can display that image.

This is a binary image the pixels with a zero for the background on that tomato and 1 if the do belong to the tomato. We can clearly see here that the fruit are not complete. The bottom fruit has got a couple of holes in it and this is because we are seeing a reflection of an overhead light is being reflected off the shiny skin of the fruit.

The other fruit has got reflections but it's also partly obscured by some leaves.

One of them looks like a round object which is what we would expect if for instance we were building a tomato harvesting robot we would for round red things the second fruit isn't very round at all. So we can use some morphological processing operations to patch the holes in these fruit.

Let's get some idea about how big the holes are. I am going to zoom in on this part of the fruit here. Now we can see that they go from about 80 to 105 so there almost 25 pixels wide this particular hole in the fruit.

If I am going to have a structuring element, I am going to perform a morphological closing operation, it needs to be a very large kernal.

Let's perform a morphological closing operation on this binary image. I will call the result closed and I will use the iclose function I am going to pass in the binary image of the tomato and I am going to pass in a square structuring element that's 15 pixels by 15 pixels. And I am going to compute that.

Now let me display that particular image. Here we see that it has certainly it has patched the holes in the bottom fruit and it's done something to the top fruit. But we see it's introduced quite a lot of jagged edges let's have a look in here and we can see that it's introduced some quite big steps into the shape of the fruit.

The reason for this is we use the structuring element that was a square 15 by 15 pixels square. The fruit that we are interested in we know is round so we could use an alternative structuring element which is a circle and I can use the toolbox function kcircle and I am going to create a circle with a radius of 15 pixels and I can display that. And here is a structuring element that is a disk that has got a radius of 15 and it sits within a 31 by 31 matrix.

If I consider this matrix as the structuring element, then the white pixels that are shown here are part of the structuring element itself and the black or zero value pixels are not part of the structuring element.

What I have done now is created a structuring element which is a circle or disk. It sits within a square matrix. Let's apply this circular structuring element to our image. So I am going to compute the closure and pass in the tomato image; the binary tomato image and I pass in the round structuring element and I will display that and we see now has turned a slightly better job of patching up the hole in a fruit.