Feature extraction

Let’s talk about object features.

We’ve already talked about describing the position of an object. We’ve worked out how to describe the centroid of an object, which is a useful way of describing where something is within the image. We’ve also talked about determining the size of an object—its area—and we can get that information from the zeroth moments.

We might be interested to describe the shape of an object; does it look like a shark, or a triangle, or a circle, or a square, and we might be interested in describing the orientation of the object. It’s quite important. Is the shark pointing upwards, or to the left or to the right? So let’s talk about how we can compute some of these additional object features.

Position we’ve already described in terms of the bounding box and the centroid, which is derived from first moments. The size of the object; we also call it its area, and it can be described form the zeroth moments. What we haven’t described so far is shape and orientation, so let’s talk about that.

The way we’re going to do this is to use a technique called the equivalent ellipse. So for each of these shapes what we’re going to do is to consider an ellipse that is centred at the object’s centroid, and that has got the same moment of inertia about that centroid as the original shape has. So this means for these different kinds of shapes we’re going to get different shaped equivalent ellipses and they’re going to tell us something about the shape and the orientation of the object. This is quite a classical technique, but it’s actually quite useful.

Let’s put it into practice. Here is our shark image. Here’s the centroid. We’re going to define a set of orthogonal axes; an x-axis and a y-axis that pass through the centroid of the shark, and they’re parallel to the u and v coordinates. Now once I’ve got this coordinate system, it’s easy to compute the inertia matrix for the blob around the y and x axes about the centroid. And we do that by computing what are called the central moments of the blob: mu, underscore, p,q. And this is very similar to the moment expression we had earlier, except inside the brackets we now subtract the coordinates of the centroid.

So if we compute these moments: mu,2,0, mu,0,2 and mu,1,1 for the blob we end up with a matrix that contain the inertia values. We want to find an ellipse that has got the same inertia as those we derived from the blob. Now if this is the inertia matrix we can compute its eigenvalues, and from those eigenvalues we can compute the major and minor axis of the ellipse. That’s basically half the long axis of the ellipse, and half the short axis of the ellipse. These radii a and b are derived from the eigenvalues of the matrix J. Lambda 1 is the largest eigenvalue, and lambda 2 is the smaller eigenvalue.

We’re also interested in the orientation of the ellipse; what’s the value of θ. And we obtain θ from the elements of the eigenvector that corresponds to the largest eigenvalue. So we take the ratio of the two elements in the eigenvector, the arc tangent, and that gives us θ. That tells us the orientation of the ellipse, and the orientation of the ellipse tells us something about the orientation of the object itself.

Another parameter that’s useful that we can derive from the equivalent ellipse is called the aspect ratio, that’s b/a. B is the short length, a is the long length, b/a is always less than 1. And it says something about the compactness of the shape. So this shark, for instance, has got a ratio of b over a that’s roughly 1, where this shark has got a value b/a that’s much smaller than 1. So b/a says something about the compactness of the shape.

Let’s load yet another variant of our shark image, and we will display this.

And here we see two sharks, one pitched downwards, and one pitched a little bit upwards. We will compute the parameters of these blobs using a familiar command to us, and we see we’ve found three blobs: two shark blobs and one background blob.

Let’s simplify things and get rid of the background blob, since we’re really only interested in the shark objects. We use the iblobs function again; pass in the image. And we ask for just those blobs comprising pixels that belong to class 1; that is, that the colour 1 they are the white pixels within the scene. And now we have an array of blob objects but only those objects that are white within the scene: the two sharks. Where each blob is displayed we see a number of parameters or attributes of those blobs, and some of those are quite familiar to use now. For instance, the area and the centroid. The parameter θ is the orientation of the blob.

Orientation is given in unit of radians, and it is in respect to a coordinate frame attached to the shark that’s parallel to the u and v axes. An orientation is positive in the clockwise direction. We can see that one shark has got a θ value of 29 and the other shark has a θ value of -2.58, so this suggests that they have different orientations.

Another parameter here is the ratio b/a, and we refer to that as the aspect ratio. And what’s interesting here is that the two sharks have got the same aspect ratio, around about 0.585, or 0.586 and this is important and interesting because the aspect ratio is the same even though the sharks have got different positions and different orientations, so we refer to this as a shape invariant. It’s a characteristic of the shark and it’s independent of its position or its orientation.
Now we’ve talked a bit before about the equivalent ellipse, and it’s pretty easy to plot. We apply the plot ellipse method to the vector of shark blobs, and what this will do is plot an equivalent ellipse for every element of that vector, all of the objects in the vector b, and here we see the two equivalent ellipses plotted. We can plot the centroids almost as easily using the plot method, applied to every object in the vector b.

So to summarise the object features that we can derive for a blob. We can describe its position in terms of bounding box or centroid. We can determine its size, its area. We can look at its shape and that comes from the aspect ratio of the equivalent ellipse, and we can determine the orientation in the object from the orientation of the equivalent ellipse.