In simple terms, image segmentation is the process of taking a small part of a whole image. Look at the image an apple below with red and green parts. One may use an imaging software such as Paint to separate the red part from the whole image by using the crop tool.
However, trying to separate the pink pink sheep only becomes a lot harder. Using only the crop tool may result to a lot of other issues such as the accuracy of the edge (line around the sheep) and other details.
We now show image processing methods/algorithms in segmenting images.
Segmenting gray scale images
Consider an image of a pay check in gray scale. Our goal is to remove the background from the text. Let us call the text us our Region of Interest (ROI).
Since the image is in gray scale, it has range values of 0-255 (dark-light). From the pay check image, we observe that the background would would have higher values compared to the ROI (text). One way to separate the two is to set a threshold value t. Let us set that any values greater that t be the background and any values less than t be the text. It is also easier if we check the histogram of gray scale images. The peak (value=193) corresponds to the background.
Below shows different results of different threshold values.
For different threshold values, there are parts were the segmentation was good while some parts were bad (look at progression of the signature). Again this is somehow subjective depending on the purpose or the region of interest.
Segmenting colored images
We first introduce a new concept here which is the normalized chromaticity coordinates. Instead of the usual RGB values, we represent the image as rgI (red, green, intensity) which are defined as
Below shows the plot for the rgI coordinates. Color information is stored in r and g while brightness in formation is stored in I. Note that it is enough to use only r and g due to the relation that r+g+b=1.
We took the color histogram of the cropped sheep below. Comparing it to the plot above shows high points near the violet and blue regions, and some on the green part.
In segmenting colored parts of the images, what is done is to get a region of interest ROI (say blue part of the blue sheep) and create a color distribution from it. From the color distribution, whatever value it has is “back-projected” to the original whole image. Back-projection is the term used when we modify the values of the original image based on the color distribution of the ROI.
In this method, we assume a Gaussian distribution fit on our ROI. The equation below is used to fit a Gaussian dist from the mean and variance of the ROI. This was done for both r and g color values.
Above shows example of parametric segmentation. The ROI (right) shows a small patch of the original image. It is clear that the method was successful in isolating the apple and the grass from the original image.
In this method, instead of trying to fit a distribution, the histogram of the ROI itself is used (similar to the histogram taken above). So from the original image, we pick its corresponding intensity value from the histogram plot.
Above is the results for the non parametric. We call that the non-parametric method was also successful in segmenting the region of interest.
Comparison between the two methods
We used other images to see some similarities and differences of the two methods. Here is of the apple shown earlier and a set of colored flowers.
For the flower, we choose to ROI’s, one of the dark colored flowers (lower right) and one of the violet (upper left) flowers. Below shows the comparison of the two methods.
The left is the ROI, the center is the result using parametric method and right part is the result on using the non-parametric method. One observation especially on the dark ROI is that the upper left flowers were also included for both methods. However, no significant differences were observed for the two.
Next was to use the apple and segment its green part. Below shows the results for the two methods.
The center is the parametric method and the right part is the non-parametric. We see that the non-parametric was more successful in identifying the green parts compared to the other method.