An interesting project I worked on in my AI class is algorithmically recoloring photos. We worked on coloring this photo of a beach to the right.
Visual by Chire - Own work, CC BY-SA 4.0, Link Thanks Chire!
In order to makes things simpler, as this was our first project involving photos, we reduced the space
of colors we were dealing with from over 16 millions with regular RGB down to the 5 most common colors.
To find these five most common colors, we used a method called K means clustering.
The visual to the right
helps explain what K means clustering is. In this example, they are finding 3 centers of 2 dimensional data.
But in my case, I would be finding 5 centers of 3 dimensional data. Each dimension would correspond to one color value
(red, green or blue). Then the program will graph out every single pixel from the image by RGB value.
Then, the program randomly decides on 5 centers. Every single pixel is placed in to a group based off of
it's nearest center. Then the program looks at all of those pixels in each group, and calculates a new center
for each group. These new centers replace the old ones, and the process can be repeated with more accurate centers each time,
quickly converging to the true value. Once this process has been repeated to have converged, the centers become
the 5 chosen colors, and all the pixels in that center's group are assigned that color.
Here we can see the results of this process! I would say this is not too bad for having so few colors.
Now the program is ready to color the image. When deciding on what color to use for a given pixel, the program will use this simplified reference image.
In order to color a given pixel, the computer has to look at more than just the greyscale value of one pixel as one value is simply too little data to go off of.
To get a better idea of what that pixel might be, it can look at a small patch surrounding the pixel.
These "pixel patches" can be thought of as a simple vector of values. We can then look at our reference
image to get an idea of what color that patch may be. We can calculate euclidean distances to find the closest pixel patch.
We can then take that patches color, and color the unknown pixel the same way.
Left - Reference Colors | Right - Recolored Black and White Photo
Here we can see the end result of all of the work. While the recoloring is not perfect, I think it still did a good job for how primitive the technique is. By only looking at a small patch of 3x3 pixels, the computer was able to tell what was foliage, what was sand and what was water. While it did get a bit by the clouds, I think this is understandable. If you look at the reference image, there are very few clouds in this half of the image. Most of them are on the right half.