Two researchers from Rutgers, Ahmed Elgammal and Babak Saleh, created an algorithm capable of classifying the works of art and determine, which paintings are the most innovative and unique. They built a database of 81,449 digital images of paining created by 1,119 artists from the 15th century to the present day. Combined with Wikiart collection, which has some 62,000 images, it gave their program plenty of diverse data to work with.
The accuracy with which the algorithm identifies paintings by artist reaches 63%, by genre — 60% and 45% by style. Not terribly earth-shuttering results, but the algorithm is still in development.
Elgammal and Saleh use a visual classification system that breaks down objects and types of scenes into categories called “classemes.” These groups can be simple (like basic shapes and colors), more complicated (like spotting a barn or the Empire State Building), or complex (a dead body, a figure running away). The algorithm can break a painting down into as many as 2,559 classemes.
As the algorithm goes through paintings, it also forms connections (like a web or network) between paintings based on their chronological age and what classemes they include.
This is how the algorithm draws conclusions about creativity and innovation. When a painting component shows up for the first time or in a novel way, it indicates originality. As MIT Technology Review points out, this approach uses network theory in a similar way to tracking epidemics, finding the source of traffic, or tracing popular people in social networks.
“In most cases the results of the algorithm are pieces of art that art historians indeed highlight as innovative and influential,” Elgammal and Saleh told Tech Review.
While at it, the algorithm made a discovery, however modest.
The algorithm found that the picture of the Frenchman Frederic Bazille L’Atelier de la rue de la Condamine (1870) at the beginning of this post, and the painting of the American artist Norman Rockwell Shuffleton’s Barbershop (1950) are quite similar. The art history has no mention of this fact.
The algorithm determined that the objects in yellow circles are very similar, marked in red — have similar composition while the ones outlined in red have similar structural elements. The researchers are convinced that without any exaggeration it can be concluded that their computer algorithm has made a discovery, albeit small.
In their exemplary modesty, Elgammal and Saleh do not believe that such algorithms will replace art historians any time soon but there is no doubt that as computer programs learn to “understand” painting paintings better and better, the accuracy of identification will significantly improve making many interesting discoveries.
Those who have met the article with particular enthusiasm voice a hope that algorithms will learn not only “understand”, but eventually learn to create masterpieces entirely on their own.
In all honesty, if I won’t live long enough to see it happen, I won’t regret all that much.
Google “dreams,” created entirely by artificial neural networks, weird and mesmerizing, impress me, yes, but not in a way paintings of old (and no so old) masters do.