2/22/2022 Jenny Applequist, Coordinated Science Lab
Prof. Lav Varshney and visiting scholar Alayt Issak have published a book on botanical farming that uses AI art to engage young children.
Written by Jenny Applequist, Coordinated Science Lab
The potential for creating artworks with the help of artificial intelligence (AI) has been attracting increasing interest. In February, CSL predoctoral scholar Alayt Issak and her advisor, Lav Varshney, published a children’s book on which they collaborated, Young McDonald Had a Botanical Farm, whose illustrations were created by Issak using AI tools.
Issak arrived at UIUC in Fall 2021 to conduct research at the intersection of visual art, creativity, and AI. She explains that she and Varshney knew from the start that they wanted to publish a book showcasing AI-generated art, but it took time to choose the direction. For example, since Issak has been curating a collection of AI art from a broad range of fields—so far including not only visual arts but music and biology, among others—they have been (and still are) considering the possibility of a coffee table book reflecting that effort.
The eureka moment happened when Varshney, who is an associate professor in ECE and CSL, was reading a bedtime story to his son and daughter, and realized that a child’s picture book could be an intriguing venue for AI art.
But what would be the subject? “We were in a meeting for research on the information theory of plant communication... and [Varshney] was inspired to write poetry in regard to a farm of sorts,” says Issak.
Young McDonald Had a Botanical Farm consists of new lyrics to the tune of “Old McDonald Had a Farm,” but each verse is about a different kind of edible plant, algae, or fungus rather than an animal. To reflect plants’ real-life communication ability, each species uses some form of audible communication. For example, peas “pop” and tomatoes “squeal,” as recent biological research shows they actually do. The story also features a modern agricultural robot.
Once the words were in place, Issak launched into making the illustrations with AI tools. Specifically, she used a combination of VQGAN (a “vector quantized generative adversarial network”), which can be used to create new images based on existing input given to it, and CLIP (Contrastive Image-Language Pretraining), which is a neural network model that’s been trained on millions of image + text pairs to “understand” language.
CLIP is a bit like a steering wheel, says Issak, “to help VQGAN understand the images that it’s trying to create. It’s like an assistant... that allows the images to be more realistic, because it understands natural language within context. So you give a ‘prompt,’ which is like an instruction, to CLIP; and via that prompt, and an image that you provide as an input, that would collectively create the desired output.”
Ultimately, “You, as the artist and the human, engineer the prompt to capitalize on the novelty of what could come about.”
For each illustration, Issak had the architecture generate numerous options—sometimes around a thousand—and then she chose her favorites for use in the book.
Varshney notes that he has a strong interest in plant-based foods so thinks of farms as places where plants should be growing rather than “pigs oinking.” While the book has connections to important themes of his research, he says, “this was really just a fun project that I hope engages kids!”
Young McDonald Had a Botanical Farm was released in February 2022 and is currently for sale only on Amazon, in Kindle and paperback formats. Proceeds will be donated to nonprofit organizations in Issak’s home country of Ethiopia, with initial donations going to humanitarian work in Tigray, and subsequent proceeds going to agricultural research.