Our Fruit Ontology has been designed with the assistance of three expert agronomists to visually describe the fruit domain.
Moreover, it extends a generic Annotation Ontology, which provides the structure necessary to embed a domain-specific ontology in the annotation tool in order to guide and constrain the annotation process.
See the following VOWL (Visual OWL); high resolution image: zoom in to see classes and properties.
Click here to download the owl file containing the Fruit Ontology definition and its 27 instances (one for each fruit variety in our dataset).
The developed Annotation Tool exploits the knowledge base encoded in the ontology to ensure the annotations correctness and to reduce the experts working time.
Left: experts had only to annotate one target object per image.
Right: bounding box annotations provided by non-experts. All the bounding box labels were inferred automatically from expert annotation through ontology.
Fruit Image Dataset
We used the tool to build the Fruit Image Dataset, a collection of 5,547 images of 3 common fruit species, namely, malus domestica (apple), prunus avium (cherry) and pyrus communis (pear), and 27 fruit varieties in total. We also generated over 110,000 bounding boxes (depicting the different varieties of fruits, leaves, peduncles, etc.) and over 1,000,000 OWL triples (representing high-level knowledge on context and attributes).
|Malus Domestica||Pyrus Communis||Prunus Avium|
|Braeburn||149||4,561||Colombia Red Anjou||250||1,897||Black Tartarian||96||3,683|
|Golden Delicious||289||5,813||Doyenne du Comice||125||1,556||Lapins||118||3,847|
|Pink Lady||260||6,612||Green Anjou||116||1,841||Stella||74||2,698|
|Royal Gala||113||3,520||Gebhard Red Anjou||122||2,347|
|Total Dataset||Total Images||Total Bounding Boxes|
We developed a simple semantic CNN-based object classifer, which is able to exploit the real-world semantics available in the Fruit Image Dataset.
The underlying idea of our classifier consisted of building an ontology instance for test image I and comparing such instance with the C ground-truth instances
(in our case, C = 27, corresponding to the considered varieties as defined in the Fruit ontology), provided by the domain experts, to find the best match. In particular, we treated the
classification problem as a graph matching one; in practice, we grounded ontology instances to weighted graphs and computed the similarity between graphs.
In the following table we report the Mean Classification Accuracy achieved a) when using only low and middle-level visual descriptors (first 4 columns) and b) when integrating high- level knowledge. In the latter case we had a performance increase of about 11%.
|Learning Visual Descriptors||Exploiting High-Level Knowledge|