Download Query processing in Multimedia Database. Dheeraj Kumar Mekala

Query processing in Multimedia
Database.
Dheeraj Kumar Mekala
Devarasetty Sai Bhanu Kiran
Introduction
The increasing development of advanced multimedia applications requires technologies to organize and
retrieve multimedia data in an effective manner, such type of multimedia data are stored in a Multimedia
Database. “A multimedia database is a controlled collection of multimedia data items such as text, images,
graphic objects, video and audio [6]”.
Major Approaches
“Content Based Approach” and “Tree pattern matching Approach” are two major approaches to query a
Multimedia database. Content Based Approach queries the Database on the basis of the content that is
present in the multimedia data files [1]. Querying a database based on the color or shape of the content
present in the multimedia data file is an example of Content Based Approach. In Tree pattern matching
Approach, data is stored in the form of objects that are hierarchically stored in a tree fashion. In the tree
pattern matching approach the user query is converted into a tree structure and the query tree is matched
with the data tree to retrieve the output.
Query Based on Image Content (QBIC) [1]
IBM’s QBIC is based on the notion of “A picture is worth thousand words”. A research team at IBM’s
Almaden Research Center has developed a database technology which allows users to query images and
videos based on its contents rather than writing it in a query language. The contents of an image would be
its color, shape and the text associated with it. When an image is being scanned, the QBIC the system
performs various calculations on the image and extracts the details about its color, layout, contrast and
directionality. The user is provided with a GUI where he can enter his query in the form of colors, layout or
by using text.
Color Query Interface [3]
Layout Query Interface [4]
When a user performs a query the QBIC looks for all the images that match the user’s query condition and
pulls up all the related images and displays them as thumbnails (10 or 20 images in number) on the screen.
In QBIC complex queries like “Find all images with blue color at the top and a triangle shape in the center
of the image” can be performed. QBIC is already being used by some of the IBM clients one the major one
being Russia’s Hermitage Museum which is located in St.Petersburg [5]. People can search for historical
images on the museums website and the image search is powered by QBIC [5]. IBM has already
incorporated this technology into two of their products namely Ultimedia® Manager and DB2 Extensions.
Tree Pattern Matching Approach [2]:
This framework assumes that both the user query and the multimedia data abide by the MPEG-7 standard
which is used to describe multimedia data. In their framework the authors used a tree embedding
approximation algorithm and ontology to support semantic retrieval of multimedia data. The MPEG–7
(Multimedia Content Description Interface) standard uses XML to describe the multimedia data. The major
drawback of XML is that it cannot retrieve implicit data because XML does not have inference capabilities
associated with its elements. In order to supplement the semantic search the authors used ontology which is
a data model that defines a set of classes and the relationships between those classes. The advantage of
using ontology in their framework can be explained as follows. If we have a set of MPEG-7 descriptions of
insect images as shown in the figure1.1 below.
1.1 MPEG – 7 Descriptions of Insect Images
A query to retrieve images of “flying insect” would just retrieve the image ‘a’. This is due to the lack of
inference capabilities in XML. But if we have an ontology which represents the insect domain as shown in
the figure1.2 the result for the same query would retrieve all the three images.
1.2 Ontology
The complete framework for the semantic retrieval is as shown below
There are four major components in this framework namely:
MPEG-7 Metadata Generator: This component is used for the generation of metadata (color, size, etc)
which is guided by the appropriate ontology.
MPEG-7 Query Generator: This component is used to convert the user queries into MPEG-7 format.
Tree Generator: This component is used to convert the MPEG-7 format query into a labeled ordered tree
structure. A labeled tree is the one in which each node has specific label and an ordered tree is the one in
which the parent child relationship and the left to right ordering among siblings are significant. The tree
generator is also used to convert the MPEG-7 data format into a data tree.
Searching Strategy: This component is based on the tree embedded approximation algorithm which is
used to match the user query tree against the MPEG-7 data tree and retrieve the appropriate results for the
user’s query.
Suppose if we want to query a basketball video segment where the referees whistle a fault. The query tree
for such is as shown in figure (a) and the MPEG-7 data tree is shown in figure (b). We note that D4 is
retrieved for Q1 even though both the labels are different. This is because the label umpire is subsumed by
the label referee. This shows the framework’s support for semantic query retrieval.
Conclusion
After comparing the two querying techniques we conclude that the semantic tree pattern matching approach
works better than the QBIC’s content based retrieval approach since the tree pattern matching approach
supports semantic queries which are not supported by QBIC. Our claim can be proved by the following
example: Consider an image database containing thousands of fish images. In the content based image
retrieval scenario a user can only retrieve images of a specific kind of fish based upon its color and layout
whereas in the semantic tree pattern matching retrieval scenario the user can perform semantic query to
retrieve all fish images irrespective of their color or shape. QBIC’s content based retrieval can be improved
by incorporating an ontology similar to the semantic tree pattern matching approach.
References
[1] “IBM’s Query by Image Content”. [Online]. Available:
http://domino.research.ibm.com/comm/wwwr_thinkresearch.nsf/pages/image396.html [Accessed: February
12, 2007]
[2] Samira Hammiche, Salima Benbernou, MohandSa¨ıd Hacid, Athena Vakali “Multimedia database
query processing and retrieval: Semantic retrieval of multimedia data”, Proceedings of the 2nd ACM
international workshop on Multimedia databases MMDB '04, ACM Press 2004
[3] “Hermitage Museum”, [Online]. Available: http://www.hermitagemuseum.org/fcgibin/db2www/qbicColor.mac/qbic?selLang=English [Accessed: February 20, 2007]
[4]”Hermitage Museum”, [Online]. Available: http://www.hermitagemuseum.org/fcgibin/db2www/qbicLayout.mac/qbic?selLang=English [Accessed :February 20, 2007]
[5] “IBM Research Almaden News”, [Online]. Available:
http://www.almaden.ibm.com/almaden/hermitage.html [Accessed: February 19, 2007]
[6] Oya Kalipsiz, “Multimedia Databases”, F o u r t h I n t e r n a t i o n a l C o n f e r e n c e o n I n f o r m a t i o n
V i s u a l i z a t i o n , IEEE, pp. 111, Date: July 2000.