Download View slides

yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Document related concepts

Cluster analysis wikipedia , lookup

John Tait
Chief Scientific
• A few half formed ideas from the world of
image and video indexing which may be of
interest to MT people
• Not original ideas (apart from I think the
• In fact a line of work which derives
originally from MT
The nub
• Unsupervised Clustering of bundles of
– Colour, texture from image segments
– Words, phrases from sentences or paragraphs ?
• Associate these bundles with
“translations” by supervised machine
– Categorised images
– Parallel texts
• “Matching Words and Pictures”: Barnard,
Duygulu, Forsyth, de Freitas, Blei, Jordan.
Journal of Machine Learning Research 3
(2003) 1107-1135
• “Image Classification Using Hybrid Neural
Networks” Tsai, McGarry and Tait.
Proceedings of the 26th ACM SIGIR
Conference on Research and Development
in Information Retrieval (SIGIR 2003),
Toronto, July, 2003. pp 431-432.
More or less general
General Concepts
Specific Concepts
• Derived from Visiterms
– These feature cluster nodes
– A notion of an area of the “semantic field”
– Remember these are colour, texture etc. for
an area of an image …. No relation to
language … or at least a very deep one
L1 General Concepts
L1 Specific Concepts
L2 General Concepts
L2 Specific
Fast Forward to 2009
• Better statistical models tuned to the data
• Much Bigger vocabularies of words
• … and lots of other advances
A question
• Is there anything like this current MT
research ?
Concluding remarks
• I’m surprised this worked at all
– Why should image data be coherent and
cohesive ?
• But text is !!!
• Is this a better way to deal with unknown
and changing vocabulary
Some other references
A Correlation Approach for Automatic Image Annotation Hardoon, D.,
Saunders, C., Szedmak, S. and Shawe-Taylor, J. (2006) A Correlation
Approach for Automatic Image Annotation. In: Second International
Conference on Advanced Data Mining and Applications, ADMA 2006,
August, Xi'an, China.
Kucuktunc, O., Sevil, S. G., Tosun, A. B., Zitouni, H., Duygulu, P., and
Can, F. 2008. Tag Suggestr: Automatic Photo Tag Expansion Using Visual
Information for Photo Sharing Websites. In Proceedings of the 3rd
international Conference on Semantic and Digital Media Technologies:
Semantic Multimedia (Koblenz, Germany, December 03 - 05, 2008). D.
Duke, L. Hardman, A. Hauptmann, D. Paulus, and S. Staab, Eds.
Lecture Notes In Computer Science, vol. 5392. Springer-Verlag, Berlin,
Heidelberg, 61-73. DOI=