Download PowerPoint Presentation - VARS Overview

MBARI’s Video Annotation and
Reference System (VARS)
Brian Schlining and Nancy
Jacobsen-Stout
MBARI
MBARI
MBARI Video Data
• More than 300 ROV dives per year
• More than 12,000 hours of video observations
generated in 16 years
• More than 1.6 million individual observations
Scientific Applications
Video observations and related data
• Used by MBARI researchers in well over 200 professional publications and
presentations, in addition to countless education and outreach projects
• Span multiple sub-disciplines within biology, geology, chemistry
“A picture can be worth a thousand words”
(if you can find it!)
Benthic Diversity
Interesting Taxa (gorgonocephalus)
Annotations
• Trained science staff identify animal
taxonomy, animal behavior, geological
features, evidence of human impact, and
other objects recorded in the ROV video
stream
• Annotations are stored in a database
shared between MBARI scientists.
• Database continues to evolve as our
researchers learn more
• Work closely with and rely on scientists
to keep updated on new discoveries and
publications in a timely fashion
Annotation History
1st generation – Free Text
– Inconsistent annotations
– Difficult to search
2nd generation – VICKI/VIMS
– Constrained annotations
– Written in Smalltalk
– Generated files
3rd generation – VARS
– Constrained annotations
– Written in Java
– Writes information directly to database
VARS - System Components
Knowledge Base = hierarchical constraint lexicon & reference
More than 3500 taxonomic, geologic, technical concepts.
Reference media and descriptions.
Annotation = catalog video observations, inc. visual media
Uses the KB data to create and edit structured annotations for
objects in the video stream.
Query = retrieve video-related data
Uses the KB to structure simple to complex queries against
annotations database
VARS - Knowledge Base
Manages constraint lexicon and references
VARS - Annotation
Generates video-annotations and frame-grabs
Frame capture
(QuickTime for Java)
VCR Control
(RS-422)
VARS - Annotation
Generates video-annotations and frame-grabs
Knowledge Base
Customizable Interface
VARS - Query
Searches for and retrieves information and frame-grabs.
VARS - Query
Searches for and retrieves information and frame-grabs.
VARS - Query
Searches for and retrieves information and frame-grabs.
Example – “Cold Seeps”
Vesicomyid Clams
Vestimentiferan
Tubeworms
“Cold Seep” Map Vintage 1999
“Cold Seep” Map Vintage 2004
Monterey bay is one of
the most observed
continental margins in
the world
Visited 0.9% pixels in
this map
Example – “Cold Seeps”
• Previous “seep” models assumed seeps
are associated with faults (fluid low
conduits)
Chemosynthetic Biological Communities
occur preferentially on steep slopes?
Steep slopes imply areas of recent erosion.
VARS Development Goals
VARS user goals include:
1) Provide responsive user interfaces to maximize user efficiency and
accommodate real- and greater than real-time analyses.
2) Provide a robust system.
3) Provide clear and intuitive user interfaces.
VARS developmental goals include:
1) Create maintainable (modular, understandable) software for future
developers.
2) Maximize use of existing software (e.g. old VIMS query and commercially
available software)
3) Provide a solution that can be exported to other institutions.
4) The database should be simple to maintain by MBARI IS administrators.
VARS Requirements
Vars system requirements:
1) Java 1.4 or greater
2) Java Comm (or RXTX)
3) Quicktime for Java (For frame-capture)
4) Relational Database
VARS Deployment
VARS Deployment
Point Lobos
Shore
Western Flyer
VARS Deployment
Point Lobos
Shore
Western Flyer
VARS - Annotation data
VARS - Annotation data
A related group of tapes.
e.g. tapes from a single expedition
VARS - Annotation data
A video source, such as a
tape or QuickTime movie
VARS - Annotation data
A single video frame
VARS - Annotation data
An object observed in a particular video-frame.
The name is constrained by the knowledgebase
VARS - Annotation data
Descriptive information. Can also be used to
link ‘concepts’. For example, ‘nanomia eating krill’
VARS - Annotation data
Position, physical data, expedition information,
and camera data,
VARS - Annotation data
VARS - Knowledgebase data
VARS - Knowledgebase data
~ Hierarchical, Allows representation of
relations or phylogeny
~ Linked to descriptive info. (History,
authors, images, video, etc. [not shown])
VARS - Knowledgebase data
~ A concept may have one or more
concept-names.
~ One concept-name is designated as a
primary name.
~ User may annotate with any concept-name.
However, only the primary name is stored
in the database.
VARS - Knowledgebase data
VARS – Available to you
VARS is available to other science institutions.
See http://www.mbari.org/vars/
Fall 2005 – Release VARS as open-source
December 2005 – MBARI will host a developer workshop
If you are interested in using VARS contact:
Judith Connor ([email protected])
Director of Information & Technology Dissemination