Download Life-long Learning Perception using Cloud Database

Nov 3rd 2013 – IROS 2013 – Cloud Robotics Workshop
Life-long Learning Perception
using Cloud Database Technology
Tim Niemueller, Stefan Schiffer, and Gerhard Lakemeyer
Knowledge-based Systems Group, RWTH Aachen University
Safoura Rezapour-Lakani
Intelligent and Interactive Systems, University of Innsbruck
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Motivation
Identify specific objects of interest in sensor range.
Describe objects by attributes instead of classes
Learn objects over time
Integrate new perception approaches over time
Accommodate various sensor modalities
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
1 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Motivation
Identify specific objects of interest in sensor range.
Describe objects by attributes instead of classes
Learn objects over time
Integrate new perception approaches over time
Accommodate various sensor modalities
Distributed Robot Perception Database
Extend perception system and object database over time
and share it among robots.
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
1 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Perception Classifier Cascades
Find a red apple.
apple
No
Yes
color:red
Matches?
No
Yes
Result
Candidates
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
2 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Architecture
Data
...
Objects
Classifiers
Base
SURF
Color
Attribute
pepper
color:red
Meta
Niemueller, Schiffer, Lakemeyer, Lakani
···
Haar
color:yellow
VFH
apple
Query: {color : red, apple}
Shape
...
Descriptors
Attributes
Queries
Nov 3rd 2013 @ IROS 2013
3 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Perception Database Requirements
Flexible Data Structures
varying/evolving data structures
Data Management
unified storage architecture
replication, backup and restore
Flexible and Efficient Retrieval
query for specific data
low-overhead retrieval of diverse and large data
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
4 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Perception Database Requirements
Flexible Data Structures
varying/evolving data structures
Data Management
unified storage architecture
replication, backup and restore
Flexible and Efficient Retrieval
query for specific data
low-overhead retrieval of diverse and large data
MongoDB, the document-oriented,
schema-less database, is particularly
well-suited to fulfill these criteria.
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
4 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
MongoDB and its Building Blocks
Document-oriented
Grouped key-value pairs
Schema-less
No declaration or enforcement of
particular structure by DB
Collections
Similarly structured documents
Indexing reference frame
Queries
JavaScript based query language
Select based on document fields
Niemueller, Schiffer, Lakemeyer, Lakani
{ // attributes/classifiers
// for specific object
"_id" : ObjectId("50e..."),
"data_id" : "apple_1_1_10_c"
"scene_id" : "apple_1_1_10"
"attributes" : {
"apple" : true,
"color" : "red"
}
"classifiers" :
["SIFT", "SURF", "Gabor",
"Haar", "Color", "VFH" ...]
}
{ // classifier info excerpt
// for attribute doc
"_id" : ObjectId("52..."),
"data_id" : "apple_1_1_10_c"
"VFH" : {
"model_file" :
"apple_1_1_10_c_vfh.txt"
"extract_time" : 20
},
// [...]
}
Nov 3rd 2013 @ IROS 2013
5 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
MongoDB in our Perception System
Scenes
Objects
Raw Data
Soda
Image, Point
Cloud, ...
Data
Sharing
Classifiers
Apple
SIFT, VFH, ...
Attributes
Scene
Cup
attributes:
apple, color:red
classifiers:
sift, vfh, ...
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
6 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Query Example
Candidates
{
attributes: {
"apple": true,
"color": "red"
}
Q = {color : red, cup}
}
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
7 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Query Example
apple
color:red
Candidates
CqD = D.classifiers,
where q ∈ D.attributes
Cq
=
T
CqD
CqD 6=∅
Niemueller, Schiffer, Lakemeyer, Lakani
docs = db.attributes.aggregate(
{$match: { "attributes.color": "red"}},
{$project: {classifiers: 1}},
{$group: {_id: "$classifiers"}})
}
set_intersect(docs, "classifiers");
Nov 3rd 2013 @ IROS 2013
7 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Query Example
apple
Yes
color:red
No
No
Yes
Result
Matches?
Candidates
Oq = Cq (input)
T
O =
Oq
Apply classifiers
Filter through cascade
q∈Q
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
7 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Why a Cloud Database?
Flexible Data Structures
Easily accommodate various data types
Quickly grow w/o tedious specification
Query Capabilities
Formulate queries deep into data structures
Major benefit over traditional file system storage
Distributed Data
Replicate for off-line training
Quick boot-strapping for new robots/methods
Sharding for multi-host robots
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
8 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Full Retraining
500
450
DB write
SIFT
Gabor
Color
Cylinder
400
DB read
SURF
Shape
VFH
Sphere
Time [ms]
350
300
250
200
150
100
50
0
0
500
1000
1500
2000
# of iterations
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
9 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Full Retraining
Time [ms]
Gabor training magnified 4x
500
450
DB write
50SIFT
Gabor
Color
Cylinder
400
DB read
SURF
Shape
VFH
Sphere
Time [ms]
350
DB Read
300
250
DB Write
200
Training
150
100
50
500
0
0
500
Niemueller, Schiffer, Lakemeyer, Lakani
1000
1500
2000
# of iterations
Nov 3rd 2013 @ IROS 2013
1
9 / 10
Life-long Learning Perception using Cloud Database Technology
Motivation
Perception Database
Evaluation
Conclusion
Conclusion and Questions
Cloud database for perception which is extensible
in terms of known objects and perception methods.
Attribute-based perception
Flexible storage w/ MongoDB
Increasing number of objects
Capable query features
Evolve perception methods
Share data among robots
www.fawkesrobotics.org
Niemueller, Schiffer, Lakemeyer, Lakani
Nov 3rd 2013 @ IROS 2013
10 / 10