Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ShareCam: Interface, System Architecture, and Implementation of a Collaboratively Controlled Robotic Webcam Dezhen Song (TAMU) and Ken Goldberg (UC Berkeley) IEEE/RSJ International Conference on Intelligent Robots and Systems Finalist for New Technology Foundation (NTF) Award for Entertainment Robots and Systems Teleoperation: Related Work • • • • • • • • • • • • Tesla, 1898 Goertz, ‘54 Mosher, ‘64 Tomovic, ‘69 Salisbury,Bejczy, ‘85 Ballard, ’86 Volz, ’87Sheridan, ‘92 Sato, ’94 Goldberg, ’94Presence Journal ‘92O. Khatib, et al. ’96 Internet networked robot: Taxonomy (Tanie, Matsuhira, Chong 00) Single Operator, Single Robot (SOSR): Single Operator, Multiple Robot (MOSR): Multiple Operator, Single Robot (MOSR): sharecam Pan, Tilt, Zoom robotic video camera Entertainment Applications Playing Games Related Work • Networked robots – – – – – – – Tanie, K., Chong, N. et al(01) Jia, S. and K. Takase (01) Hu, H., Yu, L., Tsui, P., Zhou, Q (01) Safaric, R. et al. (01) Goldberg and Siegwart (02) Coppin, P. and Wagner, M.D. (02) Konukseven, I., Erkmen, A. et al (02) • SOSR – – – – – Siegwart, R. and Saucy P. (99) Paulos, E. and Canny, J. (99) Tanie, K., Arai, H. et al. (00) Lynch, K. and Liu, C. (00) Fong, T., Thorpe, C., et al(01) Related Work • SOMR – Hu, Yu, Tsui, Zhou (01) – Jia, Takase (01) • MOMR – Fukuda, Xi, Liu, Elhajj et al. (00,02) – Tanie, Chong, et al. (00) • MOSR – Cinematrix (91) – Cannon, McDonald, et al. (97) – Goldberg, Chen, et al. (00, 01) Sharecam: System Architecture Users ShareCam Server Internet Video Server Sharecam Software Login CGI Console/Log Communication Java Core (with shared memory segments) Apache module Apache module User database Registration MySQL Apache module ShareCam web server TCP/IP Calibration Camera control RS232C Canon VCC3 Camera Gnu C++ InetCam server Panoramic image generation HTTP ShareCam applet PERL TCP/IP Video server InetCam applet Client frame selection problem: given n requests, find optimal frame Problem Definition Requested frames: i=[xi, yi, zi], i=1,…,n Problem Definition • “Satisfaction” for frame i: 0 Si 1 = i = i Si = 0 Si = 1 Similarity Metrics • Symmetric Difference Area (i ) Area (i ) SD Area (i ) • Intersection-Over-Union Area (i ) IOU 1 SD Area (i ) Nonlinear functions of (x,y) Intersection over Maximum: s (i , ) ( pi / ai ) min(( zi / z ) ,1) b pi Area (i ) max( ai , a) Max (i , ) Requested frame i , Area= ai Candidate frame Area = a pi • global satisfaction: n S ( ) ( pi / ai ) min(( zi / z ) b ,1) i 1 n S ( x, y ) i pi ( x, y ) i 1 for fixed z find * = arg max S() approximation Compute S(x,y) at lattice of sample points: y d d : lattice spacing d z : resolution spacing whg O( 2 n) d dz x w, h : width and height, g: size range error bound c* Optimal frame ĉ Smallest frame on lattice that encloses c* c~ Optimal at lattice s(c ) s(c~) s(cˆ) s(c~ ) s(cˆ) 1 * s ( c ) s ( c* ) * run time: O(n / 3 ) zmin ... zmin 2d z 1 frame selection algorithms Processing Zoom Type Complexity Centralized Discrete Exact O(n2) Centralized Discrete Approx O(nk log(nk)), k=(log(1/ε)/ε)2 Centralized Contin. Exact O(n3) Centralized Contin. Approx O((n + 1/3) log2 n) Distributed Discrete Exact O(n), Client: O(n) Distributed Contin. Approx O(n), Client O(1/3) ShareCam Application: Game based learning : global environment Collaborative Observatories for Natural Environments (CONEs) sensor networks humans: amateurs and profs. timed checks robotic video cameras motion sensors Thank you. ShareCam: Interface, System Architecture, and Implementation of a Collaboratively Controlled Robotic Webcam Frank van der Stappen (CS, Utrecht) Vladlen Koltun (EECS, UC Berkeley) George Bekey (CS, USC) Karl Bohringer (CS, UW) Anatoly Pashkevich (Informatics, Belarus) Judith Donath (Media Lab, MIT) Eric Paulos (Intel Research Lab, Berkeley) Dana Plautz (Intel Research Lab, Oregon) Sariel Har-Peled (CS, UIUC) Satellite Imaging MIT Press, 2002 Networked Robots • • • • • • Tele-Operation Internet Tele-Operation Collaborative Tele-Operation Tele-Actor Co-Opticon Co-Opticon Algorithms www.ken.goldberg.net Infiltrate Networked Robots internet tele-robot: RoboMotes: Gaurav S. Sukhatme, USC Smart Dust: Kris Pister, UCB (Image: Kenn Brown) Networked Cameras Where to look? Sensornet detects activity • “Motecams” • Other sensors: audio, pressure switches, light beams, IR, etc • Generate bounding boxes and motion vectors • Transmit to PZT camera Activity localization 1. 2. 3. 4. 5. Network Standards: HTML, Browsers, Java Infrastructure: Backbone, Routers, ADSL Public Adoption Bandwidth: 10 Mbps, 100Mbps, Gbps Video/Audio Compression: MP2,3,4 Networked robots Systems that couple communication to control of one or more robots in a network that often includes sensors and remote human operators. Two subclasses : 1) Tele-operated, where human supervisors send commands and receive feedback via the network. Such systems support research, education, and public awareness by making valuable resources accessible to broad audiences. 2) Autonomous, where robots and sensors exchange data via the network. In such systems, the network extends the effective sensing range of individual robots. Challenges: 1. 2. 3. 4. 5. 6. 7. 8. Variable Time Delays, Congestion Latency Access Control, Security, Interfaces Protocol Design Noise, Error Detection and Recovery Deployment, Dynamic Routing Power Management Hybrid Architectures Conventional Security Cameras • Immobile or Repetitive Sweep • Low resolution Future Work • Continuous zoom (m=) • Multiple outputs: – p cameras – p views from one camera • “Temporal” version: fairness – Integrate si over time: minimize accumulated dissatisfaction for any user • Network / Client Variability: load balancing • Obstacle Avoidance Outline • • • • Collaborative Teleoperation Cinematrix Co-opticon Tele-Twister "In times of terror, when everyone is something of a conspirator, everyone will be in a situation where he has to play detective." -- Walter Benjamin (1938) Statistics of Satellite Imaging • 2.5 Billion Market in 2003 • Increasing 300% per year since 1999 • Major clients – Government / Military – Oil exploration – Weather Prediction – Agriculture Ikonos, 1999 Intersection over Maximum: si( ,i) Requested frame i Candidate frame si = 0.20 0.21 0.53 • Staircase Approximation – Exact algorithm O(n3/2 log3 n) [data structure] – Approximation Algorithm O(nk log(nk)), k=(log(1/ε)/ε)2 • Staircase approximation, large constant factors