Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The Vision and Reality of Ubiquitous Computing Prof. Henning Schulzrinne Dept. of Computer Science Columbia University (with Arezu Moghadam, Ron Shacham, Suman Srinivasan, Xiaotao Wu and other IRT members; parts in cooperation with DoCoMo Eurolabs) August 8, 2007 Mobiquitous 2007 Overview • The original vision of ubiquitous computing • User challenges • Beyond terminal mobility • Location as new core service • Universality: 7DS August 8, 2007 Mobiquitous 2007 Ubiquitous computing ubiquitous communications • “It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.” • Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”, 1996) – – – – – “one person, many computers” many computers embedded in environment dynamic ownership PC phonebooth “IR use will grow rapidly” • Updated version, 2007 – – – – – not physically invisible, but transparent emphasis on communications, not computing most devices are mobile (or nomadic) cheap electronics personal devices radio (channelized and UWB) August 8, 2007 Mobiquitous 2007 Overview • The original vision of ubiquitous computing • User challenges • Beyond terminal mobility • Location as new core service • Universality: 7DS August 8, 2007 Mobiquitous 2007 User challenges vs. research challenges • Are we addressing real user needs? – Engineering vs. sports • My guesses ease of use reliability no manual no re-entry no duplication integration phishing data loss cost August 8, 2007 limited risk Mobiquitous 2007 Example: Email configuration • Application configuration for (mobile) devices painful • SMTP port 25 vs. 587 • IMAP vs. POP • TLS vs. SSL vs. “secure authentication” • Worse for SIP... August 8, 2007 Mobiquitous 2007 Example: SIP configuration • • • • • partially explains highly technical parameters, with differing names inconsistent conventions for user and realm made worse by limited end systems (configure by multi-tap) usually fails with some cryptic error message and no indication which parameter out-of-box experience not good August 8, 2007 Mobiquitous 2007 Mobile why’s • • • • • • • • • Not research, but examples of real annoyances Why does each mobile device need its own power supply? Why do I have to adjust the clock on my camera each time I travel? Why do I have to know what my IMAP server is and whether it uses TLS or SSL? Why do I have to type in my address book? Why do I have to “synchronize” my PDA? Why do I have to manually update software? Why is connecting a laptop to a projector a gamble? Why do we use USB memory sticks when all laptops have 802.11b? August 8, 2007 Mobiquitous 2007 Consumer wireless & mobile devices Prius key Garage door opener TAN display Water leak alarm wireless door bell MSN Direct weather August 8, 2007 Mobiquitous 2007 Mobile systems - reality GPS • idea: special purpose (phone) --> universal communicator – • idea is easy... QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. mobile equipment: laptop + phone – • • QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. sufficiently different UI and capabilities location data we all know the ideal (converged) cell phone difficulty is not technology, but integration and programmability – – – – (almost) each phone has a different flavor of OS doesn’t implement all functionality in Java APIs no dominant vendor (see UNIX/Linux vs. Microsoft) external interfaces crippled or unavailable • QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. e.g., phone book access QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime™ and a TIFF (Uncompress ed) dec ompres sor are needed to s ee this pic ture. August 8, 2007 Mobiquitous 2007 Example: displays and speakers August 8, 2007 Mobiquitous 2007 The mobile ubiquitous challenge Mobile phone Mobile Internet access Interconnected devices “Internet of things” August 8, 2007 Mobiquitous 2007 What do we need? • Standards, not new technology • Radio connectivity – 802.11a/b/g/n, 802.15.4 – better discovery of networks • Location information everywhere • Discovery: devices & services – network-local discovery via Bonjour (mDNS) – missing: location-based discovery • Advanced mobility: session, personal, service • Event notification • Data formats – location, sensor events, ... August 8, 2007 Mobiquitous 2007 Examples of “invisible” behavior • MP3 player in car automatically picks up new files in home server • A new email with vcard attachment automatically updates my cell phone address book • The display of my laptop appears on the local projector – without cable or configuration • I can call people I just met at Mobiquitous – without exchanging business cards • • • • My car key opens my front door My cell phone serves as a TAN (one-time password) generator My cell phone automatically turns itself off during a lecture My camera knows where the picture was taken August 8, 2007 Mobiquitous 2007 An interconnected system opens doors generates TAN incoming call updates location time, location address book alert, events any weather service school closings August 8, 2007 acoustic alerts Mobiquitous 2007 Thinking beyond 802.11 and UMTS • Many interesting networks beyond those covered in conferences – ease of access by researchers vs. importance – 90% of papers on 802.11b and maybe GPRS, BlueTooth • New wireless networks – – – – – – – – broadcast instead of unicast -- useful for many ubiquitous applications S5 for low-rate sensors (city scale) QuickTime™ and a Zigbee (802.15.4) for local sensors (20 - 250 kb/s) TIFF (Uncompressed) decompresso are needed to see this picture. FM subcarrier (not really new) -- MSN Direct FM Radio Data System -- TMC Sirius / XM HD radio paging August 8, 2007 Mobiquitous 2007 Overview • The original vision of ubiquitous computing • User challenges • Beyond terminal mobility • Location as new core service • Universality: 7DS August 8, 2007 Mobiquitous 2007 Application-layer mobility • terminal mobility – one terminal, multiple network addresses • Personal mobility – one person, multiple terminals – e.g., Grandcentral • session mobility – one user, multiple terminals in sequence or in parallel • service mobility – services move with user August 8, 2007 Mobiquitous 2007 Session mobility • Walk into office, switch from cell phone to desk phone – call transfer problem SIP REFER • related problem: split session across end devices – e.g., wall display + desk phone + PC for collaborative application – assume devices (or stand-ins) are SIP-enabled – third-party call control August 8, 2007 Mobiquitous 2007 R. Shacham, H. Schulzrinne, S. Thakolsri, W. Kellerer, “Ubiquitous device personalization and use: The next generation of IP multimedia communications”, ACM TOMCCAP, May 2007 How to find services? • Two complementary developments: – – smaller devices carried on user instead of stationary devices devices that can be time-shared • • • • • • Need to discover services in local environment – SLP (Service Location Protocol) allows querying for services • • – – – • “find all color displays with at least XGA resolution” slp://example.com/SrvRqst?public?type=printer SLP in multicast mode SLP in DA mode Apple Bonjour Need to discover services before getting to environment – – – August 8, 2007 large plasma displays projector hi-res cameras echo-canceling speaker systems wide-area network access “is there a camera in the meeting room?” SLP extension: find remote DA via DNS SRV LoST to find services by geographic location Mobiquitous 2007 Session mobility Local Devices Transcoder Internet SLP DA SLP SA SLP UA SIP SM SIP UA SIP UA Correspondent Node (CN) August 8, 2007 SIP SM SIP UA SLP UA Mobiquitous 2007 Mobile Node (MN) SLP SIP RTP Overview • The original vision of ubiquitous computing • User challenges • Beyond terminal mobility • Location as new core service • Universality: 7DS August 8, 2007 Mobiquitous 2007 Context-aware communication • • • context = “the interrelated conditions in which something exists or occurs” anything known about the participants in the (potential) communication relationship both at caller and callee time chronology, uniqueness capabilities caller preferences location location-based call routing location events (emergency alerts) security (access control) service discovery activity/availability presence sensor data (mood, bio) privacy issues similar to location data August 8, 2007 Mobiquitous 2007 GEOPRIV and SIMPLE architectures rule maker DHCP XCAP (rules) target presentity caller August 8, 2007 publication interface PUBLISH INVITE location server presence agent notification interface location recipient GEOPRIV watcher SIP presence SUBSCRIBE NOTIFY INVITE Mobiquitous 2007 callee SIP call Presence data architecture presence sources PUBLISH create view (compose) raw presence document XCAP select best source resolve contradictions depends on watcher XCAP privacy policy composition policy (not defined yet) August 8, 2007 privacy filtering draft-ietf-simple-presence-data-model Mobiquitous 2007 Presence data architecture candidate presence document remove data not of interest watcher August 8, 2007 raw presence document watcher filter post-processing composition (merging) SUBSCRIBE difference to previous notification NOTIFY Mobiquitous 2007 final presence document Presence data model person “calendar” “cell” “manual” (presentity) (views) services [email protected] audio, video, text [email protected] video devices August 8, 2007 Mobiquitous 2007 RPID = rich presence • • Provide watchers with better information about the what, where, how of presentities facilitate appropriate communications: – – – • designed to be derivable from calendar information – • “wait until end of meeting” “use text messaging instead of phone call” “make quick call before flight takes off” or provided by sensors in the environment allow filtering by “sphere” – the parts of our life – don’t show recreation details to colleagues August 8, 2007 Mobiquitous 2007 Rich presence • More information: for (authorized) people and applications • automatically derived from – sensors: physical presence, movement – electronic activity: calendars • Rich information: – multiple contacts per presentity • • – – – – – – device (cell, PDA, phone, …) service (“audio”) activities, current and planned sphere (home vs. work) current user mood surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …) August 8, 2007 Mobiquitous 2007 Presence and privacy • • All presence data, particularly location, is highly sensitive Basic location object (PIDFLO) describes – distribution (binary) – retention duration • Policy rules for more detailed access control – who can subscribe to my presence – who can see what when August 8, 2007 <tuple id="sg89ae"> <status> <gp:geopriv> <gp:location-info> <gml:location> <gml:Point gml:id="point1“ srsName="epsg:4326"> <gml:coordinates>37:46:30N 122:25:10W </gml:coordinates> </gml:Point> </gml:location> </gp:location-info> <gp:usage-rules> <gp:retransmission-allowed>no </gp:retransmission-allowed> <gp:retention-expiry>2003-06-23T04:57:29Z </gp:retention-expiry> </gp:usage-rules> </gp:geopriv> </status> <timestamp>2003-06-22T20:57:29Z</timestamp> </tuple> Mobiquitous 2007 Events as missing Internet capability • aka PUB/SUB • Used across applications, e.g., – – – – – – – – email and voicemail notification presence replace RSS (= poll!) web service completion emergency alerts (“reverse 9-1-1”) network management home control data synchronization • Rich research history – but too complex, optimize the wrong thing • XMPP and SIP as likely transport candidates August 8, 2007 Mobiquitous 2007 Local Switch Automatic Number Identification Automatic Location Identification August 8, 2007 Collaboration between local phone providers and local public safety agencies Mobiquitous 2007 911 technology failures • NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07: – “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the location of the cellphone callers, though the technology to do so has been available for at least five years.” “In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died in a house fire after a 911 operator who lacked the technology to pinpoint the call misheard the address.” • Phase II wireless; billions of dollars spent • In Mississippi, only 1 of out 5 counties – “As it ages, it is cracking, with problems like system overload, understaffing, misrouted calls and bug-ridden databases leading to unanswered calls and dangerous errors.” • operator (CAMA) trunks, with 8-digit number delivery • MSAG and ALI databases August 8, 2007 Mobiquitous 2007 Location delivery GPS HELD HTTP wire map DHCP LLDP-MED August 8, 2007 Mobiquitous 2007 Location, location, location, ... Voice Service Provider (VSP) sees emergency call but does not know caller location ISP/IAP knows user location but does not handle call August 8, 2007 Mobiquitous 2007 Options for location delivery • Wireless – – – – • GPS S5 wireless (active sensors + triangulation) Skyhook (802.11 in urban areas) HDTV L2: LLDP-MED (standardized version of CDP + location data) – periodic per-port broadcast of configuration information • L3: DHCP for – geospatial (RFC 3825) – civic (RFC 4676) • L7: proposals for retrievals: HELD, SIP, … – – – – – for own IP address or by third party (e.g., ISP to infrastructure provider) by IP address by MAC address by identifier (conveyed by DHCP or PPP) HTTP-based August 8, 2007 Mobiquitous 2007 Locating Caller using LLDP-MED LLDP-MED stands for: * * From Wikipedia Link Layer Discovery Protocol “a vendor-neutral Layer 2 protocol that allows a network device to advertise its identity and capabilities on the local network.” Media Endpoint Discovery “an enhancement to the LLDP that allows discovery of other things including location “ “I am LLDP-MED Capable. I can process location information.” LLDP-MED SWITCH CALLER EQUIPMENT August 8, 2007 “Your location is: 500 W 120TH st. New York NY 10027” Mobiquitous 2007 Location determination options Method CDP or LLDPMED DHCP HELD GPS manual entry Layer L2 L3 L7 (HTTP) - user advantages • simple to implement • built into switch • direct port/room mapping • simple to implement • network locality • traverses NATs • can be operated by L2 provider • accurate • mobile devices • no carrier cooperation • no infrastructure changes • no carrier cooperation problems may be hard to automate for large enterprises mapping MAC address to location? mapping IP address to switch port? • indoor coverage • acquisition time • fails for mobile devices • unreliable for nomadic Use Ethernet LANs Enterprise LANs Some ISPs DSL, cable mobile devices fall back August 8, 2007 Mobiquitous 2007 SIP message for Location Info. INVITE urn:service:sos SIP/2.0 To: urn:service:sos Call-ID: [email protected] Via: SIP/2.0/TCP 192.168.1.106:4064;rport Content-Type: multipart/mixed; boundary From: sip:[email protected] Contact: <sip:[email protected]:5060> CSeq: 1 INVITE Content-Length: 1379 request line header fields ------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY= MIME-Version: 1.0 content-Type: application/sdp Content-Transfer-Encoding: 8bit v=0 o=eddie 1127764654 1127764654 IN IP4 192.168.1.106 s=SIPC Call c=IN IP4 160.39.54.70 t=0 0 m=audio 10000 RTP/AVP 0 3 m=video 20000 RTP 31 SDP August 8, 2007 ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY= MIME-Version: 1.0 Content-Type: application/pidf+xml Content-Transfer-Encoding: 8bit <?xml version="1.0" encoding="ISO-8859-1"?> <presence xmlns="urn:ietf:params:xml:ns:pidf" xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10" xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc" xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0" entity="sip:[email protected]"> <tuple id="28185"> <status> <gp:geopriv> <gp:location-info> <cl:civilAddress> <cl:country>us</cl:country> <cl:A1>ny</cl:A1> <cl:A2>new york</cl:A2> <cl:A3>new york</cl:A3> <cl:A6>amsterdam</cl:A6> <cl:HNO>1214</cl:HNO> </cl:civilAddress> </gp:location-info> <gp:method>Manual</gp:method> </gp:geopriv> </status> <contact priority="0.8">sip:[email protected]:5060</contact> <timestamp>2005-09-26T15:57:34-04:00</timestamp> </tuple> </presence> ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=-- Mobiquitous 2007 PIDF-LO August 8, 2007 Mobiquitous 2007 Tracking August 8, 2007 Mobiquitous 2007 Data formats: location • Civic (street) – jurisdictional & postal • Geo (longitude & latitude) – point, polygon, circle, … • see GeoRSS for simple example August 8, 2007 Mobiquitous 2007 ECRIT: LoST Functionality • Civic as well as geospatial queries – • • civic address validation Recursive and iterative resolution Fully distributed and hierarchical deployment – – • Indicates errors in civic location data debugging – • but provides best-effort resolution Can be used for non-emergency services: – – directory and information services pizza delivery services, towing companies, … can be split by any geographic or civic boundary same civic region can span multiple LoST servers August 8, 2007 <findService xmlns="urn:…:lost1"> <location profile="basic-civic"> <civicAddress> <country>Germany</country> <A1>Bavaria</A1> <A3>Munich</A3> <A6>Neu Perlach</A6> <HNO>96</HNO> </civicAddress> </location> <service>urn:service:sos.police</service> </findService> Mobiquitous 2007 LoST: Location-to-URL Mapping VSP1 cluster serving VSP1 replicate root information cluster serves VSP2 123 Broad Ave Leonia Bergen County NJ US LoST NJ US root NY US sip:[email protected] nodes search referral Bergen County NJ US Leonia NJ US August 8, 2007 Mobiquitous 2007 LoST Architecture G tree guide G G G T1: .us G broadcast (gossip) T2: .de resolver seeker 313 Westview Leonia, NJ US T2 T1 (.us) August 8, 2007 (.de) T3 (.dk) Leonia, NJ sip:[email protected] Mobiquitous 2007 Left to do: event notification • notify (small) group of users when something of interest happens – – – – – • presence = change of communications state email, voicemail alerts environmental conditions vehicle status emergency alerts kludges – HTTP with pending response – inverse HTTP --> doesn’t work with NATs • • Lots of research (e.g., SIENA) IETF efforts starting – SIP-based – XMPP August 8, 2007 Mobiquitous 2007 Overview • The original vision of ubiquitous computing • User challenges • Beyond terminal mobility • Location as new core service • Universality: 7DS August 8, 2007 Mobiquitous 2007 Problems with Wide Area Wireless • 802.11 currently hard to deploy across city or large area • Problem: How can mobile devices / gadgets get information while on the move? • Use local peer-to-peer wireless networks to exchange information – Peers can get information they do not have from another peer Solution: 7DS! August 8, 2007 Mobiquitous 2007 S. Srinivasan, A. Moghadam, S.G. Hong, H. Schulzrinne, “7DS - Node Cooperation and Information Exchange in Mostly Disconnected Networks”, ICC '07. June 2007. How 7DS Works 1. When devices are in the same BSS (Basic service set) of 802.11 ad-hoc network, they discover each other using service discovery of Zeroconf zeroconf August 8, 2007 Mobiquitous 2007 How 7DS Works 2. If there is no Internet connection, the devices can communicate with each other to exchange information August 8, 2007 Internet Mobiquitous 2007 Web Delivery Model • 7DS core functionality: Emulation of web content access and e-mail delivery August 8, 2007 Mobiquitous 2007 Design • Peer-to-peer network set up using zeroconf – Protocol enables devices to communicate with each other without a DHCP server, a DNS server and a Directory server • • • • August 8, 2007 Proxy server serves content Search engine searches for local data MTA store and forward In progress: File synchronization, BBS Mobiquitous 2007 Store and Forward • Forwarding e-mail in the ad-hoc network • Acts as an MTA August 8, 2007 Mobiquitous 2007 Search Engine • Provides ability to query self for results • Searches the cache index using Swish-e library • Presents results in any of three formats: HTML, XML and plain text • Similar in concept to Google Desktop August 8, 2007 Mobiquitous 2007 Query Multicast Engine • • • • Used to actually exchange information among peers Requesting peer broadcasts a query to the network Responding peers reply if they have information – Send encoded string with list of matching items Requesting peer retrieves suitable information August 8, 2007 Mobiquitous 2007 File Synchronization SERVICE ANNOUNCEMENT FILE SYNCHRONIZATION RESOLUTION USING RSYNC PROTOCOL SRV : 7ds-fs1.filesync._7ds._udp.local. “I want 7ds-device1.local:2525 Word.doc and presentation.ppt” TXT : file1.xml TXT : file2.xml SRV : 7ds-fs2.filesync._7ds._udp.local. “I want 7ds-device2.local:2525 File1.xml and file2.xml” TXT : word.doc File1.xml File2.xml TXT : presentation.ppt Word.doc Presentation.ppt August 8, 2007 Mobiquitous 2007 Word.doc Presentation.ppt File1.xml File2.xml Conclusion • Motivate mobile ubiquitous research by user problems • From stovepipe mobile & wireless systems to personal and shareable wireless networks • Thinking beyond single applications – presence event notification – 9-1-1 location time & location as infrastructure • Need new models of creating services – domain-specific languages, not Java APIs August 8, 2007 Mobiquitous 2007