Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
15-829A/18-849B/95-811A/19-729A Internet-Scale Sensor Systems: Design and Policy Lecture 12 – Name Lookup Name Lookup What do names/descriptions look like? How is the searching done? What type of searches? Search for particular service, browse available services or find collection of services (composition)? Exact match queries or richer queries? Find any one matching instance or find all matching instances? 02-20-03 Which instance to choose what are the right metrics Lecture #12 2 Relevance Finding sensor data Alternatives to the XML hierarchical query processing in IrisNet Finding useful nodes Sensors Nodes for query processing Services’ web interface 02-20-03 Lecture #12 3 Naming and service discovery Wide-area naming DNS, Global Name Service, Grapevine Attribute-based systems X.500, Information Bus, Discover query routing Service location IETF SLP, Berkeley service discovery service Device discovery Jini, Universal plug-and-play Intentional Naming System (INS) Geographic Hash Tables 02-20-03 Lecture #12 4 Outline Overview INS GHT 02-20-03 Lecture #12 5 Name Types Flat names (single attribute-value pair) Basically E.g. Frank, Joe, Fred, etc… Good a single string that identifies all entities match with lookup techniques like DHTs Hierarchical names (single hierarchy of attribute-value pairs) Entities E.g. Frank.Smith.American, www.cmu.edu Good 02-20-03 classified by category, subcategory, etc. match with lookup techniques like DNS Lecture #12 6 Name Types Multiple hierarchical value-attribute pairs Type = printer memory = 32MB, lang = PCL Location = CMU building = WeH Hierarchy based on attributes or attributes-values? E.g. Country state or country=USA state=PA and country=Canada province=BC? Can be done in something like XML Good match with lookup techniques like INS Interface description Jini 02-20-03 Lecture #12 7 Query Types Exact match name=fred Ranges name > fred & name < george Simple wildcards Name & Age = 32 = * & Age = 32 Regular expression Name 02-20-03 = fr*d Lecture #12 8 Lookup Mechanisms (Multicast/Broadcast/Flooding) Services listen on well known discovery group address Client multicasts query to discovery group Services unicast replies to client IP Multicast Like a radio system Receivers subscribe to multicast groups (tune) Senders transmit with particular TTL to group (transmission power to particular frequency) Multicast not widely deployed due to scalability (and other) problems 02-20-03 Lecture #12 9 Lookup Mechanisms (Multicast/Broadcast/Flooding) Used by many systems – SLP, Jini, UPNP Tradeoffs very scalable effectively broadcast search Requires no dedicated infrastructure or bootstrap Easily adapts to availability/changes Can scope request by multicast scoping and by information in request Not 02-20-03 Lecture #12 10 Lookup Mechanisms (Directory Based/Centralized) Services register with central directory agent Soft state registrations must be refreshed or they expire Clients send query to central directory replies with list of matches Used by many systems – SLP, Jini, UPNP Typically central directory per domain 02-20-03 How do directories interact? often don’t Lecture #12 11 Lookup Mechanisms (Directory Based/Centralized) Tradeoffs How do you find the central directory service? Typically using multicast based discovery! SLP also allows directory to do periodic advertisements Need dedicated infrastructure Well suited for browsing and composition knows full list of services 02-20-03 Lecture #12 12 Lookup Mechanisms (Routing Based) Client issues query to overlay network Overlay network routes query to desired service[s] How is overlay structured/created Query can include both service description and actual request for service DNS administrative hierarchy DHT structured (circle, multi-dimensional torus, plaxton, etc.) INS self-organized network latency based Tradeoffs 02-20-03 Routing on complex parameters can be difficult/expensive Can work especially well in ad-hoc networks Can late-binding really be used in many applications? Lecture #12 13 Other Issues Security Don’t want others to serve/change queries Also, don’t want others to know about existence of services Srini’s home SLP server is advertising the $50,000 MP3 stereo system (come steal me!) Addressed in directory based systems through the use of capabilities certificates that grant access to particular service discovery records 02-20-03 Lecture #12 14 Outline Overview INS GHT 02-20-03 Lecture #12 15 Applications Location-dependent mobile applications Floorplan: An map-based navigation tool Camera: A mobile image/video service Load-balancing printer TV & jukebox service Sensor computing Network-independent “instant messaging” 02-20-03 Lecture #12 16 Environment Heterogeneous network with devices, sensors and computers Dynamism Example applications Mobility Performance variability Services “come and go” Services may be composed of groups of nodes Location-dependent mobile apps Network of mobile cameras Problem: resource discovery 02-20-03 Lecture #12 17 Design goals and principles Expressiveness Names are intentional; apps know what, not where Responsiveness Integrate name resolution and message routing (late binding) Robustness Decentralized, cooperating resolvers with soft-state protocol Easy configuration Name resolvers self-configure into overlay network 02-20-03 Lecture #12 18 Name-specifiers Expressive name language (like XML) Resolver architecture decoupled from language Providers announce descriptive names Clients make queries Attribute-value matches Wildcard matches Ranges [vspace = lcs.mit.edu/camera] [building = ne43 [room = 510]] [resolution=800x600]] [access = public] [vspace = mit.edu/thermometer] [building = ne43 [floor = 5 [room = *]] [temperature < 600F] data [status = ready] 02-20-03 Lecture #12 19 Name Lookups Lookup Tree-matching algorithm AND operations among orthogonal attributes 02-20-03 Lecture #12 20 Resolver Network Resolvers exchange routing information about names Uses triggered updates to rapidly adapt to changes Decentralized construction and maintenance Implemented as an “overlay” network over UDP tunnels Not every node needs to be a resolver Too many neighbors causes overload, but need a connected graph Overlay link metric should reflect performance 02-20-03 Lecture #12 21 INS Architecture: Message routing using intentional names Client Service Name resolver Name Overlay network of resolvers Name 02-20-03 Lecture #12 22 Robustness Decentralized name resolution and routing in “serverless” fashion Names are weakly consistent, like network-layer routes Routing protocol with periodic & triggered updates to exchange names Routing state is soft Expires if not updated Robust against service/client failure No need for explicit de-registration 02-20-03 Lecture #12 23 Routing Protocol Scalability Name-tree at resolver vspace=camera Routing updates for all names vspace=5th-floor Delegate this to another INR vspace = Set of names with common attributes Virtual-space partitioning: each resolver now handles subset of all vspaces 02-20-03 Lecture #12 24 Lookups Two styles of message delivery Anycast Multicast Two types of lookup Early binding Late binding 02-20-03 Lecture #12 25 Lookup Types If query only contains description, subsequent interactions must be outside overlay (earlybinding) Use IP address for subsequent messages If query includes request, client can send subsequent queries via overlay (late-binding) Subsequent requests may go to different services agents Enables easy fail-over/mobility of service 02-20-03 Lecture #12 26 Intentional Anycast lookup(name) yields all matches Resolver selects location based on advertised service-controlled metric E.g., server load Tunnels message to selected node Application-level vs. IP-level anycast Service-advertised metric is meaningful to the application 02-20-03 Lecture #12 27 ASIDE: Server Selection Service is replicated in many places in network How do direct clients to a particular server? As part of routing anycast, cluster load balancing As part of application HTTP redirect As part of naming DNS Which server? Lowest load to balance load on servers Best performance to improve client performance 02-20-03 Based on Geography? RTT? Throughput? Load? Any alive node to provide fault tolerance Lecture #12 28 ASIDE: Routing Based Server Selection Anycast Give service a single IP address Each node implementing service advertises route to address Packets get routed routed from client to “closest” service node Closest is defined by routing metrics May not mirror performance/application needs What 02-20-03 about the stability of routes? Lecture #12 29 Intentional Multicast Use intentional name as group handle Each resolver maintains list of neighbors for a name Data forwarded along a spanning tree of the overlay network Shared tree, rather than per-source trees Enables more than just receiver-initiated group communication 02-20-03 Lecture #12 30 INS Architecture: Message routing using intentional names Client Service Name resolver Name Late binding Name with message Intentional multicast Intentional anycast Overlay network of resolvers Name 02-20-03 Lecture #12 31 Discussion Distributed without relying on multicast Late-binding – how useful is this? Nice for fault recovery, but … Need stateless messaging and careful application design Soft-state critical to robustness of such designs Application level metrics for routing Handling dynamic attributes Difficult to scale with such attributes How to scale? 02-20-03 Lecture #12 32 Wide Area Scaling How do we scale INS to wide area? Hierarchy or DHTs? Hierarchy must be based on attribute of services All services must have this attribute All queries must include (implicitly or explicitly) this attribute Tradeoffs What attribute? Administrative (like DNS)? Geographic? Network Topologic? Should it have multiple hierarchies? Can support range queries nicely 02-20-03 Lecture #12 33 Wide Area Scaling INS over Chord TWINE DHTs what are the keys Must insert service at all possible lookup combinations One entry per each value-attribute pair for service What about popular pairs? e.g., country=USA Will overload nodes in DHT! Tradeoffs 02-20-03 Load-balancing and updates difficult Search styles limited to exact match Robust to failures Lecture #12 34 Outline Overview INS GHT 02-20-03 Lecture #12 35 Motivating Example Name-addressed, or data-centric, queries appropriate: Query(“whale”) {(whale, i, [u,v]), (whale, j, [x,y])} Expressiveness: single attribute name lookup 02-20-03 Lecture #12 36 Solution 1: Local Storage Broadcast query, collect results For n nodes, Q events, Dq detected & queried events Total msg-links = Q * n + Dq * sqrt(n) Hotspot (at access point) = Q + Dq 02-20-03 Lecture #12 37 Solution 2: External Storage Collect all events For n nodes, Q events, Dt total detected events 02-20-03 Total msg-links = Dt * sqrt(n) Hotspot (at access point) = Dt But Dt might be large Lecture #12 38 Solution 3: Data-Centric Storage (DCS) Rendezvous for queries & data For n nodes, Q events, Dt total detected events, Dq detected & queried events Total msg-links = Q * sqrt(n) + Dq * sqrt(n) + Dt * sqrt(n) Hotspot (at access point) = Q + Dq With summarization Total msg-links = Q * sqrt(n) + Q * sqrt(n) + Dt * sqrt(n) Hotspot (at access point) = 2Q 02-20-03 Lecture #12 39 Tradeoffs Local storage has greatest total message count as n grows External storage always sends fewer messages than DCS When many more event types detected than queried for, DCS has least hotspot message count DCS permits summarization of events (return multiple events in one packet) Need a simple way to implement DCS 02-20-03 Lecture #12 40 Geographic Hash Table Two operations supported: Put(k;v) stores v, the event, according to key k Get(k) retrieves the value associated with key k Hash a key k into geographic coordinates; store and retrieve events for that key at that location Spreads load evenly across key space! 02-20-03 Lecture #12 41 Geographic Routing (GPSR) Greedy geographic routing – with fixes for empty spaces Routes data to nodes surrounding geographic destination 02-20-03 Node closest stores data for that coordinate hash Replication on all nodes that enclose the coordinates to ensure persistence of data Lecture #12 42 Discussion 3 key objectives scale, adaptive to change and energy efficient Traffic concentration, data concentration and message counts are good metric for scaling and energy efficiency Data must be stored persistently and retrievable consistently from same place to allow adaptation to change 02-20-03 Lecture #12 43 Next Lecture Lecturer: Srini Topic: positioning Real world: GPS, 802.11 Radar, Bat, Cricket Network: IDMAPS, GNP, GeoPing Readings and questions posted on Saturday Announcements: exam: May 5th 8:30 – 11:30AM Mar 4th – project checkpoint Final 02-20-03 Lecture #12 44