Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Introduction to the Internet • Internet service concepts ... • • • • • • • • • Introduction to information services Finger-, Inetd-, and Telnet-based services FTP WAIS Gopher World-Wide Web Email services Security Future topics and research opportunities I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 1 Internet service concepts … • • • • What is the Internet? ... Internet services structure ... TCP/IP ... Info-clients info-servers comm. I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 2 What is the Internet? … • Federation of computer networks ... • Three roles ... • What does it mean to be on the Internet? ... • What is on the Internet? ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 3 Federation of computer networks ... • Speak same protocols • Computer network protocols • TCP/IP • Connected with high-speed telephone circuits I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 4 Three roles ... • Information providers • Users (customers) • Connection providers I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 5 What does it mean to be on the Internet? ... • “Really” on the Internet • On-demand; no Internet protocols • Hybrid I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 6 Full-time IP connection ... • IP protocol TCP/IP over data phone line (leased) LAN Internet Connection Provider Router Modem I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 7 Dial-up access • terminal emulation Characters over voice phone line Terminal Internet Modem Modem (or ISDN interface) I VPR (or ISDN interface) Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz Computer 8 Hybrid ... • SLIP: Serial Line Internet Protocol • PPP: Point-to-Point Protocol • ISDN: Integrated Services Digital Network ... • Running Internet protocols over dialup ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 9 ISDN: Integrated Services Digital Network ... • Voice, data, both • On-demand, full-time • Speed comparable to leased-line connection I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 10 Running Internet protocols over dial-up ... • Dial-up hybrid • Dial-up hybrid TCP/IP over voice phone line Terminal Modem (or ISDN interface) I VPR Modem Internet Connection Provider (or ISDN interface) Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 11 What is on the Internet? ... • Computers: all kinds • People: large number, variety • Places I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 12 Internet services structure … • Client/server architecture ... • Information service client/server ... • Any computer can be a service provider • Compare: commercial networks ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 13 Client/server architecture ... • Client: interact with user • Server: perform tasks as directed by user • Same/different computers I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 14 Information service client/server … • Client ... • Server ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 15 Client ... • Accept queries/other input from users • Send queries/data to server • Manage display of results I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 16 Server ... • Receive queries/other data from clients • Process • Send results/status I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 17 Compare: commercial networks ... • E.g., CompuServe • All services from computers run by network • No information by users (except email) I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 18 TCP/IP ... • Packets: data; acknowledge data received • Layered protocols ... • inetd superserver ... • Stand-alone servers ... • Naming ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 19 Layered protocols … • IP ... • TCP (connectio n-oriented) ... • UDP (connectio nless) ... I VPR Application Application TCP UDP IP Device Driver Application Device Driver Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz ICMP Device Driver 20 IP ... • • • • • Internet Protocol IP packet Addresses: 32-bit numbers ... No guaranteed delivery No guaranteed serialization I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 21 Addresses: 32-bit numbers ... • 0 to ~4.3 billion • Dotted-octet notation ... • Net, subnet hierarchy I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 22 Dotted-octet notation ... • 4 sets of 8 bits • Convert to decimal • Concatenate, separated by dots I VPR Binary 00001111000100011110100000011000 Dotted Octet 15 Net Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz . 17 . Subnet 232 . 24 Computer 23 TCP (connection-oriented) ... • Transmission Control Protocol • Two services IP doesn't • Guaranteed delivery • Serialization of data • How? ... • ==> Mimic stream over dedicated line • Also, port numbers ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 24 How? ... • Sequence numbers for IP packets • Check arriving packets for completeness, order • Get missing packets resent I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 25 Also, port numbers ... • • • • Layer below IP address (computer) Identify services within computer 16-bit “Well known,” e.g., ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 26 “Well known,” e.g., ... • 25: Simple Mail Transfer Protocol (SMTP) mailer • 23: Telnet • “Privileged” ports (UNIX) • Numbers < 1024 • Only root can start • Security • Prevent “listening” I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 27 UDP (connectionless) ... • User Datagram Protocol • Two additional features to IP • Port #s (16-bit, not same as TCP) • Checksumming (optional) • Detect modifications • Avoid TCP for 1-packet q's / responses I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 28 inetd superserver ... • Many servers, idle periods ==> waste • inetd: Internet Superserver • Listen to ports • Start requested server • Server talks directly to client • inetd.conf ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 29 inetd.conf ... • Determine servers at start-up • E.g.,: ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 30 E.g., ... telnet stream tcp nowait root /etc/telnetd telnetd New Pathname Service Connection server User (alt.: dgram for ea ID to client run UDP) (alt.: server start single server and wait) I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz Argument vector: commandline options to pass 31 To find ... • % grep telnet /etc/services • telnet 23/tcp • Connection request ==> transaction in syslog Jul 25 22:53:33 ohbear inetd[349] : telnet/tcpL Connection from babs.nsr.hp.com (15.17.232.24) at Mon Jul 25 22:53:33 1994 I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 32 Stand-alone servers ... • inetd restarts servers whenever needed • Auto handle of config change • Inefficient for busy service • Takes time for inetd to start server • ==> Run server stand-alone I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 33 Daemon ... • Listen to assigned port • Spawn copy of self to handle ea. request • + Fast • – Crash unnoticed; explicit reread of config files when changed I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 34 Naming ... • Domain names ... • Aliases ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 35 Domain names ... • • • • • cs.uml.edu (from specific to general) .edu, .com, .gov, .org, .mil 2-letter country codes (ISO 3166) Servers: Domain Name Server (DNS) Clients: resolvers • Send queries to DNS for IP addr. <--> name I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 36 Aliases ... • Pointers to other domain names • Easier to remember • E.g., ftp.uml.edu, www.cs.uml.edu I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 37 Info-clients info-servers comm. • Gopher ... • Web ... • FTP ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 38 Gopher ... • User request resource on Gopher server • Client query ... • Gopher server response ... • Client stores file • TCP connection torn down • TCP up and down for each request I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 39 Client query ... • Accept input & translate into query • Send query to Gopher server • Look in /etc/services to find port number • gopher 70/tcp • Query --> TCP packet(s) --> server (port 70) I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 40 Gopher server response ... • Listen on port 70 / inetd listen • inetd check destination • /etc/services to find port number • Map to service name • Gopher • Start server, gopherd • Gopher interpret query • Open, read file, send to client I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 41 Web ... • Similar to Gopher • Ea. doc. new TCP connection • Ea. inline graphics ==> separate TCP connection • With approp. server where graphics is I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 42 FTP ... • Different from Gopher, Web • 2 connections server <--> client • Control: up through time user logged in • Data: up when needed I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 43 Introduction to information services • Overview of Internet information services ... • What to do with Internet Services, e.g., ... • System and network requirements ... • Human requirements ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 44 Overview of Internet information services … • • • • • Mail-based ... FTP: file transfer protocol ... Telnet & finger ... Gopher ... WAIS: Wide Area Information Servers ... • WWW: The World-Wide Web ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 45 Mail-based ... • Reach largest population • Get quickest response • Grab attention • Compare: Gopher, WWW • User has to check • Two types ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 46 Two types … • Mailing lists ... • Archive servers ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 47 Mailing lists ... • List of e-mail addresses • Reach all with one address • E-mail list server programs • Majordomo • Listserv • listproc, e.g., ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 48 listproc, e.g., ... • To: [email protected] • -------• subscribe ora-news John Q. User of Some Organization I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 49 Archive servers ... • • • • • • • Send out files by e-mail Response to users' e-mail requests Also on-the-fly First index, then file name Archive: encodes, splits to pieces, sends User: reassembles, decodes ftpmail: xfer files from other hosts & deliver by e-mail I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 50 FTP: file transfer protocol ... • Restricted login • Limited set of commands (all) • Limited set of files • Anonymous ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 51 Anonymous ... • Public access • User: anonymous • Password: user's e-mail address (convention) • Supports almost all platforms, all file formats • Can define privileged access, limit # of users • – Slow to browse, no file description • Read README while logged in: get README - I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 52 Telnet & finger ... • On most OSs • Fairly primitive • Telnet: login over net • E.g., DUATS: weather briefing for pilots • duats.gtefsd.com (student), duat.gtefsd.com • Finger: info on users of remote system • Also other short info, e.g., stock quotes, weather report I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 53 Gopher ... • Text-labeled menus • Can be cascaded • Advantages ... Internet Gopher Information Client v2.0.16 • Limitations ... Home Gopher server: wildlife.ora.com --> 1. Introduction and Cover 2. Foreword 3. Country Accounts/ 4. Search Country Data <?> I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 54 Advantages ... • Easy navigation • Simple, powerful, easy to administer • Organize directory where server runs • Subdirectories become menu items • Files in directories become available • Text oriented ==> good for slow links, dial-ups I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 55 Limitations ... • Text only display • Can xfer binary • Display with other software • Link only from menu • Compare WWW: from within document • Links to FTP, WAIS, Telnet • But not same variety as WWW I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 56 WAIS: Wide Area Information Servers ... • Search data ... • Minimal support for browsing • Gopher, WWW assume user knows where to look • WAIS doesn't • ==> Good complement to WWW, Gopher • Requires graphical browser, IP connectivity ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 57 Search data ... • • • • • Start at Directory of Servers Index of all servers ... Real search in most approp. servers Search documents for keywords Similar (but not sufficient) documents: add entire document to keyword list • Relevance feedback: match & similar documents I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 58 Index of all servers ... • Indexing beforehand ==> quick search • Support many data types • ASCII, LaTeX, PostScript, GIF, TIFF, mail digests, NetNews archives • Easy to add data types I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 59 Requires graphical browser, IP connectivity ... • Text-based, pretty difficult • Most WAIS clients provide viewers for basic formats • Allow specify external programs to process & present more types • Works best with formats w / lots of words • Big database indexes ==> need lots of disk space I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 60 WWW: The World-Wide Web ... • Most powerful, many features of its own • Integrates most other info services • ==> Simple to use & understand • Web browser: GUI-based access to • FTP, WAIS, Gopher, etc. • Format: HyperText Markup Language (HTML) ... • Protocol: HyperText Transfer Protocol (HTTP) ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 61 Format: HyperText Markup Language (HTML) •... Commands to format document, specify hyperlinks • Browser formats, marks hyperlinks • Follow links, format according to type • Fire off other programs (e.g., xv to view images) • Must have IP connectivity I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 62 Protocol: HyperText Transfer Protocol (HTTP) ... • http://<server>/<other> • ftp://, gopher:// • Local: file:// I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 63 What to do with Internet Services, e.g., ... • Internal ... • External ... • Commercial transactions; charging money ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 64 Internal ... • • • • • • • Corporate info Forms, logos, style sheets Timecards Home-grown software archives Employee directory Workgroup mailing lists “Intranet” I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 65 External ... • • • • • • • Electronic magazine / newsletter Manual pages (e.g., BSDI) Product, support, price info Press releases Software product distribution Bug reports Contributed public domain software archives I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 66 Commercial transactions; charging money ... • Billing • Authentication and secure transactions • Unauthorized “sharing” • “Eavesdropping” • Free to use but not to resell • Future versions at a price I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 67 System and network requirements ... • • • • Network connection type ... Network load ... Computational load ... Disk space ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 68 Network connection type ... • Full-time IP connection • Limited Gopher, Telnet, Finger, WAIS • Minimum 28.8 Kbaud PPP connection • Standard modem, phone line • Most services • Minimum 56 Kbaud • Serious service • ISDN or T1 I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 69 Network load ... • Queries (usually small) ... • Responses (bulk of traffic) ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 70 Queries (usually small) ... • • • • • Mailing lists: short modification messages Mail archive: request index / file FTP: download short; upload can be long Gopher: short menu item / document name WAIS: long if doc’s added for relevance feedback, but use IDs • WWW: short strings identifying doc’s I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 71 Responses (bulk of traffic) ... • Estimate size of avg. reply • Size of files; likelihood of retrieval • If know frequency of files / menu items ==> better prediction • For WWW: inline images increase avg. size • Network tracing, monitoring • Monitoring devices • Usage logs: well-known ports ... • Mailing-lists, mail archives ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 72 Usage logs: well-known ports ... • • • • • FTP (data connection): 20 FTP (control connection): 21 Gopher: 70 Z39.50 (WAIS protocol): 210 HTTP (WWW protocol): 80 I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 73 Mailing-lists, mail archives ... • Hard to predict size • No dedicated out port: unreserved, > 1023 • Hard to monitor • Hard to separate • In port 25, but can't separate types • Best way: syslog file • Mailing-lists ... • Archives ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 74 Mailing-lists ... • Very little reply traffic • Other traffic can be substantial I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 75 Archives ... • Huge amounts of reply • No effective limit • No good way to predict avg. response size I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 76 Computational load ... • More guessing game • Monitor trend in peak load average • As service grows in popularity • Predict need for faster machine early • Mailing-list ... • Mail-based archive ... • FTP, Gopher, WWW ... • WAIS ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 77 Mailing-list ... • Servers don’t require much CPU • Simple op’s on aliases file • Not much network traffic • Lists can be CPU intensive, depending on size of list and message • sendmail: no efficient sending of thousands of messages at same time I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 78 Mail-based archive ... • Lots of CPU time in • File transfers • Processing of files transferred • Mailing those files I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 79 FTP, Gopher, WWW ... • CPU consumption proportional to • Number of queries received • Size of files processed • FTP bogging down processors • Wash U. FTP: special features to limit access, set of number users • WWW demanding because of inline images I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 80 WAIS ... • Most demanding • Search • Open index files • Compute scores (match search criteria) • Other non-trivial tasks • ==> Per-query load high • No way to control access based on sys load • Need fast processor to save large WAIS db I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 81 Disk space ... • • • • • • At least enough for all files offered Extra overhead WAIS ... Mail server ... Other services requirements insignificant But if multiple, might need as many copies of data as number of services • FTP copy, Gopher copy, WWW copy, etc. I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 82 WAIS ... • Index files as large as data files • ==> Consume 2 x data I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 83 Mail server ... • Header + bodies in mail queue directory • ==> Lots of space • More significant with mail-based archive servers • Large mail messages • E.g., uuencoded split binaries I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 84 Human requirements ... • Understand material offered • How to structure intuitively and effectively • Constantly update, add new, check integrity of links • “System guru” not necessarily same as “contents guru” (librarian) • Librarian’s duties ... • Librarian’s desirable qualifications ... I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 85 Librarian’s duties ... • Author & format info • Meta-info: FTP README, Gopher menus, ... • Collect & distribute info from other sources • Keep up-to-date • Provide & maintain links to other info resources • Respond to user requests for help I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 86 Librarian’s desirable qualifications ... • User-interface experience • Familiarity with subject & audience • Ability to navigate the Internet • Find new info resources to point to • Check integrity of links • Good people skills for customer service I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 87 Finger-, Inetd-, and Telnetbased services I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 88 FTP • Setting up an FTP archive • The WU archive FTP daemon • Maintaining and FTP archive I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 89 WAIS • Creating and Internet database server with WAIS • Creating WAIS sources with waisindex I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 90 Gopher • • • • • • • • Introduction Compiling the server Managing the server Preparing Information Linking services together Incorporating databases Veronica and jughead Gopher+ forms and other new features I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 91 World-Wide Web • • • • • Introduction to the World-Wide Web Setting up a Web server Authoring for the Web Gateways and forms Access control and security I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 92 Email services • Introduction • Simple mailing lists • Automating mailing list with majordomo • The majordomo list owner and moderator I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 93 Security • • • • Firewalls and information services xinetd Legal issues Protecting intellectual property I VPR Institute for Visualization and Perception Research © Copyright 1998 Haim Levkowitz 94 Future topics and research opportunities • New & improved • Authentication, verification, and other security schemes • Information encryption/decryption • Secure fund transfers • Etc. • Contents quality visualization and assessment • New and improved information search and exploration schemes • Image and other high bandwidth transfer • New interaction paradigms • Automatic contents control • E.g., parents' control of children's access 95 Visualization and Perception Research simplistic ``hit'' counts). IV R Institute • PWeb site for traffic assessment (beyond • Other topics © Copyright 1998 Haim Levkowitz