Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
VoD BitTorrent Framework Background • BT is a very popular peer to peer protocol with many implementations: – http://en.wikipedia.org/wiki/Comparison_of_Bit Torrent_clients • The first one was created in Python by Bram Cohen. Real world motivation • BitTorrent is common and Free • Most chances there is a swarm for your movie • VoD servers are expensive and complicated • Its better to use P2P for VoD Some definitions • StratUp Delay (D) = How much time I am allowed to hold before starting to play M (Movie). • Prefetch Time (P) = How much time before I watch byte B from M, it should arrive. • Playback Rate (R) = How many bytes/second I watch • B(t) – the index of the bit that must arrive by t. • B(t) = R *(P+t-D) Rules and Examples • B(t) = R *(P+t-D) • If R=5 KB/sec and D=6 seconds and P=2 seconds, than: – At t=6 seconds, I must have 10KB continuous from the start of the movie. – At t=18 seconds I must have 70KB continuous from the start of the movie. What does my server suppose to do • Your server suppose to get from the Bittorent clients pieces of data needed to play the video. • Each piece needs to arrive by its deadline. • If the piece does not arrive by the deadline you “can download it freely” from the server. – In this case you encounter a penalty • The number of penalty points you accumulated whole getting/playing the video is your (negative) score. How do I test my server/idea • Create my own VoD BitTorrent client. • Simulate VoD server – implement a function that copies from a seed directly to my file (Checkpoint 1). • Count the bytes your client received from the VoD Server (Checkpoint 1) • Aim at minimizing the part of the movie downloaded from VoD Server. Execution example Tracker BT BT BT BT BT run-all.sh BT BT VOD BT BT BT BT How does it work? • Run tracker • Run the “world” • Run your client (if given) How does it work? • Both tracker and the “world” run on the same computer • we have – pc-hanoch-w3 – Please reserve time in: • http://bt-p2p-vod.wikispaces.com/ • Or from course www - "Scheduling computers for tests." The main script - run_all.sh • Location: – /users/courses/hanochworkshop/bittorrent • run_all_vod.sh: – Parameters: • Start / stop • My client – optional run_all_vod.sh operation • Start: – Create download directory – Run the “world” • Each world client starts with different “saved” amount. – Run “my client” – if given – All upload rates are restricted (simulate practical world). • Stop – Kill all python threads To run client separately: • BitTornado-My-Client/btdownloadheadless.py – --saveas /vol/scratch/bt-download/picture.tst.bmp – --max_upload_rate 70 – --security 0 – ./picture.pc-hanoch-w3.bmp.torrent (Implement in Checkpoint 1): – -- Rate 60 – -- Delay 6 – -- Prefetch 2 • Not necessarily on the same machine as the world. What implementation we use • The clients are a modified version based on an up to date version of Bram Python Client. • Modifications: – “World” clients • do not print as it would only make a mess • reincarnate after the finish to make a real world feel. – “My Client” template – kills all clients when finished Python • Very common script language • Intuitive and readable • References: – python.org :http://docs.python.org/reference/ – http://wiki.python.org/moin/BeginnersGuide/NonProgrammers – http://wiki.python.org/moin/BeginnersGuide/Programmers • The book "Think like a Computer Scientist in Python" has a very good reputation: My Client • We supply a default “My Client” In: – BitTornado-My-Client/btdownloadheadless.py and “BitTornado - Your Code” link. • It is like the “standard” one. Getting started • Run the environment – ssh pc-hanoch-w3 or pc-hanoch-w4 – cd /users/courses/hanochworkshop/bittorrent – run_all_vod.sh • start • [BitTornado-My-Client/btdownloadheadless.py] – Note tracker prints a message for each GET. • You can connect to the tracker, from the CS network, with a browser at: – http://pc-hanoch-wk:6970/ • Do ls -l /home/pc-hanoch-w4/home/hillelav/$USER to see downloaded files What is it doing? • Run bttrack. • Copy From FreshFiles a set of “downloaded” files to download directory. • Run X seeds • Run a python client per “downloaded” file. • Run my client if given Directory Structure • BitTornado-My-Client – – high level application • BitTornado-My-Client/BitTornado – – services • BitTornado-My-Client/BitTornado/BT1 – – Actual work Client Operation • btdownloadheadless.py – First to run. – Performs initializations. • • • • get_response – parse the .torrent Infohash – the ID of the .torrent startEngine – Start connection with peers startRerequester - Start connection with tracker – Run rawserver.listen_forever from RawServer.py. – listen_forever is: • Polling port • executing func() from a task queue. Receiving messages • From listen_forever loop it goes to: – BT1/Connecter.py/got_message • • • • • • • • • CHOKE UNCHOKE INTERESTED NOT_INTERESTED BITFIELD REQUEST CANCEL PIECE HAVE Influencing decisions • Can be done in in one of the callbacks which spring from Connecter.py / got_message: • Downloader.py – HAVE->got_have->send_interested • Tune for VoD? BT tracker • Send GET request in Rerequester.py / announce • Receive GET data in thread in Rerequester.py / _rerequest Influencing decisions • In Encrypter.py / start_connection or _start_connection_from_queue For instance try different max_connections. _rerequest_single – check timout for scheduling. Policy • Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to. • Downloader.py / _request_more. • PiecePicker.py / next - rarest piece, etc. Tune for VoD? • got_unchoke->_request_more->next… More about the code • Start your project from “BitTornado - Your Code” link. • Click “BitTornado - Code Guide” for basic explanations about the code.