Download BitTorrent - VoD Framework

VoD BitTorrent Framework
Background
• BT is a very popular peer to peer protocol
with many implementations:
– http://en.wikipedia.org/wiki/Comparison_of_Bit
Torrent_clients
• The first one was created in Python by
Bram Cohen.
Real world motivation
• BitTorrent is common and Free
• Most chances there is a swarm for your
movie
• VoD servers are expensive and
complicated
• Its better to use P2P for VoD
Some definitions
• StratUp Delay (D) = How much time I am
allowed to hold before starting to play M (Movie).
• Prefetch Time (P) = How much time before I
watch byte B from M, it should arrive.
• Playback Rate (R) = How many bytes/second I
watch
• B(t) – the index of the bit that must arrive by t.
• B(t) = R *(P+t-D)
Rules and Examples
• B(t) = R *(P+t-D)
• If R=5 KB/sec and D=6 seconds and P=2
seconds, than:
– At t=6 seconds, I must have 10KB continuous from
the start of the movie.
– At t=18 seconds I must have 70KB continuous from
the start of the movie.
What does my server suppose
to do
• Your server suppose to get from the Bittorent
clients pieces of data needed to play the video.
• Each piece needs to arrive by its deadline.
• If the piece does not arrive by the deadline you
“can download it freely” from the server.
– In this case you encounter a penalty
• The number of penalty points you accumulated
whole getting/playing the video is your
(negative) score.
How do I test my server/idea
• Create my own VoD BitTorrent client.
• Simulate VoD server – implement a function that
copies from a seed directly to my file
(Checkpoint 1).
• Count the bytes your client received from the
VoD Server (Checkpoint 1)
• Aim at minimizing the part of the movie
downloaded from VoD Server.
Execution example
Tracker
BT
BT
BT
BT
BT
run-all.sh
BT
BT
VOD
BT
BT
BT
BT
How does it work?
• Run tracker
• Run the “world”
• Run your client (if given)
How does it work?
• Both tracker and the “world” run on the
same computer
• we have
– pc-hanoch-w3
– Please reserve time in:
• http://bt-p2p-vod.wikispaces.com/
• Or from course www - "Scheduling computers for
tests."
The main script - run_all.sh
• Location:
– /users/courses/hanochworkshop/bittorrent
• run_all_vod.sh:
– Parameters:
• Start / stop
• My client – optional
run_all_vod.sh operation
• Start:
– Create download directory
– Run the “world”
• Each world client starts with different “saved”
amount.
– Run “my client” – if given
– All upload rates are restricted (simulate
practical world).
• Stop
– Kill all python threads
To run client separately:
• BitTornado-My-Client/btdownloadheadless.py
– --saveas /vol/scratch/bt-download/picture.tst.bmp
– --max_upload_rate 70
– --security 0
– ./picture.pc-hanoch-w3.bmp.torrent
(Implement in Checkpoint 1):
– -- Rate 60
– -- Delay 6
– -- Prefetch 2
• Not necessarily on the same machine as the world.
What implementation we use
• The clients are a modified version based on an
up to date version of Bram Python Client.
• Modifications:
– “World” clients
• do not print as it would only make a mess
• reincarnate after the finish to make a real world feel.
– “My Client” template – kills all clients when finished
Python
• Very common script language
• Intuitive and readable
• References:
– python.org :http://docs.python.org/reference/
– http://wiki.python.org/moin/BeginnersGuide/NonProgrammers
– http://wiki.python.org/moin/BeginnersGuide/Programmers
• The book "Think like a Computer Scientist in Python" has a very
good reputation:
My Client
• We supply a default “My Client” In:
– BitTornado-My-Client/btdownloadheadless.py and
“BitTornado - Your Code” link.
• It is like the “standard” one.
Getting started
• Run the environment
– ssh pc-hanoch-w3 or pc-hanoch-w4
– cd /users/courses/hanochworkshop/bittorrent
– run_all_vod.sh
• start
• [BitTornado-My-Client/btdownloadheadless.py]
– Note tracker prints a message for each GET.
• You can connect to the tracker, from the CS network,
with a browser at:
– http://pc-hanoch-wk:6970/
• Do ls -l /home/pc-hanoch-w4/home/hillelav/$USER to
see downloaded files
What is it doing?
• Run bttrack.
• Copy From FreshFiles a set of
“downloaded” files to download directory.
• Run X seeds
• Run a python client per “downloaded” file.
• Run my client if given
Directory Structure
• BitTornado-My-Client –
– high level application
• BitTornado-My-Client/BitTornado –
– services
• BitTornado-My-Client/BitTornado/BT1 –
– Actual work
Client Operation
• btdownloadheadless.py – First to run.
– Performs initializations.
•
•
•
•
get_response – parse the .torrent
Infohash – the ID of the .torrent
startEngine – Start connection with peers
startRerequester - Start connection with tracker
– Run rawserver.listen_forever from RawServer.py.
– listen_forever is:
• Polling port
• executing func() from a task queue.
Receiving messages
• From listen_forever loop it goes to:
– BT1/Connecter.py/got_message
•
•
•
•
•
•
•
•
•
CHOKE
UNCHOKE
INTERESTED
NOT_INTERESTED
BITFIELD
REQUEST
CANCEL
PIECE
HAVE
Influencing decisions
• Can be done in in one of the callbacks
which spring from Connecter.py /
got_message:
• Downloader.py
– HAVE->got_have->send_interested
• Tune for VoD?
BT tracker
• Send GET request in Rerequester.py /
announce
• Receive GET data in thread in
Rerequester.py / _rerequest
Influencing decisions
• In Encrypter.py / start_connection or
_start_connection_from_queue
For instance try different max_connections.
_rerequest_single – check timout for
scheduling.
Policy
• Choker.py/_round_robin – decide who to
choke / optimistically un choke /
send_have to.
• Downloader.py / _request_more.
• PiecePicker.py / next - rarest piece, etc.
Tune for VoD?
• got_unchoke->_request_more->next…
More about the code
• Start your project from “BitTornado - Your
Code” link.
• Click “BitTornado - Code Guide” for basic
explanations about the code.