Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
ICS 214B: Transaction Processing and Distributed Data Management Lecture 15: Data Replication with Network Partitions & Transaction Processing Systems Professor Chen Li 1 Network partitions • Groups of nodes may be isolated or nodes may be slow in responding P ICS214B Net Notes 15 P 2 Network Partitions • No replicated data – Quorum-based 3PC abort, commit quorums • Replicated data – at most one operational group coterie propagation of updates ICS214B Notes 15 3 . a Quorums . c b . . d C1 = {{a,b,c}, {a,b,d}, {a,c,d}, {b,c,d}} A1 = {{a,b}, {a,c}, {a,d}, {b,c}, {b,d}, {c,d}} Important property: XC1 YA1 XYØ YA1 XC1 XYØ ICS214B Notes 15 4 • Some quorums can be implemented with vote assignments 1 . a To commit 3 To abort 2 . c b . 1 . d 1 1 • Votes to commit + votes to abort > total votes • Why? ICS214B Notes 15 5 Not all quorums can be implemented via votes C2 = {{a,b}, {c,d}} A2 = {{a,c}, {a,d}, {b,c}, {b,d}} b a d c ICS214B Notes 15 6 Proof C2 = {{a,b}, {c,d}} A2 = {{a,c}, {a,d}, {b,c}, {b,d}} Suppose it can be implemented via votes Let each site symbol represent its vote Let x be the commit threshold Let y be the abort threshold Commit: Abort: a + b >= x a + c >= y y>c+d x>b+d c + d >= x a + d >= y y>a+b x>b+c b + c >= y x>a+d b + d >= y x>a+c 2y > 2(a+b+c+d) >= 2x ICS214B 4x > 4(a+b+c+d) >= 4y Notes 15 7 Another Proof C2 = {{a,b}, {c,d}} A2 = {{a,c}, {a,d}, {b,c}, {b,d}} Commit: a + b >= x c + d >= x 2 * () Abort: a + c >= y a + d >= y b + c >= y b + d >= y + 1 * () 4(a+b+c+d) >= 4x + 4y a+b+c+d >= x + y contradiction! ICS214B Notes 15 8 Partitions and data replication Options: (1) All copies required for updates (2) Group may update, but at most one (at a time) (3) Any group may update ICS214B Notes 15 9 Updates by at most one group . a . c b . Coterie . d C1 = {{a,b,c}, {a,b,d}, {a,c,d}, {b,c,d}} C2 = {{a,b}, {a,c}, {a,d}, {b,c,d}} X1= {{a,b}, {c,d}} not valid Important property: SC GC, SGØ ICS214B Notes 15 10 Reading replicated data . a . c b . . d C1 = {{a,b,c}, {a,b,d}, {a,c,d}, {b,c,d}} R1 = {{a,b}, {a,c}, {a,d},{b,c}, {b,d}, {c,d}} C2 = R2 = {{a,b}, {a,c}, {a,d}, {b,c,d}} ICS214B Notes 15 11 Reading replicated data - Votes C1 = {{a,b,c}, {a,b,d}, {a,c,d}, {b,c,d}} R1 = {{a,b}, {a,c}, {a,d},{b,c}, {b,d}, {c,d}} 1 to write get 3 votes (Vw) to read get 2 votes (VR) . a 1 . c b . 1 . d 1 ICS214B Notes 15 12 Reading replicated data - Votes C2 = R2 = {{a,b}, {a,c}, {a,d}, {b,c,d}} 2 to write get 3 votes (Vw) to read get 3 votes (VR) . a 1 . c b . 1 . d 1 ICS214B Notes 15 13 A Problem Example: a 3-node system, 1 vote each node, replicated data Now: T1 .a T1 is committed at a,b. But after T1 writes data at b before it writes data at a, the network gets repartitioned. T1 . b ICS214B .c Notes 15 14 Later: T1 .a T2 reads at c (not seeing T1); .b .c then writes and commits at a, c Then c doesn’t see T1! ICS214B Notes 15 15 Solution • Each node keeps list of committed transactions • Whenever a new quorum is formed – Compare sequence numbers to see which node has the latest updates – All node apply latest updates – Then resume processing ICS214B Notes 15 16 Network Partitions: Summary • No replicated data – abort, commit quorums • Replicated data – at most one operational group coterie propagation of updates ICS214B Notes 15 17 Next: Transaction Processing Systems ICS214B Notes 15 18 Basic concepts • Business Transactions – Real-world interaction • On-line Transactions – execution of program that performs some function(s) of business transaction – steps: get inputs do work (access DB’s) produce response ICS214B Notes 15 19 Example applications • ATMs – E.g., 1 trans per minute per ATM 60 ATMs 1 trans/sec • Stock exchange ICS214B Notes 15 20 servers clients Transaction Processing System (TPS) Presentation manager Presentation manager workflow control transaction programs resource managers ICS214B Notes 15 21 TP Monitor • The “operating system” for the TPS • Functions: – manage processes that run trans. programs – load balancing – commit distributed transactions – route requests from clients to servers – etc. ICS214B Notes 15 22 TP monitor architecture Presentation server requests Network manager queues Workflow controller Transaction server ICS214B …. Notes 15 net Transaction server 23 Two-tier Three-tier Client presentation database server workflow ICS214B transaction servers Notes 15 24 TP monitor = “glue” and “veneer” TP application programs TP monitor API veneer request routing user interface services ICS214B transactional communications operating system services communication services Notes 15 2 phase commit ... database system services 25 TP monitor functions: presentation communication clients-servers workflow management commit coordination process management security queue management ... • • • • • • • ICS214B Notes 15 26 Presentation services • • • • • user interaction constructing requests authentication communication logging ICS214B Notes 15 27 User interaction = Go Airlines = agent: Fred Smith task: do it ICS214B check-in reservation query Notes 15 28 Flight no: =Check In = Date today Passenger: Search for passenger Announcements: Storm heading our way!! ICS214B Notes 15 29 Old-fashioned approach for client Display menu box of screen Wait for user input ICS214B Code that does everything --------------------------------- Notes 15 send msg to server 30 Client using TP monitor services: ... DISPLAY_FORM (CHECK_IN); ... some logic [no date validation?] Check in schema: Flight: ___ Date: ___ Do_check_in schema: Flight: ___ Date: ___ ... SEND_REQ (DO_CHECK_IN); ... some logic (what next?)... ICS214B Notes 15 31 Data validation in forms • Best done using cached copy of valid values • OK for static values e.g.: product codes, flight numbers • Not OK for dynamic values e.g.: seats left in plane; account balance do this validation at server (can access DB) ICS214B Notes 15 32 Authentication: 2 levels user client server Do I know/trust user? ICS214B Notes 15 Do I know/trust client? 33 Process Management server db client ICS214B REQ Notes 15 ? code that does the work more code 34 OS Solution server Code for all REQ’s client1 db ... process 1 db ... ... Code for all REQ’s clientX process X Problem: too many processes….. does not scale! ICS214B Notes 15 35 Old fashioned TP solution server client1 db single process: ... - code for all REQ’s db - threading implementation ICS214B todo Notes 15 waiting ... clientX 36 Problem: – complexity – need all application code in one process ICS214B Notes 15 37 Server processes server db client1 ... ... clientX workflow/ routing code for req#1 code for req#2 ICS214B Notes 15 db db 38 client1 routing Can be generalized: db code for req#1 ICS214B routing ... ... clientX code for req#2 Notes 15 db db 39 Routing REQ ? • Consider factors such as load balancing, failures… ICS214B Notes 15 40 NEXT: Inter-process communication router ICS214B do work server process Notes 15 do work another process (eg DBMS) 41 Transactional RPC • Transaction info (e.g., TRANS ID) is passed with calls • Execution of server code becomes part of transaction ... ... ... ... client: start-trans server T35 ( T35 ) commit ICS214B work ... call lock data for T35 DB involve all resources for T35 Notes 15 42 Example: credit_card ... pay_cc work 333 ... ... T88 ... ... ... ... ... ... T88 start-trans T88 lock client: pay_cc commit debit_bank ICS214B T88 ... work commit changes to credit_card and accounts accounts 777 ... ... ... Notes 15 43 lock T88 ... debit_bank Fault tolerance in RPC client: server: ____ ____ ____ ____ ____ ____ call timeout: ?? work NO REPLY Timeout: retry? give up? ICS214B Notes 15 44 Motivation for queues (a) server unavailable ? client server (b) server loses request req client ICS214B server Notes 15 45 Motivation for queues (c) client down req client server ? ans (d) queues also useful for - load balancing - request priorities ICS214B Notes 15 46 client enqueue req req req req server dequeue queue other clients ICS214B other servers (load balancing) Notes 15 47 client Trans#1: start_trans: get inputs construct req enqueue (Q1) commit Trans#3: start_trans: dequeue (Q2) decode reply process output commit ICS214B server Q1: request queue Q2: reply queue Notes 15 Trans#2: start_trans: dequeue (Q1); process req; enqueue (Q2); commit 48 TP monitors and the web web http browser web server daemon TP monitor client TP monitor protocol TP system ICS214B Notes 15 49 Summary of TPS • Architecture – Presentation services – Routing and workflow control – Process management • Communication services – RPC – Queues ICS214B Notes 15 50