Download Computing With Tensors - UTEP Computer Science

Why Tensors
19 Century Physics
Problem and How . . .
From Tensors in . . .
Computing With Tensors:
Potential Applications
of Physics-Motivated
Mathematics
to Computer Science
Martine Ceberio and Vladik Kreinovich
Department of Computer Science
University of Texas at El Paso
El Paso, TX 79968, USA
emails [email protected]
[email protected]
Modern Algorithm for . . .
Modern Algorithm for . . .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 1 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
1.
Why Tensors
• Modern computing – main problems include:
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
– large amounts of data;
Modern Algorithm for . . .
– long time required to process this data.
Quantum Computing . . .
• Similar situation – 19 century physics:
New Idea: Tensors to . . .
Computing with . . .
– large amounts of data;
Remaining Open Problem
– long time required to process this data.
Acknowledgments
• How the problem was solved then: by using tensors
• Natural idea: let us use tensors to solve the problems
with modern computing.
Title Page
JJ
II
J
I
Page 2 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
2.
19 Century Physics
• Physics starts with measuring and describing the values of different physical quantities.
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
• It goes on to equations which enable us to predict the
values of these quantities.
• A measuring instrument usually returns a single numerical value.
• For some physical quantities (like mass m), the single
measured value is sufficient to describe the quantity.
• For other quantities, we need several values.
• Example: three components Ex , Ey , and Ez describe
the electric field.
• Example: to describe the tension inside a solid body,
we need values σij .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 3 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
3.
Problem and How Tensors Helped
• 19 century: a separate equation for each component of
the field.
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
• Result: equations cumbersome and difficult to solve.
• Idea: to describe all the components of a physical field
as a single mathematical object:
– a vector ai ,
– or, more generally, a tensor aij , aijk , . . .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
• Result: simplified equations, faster computations.
JJ
II
• Originally: mostly vectors (rank-1 tensors) were used.
J
I
• 20 century:
Page 4 of 12
– matrices (rank-2 tensors) in quantum physics,
– higher-order tensors such as the rank-4 curvature
tensor Rijkl in relativity theory.
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
4.
From Tensors in Physics to Computing with Tensors
• Reminder:
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
– 19 century physics encountered a problem of too
much data;
New Idea: Tensors to . . .
– tensors helped.
Computing with . . .
• Modern computing: suffers from a similar problem.
Quantum Computing . . .
Remaining Open Problem
Acknowledgments
• Natural idea: tensors can help.
• Two examples justifying our optimism:
– modern algorithms for fast multiplication of large
matrices;
– quantum computing.
• Comment: detailed descriptions of these examples follow.
Title Page
JJ
II
J
I
Page 5 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
5.
Modern Algorithm for Multiplying Large Matrices
• Definition:


b11
a11 . . . a1n
 ... ... ... ...
bn1
an1 . . . ann
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
 
c11
. . . b1n
... ...  = ...
cn1
. . . bnn

. . . c1n
... ... ;
. . . cnn
cij = ai1 · b1j + . . . + aik · bkj + . . . + ain · bnj .
• Problem: for large n, no space for both A and B in the
fast (cache) memory.
• Result: lots of time-consuming data transfers (“cache
misses”) between different parts of the memory.
• Solution: represent each matrix as a matrix of blocks:


A11 . . . A1m
A =  ... ... ... ,
Am1 . . . Amm
Cαβ = Aα1 · B1β + . . . + Aαγ · Bγβ + . . . + Aαm · Bmβ .
Modern Algorithm for . . .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 6 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
6.
Modern Algorithm for Multiplying Large Matrices:
Tensor Interpretation
• Main idea:
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
– we start with a large matrix A of elements aij ;
– we represent it as a matrix consisting of block submatrices Aαβ .
• Tensor interpretation:
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
– each element of the original matrix is now represented as
– an (x, y)-th element of a block Aαβ ,
– i.e., as an element of a rank-4 tensor (Aαβ )xy .
Title Page
JJ
II
J
I
Page 7 of 12
• Fact: an increase in rank improves efficiency.
Go Back
• Analogy: a representation of a rank-1 vector as a rank2 spinor works in relativistic quantum physics.
Full Screen
Close
Quit
Why Tensors
19 Century Physics
7.
Quantum Computing as Computing with Tensors
• Classical bit: a system with two states 0 and 1.
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
• Quantum bit (qubit): superposition principle – we can
have states c0 · |0i + c1 · |1i.
• Probabilities: Prob(0) = |c0 |2 and Prob(1) = |c1 |2 ,
hence |c0 |2 + |c1 |2 = 1.
• n-(qu)bit system: a general state is
Modern Algorithm for . . .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
c0...00 · |0 . . . 00i + c0...01 · |0 . . . 01i + . . . + c1...11 · |1 . . . 11i.
Title Page
• Conclusion: each state is a tensor ci1 ...in of rank n.
JJ
II
• Advantage: store the entire tensor in only n (qu)bits.
J
I
• Resulting efficiency of quantum computing:
Page 8 of 12
√
– search in an unsorted array of size n in n time
(Grover);
– factoring large integers in polynomial time (Shor).
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
8.
New Idea: Tensors to Describe Constraints
• A general constraint between n real-valued quantities
is a subset S ⊆ Rn .
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
• A natural idea: represent this subset block-by-block –
by enumerating sub-blocks that contain elements of S.
• Fact: each block bi1 . . . in can be described by n indices
i1 , . . . , i n .
• Result: we can describe a constraint by a booleanvalued tensor ti1 ...in for which:
• ti1 ...in =“true” if bi1 ...,in ∩ S 6= ∅; and
• ti1 ...in =“false” if bi1 ...,in ∩ S = ∅.
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 9 of 12
• Fact: processing such constraint-related sets can also
be naturally described in tensor terms.
• Fact: this speeds up computations.
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
9.
Computing with Tensors Can Also Help Physics
• So far: we have shown that tensors can help computing.
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
• New idea: relation between tensors and computing can
also help physics.
• Example: Kaluza-Klein-type high-dimensional spacetime models of modern physics.
• Einstein’s idea: use “tensors” with integer or circular
values.
• From the mathematical viewpoint: such “tensors” are
unusual.
• In computer terms: integer or circular data types are
very natural.
• Fact: integers and circular data are even more efficient
to process than standard real numbers.
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 10 of 12
Go Back
Full Screen
Close
Quit
Why Tensors
19 Century Physics
10.
Remaining Open Problem
Problem and How . . .
• Example: tensors naturally appear in an efficient Taylor series approach to uncertainty propagation.
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
• Detail: the dependence of the result y on the inputs
x1 , . . . , xn is approximated by the Taylor series:
y = c0 +
n
X
i=1
ci · xi +
n X
n
X
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
cij · xi · xj + . . .
i=1 j=1
• Specifics: the resulting tensors ci1 ...ir are symmetric:
ci1 ...ir = cπ(i1 )...π(ir ) for each permutation π.
• Result: the standard computer representation leads to
a r! duplication.
• Problem: how to decrease this duplication.
Remaining Open Problem
Acknowledgments
Title Page
JJ
II
J
I
Page 11 of 12
Go Back
Full Screen
Close
Quit
11.
Acknowledgments
Why Tensors
19 Century Physics
This work was supported in part:
• by NSF grant HRD-0734825 and
• by Grant 1 T36 GM078000-01 from the National Institutes of Health.
Problem and How . . .
From Tensors in . . .
Modern Algorithm for . . .
Modern Algorithm for . . .
Quantum Computing . . .
New Idea: Tensors to . . .
Computing with . . .
Remaining Open Problem
The authors are thankful to Lenore Mullin for her encouragement.
Acknowledgments
Title Page
JJ
II
J
I
Page 12 of 12
Go Back
Full Screen
Close
Quit