Download The Power of Tree Series Transducers

The Power of Tree Series Transducers
Andreas Maletti1
Technische Universität Dresden
Fakultät Informatik
June 15, 2006
1
Research funded by German Research Foundation (DFG GK 334)
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
1 / 26
1
Motivation
2
Definition of Tree Series Transducers
3
Results
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
2 / 26
Machine Translation
Overview
Subfield of computational linguistics
Automatic translation shall assist (human) translator
Offer several (likely) alternatives
History
1954: High prospects and expectations after Georgetown Experiment
1966: “Perfect translation” failed (ALPAC report)
1993: Statistical machine translation system [Brown et al 93]
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
3 / 26
Machine Translation
Problem
Translate text of language X into grammatical text of language Y .
1
Preserve meaning
2
Preserve connotation
3
Preserve style
Relaxed Problem
Transform text of language X into text of language Y such that
1
the result is grammatical
2
expert for X and Y can discern original sentence
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
4 / 26
Tree-based Model [Yamada, Knight 01]
VB
PRP VB1
VB2
He adores VB
TO
listening TO NN
to music
⇒
VB
PRP
VB2
VB1
He TO
VB adores
NN TO listening
music to
⇓
VB
PRP
VB2
VB1
VB ga daisuki desu
kare ha TO
NN TO kiku no
ongaku wo
VB
PRP
VB2
VB1
⇐ He ha TO
VB ga adores desu
NN TO listening no
music to
kare ha ongaku wo kiku no ga daisuki desu
3 phases: (i) Reorder, (ii) Insert, (iii) Translate
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
5 / 26
Implementation of Phase (i): Reorder
Implementation by top-down tree series transducer
VB (0.723)
TT (vb)
VB
PRP VB1
VB2
He adores VB
TO
listening TO NN
TT (prp) TT (vb2) TT (vb1)
⇒
PRP
VB2
VB1
He VB
TO adores
listening TO NN
to music
to music
0.723
vb(VB(x1 , x2 , x3 )) → VB(prp(x1 ), vb2(x3 ), vb1(x2 ))
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
6 / 26
Implementation of Phase (i): Reorder
VB (0.723)
VB (0.723)
TT (prp) TT (vb2) TT (vb1)
PRP (1) TT (vb2) TT (vb1)
PRP
VB2
VB1
He VB
TO adores
listening TO NN
⇒
He
VB2
VB1
VB
TO adores
listening TO NN
to music
to music
1
prp(PRP(x1 )) → PRP(x1 )
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
7 / 26
Implementation of Phase (i): Reorder
VB (0.723)
VB (0.723)
PRP (1) TT (vb2) TT (vb1)
PRP (1) VB2 (0.749) TT (vb1)
He
VB2
VB1
VB
TO adores
listening TO NN
to music
⇒
He TT (to) TT (vb) VB1
TO
VB adores
TO NN listening
to music
0.749
vb2(VB2(x1 , x2 )) → VB2(to(x2 ), vb(x1 ))
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
8 / 26
Implementation of Phase (i): Reorder
VB (0.723)
VB (0.723)
PRP (1) VB2 (0.749) TT (vb1)
He TT (to) TT (vb) VB1
TO
VB adores
TO NN listening
to music
Andreas Maletti (TU Dresden)
PRP (1)
⇒
∗
VB2 (0.749)
He TO (0.893)
VB1 (1)
VB (1) adores
NN (1) TO (1) listening
music
The Power of Tree Series Transducers
to
June 15, 2006
9 / 26
Implementation of Phase (i): Reorder
VB
PRP VB1
VB2
He adores VB
TO
listening TO NN
VB (0.723)
PRP (1)
⇒∗
to music
VB2 (0.749)
He TO (0.893)
VB1 (1)
VB (1) adores
NN (1) TO (1) listening
music
to
The above reordering has probability:
0.723 · 0.749 · 0.893 = 0.484
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
10 / 26
Implementation Details
Rules
Original
PRP VB1 VB2
VB TO
TO NN
Andreas Maletti (TU Dresden)
Reordered
PRP VB1 VB2
PRP VB2 VB1
VB1 PRP VB2
VB1 VB2 PRP
VB2 PRP VB1
VB2 VB1 PRP
VB TO
TO VB
TO NN
NN TO
Probability
0.074
0.723
0.061
0.037
0.083
0.021
0.251
0.749
0.107
0.893
The Power of Tree Series Transducers
June 15, 2006
11 / 26
Tiburon [May, Knight 06]
Overview
Implements top-down weighted tree automata and top-down tree
series transducers over the probability semiring
Operations WTA: intersection, weighted determinization, pruning
Operations TST: application, composition, training
Applications
Used to implement Yamada-Knight model (custom implementation
took > 1 year, implementation in Tiburon 2 days)
Used to implement Japanese transliteration [Knight, Graehl 98]
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
12 / 26
1
Motivation
2
Definition of Tree Series Transducers
3
Results
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
13 / 26
Tree Series Transducers
Overview
tree series
transducer
weighted tree
automaton
weighted transducer
tree transducer
weighted automaton
tree automaton
generalized
sequential machine
string automaton
History
Introduced in [Kuich 99]
Extended to full generality in [Engelfriet, Fülöp, Vogler 02]
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
14 / 26
Semiring
Definition
(A, +, ·, 0, 1) semiring, if
(A, +, 0) commutative monoid
(A, ·, 1) monoid
· distributes (both sided) over +
0 is absorbing for · (a · 0 = 0 = 0 · a)
Example
Natural numbers (N, +, ·, 0, 1)
Probabilities ([0, 1], max, ·, 0, 1)
Subsets (P(A), ∪, ∩, ∅, A)
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
15 / 26
Top-down Tree Series Transducer [Engelfriet et al 02]
Definition
Polynomial top-down tree series transducer (Q, Σ, ∆, A, I , R) where
Q finite set of states
Σ and ∆ input and output ranked alphabet
A = (A, +, ·, 0, 1) semiring
I ⊆ Q set of initial states
R finite set of rules of the form
a
q(σ(x1 , . . . , xk )) → t
where t ∈ T∆ (Q(Xk ))
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
16 / 26
Properties of TST
Definition
(Q, Σ, ∆, A, I , R) top-down TST
deterministic, if there is at most one rule with a given left hand side
and at most one initial state
linear, if (for every rule) every variable appears at most once in the
right hand side
nondeleting, if (for every rule) variables that occur in the left hand
side also occur in the right hand side
Note
Bottom-up TST process input tree from leaves toward root.
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
17 / 26
Classes of Transformations
Definition
denotation
class of transformations computed by
substitution
x-TOPε (A)
top-down TST with properties x
ε-subst.
x-TOPo (A)
top-down TST with properties x
o-subst.
x-BOTε (A)
bottom-up TST with properties x
ε-subst.
x-BOTo (A)
bottom-up TST with properties x
o-subst.
In diagram:
x-TOPω (A) abbreviated to xω>
x-BOTω (A) abbreviated to xω⊥
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
18 / 26
1
Motivation
2
Definition of Tree Series Transducers
3
Results
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
19 / 26
Hasse Diagram for Deterministic TST
Probability Semiring and Semiring of Natural Numbers
d⊥
ε
d⊥
o
dt⊥
ε
dl⊥
ε
dn⊥
ε
h⊥
ε
dlt⊥
ε
dnt⊥
ε
hl⊥
ε
hn⊥
ε
dn⊥
o
dl⊥
o
dt⊥
o
dnl⊥
=
dnt⊥
o
dlt⊥
o
h=
o
dnlt⊥
=
hn=
o
hl=
o
dnt>
=
dlt>
=
dn>
=
dl>
=
hnl=
=
dnlt>
=
dnl>
=
dt>
=
d>
=
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
20 / 26
Hasse Diagram for Deterministic TST
Semiring of Subsets
d⊥
=
d>
=
dn⊥
=
dl⊥
=
dt⊥
=
dnl⊥
=
dnt⊥
=
dlt⊥
=
dt>
=
dn>
=
dl>
=
dnt>
=
dlt>
=
dnl>
=
h>
=
h⊥
o
h⊥
ε
hl>
=
dnlt⊥
=
hn=
=
hl⊥
o
dnlt>
=
hl⊥
ε
hnl=
=
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
21 / 26
Composition of Transformations
Definition
Let
ϕ : TΣ × T∆ → A
ψ : T∆ × TΓ → A
Composition of ϕ and ψ
(ϕ ; ψ) : TΣ × TΓ → A
X
(t, v ) 7→
ϕ(t, u) · ψ(u, v )
u∈T∆
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
22 / 26
Composition Results
Theorem (see [Kuich 99] and [Engelfriet et al 02])
A commutative semiring
nlp-BOT(A) ; p-BOT(A) = p-BOT(A)
p-BOT(A) ; bdth-BOT(A) = p-BOT(A)
Theorem
A commutative semiring
lp-BOT(A) ; p-BOT(A) = p-BOT(A)
p-BOT(A) ; bd-BOT(A) = p-BOT(A)
bdt-TOP(A) ; lp-TOP(A) ⊆ p-TOP(A)
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
23 / 26
References
Joost Engelfriet, Zoltán Fülöp, and Heiko Vogler.
Bottom-up and top-down tree series transformations.
J. Autom. Lang. Combin., 7(1):11–70, 2002.
Werner Kuich.
Full abstract families of tree series I.
In Jewels Are Forever, pages 145–156. Springer, 1999.
Werner Kuich.
Tree transducers and formal tree series.
Acta Cybernet., 14(1):135–149, 1999.
Jonathan May and Kevin Knight.
Tiburon: A weighted tree automata toolkit.
In Proc. 11th Int. Conf. Implementation and Application of Automata,
LNCS. Springer, 2006.
to appear.
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
24 / 26
Publications
Conferences
Myhill-Nerode theorem for sequential transducers over unique GCD-monoids.
Proc. 9th Int. Conf. Implementation and Application of Automata
Relating Tree Series Transducers and Weighted Tree Automata.
Proc. 8th Int. Conf. Developments in Language Theory
Compositions of bottom-up tree series transformations.
Proc. 11th Int. Conf. Automata and Formal Languages
The power of tree series transducers of type I and II.
Proc. 9th Int. Conf. Developments in Language Theory
Hierarchies of tree series transformations—Revisited.
Proc. 10th Int. Conf. Developments in Language Theory
The substitution vanishes.
(with A. Kühnemann)
Proc. 11th Int. Conf. Algebraic Methodology and Software Technology
Does o-substitution preserve recognizability?
Proc. 11th Int. Conf. Implementation and Application of Automata
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
25 / 26
Publications
Journals
Relating tree series transducers and weighted tree automata.
Int. J. of Foundations of Computer Science 16, 2005
Hasse diagrams for classes of deterministic bottom-up tree-to-tree-series
transformations.
Theoretical Computer Science 339, 2005
Cut sets as recognizable tree languages.
(with B. Borchardt, B. Šešelja, A. Tepavčević, H. Vogler)
Fuzzy Sets and Systems 157, 2006
Incomparability results for classes of polynomial tree series transformations.
(with H. Vogler)
J. Automata, Languages and Combinatorics, to appear, 2006
Bounds for tree automata with polynomial costs.
(with B. Borchardt, Zs. Gazdag, Z. Fülöp)
J. Automata, Languages and Combinatorics, to appear, 2006
Compositions of tree series transformations.
Theoretical Computer Science, to appear, 2006
Andreas Maletti (TU Dresden)
The Power of Tree Series Transducers
June 15, 2006
26 / 26