Download proposal - NYU Stern School of Business

Research question:
I want to investigate the performance of relational learning methods in a Bias-Variance framework
with the objective to identify and develop methods that perform well on noisy relational domains
such as business and medical.
Learning theory as well as empirical evidence suggests that “simpler” model should perform better.
What does simpler mean in the context of relational learning? Similarly to propositional problems
simplicity or complexity is defined in two places: the “simplicity” of the model class, meaning the
expressive power or bias and as the simplicity (number and level of aggregation) of the independent
features. For many model classes the two are not totally independent. Learning theory combines this
two into one measurable dimension: the VC dimension.
Suggestion: modular approach
Cool application:
Empirical motivation:
LogTree: more data, better performance even for log. Regression
But: More data: computational intractability of high complexity models in ILP
ILP can not produce probabilities (unless fumbling)
Linear as well as naïve Bayes == simple models outperform more complex models on noisy
domains.
Theory for motivation:
Link Vaillants MLT (Performance as a function of model complexity and data)
MLT error bounds for
Overview on the nature of IPL:
The model produced by ILP is a disjunction of conjunctions. This is very similar to decision trees.
Each path in the tree is a conjunction of conditions, every split in a node is a disjunction.
Overview of Propositionalization!
1) Guidance: most ILP systems rely on Declarative language bias to define possible operators to
construct relational featuers.
Relational Problems:
Any single fact can not provide evidence for a classification, only in combination they provide a
picture. Traditionally somebody (the expert would sit down and formulate those important known
relationships formally and calculate them for each object and would then use a traditional model
IPO
Questions: Success, Industry (higher level or partial information), size (given partial information)
Cooc: Count>20, Industry given partial information
Patent: Does a patent have international references? Too big, I will need a subset!
Nuclear Smuggle: Classify people into criminal or not
Mail: ? Does somebody submit code or nor? Who becomes a maintainer?
Thrombosis: Who has the sickness
Redo the bacteria!.
Customer interaction
Criminal networks
The difference in nature between relational and propositional problems:
Are relational problems new? No, they have always existed and people have always tried to find
solutions for them. So why should you care about this comparison? The answer is the same that has
motivated data mining in general: there is too much data to do it the traditional way, you only look at
the things you suspect in the first place, the degree of interaction and relation has increased
consistently over the last couple of years with the increase of communication.
In Rel, the class of an object could potentially depend on the entire network structure. Example: The
success of an IPO is mostly dependent on the reputation of the back that acts as the main
underwriter. However reputation is not measurable. It is however related to the relationships a bank
has had in the past, the number or IPO’s it participated in, the success of those IPO’s, the reputation
of the banks it had business with.