Download Coreference Reading or System Assignment

Project (individual?): Coreference resolution
Literature Review OR System Implementation
Fall 2013, CSCI 49/80/8986
DUE: The assigned activities and presentations covering the findings will be due the last week of
classes. Last two lectures Dec 10 and Dec 12 will be dedicated to the results of this assignment.
Option I: Literature Review
The WS Challenge website by E. Davise
http://www.cs.nyu.edu/davise/papers/WS.html
contains a list of references that include:
* J. Hartshorne, What is implicit causality? Language, Cognition, and Neuroscience,
2013.
* A. Kehler, L. Kertz, H. Rohde, and J. Elman, Coherence and Coreference Revisited,
Journal of Semantics, 25(1), 2008, 1-44.
* M. Lapata and F. Keller, Web-based models for natural language processing, ACM
Transactions on Speech and Language Processing 2:1, 2005.
* Doug Lenat, The Voice of the Turtle: Whatever Happened to AI? AI Magazine, 29:2,
2008, 11-22.
Hector Levesque, The Winograd Schema Challenge, Commonsense-2011.
Hector Levesque, Ernest Davis, and Leora Morgenstern, The Winograd Schema
Challenge, KR-2012. An expanded version of the previous item.
Hector Levesque, On Our Best Behaviour, IJCAI Research Excellence Award
Presentation, 2013.
Gary Marcus, Why Can't My Computer Understand Me? The New Yorker August 18,
2013.
Altaf Rahman and Vincent Ng, Resolving Complex Cases of Definite Pronouns: The
Winograd Schema Challenge EMNLP, 2012.
Terry Winograd, Understanding Natural Language, Academic Press, 1972.
You may note that you are familiar with a few of the references above. This assignment would ask
you to
a) study the references marked by *
b) write a report that would give at least a page long summary for each article,
c) pick one of the papers and prepare a 30 minute long presentation on the material.
Option II:
In this project we will attempt to partially recreate one of the systems reported on in
[1] Altaf Rahman and Vincent Ng, Resolving Complex Cases of Definite Pronouns: The
Winograd Schema Challenge EMNLP, 2012.
Rahman and Ng say “We train this ranker using Joachims' (2002) SVMlight package. It is worth
noting that we do not exploit the fact that each sentence has a twin in training or testing.” (page 4)
Instead you are asked to exploit an open source classification/regression software called WEKA:
http://www.cs.waikato.ac.nz/ml/weka/ :
Weka 3: Data Mining Software in Java
Weka is a collection of machine learning algorithms for data mining tasks. The
algorithms can either be applied directly to a dataset or called from your own Java
code. Weka contains tools for data pre-processing, classification, regression,
clustering, association rules, and visualization. It is also well-suited for developing new
machine learning schemes.
http://www.hlt.utdallas.edu/~vince/papers/emnlp12.html
contains the dataset used in Rahman and Ng's work. You are asked to use the same data set for
training and evaluation purposes. In addition, I would ask you to format the first 16 WS schemas
from a Collection of Winograd Schemas” posted on
http://www.cs.nyu.edu/faculty/davise/papers/WS.html in the same manner as the above mentioned
dataset so that you can use these 16 WS schemas for evaluation purposes also.
Rahman and Ng elaborate in great detail on a variety of features they use in creating their ranker –
basis for their corefernce resolution system. It is your decision which features that they list you
would like to explore. It were ideal if you could implement at least couple of the features used in
“The Baseline Ranker” (page 9). Rahman and Ng refer to their earlier work, “Supervised models for
coreference resolution” (EMNLP-09), for complete account on linguistic features typically used in
coreference resolution task. If you could incorporate at least one of the features introduced in
Section 4 in [1].
During the last week of classes you will be asked to present your system as well as discuss your
experiences, findings, results. You will be given 30 minutes.