Download Research Dossier

Coleman 1
Alex Coleman
Dr. Guenzel
ENC 1102-0203
October 18, 2015
Research Dossier: Artificial Intelligence
Dossier Introduction
Artificial Intelligence has become more popular of a research field recently now that more
papers are being published as bases for new research. Each new work is dependent upon the work of
other researchers before it. The goal of my paper will be to come to a conclusion about what methods
of research can be used to become a more successful researcher in the field of Artificial Intelligence. I
will use a variety of sources to examine how effective different perspectives are at leading to success. I
will also talk to researchers to see what they believe is the most effective.
The three main methods I will be examining are publishing many papers over a brief period of
time, devoting a lot of time to publishing a single paper, and publishing a paper on a subject that lays
the foundation for new research. My preliminary thesis is that researching something that is a
foundation to new research will show the strongest correlation with high success. In addition, I think
focusing on this one paper primarily will also show more success. I will be using the number of times an
author’s work has been cited as a measure of their success. Collaboration is the most important part of
research, because no paper comes up with new ideas without building on something preexisting.
Therefore, the number of times that a work has been cited demonstrates how much further research
has come from that particular paper. This is an excellent measure of success since it shows how much
people decided to build on that idea using that paper. This will be the main methodology used in my
paper. I will also look at awards won by the authors of papers and examine how that correlates with the
number of times they were cited. Rokach discusses the effective ways to predict researchers’ success in
his paper, so I will connect it with those. He examines the winners of the AAAI Fellowship awards over
the past decade. These papers have a certain set of characteristics that distinguish them as successful
works. I can apply my theoretical models to them to see what most accurately predicts success.
Rokach’s research is primarily the only research so far, so I will have the opportunity to expand or
validate the research here. Finally, I can use the news reports as indicators of success. The AAAI
conference reports and news articles discuss the latest breakthroughs and famous papers, which will tell
me what the Artificial Intelligence community values in a research paper.
I will use an encyclopedia entry (Funk & Wagnalls New World Encyclopedia EBSCO Publishing)
and a non-technical novel (Russell’s Artificial intelligence: a modern approach) on AI to provide
background information on the area to readers who are not familiar with it. I will primarily focus on the
information given by the encyclopedia entry since it will certainly have less bias. The novel may have
some bias to create a fantastical future of AI that may not be realistic. I can use this to address my
audiences prior believes though by opening with the public image of AI and transitioning into what it
really is. This will establish common ground with my readers, upon which I can build my research, but
also avoid coming to conclusions based solely on one fantasized source.
Coleman 2
The majority of the evidence behind my research will come from examination of published
research papers in the field of Artificial Intelligence. These papers make up the majority of my sources. I
will examine their characteristics and how successful they turned out to be. They will provide the
different evidence that I can use to draw a reinforced conclusion from. For example, I included Agrawal’s
Mining association rules between sets of items in large databases, the source that is the most cited
Artificial Intelligence paper from the ACM. Researchers must find this a foundation for their research. It
is from 1993, but it is still cited frequently. This paper included foundation information that was built on
by other researchers. It will serve as evidence for the success of foundation publications and also
publications that had all the researchers time devoted to them. I included a few other papers that were
cited very heavily, and then many that were not cited as often but by authors who published a lot of
papers.
I will also conduct an interview with a professor who spent time researching Artificial
Intelligence so that I can ask him what he thinks defines success for a researcher. However, this will not
be used as the only basis of my research since he only represents one perspective and I do not want to
extrapolate data out of this one case; I can simply include it to support one argument or to show the
disparity and the different sides to the situation. The interview will also serve more as a source of
analysis than evidence; it will give a person’s perspective on what defines success within the field that I
may then analyze and break apart. For evidence from this person, I will examine their actual works and
what old papers they built upon to create their own new paper.
Coleman 3
Research Map
Name: Alex Coleman
Research Questions

How do researchers in the field of Artificial Intelligence measure their success?

How can Artificial Intelligence researchers improve their chances of achieving success?
Keywords: “artificial intelligence background” “artificial intelligence” “artificial intelligence success”
“artificial intelligence ethics” “artificial intelligence award”
Kinds of research:
I will do a lot library research primarily using the library’s online QuickSearch to find published papers on
the subject of Artificial Intelligence. I will also conduct an interview with a professor who previously did
research on Artificial Intelligence to find a primary source account of what researchers defines success
as. I will also use internet research to examine what factors have led to researchers being cited more
often in the past.
Time table
OCTOBER
Sunday
* = research (done throughout), Conferences = TBA
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
1
2
3*
Programming
Practice
4*
5
6
7
8
9
10
Programming
Practice
11*
12 Digital
Paper Trail
13
14
15
16
17*
Programming
Practice
18*
19 Draft
Re.Dossier
20
21
22
23
24
Programming
Practice
25*
26 Final
Re.Dossier
27
28
29
30
31*
Programming
Practice
Coleman 4
NOVEMBER
Sunday
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
1*
2
3
4
5
6
7
Programming
Practice
8*
9
10
11 TED Talk
Pitch /
Rhetorical
Analysis
12
13
15
16
Programming
Regionals
17
18
19
20
21
Programming
Practice
22*
23
24
25
26
Thanksgiving
27 TG Break
28 TG Break
29 TG Break
30
14
Programming
Programming
Regionals
Regionals
DECEMBER
Sunday
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
1
2 Final Draft
3
4
5
Programming
Practice
6
7
8
9
FINALS 
10
11
12
Programming
Practice
13
14
15
 FINALS
16
17
18
19
Programming
Practice
20
21
22
23
24
25
26
Programming
Practice
27
28
29
30
31
Coleman 5
Annotated Bibliography
AAAI News. (2015). AI Magazine, 36(1), 103-111.
News reports from the AAAI in a 2015 edition of their magazine.
The article presents information on Association for the Advancement of Artificial Intelligence (AAAI)related awards, organizational news, and meeting minutes. It states the 2015 recipient of the AAAI
Classic Paper Award was Eugene Charniak for his paper "Statistical Parsing with a Context-Free Grammar
and Word Statistics." It presents information on the Ninth International AAAI Conference on Webl and
Social Media in Oxford, England on May 26-29. It presents minutes for a July 28, 2015 meeting.
[ABSTRACT FROM AUTHOR]
The magazine is edited by David Leake, who works at the University of Indiana. He is a professor of
Computer Science, and the Executive Associative Dean there. His research interests are in Artificial
Intelligence.
This editor has a lot of credibility behind him to verify that this report is accurate. He is a very accredited
researcher in the field of AI. This work will serve as evidence for my research by highlighting the most
important AI papers currently.
Agrawal, R., Tomasz Imieliński, and Arun Swami. 1993. Mining association rules between sets of items in
large databases. In Proceedings of the 1993 ACM SIGMOD international conference on Management of
data (SIGMOD '93), Peter Buneman and Sushil Jajodia (Eds.). ACM, New York, NY, USA, 207-216.
DOI=http://dx.doi.org/10.1145/170035.170072
We are given a large database of customer transactions. Each transaction consists of items purchased by
a customer in a visit. We present an efficient algorithm that generates all significant association rules
between items in the database. The algorithm incorporates buffer management and novel estimation
and pruning techniques. We also present results of applying this algorithm to sales data obtained from a
large retailing company, which shows the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]
Agrawal is a Technical Fellow at Microsoft’s Search Labs. He received numerous awards from ACM,
including the ACM-SIGKDD First Innovation Award, ACM-SIGMOD Edgar F. Codd Innovations Award,
ACM-SIGMOD Test of Time Award, VLDB 10-Yr Most Influential Paper Award, and the Computerworld
First Horizon Award.
Agrawal has been cited over 6500 times across his numerous papers published. His success in research is
likely not very common, so his data may not be relevant. However, it will provide another interesting
opportunity to examine correlation and what leads to success.
Albrecht, S. V., Beck, J. C., Buckeridge, D. L., Botea, A., Caragea, C., Chi-hung, C., & ... Malitsky, Y. (2015).
Reports on the 2015 AAAI Workshop Series. AI Magazine, 36(2), 90-101.
AAAI's 2015 Workshop Program was held Sunday and Monday, January 25-26, 2015, at the Hyatt
Regency Austin Hotel in Austin, Texas, USA. The AAAI-15 workshop program included 16 workshops
covering a wide range of topics in artificial intelligence. Most workshops were held on a single day. The
titles of the workshops included Algorithm Configuration; Artificial Intelligence and Ethics; Artificial
Intelligence Applied to Assistive Technologies and Smart Environments; Artificial Intelligence for Cities;
Coleman 6
Artificial Intelligence for Transportation: Advice, Interactivity, and Actor Modeling; Beyond the Turing
Test; Computational Sustainability; Computer Poker and Imperfect Information; Incentive and Trust in ECommunities; Knowledge, Skill, and Behavior Transfer in Autonomous Robots; Learning for General
Competency in Video Games; Multiagent Interaction without Prior Coordination; Planning, Search, and
Optimization; Scholarly Big Data: AI Perspectives, Challenges, and Ideas; Trajectory-Based Behaviour
Analytics; and World Wide Web and Public Health Intelligence. [ABSTRACT FROM AUTHOR]
Albrecht has his Ph.D. in computer science. He has co-chaired the AAAI workshops in recent years.
He contributes to many famous conferences. He gives many talks at these conferences, and so he must
be relatable to other Artificial Intelligence researchers. His ideas reflect those of many others effectively,
so he will be a useful sample. His work will serve as evidence for me to examine.
Artificial Intelligence. (2014). Funk & Wagnalls New World Encyclopedia, 1p. 1.
Encyclopedia entry on artificial intelligence will be useful for explaining it to people who haven’t studied
it before.
This is an encyclopedia so it does not have a direct author listed for it.
Its purpose is to discuss a topic with no bias and just cover the basics, which will be useful for when I
need to cover a general summary of Artificial Intelligence. It will serve as Background information for my
research.
Baughman, A.K.; Chuang, W.; Dixon, K.R.; Benz, Z.; Basilico, J., "DeepQA Jeopardy! Gamification: A
Machine-Learning Perspective," in Computational Intelligence and AI in Games, IEEE Transactions on,
vol.6, no.1, pp.55-66, March 2014
DeepQA is a large-scale natural language processing (NLP) question-and-answer system that responds
across a breadth of structured and unstructured data, from hundreds of analytics that are combined
with over 50 models, trained through machine learning. After the 2011 historic milestone of defeating
the two best human players in the Jeopardy! game show, the technology behind IBM Watson, DeepQA,
is undergoing gamification into real-world business problems. Gamifying a business domain for Watson
is a composite of functional, content, and training adaptation for nongame play. During
domain gamification for medical, financial, government, or any other business, each system change
affects the machine-learning process. As opposed to the original Watson Jeopardy!, whose class
distribution of positive-to-negative labels is 1:100, in adaptation the computed training instances,
question-and-answer pairs transformed into true-false labels, result in a very low positive-to-negative
ratio of 1:100 000. Such initial extreme class imbalance during domain gamification poses a big
challenge for the Watson machine-learning pipelines. The combination of ingested corpus sets,
question-and-answer pairs, configuration settings, and NLP algorithms contribute toward the
challenging data state. We propose several data engineering techniques, such as answer key vetting and
expansion, source ingestion, oversampling classes, and question set modifications to increase the
computed true labels. In addition, algorithm engineering, such as an implementation of the NewtonRaphson logistic regression with a regularization term, relaxes the constraints of class imbalance during
training adaptation. We conclude by empirically demonstrating that data and algorithm engineering are
complementary and indispensable to overcome the challenges in this first Watson gamification for realworld business problems. [ABSTRACT FROM AUTHOR]
Coleman 7
I could not find much information on the authors of this paper. I did find that they worked at IBM Special
Events.
I don’t have much of a background on these authors, so it will be difficult to apply them to any trends I
see. I will use it as evidence for how a paper depends on, and is depended on by other papers.
Dietmar Jannach, Lukas Lerche, and Michael Jugovac. (2015). Adaptation and Evaluation of
Recommendations for Short-term Shopping Goals. In Proceedings of the 9th ACM Conference on
Recommender Systems (RecSys '15). ACM, New York, NY, USA, 211-218.
DOI=http://dx.doi.org/10.1145/2792838.2800176
An essential characteristic in many e-commerce settings is that website visitors can have very specific
short-term shopping goals when they browse the site. Relying solely on long-term user models that are
pre-trained on historical data can therefore be insufficient for a suitable next-basket recommendation.
Simple "real-time" recommendation approaches based, e.g., on unpersonalized co-occurrence patterns,
on the other hand do not fully exploit the available information about the user's long-term preference
profile. In this work, we aim to explore and quantify the effectiveness of using and combining long-term
models and short-term adaptation strategies. We conducted an empirical evaluation based on a novel
evaluation design and two real-world datasets. The results indicate that maintaining short-term contentbased and recency-based profiles of the visitors can lead to significant accuracy increases. At the same
time, the experiments show that the choice of the algorithm for learning the long-term preferences is
particularly important at the beginning of new shopping sessions. [ABSTRACT FROM AUTHOR]
Dr. Jannach is a professor of Computer Science at TU Dortmund in Germany. He is a part of an Applied
Artificial Intelligence group.
Dr. Jannach provides a unique data point for my examination of success for AI research. He lives in
Germany, but a majority of the AI research is not taking place in Germany. I will be able to examine the
way he communicates with others to learn a lot about what his goals and motives are. His paper is not
cited often, but it one of the most popular. This means it is not useful as a basis for more research (or
has not been explored yet), but provides interesting information to learners. Since this paper is in the
ACM conference, I will use it as evidence of a successful paper.
Ernest Davis, Ethical guidelines for a superintelligence, Artificial Intelligence, Volume 220, March 2015,
Pages 121-124, ISSN 0004-3702, http://dx.doi.org/10.1016/j.artint.2014.12.003.
This is another article discussing the ethics associated with AI. This one was from the Artificial
Intelligence journal, so it may give a different perspective.
Ernest Davis is a researcher of Computer Science at New York University.
Davis is actively doing research unlike some of the other authors of papers. His latest paper was
published just this year in 2015. This means all of his ideas are very relevant today still. I will examine his
works for evidence.
news.com.au, 'What are the Avengers afraid of?', 2015. [Online]. Available:
http://www.news.com.au/technology/innovation/avengers-cast-wary-of-artificial-intelligence/storyfnjwubd2-1227325161532. [Accessed: 19- Oct- 2015].
Coleman 8
Artificial Intelligence in the media, Avengers actors discuss implications of it.
The article primarily gives quotes from the cast of Avengers, so I will analyze them instead. They are all
famous actors and actresses who appeared in Avengers.
Being famous actors, these people have a public image to uphold so they are going to try to appeal to
people with any quotes they give. They do, however, give a series of perspectives from people outside
the field of Artificial Intelligence. I will use their perspectives as evidence in the BEAM model. Looking at
the public image of Artificial Intelligence may reveal another motive for Artificial Intelligence researchers
that defines them as successful. Perhaps if they reach the public sphere of news, they consider
themselves successful. These opinions will show what ideas have made their way into the public sphere
of knowledge.
Paz-Lopez, A., Varela, G., Becerra, J., Vazquez-Rodriguez, S., & Duro, R. (2013). Towards ubiquity in
ambient intelligence: User-guided component mobility in the HI3 architecture. Science Of Computer
Programming,78(10), 1971-1986. doi:10.1016/j.scico.2012.10.014
Abstract: Ambient Intelligence (AmI) systems need to be as transparent as possible, that is, their users
should perceive only the effects of the features presented to them and, in some cases, some kind of
interface. They should not be conscious of how these features are implemented, from a hardware or
from a software point of view. In order to obtain such a high degree of transparency, it is necessary for
the system to be able to provide its services regardless of location, adapting to the environment and the
context in general and to the available hardware more specifically. This is known as ubiquity, and to
achieve it requires considering many aspects, from security and privacy to system interoperability. This
paper is mainly focused on the development of two elements related to ubiquity: physical migration of
components between different platforms together with their associated runtime state, and the
adaptation of those components to the destination platform and physical environment. These solutions
are being addressed within our efforts for the development of a general-purpose middleware for
Ambient Intelligence in the framework of the HI3 project. [Copyright &y& Elsevier]
Paz-Lopez is a Research and Development Computer Engineer. He works at the University of A Coruña.
His works have been cited around 60 times.
Paz-Lopez focuses on one aspect of Artificial Intelligence, Ambient Intelligence. Therefore, he is not very
representative of all researchers. I will use this as evidence.
Rokach, L., Kalech, M., Blank, I., & Stern, R. (2011). Who is going to win the next Association for the
Advancement of Artificial Intelligence Fellowship Award? Evaluating researchers by mining bibliographic
data. Journal Of The American Society For Information Science & Technology, 62(12), 2456-2470.
doi:10.1002/asi.21638
A paper discussing the up and coming researchers who will win the next AAAI fellowship award. This
could be one potential goal that artificial intelligence researchers strive towards.
Accurately evaluating a researcher and the quality of his or her work is an important task when decision
makers have to decide on such matters as promotions and awards. Publications and citations play a key
role in this task, and many previous studies have proposed using measurements based on them for
evaluating researchers. Machine learning techniques as a way of enhancing the evaluating process have
Coleman 9
been relatively unexplored. We propose using a machine learning approach for evaluating researchers.
In particular, the proposed method combines the outputs of three learning techniques (logistics
regression, decision trees, and artificial neural networks) to obtain a unified prediction with improved
accuracy. We conducted several experiments to evaluate the model's ability to: (a) classify researchers
in the field of artificial intelligence as Association for the Advancement of Artificial Intelligence (AAAI)
fellows and (b) predict the next AAAI fellowship winners. We show that both our classification and
prediction methods are more accurate than are previous measurement methods, and reach a precision
rate of 96% and a recall of 92%. [ABSTRACT FROM AUTHOR]
Rokach is a professor at Ben-Gurion University of the Negev, in Israel. His research took place on data
mining algorithms and their applications. He is the primary author, but also wrote the paper with others.
Rokach did not focus on Artificial Intelligence for his degree, so this paper is not actually about Artificial
Intelligence. Instead, he talks about who he thinks will be a recipient of the AAAI fellowship. He coauthored this paper with multiple other professors. This offers multiple unique perspectives into what
makes a successful Artificial Intelligence researcher. I will use this paper as a model for my methods. This
paper examines similar things to what I’m looking at, so seeing its methods will be beneficial.
Russell, S. J., & Norvig, P. (2003). Artificial intelligence: a modern approach. Upper Saddle River, N.J.:
Prentice Hall, c2003.
This book is a modern novel on new approaches to artificial intelligence.
Russell is a professor of Computer Science at University of California, Berkeley. He spent time as the
chair of Electrical Engineering and Computer Science at Berkeley.
This work is not a professional research paper, but rather a book targeted at lay people. While useful
because it will be easier to understand and interpret, I must be careful not to use it as an example of
actual research. Instead, I will look at the analysis it gives, and also use it as background information on
Artificial Intelligence.
Steck, H. “Gaussian Ranking by Matrix Factorization” in Proc. of the 9th ACM Conference on
Recommender Syst., ACM, 2015, pp 115-122.
The ranking quality at the top of the list is crucial in many real-world applications of recommender
systems. In this paper, we present a novel framework that allows for pointwise as well as listwise
training with respect to various ranking metrics. This is based on a training objective function where we
assume that, for given a user, the recommender system predicts scores for all items that follow
approximately a Gaussian distribution. We motivate this assumption from the properties of implicit
feedback data. As a model, we use matrix factorization and extend it by non-linear activation functions,
as customary in the literature of artificial neural networks. In particular, we use non-linear activation
functions derived from our Gaussian assumption. Our preliminary experimental results show that this
approach is competitive with state-of-the-art methods with respect to optimizing the Area under the
ROC curve, while it is particularly effective in optimizing the head of the ranked list. [ABSTRACT FROM
AUTHOR]
Harald Steck works with the technical staff and Learning Research Department of Bell Labs at Murray
Hill. He spent four semesters doing research in the Artificial Intelligence lab at Massachusetts Institute of
Coleman 10
Technology. He also spent three semesters doing research at the Swiss Federal Institute of Technology.
He applied his Artifical Intelligence research through work at Siemens Medical Solutions.
Steck is a very credible source with the variety of perspectives he offers. Even though he is only one
person, his work at three different locations doing/applying AI research gives him three different
perspectives for how collaboration takes place. This paper will be evidence of a successful paper since it
was a feature in the 9th conference.
Szumlanski, S. 'Artificial Intelligence', 2015.
Interview one of my professors, Sean Szumlanski, who has done research in the field of Artificial
Intelligence in the past. This will give me an insightful insider view of research in the field.
Sean Szumlanski is currently a professor in the Computer Science department at UCF.
I have to be careful with this source because it is only one person, and he may work differently than the
typical artificial intelligence researcher. I will use the results of this interview as analysis. I will ask him
questions about what he thinks success is for an Artificial Intelligence researcher, what the common goal
of AI research is, and how successful he thinks he was in his time researching. I will try to ask him around
10-15 questions.
Szumlanski S., F. Gomez, and V. Sims. 2013. A New Set of Norms for Semantic Relatedness Measures.
ACL '13. pp. 890—895.
This paper lists the results of Dr. Szumlanski’s research on an algorithm for determining how related
pairs of nouns are. The algorithm generates a value based on large sets of data fed into the program
(from the web) containing uses of the nouns. The algorithm attempts to mimic the way that people pair
related nouns.
Sean Szumlanski is currently a professor in the Computer Science department at UCF.
I have to be hesitant with this source because it’s a single person, and he may collaborate different than
the average artificial intelligence researcher. He only did major research at one university, so it would be
a fallacy to extrapolate general trends from this one case. It could be useful in combination with others
though. I will examine his paper as evidence that corresponds with what he said in the interview.
Zachary, G.. (2015, June 17). Let’s Shape AI Before AI Shapes Us (1st ed.) [Online]. Available:
http://spectrum.ieee.org/robotics/artificial-intelligence/lets-shape-ai-before-ai-shapes-us
Zachary takes a look at the ways that AI could be implemented such that it would not take control of
humans. It is a brief article that is more focused on drawing attention to the fact that we need to model
it rather than actually proposing a model.
Zachary is a professor at Arizona State University and wrote Endless Frontier: Vannevar Bush, Engineer
of the American Century (MIT Press, 1999).
Coleman 11
Zachary does not only include his only beliefs on the matter, but instead chooses to reference varying
thoughts of famous computer scientists of today. This may be biased by mainly including the thoughts
from the rich, but it does include over three different perspectives of different sides to the argument. I
will use his paper as analysis of other people’s thoughts on AI.