Download Guide to literature Study

Guide to Literature study
By Bo Nørregaard Jørgensen, MMMI)
(! Please note that the guide is meant for inspirational purposes and is subject to ongoing revision)
Purpose
The purpose of a literature study is to analyze critically a segment of a published body of knowledge
through summary, classification, and comparison of prior research studies, reviews of literature, and
theoretical articles.
Elements of the literature study include:
1.
2.
3.
4.
Literature search
Detailed review of selected research papers
Writing up state-of-the-art
Putting your own research into perspective of state-of-the-art
ECTS points
The literature study as described here is scheduled as 6 ECTS points (I.e., 150 work hours).
Getting started
Do a comprehensive literature search based on topics relating to your thesis subject. Search on keywords
etc. The final outcome of your literature search should be roughly 15 research papers (corresponding to
approximately 300 pages.) You should only include papers that are essential to your field of research in
your literature study report. That is, you have to discard all papers that are not relevant to your research
subject as part of the search process. During the search process, you should expect to read the abstract of
at least 3-4 times as many papers as you will include in your final literature report. You should schedule a
least 20 hours to do the literature search and initial skimming of papers for relevance. The literature study
report is expected to contain a detailed review for each of the selected papers; that is, you have to review
approximately 15 research papers. You should schedule 7 hours for each research paper. This leaves
approx. 25 hours for write the state of the art section based on the selected research papers.
How to skim a research paper
When you are going to find relevant candidate papers for your literature study you will browse through a
large amount of papers. Not all papers are relevant, so how do you quickly read a paper to decide whether
it is relevant? Here are some steps that will help you to survive:




Read the title. (What is the paper about?)
Read the abstract. (Should give you a concise overview of the paper.)
Read the introduction. (Look for motivations, relation to other work, and a more detailed
overview.)
Look at the structure of the paper. (What do the remaining sections address? How do they fit
together?)



(Read the previous/related work section. (How does this work relate? What is new or different
about this work?))
Read the conclusions. (What were their results?)
Read the body of the paper. You may want to skip over all the equations the first time through.
As soon as you find the paper irrelevant to your research subject you simply skip the rest of the steps and
continue with the next paper in your stack of papers.
The references won’t mean much to you if you’re not familiar with the area. Sometimes important parts of
the work may be contained in the references, particularly in conference papers since space is limited. The
references are very important when you are researching a topic—they point you to related research as well
as the research upon which the current paper builds upon.
Reviewing a research paper
When you have to review the work of others as part of your literature study you will have to address many
of the same elements as you do when writing an abstract about your own work. That is, do the paper’s
authors tell you what god researchers are expected to tell their readers? You should fill out the following
form for each paper that is part of your literature study.
Title of the paper
Name(s) of author / authors
What’s the name of the dude?
Motivation for doing the research.
Claimed relevance or benefits!
What is the motivation of the research work? Is there a
crisis in the research field that the paper attempts to
resolve? Is the research work attempting to overcome the
weaknesses of existing approaches? Is an existing research
paradigm challenged? In short, what is the niche of the
paper?
The research problem / objective
What is the argument of the paper?
Is this research original?
What is new in this paper? A new question is asked? A
new understanding of the research problem? A new
methodology for solving problems? A new algorithm? A
new breed of software tools or systems? A new
experimental method? A new proof technique? A new
formalism or notation? A new evidence to substantiate or
disprove a previously published claim? A new research
area? In short, what is original about this paper?
Chosen research approach /
methodology.
What have they done to investigate / find a solution to the
research problem? What is the methodology adopted to
substantiate the claims? What are the major theorems?
What experiments are conducted? Data analyses?
Simulations? Benchmarks? User studies? Case studies?
Examples? In short, what makes the claims scientific (as
opposed to being mere opinions)?
Reported results / products /
effects.
What is the outcome of their research? What did they find
or create?
What are the contributions /
conclusions?
What do they claim to have achieved? Summaries their
contributions in a clear and concise fashion!
Your evaluation of the reported
results with respect to the
research objective.
1. Are the results significant? Is the work scratching minor
itches? Are the authors solving artificial problems (aka
strawman)? Does the work enable practical applications,
deepen understanding, or explore new design space?
2. Are the contributions significant? Are the authors simply
repeating the state of the art? Are there real surprises?
Are the authors aware of the relation of their work to
existing literature? Is the paper addressing a well-known
open problem?
3. Are the claims valid? Have the authors been cutting
corners (intentionally or unintentionally)? Has the right
theorem been proven? Errors in proofs? Problematic
experimental setup? Confounding factors? Unrealistic,
artificial benchmarks? Comparing apples and oranges?
Methodological misunderstanding? Do the numbers add
up? Are the generalizations valid? Are the claims modest
enough? That is does the paper contain evidence for the
claims?
Your conclusion on the relevance
of this research with respect to the
field in general.
What have we learned from the paper? Is this a key
paper? Do the reported results contribute significantly to
state of the art? Shall the standard practice of the field be
changed as a result of the new findings? Is the result
generalizable? Can the result be applied to other areas of
the field? What are the open problems? In short, what are
the lessons one can learn from the paper?
Your conclusion on the relevance
of this research with respect to
your own research.
How does this work related to yours? Are there common
elements? Are they addressing the same research
problem? Can you use their results to solve your problem,
or part of it? Can you improve on their approach? Do their
ideas have other applications or extensions that the
authors might not have thought of?
Review a research paper implies careful reading. That is, when you read a research paper, your goal is to
understand the scientific contributions the authors are making. This is not an easy task. It may require
going over the paper several times. Expect to spend several hours to read a paper.
Here are some initial guidelines for how to read a research paper carefully:
Read critically: Reading a research paper must be a critical process. You should not assume that the
authors are always correct. Instead, be suspicious. Critical reading involves asking appropriate
questions. If the authors attempt to solve a problem, are they solving the right problem? Are there
simple solutions the authors do not seem to have considered? What are the limitations of the solution
(including limitations the authors might not have noticed or clearly admitted)? Are the assumptions
the authors make reasonable? Is the logic of the paper clear and justifiable, given the assumptions, or
is there a flaw in the reasoning?
If the authors present data, did they gather the right data to substantiate their argument, and did they
appear to gather it in the correct manner? Did they interpret the data in a reasonable manner? Would
other data be more compelling?
Read creatively: Reading a paper critically is easy, in that it is always easier to tear something down
than to build it up. Reading creatively involves harder, more positive thinking. What are the good ideas
in this paper? Do these ideas have other applications or extensions that the authors might not have
thought of? Can they be generalized further? Are there possible improvements that might make
important practical differences? If you were going to start doing research from this paper, what would
be the next thing you would do?
Make notes as you read the paper: Many people cover the margins of their copies of papers with
notes. Use whatever style you prefer. If you have questions or criticisms, write them down so you do
not forget them. Underline key points the authors make. Mark the data that is most important or that
appears questionable. Such efforts help the first time you read a paper and pay big dividends when
you have to re-read a paper after several months.
After the first read-through, try to summarize the paper in one or two sentences. Almost all good
research papers try to provide an answer a specific question. (Sometimes the question is a natural one
that people specifically set out to answer; sometimes a good idea just ends up answering a worthwhile
question.) If you can succinctly describe a paper, you have probably recognized the question the
authors started with with and the answer they provide. Once you have focused on the main idea, you
can go back and try to outline the paper to gain insight into more specific details. Indeed, if
summarizing the paper in one or two sentences is easy, go back and try to deepen your outline by
summarizing the three or four most important subpoints of the main idea.
If possible, compare the paper to other works. Summarizing the paper is one way to try to determine
the scientific contribution of a paper. But to really guage the scientific merit, you must compare the
paper to other works in the area. Are the ideas really novel, or have they appeared before? (Of course
we do not expect you to be experts and know the areas ahead of time in this class!) It is worth
mentioning that scientific contributions can take on many forms. Some papers offer new ideas; others
implement ideas, and show how they work; others bring previous ideas together and unite them
under a novel framework. Knowing other work in the area can help you to determine which sort of
contribution a paper is actually making.
Sooner or later, you will come across something that you don’t understand. What can you do? You should
try to figure out what it is and how it is being used (even though you still don’t understand it). For further
reading, see the references!
Writing state of the art
This section reviews the state of the art and related work relevant to your research. Here you list the
current knowledge in the field of your research and the respective references. You base this review on your
collection of paper reviews.
The idea is to present (critical analysis comes a little bit later) the major ideas in the state of the art right
up to, but not including, your own personal brilliant ideas. You organize this section by ideas, and not by
author or by publication. For example, if there have been three important main approaches to Software
Evolution Analysis to date, you might organize subsections around these three approaches.
In the first paragraph of the section you have to outline the contribution up to what is considered to be the
present state of the art. In the next paragraph you describe state of the art. The final paragraph summaries
gaps in research, e.g., open questions, unresolved problems or new perspective of immediate interest in
the field, some of which should relate to your own research question.
Composing the literature study report
You should organize our report as follows:
Preface
Describe the context for the literature report.
Introduction
Here you introduce the reader to your field of research. The idea is to give the reader a birds- eye
view on the subject of your PhD work before you zoom in on the particular topics that you are about
to investigate.That is, define or identify the general topic, issue, or area of concern, thus providing
an appropriate context for reviewing the literature.
State-of-the-art
If your literature study includes more than one research field you will have to create a subsection for
each field of research. You write this section after you have written all of the literature reviews!
Literature reviews
Create a subsection for each paper.
References
This section contains references for all the papers you have reviewed in the section ‘Literature
reviews’ . You are not to include all the papers that you have discard during the literature search
process. However, you should maintain an additional list over all the papers that seemed
interesting, but did not make it to the literature review. This is important, as you will properly
encounter these papers again in later literature searches.
2 State of the art (or "Literature review" or else):•
”State-of-the-art” afsnit er bygget op således: Først en kort opsumering af de vigtigst historiske bidrag fra
begyndelse op til i dag, der efterfølger en kort gennemgang af hvad der er de bedste løsninger, dvs. stateof-the-art i dag til at løse de problemer som der er beskrevet i afsnittet ”introduction / el. Motivation”. Så
kommer en kort analyse af hvorfor disse ’fantanstiske’ løsninger ikke er nok – manglerne ved dem
identificeres. Endelig beskrives hvordan vi (artiklens forfattere) vil råde bod på disse mangler og derved
lukke det hul i forskningen som vi lige har identificeret i vores analyse af state-of-the-art.