Download Seven Habits of Effective Research Paper Writers

Four Habits of Effective
Technical Paper Writers
Dr. Douglas C. Schmidt
[email protected]
Professor of EECS
Vanderbilt University
Nashville, Tennessee
“A man who has the knowledge, but
lacks the power to express it, is no
better off than if he never had any
ideas at all.”
-- Thucydides, Dean of the Vanderbilt
School of Engineering, 400 BC
Motivation
•Publishing papers in top quality venues is the single
most effective way to invest in your future
• http://citeseer.ist.psu.edu/mostcitedn.html
•Publishing papers is inherently a social process
• i.e., your work will have to pass the scrutiny of
reviewers who are experts in your field
•Reviewers often don’t have a lot of time to spend on
reading & reviewing papers
• Although reviewers are devoted to doing the best
possible & most thorough job, qualified reviewers
have a hectic & demanding schedule
•A reviewer has limited time to read your paper &
frequent distractions to confuse him/her
• Often, your paper will get exactly one (superficial)
reading by a reviewer
This talk describes concrete steps you can take to enhance the quality of
your papers & increase their changes of being accepted for publication
Types of Papers
•We write two general types of papers at ISIS:
•Research papers, which describe a new idea or
technique
•It must describe original/novel work
•It should present solid supporting evidence, not
just conjecture
•“Idea papers” should be backed up by a
convincing analysis
•Experience papers, which present new data based
on actual experience that demonstrates the
(in)effectiveness of technologies, describes
problems encountered, & suggests improvements
•It should provide new evidence (either positive or
negative) to evaluate existing ideas & techniques
•It should help provide direction for future research
& new insights of value to other practitioners
using these technologies
Synopsis of the Paper Writing Process
• Do good research & get solid results
• Avoid trying to publish speculative or premature papers in top venues
• Figure out what you want to say about your work
• Not everything you do is equally interesting to reviewers/readers…
• Figure out who your audience is & determine your venue
• Different audiences/venues have different expectations & payoffs
• Decide whether you’ve got anything useful to add
• Stay current with research in your field
• Follow four habits of effective paper writers
1. Adhere to a structure
2. Present effectively
3. Iterate continuously
4. Collect & integrate feedback
Focus is important, i.e., pick a message & communicate it effectively
Habit 1. Adhere to a Structure
Divide your paper into an abstract + ~5 sections:
• The abstract summarizes the paper & its contributions
• Its purpose is to get your paper into reviewer’s “A pile”
• Section 1 motivates the problem & sketches your solution
• Convince reviewers why it’s a problem & that it’s important to solve
• Section 2 describes how you solved the problem
• Convince reviewers qualitatively your solution really solves the problem
described in Section 1
• Section 3 provides concrete evidence showing that your solution really solves
the problem, e.g., analytical results and/or results from empirical tests
• Convince reviewers quantitatively/analytically the problem is actually
solved & that you have properly interpreted your results
• Section 4 compares you with other work in the area
• Convince reviewers you have a novel contribution
• Section 5 presents lessons learned
• Convince reviewers your work is significant
Writing a Good Abstract
• The abstract is your summary of the
conclusions of your paper & its contributions
to important research problems in your field
• A canonical form for writing good
abstracts is often:
• First provide the context
• Next state the problem being
investigated
• Then outline the contribution(s) this
paper makes to research on the
problem
• Summarize the results of the research
in this paper
An Example Abstract
Many distributed real-time and embedded (DRE) applications require a scalable
event-driven communication model that decouples suppliers from consumers and
supports advanced quality of service (QoS) properties and event filtering
mechanisms. The standard CORBA Notification Service is insufficient to enforce
predictable communication needed by DRE applications and does not leverage
Real-time CORBA capabilities, such as end-to-end priority preservation, priority
models, and scheduling.
This paper makes two contributions to the study of scalable real-time notification
services for DRE applications. First, we explain how we have addressed key
design challenges faced when implementing a Real-time Notification Service for
the TAO real-time object request broker. We discuss the optimizations used to
improve the scalability of TAO’s Real-time Notification Service, which integrates
Real-time CORBA features (such as thread pools, thread lanes, and priority
models) to provide real-time event communication by dedicating thread resources
with minimal locking overhead in the critical path of event propagation. Second, we
analyze the results of empirical benchmarks of the performance and predictability
of TAO’s Real-time Notification Service. These results show that the static realtime assurances enforced by Real-time CORBA are maintained within the more
flexible context of TAO’s Real-Time Notification Service.
An Example Abstract
Many distributed real-time and embedded (DRE) applications require a scalable
event-driven communication model that decouples suppliers from consumers and
supports advanced quality of service (QoS) properties and event filtering
mechanisms. The standard CORBA Notification Service is insufficient to enforce
predictable communication needed by DRE applications and does not leverage
Real-time CORBA capabilities, such as end-to-end priority preservation, priority
models, and scheduling.
This paper makes two contributions to the study of scalable real-time notification
services
DRE
applications. First, we explain how we have addressed key
Provideforthe
context
design challenges faced when implementing a Real-time Notification Service for
the TAO real-time object request broker. We discuss the optimizations used to
improve the scalability of TAO’s Real-time Notification Service, which integrates
Real-time CORBA features (such as thread pools, thread lanes, and priority
models) to provide real-time event communication by dedicating thread resources
with minimal locking overhead in the critical path of event propagation. Second, we
analyze the results of empirical benchmarks of the performance and predictability
of TAO’s Real-time Notification Service. These results show that the static realtime assurances enforced by Real-time CORBA are maintained within the more
flexible context of TAO’s Real-Time Notification Service.
An Example Abstract
Many distributed real-time and embedded (DRE) applications require a scalable
event-driven communication model that decouples suppliers from consumers and
supports advanced quality of service (QoS) properties and event filtering
mechanisms. The standard CORBA Notification Service is insufficient to enforce
predictable communication needed by DRE applications and does not leverage
Real-time CORBA capabilities, such as end-to-end priority preservation, priority
models, and scheduling.
This paper makes two contributions to the study of scalable real-time notification
services for DRE applications. First, we explain how we have addressed key
design challenges faced when implementing a Real-time Notification Service for
the
problem
being investigated
the TAO real-time object State
request
broker.
We discuss
the optimizations used to
improve the scalability of TAO’s Real-time Notification Service, which integrates
Real-time CORBA features (such as thread pools, thread lanes, and priority
models) to provide real-time event communication by dedicating thread resources
with minimal locking overhead in the critical path of event propagation. Second, we
analyze the results of empirical benchmarks of the performance and predictability
of TAO’s Real-time Notification Service. These results show that the static realtime assurances enforced by Real-time CORBA are maintained within the more
flexible context of TAO’s Real-Time Notification Service.
An Example Abstract
Many distributed real-time and embedded (DRE) applications require a scalable
event-driven communication model that decouples suppliers from consumers and
supports
quality of service
(QoS)
properties
and event
Outlineadvanced
the contributions
this paper
makes
to research
onfiltering
the problem
mechanisms. The standard CORBA Notification Service is insufficient to enforce
predictable communication needed by DRE applications and does not leverage
Real-time CORBA capabilities, such as end-to-end priority preservation, priority
models, and scheduling.
This paper makes two contributions to the study of scalable real-time notification
services for DRE applications. First, we explain how we have addressed key
design challenges faced when implementing a Real-time Notification Service for
the TAO real-time object request broker. We discuss the optimizations used to
improve the scalability of TAO’s Real-time Notification Service, which integrates
Real-time CORBA features (such as thread pools, thread lanes, and priority
models) to provide real-time event communication by dedicating thread resources
with minimal locking overhead in the critical path of event propagation. Second, we
analyze the results of empirical benchmarks of the performance and predictability
of TAO’s Real-time Notification Service. These results show that the static realtime assurances enforced by Real-time CORBA are maintained within the more
flexible context of TAO’s Real-Time Notification Service.
An Example Abstract
Many distributed real-time and embedded (DRE) applications require a scalable
event-driven communication model that decouples suppliers from consumers and
supports advanced quality of service (QoS) properties and event filtering
mechanisms. The standard CORBA Notification Service is insufficient to enforce
predictable communication needed by DRE applications and does not leverage
Real-time CORBA capabilities, such as end-to-end priority preservation, priority
models, and scheduling.
This paper makes two contributions to the study of scalable real-time notification
services for DRE applications. First, we explain how we have addressed key
design challenges
facedthe
when
implementing
a Real-time
Notification
Summarize
results
of the research
in this
paper Service for
the TAO real-time object request broker. We discuss the optimizations used to
improve the scalability of TAO’s Real-time Notification Service, which integrates
Real-time CORBA features (such as thread pools, thread lanes, and priority
models) to provide real-time event communication by dedicating thread resources
with minimal locking overhead in the critical path of event propagation. Second, we
analyze the results of empirical benchmarks of the performance and predictability
of TAO’s Real-time Notification Service. These results show that the static realtime assurances enforced by Real-time CORBA are maintained within the more
flexible context of TAO’s Real-Time Notification Service.
Writing a Good Introduction
• The introduction motivates the problem to be solved & sketches the solution
• A canonical form for writing good introductions includes:
• Provide the context for the work, e.g., sketch emerging trends
• Explain which research-worthy problems in this context your paper
focuses on
• Describe your approach for solving these problems & outline why this
paper is novel relative to other work
• Outline the paper’s organization, emphasizing its contribution
Writing a Good Solution Section
• Many authors of papers make the following mistakes
• Do an extremely poor job of motivating what problems their technologies
actually solve
• Focus on low-level implementation details no one cares about
• Many reviewers lose interest in solutions that aren’t well motivated/described
• It’s particular important to motivate your solutions in conferences since the
reviewer assignment process tends to be rather chaotic
• It may help to apply the following structure repeatedly to Section 2:
• Focus on key design challenges
• For each challenge describe the
Scheduling
segment
A
1
• Solution approach
• Solution applied
BSS-B
ESS-B
2
Scheduling
segment
B
Service Context
Client
• Problem
Distributable thread
ESS-A
• Context
Object (Servant)
Service Context
BSS-A
3
4
IDL
Stubs
Current locus of execution
Segment scheduling policies
IDL
Skeletons
B: MUF
A: EDF
5
Dynamic
Scheduler
A: EDF
5
Dynamic
Scheduler
Object
Adapter
ORB Core
Good diagrams are essential
Writing a Good Results Section
• The results section should provide concrete
evidence showing that your solution really
solves the problem
• e.g., analytical results and/or results
from empirical tests
• Experiments should be described using
canonical format, e.g.,
• Characterize hardware/software testbed
• Describe experiment design
• Present empirical results
• Analyze/interpret the results
4000
TSS Key Create:
• Experiments should be repeatable
• It is hard to publish papers in top venues
without thorough empirical results
Emulated
Native OS
3000
Time (nsec)
• e.g., distribute the benchmarking source
3500
2500
2000
1500
1000
500
0
1
51
101
151
201
251
301
351
Number of Keys Created
401
451
501
Writing a Good Related Work Section
• Your related work section should make the contribution of your paper
abundantly clear
• It is not the job of the reviewers to ferret out this information
• One aspect of identifying the contribution is to cite & make appropriate
comparisons with previous work (including your previous work)
• A research paper should compare & contrast the work with prior work,
demonstrating novelty
• An experience paper should compare its results with other papers that
present similar or opposing data
• An experience paper that merely confirms that which is well known
has opposed to that which is widely believed) is of little value
• Make sure that the results of your evaluation in Section 3 actually support
the contributions you claim in the related work section!
If you can’t concisely convey the contributions of the paper in
the related work section then the work is either (1) premature,
(2) insufficiently novel, or (3) inadequately motivated
Writing Good Concluding Remarks
• A common mistake is to simply restate the abstract
• It is particularly important that systems work conclude by summarizing
“lessons learned” from developing & evaluating the technologies showcased in
the paper
• This helps reviewers appreciate that the authors actually did significant
work & have sufficiently generalized their findings so that others may
benefit
• Lessons learned should include a mix of pros & cons, e.g.:
• The component middleware paradigm elevates the abstraction level of
middleware to enhance software developer quality and productivity. It also
introduces accidental complexities, however, that are hard to handle in an
ad-hoc manner for large-scale applications. For example, the CCM requires
many configuration files due to its large number of configuration points.
Model-based tools help to alleviate the accidental complexities of
component middleware
• It’s ok to briefly summarize future work, but don’t go overboard since there’s
insufficient room to validate claims
Habit 2. Present Effectively
• Since reviewers tend to be harried &
busy, it’s essential that your papers be
presented effectively so they will
spend time reading & enjoying them
• There are two general aspects to
effective presentation
• Concise & clear writing style
• Visually appealing typesetting
Harried Reviewer
Concise & Clear Writing Style
• Learning to write concisely & clearly is the single most important contribution
you can make to your future success
• Here are some tips
• Write clearly & unpretentiously
• Favor a down-to-earth style rather than a stuffy, academic one
• Clarity & ease of reading are important in most writing, but they're
especially important for technical writing
• Avoid passive voice
• Break up long sentences & paragraphs
• Use everyday words & make it sound natural
• Read a book or two on writing style, e.g.:
• Strunk & White's The Elements of Style
• Williams' Style: Ten Lessons in Clarity and Grace
• Trimble's Writing with Style: Conversations on the Art of Writing
Take heart! The world is full of people who are smarter than you, but few of
these people can convey what it is they know effectively via writing…
Visually Appealing Typesetting
• Good typesetting is a matter of skill in page layout, typography, and
graphics, not to mention printer quality
• Use the best software tools (word processor, drawing editor, etc.) you can
• Make liberal use of drawings to illustrate key points
• You may not think you need any drawings, but chances are you do
• At the least they break monotony, and at best they'll get your point across
as no amount of explanation can
• Not all drawings have to be
formal UML diagrams
• Informal drawings and even
sketches often convey just
as much information & more
• If you are ``artistically
challenged,'' then have
someone else do the drawings
for you
Habit 3. Iterate Continuously
• You won't get your paper right the first time
• You won't even get it right the first five times
• In fact, you'll probably never get it totally right
• Paper writing is an on-going process
• Expect to write & re-write your papers many times
• Even though lots of examples of good papers & books to help you write
them, technical paper writing (like any other kind of writing) is still an
iterative process
• The best way to improve is to practice, practice, practice!
• Don't look for perfection in one paper before you begin work on the next
• Remember, papers don't exist in isolation; they affect one another
• As with any iteration, your efforts should converge at some point, but that's
just the point where the papers have stabilized enough to let other people
read, understand, & comment on them
If your advisor won’t help you improve as a writer you may need a new advisor
Habit 4. Collect & Integrate Feedback
• Have your colleagues review your papers before submission
• Hold writer’s workshops in your research groups
• www.cs.wustl.edu/~schmidt/writersworkshop.html
• Respond to reviewer comments when they arrive
• This should be a formal process
• Once the feedback starts rolling in, be prepared to hear the worst
• Often something that seemed thoroughly comprehensible to the authors
thoroughly confounded someone else
• The. .negative feedback can be disheartening, especially at the beginning
when you're most vulnerable & also most likely to receive it from people
• While some criticism might not be valid or might result from a simple
misunderstanding, most of it will probably be legitimate
• Give your reviewers the benefit of the doubt & bend over backwards to make
them happy
• You may end up making many, many more people happy in the long run
The Paper Evolution Process
Tech
Report
Workshop
• The ultimate goal of any research
activity should be to publish it in an
top archival journal
• This journey is often an incremental
process
Conference
Journal
• i.e., tech report to workshop to
conference to journal
• It’s important that each variant be
significantly better
Don’t submit the same work to
multiple venues simultaneously
& don’t plagiarize!!!
http://www.schneier.com/blog/archives/2005/08/plagiarism_and.html
Concluding Remarks
• Adopting these habits won't
guarantee your success as a
paper writer, of course
• Nor is this list exhaustive
• But at least it should help you
focus your efforts profitably
• The better your papers are, the
more impact they'll have & the
more opportunities you’ll have in
your career
A computer scientist is equally a scientist & a writer -expend the effort to learn the other half of your profession
Parting Thought
Not on sad Stygian shore, nor in clear sheen
Of far Elysian plain, shall we meet those
Among the dead whose pupils we have been …
Yet meet we shall, and part, and meet again,
Where dead men meet, on lips of living men.