Download Methods of Observation and Inference

Methods of
Observation
PS 204A, Week 2
What is Science?
Science is: (think Ruse)
 Based on natural laws/empirical regularities.
 Makes predictions.



Collections of laws that generate predictions that are
empirically confirmed constitute “explanations.”
Must be falsifiable.
Is always tentative (move from grossly wrong to
more subtly wrong theories).
The Scientific Enterprise
Generalization
Theory
Observation
Hypothesis
Theory




Theories always possess a
“theoretical notion” or analogy
that simplifies reality.
This analogy is embodied in the
assumptions or premises of the
theory. Assumptions are
themselves unobservable – and
known to be simplifications (e.g.,
individuals are rational, states are
unitary actors). Prefer plausible
over less plausible premises.
Since premises are never “true” or,
at least, are unobservable, theories
are never true, only more or less
useful.
Utility is defined by the number of
empirically supported propositions
the theory generates.
Analogy
Generalization
Theory
Observation
Hypothesis
Plausibility of Premises




If in theories, premises are things we do not
agree on and are unobservable, how do we
assess their plausibility?
Utility of their predictions (Friedman).
Accordance with natural laws.
Transform premises into objects of
investigation that are themselves the subjects of
theories.
Theory of rationality.
 Theory of unitary states.

Science is a series of “boxes within
boxes”

Balance of power theory: international system is
anarchic and composed of unitary states wishing
only to survive.
Within any given “box,” we take the
premises as “given.” But any premise
may itself become an object of
investigation in another “box.”
Internal hierarchy: state “speaks” with
a single voice
If testing leads to a
revision of a premise,
then still need to plug
back into original
theory and retest.
Unitary states
wishing to survive
Anarchy: no
common authority
Act to check the power of
other states.
Hypotheses



Propositions are general
statements that follow
logically from the premises.
Hypotheses are propositions
that contain only observable
variables (i.e., if X, then Y,
when both X and Y can be
observed).
Central issue is deductive
validity: Does the hypothesis
follow logically and
axiomatically from the
premises?
Generalization
Theory
Deductive
Validity
Observation
Hypothesis
Tests

We test theories by examining
whether the hypotheses they
generate are supported by the
evidence. We make the
observations the theories
imply.



Conclusion validity: is there a
relationship between X and Y?
Internal validity: is the
relationship causal?
Construct validity: do the
observable measures capture
concepts in the theory
appropriately?
Theory
Generalization
2. Internal
Validity
Hypothesis
Observation
Test
3. Construct
Validity
1. Conclusion
Validity
Internal Validity


Is there a causal relationship in the model? You
have evidence that YOUR treatment (IV,
intervention, program) caused the outcome
(DV).
It is possible to have internal validity and not
construct validity: internal validity does not tell
you that you measured your intervention or
outcomes well. (ex. Reading program, but really
adult attention) = yes internal, not construct
Construct (Measurement) Validity

Assuming that there is a causal relationship in
the study, can you claim the IV (intervention)
reflected well your idea of the construct of the
measure? Did you implement the the IV you
intended to implement and did you measure the
outcome you wanted to measure? Did you
operationalize well the ideas of the cause and
effect?
Conclusion Validity

Is there a relationship between the two
variables? You might infer there is a positive or
negative or etc. relationship. Or no relationship.
Explanation v. Prediction



Theory offers an explanation for observed facts
and predicts new facts that, once confirmed, are
also explained.
Theories must be potentially falsifiable.
Popper/Hempel insist that known facts cannot
falsify a theory. Therefore, prediction is the goal
of all science.
Alternatively, Snyder argues that if scientific
evidence is objective, evidence is evidence
independent of the timing of it’s discovery
relative to the theory.
Who’s Right?


All evidence helps corroborate a theory, even
known facts.
Predictions are more “valuable” than
explanations in providing evidence for a theory.
Generalization
Possible
Refinements


Generalization
Theory

4. External
Validity
Observation
Hypothesis

Can we generalize our
observations to larger
populations?
Key issue here is external
validity (i.e., will conclusions
hold for other people at
other times).
Testing may lead us to refine
our theories further,
propelling the cycle another
round.
Science is interactive. Tests
suggest refinements to
theories, which then generate
new predictions and tests.
Conversation between theory
and evidence.
External validity

Assuming a causal relationship in this study
between constructs of the cause and effect, can
you generalize this effect to other persons,
places, times?
Deductive v. Inductive Reasoning



Hypothetico-Deductive method begins with theory,
then generates tests. From observations, draw
inferences about theory, which in turn lead to
generalizations about unobserved populations.
Inductive approach begins with observations and draws
draw inferences about unobserved populations. May or
may not lead to theory. (Cuts into cycle at
observations, then draws inferences. Can be
predictive.)
Induction can be science: body of replicated and
confirmed laws that are predictive. But, falsifiablility is
always an issue.
What can we learn by
“observing”?
Observation




Inference
Observation central to both deduction and
induction.
How do we draw inferences about unobservable
phenomena, premises, or populations from
observable phenomena?
How do we learn about what we can’t see, from
what we can?
Applies equally to inherently unobservable traits
(what goes on in people’s heads), future events
(predictions), and true populations (alternative
worlds).
Descriptive Inference:
Or how do we know what we saw?

What is this a case of ? What is the class of
which you observe one or more members?
Many categories are question or theory dependent.
 If observation is unique, no generalizations are
possible.
 If observation is exhaustive (all members of class),
what can you generalize to?

Descriptive Inference II



Probabilistic v. Deterministic Events: all events
have systematic and non-systematic
components.
Probabilistic events occur with some probability
< 1.0; if replayed under identical conditions, the
observed result would differ (more or less). On
average, would get same result (non-systematic
component is random).
Deterministic events occur with certainty (p =
1.0). Special case in which non-systematic
component is zero.
Descriptive Inference III

With a probabilistic event, how are we to classify
a particular instance?
What should we infer from a war in which country A
loses? We observe the loss, but can we generalize to
other similar cases? If the probability of victory was
.25, what can we infer from the actual loss?
 Inference is harder the smaller the number of cases.


Problem arises when we mistake a probabilistic
event for a deterministic event.

Incorrect to infer any “pathology” or “mistake” in
such instances.
Induction I: Empirical Laws




Analysis limited to observable phenomena only.
An empirical law is a robust “regularity” (e.g.,
the democratic peace)
By extending empirical laws, we can make
predictions (inferences) about future events.
But,
Concepts do not exist independent of theory.
 Correlations may be spurious.
 Correlation does not equal causation. We may
“explain” events by empirical laws but such laws do
not imply cause or constitute a causal test.

Induction II: Thick Description as a
Data Collection Method


“Thick description” as detailed casework.
May discover correlations


Uses observable to derive unobservable traits.


Darwin
Discriminate between a wink and a twitch by looking
at the reactions of others, context, etc.
Generates an interpretation that we can think of
as an inductive theory.
Induction III: Thick Description as Science

Use observables to infer unobservable phenomena,
typically motives, intent, purpose. Then explain
observed outcomes in terms of the actor’s selfunderstanding.





Geertz: “not an experimental science in search of law but an
interpretive one in search of meaning.”
Risk of circularity: observables used to infer unobservables,
which are then used to explain observables.
Not falsifiable.
External validity? Geertz: object is “not to generalize across
cases but to generalize within them.”
Interpretation is not science, because inductive and
non-falsifiable.
Conclusion



Observation is central to the scientific
enterprise. There can be no science without
observation.
Observation by itself can never demonstrate
cause. To explain a phenomenon requires an
empirically supported theory.
Nonetheless, much of what we do in political
science is observe.