Download Computational Impact Assessment of Social Justice Documentaries

Computational Impact Assessment
of Social Justice Documentaries
Jana Diesner, Jinseok Kim, Shubhanshu Mishra,
Kiumars Soltani, Sean Wilner, Amirhossein Aleyasen
The iSchool, Department of Computer Science,
Illinois Informatics Institute
1
Problem Statement: Measuring Impact
• Goal of (social justice) documentaries: Storytelling
– Create memories, imagination, sharing
• Goal of funders and producers: Impact
– Evoke change in people’s knowledge and/or behavior
• Common approach/ status quo:
– Big data (frequency counts) vs. thick data (interviews)
– Science: psychological effects of media on individuals
– Need: computational, empirical, scalable, rigorous, theory
• Q: How can we know if a documentary has what impact?
– Generalized: measure impact of information in terms of change
• Q: How early in a film’s life cycle can we answer this question?
– Prediction models for likely impact trajectories
• Here and now usefulness for producers
– Strategic allocation of limited resources
– Leverage existing social capital
2
Approach:
A story of microscopes and telescopes
• Assumption: documentaries produced, screened, watched
as part of larger, dynamic ecosystems of stakeholders and
information flow
• Method: identify, map, monitor, analyze social
(stakeholders) and semantic (information) networks to
study their structure, functioning and dynamics
3
DIMENSION
LEVEL
CONTENT
EXPECTED OUTCOME
INDEX
ANALYTICS
ITEM
Guiding Factor
Description
Ranking
weighing
Report by producers or funding agencies
Outreach
Stats
Number of movies, CDs distributed
Number of theatrical, Internet release
Duration of release; Sales of product
MASS MEDIA
Mass Media
Attention
Text Mining
Web Analytics
Frequency of news coverage weighted
by influence (article, opinion/editorial)
Domestic, international broadcast
USER MEDIA
User Media
Attention
PROFESSIONAL
MEDIA
Prestige
INTERPERSONAL
INTERACTION
Intimate
Attention
MESSAGE
EVALUATION PRIORITY
RESOURCE
OFFLINE
RELEASE
MEDIUM
RESPONSIVE MEDIUM
MEDIUM
TARGET
ONLINE
AUDIENCE SIZE
Reachability
HOMOGENEITY
Diversity
SINKER
Passiveness
TRANSMITTER
Leadership
AUDIENCE TYPE
COLLECTIVE ENTITY
COGNITIVE
GLOBAL
SOCIETAL
INDIVIDUAL
IMPACT
COMMUNAL
ATTITUDINAL
BEHAVIORAL
TEMPORAL
Twitter, Facebook, Blogs, webpages
Frequency of talking about, links
included, user-created contents
Text Mining
Web Analytics
Survey, Interview
Number of festival acceptance
Number of awards
Number of professional reviews
Conversation, talking on the phone or
email, lectures, exchange of letters, etc.
Text Mining
Web Analytics
Archived Data
Survey, Interview
Text Mining
Web Analytics
Network Analysis
Number of viewers or visitors
Geography & demography: location,
age, gender, education, income
Number of inactive viewers
Number of opinion leaders
Advocacy
Text Mining
Web Analytics
Survey, Interview
Number of advocacy communities,
colleges, schools, or NGOs
Awareness
Stats, Text Mining
Web Analytics,
Network Analysis
Frequency of names, ideas, thoughts, or
concepts appeared in corpus
Report of increased awareness
Sentiment
Sentiment
Analysis
Frequency of positive, negative, neutral
sentiments of comments
Personal, critics, mass media, and
organizational responses
Reaction to calls for action
Engagement
Enactment
Connectedness
Capacity
Expansiveness
Centralization
Impact
Dynamics
Text Mining
Web Analytics
Network Analysis
Longitudinal
analysis
How well connected
How much & far disseminated
How centralized is the impact
The route of diffusion
Number of action pledges
alliance and allied action of organization
Discussion or decision by organizational,
governmental, international
policy/legislation makers
sponsorship of bills, adoption, donation,
funding, implementation, social
movement or intervention
Comparison b/w multiple time points
Duration of impact
Increase vs. decrease
Change vs. stability vs. reinforcement
Introduction or shifts of topics
Detection of social norm change
This is no
computational
fishing
expedition.
We have theory:
CoMTI
Framework
Diesner J, Pak S, Kim J,
Soltani K, Aleyasen A (2014)
Computational Assessment
of the Impact of Social
Justice Documentaries.
iConference, Berlin,
Gemany
4
Scientific Logic
Baseline
Ground truth Transcript
Content Reality/ Change
Social Structure
Meta Data
Theme
Content
Social Structure
Meta Data
Movie
Content
Social Structure
Meta Data
Content
Theme
Technology: ConText
http://context.lis.illinois.edu
5
Lessons
Learned
6
Thank you!
• Acknowledgement: This work is supported by the
FORD Foundation, grant 0125-6162. We are also
grateful to feedback and advice from Dr. Susie Pak from
St. John’s University, Orlando Bagwell, former director
of JustFilms at the Ford Foundation, and Joaquin
Alvarado from the Center for Investigative Reporting.
• For questions, comments, feedback, follow-up:
Jana Diesner
Email: [email protected]
Phone: (412) 519 7576
Web: http://people.lis.illinois.edu/~jdiesner
7