Download Field Experiments in Marketing

FieldExperimentsinMarketing
DuncanSimester
MITSloanSchoolofManagement
January2015
FieldExperimentsinMarketing
Marketingisadiversefieldthatdrawsfromaricharrayofdisciplinesandabroadassortmentof
empiricalandtheoreticalmethods.Oneofthosedisciplinesiseconomicsandoneofthemethodsused
toinvestigateeconomicquestionsisfieldexperiments.Thehistoryoffieldexperimentsinthemarketing
literatureissurprisinglylong.EarlyexamplesincludeCurhan(1974)andEskinandBaron(1977),who
varyprices,newspaperadvertising,anddisplayvariablesingrocerystores.Thischapterreviewsthe
recenthistoryoffieldexperimentsinmarketingbyidentifyingpaperspublishedinthelast20years
(between1995and2014).Wereporthowthenumberofpaperspublishedhasincreasedduringthis
period,andevaluatedifferentexplanationsforthisincrease.Wethengroupthepapersintofivetopics
andreviewthepapersbytopic.Thechapterconcludesbyreflectingonthedesignoffieldexperiments
usedinmarketing,andproposingtopicsforfutureresearch
1.
PapersthatReportFieldExperiments
Wefocusonthefiveleadingmarketingjournalsthatpublishpaperswithaneconomicsfocus.They
include:theJournalofMarketing(JM),theJournalofMarketingResearch(JMR),MarketingScience
(MktSci),QuantitativeMarketingandEconomics(QME),andthemarketingdepartmentofManagement
Science(MngSci).1Toidentifyrelevantpaperswefirsthadaresearchassistantreadeveryissueofthe
journalspublishedbetween1995and2014.WethensupplementedthisinitiallistwithaWebofScience
topicsearch.
Thisprocessyieldedover300papers.Wethenreadandscreenedthesepaperstogenerateafinal
sampleof61papers.2Inthisscreeningwerestrictedattentiontostudiesinwhichtheresponsemeasure
representedabehavioralresponseinthe“field”.Weexcludedstudiesinwhichtheresponsewasa
surveyorperceptualmeasure,suchascustomerevaluationsorpurchaseintentions.Wealsoexcluded
studieswheretheresponseswereobtainedinalaboratorysetting,includingstudieswheretheresponse
environmentwascreatedbytheresearchers.
Thescreeningalsorestrictedattentiontostudiesthatreportedexperiments,ratherthanjustanalysisof
historicaldata.Anexperimentincludesatleasttwoexperimentalconditions,withexogenousvariation
introducedbytheresearchers.Inmostcasesthisexogenousvariationresultsfromvariationacross
randomlyselectedcustomersamples.Inothercasesitinvolvesrotationoftreatmentsovertime,or
matchedpairsofproductsinanauctionsetting.
The61papersactuallyreportfindingsfromatotalof86fieldexperiments,with18(30%)ofthepapers
reportingmultiplefieldexperiments.Thisincludestwelvepapersthatreporttwoexperiments,five
papersthatreportthreeexperimentsandonepaperthatreportsfindingsfromfourexperiments.
1
ThetwomajormarketingjournalsnotincludedinthislistareJournalofConsumerResearch(JCR)andtheJournal
ofConsumerPsychology(JCP).Paperspublishedinthesetwojournalshaveastronglypsychologicalperspective.
2
Althoughwebelievethatthelistof61papersisextensive,itisunlikelytobecomplete.Werecognizethatwewill
haveoverlookedsomepapersthatreportfieldexperiments,andapologizetotheseauthors.
1|P a g e DateofPublication
Ourfirstinvestigationwastocomparethedatesthepaperswerepublished.Thepublicationdatesare
summarizedinFigure1.Papersreportingfieldexperimentswererarebetween1995and1999,withjust
threeexamplesidentified.Thefrequencyhassinceincreasedsharply;between2010and2014atotalof
37fieldexperimentpaperswereidentified.
Figure1:PublicationsbyYear
40
37
35
NumberofPapers
30
25
20
13
15
8
10
5
3
0
1995to1999
2000to2004
2005to2009
2010to2014
Thisfigurereportsthenumberofpaperspublishedby5-yearperiod.Thesamplesizeis
61(papers).
Therearelikelytobemultiplereasonsforthistrend.Oneexplanationisthatthefieldhasbecomealot
moreconcernedaboutendogeneitywheninterpretingresultsestimatedusinghistoricaldata.Before
2000,themarketingliteraturecontainednumerouseconometricpapersstudyinghistoricalsupermarket
scannerdata.3Manyoftheseearlypaperspaidlittleattentiontotheendogeneityoftheindependent
variables.Thishassincechanged.Thequantitativemarketingfieldnowpaysconsiderableattentionto
thelimitationsinherentininterpretingendogenousvariables.Thefocusonendogeneitycanpartlybe
attributedtoa1999paperthathighlightedtheselimitations(Villas-BoasandWiner1999).Thechange
alsocoincideswiththedevelopmentofstructuralmodelsinthenewempiricalIOliterature.Publication
inaleadingquantitativemarketingjournalnowrequiresthattheauthorsjustifythesourceoftheir
identification.Manypapersusingeconometricsmethodsnowusestructuralmodelstoaddressthis
concern.Fieldexperimentsprovideresearcherswithanalternativemechanismtoovercomethishurdle.
Theincreasedprominenceofconcernaboutendogeneitymayhavecontributedtothesharpincreasein
thenumberofpublishedfieldexperiments.
3
ThisliteraturetracesitsoriginstoGuadagniandLittle(1983),whichdemonstratedhowtoapplymultinomiallogit
toscannerdata.
2|P a g e Asecondexplanationforthegrowthinthenumberofpublishedfieldexperimentsisthatithasbecome
easiertoconductfieldexperiments.ThedevelopmentoftheInternetcoincideswiththegrowthinthe
numberofreportedexperiments.ItisnowpossibletoconductfieldexperimentsusingeBayandGoogle
AdWords,withoutrequiringactiveparticipationfromacooperatingcompany.Evenforstudiesthatdo
requirecooperation,thiscooperationisoftenmucheasiertoobtaininanInternetsettingbecausethe
costofconductingtheexperimentsisrelativelylow,andbecausemanyfirmsalreadyconducttheirown
experiments.Firmsthatconductexperimentsaspartoftheirownoperationshaveimplicitlyrevealed
thattheyunderstandthevalueoffieldexperiments,andarelikelytohaveinvestedininfrastructureto
supporttheirimplementation.4
Wecanevaluatethisexplanationbyinvestigatingwhethertheformatusedtoimplementexperimental
variationhaschangedovertime.Inparticular,wegroupedthestudiesintothreecategoriesaccording
tothetypeofexperimentalmanipulation:
Physical
in-personinteractionsinahome,ataworkplace,orinabricksandmortarretail
store(includingshelf-signageorproductassortments)
DirectMail
telephone,catalogorotherdirectmail
Internet
searchordisplayadvertising,eBay,email,Twitter,websitecontent,oranother
computerorInternetinteraction
InFigure2wereporthowuseofthesemanipulationshaschangedovertime.Thetotalsamplesizeis60
observations.5
Inthefiveyearsbetween2010and2014therewerenineteenpaperspublishedthatusedtheInternet
toimplementexperimentalvariation,representingmorethanhalfofthefieldexperimentpaper
publishedduringthisperiod.Nineteenpapersinthisfiveyearperiodcontrastswithjustthreepapersin
thepreviousfifteenyears.WeconcludethatitislikelytheInternetcontributedtothegrowthinfield
experimentpapersacrossthedataperiod.
However,itisnotablethatwealsoseegrowthinthenumberofpapersimplementingexperiments
throughphysicalmanipulationsin-home,attheworkplace,orinphysicalretailstores.Therewere
twelvepapersreportingphysicalmanipulationsbetween2010and2014,comparedtoatotalofjust
ninepapersinthepreviousfifteenyears.Itseemsthattheeaseofconductingexperimentsonthe
Internetisnotacompleteexplanationforthegrowthinthenumberofpublishedpapers.
4
WemightexpectthatifitiseasiertoconductfieldexperimentsinInternetsettingsthenpapersthatusethis
settingaremorelikelytoreportfindingsfrommultipleexperiments.Itturnsoutthatthereverseistrue;papers
reportingfindingsfromInternet-basedfieldexperimentsareactuallylesslikelytoreportmultiplestudies.
5
Infourpapersthedescriptionofthetreatmentwasinsufficienttoallowclassification.Moreover,threepapers
eitherreportedmultiplestudiesusingdifferenttypesofvariation,oruseddifferenttypesofvariationacross
differenttreatmentswithinthesamestudy.Thesethreepaperswerecodedasappearinginmultiplecategories.
3|P a g e Figure2:PublicationsbyTypeofExperimentalManipulation
20
15
10
5
0
1995to1999
2000to2004
Physical
2005to2009
DirectMail
2010to2014
Internet
Thisfigurereportsthenumberofpaperspublishedusingeachtypeofexperimental
manipulationineach5-yearperiod.Thesamplesizeis60.
Athirdexplanationisthatthetopmarketingjournalsarepublishingmorepapers(notjustmorefield
experimentpapers).In2006MktSciincreasedfromfourissuesayeartosixissuesayear,andin2008JM
andJMRmadethesametransition.Inaddition,QMEbeganpublicationin2003.Theresultisanincrease
inthetotalnumberofpublishedpapers.Toinvestigatetheextenttowhichthisexplainstheincreasein
thefrequencyoffieldexperimentpapers,wecountedthetotalnumberofpaperspublishedacrossthe
20-yearperiod(1995to2014)andreportahistogramoftheproportionpublishedineach5-yearperiod.
Wedothisseparatelyforallpapers,andforjustthosepapersthatreportfieldexperiments.6The
findingsarereportedinFigure3.
Whiletherehasbeengrowthinthetotalnumberofpaperspublished,thisgrowthisalotsmallerthan
thegrowthinthenumberoffieldexperimentpapers.Whencountingallofthepaperspublished
between1995and2014,wefindthat31%werepublishedinthelast5years(2010-2014).Incontrast,
64%ofallfieldexperimentspaperswerepublishedinthatperiod.
Weconcludethatthesharpgrowthinthenumberofmarketingpapersreportingfieldexperimentscan
beattributedtoseveralfactors.Thefielditselfhasgrownandsotherehasbeengrowthinthenumber
ofpaperspublished,includingfieldexperimentandnon-fieldexperimentpapers.Second,theeaseof
conductingfieldexperimentsovertheInternetmayalsohavecontributedtothephenomenon.Finally,
themarketingfieldnowpaysmoreattentiontotheendogeneityofindependentvariables.Field
experimentsareaneffectivemechanismforresolvingconfoundsduetoendogeneity.
6
ThetotalnumberofpapersinMktSci,JMandJMRarecalculatedusingentriesintheWebofScience.We
excludedpaperspublishedinMngSciastheWebofSciencedoesnotidentifywhichdepartmenteditoraccepted
thepaper.UnfortunatelytheWebofSciencedoesnotindexthefirstfewissuesofQME,andsoforthisjournalwe
manuallycountedthenumberofpaperspublishedineachissue.
4|P a g e Figure3:HistogramoftheNumberofPapersPublishedinEach5-YearPeriod
AllPapersComparedtoFieldExperimentPapers
%ofallpapersacrossall20years
80%
61%
60%
40%
20%
31%
19%
18%
31%
21%
13%
0%
5%
AllPapers
FieldExperimentPapers
Thisfigurereportsahistogramofhowmanypaperswerepublishedineach5-year
period.Weseparatelyreportthefindingsforallpapers,andforjustthosepapersthat
reportfindingsfromafieldexperiment.Thesamplesizeis3,250forallpapers,and61for
thefieldexperimentpapers.Thepercentageswithineachcurveaddto100%.
ChoiceofTopics
Inthefigurebelowwecategorizethe61papersintogeneraltopicareas.Thiscategorizationrevealsthat
thepapersaredominatedbytwotopics:pricingandadvertising.Furtherinvestigationalsorevealsclear
differencesinthedistributionoftopicsovertime.Priorto2010,halfofthepapers(12of24)addressed
pricingtopics,withonlytwopapersinvestigatingadvertisingissues.However,thisdominancehas
reversed,withmorepapersinthelastfiveyearsstudyingadvertisingissues(13)thanpricingissues(11).
Understandably,therecentfocusonadvertisinghasbeendominatedbyresearchquestionsrelatedto
Internetadvertising.ManyofthesepapersstudycharacteristicsofInternetadvertisingthatsimplydid
notexistatthestartofthedatasample.Forexample,dynamicretargetingofadvertising(targeting
advertisementsusingindividualbrowsingbehaviorsonotherwebsites)wasnotpossibleearlierinthe
dataperiod.Similarly,personalizingadvertisingusinginformationpostedonFacebookpagesisa
relativelyrecentadvertisingtechnique.
Inthesectionsthatfollowwebrieflysurveythepapersoneachofthesetopics.Wealsosummarizeallof
thepapers,groupedbytopic,inatableasanAppendixtothischapter.
5|P a g e Figure4:ChoiceofTopics
30
23
20
15
11
8
10
4
0
Pricing
Advertising
Model
Product
Validation Development
Other
Thisfigurereportsthenumberofpublishedpapersbytopic.Thesamplesizeis61
papers.
2.
PricingTopics
The23papersinvestigatingpricingissuescanbegroupedintofoursub-topics:
1.
2.
3.
4.
Whototargetwithdiscounts?
Whattypesofdiscountsaremosteffective?
Priceasasignal
Multi-parttariffsandotherpricingschemes
Theeffectivenessofdiscountshasbeenafavoriteresearchtopicinthemarketingfield.Thisinpart
reflectsboththepervasivenessofdiscounts,andrecognitionthattheycanhavetremendouspositive
andnegativeimpactsonunitsalesandfirmprofits.Forexample,Ailawada,Harlam,CésarandTrounce
(2007)studytheunitsalesandprofitimpactofeverydiscountofferedbythenationalpharmacychain
CVSin2003.Theyreportthatmorethan50%ofthepromotionsareunprofitablebecausethelossof
marginisnotfullyoffsetbytheincrementalsales.Usingaverylarge-scale13-weekfieldtest,inwhich
theysimplywithheldpromotionsin15productcategoriesin400stores,theydemonstratedthatCVS
couldimproveitsprofitsbyover$50million.
WhotoTargetwithDiscounts?
Threefieldexperimentscontributetoourunderstandingofwhoshouldreceivediscounts.Dholakia
(2006)showsthatsendinga$5-offdiscountcoupontocustomerswhopreviouslypaidfullpricecan
actuallyleadtolessdemand.Thisstudywasconductedusingrandomlyselectedcustomersofalarge
6|P a g e automobileservicingfirm.7AndersonandSimester(2001a)reportthatofferingextendedpayment
termstocustomersofaprestigiousjewelrycatalogcanlowerdemand.Theauthorsinterpretthiseffect
asanadversequalitysignal,withtheinstallmentbillingsignalingthattheproductsaresuitablefor
customerswhoaremoresensitivetopricethanquality.Inalaterstudywithadifferentcatalogretailer,
thesameresearchteamreportthatdeepdiscountshavemorepositivelong-termexternalitieswhen
targetedatnewcustomerscomparedtoexistingcustomers(AndersonandSimester2004).Forthe
existingcustomersthelong-termeffectswerenegative;thedeepdiscountsresultedinthesecustomers
acceleratingtheirpurchasesandbecomingmorepricesensitive.Incontrast,deeperdiscountsincreased
futurepurchasesbynewcustomers,apparentlyduetomorefavorableexpectationsaboutfutureprices.
Collectivelythesestudiessuggestthatdiscountsaremosteffectivewhentargetedatcustomerswho
are:newer,whohavepaidlowerpricesinthepast,andwhoarerelativelymoreconcernedaboutprice
thanquality.
WhichDiscountsareMostEffective?
Fieldexperimentshavebeenusedtoaddressnotjustwhoshouldreceivediscounts,butalsowhatform
thediscountsshouldtake.Collectivelytheliteraturenowprovidesarichanswertothisquestion.Inthe
oldestpaperinoursample,HochandDhar(1996)reportthefindingsfromtwofieldexperiments
conductedat86supermarketstoresintheDominick’sFinerFoodssupermarketchain.8Theyshowed
thatcouponsleadtoa35%largerincreaseinunitsalesthananequivalentdiscountintheshelf-price.
Moreover,becauseredemptionofcouponsisincomplete,thisleadstoa108%largerincreaseinprofits.
Intwolarge-scalefieldexperimentsconductedatthe(apparently)same86supermarkets,Wansink,
KentandHoch(1998)illustratetwowaysthatretailerscanincreasethenumberofunitsthatcustomers
purchase.Imposingaquantitylimitof12unitspercustomersledtocustomerspurchasingtwiceas
manyunits(perbuyer)comparedtoaquantitylimitof4units.Framingdiscountsusingmultipleunit
promotions(e.g.2for$1.50)insteadofsingleunitpromotions(75-centseach)alsoincreasedthe
numberofunitsthatcustomerspurchased.Bothoftheseeffectsareinterpretedasexamplesof
customersanchoringonretailer-providedcuestodecidehowmanyunitstopurchase.Inanother
exampleofframing,Chen,Marmorstein,TsirosandRao(2012)showthatdescribingadiscountas“50%
morefree”ismoreeffectivethan“33%offtheregularprice”,eventhoughbothareeconomically
equivalent.Theirstudyismotivatedbyrelatedevidencethatcustomerstendtofocusonthe%
magnitudeofthediscount,andneglectthebaseoverwhichthepercentageiscalculated.
RamanathanandDhar(2010)presentfindingssuggestingthattheimpactofdifferenttypesof
promotionsmayvaryaccordingtothecustomer’spsychologicalorientation.Customersenteringa
Chicagogrocerystorewereprimedtothinkofeitherenjoyingthemselvesoractingsensibly.Those
primedtoenjoythemselvespurchasedmoreitemswhentheyreceivedcouponswithlongerexpiration
dates,andwhenthecouponswereframedas“Get$xOff”ratherthan“Save$x”.Incontrast,customers
7
RelatedfindingsfromtwofieldexperimentsarereportedbyAndersonandSimester(2010),whichispublishedin
aneconomicsjournal.Customersofapublishingcatalogandanapparelcatalogwerelesslikelytoplacea
subsequentorderiftheyreceivedacatalogcontaininglowerpricesthantheyhadrecentlypaidforthesameitem.
8
ThisisthesameretailerthatprovidedthedataforthenowwidelyusedDominick’sscandatapanel.
7|P a g e primedtoactsensiblypurchasedmoreitemswhentheyreceivedcouponswithshorterexpirationdates
andwhenthecouponsread“Save$x”.
Therehavebeenseveralstudiesinvestigatingtheimpactofframingdiscountsas“freesamples”or“free
gifts”.BawaandShoemaker(2004)usetwofieldexperimentstodisentanglethreeeffectsofoffering
freesamples:demandacceleration(temporalsubstitution),cannibalization,anddemandexpansion.
Theyshowthatunlikecoupons,freesamplescangeneratepositivelong-termeffectsthatcanendure
forasmuchas12monthsaftertheevent.Inalongitudinalstudyinthefinancialservicessector,Haisley
andLoewenstein(2011)investigatedhowofferingcustomersunexpectedgiftsaffectedtheir
subsequentdepositbalances.Theyfoundthatanincreasingsequenceofgifts($35then$100)hada
morepositiveimpactonsubsequentbalancesthanadecreasingsequence($100then$35).They
describedthisasan“aversiontodeterioration”.ShuandGneezy(2010)studyhowexpirationdateson
giftcertificatesaffectredemptionrates.Participantswhoreceivedgiftcertificateswithalonger
expirationdatahadmorefavorableperceptionsofthefirm,butwereactuallylesslikelytoredeemthe
certificates.Theyattributethisprofitablecoincidencetoprocrastination,andsupportthisclaimusinga
follow-upsurvey.LaranandTsiros(2013)investigatehowuncertaintyaboutwhichfreegiftwillbe
offeredinfluencetheeffectivenessofthesepromotions.Ifcustomerswereprimedtothinkabouttheir
decisions,theyrespondedmorefavorablyiftheyknewwhatfreegifttheywouldreceive.Incontrast,if
theywereprimedtomakemoreemotionaldecisionsthentheuncertaingiftwasmoreeffective.
Wealsohighlightthreeparticularlyinnovativestudies.Hui,Inman,HuangandSuher(2013)investigate
theimpactofsendingdiscountsthroughmobilephonetechnology.Theyshowthatsendingcouponsfor
productsthatarelocatedfurtherawayfromthecustomers’plannedin-storeshoppingpathsare
effectiveatincreasingunplannedpurchases.TsirosandHardesty(2010)studyhowtomosteffectively
removediscounts.Theyshowthatphasingoutadiscountgraduallyismoreeffectiveatraisingunitsales
thanremovingthediscountallatonce.Theyattributethisinparttohighercustomerexpectationsabout
futurepricelevels.Gaurav,ColeandTobacman(2011)studytheadoptionofrainfallinsuranceinrural
Indianvillages.Theinsuranceisarelativelysophisticatedfinancialproduct,andtheyfindthattraining
customersabouttheproductsisgenerallymoreeffectivethanofferingotherformsofmarketing
promotions.Theexceptionisa“MoneyBack”guaranteeifthefarmersneverhavecausetoclaimonthe
insurance,whichworksalmostaseffectivelyasthetrainingprograms.
PriceasaSignal
Athirdgroupofpricingstudiesfocusesonthesignalingroleofprices.Usingdatafromanapparel
catalog,AndersonandSimester(2001b)highlighttheimportanceofrationinghowoftensalesignsare
used.Theymotivatethisstudyasatestofanearlierpaperthatarguesthatsalesignsserveacredible
signalingrole,enablingpoorlyinformedcustomerstoevaluatewhichpricesarelowrelativelytoother
pricesinthemarket(AndersonandSimester1998).Akeyfeatureofthatmodelisthatthesalesignsare
self-regulating;anyonesignbecomeslesscrediblethemoreoftenitisused.Inarelatedstudy,
AndersonandSimester(2003)measuretheimpactof9-digitpriceendings(e.g.$1.99or$49).Usinga
seriesofthreefieldexperimentsconductedattwodifferentwomen’sapparelcatalogs,theyshowthat
$9priceendingsincreasedemand,andthatthisdemandincreaseisstrongeronneweritemsthanfor
establisheditems.However,theeffectappearstobeweakenedwhenthe$9dollarendingis
8|P a g e accompaniedbya“sale”sign.Theyinterpretthesefindingsasevidencethatthepriceendingsservea
similarsignalingroleto“sale”signs,revealingtocustomerswhichitemsarediscountedcomparedto
otherpricesinthemarket.
Gneezy,GneezyandLauga(2014)investigatetheinformationthatpricessignalaboutquality.When
customersseehighpricestheyanticipatehigherquality,andsoreactmoreadverselyifqualityislow.As
aresult,customersevaluatealowqualityproductwithahighpricemorenegativelythanalowquality
productwithalowprice.TheyestablishthisresultusingafieldexperimentatasmallCalifornia
vineyard.
Twootherpapersinthissignalingstreamhavealreadybeendiscussedascontributionstothe
discountingliterature.RecallthatAndersonandSimester’s(2001a)evidencethatextendedpayment
termsloweredsalesataprestigiousjewelrycatalogwasinterpretedasevidencethattargetingprice
sensitivecustomerscanunderminethecredibilityofaretailer’squalityclaim.Similarly,theevidence
thatdeeperdiscountstofirst-timecustomerscanmakethesecustomersmoreloyalalsohadasignaling
interpretation(AndersonandSimester2004),althoughinthiscasethesignalwasaboutfutureprices
ratherthanquality.
Multi-PartTariffsandOtherPricingSchemes
Thefourthstreamofpricing-relatedfieldexperimentsaddressesmulti-periodandmulti-parttariffs
togetherwithotherinnovativepricingschemes.LambrechtandTucker(2012)reportoneofthefew
fieldexperimentsconductedinabusiness-to-businesssetting.Theycollaboratewithawebhosting
providerintheUnitedKingdomtoinvestigatetheimpactofvaryingboththemonthlypriceandthe
framingof“hasslecosts”inthefirstperiodofamulti-periodcontract.Ratherthanevaluatingtheentire
contractterm,theyshowthatcustomersinthismarketevaluatetheoutcomeineachdistinctperiod.As
aresult,theyshowthatifcustomersincurhasslecostsinthefirstperiod,thenitismoreeffectiveto
lowerthefirstperiodpriceanddefersomeofthisrevenueuntillaterperiods.
AndersonandSimester(2008)showthatpricedifferentiationcanlowerdemandifitisperceivedtobe
unfair.Theirfieldexperimentwasconductedinawomen’sapparelcatalogthatsellsplus-sizes.Because
thecostofproducingverylargegarmentsishigherthanthecostofproducingsmallergarments,the
retailersoughttochargehigherpricesonitslargestsizes.Theresultsrevealedanasymmetry;therewas
nochangeindemandatthesmallersizes,butdemandfellsharplyforthelargersizes(evenafter
controllingfortheabsolutepriceitself).Theyinterpretthesefindingsasevidencethatprice
differentiationcanlowerdemandifitisperceivedtobeunfair.However,customersactuallyhaveto
experiencetheunfairness;itisnotsufficienttoseeothercustomerstreatedunfairly.
Twopapersfocusonmulti-parttariffschemes.Danaher(2002)usestheexogenousvariationintroduced
throughafieldexperimentinthetelecommunicationsmarkettodevelopanoptimalpricingmodelfora
2-parttariff.Theschemeincludesbothamonthlyaccessfee,togetherwithaper-minuteusageratefor
cellularphoneservice.Theresultshighlighttheimportanceofaccountingforbothusageandcustomer
retention.Solelyfocusingonusageandignoringattritionwillsubstantiallyunder-estimatethesensitivity
ofrevenuetoprice.Yao,Mela,ChiangandChen(2012)alsostudycellularphonepricing,althoughin
theircasetheyuseafieldexperimenttorecoverestimatesofcustomerdiscountrates(ratherthan
9|P a g e revenuepricesensitivity).Customers’planchoicesrevealmuchfasterconsumerdiscountingoffuture
periodsthanwenormallyassume.
Perhapsthemostunusualpricingschemerepresentedintheliteratureisa“pay-what-you-want”
(PWYW)scheme.Kim,NatterandSpannreportthreefieldexperimentsconductedinGerman
restaurantsandamovietheatre.ThefirmsrotatedPWYWschemesandtheirstandardpricingschemes
ondifferentdays.Ingeneral,customerschosetopaylowerpricesthanthepricesthefirmsregularly
charged.However,thiswasnotalwaysthecase.Customersinoneoftheexperimentschosetopayan
averageof$1.94forahotbeverage,comparedtotheregularpriceof$1.75.Inanotherexperiment,
customersonaveragechosetopaylessforabuffetlunch($6.44)thantheregularprice($7.99),butthe
PWYWschemeattractedenoughextrademandtogeneratesignificantlyhigherdailyrevenue.
3.
AdvertisingTopics
Thefifteenpapersthatfocusonadvertisingrelatedissuescanbegroupedintothreebroadsub-topics:
1. Doesadvertisingimpactpurchasing?
2. Whichadvertisingmessagesaremosteffective?
3. Optimizingpaidsearchadvertising
IsthereanImpactonPurchasing?
Fourpapersinvestigatewhetheradvertisingcanimpactpurchasing,includingtwopapersthatreacha
relativelynegativeconclusion.LewisandReiley(2014)studywhetherdisplayadvertisementsatYahoo!
causedofflinepurchasesatadepartmentstore.Despiteacontrolledrandomizedenvironmentanda
samplesizeof1.6millioncustomers,theyareonlyjustabletoestablishastatisticallysignificanteffect.
LambrechtandTucker(2013)studytheeffectsof“dynamicretargeting”ofadvertisements.Dynamic
retargetingdescribesthewidelyusedpracticeofusingbrowsinghistoryfromotherwebsitestoselect
whichadvertisingcontenttodisplay.Theyfindthatthispracticedoesnotincreaseadvertising
effectiveness.Theexceptioniswhenthebrowsinghistoryindicatesthatacustomerisrelativelycloseto
makingapurchase.Itisonlythenthatdynamicretargetingiseffective.
Anotherpaperwithakeyfindingthatiseffectivelyanullresultfocusesontheimpactofdigitalvideo
recorders(DVRs)onadvertisingeffectiveness.TheintroductionofDVRsledtoconcernthattelevision
advertisingwouldbecomelesseffectivebecausecustomerscouldfilteroutadvertising.Inthisvery
large-scalescalestudy,theauthorsfoundnoevidencethatthiswasthecase.Insteadtheauthors
concludedthatfilteringoutadvertisementsoccursrelativelyinfrequently.Incontrasttothetwo
previouspapers,thiscouldbeconsideredarelativelypositivepaperabouttheimpactofadvertising.It
suggeststhatthewidespreadconsumeradoptionofDVRtechnologywillnotdiminishtheimpactofTV
advertising.
Thefourthpaperinthissequenceinvestigatestheimpactofcompetitiveadvertising.Competingcatalog
retailerssharethenamesandaddressesoftheirbestcustomersonareciprocalbasisinordertolower
thecostsofprospectingfornewcustomers.Usingarandomizedfieldexperiment,Andersonand
Simester(2013)measurehowallowingcompetitorstotargetyourcustomerswithcompetingcatalogs
10|P a g e impactsafirm’sownsalestothosecustomers.Althoughwemightexpectanegativeeffectdueto
substitution,theyactuallyfindthatformanycustomersthecompetitors’advertisingincreased
purchasesfromtheoriginalfirm.Thispositiveeffectwasparticularlystronginproductcategoriesin
whichcustomerslearnproductsizesthatarefirm-specific(suchasfootwear).Thisleadstheauthorsto
attributetheirfindingstotheimportanceofproductstandards,customerlearningandswitchingcosts.
WhichMessagesAreMostEffective?
Anadditionalninepapersfocusonimprovingmessagingtoincreaseadvertisingresponserates.They
includeseveralpapersthataddresstargetingorpersonalizationofadvertisingmessages.Forexample,
Tucker(2014)reportsafieldexperimentconductedonFacebookthatstudieshowcustomersreactto
advertisementsthatarepersonalizedbasedonacustomer’spostedpersonalinformation(seealso
LambrechtandTucker2013,whichisdiscussedabove).Fortuitously,theexperimentcoincidedwitha
widelypublicizedchangeinFacebook’sprivacypolicies,whichgaveusersmorecontrolovertheir
personalizationsettings.Thefindingsrevealthatgivingcustomerstheoptionofcontrollingtheir
personalizationsettingsgreatlyimprovedtheperformanceoftargetedadvertisements.Schumann,von
WangenheimandGroene(2014)studyhowtomitigateadversecustomerreactionstotargetedInternet
advertising.Theyshowthatreciprocityappeals(“Ourserviceisfreeofchargetoyou–targeted
advertisinghelpsusfundit”)aremoreeffectivethanrelevanceclaims(“Youwillseemoreinteresting
andlessirrelevantadvertisementsinthefuture”).
Threestudiesinvestigatehowmessagingshouldbecustomizedtodifferentcustomersegmentsusing
dimensionsotherthanpriorbrowsingbehavior.BergerandSchwarz(2011)studymessagesthatprompt
customerstorecommendproductstoothercustomers(word-of-mouth).Theyshowthatmessagesthat
linkabrandtoaproductcuearemoreeffectivewhencustomersdonotalreadyhavestrong
associationsbetweenthebrandandthecue.UsingaGoogleAdWordsfieldexperimentconductedin
Israel,Kronrod,GrinsteinandWathieu(2012)demonstratethattheuseofassertivemessages(“You
mustsavetheMediterranean”)aremoreeffectivewhencustomersalreadybelieveinthecause,butare
lesseffectiveformoregeneralcauses,forwhichcustomers’preferencesareweaker.
BeyondKronrod,GrinsteinandWathieu(2012),twootherstudiesinvestigatehowtodesignmessaging
thatencouragescustomerstoengageinmoreenvironmentallyfriendlybehavior.Inanambitiousstudy,
WhiteandSimpson(2013)collaboratewithalargemetropolitancitytoencourageresidentstoleave
theirgrassclippingsontheground,ratherthandisposeofthemthroughthemunicipalwastesystem
(whichsendstheclippingstolandfills).Usinghangersplacedonthefrontdoorsofresident’shome,they
testedsixdifferentmessagingconditions,togetherwithaControlcondition.Theythenmeasuredthe
reductioninwaste.Varyingthetypeofappealafterprimingafocusoneitherindividualorsocial
benefitshadasignificanteffectonresidents’behavior.Spangenberg,Sprott,GrohmanandSmith(2003)
reporttwostudies.Inthefirststudytheyinvestigatehowmessagesinoutdooradvertisingmedia
(billboards)canincreaserecycling.Inthesecondstudytheyvarythecontentofmessagesinsertedina
healthclub’smonthlynewslettersandbillingstatement,withthegoalofincreasingmembervisitstothe
club.Theyshowthatframinganappealasaself-predictionincreasesitseffectiveness.Specifically,inthe
fitnessclubstudy,themessage“Fitnessguilt?”wasmoreeffectiveatincreasingmembervisitsthanthe
alternativetreatment“Workoutat[fitnessclubname]”.
11|P a g e TwootherexamplesofmessagingtestswerereportedbyTuckerandZhang(2010and2011).Bothof
thesestudiesinvestigatehowreportingparticipationinatwo-sidednetworkincreasesparticipationin
thenetwork.Thefirstexample(TuckerandZhang2010)isanotherrareexampleofastudyina
business-to-businessmarket.Thesettingwasawebsitethatresemblescraigslist.org,andtheoutcome
measurewasthenumberofsellerswhochosetolistonthesite.Thefieldexperimentrandomized
whethertodisplaythenumberofbuysand/orsellersand(ifso)thenumberofbuyersand/orsellersto
claim.Theyfindthatalargenumberofsellerscandeteradditionallistings,unlessonlythenumberof
sellersisrevealed(notthenumberofbuyers).Displayingmanybuyerswillattractmoresellersaslongas
thenumberofsellersisalsorevealed.Theyconcludethatmarketswithmorecompetitorscanappear
moreattractivetoentrants,astheremustbesufficientdemandtoattractsomanycompetitors(a
positivenetworkexternality).Thesecondstudy,TuckerandZhang(2011),usesdatafromawebsitethat
listsweddingservicevendors.Theyrandomizewhetherthewebsiterevealsthepreviousnumberof
clicksonavendor.Wemightexpectthatrevealingthisinformationwouldincreasethepopularityofthe
vendorswiththemostclicks.Instead,theyshowthatthesamelevelofhistoricalclicksresultsinalarger
liftforbrandswithanichemarketposition,comparedtoabrandwithamainstreamposition.
Customersappeartoinferthatthesebrandsmustprovidehighqualityinordertoovercometheir
narrowreach.
Thefinalpaperinvestigatingtheimpactofadvertisingmessagesincludesapairofstudiesconducted
usingmatchedpairsofmusicCDauctionsoneBay.DholakiaandSimonson(2005)studytheimpactof
messagingthatexplicitlyrecommendsthatpotentialbidders“comparethepriceofthisCDwiththe
pricesofsimilarCDslistednexttothisone”.Theyfindthatthesemessagesleadtomorecautious
biddingbehavior.Thewinningbidderstendedtosubmitlaterbids,submitfewerbids,andavoid
participatinginsimultaneousauctions.Theyinterpretthefindingsasevidencethatmakingcompeting
optionsmoresalientmakescustomersmoreriskaversebecauseopportunitycostsbecomemore
prominent.
OptimizingPaidSearchAdvertising
Twopapersbydifferentteamsinvestigatehowtooptimizepaidsearchadvertising.YangandGhose
(2010)beginbyusinganhistoricalpaneldatasettoinvestigatehowtheresponsetopaidsearch
advertisingisaffectedbythepresenceoforganic(unpaid)searchresults.Theythentesttherobustness
ofthemodelpredictionsusingan8-weekfieldexperimentonGoogle.Theyconfirmedthatwhenpaid
searchispresent,sothatcustomerscanseebothpaidandorganicsearchresults,thecombinedclickthroughrate(CTR)was5.1%highercomparedtowhentherewasnopaidsearchpresent.The
conversionrate(probabilityofapurchase)alsoincreasesby11.7%.
Agarwal,HosanagarandSmith(2011)investigatetheimportanceofpositioninpaidsearchadvertising.
Specifically,theyaskwhetherbeingthefirstsearchresult,secondresult,oralowerresult,impactsboth
click-throughrates(CTR)andconversion(purchases).Aswemightexpect,theclosertothefirstposition,
thehighertheCTR.However,theoppositeistrueforconversionrates.Appearinglowerintherankof
searchresultscanactuallyincreaseconversionratesbecauseconversionisconditionalonclicking.Asa
result,onlythemostmotivatedcustomersclickonlowerresults,andthesearethecustomersthatare
12|P a g e morelikelytoconvert.Givenadvertisersgenerallypayforclicksirrespectiveofconversions,itmaybe
optimalformanyadvertiserstopreferlowerpositionsinthesearchorder.
4.
Product-RelatedTopics
Weidentifiedsevenpapersthatpresentfieldexperimentsaddressingproduct-relatedtopics.This
includesfourstudiesfocusedonmarketresearchandproductdevelopmentissues,andthreepapers
focusedonproductsizes.
MarketResearchandProductDevelopment
Twostudiesinvestigatetherolethatmeremeasurementcanhaveoncustomerpurchasingbehavior.
Chandon,MorwitzandReinartz(2005)asked251customersofanonlinegrocerabouttheirpurchase
intentions.Asecondrandomlyselectedsetof140consumersservedasthecontrol(andwerenotasked
anyquestions).Thestudyrevealedthatthecustomerswhoweresurveyedweresignificantlymorelikely
tomakeasubsequentpurchase,andweresignificantlymoreprofitablefortheretailer.Twoyearslater,
adifferentresearchteamconductedarelatedstudyintheautomotiveservicesindustry(Borle,
Dholakia,SinghandWestbrook2007).Atelephonesurveywasadministeredto3,773randomlyselected
customers.Subsequentservicevisitsbythesecustomerswerethencomparedwitharandomlyselected
controlsampleof1,227customers.Thiscomparisonrevealedthatparticipatinginthesurveyincreased
purchasesandmadecustomersmoreresponsivetofirmpromotions.
Theothertwostudiesonthistopicusefieldexperimentstovalidatenewmarketresearchmethods.
Urban,Hauser,Qualls,Weinberg,BohlmannandChicos(1997)validateamethodforconducting
researchveryearlyintheproductdevelopmentprocess.Forecastsofnewproductsalesaremore
valuabletheearliertheyareavailable.Theauthorsinvestigatethepossibilityofusingmultimedia
representationsofpotentialnewproductstoprovideearlierandlesscostlyforecasts.Theyconducttwo
fieldexperiments,whichconfirmthatmultimediacomputerinteractionscanbothrealisticallyportray
customers’informationsources,andyieldforecaststhatarenotsignificantlydifferentfromtraditional
methods.Neslin,Novak,BakerandHoffman(2009)proposeamodelthatisdesignedtomaximize
responseratestoonlinemarketresearchpanels.Themodelisdynamic,optimizingoveradiscrete
numberofstudiesinafiniteperiod,wherethisfiniteperiodcanbeextendedusingarollinghorizon.
Theycomparetheirmodelagainstthecurrentmanagerialheuristic,andreportsignificantlyhigher
responserates.
ProductSizesandBundling
Consumers’assessmentsofrelativepackagevolumesareoftenbiasedwhenpackageshavedifferent
shapes.Krider,RaghubirandKrishna(2001)investigatethisbiasbycomparinghowcustomersreactto
productpackagesthathavethesamevolumebutdifferentshapes.Inauniversitycafeteriathatsells
creamcheesetocomplementbagels,theauthorscomparedemandfora¾ozroundtubofcream
cheeseanda¾ozsquaretubofthesamecreamcheese.Customersweresignificantlymorelikelyto
purchase2tubsofcreamcheesefortheirbagelondaysthatonlytheroundtubsofcreamcheesewere
available.Thisfindingisconsistentwithotherevidenceintheirpaperthatcustomersperceiveround
containerstobesmallerthanrectangularcontainersofthesamevolume.
13|P a g e LeszczycandHaubl(2010)reportfindingsfromthreefieldteststhatinvestigatetheprofitabilityof
productbundling.UsingaseriesofeBayauctionsofcollectablepostagestamps,theycomparethe
profitabilityofbundlingrelatedandunrelateditems.Theirfindingsrevealthatbundlingsubstitutesor
unrelateditemsinanauctionislessprofitablethansellingthemseparately,butbundlingismore
profitablewhentheitemsarecomplementary.Athirdstudyrelatedtoproductbundlinghasalready
beendiscussedasanexampleofapricing-relatedpaper.Recallthatinanexperimentconductedat86
supermarketstores,Wansink,KentandHoch(1998)comparedtheeffectivenessofmultipleunit
promotions(e.g.2for$1.50)versussingleunitpromotions(75-centseach).Themultipleunit
promotionsincreasedthenumberofunitsthatcustomerspurchasedbyanaverageof32%.
5.
ModelValidation
Themarketingfieldhasalongtraditionofdevelopingmodelsthatoptimizemarketingdecisions.
Traditionallythesemodelsarevalidatedbymeasuringgoodnessoffit,eitherin-sampleorwithholdout
samples.Alimitationofthisvalidationisthatitgenerallyreliesonassumptionsinherentinthemodels,
andsoerrorsintheassumptionsarealsointroducedtothevalidationprocess.
Becauseoftheselimitations,researchershavebegunusingfieldexperimentsasameansofvalidating
marketingmodels.Fieldexperimentsprovideanalmostidealvalidationsetting;differentpolicescanbe
implementedintreatmentandcontrolsettingsandtheiroutcomescompared.Thisprovidesa“modelfree”basisforvalidation,togetherwithacomprehensivetestofalloftheassumptionsinthemodel.
Forexample,Simester,SunandTsitsiklis(2006)proposeamodelfordynamicallyoptimizingcatalog(and
otherdirectmarketing)mailingdecisions.Catalogfirmsregularlysendcatalogstocustomers,andmust
decidewhoshouldreceiveeachcatalog.Traditionallythesedecisionshavebeenmademyopically;the
companiessendcatalogstocustomerswhoaremostlikelytorespondtothatcatalog.Themodel
proposedinthispaperoptimizesasequenceofmailingdecisionsoveraninfinitehorizon.Themodel
beginsbyproposingamethodfordividingcustomersintodiscreteMarkovstatesusingasetofvariables
describingeachcustomer’spurchasingandmailinghistories.Transitionprobabilitiesandrewardsare
thenestimatedforeachstrategy(mailornotmail)ineachstatespace.Finally,astandardpolicyiterationalgorithmisusedtocalculatetheoptimalpolicyineachstate.Theauthorsthentesttheir
proposedmethodbyvaryingcatalogmailingdecisionsfor60,000customersofanapparelcatalogovera
periodofsixmonthsincluding12catalogmailingdates.Thefindingsrevealedthatthemodelperformed
wellforlow-valuedandmoderately-valuedcustomers.However,duringtheinitialmonthsofthe
experiment,theresultswerelessfavorableforthemostvaluablecustomersinthesample.Further
investigationrevealedanexplanationforthepoorinitialoutcomeforthesecustomers:inthetraining
datathereweretoofewoccasionsinwhichthefirmhadnotmailedtothesecustomerstoprovidea
reliableestimateoftheoutcome.
14|P a g e Thesecomparisonsnotonlyvalidatetheproposedmodels,buthavealsoprovedvaluableasasourceof
insightsabouttheunderlyingphenomenon.9
InadditiontoSimester,SunandTsitsiklis(2006),weidentifiedsevenotherpaperspublishedinthelast
twentyyearsthatemployfieldexperimentstovalidateoptimizationmodels.Theseincludetwopricing
models,threeadvertisingmodelsandtwonewmarketresearchmethods.Allbutoneofthesepapers
waspublishedin2006orlater,suggestinggrowinginterestinthisapproach.Althoughsomeofthe
papersinvolverelativelysmallnumbersofparticipants(seeforexampleUrbanetal.1997;andBelloni,
Lovett,BouldingandStaelin2012),othersincludelargenumbersofparticipantsandextended
treatmentperiods.Forexample,Mantrala,Seetharaman,Kaul,GopalakrishnaandStam(2006)compare
theoutcomewhenimplementinganoptimalpricingmodelin200experimentalstoresoveraperiodof8
weeks.
6.
OtherTopics
Oursearchrevealedanadditional10studiesthatdonotfiteasilywithinthepreviousfourtopics.
Notably,thesepapersareallrelativelyrecent,withthefirstpublishedin2008,andeightofthem
publishedinthelastfiveyears.Wehavegroupedthesepapersintofivetopics:
1.
2.
3.
4.
5.
Salesforceoptimization
Word-of-mouthandreferrals
Participationinonlinecommunities
Encouragingpositivebehaviors
Othertopics
SalesForceOptimization
Kumar,VenkatesanandReinartz(2008)reportfindingsfromtwolarge-scalefieldexperiments.One
experimentwasconductedwithamultinationalbusiness-to-businesstechnologyfirm,whilethesecond
studywasconductedwithafirminthetelecommunicationsindustrythatsellstobothbusinessesand
retailconsumers.Thestudiescompareda“customer-focus”inwhichthetimingofsalescallswas
coordinatedwithforecastsofcustomers’purchasingdecisions.Inthiscondition,salescallswerealso
coordinatedacrossproductcategories,sothatifacustomerwasexpectedtopurchaseinmultiple
categoriesassignedtodifferentsalesteams,thecustomerreceivedajointvisitfrombothteams.This
coordinatedpolicywascomparedwithastandardpolicy,whichlackedcoordination.Thetwo
treatmentswererandomlyassignedwithinmatchedpairsofcustomers.Thefindingsrevealedthat
coordinationledtosignificantlyhigherprofitsandreturn-on-investment.Lim,AhearneandHam(2009)
alsoinvestigatesalesforceoptimizationissuesusingrandomizedfieldexperiments,althoughtheyfocus
onsalesforceincentives.Specifically,theycomparetheimpactofdifferentprizestructuresinsales
contests.Theirfindingsindicatethatasalescontestshouldincludemultipleprizewinners,andrank
orderingprizesincontestswithmultiplewinnersdoesnotincreasesalesorrevenues.
9
ThefindingsintheSimester,SunandTsitsiklis(2006)fieldexperimentledtoasubsequentpaper(Mannoretal.
2007)inwhichtheauthorsdocumentedthepotentialforpositivebiaswhenapplyingdynamicprogramming
modelstofielddata.
15|P a g e Word-of-MouthandReferrals
Inawidely-citedpaper,GodesandMayzlin(2009)investigatewhetherfirmactionsdesignedto
encourageword-of-mouth(WOM)leadtohighersales.TheyfindthatWOMismoreeffectiveat
increasingsalesifitcomesfromrelativelylessloyalcustomers.TheyalsoreportthatWOMgenerates
moresalesifitcomesfromacquaintances,ratherthanclosefriends.Kumar,PetersonandLeone(2010)
usefourfieldexperimentsconductedwithafinancialservicesfirmtoidentifywhichcustomersafirm
shouldtargetwhendesigningcustomerreferralprograms.Theyusethefindingstovalidateanapproach
forcomputingthe“customerreferralvalue”foreachcustomer.
ParticipationinanOnlineCommunity
InastudyconductedinGermanyinvolvingeBayusers,Algesheimer,Borle,DholakiaandSingh(2010)
measuretheimpactofaprogramdesignedtoincreaseparticipationinthefirm’sonlinecommunity.
Customersintherandomlyassignedtreatmentgroupreceivedmultipleemailmessagesinviting
participation,whilethoseinthecontrolgroupdidnotreceivethesemessages.Overthenextyear
customersinthetreatmentgroupspentlessandlistedfeweritemsthancustomersinthecontrolgroup.
Twoyearslateranoverlappingresearchteamconductedafollow-upstudyagainusingGermaneBay
users(Zhu,Dholakia,ChenandAlgesheimer2012).Emailmessagestoarandomlyselectedtreatment
groupwereagainusedtoinviteparticipationinthefirm’sonlinecommunity.Thefindingsrevealthat
participantsengageinmorerisk-seekingbiddingbehavior.Thiseffectisonlytruewhencommunity
membershavestrongtiestoothermembersofthecommunity.Thefindingsarereplicatedinasecond
fieldexperimentconductedwithprosper.com.
ToubiaandStephen(2013)studywhypeoplecontributetothemicrobloggingsiteTwitter(seealsoour
earlierdiscussionofBergerandSchwarz2011,whoaskwhycustomerscontributeword-of-mouth).They
experimentallymanipulatedthenumberofTwitterfollowers,andcomparedtheirpostingactivitiestoa
randomlyassignedcontrolledgroup.Theirfindingssuggestthatmanyuserscontributebecausethey
careabouthowpeopleperceivethem,ratherthanbecausetheyderiveintrinsicutilityfromtheactivity.
EncouragingPositiveBehaviors
Weearlierdescribedtwopapersthatstudyhowadvertisingmessagescanleadtopro-socialbehavior
(WhiteandSimpson2013studywastereduction,andSpangenberg,Sprott,GrohmanandSmith2003
studyrecyclingbehavior).Twootherpapershavefocusedonencouragingpositivebehaviors.Raju,
RajagopalandGilbride(2010)examinehowtoencouragechildrentochoosehealthierfoodoptions.The
mosteffectiveinterventionwastoestablisha“friendlyhealthfuleatingcompetitionwithstudentsatthe
samegradelevelfromotherparticipatingschools.”Askingthechildrentosignapledgetoeatmore
fruitsandvegetables,andprovidingdirectincentivesforhealthychoices(e.g.pencils,stickers,key
chains)alsoledtoimprovedeatinghabits,evenupto10weeksaftertheinterventionsended.
SomanandCheema(2011)comparemethodsforimprovingsavingsratesamonglow-incomelaborersin
ruralIndia.Financialplannersvisitedthe146familiesinthestudyfor15weeks,andgavethemasavings
goal,whichwasplacedinasealedenvelope.Experimentalvariationsincludedthesizeofthisgoal,
whethertheirpicturesoftheirchildrenwereprintedontheenvelope,andwhetherthesavingsgoalwas
16|P a g e pooledintoasingleenvelopeorsplitbetweentwoenvelopes.Althoughthesizeofthegoaldidnot
affectthesavingsrate,savingsweresignificantlyhigherinthephotographcondition,andwhenthe
savingswerepartitionedintomultipleenvelopes.
OtherTopics
Theremainingtwopapersstudydiversetopics.DaggerandDanaher(2014)reportfindingsfromfield
experimentsdesignedtomeasurehowremodelingastoreimpactsstoredemand.Thefirstofthese
studieswasconductedatan“equipmentretailerandserviceprovider”thatretaineditsoriginalretail
space,whileremodelingnewspaceinthebuildingtoserveasareplacementretailspace.The
researchersrandomlyrotatedtheretailoperationsbetweenthenewandoldspacesforaperiodofsix
weeks.Theyfoundthattheremodeledspaceincreasedsalessignificantlymorefornewcustomersthan
forexistingcustomers.Theyreplicatetheirfindingsinasecondstudyconductedinalargedepartment
store.
HaruvyandLeszczyc(2010)conductedaseriesofexperimentsusingpairsofsimultaneousauctionsto
measuretheimpactofarangeofauctionfeatures,including:auctiondurations,shippingcosts,thelevel
ofopenreserveprices,andtheuseofsecretreserveprices.Theresultsrevealconsiderableprice
dispersionwithintheauction-pairs.Theyalsoconfirmthattheauctionfeatureshadsignificanteffectson
finalprices,whichtheauthorsattributetotheroleofsearchcosts.
7.
DesigningExperimentsandFutureTopics
Weconcludebyreflectingonwhatisrequiredtopublishfieldexperimentsinthemarketingliterature.
Wealsodiscusstherangeoftopicsthathavebeenstudied,andidentifytopicsthatremainrelatively
under-studied.
DesigningExperimentsfortheMarketingLiterature
Fieldexperimentspublishedinmarketingobviouslysharemanyofthesamecharacteristicsasfield
experimentspublishedineconomicsjournals.However,therearesomedistinguishingfeatures.
Publishinganyempiricalstudyinthemarketingliteraturerequiresmorethanmerelydocumentingan
effect.Researchersareexpectedtoalsoshedlightonthemechanismthatcausestheeffect.For
example,itisnotsufficienttojustshowthatmultipleunitpromotions(e.g.2for$1.50)aremore
effectivethansingleunitpromotions(75-centseach).Wansink,KentandHoch(1998)werealso
expectedtoexplainthisresultasanexampleofamoregeneralphenomenon.Theyinterpretedtheir
findingasanexampleof“anchoringandadjustment”.
Thisrequirementcanbeaformidableobstacleforfieldexperiments,asitisoftennotpossibleto
interviewcustomers,ortootherwisecollectintermediateprocessmeasuresthatcanrevealunderlying
causes.Instead,therearefourapproachesthatresearchershavegenerallyusedtoinvestigatethecause
oftheireffects,andtheyoftenusetheseapproachesincombination.
First,manypapersreportinteractionsratherthanjustmaineffects.Forexample,TuckerandZhang
(2011)donotjustreportthemaineffectofrevealingpopularityinformationonawebsite.Instead,they
comparetheseeffectsfornicheversusmainstreambrands.Similarly,BergerandSchwarz(2011)
17|P a g e comparetheeffectofadvertisingmessageonword-of-mouthforcustomerswhohavestrongandweak
associationsbetweenthebrandandthemessage.Forresearchersseekingtopublishfieldexperiments
inmarketing,itisimportanttogivecarefulthoughttowhatinteractionstheycanmeasure,andhow
thoseinteractionswillhelpclarifythecauseoftheireffects.Insightfulinteractionsareoftenviewedas
the“clever”elementofastudy.Ideallytheinteractionsareconsistentwithaproposedargument,and
aredifficulttoreconcilewithalternativeexplanations.Indeed,thisisthebenefitofreporting
interactions;theyaremorelikelytobeimmunefromalternativeexplanations.
Toestimateinteractions,researcherssometimesusemultiplefieldexperiments(recallthat30%ofthe
papersreportresultsfrommultiplefieldexperiments).Althoughtheadditionalstudiesareoccasionally
positionedasreplications(e.g.Danaher2002),theyarealsooftenusedtoinvestigateinteractions(e.g.
AndersonandSimester2003).Otherstudiesreportasingleexperiment,butincludealargenumberof
experimentaltreatments.Forexample,inGaurav,ColeandTobacman’s(2011)studyofdemandfor
rainfallinsuranceamongruralIndianfarmers,theyinclude14differentexperimentalconditions.In
general,studieseitherreportmultipleexperimentsormultipletreatments(andnotboth).10
Whereapaperreportsasinglefieldexperimentwithjusttwoconditions,itiscommontocomplement
thefieldexperimentwithoneormorelaboratoryexperiments.Examplesinclude,Kronrod,Grinstein
andWathieu(2012),whoaddtwolaboratoryexperiments,andKrider,RaghubirandKrishna,who
reportfivelaboratoryexperiments.Inpapersthatincludemultiplelaboratoryexperiments,thefield
experimentoftenservesalessprominentrole.Inparticular,thelaboratoryexperimentsmayestablish
themaineffect,replicatetheeffect,andinvestigateinteractions,whereasthefieldexperimentis
relegatedtoconfirminggeneralizabilityinafieldsetting.
Athirdapproachusedtoinvestigateexplanationsistocombinethefieldexperimentwithacustomer
survey.Forexample,recallthatAndersonandSimester(2001a)measuredhowcustomersreacttoan
installmentbillingofferinapremiumjewelrycatalog.Customersdidnotpurchaseasmuchinthe
installmentbillingcondition,whichtheyinterpretedasanadversequalitysignal,revealingthatthe
productsaretargetedatcustomerswhoaremoresensitivetoqualitythanprice.Tosupportthis
interpretation,theymailedcatalogssimilartothoseusedfortheexperimentaltreatmentstoother
randomlyselectedcustomersamples,togetherwithashortsurvey.Customerswhoreceivedthe
installmentbillingversionweremorelikelytoexpressconcernaboutproductquality.Onerespondent
wroteonthesurveyinstrument:“Itmakes[catalogname]looktackytohaveinstallmentplans–kindof
likeFranklinMintdolls”(atpage326).ShuandGneezy(2010)provideasimilarexampleofusinga
surveytovalidateanexplanationfortheirexperimentalfindings.
Thefourthapproachistousefieldexperimentdatatoestimateastructuralmodel.Forexample,ina
recentworkingpaperDubé,LuoandFang(2016)reportfindingsfromtwofieldexperimentsinwhich
theysentSMS(text)messagestoChinesemobilephoneusersofferingpromotionsonmovietickets.
Theyrandomlyvaried(a)thesizeofthediscount,(b)whetherthetextindicatedtheserviceprovider
woulddonatetoalocalcharityforeveryticketpurchased,and(c)thesizeofthedonation(ifany).
Surprisingly,theyshowthatcustomersarelessresponsivetolargerdonationswhenthediscountsare
10
Thepair-wisecorrelationbetweenthenumberofexperimentsandthe(maximum)numberofconditionsis
−0.14.
18|P a g e large.Theyestimateastructuralmodeltoprovideevidencethatthiseffectresultsfrom“self-signaling”;
largerpricediscountspromptconsumerstoinferthattheirpurchasesarenolongeraltruistic.The
authorsobservethatwithoutthestructuralmodelthereisnoobviouswaytotestthismechanismand
disentangleitfromalternativeexplanations.Thisapproachofusingastructuralmodeltoexplainthe
findingsinafieldexperimentisstillrelativelynovelinthemarketingliterature.Thestructuralmodeling
andfieldexperimentliteratureshavegrownatsimilartimes,perhapspartlybecausetheyoffer
alternativesolutionstoresolvingendogeneityconcerns(seeearlierdiscussion).Thispaperrecognizes
thatthetwoapproachesarenotjustalternatives;theycanalsocomplementeachother.Giventherapid
growthininterestinbothmethods,oncethebenefitsofcombiningthemethodsaremorewidely
recognizedweshouldanticipatemanymorepapersusingthiscombinationofmethods.
Randomization
Randomizationofferswell-documentedstatisticaladvantageswhencomparingtreatmentandcontrol
conditions.However,randomizationisnotrequiredtopublishfieldexperimentsinthemarketing
literature.In29%ofthepapers,experimentaltreatmentswerenotassignedbyrandomization(inthree
papersitwasunclearwhetherthestudiesusedrandomization).Insomecasesrandomizationcouldhave
ledtocontagionbetweentheexperimentaltreatments.Forexample,inSomanandCheema’s(2011)
studyofhowtoincreasesavingslevelsamongruralIndianlaborers,participantswereassignedto
conditions“accordingtogeographicandsocialclusterstominimizethepossibilityofhouseholdsfrom
differenttreatmentconditionsmeetinganddiscussingtheirparticipation”(atpageS17).Inothercasesit
isnotobvioushowrandomizationcouldbeachieved.Forexample,inthestudiesinvolvingmatched
pairsofauctionsoneBay(DholakiaandSimonson2005,LeszczycandHaubl2010,andHaruvyand
Leszczyc2011),thematchedpairsserveastreatmentandcontrolsamples.Randomizingthe
experimentaltreatmentsbetweenproductswithinapairwouldnotbemeaningful,eitherbecausethe
productsareidentical,orbecausetheexperimentalvariationinvolvesdifferencesintheproduct
bundlesthemselves.
Intheabsenceofrandomization,acommonapproachistorotatetreatmentsacrosstime.Forexample,
inDaggerandDanaher’s(2014)studyofastoreremodeling,“theoriginalretailenvironmentservedas
thecontrol,usedinweeks1,4and5.Thenewenvironment,whichrepresentedthetreatment
condition,wasinplaceforweeks2,3and6”(page66).11OtherexamplesincludeYangandGhose
(2010),wherethetreatmentwasimplementedbyrotatingsponsorshipofkeywordsintwoweek
intervals,andKrider,RaghubirandKrishna(2001),whorotateacrossdaystheshapeofcreamcheese
packagesinastore.
FutureTopics
Thissurveyoftherecentmarketingliteraturerevealsaremarkablediversityinfieldexperimenttopics.
Thismirrorsthediversityoftopicsinthefieldgenerally.Firmshaveawiderangeofleversthattheycan
usetoinfluencedemand,andthemarketingliteratureconsequentlyhasawiderangeoftopicsto
11
DaggerandDanaher(2014)didrandomlyassignthetreatmentstothesixweeks,althoughitiscommonfor
rotationtooccurwithoutrandomization.
19|P a g e choosefrom.Notably,theInternethasincreasedtherangeofleversavailabletofirms,andthishas
furtherbroadenedthetopicsthatarestudied.
Whilethediversityofavailabletopicsmakesthefieldattractivetoresearchers,itisalsoaweakness.
Almostwithoutexceptionthepaperswehavediscussedraisenewresearchquestions,ratherthan
buildingonpreviousresearch.Otherfieldsbenefitfromgreaterfocusintheirresearchtopics;suchas
thestudyofpricerigidityinthemonetaryeconomicsliterature.Thisisnotthecaseinmarketing,which
lacksbroadagreementabouttheresearchquestionsthatshouldbeprioritized.Asaresult,therehasnot
beenatraditionofextendingpreviousfindingsandcomprehensivelyansweringwell-definedproblems.
Althoughtherearesomeexceptionstothisobservation,manyoftheexceptionsreflectadedicationtoa
researchquestionbyasingleresearchteam(orbyteamswithoverlappingresearchers).
Despitethediversityoftopics,therearetopicsthatareclearlyunder-represented.Noneofthepapers
investigateissuesrelatedtomanagingupstreamordownstreamchannelpartnerships.Forexample,
noneofthepapersinvestigatethebenefitsofexclusivityinsupplyordistributionrelationships,orthe
roleofstandardizationtofacilitatecoordination.Theabsenceofstudiesonthesetopicsmayinpart
reflectthedifficultyofimplementingexperimentalvariation.Variationinchannelrelationshipsalmost
alwaysrequiresconflict,whichdisruptsrelationshipsthatfirmshaveoftenspentconsiderabletime
building.
Thereisalsorelativelylittleresearchstudyingbusiness-to-businessmarkets.Onlyfiveofthepapers
reportfieldexperimentsinwhichfirmstargetotherbusinesses(thisincludesonestudytargetingrural
farmersinIndia).Thispaucityofstudiesmayreflectdifficultiesingeneratingsufficientsampleinthese
markets.Experimentationmayalsobehinderedbygreatertransparency,whichcouldleadtobusiness
customersinthedifferentexperimentalconditionsbecomingawareoftheexperimentalvariation
(althoughthislimitationwouldalsoseemrelevantinseveraloftheInternetstudies).Athirdpossibilityis
thatthetimerequiredtoimplementchangeandobserveoutcomescouldbelongerinbusiness-tobusinesssettingsthaninconsumermarkets.
Whatisperhapsmostsurprisingisthelackofpapersinvestigatingthefeasibilityofusingfield
experimentstooptimizemarketingdecisions.Aswehavealreadyrecognized,themarketingfieldhasa
longhistoryofdevelopingmodelstooptimizemarketingdecisions.Indeed,eightofthepapersthatwe
reviewedreportonfieldexperimentsdesignedtovalidateexampleofoptimizationmodels.However,
fieldexperimentsarethemselvesanoptimizationmethod.Byexperimentallychangingmarketing
variablesandcomparingtheoutcomes,firmscouldinprincipleusefieldexperimentstoimproveprofits
bysearchingacrossthespaceofpossibledecisions.
Noneoftheempiricalpaperswesurveyedinvestigatethefeasibilityofthisapproach.Perhapsthe
closestexampleisDanaher(2002),whichusesthedatafromafieldexperimentimplementedbya
telecommunicationscompanytodevelopastatisticalmodelthatoptimizesatwo-parttariff(pricing)
scheme.However,inthisexamplethefieldexperimentisusedasasourceofdata,whilethe
optimizationispursuedusingmoretraditionalmethods.
Althoughwecouldfindnoempiricalpapersthatstudywhetherfirmscouldusefieldexperimentsasa
practicaloptimizationmethod,thereisarecenttheoreticalpaper.Li,Rusmevichientong,Simester,
20|P a g e TsitsiklisandZoumpoulis(2015)investigatehowmanyexperimentsareneededtosetpricesina
productcategory,asthesizeofthecategorygrows.Settingpricesacrossaproductcategoryrequires
estimatingalargematrixofcross-productdemandelasticities(becauseitemsinthesamecategorymay
besubstitutesorcomplements).Theyshowthatifthecategoryhasafavorablestructure,thenumberof
experimentsneededmaygrowjustlogarithmicallywiththenumberofproducts.Theyconcludethat
firmsmaybeabletoobtainmeaningfulestimatesusingapracticallyfeasiblenumberofexperiments,
evenincategorieswithalargenumberofproducts.Toourknowledge,thisistheonlypaperthat
formallyinvestigateswhenitisfeasibletooptimizemarketingvariablesusingfieldexperimentsalone.
However,thispaperpresentstheoreticalresultsusinginformationtheory;itdoesnotreporttheresults
ofanyfieldexperiments.
8.
Conclusions
Fieldexperimentsarenolongerararityinthemarketingliterature.Thesharpgrowthinthenumberof
papers,particularlyinthelastfiveyears,meansthattheyarenowamainstreammethodfor
undertakingempiricalresearch.Thegrowthinthenumberofpapersismatchedbytremendousgrowth
inthenumberofauthorswhoareparticipatinginthesestudies.Between1995and1999just11authors
authoredorco-authoredapaperusingafieldexperiment(inoursample).Between2010and2014
therewere75authorsrepresented.12
Thereappeartobeatleastthreereasonsforthisgrowthinthenumberoffieldexperimentpapers.First,
themarketingfieldhasbecomeincreasinglyconcernedabouttheinterpretationofendogenous
independentvariables.Thisconcerndoesnotariseifexperimentaltreatmentsareexogenously
manipulated.Second,thefieldhasgrowngenerally,andsotherearemorepapersofalltypesbeing
published.Third,theInternethasmadeiteasiertoimplementfieldexperiments,oftenwithoutthe
needforcooperationfromfirms.
Oursurveyrevealsthatbefore2010,fieldexperimentpapersinmarketingweredominatedbypricing
topics.Since2010,atleastsomeofthisfocushasshiftedfrompricingtoadvertising,withmanyofthe
recentpapersinvestigatingtopicsrelatedtoInternetadvertising.However,evenwithinthepricingand
advertisingtopics,thereisaremarkablelevelofdiversityintheresearchquestions.Veryfewofthe
papersbuildonapreviousfieldexperimentpaper.Werecognizethisasbothastrengthandweakness
ofthefield.
Animportantcriterionforpublishingafieldexperimentinatopmarketingjournalistheabilityto
extendthefindingsbeyonddocumentingamaineffect,toalsoprovideinsightsaboutthecauseofthat
effect.Althoughexplainingeffectsisobviouslyvaluedineconomicsjournals,thisobjectivereceiveseven
greateremphasisinthemarketingfield.Fieldexperimentsarenotalwayswell-suitedtoevaluating
competingexplanations.Forthisreason,manyofthepapersreportinteractions,ratherthanjustmain
effects.Otherresearcherscomplementtheirfieldexperimentswithlaboratoryexperimentsorcustomer
surveys,inordertoprovidesupportfortheirinterpretations.
12
Acrossall20yearsthereare123uniqueauthorsorco-authorsofpapersinthesample.
21|P a g e Althoughanexplanationforaneffectisgenerallynecessary,randomizationisnot.Almost30%ofthe
fieldexperimentspublishedinmarketingdonotemployrandomizationtoassignexperimental
treatments.Whentreatmentsarenotrandomized,theyaregenerallyrotatedovertime.Wealso
identifiedthreepapersthatreportfieldexperimentsusingmatchedsetsofproductsauctionedoneBay.
Althoughwehavehighlightedthediversityintopics,ourreviewalsoidentifiedseveraltopicsthatare
yettoreceiveattention.Theseincludetheproblemsofmotivatingandcoordinatingupstreamand
downstreamchannelrelationships.Therearealsoveryfewstudiesofbusiness-to-businessmarkets.
Finally,werecognizedthatfieldexperimentsarethemselvesanoptimizationmethod;firmscoulduse
fieldexperimentstoimproveprofitsbysearchingacrossthespaceofpossibledecisions.Althoughthere
issomeinitialtheoreticalworkinthisarea,thefeasibilityofusingfieldexperimentsinthisroleremains
animportantbutunder-studiedtopic.
22|P a g e 9.
References
Agarwal,Ashish,KartikHosanagar,andMichaelD.Smith(2011),“Location,Location,Location:An
AnalysisofProfitabilityofpositioninonlineAdvertisingMarkets,”JournalofMarketingResearch,Vol.
XLVIII,December,1057-1073.
Ailawadi,KusmL.,BariA.Harlam,JacquesCésar,andDavidTrounce(2007),“QuantifyingandImproving
PromotionEffectivenessatCVS,”MarketingScience,26(4),566-575.
Algesheimer,René,SharadBorle,UtpalM.Dholakia,andSiddarthS.Singh(2010),“TheImpactof
CustomerCommunityParticipationonCustomerBehaviors:AnEmpiricalInvestigation,”Marketing
Science,29(4),756-769.
Anderson,EricT.andDuncanI.Simester(1998),“TheRoleOfSaleSigns”MarketingScience,17(2),139155.
Anderson,EricT.andDuncanI.Simester(2001a),“ResearchNote:PriceDiscriminationasaSignal:Why
anOffertoSpreadPaymentsMayHurtDemand,”MarketingScience,20(3),315-327.
Anderson,EricT.andDuncanI.Simester(2001b),“AreSaleSignsLessEffectiveWhenMoreProducts
HaveThem?”MarketingScience,20(2),121-142.
Anderson,EricT.andDuncanI.Simester(2003),“Effectsof$9PriceEndingsonRetailSales:
EvidencefromFieldExperiments,”QuantitativeMarketingandEconomics,1(1),93-110.
Anderson,EricT.andDuncanI.Simester(2004),“LongRunEffectsofPromotionDepthonNewVersus
EstablishedCustomers:ThreeFieldStudies,”MarketingScience,23(1),4-20.
Anderson,EricT.andDuncanI.Simester(2008),“ResearchNote:DoesDemandFallWhenCustomers
PerceiveThatPricesAreUnfair?TheCaseofPremiumPricingforLargeSizes,”MarketingScience,27(3),
492-500.
Anderson,EricT.andDuncanI.Simester(2010),“PriceStickinessandCustomerAntagonism,”Quarterly
JournalofEconomics,125(2),729-765.
Anderson,EricT.andDuncanI.Simester(2013),“AdvertisinginaCompetitiveMarket:TheRoleof
ProductStandards,CustomerLearningandSwitchingCosts,”JournalofMarketingResearch,50(4),489504.
Bawa,Kapil,andRobertShoemaker(2004),“TheEffectsofFreeSamplePromotionsonIncremental
BrandSales,”MarketingScience,23(3),345-363.
Belloni,Alexandre,MitchellJ.Lovett,WilliamBoulding,andRichardStaelin(2012),“OptimalAdmission
andScholarshipDecisions;ChoosingCustomizedMarketingOfferstoAttractaDesirableMixof
Customers,”MarketingScience,31(4),621-636.
Berger,JonahandEricM.Schwarz(2011),“WhatDrivesimmediateandOngoingWordofMouth,”
JournalofMarketingResearch,Vol.XLVIII,October,869-880.
Borle,Sharad,UtpalM.Dholakia,SiddharthS.Singh,andRobertA.Westbrook(2007),“TheImpactof
SurveyParticipationonSubsequentCustomerBehavior:AnEmpiricalinvestigation,”MarketingScience,
26(5),711-726.
Bronnenberg,BartJ.,Jean-PierreDubé,andCarlF.Mela(2010),“DoDigitalVideoRecordersInfluence
Sales?”JournalofMarketingResearch,Vol.XLVII,998-1010.
Chandon,Pierre,VickiG.Morwitz,andWernerJ.Reinartz(2005),“DoIntentionsReallyPredict
23|P a g e Behavior?Self-GeneratedValidityEffectsinSurveyResearch,”JournalofMarketing,69,April,1-14.
Chen,Haipeng(allan),HowardMarmorstein,MichaelTsiros,andAkshayR.Rao(2012),“WhenMoreis
Less:TheImpactofBaseValueNeglectonConsumerPreferencesforBonusPacksoverPriceDiscounts,”
JournalofMarketing,76,July,64-77.
Curhan,RonaldC.(1974),“TheEffectsofMerchandisingandTemporaryPromotionalActivitiesonthe
SalesofFreshFruitsandVegetablesinSupermarkets,”JournalofMarketingResearch,Vol.XI,286-94.
Dagger,TraceyS.,andPeterJ.Danaher(2014),“ComparingtheEffectofStoreRemodelingonNewand
ExistingCustomers,”JournalofMarketing,78,May,62-80.
Danaher,PeterJ.(2002),“OptimalPricingofNewSubscriptionServices:AnalysisofaMarket
Experiment,”MarketingScience,21(2),119-138.
Dhar,Sanjay,andStephenJ.Hoch(1996),“PriceDiscriminationUsingIn-StoreMerchandising,”Journal
ofMarketing,January,17-30.
Dholakia,UtpalM.(2006),“HowCustomerSelf-DeterminationInfluencesRelationalMarketing
Outcomes:EvidencefromLongitudinalFieldStudies,”JournalofMarketingResearch,Vol.XLIII,
February,109-120.
Dholakia,UtpalM.,andItamarSimonson(2005),“TheEffectofExplicitReferencePointsonConsumer
ChoiceandOnlineBiddingBehavior,”MarketingScience,24(2),206-217.
Dubé,Jean-Pierre,XuemingLuo,andZhengFang(2016),“Self-SignalingandProsocialBehavior:aCause
MarketingExperiment,”workingpaper,UniversityofChicago.
Eskin,GeraldJ.andPennyH.Baron(1977),“EffectsofPriceandAdvertisinginTest-Market
Experiments,”JournalofMarketingResearch,Vol.XIV,499-508.
Gaurav,Sarthak,ShawnCole,andJeremyTobacman(2011),“MarketingComplexFinancialProductsin
EmergingMarkets:EvidencefromRainfallInsuranceinIndia,”JournalofMarketingResearch,Vol.XLVIII,
SpecialIssue2011,S150-S162.
Gneezy,Ayelet,UriGneezy,andDominiqueOliéLauga(2014),“AReference-DependentModelofthe
Price-QualityHeuristic,”JournalofMarketingResearch,Vol.LI,April,153-164.
Godes,DavidandDinaMayzlin(2009),“Firm-Created-Word-of-MouthCommunication:Evidencefroma
FieldTest,”MarketingScience,28(4),721-739.
Guadagni,PeterM.andJohnD.C.Little(2008),“ALogitModelofbrandChoiceCalibratedonScanner
Data,”MarketingScience,15(1),1-20.
Haisley,EmilyandGeorgeLoewenstein(2011),“It’sNotWhatYouGetbutWhenYouGetIt:TheEffect
ofGiftSequenceonDepositBalancesandCustomerSentimentinaCommercialBank,”Journalof
MarketingResearch,Vol.XLVIII,February,103-115.
Haruvy,Ernan,andPeterT.PopkowskiLeszczyc(2010),“SearchandChoiceinOnlineConsumer
Auctions,”MarketingScience,29(6),1152-1164.
Hui,SamK.,J.JeffreyInman,YanliuHuang,andJacobSuher(2013),“TheEffectofIn-StoreTravel
DistanceonUnplannedSpending:ApplicationstoMobilePromotionStrategies,”JournalofMarketing,
77,March,1-16.
Ju-YoungKim,MartinNatterandMartniSpann(2009),“PayWhatYouwant:ANewParticipativePricing
Mechanism,”JournalofMarketing,73,January,44-58.
24|P a g e Krider,RobertE.,PriyaRaghubir,AradhnaKrishna(2001),“Pizzas:πorSquare?PsychologicalBiasesin
AreaComparisons,”MarketingScience,20(4),405-425.
Kronrod,Ann,AmirGrinstein,andLucWathieu(2012),“GoGreen!ShouldEnvironmentalMessagesBe
SoAssertive?”JournalofMarketing,76,95-102.
Kumar,V.,J.AndrewPeterson,andRobertP.Leone(2010),“DrivingProfitabilitybyEncouraging
CustomerReferrals:Who,When,andHow,”JournalofMarketing,74,September,1-17.
Kumar,V.,RajkumarVenkatesan,andWernerReinartz(2008),“PerformanceImplicationsofAdoptinga
Customer-FocusedSalesCampaign,”JournalofMarketing,72,September,50-68.
Lambrecht,AnjaandCatherineTucker(2012),“PayingwithMoneyorEffort:PricingWhenCustomers
AnticipateHassle,”JournalofMarketingResearch,Vol.XLIX,66-82.
Lambrecht,AnjaandCatherineTucker(2013),“WhenDoesRetargetingWork?InformationSpecificityin
OnlineAdvertising,”JournalofMarketingResearch,Vol.L,October,561-576.
Laran,Juliano,andMichaelTsiros(2013),“AnInvestigationoftheEffectivenessofUncertaintyin
MarketingPromotionsInvolvingFreeGifts,”JournalofMarketing,77,112-123.
Leszczyc,PeterT.L.Popkowski,andGeraldHaübl(2010),“ToBundleorNottoBundle:Determinantsof
theProfitabilityofMulti-ItemAuctions,”JournalofMarketing,Vol.74,110-124.
Lewis,RandallA.,andDavidH.Reiley(2014),“OnlineAdsandOfflineSales:MeasuringtheEffectsof
RetailAdvertisingViaaControlledExperiment,”QuantitativeMarketingandEconomics,12,235-266.
Li,Hongshuang(Alice),andP.K.Kannan(2014),“AttributingConversionsinaMultichannelOnline
MarketingEnvironment:AnEmpiricalModelandaFieldExperiment,”JournalofMarketingResearch,
Vol.LI,February,40-56.
Li,JimmyQ.,PaatRusmevichientong,DuncanI.Simester,JohnN.TsitsiklisandSpyrosI.Zoumpoulis
(2015),“TheValueofFieldExperiments,”ManagementScience,forthcoming.
Lim,Noah,MichaelJ.Ahearne,andSungH.Ham(2009),“DesigningSalesContests:DoesthePrize
StructureMatter?,”JournalofMarketingResearch,Vol.XLVI,356-371.
Mantrala,MuraliK.,P.B.Seetharaman,RajeeveKaul,andSrinathGopalakrishna,andAntonieStam
(2006),“OptimalPricingStrategiesforanAutomotiveAftermarketRetailer,”JournalofMarketing
Research,Vol.XLIII,November,588-604.
Neslin,ScottA.,ThomasP.Novak,KennethR.Baker,andDonnaL.Hoffman(2009),“AnOptimalContact
ModelforMaximizingOnlinePanelResponseRates,”ManagementScience,55(5),727-737.
Raju,Sekar,PriyaliRajagopal,andTimothyJ.Gilbride(2010),“MarketingHealthfulEatingtoChildren:
TheEffectivenessofIncentives,PledgesandCompetitions,”JournalofMarketing,Vol.74,May,93-106.
Ramanathan,SureshandSanjayK.Dhar(2010),“TheEffectofSalesPromotionsontheSizeand
CompositionoftheShoppingBasket:RegulatoryCompatibilityfromFramingandTemporal
Restrictions,”JournalofMarketingResearch,Vol.XLVII,June,542-552.
Schumann,JanH.,FlorianvonWangenheim,andNicoleGroene(2014),“TargetedOnlineAdvertising:
UsingReciprocityAppealstoIncreaseAcceptanceAmongUsersofFreeWebServices,”Journalof
Marketing,78,January,59-75.
Shu,SuzanneB.andAyeletGneezy(2010),“ProcrastinationofEnjoyableExperiences,”Journalof
MarketingResearch,Vol.XLVII,October,933-944.
25|P a g e Simester,DuncanI.,PengSun,andJohnN.Tsitsiklis(2006),“DynamicCatalogMailingPolicies,”
ManagementScience,52(5),683-696.
Skiera,Bernd,andNadiaAbouNabout(2013),“PROSAD:ABiddingDecisionSupportSystemforProfit
OptimizingSearchEngineAdvertising,”MarketingScience,32(2),213-220.
Soman,Dilip,andAmarCheema(2011),“EarmarkingandPartitioning:IncreasingSavingbyLow-Income
Households,”JournalofMarketingResearch,Vol.XLVIII,SpecialIssue2011,S14-S22.
Spangenberg,EricR.,DavidE.Sprott,BiancaGrohmann,andRonnJ.Smith(2003),“MassCommunicatedPredictionRequests:PracticalApplicationandaCognitiveDissonanceExplanationfor
Self-Prophecy,”JournalofMarketing,Vol.67,July,47-62.
Toubia,Olivier,andAndrewT.Stephen(2013),“Intrinsicvs.Image-RelatedUtilityinSocialMedia:Why
DoPeopleContributeContenttoTwitter?,”MarketingScience,32(3),368-392.
Tsiros,Michael,andDavidM.Hardesty(2010),“EndingaPricePromotion:RetractingItinOneStepor
PhasingItOutGradually,”JournalofMarketing,74,January,49-64.
Tucker,CatherineE.(2014),“SocialNetworks,PersonalizedAdvertising,andPrivacyControls,”Journalof
MarketingResearch,Vol.LI,October,546-562.
Tucker,CatherineE.,andJuanjuanZhang(2010),“GrowingTwo-SidedNetworksbyAdvertisingtheUser
Base:AFieldExperiment,”MarketingScience,29(5),805-814.
Tucker,CatherineE.,andJuanjuanZhang(2011),“HowDoesPopularityInformationAffectChoices?A
FieldExperiment,”ManagementScience,57(5),828-842.
Urban,GlenL.,JohnR.Hauser,WilliamJ.Qualls,BruceD.Weinberg.JonathanD.Bohlmann,and
RobertaA.Chicos(1997),“InformationAcceleration:ValidationsandLessonsfromtheField,”Journalof
MarketingResearch,Vol.XXXIV,February,143-153.
Urban,GlenL.,Guilherme(Gui)Liberali,ErinMacDonald,RobertBordley,andJohnR.Hauser(2014),
“MorphingBannerAdvertising,”MarketingScience,33(1),27-46.
Villas-Boas,MiguelJ.,andRussellS.Winer(1999),“EndogeneityinBrandChoiceModels,”Management
Science,45(10),1324-1338.
Wansink,Brian,RobertJ.Kent,andStephenJ.Hoch(1998),“AnAnchoringandAdjustmentModelof
PurchaseQuantityDecisions,”JournalofMarketingResearch,Vol.XXXV,February,71-81.
White,Katherine,andBonnieSimpson(2013),“WhenDo(andDon’t)NormativeAppealsInfluence
SustainableConsumerBehaviors?”JournalofMarketing,77,March,78-95.
Yang,Sha,andAnindyaGhose(2010),“AnalyzingtheRelationshipBetweenOrganicandSponsored
SearchAdvertising:Positive,NegativeorZeroIndependence?”MarketingScience,29(4),602-623.
Yao,Song,CarlF.Mela,JeongwenChiang,andYuxinChen(2012),“DeterminingConsumers’Discount
RateswithFieldStudies,”JournalofMarketingResearch,Vol.XLIX,December,822-841.
Zhu,Rui(Juliet),UtpalM.Dholakia,Xinlei(Jack)Chen,andRenéAlgesheimer(2012),“DoesOnline
CommunityParticipationFosterRiskyFinancialBehavior?”JournalofMarketingResearch,Vol.XLIX,
June,394-407.
26|P a g e Appendix:SummaryofthePapers
PricingIssues
Whototargetwithdiscounts?
Avoidsendingdiscountstocustomerswhopreviouslypaidfullprice.
Dholakia(2006)
Avoidofferingdiscountstocustomerswhoaremoresensitivetoqualitythanprice.
AndersonandSimester(2001a)
Deepdiscountshaveamorefavorablelong-termimpactfornewcustomersthanfor
existingcustomers.
AndersonandSimester(2004)
Whattypesofdiscountsaremosteffective?
Morethan50%ofthepromotionsareunprofitablebecausethelossofmarginisnotfully
offsetbytheincrementalsales.
Ailawada,Harlam,Césarand
Trounce(2007)
Couponsaremoreeffectiveforincreasingsalesandprofitsthandiscountsofftheregular
price.
DharandHoch (1996)
Multipleunitpromotions(e.g.2for$1.50)aremoreeffectivethansingleunitpromotions
(75-centseach).Largerquantitylimitsincreasethenumberofunitsthatcustomersbuy.
Wansink,KentandHoch(1998)
Framingadiscountas“50%morefree”ismoreeffectivethan“33%offtheregularprice”.
Chen,Marmorstein,Tsirosand
Rao(2012)
Theoptimaldesignofacoupondependsuponthecustomer’spsychologicalorientation.
RamanathanandDhar(2010)
Freesamplescanleadtopositivelong-termdemandeffects.
BawaandShoemaker(2004)
Asequenceofincreasingcustomerbonusesismoreeffectiveatdrivingusageand
retentionthanasequenceofdecreasingbonuses.
HaisleyandLoewenstein(2011)
Longerdeadlinesongiftcardsleadtolowerredemptionrates.
ShuandGneezy(2010)
Uncertaintyaboutafreegiftcaninfluencetheeffectivenessofpromotion,depending
uponwhethercustomersareprimedtomakeemotionalorthoughtfuldecisions.
LaranandTsiros(2013)
Returningpromotedpricestoregularlevelsgraduallywillgeneratemoredemand.
TsirosandHardesty(2010)
Promotionsthatextendstoretripscanleadtoanincreaseinunplannedexpenditure.
Hui,Inman,HuangandSuher
(2013)
Trainingcustomerstoappreciatethebenefitsoftheproductcanbemoreeffectivethan
promotions.
Gaurav,ColeandTobacman
(2011) PriceasaSignal
9-digitpriceendingscansignalapriceisdiscounted,particularlyfornewproducts.
AndersonandSimester(2003)
“Sale”signsarealesscrediblesignalwhentheyareusedtoooften.
AndersonandSimester(2001b)
Higherpricessethigherexpectations,andsocustomersevaluatealowqualityproduct
morenegativelywhenithasahigherprice.
Gneezy,GneezyandLauga
(2014)
Promotionscanlowerdemandbysignalingthataproductissuitableforcustomerswho
aremoresensitivetoqualitythanprice.
AndersonandSimester(2001a)
27|P a g e Usingdiscountstoattractfirst-timecustomerscanmakethesecustomersmoreloyalby
signalingthatthefirmoffersgoodvalue.
AndersonandSimester(2004)
Multi-PartTariffsandOtherSchemes
Customersevaluateoutcomeswithinaperiod,ratherthanacrosstheentirecontract.
LambrechtandTucker(2012)
Amulti-partpricingsystemcanlowerdemandifcustomersinterpretitasunfair.
AndersonandSimester(2008)
Whendesigningarevenuemaximizing2-parttariffinthetelecommunicationsmarketitis Danaher(2002)
importanttoaccountforbothusageandcustomerretention,assolelyfocusingonusage
willsubstantiallyunder-estimatethesensitivityofrevenuetoprice.
Consumersaresurprisinglyshort-termfocusedwhenchoosingbetweenlinearprice
schemesand3-parttariffs.
Yao,Mela,ChiangandChen
(2012)
Howdocustomersrespondtoapay-what-you-wantpricingscheme?
Kim,NatterandSpann (2009)
AdvertisingIssues
Isthereanimpactonpurchasing?
IttakesalotofdatatomeasuretheimpactofInternetadvertisingonconsumerspending. LewisandRiley(2014)
Retargetingisgenerallyineffective,unlessconsumersareclosetopurchasing.
LambrechtandTucker(2013)
DVR'sdonotappeartoaffecthouseholdspending.
Bronnenberg,DubeandMela
(2010)
Competitors’advertisingcanincreaseyourdemand.
AndersonandSimester(2013)
Whichmessagesaremosteffective?
Framinganappealasaself-predictionincreasesitseffectiveness.
Spangenberg,Sprott,Grohman
andSmith(2003)
Matchingthetypeofappealwithafocusoneitherindividualversussocialbenefitscan
improveadvertisingeffectiveness.
WhiteandSimpson(2013)
Highlightingthebenefitsoftargetingislesseffectivethanemphasizingreciprocityforfree Schumann,vonWangenheim
access.
andGroene(2014)
Controloverpersonalizationcanmakepersonalizedadvertisementsmoreeffective.
Tucker(2014)
Marketswithmorecompetitorscanappearmoreattractivetoentrants,astheremustbe
sufficientdemandtoattractsomanycompetitors.
TuckerandZhang(2010)
Advertisingpopularityinformationcanbenefitnichebrands,becausecustomers
recognizethatthesebrandsmustofferhighqualitytoovercometheirnarrowreach.
TuckerandZhang(2011)
Appealstoexplicitlycompareanofferwithcompetingoffersproducesmorecautious
biddingbehaviorinauctions.
DholakiaandSimonson(2005)
Theeffectofadvertisingonword-of-mouthisstrongerforthosewhodonotalready
associatethebrandwiththemessage.
BergerandSchwartz(2011)
Assertivemessagingismoreeffectiveifcustomersbelieveinthecause,butlesseffective
Kronrod,GrinsteinandWathieu
28|P a g e iftheyarenotyetconvinced.
(2012)
Optimizingpaidsearchadvertising
Clicksandrevenuesfororganicsearcharehigherinthepresenceofpaidsearch.
YangandGhose(2010)
Clicksdecreasewithsearchpositionbutconversionmayincreaseswithsearchposition.
Agarwal,HosanagarandSmith
(2011)
MarketResearchandProductDevelopmentIssues
MarketResearch
Meremeasurementofpurchaseintentionsmayincreasethecorrelationbetweenstated
purchaseintentionsandactualpurchaseprobabilities.
Chandon,MorwitzandReinartz
(2005)
Participationincustomersatisfactionsurveysincreasessubsequentpurchaselikelihoods.
Borle,Dholakia,Singhand
Westbrook(2007)
Multimediacomputerinteractionscanrealisticallyportraycustomer’sinformation
sources,andprovideforecaststhatarenotsignificantlydifferentfromtraditional
methods.
Urban,Hauser,Qualls,
Weinberg,BohlmannandChicos
(1997)
Anoptimalcontactmodelforincreasingresponseratestoonlinemarketingresearch
panels.
Neslin,Novak,Bakerand
Hoffman(2009)
ProductSizesandBundling
Ifacustomerneedsafixedquantity,theywillpurchasemoreunitswhentheproductisin
aroundpackagecomparedtoasquarepackageofequalvolume.
Krider,RaghubirandKrishna
(2001)
Bundlingsubstitutesorunrelateditemsinanauctionislessprofitablethansellingthem
separately,butbundlingismoreprofitableforcomplementaryproducts.
LeszczycandHaubl(2010)
Multipleunitpromotions(e.g.2for$1.50)aremoreeffectivethansingleunitpromotions
(75-centseach).
Wansink,KentandHoch(1998)
ValidatingOptimizationModels
PricingModels
Categorypricingofautomobileparts.
Mantrala,Seetharaman,Kaul,
GopalakrishnaandStam(2006)
Scholarshipsanduniversityadmissiondecisions
Belloni,Lovett,Bouldingand
Staelin(2012)
AdvertisingModels
Optimizingsearchengineadvertisingbids.
SkieraandNabout(2013)
Anadvertisingcontactmodelforcatalogmailing.
Simester,SunandTsitsiklis
(2006)
Estimatingtheincrementalimpactofmarketingactivitiesindifferentchannels.
LiandKannan(2014)
29|P a g e Banneradvertisingoptimization.
Urban,Liberali,MacDonald,
BordleyandHauser(2014)
MarketResearchandProductDevelopmentModels
Increasingresponseratestoonlinemarketingresearchpanels.
Neslin,Novak,Bakerand
Hoffman(2009)
Forecastingcustomerdemandforreallynewproducts.
Urban,Hauser,Qualls,
Weinberg,BohlmannandChicos
(1997)
OtherTopics
SalesforceOptimization
Coordinatingsalescallstomatchforecastsofcustomers’purchasingdecisionscan
increaseprofits.
Kumar,VenkatseanandReinartz
(2008)
Salesforcesalescontestshouldincludemultipleprizewinners,andrankorderingpricesin Lim,AhearneandHam(2009)
contestswithmultiplewinnersdoesnotincreasesalesorrevenues.
Word-of-MouthandReferrals
Word-of-mouthismoreeffectiveatincreasingsalesifitiscomesfromrelativelylessloyal
customers,andfromacquaintancesinsteadoffriends.
GodesandMayzlin(2009)
Whichcustomersshouldafirmtargetwhendesigningcustomerreferralprograms?
Kumar,PetersonandLeone
(2010)
ParticipationinanOnlineCommunity
Participationinanonlinecommunityleadstolessspendingandfewerlistings.
Algesheimer,Borle,Dholakia
andSingh(2010)
Participationinanonlinecommunitymakescustomersmorewillingtoengageinrisky
biddingbehavior,particularlywhentiestoothercommunitymembersarestrong.
Zhu,Dholakia,Chenand
Algesheimer(2012)
UserscontributetoTwitterbecausetheycareabouthowpeopleperceivethem,rather
thanbecausetheyderiveintrinsicutilityfromtheactivity.
ToubiaandStephen(2013)
EncouragingPositiveBehaviors
Competitions,pledgesandincentivescanallimprovechildren’sfoodchoices.
Raju,RajagopalandGilbride
(2010)
Earmarkingearningscanincreasesavingsrates.
SomanandCheema(2011)
Other
Remodelingaretailspaceincreasessalesmorefornewcustomersthanestablished
customers.
DaggerandDanaher(2014)
Auctionoutcomescanbeinfluencedbyshippingcosts,secretreserveprices,andother
auctionfeatures.
HaruvyandLeszczyc(2011)
30|P a g e