Download Advances in Artificial Intelligence Require Progress Across all of

AdvancesinArtificialIntelligenceRequire
ProgressAcrossallofComputerScience
February2017
GregoryD.Hager,RandalBryant,EricHorvitz,MajaMatarić,andVasantHonavar
Overthelastdecade,theconstellationofcomputingtechnologiesreferredtoas
artificialintelligence(AI)hasemergedintothepublicviewasanimportantfrontier
oftechnologicalinnovationwithpotentialinfluencesinmanyrealms.Advancesin
manydisciplinesrelatedtoAI,includingmachinelearning,robotics,computer
vision,naturallanguageprocessing,inference,decision-making,andplanning,are
contributingtonew-fieldedproducts,services,andexperiences.Offeringssuchas
navigationsystems,websearch,speechrecognition,machinetranslation,face
recognition,andrecommenderengineshavebecomepartofthedailylifeofmillions
ofpeople.Otherapplicationscomingtotheforeincludesemi-autonomousand
autonomousgroundandairvehicles,systemsthatharnessplanningandscheduling,
intelligenttutoring,robotics.Morebroadly,cyber-physicalandroboticsystems,
incorporatingvaryingdegreesofAItechnology,arepoisedtobefieldedinavariety
ofreal-worldsettings.
AlthoughAIwillbeanengineforprogressinmanyareas,creatingreal-world
systemsthatrealizetheseinnovationswillinfactrequiresignificantadvancesin
virtuallyallareasofcomputing,includingareasthatarenottraditionallyrecognized
asbeingimportanttoAIresearchanddevelopment.Indeed,weexpectfutureAI
systemswillnotonlydrawfrommethods,tools,andthemesinotherareasof
computerscienceresearch,butwillalsoprovidenewdirectionsforresearchin
areassuchasefficiency,trustworthiness,transparency,reliability,andsecurity.
Thisisnotnew–thehistoryofAIisinextricablyintertwinedwiththehistoryof
advancesinbroadercomputerscience(CS)aswellasapplicationsinrelatedareas
suchasspeechandlanguage,computervision,robotics,andothertypesof
intelligentsystems.
Inwhatfollows,wereviewseveralpromisingareasofinteractionbetweenAIand
broadercomputersciencethatproviderichopportunitiesaheadforresearchand
development.WeincludeopportunitiesthatseemespeciallyimportantasAI
systemsbecomemoreubiquitousandareplayingrolescriticaltoourindividualand
combinedhealthandwell-being.Inbrief,weseeparticularlyrichopportunitiesfor
supportingadvancesinAIviasynergiesandcollaborationwithresearchand
developmentinthefollowingareas,describedbrieflybelow,thenfurtherexpanded
upon:
1
Computingsystemsandhardware.Thereareopportunitiesaheadforleveraging
innovationsincomputingsystemsandhardware.Directionsincludethe
developmentofmethodsforspeedingupcorecomputationalproceduresemployed
inAIsystems,suchasthemethodsusedtotrainandtoexecuteclassificationfor
perceptualtasksusingconvolutionalneuralnetworks.Opportunitiesincludenew
approachestoparallelism,smartcaching,andusesofspecializedhardwarelike
FPGAstolowercostsofcomputationandtomeetthedemandsandrobustness
neededwithAIapplications.
Theoreticalcomputerscience.AIwasbuiltontheoreticalworkbasedonthe
mathematicsofcomputabilityintheearly20thcenturybyTuring,Church,and
others.AIchallengeshavelongposedandfacedcombinatorialchallengesandhas
madeuseofresultsontheperformanceandprecisionofapproximationprocedures.
TherearecontinuingopportunitiesfortheformalstudyofhardchallengesinAI
withtoolsandtechniquesdevelopedintherealmsofanalysisofalgorithms,
includingeffortsincombinatorics,computationalcomplexitytheory,andstudiesof
computability.
Cybersecurity.AIsystemsarebeingdevelopedforhigh-stakessystemsinsuch
areasashealthcareandtransportation.Thesesystemsarealsobringingtothefore
newattacksurfacesthatneedtobeunderstoodandprotected.Directionsinclude
understandingandhardeningsystemstowholenewcategoriesofattackincluding,
“machinelearningattacks,”wherecleveradversarialproceduresareemployedto
injectdataintosystemsthatwillconfuseorbiasthemintheirintendedoperation.
AIsystemsframenewchallengesthatwillrequireadvancesinsecuritythataddress
thenewattacksurfacestoensurethattheyaresafe,reliable,robustandsecure
againstmaliciousattacks.
Formalmethods.Formalmethodscanplayacriticalroleindefiningand
constrainingAIsystems,soastoensurethattheirbehaviorconformsto
specifications.Effortsincludemethodsfordoingformalverificationofprogramsand
alsotoperformreal-timeverificationofsystemsthroughnewkindsofmonitoring.
Formalmethodsarepromisingapproachestoensuringthatprogramsdonottake
actionsbeyondspecifiedgoalsandconstraints.
Programminglanguages,tools,andenvironments.Newprogramminglanguages,
tools,andprogrammingenvironmentscanhelpengineerstobuild,test,andrefine
AIsystems.Higher-levellanguagescanofferengineersandscientistsnewkindsof
abstractionsandpowertoweavetogethermultiplecompetencies,suchasavision,
speechrecognition,andnaturallanguageunderstandingsoastobeabletodevelop
anddebugprogramsthatrelyontheclosecoordinationofmultipleAIanalytical
pipelines.
Human-computerinteraction.ThekeychallengeswithAIframenumerous
opportunitiesinthebroadrealmofresearchinhuman-computerinteraction(HCI),
animportantareaofcomputerscienceresearch.Effortsincludemethodsfor
2
explainingtheresultsofAIsystemstopeople,allowingpeopletoworkinteractively
withAIsystems(e.g.,interactivemachinelearning),thathelpwiththespecification,
encoding,andunderstandingoftheimplicationsofdifferentpolicies,values,and
preferencesassumedbyautomatedsystems,andsupportingnewkindsofhuman-AI
collaboration,includingmixed-initiativeinteractionandaugmentinghuman
cognition.
ComputingSystemsandHardware
Researchincomputerhardware,operatingsystems,andcomputernetworkinghave
been,andcontinuetobe,criticaltocreatingthelarge-scalesystems,includingAI
systemsthatusevastamountsofdatatobuildpredictivemodels,optimizecomplex
objectivefunctions,performautomatedinferenceoverlargeknowledgebases,or
complexprobabilitydistributions.Someofthelargestcomputerfacilitiesinthe
worldareoperatedbyindustrialandgovernmentorganizationstocollectdata,
buildpredictivemodelsfromdatausingmachinelearningalgorithms,andthenuse
theresultingpredictivemodelstoprovideservicesrangingfromdrivingdirections
toshoppingrecommendations.
AsAIcontinuestogrow,thedemandsonthesesystemswillalsogrowandchange.
Forexample,currentusesofcloudcomputingaretypicallynothardreal-time.
Whetherasearchqueryisansweredin0.2sor0.4smaynotmattermuchin
practice.However,aqueryfromacarmovingat60mphmayneedtobeanswered
withinahardreal-timeconstrainttoensurethesafeoperationoftheautomobile.
Historically,specialpurposehardwarearchitecturesforspecifictaskse.g.,computer
vision,havebeeninandoutoffashion.However,asweapproachtheendofthe
Moore’slaw,performancegainsthatarerequiredforsuccessfuldeploymentofAI
systemsinreal-worldapplicationsarelikelytorequireinnovationsinhardwareand
systemsaswellasco-designofhardwareandsoftwaretoexploitthespecial
featuresofspecializedarchitecturesandsystems.Largeserversystemsalreadyuse
GPUsextensivelytosupportmodernlearningalgorithms.Atscale,itmaybecome
costandenergyefficienttoincludeevenmorecustomizedcapabilitythatmustbe
sharedamongonetoamillionusersdependingonthesysteminquestion.
Unlikemanycomputingtasks,machinelearningcanoftenmakeuseofapproximate
results.Theymakedecisionsusingstatisticalmodelsandtherecanbeatolerance
forsmallcomputationalerrorsintroducedbythehardwareorbyimperfect
synchronizationacrosscomputingresources.Thisopensupmanypossibilitiesfor
makingthesystemmoreenergyefficientorscalableatthehardware,software,and
systemlevels.Achievingthesebenefitswillrequireacarefulcharacterizationofthe
natureofthecomputationalerrorsthatcanbeintroducedandtheirimpactonthe
overallsystemfunctionality.
TheoreticalCS:AnalysisofAlgorithms,Combinatorics,andComplexity
3
AIhasinfluenced,andbenefitedfrom,advancesinalgorithmsinanumberofareas
includingautomatedreasoning,search,planning,optimization,andlearning.For
example,machinelearning,whichliesattheheartofmanymodernAIsystems,
enablesGoogle’sAlphaGoprogramtodevisestrategiesforplayingthegameofGoby
analyzingmillionsofmovesmadebyhumansinGotournaments;Amazon’s
recommendersystemstoanalyzelargedatasetsoftransactionsandthensuggest
productstocustomers;automobilelanetrackingsystemstodetectlanemarkers
fromvideoimagesandwarndriverswhentheyareveeringoutoftheirlanes;
languagetranslationsystemstogeneratemappingsfromonelanguagetoanotherby
processinglargecollectionsofhuman-generatedtranslations.Machinelearning
requiresalgorithmsthatconstructsophisticatedpredictivemodelstoofferuseful
insightsandactionableknowledgefromlargeandcomplexdatasets.Thedemands
ofmachinelearninghasledtoadvancesinalgorithms,especiallyforoptimizationof
complexobjectivefunctions,reasoningaboutcomplexprobabilitydistributionse.g.,
usingfactorizedrepresentations,etc.Withtheemergenceofbigdata,the
requirementthatmachinelearningalgorithmsneedtobescalabletomassive
amountsofhighdimensionaldata,robustinthepresenceofnoise,etc.,present
manychallengesindesignandanalysisofalgorithms.
Similarly,AIsystemsforplanning(e.g.,motionplanning,dialogplanning,and
activityplanning)callforadvancesindatastructuresandalgorithmsfor
representingandreasoningaboutlargestatespaces,copingwithuncertainty,and
providingcompactrepresentationsthatsupportefficientaccesstodata.The
emergenceoflargeknowledgebasesthatcodifyhumanknowledgeinavarietyof
domainscallsforrobustalgorithmsforautomatedinference;forexample,for
answeringcomplexqueriesagainstknowledgebases.Manyproblems,including
assemblingproductsthatmeetspecifiedrequirementsfromavailablecomponents,
canbeformulatedasproblemsinsearchoroptimization,andhence,candrive
advancesinconstraintprocessingandoptimization.AIsystemsforcomputervision,
naturallanguagedialogue,textprocessing,videoanalytics,andothertaskscan
similarlydriveinnovationsinalgorithms.
ThesuccessofAIinreducingmoreandmoretasksthatwerethoughttorequire
humanintelligenceintoonesthatcanbesolvedalgorithmicallywillstimulatework
onestablishingthecorrectness,performancebounds,andthetrade-offsamong
them.Thiswillhaveimmediatepracticalconsequences,aswillnegativeresultsthat
establishthetheoreticallimitsofalgorithms.Suchinvestigationscanleadto
importantinsights,forexamplewhetherataskislearnableinprinciple,inpractice,
andunderwhatconditions,suchas,fromobservations,queries,orexperimentation.
Ensuringrobustoperationwillrequirefindingboundsonthepotentialdeviationsof
anAIsystemwhenpresentedwithunanticipatedinputs.
Cybersecurity
InadditiontothemanybenefitscomputertechnologyandtheInternetprovide
society,ithasprovedtobeapowerfultoolforadversaries,includingmalicious
4
individuals,criminals,andnationstates.Attackersuseanumberofmethodsto
extractsensitiveinformationfromorganizationsandtodisableordisruptthe
activitiesofindividuals,corporations,andgovernment.
AIsystemshavevulnerabilitiesthatincludethoseofbothtraditionalandnew
computersystems.Forexample,bycorruptingthetrainingdata,anAIsystemcanbe
trickedintoconstructinganinvalidpredictivemodel.Giventhelimitedmechanisms
currentlyavailablefortestingthesemodelssuchcorruptionmaybedifficultto
detect.
AsAIsystemsaredeployedintherealworld,adversarieswillseekwaystotrick
themintobehavinginundesirableways.Forexample,imagineattemptingtodeploy
anautonomousarmoredtruck.Adversarieswouldbehighlymotivatedtoforceitto
stoportoalteritscoursebyputtingthetruck’scontrolsystemoutsidetherangeof
conditionsithasbeentrainedtoconsider.Indeed,suchlimitationswillimpedethe
deploymentofAIsystemsinsecurity-criticalenvironments.
AIcanalsoprovideapowerfultoolforbothcyberattackersandcyberdefenders.On
theattackside,anetworkofbotnetscouldusereal-timedataanalyticsto
dynamicallyadapttheirbehavior,increasingtheireffectivenessanddiminishing
theirdetectability.Conversely,machinelearningisalreadyprovidinganeffective
toolfordetectinganomaliesincomputersystemsthatcouldindicateanintrusion.
Hence,increasingrelianceonAIsystemsasintegralcomponentsofcomplex
systems,callsforadvancesincybersecurity.
FormalVerification
Computersystemsoftenfailduetoerrorsinthedesignorimplementationofits
hardwareandsoftware.Sometimes,suchfailures,suchasthelossofanimportant
file,whileannoying,areofnomajorconsequence.Butwhencomputersdirectly
controlcriticalsystems,suchasmedicaldevices,civilinfrastructure,anddefense
systems,consequencesoffailuresaremuchmoresevere.Hence,weneedtoolsfor
ensuringthatsoftware,particularly,complexAIsoftware,complywith
specifications.
Formalverificationtoolsprovidewaystotestthecorrectnessandhardwareand
softwareunderallpossibleoperatingconditions.Theygobeyondthetraditional
methodoftestingthesystemonmanydifferentindividualcasestoconsiderall
possiblecases.Suchtoolshavehadsignificantsuccesswithpurelydigital
computations.Forexample,Inteltookalossof$475millionwhenthefirstversion
ofitsPentiumprocessorcould,underveryrarecircumstances,producethewrong
resultwhendividingtwonumbers.Theysubsequentlydevelopedasetoftoolsto
ensurethecorrectnessoftheirarithmeticcircuitsforallpossibledata.Similarly,
othercompanieshavedevelopedanddeployedtoolsthatcandetectmanyclassesof
softwareerrors.
5
However,establishingthecorrectnessofcomputersystemsthatoperateinthe
physicalworld–eithersystemswhoseperformancedependsonthedatathatthey
weretrainedon,asinthecaseofAIsystemsthatrelyonmachinelearning,or
systemsthatneedtooperateinopenenvironmentsthatarehard-to-characterize
(e.g.,householdrobots,automatedvehicles,etc.)–ismuchmoredifficult.For
example,itisimpossibletoanticipateallofthesituationsanautonomousvehicle
mightencounter,muchlessguaranteethatitwillhandleeachsuchsituation
correctly.Ontheotherhand,itmaybepossibletouseformalverificationtechniques
toverifythatAIsystemswillconformtospecificboundsontheirperformancein
suchenvironments.
ProgrammingLanguages
RapiddevelopmentofcomplexAIsystemscallsforadvancesinprogramming
languagesandtools.Ofparticularinterestaredomain-specificprogramming
languageswithbuilt-inhighlevelabstractionsthatmakeiteasytodesignand
programAIsystemsforclassesofapplicationse.g.,naturallanguageprocessing,
computervision,multi-robotsystems.Alsoofinterestarenewprogramming
languagesthatsupportprobabilisticcomputations,large-scaleautomatedinference,
constraintprocessing.,etc.
Machinelearningalsoprovidesaninspirationforanewconceptionof
programming.Withthesesystems,thehumanprogrammerscreateaframeworkfor
howthesystemshouldoperate,buttheactual“program”,suchastheweightsthat
specifythedetailedconfigurationofaneuralnetwork,isderivedalgorithmicallyby
trainingthesystemoverlargeamountsofdata.Onecanimagineafutureinwhich
evenhigherlevelsofautomationareappliedtoconstructmuchofwhatisdoneby
humanprogrammerstoday.
Human-ComputerInteraction
AIwillrevolutionizethetypeofresearchthatcanandmustbedonetoenable
peopletoeffectivelyuse,understand,interactandco-existwithAIsystems.Topick
oneexample,todaymostvoicequeriesareeitherdictationofamessageororder,or
aone-timequery.FutureAIsystemswillneedtoengageinbroaderformsofdialog
todealwithambiguity,confusion,ortoimproveengagement.Withtheincreasing
adoptionofAIsystems(e.g.,automatedvehicles,robots,softwareassistants,inrealworldenvironments)thereisanincreasingneedforresearchonframeworks,
languages,abstraction,andinterfacesthatalloweffectivecommunicationand
interactionbetweenhumansandAIsystems.
Ofparticularinterestaremixedinitiativesystemsthatallowproductive
collaborationsamongAIsystems,humansandAIsystems,aswellasamong
humans,withmediationandfacilitationbyhumansorAIsystemsasneeded.AI
6
systemsmustengagewithhumansinacollaborativemanner,allowingpeopleto
workinteractivelywithAIsystems(e.g.,interactivemachinelearning),thathelp
withthespecification,encoding,andunderstandingoftheimplicationsofdifferent
policies,values,andpreferencesassumedbyautomatedsystems,andsupporting
newkindsofhuman-AIcollaboration,includingmixed-initiativeinteractionand
augmentinghumancognition.
Thus,ascapabilitiesinAIgrow,sotoowillthequestionsandopportunitiesto
connectAItopeopleinameaningfulandeffectiveway.
InConclusion
ThereisagrowingandcompellingimperativetoleveragetheadvancesinAIand
automationtoimprovehumanlivesinmanyways.Atthesametime,suchsystems
willalsobecomefarmorepresentandconsequentialtoeverydaylife,andwill
provideservicesandcapabilitiesthatwillexploitlargeamountsofdata(including
personaldata),controlphysicaldevicesofvariouskinds,includingdevicesinsafety
criticalareas,andbeempoweredtomakeandactondecisionsofvarying
importancethatcouldinfluenceindividualsandsocietiesinexplicitandimplicit
ways.
Thepathtowardabalancedportfolioofcapable,safe,andtransparentAI-based
systemswilldrawonabroadspectrumofcomputingideasandprinciples,andis
likelytobecomeadriverfornewadvancesincomputing.Byembracingthepromise
ofAI,webelievethatmanyareasofcomputersciencewillnotonlybeadvanced,but
willalsoallowAItoaddressimportantopportunitiesanddosoinawaythatissafe,
reliable,andeffective.
ThismaterialisbaseduponworksupportedbytheNationalScienceFoundationunder
GrantNo.1136993.Anyopinions,findings,andconclusionsorrecommendations
expressedinthismaterialarethoseoftheauthorsanddonotnecessarilyreflectthe
viewsoftheNationalScienceFoundation.
7