Download Artificial Intelligence and Subfields - cse.scu.edu

ArtificialIntelligenceandSubfields
Ming-HwaWang,Ph.D
Adjunct,DepartmentofComputerEngineering
SantaClaraUniversity
February4,2017
ArtificialIntelligence
• Humanintelligence:
Useofintuition,commonsense,judgment,creativity,goal-directedness,
plausiblereasoning,knowledgeandbeliefs
• Artificialintelligence:
Afieldofstudythatencompassescomputationaltechniquesforperforming
tasksthatapparentlyrequireintelligencewhenperformedbyhuman
• Turingtest
• Applications:
Computervision,speechandaudioprocessing,naturallanguageprocessing,
robotics,bioinformaticsandchemistry,videogames,searchengines,online
advertisingandfinance
• Challenges:
Solvingthetasksthatareeasyforpeopletoperformintuitivelybuthardfor
peopletodescribeformally,e.g.,speechunderstanding,facerecognition
TraditionalAI
• Fundamentalissues:
Knowledgerepresentation,search,perceptionandinference
• Classictechniques:
• Symbolic,rule-based,oralgorithm-based:emphasizetheuseofprior
knowledge,asystemwithmanuallycreatedruleshaslimitedsuccess
• Alwayshaveexceptionsforrules
• AlmostallAIproblemsareNP-hardO(2n)
• Artificialneuralnetwork(ANN):
• Reverse engineer the computational principles behind the brain and
duplicateitsfunctionality
• Thesecondbestsolutionforalmostanyproblem
• EarlyFailureofAI
• Moore’sLawandparallel/distributedcomputing
RecentAIProgress
• Bigprogressfields:
Machinelearning,datamining,naturallanguageprocessing,ComputerVision,
etc.
• Empirical,data-drivenstatisticalapproaches:
Achievedencouragingresults(statisticalrevolution),emphasizethedata
• AIDeepLearning:
The hierarchy of concepts enables the computer to learn complicated
conceptsbybuildingthemoutofsimplerones
• BigData:
A supervised deep learning algorithm will generally achieve acceptable
performancewitharound5,000labeledexamplespercategoryandwillmatch
orexceedhumanperformancewhentrainedwithadatasetcontainingatleast
10millionlabeledexamples
• ChangedMindset:
Notexact/perfectsolution
•
•
•
•
•
•
•
•
•
•
AISubfields
MachineLearning
DataMining
InformationRetrievalandSemanticWeb
SpeechRecognitionandNaturalLanguageProcessing
ImageProcessing/RecognitionandComputerVision
Robotics
Search
KnowledgeRepresentationandKnowledgeDatabase
LogicReasoningandProbabilisticReasoning
ExpertSystems
ExpertSystems
• Domain-specificcomplexapplication
LogicReasoning&ProbabilisticReasoning
• Logicreasoning:
Propositionalcalculusàpredicatecalculusàfirst-orderpredicatecalculus
• Logicprogramming:
• FunctionalProgrammingLanguage:Lisp,Scheme
• VeryHighLevelProgrammingLanguage:Prolog
• Automatedreasoning:
• Inferencerule(generatenewclausesfromexistingones),
• Subsumption(removeunnecessaryclauses),
• Demodulation(replace/rewrite/substituteclauses),
• Theoremprovingandqualification
• Probability:additivelaw,multiplicativelaw,Bayes’rule
• Probabilityinferencenetworksandfuzzylogic
• Dempster-ShafterCalculus:
Formanipulatedegreesofbelief,whichdoesn’trequireB(A)+B(¬A)=1
KnowledgeRepresentation&KnowledgeDatabase
•
•
•
•
•
•
Productionsystem:pattern-action-orientedknowledge
Inclusionorconcepthierarchies:object-oriented
Mathematiclogics:first-orderpredicatecalculus
Framesofcontext
Semanticnetworksorsemanticnet
Constraintschemataorgeneralizedconstraints:
Predicatecalculusaugmentedwithproceduralinformation
• RelationaldatabaseandNoSQL
•
•
•
•
Key-valueNoSQL(e.g.,Redis)
Column-familyNoSQL(e.g.,Google’sBigTable,HBase,Cassandra)
DocumentNoSQL(e.g.,MongoDB,CouchDB)
GraphNoSQL(e.g.,Neo4j)
• Challenges:
• Closeworldassumption
• Expensiveknowledgeacquisition
Search
• Depth-firstsearch
Usingbacktracking
• Breadth-firstsearch
Usingqueue
• Heuristicsearchmethods
Withcostfunctionandpriorityqueueorheap
• Approximation
Closebutnotexactsolutionwithdeviationguarantee
• RandomizeorProbabilisticmethods
• Planning
Makesproblemsolvingmoretractablebymakingtheoperationsconditional
Robotics:
• Sensorforperception:
• Proprioception
• Forcesensing
• Tactilesensingortouchsensing
• Sonar
• CameraandcomputerVision
• Effectorforaction:
• Actuator:convertssoftwarecommandsintophysicalmotion
• Locomotion:changethepositionoftherobotwithinitsenvironment
• Manipulation:moveotherobjectsintheenvironment,changetheshape
orotherphysicalpropertiesofobjects
• Endeffector:tool
• AutonomousSystems:
Imagingprocessing/recognitionand
computervision
• Vision:
• Therichestofthefivesensingmodalitiesofhuman,itoccupies
approximately¼ofthebrain
• Simultaneouslywithspatial,geometricalrelations,andsymbolic,semantic
structures
• Manipulation:
Elementarymanipulation,extractionofmeaningfulstructures,edge
detectionandsegmentationintoregions,analyzingshape,representingand
determining3Dstructurefrom2Dimages,specialheuristicsforhandling
blocksworldscenes
• Computervision:
Theperceptionmodel,machinerecognitionofpatterns,pictureprocessing,
automaticimageanalysis,sceneanalysis,imageunderstanding
Speech&NaturalLanguageProcessing
• Speechrecognitionandsynthesis,stemmingandlemmatization,syntaxand
parsing, semantic analysis and knowledge representation, formal language
theory, statistical methods, probabilistic models, hidden Markov models,
computational linguistic, machine translation, spoken language
understanding, question answering, conversational agents, machine
translation,summarization,andhuman-machineinteraction
• source-channelmathematicalmodelforaspeechrecognitionsystem
communicationchannel
text
generator
W speech generator
signal processing X
speech decoder
Ŵ
speechrecognizer
where W is a source word sequence, X is W’s acoustic signal, and Ŵ is the recognized words
sequence
Speech&NaturalLanguageProcessing
• basicsystemarchitectureofaspeechrecognitionsystem
voice application application signalprocessing
decoder
languagemodels
adaptation acousticmodels
• basicsystemarchitectureofaspokenlanguageunderstandingsystem
application database
discourseanalysis
dialogmanager dialogstrategy
responsegeneration sentenceinterpretation
text-to-speech speechrecognizer
accessdevice
Speech&NaturalLanguageProcessing
• basicsystemarchitectureofatext-to-speechsystem
rawtextor taggedtext TTSEngine
textanalysis:documentstructuredetection,text
normalization,linguisticanalysis
Taggedtext
phoneticanalysis:grapheme-to-phoneme
conversion
Taggedphones
prosodicanalysis:pitch&durationattachment
Controls
speechsynthesis:voicerendering
InformationRetrieval&SemanticWeb
• IR:
Process data offline and store it in inverted index to provide good search
performance
• Ranking
• Tools:Lucene,Solr,ElasticSearch,etc.
• SemanticWeb:
Semantic-basedsearchinginsteadofkeyword-basedsearching
• ResourceDescriptionFramework(RDF):
• Triples(subject,predicate,andobject)andhypergraph
• RDFàRDFS(RDFScheme)àWebOntologyLanguage(OWL)
• RDFQueryLanguageSPARQL
DataMining
• Findingsimilaritems:
Shingling,minhash,andLSH(localitysensitivehashing)
• Frequentitemsets:
Market-basketmodel,A-Priorialgorithmandvariations
• Miningdatastream:
Sampling,filtering,counting,estimatingmoments
• Linkanalysis:
PageRank,topic-sensitivePageRank,linkspam,hubsandauthority
• Textminingandsocialmining
MachineLearning
• Recommendation(collaborativefiltering),clustering,classification
• Supervised/predictive learning, unsupervised/descriptive learning, semisupervisedlearning,reinforcementlearning
• MLprojectlifecycle:
Understanding, data collection/cleaning, feature selection, model selection,
training/tuning,evaluation,deployment,review
• Error-basedlearning:linearregression,gradientdescent,multinomiallogistic
regression,modelingnon-linearrelationship),artificialneuralnetworks
• Information-basedlearning:decisiontreelearning,entropyandinformation
gain,modelensembleslikeboostingandbagging
• Similarity-based learning: feature space and distance metrics, nearest
neighboralgorithm,k-dtree
• Probability-basedlearning:naïveBayes,Bayesiannetworks
• NoFreeLunchTheorem
DeepLearning
• Usingdeepermodelscanreducethenumberofunitsrequiredtorepresent
thedesiredfunctionandcanreducetheamountofgeneralizationerror
• 2Dmatrixalgebraà3Dtensoralgebra
• Artificial neural network (ANN), convolutional neural network (CNN),
recurrentneuralnetwork(RNN),deepneuralnetwork
• Convexandnon-convexoptimization
• PerformanceenhancementwithGPU