Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Unsupervised Transfer Classification
Document related concepts
no text concepts found
Transcript
Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University Overview Introduction Related Work Unsupervised Transfer Classification Problem Definition Approach & Analysis Experiments Conclusions Introduction Classification: supervised learning semi-supervised learning supervised What if No label information is available? semi-supervised impossible but not with some additional information unsupervised classification Introduction Unsupervised transfer classification (UTC) a collection of training examples and their assignments to auxiliary classes to build a classification model for a target class auxiliary class 1 auxiliary class K …. conditional probabilities No Labeled training examples target class prior Introduction: Motivated Examples Image Annotation auxiliary classes grass target classes Social Tagging google phone verizon apple sky sun water 1 0 0 ? 1 0 0 ? 1 1 0 ? 1 1 0 ? 0 1 1 ? 0 1 1 ? 0 0 1 ? 0 0 1 ? / / / ? / / / ? How to predict an annotation word/social tag that does not appear in the training data ? Related Work Transfer Learning transfer knowledge from source domain to target domain similarity: transfer label information for auxiliary classes to target class difference: assume NO label information for target class Multi-Label Learning, Maximum Entropy Model Unsupervised Transfer Classification Data for auxiliary class Examples assignments to auxiliary classes Auxiliary Classes Class Information Prior probability Goal target class conditional probabilities target class label target classification model Maximum Entropy Model (MaxEnt) Favor uniform distribution Feature statistics computed from conditional model : the jth feature function Feature statistics computed from training data Generalized MaxEnt Equality constraints With a large probability Inequality constraints Generalized MaxEnt Generalized MaxEnt is unknown for target class How to extend generalized MaxEnt to unsupervised transfer classification ? Unsupervised Transfer Classification Estimating feature statistics of target class from those of the auxiliary classes Unsupervised Transfer Classification Build up Relation between Auxiliary Classes and Target Class Independence Assumption Unsupervised Transfer Classification Estimating feature statistics for the target class by regression Feature Statistics for Auxiliary Classes Class Information Feature Statistics for Target Class Unsupervised Transfer Classification Dual problem : function of U; definition can be found in paper Consistency Result With a large probability The optimal dual solution The dual solution using the label information obtained by the proposed approach for the target class Experiments Text categorization Data sets: multi-labeled data Protocol: leave one-class out as the target class Metric: AUC (Area under ROC curve) Experiments: Baselines cModel cLabel predict the assignment of the target class for training examples by linearly combining the labels of auxiliary classes train a classifier using the predicted labels for target class GME-avg train a classifier for each auxiliary class linearly combine them for the target class use generalized maxent model compute the feature statistics for the target class by linearly combining those for the auxiliary classes Proposed Approach: GME-Reg Experiment (I) Estimate class information from training data Experiment (I) Estimate class information from training data Compare to the classifier of the target class learned by supervised learning 1500 2500 Experiment (II) Obtain class information from external sources Datasets: bibtex and delicious bibsonomy www.bibsonomy.org/tagsbibtex ACM DL www.portal.acm.orgbibtex deli.cio.us www.delicious.com/tagdelicious Experiment (II) Comparison with Supervised Classification 650 1000~1200 Conclusions A new problem: unsupervised transfer classification A statistical framework for unsupervised transfer classification based on generalized maximum entropy robust estimate feature statistics for target class provable performance by consistency analysis Future Work relax independence assumption better estimation of feature statistics for target class Thanks Questions ?
