Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
On the Use of Computable Features for Film Classification Zeeshan Rasheed,Yaser Sheikh Mubarak Shah IEEE TRANSCATION ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, JAN 2005 Outline • Introduction • Computable video features – – – – Average Shot Length Color Variance Motion Content Lighting Key • Mean Shift Classification • Results • Conclusion Introduction • Films are a means of expression – Explicitly, with the delivery of lines by actors – Implicitly, with the background music, lighting, camera movements and so on • Study domain is the movie preview, which often emphasizes the theme of a film Introduction • Maybe a need to extract the “genre” of scenes • With scene-level classification, it would allow a more flexible system of scene ratings, ex filter and recommendation of movies Related Work • Work by Fischer et al. and Truong et al. distinguished between newscasts, cartoon, commercials, sports through decision tree with examples • Kobla et.al used DCT coefficients, and motion vector information of MPEG video for indexing and retrieval Computable video features • Identify four major genres – Action, Comedy, Horror, and Drama – because most movies can be classified and low-level discriminant analysis is most likely to succeed • Employ for features – Average Shot Length, Shot Motion Content, Lighting Key, Color Variance Shot Detection and Average Shot Length • First proposed by Vasconcelos • Can direct audience’s attention with controling the tempo of the scene • Ex. Dramas have larger average length, whereas action movies shorter shot length Shot Detection and Average Shot Length • Detection of shot boundaries using color histogram intersection in the HSV space – H: hue (color), 8 bins – S: saturation, 4 bins – V: value (brightness), 4 bins • S(i) represent the intersection of histograms H and H of frames i and i-1 i i 1 Shot Detection and Average Shot Length min bin1 bin2 Frame i bin1 bin2 Frame i-1 Shot Detection and Average Shot Length min bin1 bin2 bin1 bin2 When S(i) is less than a fixed threshold Frame i Frame i-1 shot boundaries !! Shot Detection and Average Shot Length 17 shots identified by a human observer Number of shots detected: 40; Correct: 15; False positive: 25; False negative: 2 Shot Detection and Average Shot Length To improve accuracy, an18; iterative smoothing of the 1-D function Number of the shots detected: Correct: 16; isFalse performed first2; False negative: 1 positive: Color Variance • variance of color has a strong correlational structure with genres intuitively – For instance, comedies with a large variety of bright colors – whereas horror films with only darker hues • Employ variance of CIE Luv – L: luminancy (發光度) – u,v: chrominancy (色差) Color Variance Generalized variance is obtain ps. All key frames presented in a preview are used to find this feature Motion Content • The visual disturbance of a scene can be represented as the motion content present • Action films with higher value for such a measure, and dramatic or romantic movies with less visual disturbance Motion Content Horizontal slice: I(x,t) Motion Content Hx, Ht are the partial derivatives of I(x,t) Lighting Key • There are numerous ways to illuminate a scene, one of the common used is Three Point Lighting Fill-light: Keylight: Backlight: Secondary illumination The main Help emphasize sourcethe of light source which helps to on the subject contour of the object, and it isand soften some of the itthe also source separates of greatest it from a shadows thrown by the illumination dark background keylight and backlight Lighting Key • High-key lighting: – An abundance of bright light – More action, less dramatic – Ex. Comedy & action movies • Low-key lighting: – Ex. Film noir or horror films Lighting Key • Many algorithms exist that compute the position of a light source in a given image • Unfortunately, assumptions typically made in existing algorithms are violated, for example, single light source • Compute the key of the lighting with brightness value of pixels Lighting Key Lighting Key • Key frame i with m*n pixels, find the mean and standard deviation of the value component of the HSV space • Lighting quantity i ( , ) i i – Horror movies with small value – Comedy movies with large value Mean Shift Classification • Mean shift procedure has been shown to have excellent properties for clustering and mode-detection with real data Xi: video features hi: their bandwidth parameters Action + drama drama Comedy+ drama comedy Action+ comedy horror Results • Conduct 101 film previews obtained from the Apple website • The total number of outliers in the final classification was 17 and 83% genre classification accurate Conclusions • Propose a method to perform genre classification of previews using low-level computable features • Classification is performed using mean shift clustering in the 4-D feature space of average shot length, color variance, motion content, and the lighting key