Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Online Construction of Surface Light Fields By Greg Coombe, Chad Hantak, Anselmo Lastra, and Radek Grzeszczuk 1 Image-Based Modeling * Benefits: * Photorealistic content Modeling by acquisition Difficulties: Sampling issues Lack of feedback 2 Motivation Casual Capture Pick up a video camera, wave it around the model, and capture reflectance model 3 Surface Light Field A surface light field is a representation of the appearance of a model with known geometry and static lighting: (θ,Φ) surface position f ( , , u, v) (u,v) viewing direction 4 Our system Incremental system for online capture, construction, and rendering of surface light fields Each image is processed as it is captured Interactive feedback Guide user to undersampled regions Data-driven heuristic Structured method for handling missing data 5 Previous Work Regular parameterizations light field data [Levoy96, Gortler96] Sparse, scattered data [Debevec96, Debevec98, Buehler01] BRDF capture systems [Dana99, Lafortune97, Marschner99, Debevec01, Gardner03] Function fitting Lafortune BRDF [McAllister02], Torrance-Sparrow BRDF [Sato97], Clusters of BRDFs [Lensh01], Homomorphic Factorization [McCool?], Bi-quadratic polynomials [Malzbender01] Online Methods Fixed viewpoint, progressive refinement [Matusik04] Adaptive Meshing of light field [Schirmacher99] Streaming non-linear optimization [Hillesland04] Data Mining [Brand02, Roweis97] 6 SLFs using PCA Represent 4D function as a matrix Use Principal Component Analysis [Noshino01, Chen02] For each surface patch, break the 4D function into two 2D functions Store the principal components in texture maps rank f ( , , u, v) g ( , ) * h(u, v) 7 SLFs using PCA Problems: Requires that the entire data set be available at once. Difficult to locate undersampled regions Requires recomputation when new images are added SVD * 8 Online SVD Online SVD [Brand02] is a incremental PCA Update output matrices one sample at a time Advantages: Never store the entire data matrix at once Stream images ... Online SVD * 9 Online SVD Algorithm 1. Existing rank-r PCA A USV T 2. Project new image samples onto eigenspace m UTa p 3. Compute the orthogonal component j p a - Um 4. If ||p|| is less than a threshold, then we just compute the rotations and update the eigenspace. U’ = URu 5. V’ = VRv Otherwise, the current rank r is insufficient, so we append a new eigenvector. U’ = [U; m]Ru V’ = VRv 6. The rotations are computed by re-diagonalizing this matrix diag ( s ) m diagonalize [U ', S ',V '] p 0 10 Error Rank Reference PCA Online SVD 11 Imputation Matrix factorization approach requires fully resampled data matrices Missing data is a common problem Occlusion, meshing errors Imputation is the process of filling in these holes with “reasonable” guesses Better than zeros or mean Use the current Online SVD approximation to generate these missing values In practice, need about 5-10 initial images, and 50%+ coverage Missing data in red Imputed Further details in [Brand03] 12 Automatic Feedback Aid the user in capturing surface light fields Direct attention towards undersampled areas Use a data-driven quality heuristic E = sqrt( e0^2 + e1^2) e0 – variation over hemisphere e1 – variation over surface 13 System Video Camera Real Time Pose Estimation Capture PC Visibility* Resampling* Render PC Online SVD Rendering Quality Heuristic * From Intel’s OpenLF system 14 Results Captured image SLF before incorporating new image SLF after incorporating new image 15 Results 16 Timing 17 Conclusion We present a method with Incremental construction Data-driven quality heuristic Online SVD is well-suited for streaming model Combine distribution and solution Requires only one pass over data 18 Future Work We are still a long ways from “Casual Capture” Geometry must be known a priori Fixed illumination conditions Implementation issues Real-time pose estimation errors 19 20 Online SVD Problem: Multiple small rotations can accumulate error Brand proposes splitting output matrices to avoid accumulating error A SVD USV T A SVD UU ' SV 'V T Advantages: Due to small working set, most data in cache (fast) 21 Higher-Dimensional Problems We would like to extend Online SVD to higher-order factorizations Moveable light source Introduces another dimension of data f (1 ,1 , u, v,2 , 2 ) g (1 , 1 ) * h(u, v) *k (2 , 2 ) View maps Surface maps Light maps Requires tensor product expansion 22 2. Convergence Averaged over twenty random vertices 23 3. Sensitivity 24 Imputation Incomplete sample: (-1, ??) Existing SVD approximation Impute using SVD 25