(1) Right Component
(2) Left Component
(3) Front Component
(4) Top Component
(5) Back Component
This project attempts to determine the direction of light on a face from a single picture. Two approaches were taken for this project: detection through Principal Component Analysis and Gradient Direction Histogram comparison.
Principal Component Analysis was performed on a set of six faces under five different lighting angles. The chosen angles were front, left side, right side, top, and back. This data set can be viewed here. All faces were labeled with key points by hand and morphed to the mean face before PCA was performed. The analysis was able to determine that the five most important components were the different directions. Additional components started differentiating between facial features.
(1) Right Component
(2) Left Component
(3) Front Component
(4) Top Component
(5) Back Component
These components were saved for use later. Next, the average of the faces in each of the five different lighting conditions was taken.
(1) Right Average
(2) Left Average
(3) Front Average
(4) Top Average
(5) Back Average
These averages were projected into the PCA space and their coordinates were saved. To detect the direction of light, the test face is morphed to the mean face of the PCA set, projected into the PCA space, and the closest point of the five angles to the projected coordinate was determined to be the primary light direction.
Histogram for the Right Average, threshold 0.85
An alternate approach was also investigated. This approach also takes a test image that has been warped to the mean face, but instead of projecting it into the PCA space, a histogram of the gradient diection is computed for it. The direction is only added to the histogram is its magnitude is greater than a threshold value. The magnitudes are normalized to the [0.0, 1.0] range before thresholding. The histogram uses bins in the range from -180 to 180 separated by 30. Similar histograms are also computed for the average faces for each of the lighting angles. All histograms are normazlied, then the test image histogram is compared to the average histograms using the Chi Square distance metric and the light angle with the smallest distance is determined to be the primary light direction.
Test photos were drawn from google searches and one was from my portfolio. Values in parentheses are distance measures returned by the algorithm. Smaller is better.
PCA
Ranking: front (158), top (184), right (186), left (191), back (210)
Histogram Distance
Ranking: front (0.692), back (0.811), right (0.946), top (.949), left (0.955)
PCA
Ranking: left (93.1), right (113), top (126), back (138), right (145)
Histogram Distance
Ranking: left (0.223), front (0.744), back (0.824), right (1), top (1)
PCA
Ranking: right (169), front (177), top (191), back (218), left (220)
Histogram Distance
Ranking: right (0.353), front (0.751), top (8.63), back (0.959), left (1)
PCA
Ranking: left (60.2), front (60.8), top (76.2), right (97.4), back (98.0)
Histogram Distance - Failure Case
Ranking: front (0.188), right (0.379), back (0.386), top (0.388), left (0.649)
The brightness of the image may have contributed the the complete failure of the histogram distance metric.
PCA
Ranking: front (155), right (164), top (174), left (201), back (205)
Histogram Distance
Ranking: top (0.676), back (0.691), front (0.867), right (0.986), left (1)
The light in this image is primarily from the right, however there is a lot of spill onto the center part of the face, which tilts the PCA analysis towards front light.
PCA
Ranking: front (65.3), right (85), top (89.3), left (99.7), back (111)
Histogram Distance
Ranking: right (0.161), top (0.337), front (0.514), back (0.573), left (0.845)
I suspect that the front light illuminating the face is throwing off the criteria for the back light in the PCA algorithm.