𝐏𝐫𝐢𝐧𝐜𝐢𝐩𝐚𝐥 𝐂𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬 (𝐏𝐂𝐀), was one of the earliest methods used to 𝐝𝐞𝐭𝐞𝐫𝐦𝐢𝐧𝐞 𝐭𝐡𝐞 𝐜𝐨𝐫𝐫𝐞𝐥𝐚𝐭𝐢𝐨𝐧𝐬 among data and to reduce the dimensionality of the space, 𝑿, by projecting it into a lower dimensional space 𝒁.

𝐏𝐂𝐀 attempts to find a mapping 𝒇 that preserves, to the 𝐠𝐫𝐞𝐚𝐭𝐞𝐬𝐭 𝐞𝐱𝐭𝐞𝐧𝐭 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞, 𝐭𝐡𝐞 𝐯𝐚𝐫𝐢𝐚𝐧𝐜𝐞 of the data points once they are projected into the reduced space 𝒁. 𝑿 can be taken to 𝐛𝐞 𝐚 𝐳𝐞𝐫𝐨-𝐦𝐞𝐚𝐧 𝐝𝐚𝐭, corresponding to 𝑿 − µ𝑿.

Because the 𝐯𝐚𝐫𝐢𝐚𝐧𝐜𝐞 of a feature can be 𝐢𝐧𝐟𝐥𝐮𝐞𝐧𝐜𝐞𝐝 𝐛𝐲 𝐭𝐡𝐞 𝐦𝐞𝐚𝐬𝐮𝐫𝐞𝐦𝐞𝐧𝐭 𝐮𝐧𝐢𝐭𝐬 used to express it, a commonly used preprocessing procedure for 𝑿 is normalization.

𝐏𝐂𝐀 is applied in various domains, and its 𝐦𝐚𝐭𝐡𝐞𝐦𝐚𝐭𝐢𝐜𝐚𝐥 𝐚𝐧𝐝 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 properties can be derived.

𝐌𝐚𝐭𝐡𝐞𝐦𝐚𝐭𝐢𝐜𝐚𝐥𝐥𝐲, in PCA, the mapping 𝒇 generates a 𝐥𝐢𝐧𝐞𝐚𝐫 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐦𝐚𝐭𝐫𝐢𝐱 𝑼, whose columns map each data vector 𝒙 to corresponding elements 𝒛, 𝐜𝐚𝐥𝐥𝐞𝐝 𝐭𝐡𝐞 𝐩𝐫𝐢𝐧𝐜𝐢𝐩𝐚𝐥 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭𝐬 of 𝒙; 𝒁= 𝑿𝑼.

Because the columns of 𝑼 are 𝐜𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐞𝐝 𝐭𝐨 𝐛𝐞 𝐮𝐧𝐢𝐭 𝐯𝐞𝐜𝐭𝐨𝐫𝐬 that are normal to each other, they constitute an 𝐨𝐫𝐭𝐡𝐨𝐧𝐨𝐫𝐦𝐚𝐥 𝐛𝐚𝐬𝐢𝐬 for 𝒁.

The 𝐨𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐢𝐬 𝐜𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐞𝐝 such that the variances of the transformed dataset along the 𝑼 basis directions 𝐚𝐫𝐞 𝐬𝐨𝐫𝐭𝐞𝐝 𝐢𝐧 𝐝𝐞𝐜𝐫𝐞𝐚𝐬𝐢𝐧𝐠 𝐨𝐫𝐝𝐞𝐫 w.r.t. the basis element order, i.e., the 𝐟𝐢𝐫𝐬𝐭 𝐜𝐨𝐥𝐮𝐦𝐧 of 𝑼 matrix corresponds to the 𝐡𝐢𝐠𝐡𝐞𝐬𝐭 𝐞𝐢𝐠𝐞𝐧𝐯𝐚𝐥𝐮𝐞 of 𝑪𝑿, and so on.

In the following figure, 𝐭𝐰𝐨 𝐩𝐫𝐢𝐧𝐜𝐢𝐩𝐚𝐥 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭𝐬 𝐚𝐫𝐞 𝐭𝐚𝐤𝐞𝐧 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐝𝐞𝐜𝐨𝐦𝐩𝐨𝐬𝐢𝐭𝐢𝐨𝐧 𝐨𝐟 𝐭𝐡𝐞 𝐜𝐨𝐯𝐚𝐫𝐢𝐚𝐧𝐜𝐞 𝐦𝐚𝐭𝐫𝐢𝐱, where the 𝐩𝐮𝐫𝐩𝐥𝐞 𝐥𝐢𝐧𝐞 arrow is the 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 of 𝐭𝐡𝐞 𝐟𝐢𝐫𝐬𝐭 𝐩𝐫𝐢𝐧𝐜𝐢𝐩𝐚𝐥 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 which is corresponded to the 𝐥𝐚𝐫𝐠𝐞𝐬𝐭 𝐞𝐢𝐠𝐞𝐧𝐯𝐚𝐥𝐮𝐞 (λ = 9).

PCA can also be derived 𝐨𝐧 𝐭𝐡𝐞 𝐛𝐚𝐬𝐢𝐬 𝐨𝐟 𝐠𝐞𝐨𝐦𝐞𝐭𝐫𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐬𝐢𝐝𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬. Here, the process can be 𝐢𝐭𝐞𝐫𝐚𝐭𝐢𝐯𝐞𝐥𝐲 described as 𝐟𝐢𝐧𝐝𝐢𝐧𝐠 𝐨𝐧𝐞 𝐨𝐫𝐭𝐡𝐨𝐧𝐨𝐫𝐦𝐚𝐥 𝐛𝐚𝐬𝐢𝐬 𝐜𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭, 𝒖𝒊, at a time. The 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 of the first component, 𝐮1, is defined by the 𝐛𝐞𝐬𝐭-𝐟𝐢𝐭 𝐥𝐢𝐧𝐞 𝐰𝐡𝐢𝐜𝐡 𝐩𝐚𝐬𝐬𝐞𝐬 𝐭𝐡𝐫𝐨𝐮𝐠𝐡 𝐭𝐡𝐞 𝐨𝐫𝐢𝐠𝐢𝐧.

𝐒𝐞𝐯𝐞𝐫𝐚𝐥 𝐞𝐱𝐭𝐞𝐧𝐬𝐢𝐨𝐧𝐬 𝐨𝐟 𝐭𝐡𝐞 𝐏𝐂𝐀 methods have been reported in the literature, among the others 𝐃𝐮𝐚𝐥 𝐏𝐂𝐀, 𝐤𝐞𝐫𝐧𝐞𝐥 𝐏𝐂𝐀, and 𝐩𝐫𝐨𝐛𝐚𝐛𝐢𝐥𝐢𝐬𝐭𝐢𝐜 𝐏𝐂𝐀 have been developed.

𝓑𝓎: 𝓜𝓸𝒽𝒶𝓃𝒶𝒹 𝒜𝒷𝓊𝓀𝓂𝓮𝒾𝓁 𝒫𝒽.𝒟. 𝓮𝓍𝒸𝓮𝓇𝓅𝓉𝓈