Algebra II

Linear algebra and Principal Component Analysis

Principal Component Analysis and Alg II

12h, 6x2h par semaine

Illustration of Principal Component Analysis (PCA) on sentences, represented as Gaussian distributions with diagonal covariance matrices. Deudon, Michel. [Learning semantic similarity in a continuous space](https://papers.nips.cc/paper_files/paper/2018/file/97e8527feaf77a97fc38f34216141515-Paper.pdf). Advances in neural information processing systems 31 (2018).
Illustration of Principal Component Analysis (PCA) on sentences, represented as Gaussian distributions with diagonal covariance matrices. Deudon, Michel. Learning semantic similarity in a continuous space. Advances in neural information processing systems 31 (2018).

Course

ACP et reduction de dimension.

1. Definitions and notations

2 PCA from 2D to 1D

3 PCA in 3D and more

3.1 Formulation of the problem
3.2 Decomposition of matrices
3.3 Back to PCA and lower rank approximation

4 PCA in practice

4.1 Data preprocessing
4.2 Pseudo code
4.3 Concept of similarity

5 Other Matrix Factorization Methods

6 Preservation of distances

Conclusion

TP / Pratique

te62mi-algebre-bilineaire

TP1. SVD.

TP2. PCA.

TP3. MDS.

TD / Exercices

Coming soon.

References

    1. R. Bellman. The curse of dimensionality. Princeton. 1961.
    1. P. Comon. Independent component analysis, a new concept? Signal processing. 1994.
    1. P. Pentti et U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics. 1994.
    1. A. J. Bell et T. J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural computation. 1995.
    1. A. Hyvärinen et O. Erkki. Independent component analysis: algorithms and applications. Neural networks. 2000.
    1. J. Mairal, F. Bach et al. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research. 2010.
    1. F. Pedregosa, G. Varoquaux et al. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research. 2011.
    1. C.R. Harris, K.J. Millman et al. Array programming with NumPy. Nature. 2020.
linear algebra dimensionality reduction principal component analysis matrix factorization singular value decomposition
Next

Last updated on Jul 25, 2023