Restricted Boltzmann machine (RBM)

RBMs are energy-based models, meaning they define a probability distribution through an energy function. They consist of two layers: a visible layer representing the input data and a hidden layer capturing latent features. The key restriction—hence 'restricted'—is that units within the same layer are not connected, simplifying the learning process. Learning in RBMs typically involves Contrastive Divergence, an algorithm that approximates the gradient of the log-likelihood function. This algorithm aims to adjust the weights connecting visible and hidden units so that the energy function reflects the data distribution. RBMs can be stacked to form Deep Belief Networks (DBNs) and used for dimensionality reduction, feature learning, collaborative filtering, and pre-training deep neural networks. Though less commonly used directly in modern deep learning architectures, the principles behind RBMs contributed significantly to the development of more complex generative models and unsupervised learning techniques.