Generative models

Generative models are a class of artificial intelligence systems designed to learn the underlying joint probability distribution of a dataset, allowing them to simulate how data is generated in the real world. Unlike discriminative models, which focus on predicting a label given an observation, generative models solve the more general problem of modeling all variables to create new, original content that resembles the training data.

The primary distinction in machine learning is between generative and discriminative modeling

Generative Models: These models describe the full data-generating process, typically modeling P(X,Y) or the distribution of inputs within each class, P(X∣Y). This enables them to generate random instances of an observation.

Discriminative Models : These models focus on P(Y∣X), mapping inputs directly to labels to categorize signals without caring how the data was generated.

The roots of generative AI trace back to Markov Chains in the late 1950s, which used statistical models to generate sequences of data. While neural networks gained traction in the 1990s and 2000s as hardware improved, the field was reshaped by pivotal milestones such as WaveNet (2016) for audio, GPT models (2019-2023) for text, and DALL-E (2022) for image generation.