Overview
I wanted to understand how Denoising Diffusion Probabilistic Models (DDPMs) generate images, so I built a small one from scratch.
This was a learning project rather than a polished library. The repository contains the main experiments and results, but some exploratory code is incomplete or only lightly documented.
Results
I first trained the model on MNIST, a dataset of handwritten digits. Its simple images made it a useful starting point. The model is unconditional and does not accept a text prompt.
I later used the sprite dataset from How Diffusion Models Work. It is small, but its images are more complex than MNIST.
Many images in the dataset repeat with only slight variations. The model therefore often generated images close to the training examples, which suggests overfitting. A more varied dataset would likely reduce this problem.
Project details
I began with a simple experiment: add noise to an image, then train a neural network to remove it. I did not yet understand the complete DDPM process, and the results were limited, but the experiment helped me identify what I needed to study next.
The final implementation is based on the principles outlined in the paper Denoising Diffusion Probabilistic Models. I also referred to several implementations available online.
Pitfalls
The forward and reverse diffusion processes were not difficult to understand on their own. The implementation became harder when it required details such as adding noise during denoising. Without them, the model did not learn to generate usable images.
The noise schedule, learning rate, and model architecture all affected the output. I tested several configurations because early models produced blurry images or repeatedly generated the same image, indicating mode collapse.
Training also required more compute than I had locally. I reduced the architecture and dataset sizes, optimized the training process, and still rented cloud GPU time to finish some runs in a reasonable time.