Industry news

Research Debt
Research Debt

Achieving a research-level understanding of most topics is like climbing a mountain. Aspiring researchers must struggle to understand vast bodies of work that came before them, to learn techniques, and…

Learn more
Why Momentum Really Works
Why Momentum Really Works

Step-size α = 0.02 Momentum β = 0.99 We often think of Momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. But it…

Learn more
Feature Visualization
Feature Visualization

There is a growing sense that neural networks need to be interpretable to humans. The field of neural network interpretability has formed in response to these concerns. As it matures,…

Learn more
Sequence Modeling with CTC
Sequence Modeling with CTC

Introduction Consider speech recognition. We have a dataset of audio clips and corresponding transcripts. Unfortunately, we don’t know how the characters in the transcript align to the audio. This makes…

Learn more