C4: Activation Functions Gallery

Compare functions, derivatives, and the vanishing gradient problem

Functions
Display Options
Show Derivatives Only
Show Both (f and f')
Function Comparison
x = 0.00
Chain Configuration
Number of Layers 10
Initial Input (x₀) 0.5
Final Gradient (after N layers)
⚠️ Gradient has vanished!
How it works:
The gradient at layer N is computed as:
∏ f'(xᵢ) where xᵢ₊₁ = f(xᵢ)

For sigmoid/tanh, f'(x) < 1 always, so gradients shrink exponentially. ReLU maintains f'(x) = 1 for positive inputs.
Gradient Magnitude vs Layer Depth (Log Scale)