Softmax, explained: the idea behind the Inferenz logo

The Inferenz logo is a row of bars with one tall bar that doubles as the letter i. It's a picture of the softmax function, which sits at the end of almost every classifier and language model. Here is what it does.

From scores to probabilities

A neural network's last layer produces raw scores, called logits, one per option. A language model produces one for every token in its vocabulary. These numbers can be any size and sign, so they aren't probabilities yet. Softmax turns them into a probability distribution: every value between 0 and 1, all summing to 1.

softmax(z)_i = exp(z_i) / sum_j exp(z_j)

Exponentiating makes every score positive and exaggerates the differences, and dividing by the total normalises them. The biggest score gets the biggest share, but the others keep a little.

A tiny example

import numpy as np

def softmax(z):
    z = z - np.max(z)          # subtract the max for numerical stability
    e = np.exp(z)
    return e / e.sum()

logits = np.array([2.0, 1.0, 0.1])
print(softmax(logits))         # about [0.66, 0.24, 0.10]

Subtracting the maximum doesn't change the result but prevents overflow when scores are large.

Temperature: how confident to be

Divide the logits by a temperature T before the softmax. A low temperature sharpens the distribution toward the top choice, and a high temperature flattens it. This is the "temperature" setting you see in language model APIs.

def softmax_t(z, T=1.0):
    return softmax(np.asarray(z) / T)

softmax_t([2.0, 1.0, 0.1], T=0.5)   # sharper: the top option dominates
softmax_t([2.0, 1.0, 0.1], T=2.0)   # flatter: the options are closer

Picking an answer: argmax or sampling

Once you have the probabilities you can pick the most likely option (argmax), which is deterministic, or sample from the distribution, which gives variety. Language models usually sample, which is why the same prompt can produce different answers.

Why it's our logo

Inference is the moment a trained model turns an input into a decision, and softmax is the picture of that decision: many candidates, one that wins. The tall bar with the dot is the winning answer, and the shorter bars are the alternatives the model considered. The lowercase i is for Inferenz.

You can play with related ideas in the interactive studio on our home page.