- To sum it up, the logistic regression classifier has a non-linear activation function, but the weight coefficients of this model are essentially a linear combination, which is why logistic regression is a “generalized” linear model.
- We put the net input z through a non-linear “activation function” — the logistic sigmoid function where.
- In linear regression, we compute a linear combination of weights and inputs (let’s call this function the “net input function”).
- We only have 3 units in the input layer (x_0 = 1 for the bias unit, and x_1 and x_2 for the 2 features, respectively); there are 200 of these sigmoid activation functions (a_m) in the hidden layer and 1 sigmoid function in the output layer, which is then squashed through a unit step function (not shown) to produce the predicted output class label y^ .
- Let’s consider logistic regression.

Confused as to exactly what the activation function in a neural network does? Read this overview, and check out the handy cheat sheet at the end.

Continue reading “What is the Role of the Activation Function in a Neural Network?”