Review of classification

Step 1 function set

Step 2:Goodeness of function

Suppose it comes from a set of posterior probabilities training data, A certain group w and b Generate a set N individual training data Probability


Is most likely w and b, Is the biggest L, namely

Convert the determined maximum to the minimum , namely

take Ln Can simplify the calculation , We often see , This will change multiplication into addition

y hat Indicates the conclusion of correct classification ,1 Corresponding class 1 0 Corresponding class 2

Put in the above formula



The cross entropy represents how close the two distributions are
Compare the two regression, as follows

Find the best function

use loss function Yes w And for b Partial differential of

Treat the second item in the same way

Yes w The partial differential of is

When updating this parameter ：

It is related to three parameters ,xi Is the sample , The green underline indicates the goal and the present model Of output The gap between , Learning rate
larger difference, large update

Limitation of Logistic Regression

Samples like this , adopt logistic Got boundary It's a straight line , So it's hard to separate two categories .
Solutions to problems ：Feature Transformation, The way to change with the help of a machine is cascading logistic regression models

Map the original feature plane to another feature plane , Adjustment here w and b, Easy to use one boundary Distinguish between different kinds