Common algorithm interview has been out! Machine learning algorithm interview - KDnuggets

If the common algorithm is the common programmer's necessary knowledge , So is a more practical machine learning algorithm ？ Or is it a necessary knowledge for data scientists ？

In preparing for an interview in Data Science , It is necessary to have a clear understanding of the various machine learning models - Give a brief description of each ready-made model . ad locum , We summarize various machine learning models by highlighting the main points , To help you communicate complex models .

Linear regression  

Linear regression involves the use of the least square method to find “ Best fit line ”. The least squares method involves finding a linear equation , The equation minimizes the sum of squares of residuals . The residual is equal to the actual negative predictive value .

for instance , The red line is a better fit than the green line , Because it's closer to the point , So the residuals are small .

The picture was created by the author .

Ridge Return to

Ridge Return to （ Also known as L2 Regularization ） It's a regression technique , A small amount of deviation can be introduced to reduce over fitting . It works by minimizing the square of residuals   And plus   Penalty points to achieve this goal , The penalty is equal to λ Times the slope squared .Lambda It means the severity of the punishment .

The picture was created by the author .

If there is no punishment , Then the slope of the best fit line becomes steeper , That means it's good for X More sensitive to subtle changes in . By introducing punishment , Best fit line pairs X It becomes less sensitive . Back of the ridge return .

Lasso Return to

Lasso Return to , Also known as L1 Regularization , And Ridge Return to similar . The only difference is , The penalty is calculated using the absolute value of the slope .

Logical regression

Logistic Regression is a classification technique , You can also find “ The most suitable straight line ”. however , Unlike linear regression , In linear regression , Use the least square to find the best fit line , Logistic regression uses maximum likelihood to find the best fit line （ The logic curve ）. This is because y Value can only be 1 or 0.  watch StatQuest In the video , Learn how to calculate the maximum likelihood .

The picture was created by the author .

K Nearest neighbor

K Nearest neighbor is a classification technique , Classify the new samples by looking at the nearest classification point , So called “ K lately ”. In the following example , If k = 1, Then unclassified points are classified as blue dots .

The picture was created by the author .

If k The value of is too low , There may be outliers . however , If it's too high , It is possible to ignore classes with only a few samples .

Naive Bayes

Naive Bayes classifier is a classification technique inspired by Bayes theorem , The following equation is stated ：

Because of naive assumptions （ Hence the name ）, Variables are independent in the case of a given class , So it can be rewritten as follows P（X | y）：

Again , Because we have to solve y, therefore P（X） It's a constant , This means that we can remove it from the equation and introduce proportionality .

therefore , Each one y The probability of value is calculated as given y Conditional probability of x n The product of the .

Support vector machine

Support vector machine is a classification technique , We can find the best boundary called hyperplane , This boundary is used to separate different categories . Find hyperplanes by maximizing the margin between classes .

The picture was created by the author .

Decision tree

Decision tree is essentially a series of conditional statements , These conditional statements determine the path taken by the sample before it reaches the bottom . They are intuitive and easy to build , But it's often inaccurate .

Random forests

Random forest is an integrated technology , This means that it combines multiple models into one model to improve its predictive power . say concretely , It uses bootstrap data sets and random subsets of variables （ Also known as bagging ） Thousands of smaller decision trees have been built . With thousands of smaller decision trees , Random forest use “ The majority wins ” Model to determine the value of the target variable .

for example , If we create a decision tree , The third decision tree , It will predict 0. however , If we rely on all 4 A decision tree model , Then the predicted value will be 1. This is the power of random forests .

AdaBoost

AdaBoost It's an enhancement algorithm , Be similar to “ Random forests ”, But there are two important differences ：

1. AdaBoost It's not usually made up of trees , It's a forest of stumps （ A stump is a tree with only one node and two leaves ）.
2. The decision of each stump has a different weight in the final decision . The total error is small （ High accuracy ） The stump has a higher voice .
3. The order in which the stumps are created is important , Because each subsequent stump emphasizes the importance of samples that were not correctly classified in the previous stump .

Gradient rise

Gradient Boost And AdaBoost similar , Because it can build multiple trees , Each of these trees was built from the previous tree . And AdaBoost You can build stumps differently ,Gradient Boost Can be built, usually with 8 to 32 A leafy tree .

what's more ,Gradient Boost And AdaBoost The difference is in the way decision trees are constructed . Gradient enhancement starts with the initial prediction , It's usually the average . then , The decision tree is constructed based on the residuals of samples . By using the initial prediction + The learning rate is multiplied by the result of the residual tree to make a new prediction , Then repeat the process .

XGBoost

XGBoost Essentially with Gradient Boost identical , But the main difference is how to construct the residual tree . Use XGBoost, The residual tree can be determined by calculating the similarity score between the leaf and the previous node , To determine which variables are used as roots and nodes .