当前位置：网站首页>Machine learning by Li Hongyi 5 Tips for neural network design

Machine learning by Li Hongyi 5 Tips for neural network design

2022-07-22 08:50:00 【InfoQ】

Author：AXYZdong

Automation Engineering Male

A little bit of thinking , Have a little idea , A little bit rational ！

Set a small goal , Try to be a habit ！ Meet a better self in the most beautiful years ！

One 、 Local minimum and saddle point

Optimization Fails because .....

local minima（ Local minimum ）

saddle point（ saddle point ）

Two 、 Batch and momentum

Optimization with Batch

Gradient Descent + Momentum

3、 ... and 、 Automatically adjust the learning rate

RMSProp(root mean square prop, Root mean square )

Learning principles ∶ The attenuation factor is introduced on the basis of adaptive gradient , When the gradient accumulates , Would be right “ In the past ” And “ Now? ” Make a balance , Adjust the attenuation through super parameters . Suitable for dealing with non-stationary targets （ That is, time related ), about RNN The effect is very good .

Adam(Adaptive momentum optimization, Adaptive momentum optimization )

It is the most popular optimization method in deep learning , It combines the advantages that adaptive gradient is good at dealing with sparse gradient and root mean square is good at dealing with non-stationary targets , It is suitable for large data sets and high-dimensional space .

Four 、 The effect of loss function

Mainly the impact on classification tasks .

5、 ... and 、 Batch standardization

Batch standardization （Batch Normalization）, Change the range of different features （changing landscape）.

6、 ... and 、 summary

Datawhale Team learning , Li Hongyi 《 machine learning 》Task5. Tips for neural network design（ Neural network design skills ）. Including local minimum and saddle point 、 Batch and momentum 、 Automatically adjust the learning rate 、 Effect of loss function and batch standardization . Mr. Li clearly explained the principles of mathematics in his course , However, the emphasis on application can be ignored , Know the design skills .

This sharing is here

If my article helps you 、 If you like the content of my article , please

“ give the thumbs-up ” “ Comment on ” “ Collection ”

One button, three links ！

hear

give the thumbs-up

It's not that bad luck for people who are , Every day will be full of vitality ！^ _ ^

It's not easy to code words , Everyone's support is my driving force to stick to it . Don't forget to like it

Focus on

I oh ！

If any of the above is inaccurate , Welcome to leave a message below . Or you have a better idea , Welcome to exchange and study together ~~~

原网站