Distribution rules of weights of binary neural networks

In the previous experiments, the rules of weight distribution of binary neural network are obtained ：

Distribute by column ,

Positive and negative symmetry ,

A0 Column or B0 The column height is prominent , And A0 Column or B0 The positive and negative relationships between columns are opposite

coincidence 1 The height of the column is small , And the positive and negative relationship is A0 Column or B0 The column is opposite

coincidence 0 The column does not change

A0 Column ,B0 Column , coincidence 1 Column , coincidence 0 The columns are almost consistent with each other .

Test this rule this time

(A,B)---9*9*2---(1,0)(0,1)

Classify with neural network A and B,A Yes 6 individual 1,B Yes 5 individual 1. Export convergence weights , Observe the relationship between training set and weight .

for the first time

convergence w1

Draw a picture

Focus on training

A0 As a x=0,6

B0 As a x=1,2,5,

AB Coincident 1 Column x=3,7,8

AB Coincident 0 Column x=4

It is obvious that , The first 1,2,5 The column distribution is consistent , The first 0,6 The column distribution pattern is consistent , And with the third 1,2,5 The positive and negative of the column are opposite . The first 3,7,8 The columns are distributed uniformly , And with the third 1,2,5 The column distribution is positive and negative .

The initialization weight of this network w1 export

Compare initial weights w1 And convergence weight w1 Of the 4 Column , They are consistent . therefore AB Coincident 0 Column weights do not change , This is consistent with the rules .

Do the second group

The initialization of the w1

Convergent w1

Will converge w1 Draw a picture

X=1,2,5 The column distribution corresponds to B0 Column ,x=0,6 The column distribution corresponds to A0 Column , And with the B0 The positive and negative relations of columns are opposite .

The two weights are 4 Column consistency , There is no change , accord with AB coincidence 0 Rules for columns . and x=3,7,8 The distribution is consistent , And with the B0 The positive and negative relationships of columns are opposite , Corresponding AB coincidence 1 Column .

According to the shift distance and assumption ,A0,B0 The column corresponds to the shift particle , Is active column , Therefore, the amplitude is large . and AB coincidence 1 Column , Is the destination column of the shift particle , Also passively participated in the shift , Therefore, there is movement but the amplitude is not obvious . and AB coincidence 0 Column neither corresponds to shift particle nor destination column , Therefore, he did not participate in the displacement movement , The amplitude is 0 There is no change .

Therefore, the experiment shows that the weight convergence form can be constructed from the training set based on several clear rules .