Interface idempotency

Interface idempotence

One 、 Definition

idempotent （idempotent、idempotence） It is a concept of mathematics and computer science , Common in abstract algebra .

The characteristic of an idempotent operation in programming is that any multiple execution has the same effect as one execution .

Idempotent function , Or idempotent method , It means that the same parameters can be used for repeated execution , And can get the same result function . These functions do not affect the state of the system , You don't have to worry about repeated execution that will change the system .

for example ,“setTrue()” A function is an idempotent function , No matter how many times , The results are the same . More complex operation idempotent guarantee is to use unique transaction number ( Serial number ) Realization .

This is the explanation from Baidu Encyclopedia , Speaking of idempotency means Interface can be called repeatedly , In the case of multiple calls by the caller , The final result of the interface is consistent .

Two 、 Use scenarios

For the excuse of query , Idempotency can be naturally realized , Because no matter how many times you query, the result is the same .

As for the excuse of adding, deleting and modifying , We also need to ensure the idempotency of its interface .

For example, when the user clicks several times to create a product , At this time, the front end will repeatedly request a new interface （ The background is required to perform a creation ）, At this time, the backend repeatedly creates multiple records , This is the result of not implementing interface idempotency bug, It is also a situation we need to avoid .

The following are the scenarios where interface idempotency can be used ：

1. The front end repeatedly submits the form ： When filling out some forms , User completes and submits , Most of the time, due to network fluctuations, there is no timely response to user submission , Causes the user to think that has not submitted successfully , And then keep clicking on the submit button , At this point, repeated submission of form requests will occur .
2. Users swipe orders maliciously ： For example, in the implementation of the user voting function , If a user repeatedly submits a vote against a user , This will cause the interface to receive voting information repeatedly submitted by users , This will make the voting result seriously inconsistent with the facts .
3. The interface timed out and submitted repeatedly ： A lot of times HTTP By default, client tools enable the timeout retry mechanism , Especially when a third party calls an interface , In order to prevent the request failure caused by network fluctuation, timeout, etc , Will add a retrial mechanism , Causes a request to be submitted multiple times .
4. Repeat consumption of news ： When using MQ When it comes to message middleware , If there is an error in the message middleware, the consumption information is not submitted in time , Lead to repeated consumption .

3、 ... and 、 influence

Idempotency is to simplify client logic processing , Can place repeated submission and other operations , But it increases the logical complexity and cost of the server , It is mainly ：

1. Change the function of parallel execution to serial execution , Reduced the efficiency of execution .
2. Added additional control idempotent business logic , Complex business functions ;

Therefore, it is necessary to consider whether to introduce idempotency when using it , According to the actual business scenario, specific analysis , In addition to special business requirements , In general, there is no need to introduce interface idempotence .

The greatest advantage of using idempotency is that the interface guarantees any idempotent operation , Avoid the unknown problems caused by system retrying

Four 、 Realization way

1. unique index ： Take advantage of the unique constraint of the primary key , Ensure that only one piece of data is inserted

   • Generally speaking, this primary key does not use the self incrementing primary key in the database , It's distributed ID Act as a primary key , Only in this way can we ensure that ID Global uniqueness of

2. Anti weight Token token ： The cluster environment adopts token+redis（redis It's single threaded , Processing needs to be queued ）, single JVM The environment adopts token+jvm Memory , For the continuous click of the client or the timeout retry of the caller

   • Apply to the backend before calling the interface token,token Put in redis( As key, User information as value) or jvm In the memory , And set the effective time
   • Submit the request with token, Verify whether the background exists token, as well as value match , If it exists and matches, the request succeeds and is deleted token, Otherwise, the request fails
   • perform redis There is a concurrency problem in the search and deletion of , In the case of concurrency , perform Redis Finding and deleting data needs to be atomic , You can use distributed locks or Lua Expression to unregister queries and delete operations .

3. The state mechanism is idempotent ： Change of state , Judge the status when updating data

   • If the state machine is already in the next state , At this time, there is a change of the previous state , In theory, it can't be changed
   • Electricity supplier order , Order payment status 0 To be paid ,1 In the payment ,3 Successful payment ,4 Failure to pay
   • update order set status = 1 where status =0 and orderId = “201251487987”
     The sql Statement uses state CAS Ensure that the operation is idempotent
   • For example, order payment , Come up and use CAS Update order status
     The number of returned influences is 1 It means that the modification was successful , Can pay for , Continue to execute the payment business code
     The number of returned influences is 0 Indicates that the modification failed , This order is no longer a pending order

4. Pessimistic locking ： When getting data Lock acquisition

   • Lock the table or row before the user requests to operate the database , Release the lock after finishing the operation
   • At this time, if a second request attempts to access the database, it will fail when trying to obtain the lock

5. Optimism lock ： It's just When updating data Lock table , Do not lock the meter at other times , So compared with pessimistic lock , More efficient

   • Add one more field in the corresponding data table in advance , Act as the version identifier of the current data
   • Every time the data of this table in this database is updated , Will use the version ID as a condition , The value is the value of the version ID in the last data to be updated , Ensure that the information corresponding to a certain version is updated
   • When updating repeatedly , In the second update, the version has changed , Therefore, it cannot be updated successfully to achieve idempotence

6. Distributed lock ： Through third party systems (redis or zookeeper), Insert data or update data in the business system , Acquire distributed lock , And then do it , Then release the lock

   • Examples of inserting data , If the distribution is a system , It's hard to build globally unique indexes , For example, a unique field cannot be determined , In this case, distributed locks can be introduced
   • A long process requires that it cannot be executed concurrently , You can follow a certain flag before the process is executed ( user ID+ Suffixes, etc ) Acquire distributed lock , Other processes fail to acquire locks when they execute , That is to say, only one of the processes can be successfully executed at the same time , After execution , Release distributed lock ( Distributed lock to be provided by third party system )

7. Cache queue ： Put the request in the queue , Subsequently, asynchronous tasks are used to process the data in the queue , Filter out duplicate messages . It's the same as preventing repeated consumption