当前位置：网站首页>Best practices of haproxy in tidb

Best practices of haproxy in tidb

2022-07-22 13:39:00 【Tianxiang shop】

In this paper, HAProxy stay TiDB The best configuration and usage in .HAProxy Provide TCP Load balancing capability under the Protocol ,TiDB The client connects HAProxy Float provided IP You can operate the data , Realization TiDB Server Layer load balancing .

HAProxy brief introduction

HAProxy By C Free and open source software written in , Based on TCP and HTTP The application of the protocol provides high availability 、 Load balancing and proxy services . because HAProxy Can fast 、 Use it efficiently CPU And memory , So it is widely used at present , Many well-known websites such as GitHub、Bitbucket、Stack Overflow、Reddit、Tumblr、Twitter and Tuenti And Amazon Web service system HAProxy.

HAProxy from Linux Core contributors to the kernel Willy Tarreau On 2000 Year compilation , He is still in charge of the maintenance of the project , And provide version iteration free of charge in the open source community . This article uses HAProxy 2.5.0. It is recommended to use the latest stable version HAProxy, For details, see Published HAProxy edition .

HAProxy Introduction to some core functions

High availability ：HAProxy Provides elegant shut down services and seamless switching of highly available features ;
Load balancing ：L4 (TCP) and L7 (HTTP) Two load balancing modes , At least 9 Class equalization algorithm , such as roundrobin,leastconn,random etc. ;
health examination ： Yes HAProxy Configured HTTP perhaps TCP Check the mode status ;
Conversation retention ： In cases where the application does not provide session persistence ,HAProxy This function can be provided ;
SSL： Support HTTPS Communication and parsing ;
Monitoring and statistics ： adopt web The page can monitor the service status and specific traffic information in real time .

Prepare the environment

In the deployment HAProxy Before , The following environment should be prepared .

Hardware requirements

According to official documents , Yes HAProxy The following recommendations are provided for the server hardware configuration of , It can also be calculated according to the load balancing environment , On this basis, improve the server configuration .

Hardware resources	Minimum configuration
CPU	2 nucleus ,3.5 GHz
Memory	16 GB
storage capacity	50 GB（SATA disc ）
network card	Wan Zhao nic

Dependent software

according to HAProxy Official documents , There are the following suggestions for the operating system and dependent packages , If you pass yum Source deployment installation HAProxy Software , Dependent packages do not need to be installed separately .

operating system

Linux operating system	edition
Red Hat Enterprise Linux	7 perhaps 8
CentOS	7 perhaps 8
Oracle Enterprise Linux	7 perhaps 8
Ubuntu LTS	18.04 Or above

Be careful

Other operating system support , See HAProxy file .

Dependency package

epel-release
gcc
systemd-devel

Execute the following command to install the dependent package ：

yum -y install epel-release gcc systemd-devel

Deploy HAProxy

HAProxy To configure Database The load balancing scenario is easy to operate , The following deployment operations are universal , Not special , It is suggested that according to the actual situation , Personalized configuration related The configuration file .

install HAProxy

download HAProxy 2.5.0 The source package ：
wget https://github.com/haproxy/haproxy/archive/refs/tags/v2.5.0.zip
Decompress the source package ：
unzip v2.5.0.zip
Compile from source HAProxy application ：
cd haproxy-2.5.0 make clean make -j 8 TARGET=linux-glibc USE_THREAD=1 make PREFIX=${/app/haproxy} SBINDIR=${/app/haproxy/bin} install # take `${/app/haproxy}` and `${/app/haproxy/bin}` Replace with the customized actual path .
Reconfiguration profile file ：
echo 'export PATH=/app/haproxy:$PATH' >> /etc/profile
Check HAProxy Is the installation successful ：
which haproxy

HAProxy Command Introduction

Execute the following command to check the command line parameters and basic usage ：

haproxy --help

Parameters	explain
`-v`	Display brief version information .
`-vv`	Show detailed version information .
`-d`	Turn on debug Pattern .
`-db`	Disable background mode and multi process mode .
`-dM [<byte>]`	Execute allocate memory .
`-V`	The startup process displays configuration and polling information .
`-D`	Turn on the daemons mode .
`-C <dir>`	Change the directory location to... Before loading the configuration file `<dir>`.
`-W`	A master-slave mode .
`-q`	silent mode , Don't output information .
`-c`	Just check the configuration file and exit before attempting to bind .
`-n <limit>`	Set the maximum total number of connections per process to `<limit>`.
`-m <limit>`	Set the maximum available memory for all processes to `<limit>`（ Company ：MB）.
`-N <limit>`	Set the maximum number of single point connections to `<limit>`, The default is 2000.
`-L <name>`	Change the local instance peer name to `<name>`, The default is the local host name .
`-p <file>`	take HAProxy Of all subprocesses PID iw `<file>`.
`-de`	No use epoll(7),epoll(7) Only in Linux 2.6 And some customized Linux 2.4 Available on the system .
`-dp`	No use epoll(2), Switch to select(2).
`-dS`	No use splice(2),splice(2) In some old versions Linux Not available on the kernel .
`-dR`	No use SO_REUSEPORT.
`-dr`	Ignoring server address resolution failed .
`-dV`	Do not use... On the server side SSL.
`-sf <pidlist>`	After starting , towards pidlist Medium PID send out `finish` The signal , The process receiving this signal will wait for all sessions to complete before exiting , That is, stop serving gracefully . This option must specify at the end , Any number of PID. Technically speaking ,SIGTTOU and SIGUSR1 All sent .
`-st <pidlist>`	After starting , towards pidlist Medium PID send out `terminate` The signal , The process receiving this signal will immediately terminate , Close all active sessions . This option must specify at the end , Any number of PID. Technically speaking ,SIGTTOU and SIGTERM All sent .
`-x <unix_socket>`	Connect the specified socket And get all from the old process listening socket, then , Use these socket Instead of binding new .
`-S <bind>[,<bind_options>...]`	In master-slave mode , Create a binding to the main process socket, this socket Access to each child process socket.

More information about HAProxy Information about command parameters , Please see the Management Guide of HAProxy and General Commands Manual of HAProxy.

To configure HAProxy

yum The configuration template will be generated during installation , You can also customize the following configuration items according to the actual scene .

global # Global configuration . log 127.0.0.1 local2 # Defining the global syslog The server , You can define up to two . chroot /var/lib/haproxy # Change the current directory and set superuser permissions for the startup process , To improve security . pidfile /var/run/haproxy.pid # take HAProxy Process PID write in pidfile. maxconn 4096 # Single HAProxy The maximum number of concurrent connections that the process can accept , Equivalent to command line arguments "-n". nbthread 48 # Maximum number of threads . The maximum number of threads is the same as CPU The same number . user haproxy # Same as UID Parameters . group haproxy # Same as GID Parameters , A dedicated user group is recommended . daemon # Give Way HAProxy Work in the background as a daemon , Equivalent to command line arguments “-D” The function of . Of course , You can also use the “-db” Parameter to disable it . stats socket /var/lib/haproxy/stats # Statistics save location . defaults # The default configuration . log global # The log inherits the settings of the global configuration section . retries 2 # The maximum number of attempts to connect to the upstream server , Beyond this value, the back-end server is considered unavailable . timeout connect 2s # HAProxy Timeout for connection with back-end server . If you are in the same LAN , It can be set to a shorter time . timeout client 30000s # The client and HAProxy After connection , Data transmission completed , That is, the timeout for inactive connections . timeout server 30000s # Timeout of server-side inactive connection . listen admin_stats # frontend and backend The combination of , The name of this monitoring group can be customized as needed . bind 0.0.0.0:8080 # Listening port . mode http # Monitor the mode of operation , Here is `http` Pattern . option httplog # Start enabling logging HTTP Requested logging capabilities . maxconn 10 # Maximum number of concurrent connections . stats refresh 30s # every other 30 Seconds to automatically refresh the monitoring page . stats uri /haproxy # Monitor the URL. stats realm HAProxy # The prompt information on the monitoring page . stats auth admin:pingcap123 # Monitor the user and password of the page , Multiple user names can be set . stats hide-version # Hide... On the monitoring page HAProxy Version information . stats admin if TRUE # Enable or disable the backend server manually （HAProxy 1.4.9 And later versions support ）. listen tidb-cluster # To configure database Load balancing . bind 0.0.0.0:3390 # float IP and Listening port . mode tcp # HAProxy To use page 4 Transport layer of layer . balance leastconn # The server with the least number of connections has priority in receiving connections .`leastconn` Recommended for long session services , for example LDAP、SQL、TSE etc. , Instead of the short session protocol , Such as HTTP. The algorithm is dynamic , For slow start servers , The server weight will be adjusted during operation . server tidb-1 10.9.18.229:4000 check inter 2000 rise 2 fall 3 # testing 4000 port , The detection frequency is per 2000 One millisecond . If 2 This test is successful , The server is considered available ; If 3 The first detection is failed , The server is considered unavailable . server tidb-2 10.9.39.208:4000 check inter 2000 rise 2 fall 3 server tidb-3 10.9.64.166:4000 check inter 2000 rise 2 fall 3

If you want to pass SHOW PROCESSLIST Check the connection source IP, Need to be configured to use PROXY agreement Connect TiDB.

server tidb-1 10.9.18.229:4000 send-proxy check inter 2000 rise 2 fall 3 server tidb-2 10.9.39.208:4000 send-proxy check inter 2000 rise 2 fall 3 server tidb-3 10.9.64.166:4000 send-proxy check inter 2000 rise 2 fall 3

Be careful

Use PROXY When the agreement , You need to tidb-server Of proxy-protocol.networks Add the corresponding parameters in the configuration file .