Core concepts of elastic search learning Introduction (4)

​​​​1.1 Cluster

Represents a cluster , There are multiple nodes in the cluster , One of them is the main node , This master node can be elected , The primary and secondary nodes are internal to the cluster .ES One of the concepts is decentralization , Literally, it's a centerless node , This is for the outside of the cluster , Because from the outside ES colony , Logically it's a whole , You communicate with any node and with the whole ES Cluster communication is equivalent .

The primary node is responsible for managing the cluster state , This includes managing the state of shards and replicas , And the discovery and deletion of nodes .

You only need to start multiple within the same network segment ES node , It can automatically form a cluster .

By default ES Nodes in the same network segment will be found automatically , Automatically form a cluster .

Viewing status of the cluster ： http://hadoop01:9200/_cluster/health?pretty

1.2 shards

For index fragmentation ,ES A complete index can be divided into multiple slices , The advantage is that you can split a large index into multiple indexes , Distributed to different nodes , Form a distributed search . The number of slices can only be specified before index creation , And the index cannot be changed after creation .

You can specify... When creating the index library

curl -XPUT 'localhost:9200/test1/' -d '{"settings":{"number_of_shards":3}}'

The default is an index library with 5 A shard ：    index.number_of_shards:5

1.3 replicas

For index copy ,ES You can set a copy for the index , The first function of replica is to improve the fault tolerance of the system , A node can recover from a replica when a partition is damaged or lost . The second is to improve ES The query efficiency of ,ES Will automatically load balance the search request .

You can specify... When creating the index library

curl -XPUT 'localhost:9200/test2/' -d'{"settings":{"number_of_replicas":2}}'

The default is a partition with 1 Copies ：    index.number_of_replicas:1

1.4 recovery

Represents data recovery or data redistribution ,ES When a node is added or exited, index fragmentation will be reallocated according to the load of the machine , Data recovery will also be performed when the suspended node is restarted .

1.5 gateway

representative ES Persistent storage of index ,ES The default is to store the index in memory first , When the memory is full, persist to the hard disk . When this ES When the cluster is shut down and restarted, it will start from gateway Read index data from .Es Support multiple types of gateway, There's a local file system ( Default ), distributed file system ,Hadoop Of HDFS and amazon Of s3 Cloud storage services .

1.6 discovery.zen

representative ES Automatic discovery of nodes mechanism ,ES It's based on p2p The system of , It first looks for the existing nodes through broadcasting , Then through the multicast protocol to carry out the communication between nodes , It also supports point-to-point interaction .

If the nodes of different network segments are composed ES colony ​​​​​​​

 Disable auto discovery 

discovery.zen.ping.multicast.enabled:false

 Set the annotation list that can be found when the new node is started 

Discovery.zen.ping.unicast.hosts:["192.8.50.150", "192.8.53.124:9300"]

1.7 Transport

representative ES How internal nodes or clusters interact with clients , The default internal is to use tcp Protocol to interact , And it supports http agreement (json Format )、thrift、servlet、memcached、zeroMQ Wait for the transmission protocol ( Through plug-in integration ).