atextcrawler/doc/source/elasticsearch.md

3.0 KiB

Howto elasticsearch

Prerequisites

On the host (virtualization host) we need:

# cat /etc/sysctl.d/virtual_memory.conf
vm.max_map_count=262144
# sysctl -p /etc/sysctl.d/virtual_memory.conf

If this cannot be done, change this file after installing or upgrading elasticsearch:

/usr/lib/sysctl.d/elasticsearch.conf

Setup

Install package

In general, see the elaticsearch reference.

We do a manual install. If you configure the apt repo instead, also think about setting RESTART_ON_UPGRADE=true in /etc/default/elasticsearch.

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-amd64.deb.sha512
shasum -a 512 -c elasticsearch-7.15.2-amd64.deb.sha512
dpkg -i elasticsearch-7.15.2-amd64.deb
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

First test:

http -j GET 127.0.0.1:9200/

Storage

systemctl stop elasticsearch.service
mv /var/lib/elasticsearch/ /srv/
systemctl start elasticsearch.service

Edit /etc/elasticsearch/elasticsearch.yml

cluster.name: org.a-text.search
node.name: atext1
path.data: /srv/elasticsearch
path.logs: /var/log/elasticsearch
discovery.seed_hosts: ["atext1.multiname.org"]
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
systemctl restart elasticsearch

The logfile now is at

/var/log/elasticsearch/org.a-text.search.log

Setup passwords

Setup passwords:

# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y

Copy output to /etc/elasticsearch/passwords and

chmod 400 /etc/elasticsearch/passwords

Check login as user elastic:

http --auth elastic:************** -j GET http://127.0.0.1:9200/

Memory limitation

To limit memory usage

mkdir /etc/systemd/system/elasticsearch.service.d
cat >/etc/systemd/system/elasticsearch.service.d/override.conf <<EOF
[Service]
LimitMEMLOCK=8G

systemctl stop elasticsearch
systemctl daemon-reload
systemctl start elasticsearch
EOF

and restart the service.

Usage

Some useful requests:

List indices

http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/indices

Health

http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/health

Node attributes

http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/nodeattrs

Create API key

http --auth elastic:$PASS -j POST http://127.0.0.1:9200/_security/api_key name=anarchism role_descriptors:='{"anarchism": {"cluster": [], "index": [{"names": ["anarchism_*"], "privileges": ["all"]}]}}'