Make sure you have SSL certificates set up for secure communication.

Setting up the Elastic Stack (formerly ELK Stack)
- ELK-stack
- ELK
- elastic-stack
- elasticsearch
- logstash
- kibana
The Elastic Stack, formerly known as the ELK Stack, is a powerful suite of open-source tools designed for real-time data search, analysis, and visualization. It offers comprehensive capabilities for collecting, processing, and visualizing large volumes of data. Its components are:
- Elasticsearch: A distributed, RESTful search and analytics engine based on the Lucene library.
- Logstash: A flexible data collection, processing, and enrichment pipeline.
- Kibana: A visualization and exploration tool for analyzing and visualizing data stored in Elasticsearch.
- Beats: Lightweight data shippers for ingesting data into Elasticsearch or Logstash.
Before you startLink to this anchor
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- An SSH key
- An Instance or an Elastic Metal server with at least 4 GB of RAM
Install ElasticsearchLink to this anchor
-
Download and install the Elasticsearch signing key:
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor -o /usr/share/keyrings/elasticsearch-archive-keyring.gpg -
Add the Elasticsearch repository:
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-archive-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-8.x.list -
Update the
apt
package repositories:apt update -
Install Elasticsearch:
apt install elasticsearch -
Start and enable the Elasticsearch service:
systemctl start elasticsearchsystemctl enable elasticsearch -
Configure Elasticsearch for production: Modify the
elasticsearch.yml
file to optimize Elasticsearch for production use:nano /etc/elasticsearch/elasticsearch.ymlAdd the following:
cluster.name: "my-cluster"node.name: "node-1"network.host: 0.0.0.0xpack.security.enabled: truexpack.security.transport.ssl.enabled: truexpack.security.http.ssl.enabled: truexpack.security.http.ssl.keystore.path: /etc/elasticsearch/certs/keystore.p12xpack.security.http.ssl.truststore.path: /etc/elasticsearch/certs/truststore.p12Note
Install and configure LogstashLink to this anchor
-
Install Logstash using the same repository added for Elasticsearch:
apt install logstash -
Create and modify configuration files for Logstash: The configuration files for Logstash are typically located in
/etc/logstash/conf.d/
. You can create pipelines to manage your data processing. -
Start and enable the Logstash service:
systemctl start logstashsystemctl enable logstash
Install and configure KibanaLink to this anchor
-
Install Kibana:
apt install kibana -
Start and enable the Kibana service:
systemctl start kibanasystemctl enable kibana -
Configure Kibana for remote access: By default, Kibana is accessible on
http://localhost:5601
. To make Kibana accessible remotely, edit the Kibana configuration file:nano /etc/kibana/kibana.ymlChange the server host to:
server.host: "0.0.0.0" -
Secure Kibana: Ensure Kibana uses SSL to encrypt communications by adding SSL certificates in the
kibana.yml
file:server.ssl.enabled: trueserver.ssl.certificate: /etc/kibana/certs/kibana.crtserver.ssl.key: /etc/kibana/certs/kibana.keyelasticsearch.ssl.certificate: /etc/kibana/certs/kibana.crtelasticsearch.ssl.key: /etc/kibana/certs/kibana.key
Install and configure FilebeatLink to this anchor
-
Install Filebeat:
apt install filebeat -
Configure Filebeat to ship logs to Elasticsearch: Edit the Filebeat configuration file to point to your Elasticsearch instance:
nano /etc/filebeat/filebeat.ymlSet the output to Elasticsearch:
output.elasticsearch:hosts: ["http://localhost:9200"]Alternatively, configure Filebeat to send logs to Logstash:
output.logstash:hosts: ["localhost:5044"] -
Start and enable the Filebeat service:
systemctl enable filebeatsystemctl start filebeat
Secure the Elastic StackLink to this anchor
Securing your Elastic Stack is essential, especially if exposed to the internet. Following are some recommendations:
-
Enable built-in security features (as shown above in Elasticsearch and Kibana setup).
-
Use a firewall: You can use
ufw
oriptables
to restrict access to only the necessary IPs:ufw allow from <your_ip> to any port 9200ufw allow from <your_ip> to any port 5601 -
Set up an HTTPS reverse proxy: You can secure Kibana by setting up an HTTPS reverse proxy with Nginx: Set up an HTTPS reverse proxy with Nginx.
-
Set up TLS/SSL for Elasticsearch and Kibana: Ensure communications are encrypted between components using SSL/TLS as shown above.
Test the installationLink to this anchor
After completing the setup, you can verify if everything is working:
-
Elasticsearch: Run the following command to check Elasticsearch health:
curl -X GET "localhost:9200/_cluster/health?pretty" -
Kibana: Navigate to
http://your_server_ip:5601
in your web browser. -
Filebeat: Ensure logs are being shipped by checking the status:
curl -X GET "localhost:5601/api/status"
Now, you should have a basic Elastic stack up and running! Adjust configurations as needed for your specific use case and further secure and optimize your setup for production use. Refer to the official Elastic documentation for the most accurate and up-to-date instructions and advanced configuration information.