Elasticsearch 8 Tutorial - Configuring Elasticsearch and Kibana with Credentials and SSL

Explanation of the handshake process:

SSL/TLS Handshake Flow: Browser, Kibana, and Elasticsearch

1. Browser to Kibana Handshake

Browser initiates connection to Kibana (Client Hello)
Kibana responds with Server Hello and sends its certificate (kibana.crt)
Kibana proves its identity:
- Creates a summary of all handshake messages so far
- Signs this summary with its private key (kibana.key)
- Sends the signed summary to Browser
Browser verifies the signature using Kibana's public key (from kibana.crt)
If signature is valid, Browser confirms it's talking to the real Kibana
Browser generates a pre-master secret
Browser encrypts the pre-master secret with Kibana's public key
Browser sends the encrypted pre-master secret to Kibana
Kibana decrypts the pre-master secret using its private key
Both Browser and Kibana independently generate session keys from the pre-master secret
They exchange "Finished" messages to establish a secure connection

2. Kibana to Elasticsearch Handshake

Kibana initiates connection to Elasticsearch (Client Hello)
Elasticsearch responds with Server Hello and sends its certificate (elastic.crt)
Elasticsearch proves its identity:
- Creates a summary of all handshake messages so far
- Signs this summary with its private key (elastic.key)
- Sends the signed summary to Kibana
Kibana verifies the signature using Elasticsearch's public key (from elastic.crt)
If signature is valid, Kibana confirms it's talking to the real Elasticsearch
Kibana generates a pre-master secret
Kibana encrypts the pre-master secret with Elasticsearch's public key
Kibana sends the encrypted pre-master secret to Elasticsearch
Elasticsearch decrypts the pre-master secret using its private key
Both Kibana and Elasticsearch independently generate session keys from the pre-master secret
They exchange "Finished" messages to establish a secure connection

3. Secure Communication

Browser sends HTTPS requests to Kibana
Kibana forwards HTTPS requests to Elasticsearch
Elasticsearch sends HTTPS responses to Kibana
Kibana forwards HTTPS responses to Browser
All communication is end-to-end encrypted using the established session keys

Key Points:

Private keys (kibana.key and elastic.key) are used for decryption and signing
Public keys (from certificates) are used for encryption and signature verification
Pre-master secrets are used to generate session keys for ongoing encrypted communication
The process ensures authentication of servers and secure key exchange

Prerequisites

Download Elasticsearch 8.15.2 from Elasticsearch downloads page

Download Kibana 8.15.2 from Kibana downloads page

Step 1: Initial Elasticsearch Setup

Unzip the Elasticsearch installation.
Navigate to the bin folder and start Elasticsearch by running elasticsearch.bat
Access https://localhost:9200. You'll be prompted for credentials:
- Username: elastic
- Password: (found in the command line console when Elasticsearch starts)

Note: By default, Elasticsearch creates http.p12, http_ca.crt, and transport.p12 for us.

Step 2: Reset Passwords

Reset passwords for the elastic and kibana users:

.\elasticsearch-reset-password -u elastic --interactive
.\elasticsearch-reset-password -u kibana --interactive

Set both passwords to their respective usernames for this tutorial.

Step 3: Create SSL Certificates

Create a Certificate Authority (CA):
```
.\elasticsearch-certutil ca --pem
```
This generates ca.crt and ca.key.

Create Elasticsearch certificate:

.\elasticsearch-certutil cert --name elastic --ca-cert D:\elk\ad\elasticsearch-8.15.2\config\certs\ca\ca.crt --ca-key D:\elk\ad\elasticsearch-8.15.2\config\certs\ca\ca.key --dns javainuse.com --pem

$\elasticsearch-certutil cert --name elastic$

This creates elastic.crt and elastic.key.

Create Kibana certificate:

.\elasticsearch-certutil cert --name kibana --ca-cert D:\elk\ad\elasticsearch-8.15.2\config\certs\ca\ca.crt --ca-key D:\elk\ad\elasticsearch-8.15.2\config\certs\ca\ca.key --dns javainuse.com --pem

$\elasticsearch-certutil cert --name elastic$

This creates kibana.crt and kibana.key.

Step 4: Configure Elasticsearch

Modify the elasticsearch.yml configuration file:

# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  certificate: certs/elastic/elastic.crt
  key: certs/elastic/elastic.key

cluster.initial_master_nodes: ["LAPTOP-0ELSI3NO"]
http.host: 0.0.0.0

Start Elasticsearch. Access https://localhost:9200 and enter the credentials (elastic/elastic).

Step 5: Configure Kibana

Modify the Kibana configuration file:

server.host: "0.0.0.0"
server.ssl.enabled: true
server.ssl.certificate: D:\elk\ad\elasticsearch-8.15.2\config\certs\kibana\kibana.crt
server.ssl.key: D:\elk\ad\elasticsearch-8.15.2\config\certs\kibana\kibana.key
elasticsearch.ssl.verificationMode: none
elasticsearch.hosts: ["https://127.0.0.1:9200"]
elasticsearch.username: "kibana"
elasticsearch.password: "kibana"

Start Kibana. Access https://localhost:5601 and enter the elastic user credentials.

Conclusion

You have now successfully set up Elasticsearch and Kibana with proper security measures, including user authentication and SSL encryption. This configuration ensures that your Elastic Stack is protected against unauthorized access and that data transmission is secure.

Remember to keep your certificates and passwords safe, and consider implementing more advanced security measures for production environments.

Search Tutorials