GCP ELK on Ubuntu 22.04

GCP Marketplace Image User Guide

This documentation describes Classmethod Canada’s ELK stack product listed on the Google Cloud Marketplace.

The package runs on Ubuntu 22.04 and contains ELK 7.17.8 (Elasticsearch, Logstash, Kibana, note that Beat is not included in this installation). License file can be found under this directory

/usr/share/doc/ELK_licenses/

GCP Marketplace Image User Guide

This product features a built-in ELK (Elasticsearch, Logstash, and Kibana) stack on Google Cloud Platform which can be customize to each users’ preference. This solution is ideal for user who wants the full functionality of an ELK stack and want to save some time on installing the software. We have installed all the ELK components on an Ubuntu environment, so that you can start your data analysis immediately. As easy as it sound, with just a few clicks, you can provision an ELK instance in less than a minute.

After you have provisioned the VM instance, go to the GCP Compute Engine console. Check that the status is running. Note the External IP address to access Kibana on your browser.

Wait a few minutes to provision Kibana. Then paste the IPv4 Public IP onto your web browser and it will bring you to the Kibana launch page:

{https://{External IP}:5601

You can further customize the ELK stack by SSH into the instance using the following command:

gcloud beta compute ssh --zone "us-west1-a" "elk"  --project "classmethod-can-public"

GCP allow users to SSH into the instance directly on the console. Click on the arrow next to SSH of your instance to see more SSH options. If you wish to SSH using terminal, make sure you set up gcloud and OAuth on your local machine.

SSH documentation: https://cloud.google.com/sdk/gcloud/reference/compute/ssh

Elasticsearch

This documentation guide you through how to get started with Elasticsearch.

https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html

Run this command to see if Elasticsearch is running.

curl -X GET "localhost:9200/"

It should give the following if successful:

{
  "name" : "elk-ubuntu2204-jammy",
  "cluster_name" : "my-application",
  "cluster_uuid" : "R4UdK2TpREWQfy-7iuRd3g",
  "version" : {
    "number" : "7.17.8",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "120eabe1c8a0cb2ae87cffc109a5b65d213e9df1",
    "build_date" : "2022-12-02T17:33:09.727072865Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Alternatively, you can also check the status by typing

sudo service elasticsearch status

If the service is down, restart or stop elasticsearch service.

sudo service elasticsearch restart

or

sudo service elasticsearch stop

Wait at least 30s to check the status again.

By default, all three services will automatically start when you launch the instance.

Start using Elasticsearch with Introduction to Indexing.

Logstash

Check status using this command:

sudo service logstash status

Follow this tutorial to get started on a Logstash pipeline.

Kibana

Check status using this command:

sudo service kibana status

Refer to these official user guides to start adding sample data!

Configuring Elasticsearch

Kibana Guide — Get Started

Troubleshooting

Scenario 1.

If Kibana doesn’t show up on web browser, reboot the instance on GCP console.

Alternatively, you can also SSH into the instance and run:

sudo reboot

Scenario 2.

If you encounter the following page indicating that the connection is not private, there’s a workaround.

Select anywhere on the error page, and type the following code. This will bring you to the Kibana launch page. (Refer to reference number 4 & 5)

thisisunsafe

Disable Unattended Upgrades

Unattended Upgrades Are Turned On by default. If you want to turn unattended upgrades off please following this instruction:

1. Run sudo dpkg-reconfigure -plow unattended-upgrades and select “No” when Configuring unattended-upgrades prompted, OR
2. sudo apt remove unattended-upgrades to remove

Reference

1. Install Kibana with Debian Package
2. Installing Logstash
3. No “Proceed Anyway” option on NET::ERR_CERT_INVALID in Chrome on MacOS
4. NET::ERR_CERT_REVOKED Chrome MacOS home server
5. How to Ensure Ubuntu’s Unattended Upgrades Are Turned On
6. How To Disable Unattended Upgrades On Ubuntu
7. How do I stop/disable unattended upgrades from being launched automatically? [duplicate]

Tags ELK, GCP