Pushing metrics and logs from a Scaleway Instance

• cockpit
• observability
• metrics
• logs

This page shows you how to push your metrics and logs from a Scaleway Instance. You can also use this tutorial to push metrics and logs from external Instances (i.e. Instances outside of Scaleway).

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization

Configure your environment

1. Create an Instance from the Compute section of the Scaleway console.

2. Copy your Instance’s public IP as you will need it in the next steps.

3. Open a terminal and type the following command to connect to your Instance:

   ssh root@<your Instance public IP>
   

Clone the GitHub repository

We have created a GitHub repository that contains all the code you need to configure the Grafana agent, to push your data.

Files in the directory

The tuto-cockpit-instance directory contains a workshop folder with several files inside of it:

• download.sh is a script responsible for downloading the necessary binaries and tools you need for the monitoring setup. It fetches the Grafana agent, for example.
• agent.yaml is the Grafana agent configuration file responsible for collecting metrics and logs from your data sources, and pushing them to your Cockpit.
• docker-compose.yaml is the Docker compose configuration file
• setup.sh ensures that all dependencies required for monitoring are properly installed and configured
• nginx.conf specifies server settings
• run.sh executes the Grafana agent with the specified configuration file (agent.yaml).

The only file we will be editing in this tutorial is the agent.yaml file to add your Cockpit token.

1. Open another terminal and type the following command to clone our tuto-cockpit-instance directory on your local machine:

   git clone git@github.com:scaleway/tuto-cockpit-instance.git
   

2. In the same terminal, run the following command to send the tuto-cockpit-instance directory and its content into your Instance:

   scp -r tuto-cockpit-instance root@<your Instance public IP>:~/
   

3. Make sure the tuto-cockpit-instance directory is inside your Instance with the following command:

   ls
   

4. Access the workshop folder inside tuto-cockpit-instance, using the following command:

   cd tuto-cockpit-instance/workshop
   

5. Install Golang on your Instance using the following commands:

   wget https://github.com/grafana/agent/releases/download/v0.34.3/grafana-agent-0.34.3-1.amd64.deb
   sudo tar -C /usr/local/ -xzf go1.20.linux-amd64.tar.gz
   

6. Run the following command to set Golang in your environment:

   export PATH=$PATH:/usr/local/go/bin
   

7. Run the following command to install docker. Docker allows you to run the docker-compose.yaml file.

   apt install docker.io
   

8. Run the following command to unzip the files in the workshop folder:

   sudo apt-get install zip unzip
   

Configure the Grafana agent file and push your data

1. Generate a token and select the push permissions for metrics and logs. Make sure you note down your token’s secret key as you will need it in the following steps.
2. In the same terminal where you are connected to your Instance, run the following command to download the Grafana agent:

   bash ./download.sh
   

3. Run vim agent.yaml to access the Grafana agent configuration file.
4. Type i to edit the file.
5. Add your token’s secret key next to X-Token in the metrics section.
6. Add your token’s secret key next to tenant_id in the logs section.
7. Press escape, :wq, and Enter to save and exit the file.
8. Run the following command to launch the Grafana agent:

   bash ./run.sh
   

Visualize your metrics and logs

1. Log in to the Scaleway console and click Cockpit in the side menu.

2. Click Open dashboards at the top right corner of your screen.

3. Log in to Grafana using your credentials.

4. Click the downward pointing arrow at the top right corner of your screen and select Import dashboard.

   

5. Paste the following dashboard ID 12486 in the Import via grafana.com section and click Load. Alternatively, download the dashboard’s JSON template and upload it in the Import dashboard section.

6. Click Import.

7. Configure your dashboard as you please.

8. Select Scaleway metrics in the drop-down box and click Import.

   Important

   Select Metrics if you are using custom data.

   Your metrics should display in the dashboard.

   

9. Click Toggle menu and Explore.

   Important

   Select Logs if you are using custom data.

10. Select the labels you want to apply in the Label filters section.

11. Click Run query at the top right corner of your screen. Your logs should display.