Retrieving logs programmatically using LogCLI

Reviewed on 19 September 2024 • Published on 22 May 2024

This documentation page shows you how to retrieve your Scaleway and custom logs stored in Cockpit programmatically, using LogCLI.

Retrieving your logs programmatically can be useful in environments where automation, scalability, and integration with other systems are crucial. It enables advanced analytics, greater efficiency, and deeper insights than manual log retrieval through a user interface such as Grafana.

Before you start

To complete the actions presented below, you must have:

Owner status or IAM permissions allowing you to perform actions in the intended Organization
A Scaleway account logged into the console
Created a token with the query permission for logs
Custom and/or Scaleway logs to query
Installed LogCLI

Exporting your environment variables

Open a terminal and run the following command to export your environment variables. Make sure that you replace <DATA_SOURCE_URL> and <COCKPIT_TOKEN_SECRET_KEY> with your values.

export LOKI_ADDR=<DATA_SOURCE_URL>
export LOKI_BEARER_TOKEN=<COCKPIT_TOKEN_SECRET_KEY>

Note

You can retrieve your data source URL in the Data sources tab under the API URL section.

Retrieving logs

Retrieving an Object Storage bucket’s logs

Open a terminal and paste the following command to retrieve logs from your desired data source. For the sake of this documentation, we are retrieving logs from a Scaleway Object Storage bucket. You can replace object_storage_bucket with any other Scaleway resource type, provided that the resource has logs available.

logcli query '{resource_type="object_storage_bucket"}'

An output similar to the following should display:

2024/05/20 17:04:33 <DATA_SOURCE_URL>_range?direction=BACKWARD&end=1716390273303145000&limit=30&query=%7Bresource_type%3D%22object_storage_bucket%22%7D&start=1716386673303145000

Retrieving a Kubernetes Kapsule cluster’s logs

Open a terminal and paste the following command to retrieve logs from your Kapsule cluster.

logcli query '{resource_type="kubernetes_cluster"}'

An output similar to the following should display:

2024/05/22 17:33:19 <DATA_SOURCE_URL>_range?direction=BACKWARD&end=1716391999461784000&limit=30&query=%7Bresource_type%3D%22kubernetes_cluster%22%7D&start=1716388399461784000
2024/05/22 17:33:19 Common labels: {level="I", log_type="control-plane", region="fr-par", resource_id="36f08aaf-f208-4d58-9385-5064cde63792", resource_name="k8s-sharp-robinson", resource_type="kubernetes_cluster"}
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 legacy.go:727] No candidates for scale down
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 static_autoscaler.go:544] Starting scale down
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 static_autoscaler.go:535] Scale down status: lastScaleUpTime=2024-05-22 14:20:23.13076564 +0000 UTC m=-3598.920416647 lastScaleDownDeleteTime=2024-05-22 14:20:23.13076564 +0000 UTC m=-3598.920416647 lastScaleDownFailTime=2024-05-22 14:20:23.13076564 +0000 UTC m=-3598.920416647 scaleDownForbidden=false scaleDownInCooldown=false
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 pre_filtering_processor.go:57] Node scw-k8s-sharp-robinson-default-e110ac268b5546d should not be processed by cluster autoscaler (no node group config)
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 pre_filtering_processor.go:57] Node scw-k8s-sharp-robinson-default-7cefec16593342e should not be processed by cluster autoscaler (no node group config)
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 pre_filtering_processor.go:57] Node scw-k8s-sharp-robinson-default-bfb90f82c4b949c should not be processed by cluster autoscaler (no node group config)
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 static_autoscaler.go:492] Calculating unneeded nodes
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 static_autoscaler.go:445] No unschedulable pods
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 filter_out_schedulable.go:87] No schedulable pods
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 filter_out_schedulable.go:177] 0 pods marked as unschedulable can be scheduled.
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 filter_out_schedulable.go:176] 0 pods were kept as unschedulable based on caching
2024-05-22T17:33:04+02:00 {component="kapsule-autoscaler"} I0522        1 filter_out_schedulable.go:137] Filtered out 0 pods using hints

Tip

You can tailor your logcli query command according to your needs using the LogCLI flags and commands. Enter logcli help in your terminal to see the available flags and commands, and examples of how to use them.