How to ingest application-level metrics from Managed Service for Kubernetes® using Prometheus Operator

​

Prerequisites

1. Install and configure Nebius AI Cloud CLI.
2. If you don’t have a service account for observability services, create one.
3. Make sure that the service account is in a group that has at least the editor role within your tenant; for example, the default editors group. You can check this in the Administration → IAM section of the web console. If the service account is not in the required group, click  → Add to group, and select editors.
4. Issue a static key for the service account using the following command:
   nebius iam static-key issue \
     --name <name_for_the_key> \
     --account-service-account-id <service_account_ID> \
     --service observability
   
   Copy the value of the static key from the token parameter of the response. You will need it later in the configuration steps.

​

How to start ingesting metrics

1. Connect to your Managed Service for Kubernetes cluster.
2. Create a namespace for the observability components:
   kubectl create namespace observability
   
3. Create a Kubernetes Secret for your token:
   kubectl create secret generic nebius-o11y-token \
     --from-literal token=<token_value> --namespace observability
   
   In this command, specify the token value of the static key that you got earlier.
4. Install and update the Prometheus Monitoring Community repository with Helm charts:
   helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
   helm repo update
   
5. Customize the kube-prometheus-stack chart:
   1. Open the values.yaml file:
      helm show values prometheus-community/kube-prometheus-stack > values.yaml
      
   2. Paste the following configuration into the values.yaml:
      grafana:
        enabled: false
      
      alertmanager:
        enabled: false
      
      prometheus:
        agentMode: true 
        
        prometheusSpec:  
          remoteWrite:      # Configure Prometheus to write scraped samples to a remote endpoint
            - name: Nebius
              url: https://write.monitoring.<region_ID>.nebius.cloud/projects/<project_ID>/prometheus/api/v1/write
              authorization:
                credentials:
                  name: nebius-o11y-token
                  key: token
      
      In this configuration, specify your region ID from the region list, and your project ID. The configuration disables Grafana® and Prometheus in agent mode and configures a write endpoint to push metrics to Monitoring.
6. Install the chart:
   helm install prometheus-stack prometheus-community/kube-prometheus-stack \
     --namespace observability \
     --values values.yaml
   
7. Check the installation:
   kubectl get pods --namespace observability