How to create a custom cluster role in Rancher v2.x to grant permission on the metrics endpoint of the kube-apiserver in a Rancher managed cluster

Follow
Table of Contents

Task

This article details how to create a cluster role to grant users access to the /metrics endpoint of the Kubernetes API Server, in Rancher-managed Kubernetes clusters.

Pre-requisites

  • A Rancher v2.x instance
  • A Rancher-managed Kubernetes cluster, with Metrics Server deployed. This is deployed by default in Rancher-provisioned clusters.

Resolution

In Rancher v2.4+ it should be possible to define a non-resource URL grant via role creation within the Rancher UI. However, this is affected by the issue tracked in Issue #30321 and use of the Rancher v3 API is therefore required to create the role.

  1. The first step is to create a custom cluster role within Rancher, that grants get permission on the non-resource URL /metrics endpoint.

    As an admin user, generate an un-scoped Rancher API token, and execute the following API request via cURL, to create the required role. You will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and RANCHER_URL to reflect the generated API token and your Rancher URL. You can also edit the role name, as desired, which is set to kube-api metrics in this example.

    export CATTLE_ACCESS_KEY=token-8jn92
    export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
    export RANCHER_URL=rancher.example.com
    curl -k -u "${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}" \
    -X POST \
    -H 'Accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{"context":"cluster","clusterCreatorDefault":false," projectCreatorDefault":false,"name":"kube-api metrics","rules":[{"nonResourceURLs":["/metrics"],"type":"/v3/schemas/policyRule","verbs":["get"]}]}' \
    "https://${RANCHER_URL}/v3/roletemplates"
  2. After creating the cluster role, you can then grant this for a user or group. To do so, follow the steps in the Rancher documentation on assigning a cluster role to a user or group.

  3. Once the role is granted to a user, they will be able to test their access to the /metrics endpoint.

    The user can access the endpoint, with the applicable cluster id, via the Rancher proxied Kubernetes API Server endpoint, by generating a cluster-scoped or un-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY, RANCHER_URL and CLUSTER_ID to reflect the generated API token, Rancher URL and cluster id.

    export CATTLE_ACCESS_KEY=token-8jn92
    export CATTLE_SECRET_KEY=l2r4nq9sx6pdhpm4bgwntvgk49qn6rvvmtsvlvkmjk9rjsfd7n65fz
    export RANCHER_URL=rancher.example.com
    export CLUSTER_ID=c-wwdjc
    curl -k https://${RANCHER_URL}/k8s/clusters/${CLUSTER_ID}/metrics \
    -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

    For Rancher-provisioned Kubernetes clusters with Authorized Cluster Endpoint enabled, the user can also query the endpoint by connecting to the Kubernetes API Server on the cluster's controlplane nodes directly, using a cluster-scoped API token. The user will need to set CATTLE_ACCESS_KEY, CATTLE_SECRET_KEY and AUTHORIZED_ENDPOINT_ADDRESS to reflect the generated API token, and the authorized endpoint address.

    export CATTLE_ACCESS_KEY=token-d6cls
    export CATTLE_SECRET_KEY=b6gk6lmgrhsb4rjccktzkwxn5df7tm87msggq87lpmls2pkbpc5t5r
    export AUTHORIZED_ENDPOINT_ADDRESS=controlplane-01.example.com
    curl -k https://${AUTHORIZED_ENDPOINT_ADDRESS}:6443/metrics \
    -H "Authorization: Bearer ${CATTLE_ACCESS_KEY}:${CATTLE_SECRET_KEY}"

Further reading

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.