`Get secret: secrets "webhook-receiver" not found` errors observed in Rancher v2.4 and v2.5 server logs

Issue

Sometimes an admin may see an error in the Rancher server logs like the one below:

[project-alert-group-controller] failed with : Update Webhook Receiver Config: Get secret: secrets "webhook-receiver" not found

Pre-requisites

• A Rancher v2.4.x instance prior to v2.4.11, or a Rancher v2.5.x instance prior to v2.5.4
• Rancher Monitoring v1 enabled and an Alert Group configured with a non-webhook notifier

Root cause

The alert controller in Monitoring v1 would log an error when a secret containing a webhook-notifier configuration could not be found, even if no webhook based notifiers have been configured on alerts. This log message is harmless and can be ignored.

Resolution

This erroneous log message has been fixed in Rancher v2.4.x, at v2.4.11 and above, and in Rancher v2.5.x, at v2.5.4 and above. Following a Rancher upgrade, the error should no longer show in the Rancher server logs.

Further Reading

• Github Issue #28954 tracking this behaviour in Rancher v2.5.x
• Github Issue #30088 tracking this behaviour in Rancher v.4.x