How to configure Nginx Ingress controller to write FQDN into access logs

Follow
Table of Contents

Task

Configure Nginx Ingress controller to write FQDN into access logs for better log filtering.

Resolution

To change the logging format, you have to change the value of log-format-upstream in ConfigMap of Nginx Ingress nginx-configuration, present in namespace ingress-nginx.

You may confirm the default log format by checking the source code of Nginx Ingress.

For example, Nginx Ingress version v0.35.0 has the following default logging configuration (Ref: Code).

logFormatUpstream = `$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time [$proxy_upstream_name] [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id`

With the default configuration, Ingress Nginx logs access entry without FQDN of the ingress.

Default access log output will look like below.

192.168.110.100 - - [12/Jan/2021:06:22:54 +0000] "GET / HTTP/1.1" 200 517 "-" "curl/7.64.1" 85 0.028 [default-ingress-4e51729d468a17ac2588f6c756ad4a2b-80] [] 10.42.189.81:80 517 0.028 200 81f29bcb4b96de6bc83ee110632908d1

We can modify the default values using config map variable log-format-upstream and add one more variable called $host to include Ingress FQDN in the logs.

For making the change persistent across RKE cluster changes, the modified value should be updated via Cluster's YAML spec.

You may perfom this by navigating to Rancher UI -> Edit Cluster -> Edit as YAML and then updating the ingress spec like below.

  ingress:
    provider: nginx
    options:
      log-format-upstream: >-
        $remote_addr - $remote_user [$time_local] $host \"$request\" $status
        $body_bytes_sent \"$http_referer\" \"$http_user_agent\" $request_length
        $request_time [$proxy_upstream_name [$proxy_alternative_upstream_name]
        $upstream_addr $upstream_response_length $upstream_response_time
        $upstream_status $req_id

If you are using API call to update cluster configuration, then make sure to escape the quotes in JSON payload like below.

         "options":{
            "log-format-upstream":"$remote_addr - $remote_user [$time_local] $host \\\"$request\\\" $status $body_bytes_sent \\\"$http_referer\\\" \\\"$http_user_agent\\\" $request_length $request_time [$proxy_upstream_name [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id"
         },

After applying the change, the cluster will go through a reconciliation cycle.

Once reconcilation completes, you may verify the change by checking the ConfigMap of Nginx Ingress using below command.

kubectl get cm -n ingress-nginx nginx-configuration -o yaml

Sample output:

apiVersion: v1
data:
  log-format-upstream: $remote_addr - $remote_user [$time_local] $host "$request"
    $status $body_bytes_sent "$http_referer" "$http_user_agent" $request_length $request_time
    [$proxy_upstream_name [$proxy_alternative_upstream_name] $upstream_addr $upstream_response_length
    $upstream_response_time $upstream_status $req_id
kind: ConfigMap
metadata:
  labels:
    app: ingress-nginx
  name: nginx-configuration
  namespace: ingress-nginx

Now onwards, the Nginx Ingress controller will add FQDN of the Ingress when a client access the URL.

In below sample, you can see the FQDN before the GET call.

192.168.110.100 - - [12/Jan/2021:06:24:02 +0000] ingress-test.domain.io "GET / HTTP/1.1" 200 518 "-" "curl/7.64.1" 85 0.097 [default-ingress-4e51729d468a17ac2588f6c756ad4a2b-80] [] 10.42.235.172:80 518 0.096 200 9bf011c2960a472e2910c5c3b8eeb956

Further reading

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.