Redirect Docker/Kubernete log out of /var/log/message for Cloudera Data Science Workbench

After you installed Cloudera Data Science Workbench (CDSW) in a cluster, by default, the log messages for Docker and Kubernete all go into /var/log/messages. This is OK in most cases, but some user might not want the logs to be shared with other processes, so separating those logs outside of /var/log/messages might make sense. To do this, please follow steps below: 1. Create the file under /etc/rsyslog.d/10-docker.conf 2. Put the below contents into the above file:

# Docker logging
*.* {
 /var/log/docker.log
 stop
}

3. Configure logrotate to roll and archive these files by creating file: /etc/logrotate.d/docker 4. Put the following contents into file /etc/logrotate.d/docker:

/var/log/docker.log {
    size 100M
    rotate 2
    missingok
    compress
}

5. Finally, restart rsyslog:

service rsyslog restart

After all above steps, the CDSW logging should be redirected to /var/log/docker.log, no restart of CDSW is required. Note: this blog was based on steps from Mark Wolfe’s Blog. The difference is that instead of “daemon.*” in step 2, I need to use “*.*“, otherwise only kubernete log will go into /var/log/docker.log, not dockerd logs.