Log rotation

The log file generated by the application can quickly increase in size, and if you want to make sure it doesn't take up too much disk space, you can introduce log rotation. Log rotation will periodically clear the old logs, thus preventing the log file from taking up all the disk space.

On Linux systems, the logrotate command is usually already present by default (generally located at /usr/sbin/logrotate). The way to set up log rotation can differ based on the kind of Linux distribution you use, whether you set up the application as root or as a simple user, whether you placed the application in /opt or in /home, and the exact location where the logs are placed. Two typical use cases are described below:

When the log files are placed inside /opt/timeline or a similar location, and docker is executed as a root user:

1. Create a log rotation config file, for example at: /etc/logrotate.d/timeline

   The file should contain the following:

   Copy

   /opt/timeline/logs/* {
       size 1G
       copytruncate
       rotate 1
   }

2. The path should point to the log file generated by the docker-compose up command. This particular configuration would clear the log file when it exceeds the size of 1 MB and copy its original content to another file called /home/<USER>/timeline/logs/docker-compose.log.1. The next time the log rotation runs and finds that the log size exceeds 1 MB again, it overrides the docker-compose.log.1 with its new contents and clears the original log file.

3. This assumes that the Linux system already has a logrotation installed and registered as a cron job. You can verify this by checking that an /etc/logrotate.conf file exists, and it contains the line include /etc/logrotate.d, and also that there is a file called /etc/cron.daily/logrotate that runs the command /usr/sbin/logrotate /etc/logrotate.conf. Different Linux distributions might have these files arranged in a different way.

When the log files are placed inside /home/<USER>, and are written by a non-root user:

1. Create a log rotation config file, for example at: /home/<USER>/logrotate.conf

   The file should contain the following:

   Copy

   /home/<USER>/timeline/logs/* {
       size 1G
       copytruncate
       rotate 1
   }

   The path should point to the log file generated by the docker-compose up command. This configuration works the same way as described in the previous case.

2. Register a cron job to run the log rotation procedure once a day.

   Run the following command to create a user-specific cron job:

   crontab -e

   This will open a text editor where you can register cron jobs by adding lines like the following:

   0 * * * * /usr/sbin/logrotate /home/$USER/logrotate.conf --state /home/$USER/logrotate-state.txt

For log rotation to work correctly, you have to write the log files in append mode (in bash '>>' instead of just '>'), otherwise the log file cannot be cut, since the process would keep writing at its current offset at the location that used to be the end of the file, even after the file was cleared.