Connect Layer - Managing Hard Disk Space
- Product: campusM
- Operating system: iOS, Android
Typically Connect Layer servers are dedicated to campusM; we don’t store any data on these servers. There are exceptions (e.g. servers which also host databases for integrations, or other apps for the University) but this document covers the usual case of the campusM app being the sole use of a server.
df (stands for “disk filesystem”) shows how much space remains on each partition on a linux server. It’s a simple command to use and is worth doing as a precaution whenever you visit a server, particularly if there are any issues reported.
A typical output from
df. Note that
df -h will list the storage amounts in human readable format instead of bytes.
Of particular interest are the two columns on the right. “
Use%” shows the percentage of the partition is used. “
Mounted on” shows the area of the file system that is mounted on a particular partition (as opposed to the physical location, which is show under “
Filesystem”, but not particularly relevant to our clearing disk space).
The exact partition used to house the campusM files varies a great deal. Often it's
/ but there are no hard and fast rules, and different files may be spread across more than one mounted drive.
If a partition shows high usage, it may need some space clearing. If it is showing 100% use, it absolutely will need space clearing; and may be the cause of any issues reported since it would prevent campusM from handling any traffic at all with the inability to write log file activity.
A Typical Tomcat logs directory.
The Tomcat logs directory usually contains catalina.out, localhost, localhost_access and manager, with several date-stamped archive files of each. These contain log messages written to stdout and stderr (in Java terms, outputs such as
While developers should be using log4j, in practice these files can still gather a lot of logs, and will grow over time.
We do not need to retain many archived log files. When an error is reported, it will usually be present in the current, or previous day’s logs. It’s very rare for us to need to look at anything older. With this in mind, it is considered safe to delete any log files more than a week old (although the standard log rotate recommendation in the typical setup is to rotate anything older than two weeks.
There will also be one or more subdirectories within the base logs directory. Sometimes a subdirectory will have the same name as the app’s WAR file; either“campusM” (older) or “campusm” (newer) depending on the generation of Connect Layer deployment. Because Linux directory names are case-sensitive “campusm” and “campusM” may exist in the same logs directory.
The content of the subdirectories also follows a similar pattern, with a number of types of log, and a number of date-stamped archives of each. In order to manage disk space effectively we need to manage these subdirectories as well as the base logs directory.
Managing Disk Space
rm -f /var/log/tomcat/logs/*2017*will quickly clear all logs from 2017). Repeat this, deleting any log files older than, say, a week (unless you ohave a compelling reason to keep anything older, in which cases it would be recommended to archive those older logs and offload them to a different disk).
Prevention is better than cure, and we can prevent log files consuming all disk space with log rotation. This means old files are deleted regularly, so only the latest logs are retained, and disk space is preserved as recommended in our Connect layer setup guide log rotate recommendation and additional details below.
Logging Properties Files
Part of defining log rotation is through the log4j.properties file (in the Tomcat conf directory or deployed in the WAR file). The logging behaviour of the subdirectories are defined in this file.
# LOGFILE is set to be a File appender using a PatternLayout.
log4j.appender.LOGFILE.layout.ConversionPattern=%d [%t] %-5p %c %x - %m%n
The last line:
log4j.appender.LOGFILE.MaxBackupIndex=7 means only seven days’ log files will be retained. Each log file defined in log4j.properties must have a similar line.
Logrotate is a linux system utility that manages the automatic rotation and compression of log files. It is used to manage the logs in the Tomcat base logs directory.
Logrotate is installed by your package manager in the usual way:
aptitude update && aptitude install logrotate (Debian and Ubuntu)
yum update && yum install logrotate (CentOS, Red Hat, etc.)
(The client is responsible for installing software.)
The main config for logrotate is usually found in
/etc/logrotate.d, with specific config for individual applications (in our case Tomcat) within
/etc/logrotate.d. In this directory edit, or create a file called tomcat.
In there you will need to create a config entry for each log directory that campusM creates. This example manages the all.log file. Similar entries will be required for campusM.log and others:
create 0644 tomcat:tomcat
There is a good deal more information in the logrotate man file:
It is recommended to user INFO or WARN on the Production Connect Layer and DEBUG on the Sandbox Connect Layer.
- Article last edited: DD-Mmm-YYYY