UW Information Technology supports the Personal Log File service so you can accurately track requests to your website. This article describes the different log files to choose from and how to turn them on and off. It also provides advice on managing your log files so you don’t waste space or exceed your disk quota.
What are Personal Log Files?
Personal log files are individual files in your Web directory that accrue information about the requests made to your website. Here’s how they work. During the day, the Web server logs information about every request it serves. Around midnight, we go to work and sort out the information relevant to your website. All you have to do to receive this information is turn on the corresponding personal log file.
You have three different personal log files to choose from: access_log, referer_log, and extended_log. Their formats are described below.
The access_log file
Your personal access_log file contains information formatted in the common logfile format. This is a popular format used by many Web servers. Most log file analysis tools can process this format.
Each line in your access_log file represents one request. For example:
bu.bu.edu – – [03/Aug/2001:22:41:42 -0700] “GET /webdemo/page.html HTTP/1.0” 200 988
In this case, bu.bu.edu is the computer that made a request for /webdemo/page.html on the night of August, 3, 2001. The request was successful (that’s what 200 means) and 988 bytes were sent.
The referer_log file
Your personal referer_log file contains information about references to your website. It is often helpful to turn on your referer_log file to determine what Web pages link to yours and thus how people find your website.
Each line in your referer_log file represents one request that was logged with referring information. Here are some hypothetical examples:
http://www.washington.edu/computing/Web/index.html -> /webdemo/index.html
http://search.yahoo.com/bin/search?p=webdemo -> /webdemo/index.html
In both cases, the page on the left referred someone to the webdemo home page and the resulting request was logged. The first example is typical referer_log file information: it lists one address that links to the webdemo home page. The second example illustrates how someone might have found the webdemo home page by searching the Yahoo subject catalog. If your website has been indexed by Internet search services, such as Yahoo or AltaVista, your referer_log reveals what services and search words people use to find your website.
The extended_log file
Your personal extended_log file contains a mixture of access_log and referer_log information, with Web browser information added for good measure.
Heed This Warning
Before you turn on any personal log file, plan for the incoming information. Remember you continue to receive nightly installments until you run out of disk space. Avoid the fate of the Sorcerer’s apprentice: keep an eye on your personal log files!
Turning On a Log File
The Personal Log File service hinges on the existence of specific files in your Web directory, called, appropriately, access_log, referer_log, and extended_log. When one of these files exists, it accrues data via nightly installments. You can turn on one, two, or all three of these files, depending on the information you want to receive.
To turn on a personal log file:
- Log into your web development server using a terminal emulator. If you’re not sure how to do this, click here for instructions.
- At the prompt, enter the following command to change directories to your Web directory:
cd public_html
- Use the touch command to turn on a personal log file. For example, to turn on your personal extended_log file enter:
touch extended_log
At this point, your log file is turned on, but it will be empty until sometime around midnight when the first batch of data is delivered.
- Optional: If you do not want other people to be able to look at the personal log file you just turned on, change the file permissions so only you can read it. For example, enter:
chmod 600 extended_log
Turning Off a Log File
Since the Personal Log File service hinges on the presence of files in your Web directory, all you have to do to turn off a log files is rename or remove the appropriate file.
To turn off a personal log file:
- Log into your web development environment using a terminal emulator. If you’re not sure how to do this, click here for instructions.
- At the prompt, enter the following command to change directories to your Web directory:
cd public_html
- To turn off a personal log file, remove or rename the appropriate file in your Web directory. For example, to turn off your personal extended_log file and remove the data in it, enter:
rm extended_log
Alternatively, to turn off your personal extended_log file and keep what’s in it, enter:
mv extended_log extended_log.bak
UW Information Technology does not provide access to log file data except through the Personal Log File service. Remember you receive no installments while your personal log files are turned off.
Archiving Your Log Files
After a while, you should delete or archive your personal log file data to save disk space. If you want to archive your data for future use, pick a good time, perhaps the first day of each month, and archive any personal log files you have turned on. One way to do this is illustrated below, using the extended_log file as an example.
To archive your extended_log file:
- Log into your web development environment using a terminal emulator. If you’re not sure how to do this, click here for instructions.
- At the prompt, enter the following command to change directories to your Web directory:
cd public_html
- At the prompt, enter the following command to rename your extended_log file:
mv extended_log extended_log.jun.2001
Now your data is archived in a file called extended_log.jun.2001.
- Now compress the file to save space. Enter:
gzip extended_log.jun.2001
Note: compressing the file with the gzip command automatically adds a .gz filename extension.
- Finally, move the compressed file out of your Web directory to your home directory on either Homer or Vergil. To do so, enter:
mv extended_log.jun.2001.gz ~
- Renaming your extended_log effectively turned it off. If you want to accrue new extended_log data, turn your extended_log back on.
If Logs Stop Updating
From time to time, our log splitting tool will stop working. While we usually catch these errors quickly, if you notice your log hasn’t been updated for a few days, please let us know.