Stat Tables

• The online tables (STAT_OBJECT and STAT_TARGET_SERVICE) are used during the SFX request time. Details regarding all the interactions with the SFX end user are stored in these tables.
• The offline tables (STAT_OBJECT_OFFLINE and STAT_TARGET_SERVICE_OFFLINE) are used by the statistics queries that the librarian or SFX administrator runs.

Why Stat Queries Are Run on the Offline Statistics Tables

The runtime for statistics queries varies depending on the amount of data.

The database engine locks the table in order to keep data integrity when reading from tables. Therefore, if the SFX administrator runs the queries directly on the online table, SFX end users may have long wait periods for their SFX menu to display. For this reason, offline tables are used in the Statistics module.

You can and may want to remove information from the offline stat tables and archive the data into TXT files when the offline tables contain a large amount of statistical information.

The diagram below illustrates a complete flow of the statistics information on the SFX server:

Flow of Statistics Information

Note how the arrows point in the direction of the statistical data flow. The gray squares indicate the processes that move the data, the blue cylinders represent the online and offline tables, and the yellow figures indicate the statistical data stored in a static medium such as a text or an HTML file.

The following illustration describes the online and offline tables:

Guidelines for SFX Statistics Maintenance

• Moving data from the online to offline tables
• Running your queries
• Archiving the offline data

• Moving data from the online to the offline tables

As described in Stat Tables, the statistics queries are always run on the offline tables. You should move data from the online tables to the offline tables as often as possible.

Note that the live/production SFX menu writes to the online tables when library end users click the SFX menu items. Therefore, if you want to store all possible interaction data for a given instance, you must move from the online to offline process during a time when there are no users connected to that particular instance (if possible). In most cases, a good time to move data from online to offline is as early as possible in the morning.

• Running your queries

Running statistics queries irregularly can be a time-consuming task if the offline table is not maintained correctly. The SFX Apache Web server is set by default to time out after 300 seconds. As a result, the display of queries you submit via the Statistics module in the SFX Admin Center may be affected.

For example, the average SFX Apache server times out after a query involving approximately 1 million records. The results of such a query are not viewable via the SFX Admin Center. This estimate may vary depending on your hardware and system load. It is therefore important that you run your queries on a regular basis and that you clean out the offline table once the data has been analyzed. If you experience timeouts that cause queries involving large amounts of data to not result in an HTML display of results via the SFX Admin Center, you may want to schedule the queries to receive the results via e-mail and archive data to reduce the burden on your offline table.

• Archiving the offline data

  It is recommended that you archive statistics data from the offline tables to prevent poor performance of your queries. The recommended practice is to not store more than 1 million records in the offline tables.

  The following table contains guidelines—based on the overall activity of your SFX installation—as to how often to move data from the online table to the offline table, run your queries, and archive your offline data.

1. Set up a scheduled job (cron job) to move the online statistics data to the offline table every day/week/month depending on the usage load of your SFX server.
2. Run statistics every week/month using the SFX Admin Center Web interface or the scheduled query options.
3. After running the statistical queries, move the data stored in the offline table to an archive file and save this archive file to a location outside the server.
4. Choose meaningful names for the archive files in order to easily retrieve them in case they need to be reloaded into the offline table for further analysis.
5. Make sure, before reloading old archived information into the offline table, that all the information in the offline table is archived and well-labeled. It is recommended that you empty the offline table before reloading old archived information to ensure that the results of your queries are accurate.