Project: refs

Lightweight, plain text analysis of Apache's access_log.

Download (tgz, 5KB)

An ancient piece of perl, hacked together aeons ago to provide a simple method of examining recent web traffic statistics. It's ugly and relatively inefficient, but I've used it with great frequency for many years.

Features

Lists recent referrers, search engine strings and visitor hostnames
Analysis of the previous week's visits, segmented into raw and unique hits, referrals from search engines and external sites, and spider visits.
Formatted plaintext output
Hyperlinked referrers when used in the context of a CGI script

Usage

tail -2000 access_log | refs

Example Output

---------------------------------+
STATISTICS FOR DOMAIN: erase.net
                 TIME: [17/02 15:35:21]

LAST 10 REFERRERS -----------------------------------------------------------+
  [16/02 20:23:31]        google: daniel jones
  [16/02 20:37:58]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 21:12:23]  http://o-db.com/en/dictionaries/russian_belorussian/
  [16/02 21:49:43]           msn: erase
  [16/02 22:25:59]  http://www.processing.org/reference/libraries/index.html
  [16/02 22:54:09]        google: mpvt vinyl
  [17/02 00:04:57]  http://www.netcraft.com/survey/
  [17/02 00:07:55]  http://processing.org/reference/libraries/index.html
  [17/02 00:19:39]        google: p5_sc daniel jones
  [17/02 00:57:29]        google: erase

LAST 10 SEARCHES -------------------------------------------------------------+
  [16/02 02:25:56]        google: erase
  [16/02 11:26:24]        google: erase
  [16/02 14:05:37]        google: processing supercolider
  [16/02 16:17:06]        google: erase net project
  [16/02 17:08:53]         yahoo: erase
  [16/02 20:23:31]        google: daniel jones
  [16/02 21:49:43]           msn: erase
  [16/02 22:54:09]        google: mpvt vinyl
  [17/02 00:19:39]        google: p5_sc daniel jones
  [17/02 00:57:29]        google: erase

LAST 10 REFERRING SITES ------------------------------------------------------+
  [16/02 16:54:05]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 17:18:48]  http://processing.org/reference/libraries/
  [16/02 18:02:13]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 18:30:52]  http://processing.org/reference/libraries/
  [16/02 19:51:21]  http://processing.org/reference/libraries/index.html
  [16/02 20:37:58]  http://forum.hardware.fr/hfr/Hardware/Materiels-problemes-
  [16/02 21:12:23]  http://o-db.com/en/dictionaries/russian_belorussian/
  [16/02 22:25:59]  http://www.processing.org/reference/libraries/index.html
  [17/02 00:04:57]  http://www.netcraft.com/survey/
  [17/02 00:07:55]  http://processing.org/reference/libraries/index.html

LAST 10 VISITORS ------------------------------------------------------------+
  [17/02 01:28:32]  74.6.28.224
  [17/02 02:09:06]  122.152.128.15
  [17/02 02:28:06]  74.6.7.126
  [17/02 02:44:11]  74.6.29.37
  [17/02 02:53:53]  65.55.104.12
  [17/02 02:54:26]  65.55.208.76
  [17/02 03:29:33]  66.249.65.52
  [17/02 03:53:35]  74.6.25.224
  [17/02 03:53:35]  74.6.19.103
  [17/02 04:03:55]  65.55.104.156

LAST 5 SPIDERS --------------------------------------------------------------+
  [13/02 04:35:23]  voilabot
  [15/02 04:39:12]  gigablast
  [16/02 20:39:02]  askjeeves
  [17/02 03:29:33]  google
  [17/02 03:53:35]  inktomi

PREVIOUS 7 DAYS
+-------+---------------+------------------------+----------------------------+
|       | HITS:         | REFERRERS:             | MISC:                      |
| DATE  | RAW    | UNIQ | SEARCH | SITE  | NONE  | SPIDER/GB | FAVS | DATA    |
+-------+--------+------+--------+-------+-------+-----------+------+---------+
| 02/17 |    142 |   12 |      2 |     2 |     8 |   31/28   |    3 |   1.6Mb |
| 02/16 |    697 |   68 |      8 |    18 |    42 |   23/13   |   23 |   5.8Mb |
| 02/15 |    944 |   67 |      9 |    16 |    42 |  153/144  |   27 |   7.7Mb |
| 02/14 |    939 |   66 |      5 |    17 |    44 |   39/29   |   24 |  12.3Mb |
| 02/13 |   1193 |   90 |      5 |    29 |    56 |   21/15   |   31 |  37.5Mb |
| 02/12 |    964 |   73 |     11 |    22 |    40 |   47/43   |   15 |  35.9Mb |
| 02/11 |   1079 |   89 |      9 |    45 |    35 |   20/12   |   27 |  10.5Mb |
+-------+--------+------+--------+-------+-------+-----------+------+---------+