Analyzing Web Statistics with AWStats

Step 6. Build the Reports
After you define your IIS Web site, it's time to get to the fun stuff—building the reports. Change to the C:\Program Files\AWStats\tools directory and execute the command

awstats_
  buildstaticpages.pl
-config=WebSiteName
-update -lang=en
-dir="C:\Inetpub\awstats"
-awstatsprog=
  "C:/Program Files/
  AWStats/wwwroot/
  cgi bin/awstats.pl"

where WebSiteName is your Web site's name (e.g., www.toombspartners.com for the Toombs Partners example). The report-generation component (i.e., awstats_buildstaticpages.pl) builds the main statistics report page, along with all related subpages, for your site and writes them to the directory specified by the -dir switch.

If everything is working properly, AWStats should create one report after another, until the processing cycle is complete. After the reporting cycle finishes, you can open a Web browser on your system and navigate to your newly created AWStats Web site. The main page of the AWStats report should be your default page. Figure 5 shows a portion of a sample main output page.

The main output page contains a number of high-level statistics and clickable links that let you drill-down into more detail. For example, if you click the Countries link under the Who section in Figure 5, you'll be sent to the Countries (Top 10) table that's further down in the main output page. Clicking the Full list link in the table's title bar takes you to a table that lists all the visitors' countries. Alternatively, you can go directly to the full list of countries by clicking the Full list link that appears after the Countries link in the Who section.

But wait—you don't have country statistics in your report? Not to worry, this is the default behavior for AWStats. Let's look at how to fix that.

One of the primary mechanisms used to determine where a visitor is coming from is to perform a reverse lookup on the visitor's IP address to see whether there's a Fully Qualified Domain Name (FQDN) associated with that address. If there's a proper FQDN associated with the IP address, the address should end in an extension such as .com, .ca, or .jp. AWStats uses this extension to determine where the visitor is coming from.

In its default configuration, AWStats doesn't perform reverse DNS lookups because it takes a significant amount of time to perform a reverse DNS lookup for every IP address that comes to a site. AWStats is highly efficient in its own processing, but when it must submit a query to an external DNS server and wait for the response, the process literally slows down exponentially. For example, the three test log files that I'm using for this article contain about 350,000 addresses. Without performing reverse DNS lookups, AWStats analyzed this much data in only a few minutes. With reverse lookups enabled, the process took more than 6 hours to complete. So, there's a significant performance hit involved with performing reverse DNS lookups.

If you want AWStats to perform reverse DNS lookups, you need to find the DNSLookup parameter in your site's configuration file and change the parameter's value from 2 to 1. After you set that parameter value, AWStats will attempt to perform a reverse DNS lookup for every IP address in any new log files you analyze. In other words, you need to manually reanalyze any existing log files. To do so, delete the AWStats database file for your site, then start the analysis process again. This time, AWStats will perform a reverse DNS lookup for all your records—but be prepared for it to take a long time. After AWStats has analyzed your log files with reverse lookups, simply create the reports again and you should have the statistics about which countries your visitors are from.

Step 7. Schedule AWStats to Run Nightly
You'll probably want to have AWStats process your log files every night so that you don't have to manually execute the analysis and report-generation applications. To do this, you can use the NightRun.bat file, which Listing 1 shows. This batch file first calls the analysis application, which analyzes the IIS log files. After the analysis process completes, the batch file then calls the report-generation application, which creates the reports and places them in the IIS directory for AWStats.

To use NightRun.bat, first download the file from the Windows IT Pro Web site. Go to http://www.windowsitpro.com, enter 45264 in the InstantDoc ID text box, then click the 45264.zip hotlink. Next, open the file in Notepad or another text editor, and replace each occurrence of www.toombspartners.com with your Web site's name. Finally, open the Control Panel Scheduled Tasks applet and schedule the batch file to run every night at some point after midnight. Why after midnight? When I showed you how to execute AWStats in Step 4, I used the -logfile switch to supply a specific log-file name. If that switch is missing, as it is in NightRun.bat, AWStats automatically chooses the previous day as the date to use for any wildcards defined in the configuration file's Logfile= parameter. So, if this batch file executes after midnight on June 4, AWStats analyzes the log files for June 3 because a wildcard date format was used in the LogFile parameter's value.

Web Statistics That Are Just a Click Away
It's nice to have a fresh set of Web statistics ready and waiting for you every morning. If you're not mining your company's Web site data, you should be, even if you aren't selling products on your site. The statistics can give you a better idea as to what people are looking for when they find your Web site—and in today's age of customer service, that's always a good thing to know.