Introduction

In this series, I will introduce to you a popular backend module in real ASP.NET websites - site traffic and statistics monitoring. As the name implies, its main functionality is provide services for internal management support for the target site, rather than acts as external information dissemination and customer interaction services. For simplicity, I choose the development tools as listed in the note part below. You can, however, run the sample application in the higher ASP.NET platforms.

Traffic statistics are a simple and practical statistical analysis system, which, through real-time monitoring, can provide detailed traffic statistics reports. For example, in this kind of application you can acquire how many times the site is visited by how many people every day, where the users come from, what operating systems and the browsers the users are using. This information will be very helpful for website development strategy.

The so-called "website traffic", refers to the visit traffic, used to describe the number of users to access a site and the number of pages viewed by users. The commonly used statistical indicators include the number of independent users of the site, the total number of users (including repeated visitors), web pages browsing quantity, the number of page views per user, the average stay time of users on the site, etc.

These basic data traffic is the basis for statistical analysis of site visits. General site traffic analysis systems have certain statistics features, but most traffic statistics are limited to existing data records and a simple related summary, and accordingly draw graphs and charts based upon the analysis.

According to traffic statistics implementation process, the system modules can be divided into four design phases (also the case in our sample), as shown in Figure 1.

Figure 1: Main modules in the site traffic and statistics monitoring

In detail, the four components consist of the following info:

• Obtain customers access information. Generally you use the Request object to get the customer access information. There are no fixed requirements in the obtained information. Readers can determine the IP information according to your practical requirements. In general, this info includes the clients' IP information, client info, visit URL, stay time, origin URL information and other valuable information.
• Store customers visit information. In order to facilitate the information statistics and analyses, the obtained information must be recorded, separately processed according to different needs. For example, some info can be directly added to the database, or you can update records in the database, some info can be stored in the database after initial processing, while other info can be stored inside the Cookie, Session or Application related variables.
• Statistics and analysis. This stage is the most complex and critical step, involving a large amount of logical read and write code. Readers must make development based on targeted objectives, for example, whether making traffic statistics, and how to make the statistics, how to make classification, etc.
• Report generation. What is produced by statistical analysis is only the result data. How to make these results more intuitively displayed is the task to solve in this stage. For example, you can select to use text display, or use charts/graphs show, or use the data table, or even use the data comparison methods, and so on. Anyway, you can make full use of ASP.NET provided plentiful chart display functionalities to meet the needs of various types of charts.

For the readers to gain a more intuitive understanding with the sample application to be introduced in this series, Figure 2 illustrates one of the main running-time screenshots.

Figure 2: One of the main running-time screenshots of the sample system

Next, let's first look at the database design of the sample application.

Database Design

In real cases, database design is a complex project. Well-shaped database structure will help to optimize code writing, as well as diminish developers' workload. In this sample application, the database counter is designed using SQL Server 2008. Since what we are discussing is mainly around traffic and related statistics, there are only four tables contained in the database (but there is a lot of info inside them):

• CountContent, responsible to record data to be updated in real time;
• CounterInfo, used to store system info (relatively stable, which is not frequently written or updated);
• CounterView, responsible for the accumulation of the visitors' access information;
• IP, designed to store IP information.

Now, let's look at the detailed table structure definitions.

Table 1: Structure for table ‘CountContent’

Table CounterInfo contains all the system info, which is not needed to be updated or written frequently. Below is the structure for table CounterInfo.

Table 2: Structure for table ‘CounterInfo’

Table CounterView contains data that accumulates the visiting info of the browser. Below gives the structure for this table.

Table 3: Structure for table ‘CounterView’

Table IP contains all the IP related info. Below indicates the structure for this table.

Table 4: Structure for table ‘IP’

Database Access Helper Class

Database manipulation is a very tedious and repetitive work. To improve efficiency, we encapsulate all the database related code into a custom helper class named SQLConn which is put under the ASP.NET folder App_Code.

There are totally four public static methods defined in the class SQLConn. The first method GetSqlConnString is used to obtain the database connection string. The second method ExecuteSql is used to execute database write, update and delete operations. The third method ExecuteSqlForDataSet is responsible for query, which will return a DataSet. The last method ExecuteSqlForDataSetPageing is also used for retrieval but returns a friendlier paged DataSet.

We are not going to delve into the above four methods since they are not difficult to understand (you are sure to have found similar familiar appearances like them in your other ASP.NET projects). However, you'd better take a look at the database connection string set up in the site configure file Web.config, as follows:

As is seen, the above string definition is suitable for local trusting connection. If connected to real servers, you have to provide the user name and password (both of which are 'admin') of the database.

Obtain User Info

As is mentioned previously, the first step for traffic statistics is to obtain user visit info. According to the need of statistics, this kind of info generally includes client-side browser info, user IP address, and user operation system, etc. In real cases, there is possibly other more information involved.

In our case, the task to obtain user info is achieved through the behind file stat.aspx.cs. Since the code is very long, let's discuss them piece by piece.

Initialization

We mainly use the server side variable ServerVariables to accumulate the user info. Let's look at the first part, as shown in Listing 2 below.

There are several points worthy to be noted above. First, in the method Page_Load we define the current page immediately expires, i.e. in the successive period the page becomes overdue to prevent the browser from visiting the page info in the buffer to ignore the related statistic info.

Second, in general, when the user makes request to the Web server, the server will send back the processed info to the user. On the client side, the browser uses the buffer as a means to improve speed. In some cases, however, to prevent repeatable submissions or guarantee strict access order, we must make the user data expired as soon as it is used. Hence, when the user presses the BACK button of the browser, the system will show the data expired and cannot be used any more. As indicated above, Response.Expire is usually put at the topmost position of the method, followed by an overdue time (here 0 means immediate expire).

Next, we use ServerVariables["HTTP_REFERER"] to obtain the current user visit URL.

And next, we use the following code to judge whether the current page is an enforced one.

Next, let's look deeply into the previously-mentioned object ServerVariables.

Using ServerVariables to obtain user basic info

There are two kinds of values contained in the ServerVariables collection. One is the values sent from the client side to the HTTP header of the server side; the other is the values provided by the server itself when it receives the requests. The returned values in the ServerVariables collection include the detailed Web server info and current page related path info. You can use this info when you create a page anywhere.

For simplification, we will no more delve into the ServerVariables collection; please refer to the online MSDN related pages for more info.

Next, let's look at the related implementation in our case.

Now that we've grabbed the basic user info, it is time for us to figure out new required info.

Figure out the user info

As you've guessed, some info cannot be directly obtained through the server-side environment variable ServerVariables. However, with the trivial info ready, we can make further logical calculation to get our required data, such as user visit time, whether the user refreshing the page, user location, online user count. Refer to the following code.

Besides the above means, you can also use the Request object to get user info.

Using Request to get more user info

There are many ways to get user information. The most direct information is from the client's request; this kind of information can be obtained by resolving the client request. Since Web access is based on the HTTP protocol, this information usually comes from the HTTP request header information. ASP and ASP.NET both encapsulate objects through which to access the HTTP protocol related information. In ASP.NET, you mainly through the Request object get the client's information; in ASP, however, the Request object derives from the HttpRequest class located inside the System.Web namespace of the .NET framework class library.

There are many properties and methods defined in the Request object. You can refer to MSDN to give them a thorough study; we'll no more list them here.

Summary

In this first part of the series, you've learned the following: why establish the traffic and statistics monitoring module; the general flow and main sub modules to achieve such a task; database design in the sample application; the first step in the system – to obtain user info. In the subsequent part, we'll continue to discuss how to store the grabbed user info and make related appropriate treatment.

About Xianzhong Zhu

I'm a college teacher and also a freelance developer and writer from WeiFang China, with more than fourteen years of experience in design, and development of various kinds of products and applications on Windows platform. My expertise is in Visual C++/Basic/C#, SQL Server 2000/2005/2008, PHP+MyS... This author has published 81 articles on DotNetSlackers. View other articles or the complete profile here.