Tools - Website Analytics

What is Web Analytics (Web Log Analysis)

Web Log Analysis is the process concerned with the collection, measurement, analysis & reporting of user traffic insights for the purpose of understanding the workload across the website. In short Web Log Analysis is about gathering comprehensive data on end user access patterns, access times, traffic insights of customers/users accessing the system from different parts of the world, navigation pattern across the application, typical session duration, browsers used, platform used, etc. Web Log Analysis helps with obtaining a strong understanding of the usage profile of the entire web application. The web application could be an internally hosted application or an externally hosted application i.e. hosted on the internet for anyone to access.

Web site analytics is a key activity that can provide metrics required to understand traffic patterns, current system performance, end user behaviour, hotspots across the application which receive the most amount of hits, application components that generate the most amount of errors, user demand with regards to times of day when peaks and troughs (user access) occurs, etc. All of these metrics are key to build a workload model which you can then use as an input into performance testing, performance modelling and capacity management.

Analysis of Web Site Statistics

There are two main methods for performing website analytics - Web log file analysis and Script-based analysis. Every time a user visits a website, the web server has the ability to record all the relevant information about the user access request. This typically includes the time of access, types of files accessed & downloaded, browser used, operating system used, geographic location, IP address, bandwidth used, referring websites, HTTP response codes, Page Views, page processing time, search spiders, etc. All of this data is generally stored by the web server in a log files. Web log analyzers can then be used to interpret the data stored within the log files to obtain meaningful insights & stores them in a database to generate reports on demand. There are a few different Web Log File formats but some of the most common ones are -

• Apache log file formats like NCSA combined/XLF/ELF log format or common/CLF log format
• IIS log file format like W3C log format, etc.

Alternatively, Script based analyzers use a JavaScript based tracking code to be added on every webpage in the website that needs to be included in the web analytics. This tracking code places a cookie on the visitor’s computer/device to track the webpage usage & captures the user access details. Java Script based analyzers e.g. Google Analytics, sends data for all of the metrics (outlined above) to a remote server generally in an asynchronous fashion ( to avoid penalizing end user performance).

Challenges Using Web Site Analytics

It’s possible that the traffic data reported by Log Analyzer tools are higher as they process the log file data from Web server which tracks every user interaction with the website whereas Script based Analyzers record data from pages that have JavaScript tracking enabled & captures user details who use cookie/JavaScript enabled browsers (so they rarely track 100% of visitors). There’s no right or wrong way or obtaining Web Site Usage statistics. We would recommend using a combination of the above techniques.

Hope you’ve enjoyed the content in this section at Practical Performance Analyst and have learnt something new. Please help us grow the community by taking a moment and sharing this content with rest of community using your preferred Social Media Platform (links provided below). We are looking for the bright spark and if you think you have what it takes to build and grow this community reach out to me by Sending us an email.

Trevor Warren is passionate about challenging the status-quo and finding reasons to innovate. Over the past 16 years he has been delivering complex systems, has worked with very large clients across the world and constantly is looking for opportunities to bring about change. Trevor constantly strives to combine his passion for delivering outcomes with his ability to build long lasting professional relationships. You can learn more about the work he does at LinkedIn. You can download a copy of his CV at VisualCV. Visit the Github page for details of the projects he’s been hacking with.