I've started working on displaying the path a particular user followed through the website; here is the first such map.
The yellow line represents the path the user followed. In the larger version, the page name and the total number of seconds elapsed are readable along the bottom. The larger version is, however, 6 megs large so i'm not linking to it from here.
I've also made it possible to view only one subset of the site in question.
This is my website starting from /other.
Code
There are currently 3 scripts:
- vw-spider - spiders through the website
- vw-map - creates map of data from vw-spider
- vw-track - creates multiple png's following a user through the website
These haven't been tested on any website other than my own and there is no installation instructions or the like. Use at your own risk; it is probably best to use them as a guide for your own projects, at least until i tighten up the code a bit.
To Do
- Need to implement a method to display more than one user at a time, perhaps a daily view of who is visiting the site.
- a Flash animation instead of an animated gif would be nice since the data is already pretty much vector based. now if only i could get the perl Flash module to work on OpenBSD.
- If i had some programmable method to display the animations, then i could also provide timings as in the amount of time the browser spent at a given page. currently the lower right displays the total elapsed time of the visitor's stay and the left shows the page in question, but in the small version this is too small to read.
- I suppose i should try spidering some other site to see how the code will break and if i can fix it. i hope i remember to change the sleeptime between hits before i spider through microsoft.com...
- My original goal was to have a t-shirt with a graphical representation of my website printed on it, so i have to get the silkscreen goods outta the closet and get it done.
Links
Stuff i found while exploring this topic.
Appearance here is no endorsement. Descriptions have been taken from either the project's webpage or the project's freshmeat entry.
nSite
nSite generates site maps for a given WWW site. It walks a site from the root URL and generates an HTML, TEXT, or XML link page which illustrates the structure and links of the site. This is a highly configurable Perl 5 script and companion module. The site map can contain the page url, title, unique fingerprint, summary, and list of internal (blue) and external (orange) links. Using this tool can be a quick way to determine the structure of an unknown or complex web site. This tool was inspired by and extends the sitemapper.pl v1.016 and WWW:Sitemap V0.002 utilities.
LinkScan
LinkScan is the industrial strength link checking and website management tool for Unix, Windows 98 and Windows NT operating systems. It is the most accurate, fastest and most scalable product available and it is highly customizable. It also does HTML validation. LinkScan creates two types of publication quality SiteMaps and it is professionally supported. LinkScan requires web server software and Perl 5. Reports may be viewed using all industry standard browsers. LinkScan comes in two versions: LinkScan WorkStation and LinkScan Server. Both operate on all Linux, Unix and Windows systems.
Sitemapper
Hierarchical Sitemapper is a Python script that crawls the file system to create an HTML site map.
html-tree
HTML-Tree is a Perl program that recursively decends directories, and creates a web-page based graphical map of HTML pages on a webserver. A configuration file provides control over the "root" directory for the map, map page title and header, directories to be excluded, link substitution strings, and map page background image. This mapper may be run as a cron task to provide an up-to-date roadmap of a webserver. It is primarily useful as a web site development and administration tool, since it shows all pages available to web browsers, and can identify where links are needed.
Visualizing the Crowds at a Web Site
A visualization of the crowds of people visiting a web site is developed. Visitors are drawn as icons on a map of the web site; the animation of people's movements conveys the crowd dynamics of visitors. The visualization combines three pieces: a map of the web site, an iconic representation of individual visitors, and an interpretation of the dynamics of visitor patterns to show crowd phenomena. The effect is to make a web site look like a social, active space.
