We’ve been doing some work lately to make Vantage take advantage of multi-cpu and multi-core processors. It looks like we’ve improved the performance of importing multiple log files by around 50%, and the most common reporting scenarios by around 30% (tested on a quad-core CPU). Before we release this build into the wild, I’d like to give it as much field testing as possible. If you are interested in obtaining a copy of this build, please contact me using the comments form below, or email me at scottg at webspy dot com.
The multi-processing build features a new tab called ‘Performance’ in Tools | Options. Ensure ‘Use multi-processing’ is checked, and set the ‘Maximum Concurrent Threads’ value to twice the number of logical/physical CPUs you have. For example, set it to 4 on a dual core, 8 on a quad-core, or 16 on a machine with two quad-core CPUs. Feel free to also play around with this value, but we have found this formula to be optimal and we’ll automatically set this as default in future builds.
The new Performance tab in Vantage
You’ll notice that if you import a folder of logs, about 6-8 logs will import simultaneously. Simply importing one log file will not show any speed improvement.
Multi-processing will only benefit report generation if your report template contains side-by-side summaries as opposed to drilldown summaries. Here’s what I mean by a template with side-by-side summaries:
A report template that only has side-by-side summaries
And here’s what I mean by a template with drilldown summaries
A report template with only drilldown summaries
Basically, each side-by-side summary will get processed simultaneously, but any drilldowns will be processed sequentially.
Most report templates consist of both side-by-side AND drilldown summaries, such as:
A report template with both side-by-side and drilldown summaries
The amount that these templates benefit from multi-processing will depend on the number of side-by-side summaries.
If this sounds like something you’d like to try, please contact me for a copy of the build!
We use Vantage extensively, and have been for the last two years. We use the task automation to produce 2 dozen reports from approx. 30GB of data each week. This was taking up to 48 hours to complete because of our extensive drilldowns. So far with the new build we’ve seen that task time drop by almost half – just over 24 hours. We have quad core x64 server and it Vantage never really made it sweat – now we can make it push the system resources as far as we want. Very happy so far with this build, will keep testing.
We are testing Vantage to generate individual reports for the top browser users and a variety of trend reports. They are generated from logs of about 160gb a month and support for multi-processor short by half the time required. Now we take advantage of all hardware. Very happy with this change, will keep testing.
Simply Amazing
Applause for the webspy guys.
Thanks for the positive comments guys! Really glad to hear it’s going well.
Cheers!
Scott.
does the current version of Vantage Giga 2.2.0.32 support multip processors – it appears to import multiple files at once into storage – but wondered if there was more I could do to enhance progressing – your Tools, Options, use multiple processing doesn’t seem ot be there in this version – is it already in corporate or is there something I can set to get better through put ?
Hi Mark,
I have posted your comment in WebSpy Forums where our support team will respond to it, and any subsequent queries. Here’s the link: http://www.webspy.com.au/forums/viewtopic.php?f=4&t=16
All the best,
Asa