Vantage Giga can be a very effective tool for generating useful management reports on Internet, email, and network activity in large organizations. Because large organizations generally have a large volume of log data to report on, here are a few guidelines on how to report on enterprise storages to help make the reporting process as efficient as possible.

Partition your storage

By default Vantage Ultimate internally structures your storage by date. This means that if you run a report filtered for a specific date, Vantage Ultimate will only look at the storage partition containing the data for that date. If you run a report filtered by anything other than date, Vantage Ultimate needs to look through the entire storage to find that data. This can take some time when dealing with large storages.

If you are filtering your reports by department, which is based on the user field, it will be a good idea to partition your storage by User. When you import data into your storage, the information for each individual user will be grouped together in its own partition. When Vantage Ultimate is asked to report on that user, it only has to read that partition.

Note: if you are always reporting on all users, partitioning by the ‘User’ summary will not help. Vantage Ultimate will still have to read every partition. In fact, there is some overhead incurred when jumping between partitions, so a report on All Users using a storage partitioned by User will take longer than the same report using a storage partitioned by date (assuming there are less dates than users in your data).

You should not partition your storage by a summary that contains many highly unique items, such as URLs. This will create hundreds of thousands of small partitions, and each partition will need to be read for reports that are not filtered by that summary.

Partitioning can be very useful when using the Summaries screen in order to cut back on the time it takes to perform a drilldown, as long as the chosen partitioning matches the first summary you drilldown into. You can partition your storage on more than one summary, such as date and user. If you had three users and three dates, this will create the following 6 partitions:
• User1 date1
• User1 date2
• User1 date3
• User2 date1
• User2 date2
• User2 date3

Avoid partitioning schemes that will create many small partitions.

Ensure fast access to the log files

Ensure Vantage Ultimate has fast access to your log files to avoid network latency delaying the import. Try scheduling regular tasks to automatically transfer log files from remote office locations closer to the machine Vantage is running on.

Compress your storage folder

The size of a storage in Vantage Ultimate is generally 50-80% of the original log file size. This varies depending on the type of log file, and the amount of fields being imported (see Do not import unnecessary fields below).

Storages in Vantage Ultimate are folders containing a range of files. You can compress your storages folder using NTFS compression to reduce the size of the storage. This also increases the speed in which Vantage Ultimate reads information the storage.

To do this:
1. Navigate to your storages folder in Windows Explorer
2. Right-click the folder and select ‘Properties’
3. On the General tab, click the ‘Advanced’ button.
4. Check the ‘Compress contents to save disk space’ checkbox.
5. Select ‘Apply changes to this folder, subfolders and files’ if prompted and click OK.
6. Click Apply.

Turn off ‘Analyze data during import’

Analyzing data during import is a useful feature if you are interactively drilling down into your data on the Summaries screen. This interactive analysis becomes less feasible when you are working with very large datasets as each drilldown can take a long time.

When you are ready to import all of your logs, speed up the import process by turning this feature off. To do this select Tools | Options from the main menu and un-check the ‘Analyze data during import’ checkbox.

Figure 4: Uncheck ‘Analyze data during import’

Do not import unnecessary fields
Log files can contain many fields, some of which are not necessary for your reports. Importing these fields bloats your storage resulting in longer import and reporting times.

Use the Field Selection page of the Input Dialog tab to uncheck Summaries that you are not using in your reports. This will uncheck the fields required to generate those Summaries. You can switch between the Summaries and Fields view by selecting the appropriate option from the ‘View’
dropdown list on the toolbar.

Figure 5: Excluding unnecessary fields

Define import filters

Use the Summaries screen to look for information that you are not interested in reporting on (such as traffic generated by particular PCs, applications or certain protocols), and prevent this information from bloating your storage using import filters. These are defined on the Import Filters page of the Input Dialog when
importing your log files.

Figure 6: Excluding Windows update traffic using import filters

Distribute reporting

Distribute your reporting across multiple licenses and sites. Import logs from one network into one license and logs from another network into another license. Run these licenses on their own dedicated PCs.

Define efficient tasks
Once you have your import and report templates configured to be as efficient as possible, define tasks to automatically import data and generate reports at convenient times. For example, import tasks can be scheduled each night and reporting tasks for the end of each week or month. It is often a good idea to define tasks to import data incrementally each night, so that the storage is ready when the reports need to be generated. This avoids delaying the reporting process with long import tasks before the reporting tasks can begin.

Run Vantage Ultimate on dedicated machines

When generating large reports, Vantage Ultimate can consume a large amount of system resources (CPU and RAM). Depending on the report and the environment, it could take many hours or possibly days to generate reports. It is therefore advisable is to run Vantage Ultimate on a dedicated and highly specified reporting PC. It is NOT advised to run Vantage Ultimate on mission critical servers such as proxy servers or firewalls.

See Also: http://www.webspy.com/blogs/index.php/vantage-guidelines-for-enterprise-reports

See also:

• Management Employee Internet Reports – Enterprise Guidelines
• Increase Report Speed : Reduce What You Import
• How To: Report On Date and Time A Site Was Visited
• Report Templates: Making The Most of Your Data
• Web Module – Dynamic Employee Internet Reports