Installation, usage, configuration (Windows – Microsoft Visual C++)

Download and Extraction

• Install the Boost libraries (required).
• Download the latest Stxxl zip file from SourceForge
• Unpack the zip file in some directory (e. g. 'C:\' )
• Change to stxxl base directory: cd stxxl-x.y.z

Library Compilation

• Create make.settings.local in the base directory according to your system configuration:
  • set BOOST_ROOT variable according to the Boost root path, e. g. BOOST_ROOT = "C:\Program Files (x86)\boost\boost_1_40_0"#
  • (optionally) set STXXL_ROOT variable to STXXL root directory
  • (optionally) set OPT variable to /O2 or any other VC++ optimization level you like, add -D_SECURE_SCL=0 to switch off iterator checking, which improves performance
  • (optionally) set DEBUG_COMPILER=/MDd /Zi and DEBUG_LINKER=/DEBUG enable debugging
• Open the stxxl.vcproj file (VS Solution Object) in Visual Studio. The file is located in the STXXL_ROOT directory Press F7 to build the library. The library file (libstxxl.lib) should appear in STXXL_ROOT\lib directory Or build the library and the STXXL test programs by pressing Ctrl-Alt-F7 (or choosing from 'Build' drop-down menu Rebuild Solution)
• (alternatively) Compile the library by executing nmake library_msvc and the tests by executing nmake tests_msvc, with all the appropriate environment set (e. g. by using the VS Command Shell)

Building an Application

Programs using Stxxl can be compiled using options from compiler.options file (in the STXXL_ROOT directory). The linking options for the VC++ linker you can find in linker.options file. In order to accomplish this do the following:

• Open project property pages (menu Project->Properties)
• Choose C/C++->Command Line page.
• In the 'Additional Options' field insert the contents of the compiler.options file. Make sure that the Runtime libraries/debug options (/MDd or /MD or /MT or /MTd) of the Stxxl library (see above) do not conflict with the options of your project. Use the same options in the Stxxl and your project.
• Choose Linker->Command Line page.
• In the 'Additional Options' field insert the contents of the linker.options file.

If you use make files you can include the make.settings file in your make files and use STXXL_COMPILER_OPTIONS and STXXL_LINKER_OPTIONS variables, defined therein.

For example:

cl -c my_example.cpp $(STXXL_COMPILER_OPTIONS) 

link my_example.obj /out:my_example.exe $(STXXL_LINKER_OPTIONS) 

The STXXL_ROOT\test\WinGUI directory contains an example MFC GUI project that uses Stxxl. In order to compile it open the WinGUI.vcproj file in Visual Studio .NET. Change if needed the Compiler and Linker Options of the project (see above).

Before you try to run one of the STXXL examples (or your own STXXL program) you must configure the disk space that will be used as external memory for the library. For instructions how to do that, see the next section.

Disk space

To get best performance with STXXL you should assign separate disks to it. These disks should be used by the library only. Since STXXL is developed to exploit disk parallelism, the performance of your external memory application will increase if you use more than one disk. But from how many disks your application can benefit depends on how "I/O bound" it is. With modern disk bandwidths of about 50-75 MiB/s most of applications are I/O bound for one disk. This means that if you add another disk the running time will be halved. Adding more disks might also increase performance significantly.

Disk configuration file

You must define the disk configuration for an STXXL program in a file named '.stxxl' that must reside in the same directory where you execute the program. You can change the default file name for the configuration file by setting the environment variable STXXLCFG .

Each line of the configuration file describes a disk. A disk description uses the following format:
disk=full_disk_filename,capacity,access_method

Description of the parameters:

• full_disk_filename : full disk filename. In order to access disks STXXL uses file access methods. Each disk is represented as a file. If you have a disk called e: then the correct value for the full_disk_filename would be e:\some_file_name ,
• capacity : maximum capacity of the disk in megabytes
• access_method : STXXL has a number of different file access implementations for WINDOWS, choose one of them:
  • syscall : use read and write POSIX system calls (slow)
  • wincall: performs disks transfers using ReadFile and WriteFile WinAPI calls This method supports direct I/O that avoids superfluous copying of data pages in the Windows kernel. This is the best (and default) method in Stxxl for Windows.
  • boostfd : access the file using a Boost file descriptor
  • fileperblock_syscall, fileperblock_wincall, fileperblock_boostfd : same as above, but take a single file per block, using full_disk_filename as file name prefix. Usually provide worse performance than the standard variants, but release freed blocks to the file system immediately.
  • memory : keeps all data in RAM, for quicker testing
  • wbtl : library-based write-combining (good for writing small blocks onto SSDs), based on syscall

See also the example configuration file 'config_example_win' included in the archive.

Precreating external memory files

In order to get the maximum performance one should precreate disk files described in the configuration file, before running STXXL applications.

The precreation utility is included in the set of STXXL utilities ( utils\createdisks.exe ). Run this utility for each disk you have defined in the disk configuration file:

utils\createdisks.exe capacity full_disk_filename...