Regression test for Robust04
This tutorial explains how to run regression tests on the Robust04 collection.
Requirements
Due to the collection's license, we cannot distribute it, and thus you will need to obtain it separately to run this test. See https://trec.nist.gov/data/cd45/index.html for more information.
Docker Image
In the repository, we provide a Docker image definition under
test/docker/benchmark
to make reproducing results easier. First, to
build the image, run the following command from the repository's root
directory:
docker build -t pisa-bench -f- .. < test/docker/benchmark/Dockerfile
You can use any name instead of pisa-bench
but if you use a different
one, make sure to substitute it in any subsequent command.
Building the image may take a while because the tools will be compiled.
Once the image is built, you can run it with:
podman run -v path/to/disk45:/opt/disk45:ro -v your/workdir:/opt/workdir --rm pisa-bench
Replace path/to/disk45
with your local path to the Robust04 disk 4 & 5
directory, and your/workdir
with the path to a local working
directory, where artifacts will be written.
By default, the script will build all indices and run evaluation. If you want to inspect the image or run some custom commands, you can execute the image interactively:
podman run -v $HOME/data/disk45:/opt/disk45:ro -v $HOME/workdir:/opt/workdir \
--rm -it pisa-bench /bin/bash