Benchmarking

Would you like to help us benchmark servers?

We are gathering data on the performance of DISCOVAR on different servers and under different configurations, and we are looking to you to help. We would like to generate a table comparing performance statistics, based on a benchmarking test we have designed. This will help us, and hopefully you too, when deciding on future hardware purchases or determining if existing servers are running efficiently.

8.6m (0.42m) 64=4x8x2 cores Intel Xeon E5-4620 2.2 GHz Centos 6.4 (Dell; PowerEdge R820)
8.3m (0.06m) 64=4×16 cores AMD Opteron 6276 2.3 GHz Centos 6.4 (Dell; PowerEdge C6145)

If you would like to participate, please follow the instructions given below. In the meantime, here is the table so far.

Server Benchmarks

Runtime* Model Hardware OS
5.8 (0.12) SGI UV2 64 (4x8x2) Intel Xeon E5-4640 2.7 GHz CentOS 6.4
6.4 (0.04) Dell R820 32 (4×8) Intel Xeon E5 4650 2.7 GHz CentOS 6.3
8.2 (0.04) BioTeam Slipstream Appliance 32 (2x8x2) Intel Xeon E5 2660 2.2 GHz CentOS 6.4
8.3 (0.06) Dell C6145 64 (4×16) AMD Opteron 6276 2.3 GHz CentOS 6.4
8.6 (0.42) Dell 820 64 (4x8x2) Intel Xeon E5-4620 2.2 GHz CentOS 6.4
8.7 (0.26) IBM X3550 M4 32 (2x8x2) Intel Xeon E5-2665 2.4 GHz CentOS 6.3
9.7 (0.09) Supermicro 64 (4×16) AMD Opteron 6274 2.2 GHz CentOS 5.9
9.7 (0.10) Dell R815 48 (4×12) AMD Opteron 6174 2.2 GHz CentOS 5.5
10.2 (0.10) HP BL406 G6 24 (2x6x2) Intel Xeon 5670 2.9 GHz CentOS 5.8
10.4 (0.04) Dell R510 24 (2x6x2) Intel Xeon 5660 2.8GHz CentOS 6.3
10.4 (0.07) Thinkmate 32 (4×8) AMD Opteron 6132HE 2.2 GHz CentOS 5.9
12.7 (1.6) Dell R910 80 (4x10x2) Intel Xeon E7 4850 2.0 GHz CentOS 5.8
12.9 (0.11) Thinkmate 48 (4×12) AMD Opteron 6174 2.2 GHz CentOS 5.9
28.3 (4.2) Dell R910 80 (4x10x2) Intel Xeon E7 4850 2.0 GHz Ubuntu

* mean (standard deviation) in minutes
total cores (processors x physical cores x logical cores) chipset clockspeed

We will update this table when we obtain new information. Note that we cannot include prices as they are generally confidential.

Running the benchmark test

Download the benchmarking test and run it on your hardware. For your convenience we have provided statically compiled binaries, but also the source code if you wish to build it yourself.

The benchmark test needs to be done on a server that has essentially no other user processes running during the duration of the test. The runtime will depend on your hardware, but we estimate it should take around 2 hours. The heart of the test is a short DISCOVAR example, which is run 10 times in order to compute a mean and standard deviation. The test does not require large amounts of memory or produce a heavy IO burden.

Send us the result by posting the LOGFILE.txt file produced by the test to this thread on the forum. Please also include a description of your hardware.

What information is collected?

Note that no information about users or jobs running on the system is collected, and nothing is transmitted back to us – other than the contents of the LOGFILE.txt file that you send us. This file contains the runtime benchmarking information, plus the following:

OS
Machine name
/proc/cpuinfo – information about the microprocessors in the system
/proc/loadavg – information about the load average
/proc/version – kernel version number, build name, etc.

Difficulties running the Benchmark?

We recommend you try running the binary test first – it is statically linked with all the required dependencies, and also includes SAMtools. If you encounter problems please let us know via the user forum. In addition to the binaries you can also download the DISCOVAR source code for the benchmark test release.