|Tool||UDP||TCP||TLS||Pipelining||Uses query file||Replay pcaps||Comments|
|dnsperf from Nominum||Y||N||N||N||Y||N||https://www.nominum.com/measurement-tools/|
|resperf from Nominum||Y||N||N||N||Y||N|
For testing resolvers
|dnsperf-tls||Y||Y||Y||Y||Y||N||https://github.com/Sinodun/dnsperf-tcp/tree/feature/tls_openssl. A re-factor was required to accomodate TLS usage within the threading model used here and we believe this introduces a performance overhead at very low queries per connection (below 500). This is being investigated.|
|perftcpdns||N||Y||N||N||Y - in hex format||N||A performance testing tool for DNS over TCP, available in the contrib directory of all recent versions of BIND9|
|queryperf++||Y||Y||N||N||Opensource framework for testing DNS servers that uses both UDP and TCP: https://github.com/jinmei/queryperfpp|
|ISC Performance lab||Y||N||N||N||Y||?||https://github.com/isc-projects/perflab|
We've also looked a some tools to see if we can reuse anything from the HTTP measurement world to help with benchmarking of Dot or DoH.
As a minimum requirement a tool must be capable of making test appear to come from at least 1000 individual clients (aka virtual users or VU in web server performance speak) but this would still require some orchestration to reach (ideally) ~30,000 VU per test VM to minimise the total number of test VM needed.
tsung status, anyway.
jim-dev-linux. Test startup took 2s on 10 VU, 9s on 100 VU, 41s on 500 VU, 63s on 750 VU and 26m9s on 1000 VU. The latter was because it went well into swap on a 4Gb RAM VM.
Others surveyed but not tested:
After further work on Tsung we concluded that peak traffic generation for single client instance of 30k clients was limited to 100kqps, as discussed below. This would mean many client VM's would be needed to achieve moderate DNS traffic levels. We suspect the fundamental reason behind this is twofold:
We did do some development work on it to add DNS as a proof of concept, the result can be found in the dns-client branch in this GitHub repo. It proved reasonably easy to get a DNS plugin doing synchronous queries working over UDP, TCP and TCP/TLS. Even taking successive lookups from a queries file as used by
dnsperf. Our specific observations are:
So, the problem is that we're unable to use Tsung to generate baseline performance numbers to compare with existing numbers. While it might (after some work) be useful for real-life type scenarios, there are concerns about how well it will scale horizontally (though that may well be the dynamic variable issue above). The original author does designed for many parallel runs of complex sessions - so, probably good for real life tests, but a problem if you're trying to establish a baseline of maximum queries a server can handle.