Hadoop - Benchmark

Tool

Gridmix

MapReduce workload.

GridMix is a benchmark for Hadoop clusters. It works from a MapReduce job trace describing the workload. Such traces are typically generated by Rumen. Rumen mines JobHistory logs to extract meaningful data and stores it in an easily-parsed, condensed format or digest.

Yarn Scheduler Load Simulator (SLS)

See Yarn - Scheduler Load Simulator (SLS)

Distributed System Testing

• open source software library for systems testing: https://jepsen.io/
• https://www.consul.io/docs/internals/jepsen.html

Hive

See Hive - Benchmark

Spark

See Spark - Benchmark

Documentation / Reference

• https://engineering.linkedin.com/blog/2018/02/dynamometer--scale-testing-hdfs-on-minimal-hardware-with-maximum