This project empirically evaluates Memcached, Redis and Aerospike key-value store leveraging Yahoo! Cloud System Benchmark (YCSB) with various workloads and benchmarking settings aiming to provide useful insight for each kv-store.

1. The code in this repository are based on YCSB repository.
2. Modified code is in redis and redis-binding directory. (Diff)
3. Automated scripts are added for benchmarking the three databases.
4. Configuration files for running YCSB in our benchmarking is available in benchmark directory.
5. Sample configuration files for running YCSB in our benchmarking is available in benchmark/db_configs directory.

Installing via apt-get
apt-get -y update
apt-get -y install memcached

Changing the configuration
vim /etc/memcached.conf

Running Memcached server
service memcached start

Installing via tarball wget http://download.redis.io/redis-3.2.5.tar.gz
tar xvzf redis-3.2.5.tar.gz
cd redis-3.2.5
make
make install

Changing configuration file
vim redis-3.2.5/redis.conf

Running Redis server
redis-server redis-3.2.5/redis.conf

Instsalling Aerospike server
wget -O aerospike.tgz 'http://aerospike.com/download/server/latest/artifact/ubuntu12'
tar -xvf aerospike.tgz
cd aerospike-server-community-*-ubuntu12
sudo ./asinstall

Changing configuration file
vim /etc/aerospike/aerospike.conf

Runing Aerospike server
`sudo service aerospike start &&
sudo tail -f /var/log/aerospike/aerospike.log | grep cake

Start the databases server(s) as instructed above.

Install the modified YCSB project
git clone https://github.com/anthonyaje/ycsb-kvstore-comparison.git

Notice before running cluster mode benchmarking
cd ~/ycsb-kvstore-comparison/redis-binding
cp -rf ./cluster-lib/* ./lib/

Notice before running single node benchmarking
cd ~/ycsb-kvstore-comparison/redis-binding
cp -rf ./single-lib/* ./lib/

Execute the YCSB load phase
./load_benchmark.sh [redis|riak|memcached] [single|cluster] [balanced|write_heavy|read_heavy]

Execute the YCSB run phase
./run_benchmark.sh [redis|riak|memcached] [single|cluster] [balanced|write_heavy|read_heavy]
This script will execute 4 iterations of YCSB run phase for each number of threads 1 4 8 12 16 20 24 28 32.
The results will be generated in results/$1/$2/$3/out_run_$t-$k where
$1 $2 $3 are the parameters passed when executing the scripts
$t and $k are #thread and #iteration.

Remove the result for the 1st run (the result for the first cold start)
cd to the results directory
rm out_run_*-1

Calculate the average throughput and latencies
cd to the results directory
cp ~/ycsb-kvstore-comparison/calc_* ./
./calc_tp_avg.sh ./calc_lat_read_avg.sh
./calc_lat_read_avg.sh
This scripts generate tp_avg, lat_read_avg, and lat_write_avg that contain the average throughput, read latency, and write latency for each #thread respectively.

Anthony
Yaganti Naga Malleswara Rao