Launch a MongoDB Cluster for testing

Here’s a bash script I use to create a sharded MongoDB Cluster for testing purposes. The key functions are mongo_setup_cluster and mongo_teardown_cluster. The script will created a Mongo Cluster with 2 shards, with 3 nodes each, 3 config server and 3 mongos servers.

UPDATE 2015/10/02 I’ve found out about an undocumented option available to set the WiredTiger cache size in megabytes rather than gigabytes. Useful on test machines with limited RAM. The option is –wiredTigerEngineConfigString . To set the cache size to 200 megabytes you would do… –wiredTigerEngineConfigString=”cache_size=200M”

set -e;
set -u;
 
function mongo_teardown_cluster()
{
	killall --quiet mongos && echo "mongos processes have been murdered.";
	killall --quiet mongod && echo "mongod processes have been murdered.";
	cd ~;
	rm -Rf rhys_sharded_cluster_test_temp && echo "Directory rhys_sharded_cluster_test_temp removed.";
}
 
function mongo_create_directories()
{
	cd ~;
	mkdir -p rhys_sharded_cluster_test_temp;
	cd rhys_sharded_cluster_test_temp;
	mkdir config1 config2 config3 mongos1 mongos2 mongos3 shard0_30001 shard0_30002 shard0_30003 shard1_30004 shard1_30005 shard1_30006;
}
 
function mongo_create_config_servers()
{
	mongod --configsvr --port 27019 --dbpath ./config1 --logpath config1.log --smallfiles --nojournal --fork
	mongod --configsvr --port 27020 --dbpath ./config2 --logpath config2.log --smallfiles --nojournal --fork
	mongod --configsvr --port 27021 --dbpath ./config3 --logpath config3.log --smallfiles --nojournal --fork
}
 
function mongo_create_mongos_servers()
{
		mongos --configdb "localhost.localdomain:27019,localhost.localdomain:27020,localhost.localdomain:27021" --logpath mongos1.log --port 27017 --fork
		mongos --configdb "localhost.localdomain:27019,localhost.localdomain:27020,localhost.localdomain:27021" --logpath mongos2.log --port 27018 --fork
		mongos --configdb "localhost.localdomain:27019,localhost.localdomain:27020,localhost.localdomain:27021" --logpath mongos3.log --port 27016 --fork
}
 
function mongo_create_mongod_shard_servers()
{
	# shard0 mongod instances
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard0_30001  --port 30001 --replSet "rs0" --logpath shard0_30001.log --fork
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard0_30002  --port 30002 --replSet "rs0" --logpath shard0_30002.log --fork
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard0_30003  --port 30003 --replSet "rs0" --logpath shard0_30003.log --fork
	# shard1 mongod instances
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard1_30004  --port 30004 --replSet "rs1" --logpath shard1_30004.log --fork
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard1_30005  --port 30005 --replSet "rs1" --logpath shard1_30005.log --fork
	mongod --smallfiles --nojournal --storageEngine wiredTiger --dbpath ./shard1_30006  --port 30006 --replSet "rs1" --logpath shard1_30006.log --fork	
}
 
function mongo_configure_replicaset_rs0()
{
	mongo --port 30001 <<EOF
	rs.initiate();
	while(rs.status()['myState'] != 1) {
		print("State is not yet PRIMARY. Waiting...");
	}	
	rs.add("localhost.localdomain:30002");
	rs.add("localhost.localdomain:30003");
EOF
	STATUS=$?;
	return $STATUS;
}
 
function mongo_configure_replicaset_rs1()
{
	mongo --port 30004 <<EOF
	rs.initiate();
	while(rs.status()['myState'] != 1) {
		print("State is not yet PRIMARY. Waiting...");
	}	
	rs.add("localhost.localdomain:30005");
	rs.add("localhost.localdomain:30006");
EOF
}
 
function mongo_configure_sharding()
{
	mongo <<EOF 
	sh.addShard( "rs0/localhost.localdomain:30001" );
	sh.addShard( "rs1/localhost.localdomain:30004" );
	sh.enableSharding("test");
	sh.shardCollection("test.test_collection", { "a": 1 } );
EOF
}
 
function mongo_setup_cluster()
{
	mongo_create_directories && echo "Successfully created directories";
	mongo_create_config_servers && echo "Successfully started configuration servers." && sleep 5;
	mongo_create_mongos_servers && echo "Successfully started mongos servers." && sleep 5;
	mongo_create_mongod_shard_servers && echo "Successfully started mongod shard servers";
	echo "Sleeping for sixty seconds before attempting replicaset & shard configuration." && sleep 60;
	mongo_configure_replicaset_rs0 && echo "Successfully configured replicaset rs0." && sleep 5;
	mongo_configure_replicaset_rs1 && echo "Successfully configured replicaset rs1." && sleep 5;
	mongo_configure_sharding && echo "Successfully configured Sharding and sharded test.collection by t_u.";
	echo "TODO.... add function for loading data into test.test_collection.";
}

Usage is as follows…

. /path/to/mongo_cluster_script.sh; # Source the above bash functions
mongo_setup_cluster(); # Fire up a sharded cluster

To destroy the cluster…

mongo_teardown_cluster();

Note this will kill all mongo processes and remove all data directories!


Leave a Reply