Archive for the ‘Big Data’ Category

Elasticsearch: Turn off index replicas

If you’re playing with elasticsearch on a single host you may notice your cluster health is always yellow. This is probably because your indexes are set to have one replica but there’s no other node to replicate it to. To confirm if this is the case or not you can look in elasticsearch-head. In the […]

Modifying elasticsearch index settings

To view the settings of an index run the following at the command-line… ?View Code BASHcurl -XGET http://hostname:9200/indexname/_settings From here you can indeify the setting you need and modify it as you wish. This example sets the number of replicas to zero. ?View Code BASHcurl -XPUT http://hostname:9200/indexname/_settings -d ‘{ "index": {"number_of_replicas":"0"}}’ For further details see […]

Removing logstash indicies from elasticsearch‏

I’ve been playing with EFK and elasticsearch ended up eating all of the RAM on my test system. I discovered this was due to it attempting to cache all these indexes. Since this is a test system I’m not too bothered about having a long history here so I wrote this bash script to remove […]

EFK: Free Alternative to Splunk Using Fluentd

Here is an updated version of the instructions given at Free Alternative to Splunk Using Fluentd. The installation was performed in CentOS 6.5. 1. Install ElasticSearch mkdir /opt/src cd /opt/src wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.2.1.noarch.rpm rpm -ivh elasticsearch-1.2.1.noarch.rpm /sbin/chkconfig –add elasticsearch service elasticsearch start # Move default file locations if required mkdir /data/elasticsearch mkdir /data/elasticsearch/data mkdir /data/elasticsearch/tmp mkdir /data/elasticsearch/logs […]

Compiling Hadoop example MaxTemperature.java

I’m working through some of the examples in this Hadoop book. I’m a little rusty on compiling java programs and had a little trouble with this one so I’m documenting it here for anyone else how might be having issues. Firstly, I tried compiling the examples like this; javac MaxTemperature.java That wasn’t too successful; MaxTemperature.java:3: […]