Archive for the ‘Big Data’ Category

Modifying elasticsearch index settings

To view the settings of an index run the following at the command-line… ?View Code BASHcurl -XGET http://hostname:9200/indexname/_settings From here you can indeify the setting you need and modify it as you wish. This example sets the number of replicas to zero. ?View Code BASHcurl -XPUT http://hostname:9200/indexname/_settings -d ‘{ "index": {"number_of_replicas":"0"}}’ For further details see […]

Removing logstash indicies from elasticsearch‏

I’ve been playing with EFK and elasticsearch ended up eating all of the RAM on my test system. I discovered this was due to it attempting to cache all these indexes. Since this is a test system I’m not too bothered about having a long history here so I wrote this bash script to remove […]

EFK: Free Alternative to Splunk Using Fluentd

Here is an updated version of the instructions given at Free Alternative to Splunk Using Fluentd. The installation was performed in CentOS 6.5. 1. Install ElasticSearch mkdir /opt/src cd /opt/src wget rpm -ivh elasticsearch-1.2.1.noarch.rpm /sbin/chkconfig –add elasticsearch service elasticsearch start # Move default file locations if required mkdir /data/elasticsearch mkdir /data/elasticsearch/data mkdir /data/elasticsearch/tmp mkdir /data/elasticsearch/logs […]

Compiling Hadoop example

I’m working through some of the examples in this Hadoop book. I’m a little rusty on compiling java programs and had a little trouble with this one so I’m documenting it here for anyone else how might be having issues. Firstly, I tried compiling the examples like this; javac That wasn’t too successful; […]

Hadoop VersionInfo Issue on OpenSuSE 12

I was getting the following error when attempting to run hadoop version. The java class is not found: org.apache.hadoop.util.VersionInfo Unable to determine Hadoop version information. ‘hadoop version’ returned: The java class is not found: org.apache.hadoop.util.VersionInfo This was due to having the OpenJDK installed rather than the one from Sun/Oracle. To resolve this simply uninstall the […]