NoSuchMethodError Sets.newConcurrentHashSet() while running jar using hadoop
You are basically running into a version
conflict. The problem goes like this,
Both hadoop native libraries and cassandra
uses google guava.
But your hadoop version is using an older
version of guava (11.xx) while your cassandra is
update and uses guava 16.0. It is not very common
for enterprise scale hadoop setups to update their
environment with every new release.
cassandra config loader uses
newConcurrentHashSet() method which is not present
in your older version.
jars used by hadoop are always loaded before
any third party jars. Hence even though a correct
version of guava is present in your "with
dependencies" jar, an older version of guava jar
was being loaded from hadoop classpath and
distributed to your mappers/reducers.
Set the configuration parameter
“mapreduce.job.user.classpath.first” to true
in the run method of your Job :
I've personally used the command mvn clean package -Pdist,native-win
-DskipTests -Dtar -Dmaven.javadoc.skip=true succesfully for building
version 2.4.1 on windows 8. I had lots of different problems, but
don't remember this being one of them. But it does seem to complain
The problem here is that the spark uses old guava version - 14.0.1.
Looks like some of you dependencies bring newer version of guava. Just
force to use 14.0.1 in you build tool and everything should be fine
I had the wrong version of ElasticSearch installed.
For the attachment-mapper plugin I had installed, I needed
elasticsearch version 1.4.
Removed old version, installed new version, installed
attachment-mapper plugin, started service, and ran through tutorial
again and it worked.