spot7.org logo
Home PHP C# C++ Android Java Javascript Python IOS SQL HTML Categories

NoSuchMethodError Sets.newConcurrentHashSet() while running jar using hadoop


You are basically running into a version conflict. The problem goes like this,

  • Both hadoop native libraries and cassandra uses google guava.
  • But your hadoop version is using an older version of guava (11.xx) while your cassandra is update and uses guava 16.0. It is not very common for enterprise scale hadoop setups to update their environment with every new release.
  • cassandra config loader uses newConcurrentHashSet() method which is not present in your older version.
  • jars used by hadoop are always loaded before any third party jars. Hence even though a correct version of guava is present in your "with dependencies" jar, an older version of guava jar was being loaded from hadoop classpath and distributed to your mappers/reducers.

Solution:

  • Set the configuration parameter “mapreduce.job.user.classpath.first” to true in the run method of your Job :

    job.getConfiguration().set("mapreduce.job.user.classpath.first",
    "true");
    
  • Now, in your bin/hadoop , add the statement

    export HADOOP_USER_CLASSPATH_FIRST=true
    
    which will tell hadoop to load user defined
    libraries first. 
    
  • Make sure the latest version of your library is present in your hadoop classpath before the older one.

Categories : Java

Related to : NoSuchMethodError Sets.newConcurrentHashSet() while running jar using hadoop
Hadoop 2.5.2 mvn build fail Windows 7 x64 - compile-ms-native-dll on hadoop-common?
I've personally used the command mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true succesfully for building version 2.4.1 on windows 8. I had lots of different problems, but don't remember this being one of them. But it does seem to complain about zlib.

Categories : Java
NoSuchMethodError on Spark applications
The problem here is that the spark uses old guava version - 14.0.1. Looks like some of you dependencies bring newer version of guava. Just force to use 14.0.1 in you build tool and everything should be fine

Categories : Java
NoSuchMethodError: javax.persistence.JoinColumn.foreignKey()
Add this Maven dependency to your pom.xml: <dependency> <groupId>org.hibernate.javax.persistence</groupId> <artifactId>hibernate-jpa-2.1-api</artifactId> <version>1.0.0.Final</version> </dependency>

Categories : Java
NoSuchMethodError: registerAutoProxyCreatorIfNecessary with Spring 4, Hibernate 4 and Maven 3
I resolved my issue with this issue: Why is intelliJ not using the correct version of .xsd file My spring-context-4.1.xsd file was using a .jar in my jre/lib from my Java installation.

Categories : Java
NoSuchMethodError when creating mapping for attachment type in ElasticSearch
I had the wrong version of ElasticSearch installed. For the attachment-mapper plugin I had installed, I needed elasticsearch version 1.4. Removed old version, installed new version, installed attachment-mapper plugin, started service, and ran through tutorial again and it worked.

Categories : Elasticsearch
Recently Add
Redirecting the output directory of 'mvn package' or 'mvn compile' command
No suitable constructor found for ProductoExtranjero
java Composite design pattern(Directory &File)
Java JTree's ui refresh after removing node from parent
First REST Spring application
How to cancel Indexing of a Solr document using Update Request Processor
PowerMock - Mock a Singleton with a Private Constructor
Calling a Postgres stored function SQL error
Where to store Morphlines Java custom command class?
Generic repository using map
How can I scroll a ScrolledComposited in Eclipse SWT Design view?
2 Frames/layout in 1 Activity
Writing a switch differently
Next button opens another activity when its reaches the array limit
Is EclipseLink MOXy capable of applying JSR-303 Bean Validation when unmarshalling XML to object?
Why my jdk can't work,and before the java_home, there is a space that is not from me
How to add List of objects in a Map
How to make notepad++ function like regular notepad in cmd?
Cell renderer and the lost focus
how can I implement iterable for LinkedList>
Disable Androids image-crunch in eclipse (run as) builds
java 8 lambda myMap.stream().count() != myMap.size() after merging myMap
Issue with Calendar calculation that spans 2 calendar years
JSF 2.0 Spring bean injection
Java Regex ReplaceAll with grouping
Getting any word and last word using sed
Clicking on link on JEditorPane throws IOException
printing out difference of two arrays
Spring Bean Alias in JavaConfig
Using Factory Method to Create Generics
© Copyright 2017 spot7.org Publishing Limited. All rights reserved.