Spark Jar : Caused by: java.lang.ClassNotFoundException: csv.DefaultSource

 Exception

Caused by: java.lang.ClassNotFoundException: Failed to find data source: csv. Please find packages at http://spark.apache.org/third-party-projects.html

at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:660)

at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:194)

at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:619)

at org.apache.spark.sql.DataFrameReader.csv(DataFrameReader.scala:468)

at SparkGetAddressInfo$.<init>(SparkGetAddressInfo.scala:24)

at SparkGetAddressInfo$.<clinit>(SparkGetAddressInfo.scala)

... 11 more

Caused by: java.lang.ClassNotFoundException: csv.DefaultSource

at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

at java.lang.ClassLoader.loadClass(ClassLoader.java:418)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)

at java.lang.ClassLoader.loadClass(ClassLoader.java:351)

at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$5(DataSource.scala:634)

at scala.util.Try$.apply(Try.scala:213)

at org.apache.spark.sql.execution.datasources.DataSource$.$anonfun$lookupDataSource$4(DataSource.scala:634)

at scala.util.Failure.orElse(Try.scala:224)

at org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:634)

Solution

Add below in the build.sbt of the project

assemblyMergeStrategy in assembly := {

case path if path.contains("META-INF/services") => MergeStrategy.concat

case PathList("META-INF", xs @ _*) => MergeStrategy.discard

case x => MergeStrategy.first

}