Spark : Modify CSV file and write to other folder

Vote count: 0

Folks,

We have one requirement where we wanted to do a minor transformation on CSV file and write the same in to other HDFS folder using spark.

e.g /input/csv1.txt

ID,Name,Address
100,john,some street

output should be in file (output/csv1.txt). Basically two new columns will be added ( Order of record should be same as input file)

ID,Name,Address,Country,ZipCode
100,Name,Address,India,560001

Looks like there is no easy to do this with spark.

asked 15 secs ago

Mihirkumar Joshi

Let's block ads! (Why?)

Spark : Modify CSV file and write to other folder