Read huge csv file in R from Hadoop

Vote count:

0

I am trying to read huge (250 MB, 0.6 million rows, 60 columns) CSV files in R from Hadoop. The field seperator I am using is "@" and the R code I am using is

PC2009 <- as.matrix(from.dfs("hdfs://..../apps/g00008/moloy/fuzzy/TUSOpsSODetails_2009.csv",make.input.format(format="csv",sep="@", fill=TRUE))$val)

But from 5th column onwars the data is getting corrupted. I found data to be correct in HDFS by doing a "cat". Could anyone please eductae me how to read such big files in R from Hadoop? Please note that I am new to MapReduce.

asked 45 secs ago

Dr Moloy De

6

Read huge csv file in R from Hadoop