How to convert a big csv file into json array quickly in java -



How to convert a big csv file into json array quickly in java -

i want convert big csv file 20000 50000 record file json array takes 1 min convert in there way accomplish in less 5 sec.

resourcebundle rb = resourcebundle.getbundle("settings"); string path = rb.getstring("fileandfolder.path"); system.out.println(path + "ssdd"); string csvpath = request.getparameter("dp") != null ? request .getparameter("dp").tostring() : ""; string orname = path + csvpath; file file = new file(orname); filereader fin = new filereader(file); //read file 1 1 bufferedreader bi = new bufferedreader(fin); int res; string csv = ""; while ((res = fin.read()) != -1) { csv = csv + ((char) res); //converted int char , stored in csv } long start3 = system.nanotime(); jsonarray array = cdl.tojsonarray(csv); string csvs = array.tostring(); long time3 = system.nanotime() - start3; system.out .printf("took %.3f seconds convert %d mb file, rate: %.1f mb/s%n", time3 / 1e9, file.length() >> 20, file.length() * 1000.0 / time3);

there 2 glaring performance problems in code, both of them in snippet:

while ((res = fin.read()) != -1) { csv = csv + ((char) res); }

first problem: fin unbuffered filereader, each read() phone call doing scheme call. each syscall hundreds or thousands of instructions. , doing each , every character in input file.

remedy: read bi rather fin. (that's created ... presumably.)

second problem: each time execute csv = csv + ((char) res); creating new string 1 character longer previous one. if have n characters in input file, end copying n^2 characters build string.

remedy: instead of concatenating strings, utilize stringbuilder ... this:

stringbuilder sb = new stringbuilder(); .... sb.append((char) res); .... string csv = sb.tostring();

at point, not clear me if there performance problem in converting csv string json; i.e. in snippet.

jsonarray array = cdl.tojsonarray(csv); string csvs = array.tostring();

unfortunately, don't know jsonarray , cdl classes using. hence, hard why slow, or whether there faster way conversion. (but suspect, biggest performance problems in before snippet.)

java json

Comments

Popular posts from this blog

php - Android app custom user registration and login with cookie using facebook sdk -

django - Access session in user model .save() -

php - .htaccess Multiple Rewrite Rules / Prioritizing -