我正在阅读帖子,它说使用BufferedReader或MappedByteBuffer.我决定用291.0 MB文件自己测试,但仍然无法决定
BufferedReader reader = new BufferedReader(new FileReader("/Users/rachana/part-00000")); String line = null; while((line = reader.readLine())!=null) { System.out.println(line); } ~~~~~~ Heap utilization in MB ~~~~~~ Start Date 21:10:20 End Date 21:17:48 Time used 448 second 7.50 min Used Memory In MB:28 Free Memory:81 Total Memory:109 Max Memory:1820
使用MappedByteBuffer
RandomAccessFile aFile = new RandomAccessFile ("/Users/rachana/part-00000", "r"); FileChannel inChannel = aFile.getChannel(); MappedByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size()); buffer.load(); for (int i = 0; i < buffer.limit(); i++) { System.out.print((char) buffer.get()); } buffer.clear(); // do something with the data and clear/compact it. inChannel.close(); aFile.close(); ~~~~~~ Heap utilization in MB ~~~~~~ Start Date 21:20:40 End Date 21:33:52 Time used 792 sec / 13.2 min Used Memory In MB:4 Free Memory:104 Total Memory:109 Max Memory:1820
它清楚地表明MappedByteBuffer使用更少的内存,但更多的时间,因为BufferedReader使用更多的内存但更少的时间.
我试图找到平衡也使用MappedByteBuffer读取行.
任何建议都会有所帮助