IO拷贝
public class IOFileCopy {
private static final int buffer _SIZE = 1024;
public static void copyFile(String source, String target) {
long start = System.currentTimeMillis();
try(InputStream in = new FileInputStream(new File(source));
OutputStream out = new FileOutputStream(new File(target))) {
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
System.out.println(String.format("IO file copy cost %d msc", System.currentTimeMillis() - start));
} catch (Exception e) {
e.printStackTrace();
}
}
}
传统IO中文件读取过程可以分为以下几步:
- 内核从磁盘读取数据到缓冲区,这个过程由磁盘操作器通过 DMA 操作将数据从磁盘读取到内核缓冲区,该过程 不依赖CPU
- 用户进程在将数据从内核缓冲区 拷贝 到用户空间缓冲区
- 用户进程从用户空间缓冲区读取数据
NIO拷贝
NIO进行文件拷贝有两种实现方式,一是通过管道,而是通过文件内存 内存映射
public class NIOFileCopy {
public static void copyFile(String source, String target) {
long start = System.currentTimeMillis();
try(FileChannel input = new FileInputStream(new File(source)).getChannel();
FileChannel output = new FileOutputStream(new File(target)).getChannel()) {
output.transferFrom(input, 0, input.size());
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(String.format("NIO file copy cost %d msc", System.currentTimeMillis() - start));
}
}
文件内存映射:
把内核空间地址与用户空间的虚拟地址 映射到同一个物理地址 ,DMA 硬件可以填充对内核与用户空间进程同时可见的缓冲区了。用户进程直接 从内存中读取 文件内容,应用只需要和内存打交道,不需要进行缓冲区来回拷贝,大大提高了IO拷贝的效率。加载 内存映射文件 所使用的内存在Java堆区之外
public class NIOFileCopy2 {
public static void copyFile(String source, String target) {
long start = System.currentTimeMillis();
try(FileInputStream fis = new FileInputStream(new File(source));
FileOutputStream fos = new FileOutputStream(new File(target))) {
FileChannel sourceChannel = fis.getChannel();
FileChannel targetChannel = fos.getChannel();
MappedByteBuffer mappedByteBuffer = sourceChannel.map(FileChannel.MapMode.READ_ONLY, 0, sourceChannel.size());
targetChannel.write(mappedByteBuffer);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(String.format("NIO memory reflect file copy cost %d msc", System.currentTimeMillis() - start));
File targetFile = new File(target);
targetFile.delete();
}
}
NIO内存映射文件拷贝可以分为以下几步
NIO的内存映射实际上就是少了一次从内核空间拷贝到用户空间的过程,将对用户缓冲区的读改为从内存读取
Files#copyFile方法
public class FilesCopy {
public static void copyFile(String source, String target) {
long start = System.currentTimeMillis();
try {
File sourceFile = new File(source);
File targetFile = new File(target);
Files.copy(sourceFile.toPath(), targetFile.toPath());
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(String.format("FileCopy file copy cost %d msc", System.currentTimeMillis() - start));
}
}
FileUtils#copyFile方法
使用FileUtils之前需先引入依赖
- 依赖 <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.4</version> </dependency>
- FileUtils#copyFile封装类:FileUtilsCopy.javapublic class FileUtilsCopy { public static void copyFile(String source, String target) { long start = System.currentTimeMillis(); try { FileUtils.copyFile(new File(source), new File(target)); } catch (IOException e) { e.printStackTrace(); } System.out.println(String.format(“FileUtils file copy cost %d msc”, System.currentTimeMillis() – start)); } }
性能比较
既然有这么多种实现方法,肯定要从中选择性能最佳的
测试环境:
- windows 10
- CPU 6核
- JDK1.8
测试代码:PerformTest.java
public class PerformTest {
private static final String source1 = "input/test1.txt";
private static final String source2 = "input/test2.txt";
private static final String source3 = "input/test3.txt";
private static final String source4 = "input/test4.txt";
private static final String target1 = "output/test1.txt";
private static final String target2 = "output/test2.txt";
private static final String target3 = "output/test3.txt";
private static final String target4 = "output/test4.txt";
public static void main(String[] args) {
IOFileCopy.copyFile(source1, target1);
NIOFileCopy.copyFile(source2, target2);
FilesCopy.copyFile(source3, target3);
FileUtilsCopy.copyFile(source4, target4);
}
}
总共执行了五次,读写的文件大小分别为9KB、23KB、239KB、1.77MB、12.7MB
注意:单位均为毫秒
从执行结果来看:
- 文件很小时 => IO > NIO【内存映射】> NIO【管道】 > Files#copy > FileUtils#copyFile
- 在文件较小时 => NIO【内存映射】> IO > NIO【管道】 > Files#copy > FileUtils#copyFile
- 在文件较大时 => NIO【内存映射】> > NIO【管道】> IO > Files#copy > FileUtils#copyFile
- 修改IO缓冲区大小对拷贝效率有影响,但是并不是越大性能越好,稍大于拷贝文件大小即可
文件较小时,IO效率高于NIO,NIO底层实现较为复杂,NIO的优势不明显。同时NIO内存映射初始化耗时,所以在文件较小时和IO复制相比没有优势
如果追求效率可以选择NIO的内存映射去实现文件拷贝,但是对于大文件使用内存映射拷贝要格外关注系统内存的使用率。推荐: 大文件 拷贝使用 内存映射 ,原文是这样的:
For most operating systems, mapping a file into memory is more
expensive than reading or writing a few tens of kilobytes of data via
the usual {@link #read read} and {@link #write write} methods. From the
standpoint of performance it is generally only worth mapping relatively
large files into memory
绝大多数操作系统的内存映射开销大于IO开销
同时通过测试结果来看,工具类和JDK提供的文件复制方法效果并不高,如果不追求效率还是可以使用一下,毕竟能少写一行代码就少写一行代码,写代码没有摸鱼来的快乐
来源: