Java开发FTP功能的apache工具包，留神使用为妙

Java开发FTP功能的apache工具包，小心使用为妙

项目是一个报表系统，使用apache-commons-net网络工具包实现文件上传与下载。实际测试中报表数量比较小，没有发现大问题，并且开发时也是本地开发，本地FTP服务器处理。后来测试发现，该处理使用的是单字节不刷新读取模式，一个文件如果几百M,读入缓冲字节流的数据也会在内存中占用几百个M,这一点实际开发测试可以断点调试，使用VisualVM监控即可。

查看FTP上传下载源码，发现可以再FTPClient注入缓冲字节大小，实际测试8*1024性能比较好：

ftpClient.changeWorkingDirectory(new String(path.getBytes("utf-8"), "ISO8859-1"));
			ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
			// long start = System.currentTimeMillis();
			ftpClient.setBufferSize(1024 * 8);
			ftpClient.storeFile(new String(fileName.getBytes("utf-8"), "ISO8859-1"), input);
			// System.out.println(System.currentTimeMillis()-start);

设置后，发现效率并没有显著提高，查看storeFile文件上传源码了，里面的核心处理还记基于流的读写，如下所示：

protected boolean _storeFile(String command,
			String remote,
			InputStream local) throws IOException {
		Socket socket = _openDataConnection_(command, remote);

		if (socket == null) {
			return false;
		}

		OutputStream output = getBufferedOutputStream(socket.getOutputStream());

		if (__fileType == ASCII_FILE_TYPE) {
			output = new ToNetASCIIOutputStream(output);
		}

		CSL csl = null;
		if (__controlKeepAliveTimeout > 0) {
			csl = new CSL(this, __controlKeepAliveTimeout, __controlKeepAliveReplyTimeout);
		}

		// Treat everything else as binary for now
		try {
			Util.copyStream(local, output, getBufferSize(), CopyStreamEvent.UNKNOWN_STREAM_SIZE,
					__mergeListeners(csl), false);
		} catch (IOException e) {
			Util.closeQuietly(socket); // ignore close errors here
			if (csl != null) {
				csl.cleanUp(); // fetch any outstanding keepalive replies
			}
			throw e;
		}

		output.close(); // ensure the file is fully written
		socket.close(); // done writing the file
		if (csl != null) {
			csl.cleanUp(); // fetch any outstanding keepalive replies
		}
		// Get the transfer response
		boolean ok = completePendingCommand();
		return ok;
	}

其中的

Util.copyStream

是主要的处理方法，这里的最后一个参数为false,表示是否实时flush,但是这个方法的属性设置包中没有外部接口，坑爹了。进去看一看这个上传的源码：

 public static final long copyStream(InputStream source, OutputStream dest,
                                        int bufferSize, long streamSize,
                                        CopyStreamListener listener,
                                        boolean flush)
    throws CopyStreamException
    {
        int bytes;
        long total = 0;
        <strong>byte[] buffer = new byte[bufferSize >= 0 ? bufferSize : DEFAULT_COPY_BUFFER_SIZE];</strong>

        try
        {
            while ((bytes = source.read(buffer)) != -1)
            {
                // Technically, some read(byte[]) methods may return 0 and we cannot
                // accept that as an indication of EOF.

                if (bytes == 0)
                {
                    bytes = source.read();
                    if (bytes < 0) {
                        break;
                    }
                    dest.write(bytes);
                    <strong>if(flush) {
                        dest.flush();
                    }</strong>
                    ++total;
                    if (listener != null) {
                        listener.bytesTransferred(total, 1, streamSize);
                    }
                    continue;
                }

                dest.write(buffer, 0, bytes);
                <strong>if(flush) {
                    dest.flush();
                }</strong>
                total += bytes;
                if (listener != null) {
                    listener.bytesTransferred(total, bytes, streamSize);
                }
            }
        }
        catch (IOException e)
        {
            throw new CopyStreamException("IOException caught while copying.",
                                          total, e);
        }

        return total;
    }

看看加粗的代码，一个是缓冲流大小，一个是是否刷新流。找了半天flush没有注入接口，没办法，最后想通过继承的方式，在应用中实现这个类，并重写方法，最后发现，难度有点大，依赖属性太多。最好把FTPClient直接拉到自己应用中，全部复制吧，如下：

public class<strong> FTPClient extends org.apache.commons.net.ftp.FTPClient</strong> {

	/**
	 * The system property ({@value} ) which can be used to override the system type.<br/>
	 * If defined, the value will be used to create any automatically created parsers.
	 * 
	 * @since 3.0
	 */
	public static final String FTP_SYSTEM_TYPE = "org.apache.commons.net.ftp.systemType";

	/**
	 * The system property ({@value} ) which can be used as the default system type.<br/>
	 * If defined, the value will be used if the SYST command fails.
	 *

其它代码照搬吧。不管怎么样，最后的传输效率大大提高，而且是时刻刷新，减轻了服务器压力。

可能对此还不是深入理解，遗漏出望指教。

Java开发FTP功能的apache工具包，留神使用为妙

相关推荐