在尝试处理文件之前如何验证文件是否已完成?

问题描述:

很难对我的问题给出一个好的标题,但它如下。首先,我在Windows上执行此操作,它可能也可以在Linux机器上使用,因此我需要修复程序才能在两个系统上运行。我正在监视新文件的目录。我基本上查看目录的文件并反复比较它们,只处理新文件。问题是我在尝试处理之前一直收到文件未完成的错误。

Hard to give a good title to my problem but it is as follows. First I'm doing this on windows and it could possibly be used on a linux box also so I'd need the fix to work on both systems. I am monitoring a directory for new files. I basically looking at the directory's files and comparing them over and over and only processing the new files. Problem is I keep getting an error where the file isn't finished being written before I attempt to process.

public class LiveDetectionsProvider extends DetectionsProvider {
protected LiveDetectionsProvider.MonitorDirectory monitorDirectory = null;
protected TimeModel timeModel = null;
private ArrayList<String> loaded = new ArrayList();
private File topLayerFolder = null;
public LiveDetectionsProvider(String directory, String id) {
    super(directory, id);
    timeModel = super.timeModel;
}
/**
 * Initialize the data provider.
 */
public void initialize() {
    try {
        topLayerFolder = new File(directory);
        File[] dir = topLayerFolder.listFiles();
        for (File file : dir) {
            loaded.add(file.getName());
        }
        monitorDirectory = new MonitorDirectory();
        monitorDirectory.execute();
    }
    catch (Exception ex) {
        Logger.getLogger(LiveDetectionsProvider.class.getName()).log(Level.SEVERE, "Failed to read detection\n{0}", ex.getMessage());
    }
    super.initialize();
}
/**
 * Un-initialize the data provider.
 */
public void uninitialize() {
    super.uninitialize();
    if (monitorDirectory != null) {
        monitorDirectory.continuing = false;
    }
}
/**
 * The class that is used to load the detection points in a background
 * thread.
 */
protected class MonitorDirectory extends SwingWorker<Void, Void> {
    public boolean continuing = true;
    /**
     * The executor service thread pool.
     */
    private ExecutorService executor = null;
    /**
     * The completion service that reports the completed threads.
     */
    private CompletionService<Object> completionService = null;
    @Override
    protected Void doInBackground() throws Exception {
        int count = 0;
        executor = Executors.newFixedThreadPool(1);
        completionService = new ExecutorCompletionService<>(executor);
        while (continuing && topLayerFolder != null) {
            File[] dir = topLayerFolder.listFiles();
            Thread.sleep(10);
            ArrayList<File> filesToLoad = new ArrayList();
            for (File file : dir) {
                if (!loaded.contains(file.getName())) {
                    long filesize = 0;
                    boolean cont = true;
                    while (cont) {
                        if (file.length() == filesize) {
                            cont = false;
                            Thread.sleep(3);
                            filesToLoad.add(file);
                        }
                        else {
                            filesize = file.length();
                            Thread.sleep(3);
                        }
                    }
                    Thread.sleep(3);
                }
            }
            for (File file : filesToLoad) {
                timeModel.setLoadingData(LiveDetectionsProvider.this.hashCode(), true);
                completionService.submit(Executors.callable(new ReadDetection(file, false)));
                while (completionService.take() == null) {
                    Thread.sleep(2);
                }
                loaded.add(file.getName());
                count++;
                Logger.getLogger(LiveDetectionsProvider.class.getName()).log(Level.SEVERE, "Detection Message Count:" + count);
            }
            detectionsModel.fireStateChanged(DetectionsModel.CHANGE_EVENT_DETECTIONS);
            timeModel.setLoadingData(LiveDetectionsProvider.this.hashCode(), false);
        }
        return null;
    }
}
}

该文件在line with with

The file is processed at the line with

completionService.submit(Executors.callable(new ReadDetection(file, false)));

此时的文件仍未完成写入,因此失败。我试过睡觉我的线程慢下来,我已经尝试验证文件大小没有改变。我的测试用例是我解压缩包含大量1,000 KB文件的tar文件。

The file at this point still hasnt finished being written and thus fails. I've tried sleeping my thread to slow it down, and I've tried verifying the file size hasn't changed. My test case for this is I'm unzipping a tar file which contains tons of 1,000 KB files.

首先是编译看看我发布的与代码相关的解决方案在我的问题中。你替换

First yes it compiles look a the solution I posted in relation to the code In my question. You substitute

For(File file: dir){
while(!file.renameTo(file)){
Thread.sleep(1)
}
// In my code I check to see if the file name is already in the list which 
// contains files that have been previously loaded if its not I add it to a list
// of files to be processed
}

in for

        for (File file : dir) {
            if (!loaded.contains(file.getName())) {
                long filesize = 0;
                boolean cont = true;
                while (cont) {
                    if (file.length() == filesize) {
                        cont = false;
                        Thread.sleep(3);
                        filesToLoad.add(file);
                    }
                    else {
                        filesize = file.length();
                        Thread.sleep(3);
                    }
                }
                Thread.sleep(3);
            }
        }

抱歉,我忘了将评论标签放入说你要做的就是你需要做的事情。

sorry I forgot to put comment tags // in on the line that said do what every you need to do here.

它的作用是查看目录中的每个文件并检查是否可以重命名,如果重命名失败则会睡眠并继续检查,直到它成功完成重命名,在这一点上你可以用你需要的文件做我需要的东西,在替换的for循环之后的所有东西。我很好奇为什么我的芒果被视为低于标准删除和锁定。此解决方案确实有效并解决了我的问题,并且任何有同样问题的人都会尝试处理仍在编写或复制到受监控目录的文件中的文件。

What it does is it looks at each file in the directory and checks to see if you can rename it if the rename fails it sleeps and continues checking till it sucessfully is able to rename at which point you can do what you need to with the file which in my case was everything after the for loop that was replaced. I'm curious why my awnser was viewed as sub par deleted and locked. This solution does work and solved my problem and would anyone else who's having the same issue attempting to process a file that still being written or copied to a directory thats being monitored for changes.