Why does DistributedCache mangle my file names
I have a weird problem, DistributedCache appears to change the names of my files, it uses the original name as the parent folder and adds the file as a child.
i.e. folder\file开发者_Go百科name.ext becomes folder\filename.ext\filename.ext
Any ideas, my code is below. Thanks Akintayo
String paramsLocation="/user/fwang/settings/ecgparams.txt";
DistributedCache.addCacheFile(new URI(paramsLocation), firstStageConf);
Path[] paths = DistributedCache.getLocalCacheFiles(job);
for (Path path: paths) {
if (path.getName().equals(ecgParamsFilename)) {
File f = new File(path.toString());
if (f.exists()) {
ecgParamsFullFileName = f.getAbsolutePath();
//this becomes /user/fwang/settings/ecgparams.txt/ecgparams.txt
}
}
}
Update: 16 August 2010 Replacing the original call with the following, stops the mangling of the names.
DistributedCache.addCacheArchive(new URI(ecgParamsLocation), firstStageConf);
Are you sure you want to use addCacheArchive()
? There is a addCacheFile()
http://hadoop.apache.org/common/docs/r0.15.2/api/org/apache/hadoop/filecache/DistributedCache.html#getCacheFiles%28org.apache.hadoop.conf.Configuration%29 as well.
精彩评论