开发者

Why does DistributedCache mangle my file names

I have a weird problem, DistributedCache appears to change the names of my files, it uses the original name as the parent folder and adds the file as a child.

i.e. folder\file开发者_Go百科name.ext becomes folder\filename.ext\filename.ext

Any ideas, my code is below. Thanks Akintayo

String paramsLocation="/user/fwang/settings/ecgparams.txt";
DistributedCache.addCacheFile(new URI(paramsLocation), firstStageConf);

Path[] paths = DistributedCache.getLocalCacheFiles(job);
 for (Path path: paths) {
  if (path.getName().equals(ecgParamsFilename)) {
   File f = new File(path.toString());
    if (f.exists()) {
     ecgParamsFullFileName = f.getAbsolutePath();
     //this becomes /user/fwang/settings/ecgparams.txt/ecgparams.txt 
    }
  }
 }

Update: 16 August 2010 Replacing the original call with the following, stops the mangling of the names.

DistributedCache.addCacheArchive(new URI(ecgParamsLocation), firstStageConf);


Are you sure you want to use addCacheArchive()? There is a addCacheFile() http://hadoop.apache.org/common/docs/r0.15.2/api/org/apache/hadoop/filecache/DistributedCache.html#getCacheFiles%28org.apache.hadoop.conf.Configuration%29 as well.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜