开发者

MapReduce Distributed Cache

I am adding a file to distributed cache of Hadoop using

     Configuration cng=new Configuration();
     JobConf conf = new JobConf(cng, Driver.class);
     DistributedCache.addCacheFile(new Path("DCache/Orders.txt").toUri(), cng);

where DCache/开发者_Go百科Orders.txt is the file in HDFS.

When I try to retrieve this file from the cache in configure method of mapper using:

    Path[] cacheFiles=DistributedCache.getLocalCacheFiles(conf);

I get null pointer. What can be the error?

Thanks


DistributedCache doesn't work in single node mode, it just returns a null pointer. Or at least that was my experience with the current version.

I think the url is supposed to start with the hdfs identifier.

http://hadoop.apache.org/common/docs/current/mapred_tutorial.html#DistributedCache

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜