lucene file index

I have to index log record from captured from enterprice networks.In current implementation every protocol,has index files as year/mont/day/lucene file ,i want to know if i use only one single lucene index file and every day i update this single file how this effect search time ? .is it Considerable increase,in current sitiuation when i search i am querying exacly for that day. Current: smtp/year/month/ay/luceneindex

if i do smtp/luceneindex all idex in a si开发者_开发问答ngle file.Let me know prons and cons

That depends on a whole range of factors.

When you say a single lucene file? Lucene stores an index, using multiple types of files and has segments, so there is more than one file anyway.

What and how are you indexing log data?

What do you use for querying across lucene indexes, solr, elasticsearch, custom?

Are you running a single instance, single machine configuration.

Can you run multiple processes, on separate hosts, use some for search tasks and others for index updates?

What are your typical search queries like, optimise for those cases.

Have a look at http://elasticsearch.org/ or http://lucene.apache.org/solr/ for distributed search options.

lucene has options to run in memory, like RAMDirectory, you may like to investigate.

Is the size of the one-day file going to be problematic for administration? Are the File sizes going to be so large relative to disk, bandwidth constraints that copying, moving introduces issues.