Subset counting algorithm

I have a following problem I want to solve efficiently. I am given a set of k-tuples of Boolean values where I know in advance that some fraction of each of the values in each of the k-tuples is true. For example, I might have the following 4-tuples, where each tuple has at least 60% of it's Boolean values set to true:

(1, 0, 1, 0)
(1, 1, 0, 1)
(0, 0, 1, 0)

I am interested in finding se开发者_高级运维ts of indices that have a particular property: if I look at each of the values in the tuples at the indicated indices, at least the given fraction of those tuples have the corresponding bit set. For example, in the above set of 4-tuples, I could consider the set {0}, since if you look at the zeroth element of each of the above tuples, two-thirds of them are 1, and 2/3 ~= 66% > 60%. I could also consider the set {2} for the same reason. However, I could not consider {1}, since at index 1 only one third of the tuples have a 1 and 1/3 is less than 60%. Similarly, I could not use {0, 2} as a set, because it is not true that at least 60% of the tuples have both bits 0 and 2 set.

My goal is to find all sets for which this property holds. Does anyone have a good algorithm for solving this?

Thank you.

As you've wrote, that can be assumed that architecture is x86_64 and you are looking for implementation performance, cause asymptotic complexity (as it is not going to go under linear - by definition of problem ;) ), I propose following algorithm (C++ like pseudocode):

/* N=16 -> int16; N=8 -> int8 etc. Select N according to input sizes. (maybe N=24 ;) ) */
count_occurences_intN(vector<intN> t, vector<long> &result_counters){
   intN counters[2^N]={};
   //first, count bit combinations
   for_each(v in t)
       ++counters[v];
   //second, count bit occurrences, using aggregated data 
   for(column=0; column<N; ++column){
      mask = 1 << column;
      long *result_counter_ptr = &(result_counters[column]);
      for(v=0; v<2^16; ++v)
         if( v & mask )
            ++(*result_counter_ptr);
   }
}

Than, split your input k-bit vectors into N-bit vectors, and apply above function.

Depending on input size you might improve performance you choosing N=8, N=16, N=24 or applying naive approach.

As you've wrote, you can not assume anything on client side, just implement N={8,16,24} and naive and select one from four implementations depending on size of input.

Make a k-vector of integers, describing how many passes there were for each index. Loop through your set, for each element incrementing the k-vector of passes.

Then figure out the cardinality of your set (either in a separate loop, or in the above one). Then loop through your vector of counts, and emit a pass/fail vector based on your criteria.