LINQ foreach during list creation

I have a simple LINQ statement that builds a huge monster list of around 8 billion items. Each of the items is then looped through and an operation is perform开发者_JAVA技巧ed on the item. The problem is that the foreach is not even run until the list is fully created. Is it possible to use LINQ and have it execute the foreach for each item as it builds it instead of waiting?

I am not doing anything in the list like sorting. Just straight list building.

Edit: This may seem wierd, but I have some database rows with several numeric values. I know that some arithmetic combination of those values yields a number I want, but am not sure which. Thus I created the following LINQ:

(
 from I1 in Items
 from I2 in Items
 from I3 in Items
 from I4 in Items
 select new Item[] { I1, I2, I3, I4 }
).ToList().ForEach(x => CalculateValues(x);

I did the above statement for 1 item then 2 items and so on. My computer hung for a while at the 3 items with is about 28 million items.

I do know that this is inefficient, but I was going to do a quick program to run overnight.

Sounds like you want to be using yield when constructing your list, if possible, so that you don't have 8 billion items kicking around.

e.g.

public IEnumerable<MyItem> ListBuilder()
{
   while (stillBuilding)
   {
       yield return currentItem;
   }
}

Then foreach the result of this.

If you want something concurrent/multithreaded then it's more complex and you should probably have one thread building items and feeding them into a queue and one or more consumer threads pulling items off the queue and performing operations.

Don't call ToList() that method will try to load the whole data in memory

Try this,

var q = from I1 in Items
        from I2 in Items
        from I3 in Items
        from I4 in Items
        select new Item[] { I1, I2, I3, I4 };

foreach(var x in q)
{
    CalculateValues(x);
}

that might still be slow for your input, but it will stream the results and process them one at a time if I am correct.