Design Pattern for .Net data reader

What do you think the "ideal" design pattern for reading a certain file type into my application.

I want to be able to reuse the code in various places, so clearly I want to have this in a separate dll, however although I have no problem passing a filename or stream into the code, what's the recommended design pattern for handling the read data.

Reading all the data into an array and passing t开发者_如何学编程his out is obviously not ideal from a memory management point of view, but the only other way I can think of doing it is by raising events when a record is read, however this doesn't feel "right" in that I would be worried that some of the events could go astray.

I'm sure there's a really slick way of doing this, but I can't for the life of me think of it.

Thanks

What is the format? If it is text, passing in a TextReader would be ideal; and for xml, an XmlReader. If it is arbitrary binary, pass in a Stream.

For large data, the ideal approach where possible is to read the data in a non-buffered way - i.e. don't load it all into memory. Iterator blocks can be useful here, for example here's a silly example that shows reading lines (but it could just as easily yield return objects built from data in a stream):

    public IEnumerable<string> ReadLines(TextReader source)
    {
        string s;
        while ((s = source.ReadLine()) != null)
            yield return s;
    }

obviously in the general case a lot more processing may be necessary per item!

The caller now has a lazily-spooling, non-buffered source of your data, for example:

using(var file = File.OpenRead(path))
{
    foreach(var customer in YourSpiffyParser(file))
        DoSomethingFun(customer);
}

And handling stream/textreader etc allows them to use it in a decorator chain with things other than files; network streams, in-memory streams, compression/crypto streams, etc.

It's difficult to define something so general. You would have to specify more what you really need, or the file type. Then you can have a look at the .NET Framework itself, there are a lot of "XXXReader" classes, for example:

• BinaryReader
• TextReader
• StreamReader
• StringReader
• XmlReader
• XmlTextReader
• IDataReader
• EventLogReader
• XamlReader
• EntityDataReader

And each one is really different from the others... Some are abstract, some aren't, etc...