.Net 4: Easy way to dynamically create List<Tuple<...>> results

For a remoting scenario, the result would be very good to receive as an array or list of Tuple ob开发者_Python百科jects (among benefits being strong typing).

Example: dynamically convert SELECT Name, Age FROM Table => List<Tuple<string,int>>

Question: are there any samples out there that, given an arbitrary table of data (like SQL resultset or CSV file), with types of each column known only at runtime, to generate code that would dynamically create a strongly-typed List<Tuple<...>> object. Code should be dynamically generated, otherwise it would be extremely slow.

Edit: I changed the code to use the Tuple constructor instead of Tuple.Create. It currently works only for up to 8 values, but to add the 'Tuple stacking' should be trivial.

This is a little bit tricky and implementation is kind of dependent on the datasource. To give an impression, I created a solution using a list of anonymous types as a source.

As Elion said, we need to dynamically create an expression tree to call it afterward. The basic technique we employ is called projection.

We have to get, at runtime the type information and create a ConstructorInfor of the Tuple(...) constructor according to the properties count. This is dynamic (although needs to be the same per record) per each call.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;

class Program
{
    static void Main(string[] args)
    {

        var list = new[]
                       {
                           //new {Name = "ABC", Id = 1},
                           //new {Name = "Xyz", Id = 2}
                           new {Name = "ABC", Id = 1, Foo = 123.22},
                           new {Name = "Xyz", Id = 2, Foo = 444.11}
                       };

        var resultList = DynamicNewTyple(list);

        foreach (var item in resultList)
        {
            Console.WriteLine( item.ToString() );
        }

        Console.ReadLine();

    }

    static IQueryable DynamicNewTyple<T>(IEnumerable<T> list)
    {
        // This is basically: list.Select(x=> new Tuple<string, int, ...>(x.Name, x.Id, ...);
        Expression selector = GetTupleNewExpression<T>();

        var expressionType = selector.GetType();
        var funcType = expressionType.GetGenericArguments()[0]; // == Func< <>AnonType..., Tuple<String, int>>
        var funcTypegenericArguments = funcType.GetGenericArguments();

        var inputType = funcTypegenericArguments[0];  // == <>AnonType...
        var resultType = funcTypegenericArguments[1]; // == Tuple<String, int>

        var selects = typeof (Queryable).GetMethods()
            .AsQueryable()
            .Where(x => x.Name == "Select"
            );

        // This is hacky, we just hope the first method is correct, 
        // we should explicitly search the correct one
        var genSelectMi = selects.First(); 
        var selectMi = genSelectMi.MakeGenericMethod(new[] {inputType, resultType}); 

        var result = selectMi.Invoke(null, new object[] {list.AsQueryable(), selector});
        return (IQueryable) result;

    }

    static Expression GetTupleNewExpression<T>()
    {
        Type paramType = typeof (T);
        string tupleTyneName = typeof (Tuple).AssemblyQualifiedName;
        int propertiesCount = paramType.GetProperties().Length;

        if ( propertiesCount > 8 )
        {
            throw new ApplicationException(
                "Currently only Tuples of up to 8 entries are alowed. You could change this code to allow stacking of Tuples!");
        }

        // So far we have the non generic Tuple type. 
        // Now we need to create select the correct geneeric of Tuple.
        // There might be a cleaner way ... you could get all types with the name 'Tuple' and 
        // select the one with the correct number of arguments ... that exercise is left to you!
        // We employ the way of getting the AssemblyQualifiedTypeName and add the genric information 
        tupleTyneName = tupleTyneName.Replace("Tuple,", "Tuple`" + propertiesCount + ",");
        var genericTupleType = Type.GetType(tupleTyneName);

        var argument = Expression.Parameter(paramType, "x");

        var parmList = new List<Expression>();
        List<Type> tupleTypes = new List<Type>();

        //we add all the properties to the tuples, this only will work for up to 8 properties (in C#4)
        // We probably should use our own implementation.
        // We could use a dictionary as well, but then we would need to rewrite this function 
        // more or less completly as we would need to call the 'Add' function of a dictionary.
        foreach (var param in paramType.GetProperties())
        {
            parmList.Add(Expression.Property(argument, param));
            tupleTypes.Add(param.PropertyType);
        }

        // Create a type of the discovered tuples
        var tupleType = genericTupleType.MakeGenericType(tupleTypes.ToArray());

        var tuplConstructor =
            tupleType.GetConstructors().First();

        var res =
            Expression.Lambda(
                Expression.New(tuplConstructor, parmList.ToArray()),
                argument);

        return res;
    }
}

If you want to use a DataReader or some CVS input, you would need to rewrite the function GetTupleNewExpression.

I cant speak about the performance, although it should not be much slower as a native LINQ implementation as the generation of the LINQ expression only happens once per call. If its too slow you could go down the road of generating code (and keep it stored in a file) for example using Mono.Cecil.

I couldn't test this in C# 4.0 yet and but it should work. If you want to try it in C# 3.5 you need the following code as well:

public static class Tuple
{

    public static Tuple<T1, T2> Create<T1, T2>(T1 item1, T2 item2)
    {
        return new Tuple<T1, T2>(item1, item2);
    }

    public static Tuple<T1, T2, T3> Create<T1, T2, T3>(T1 item1, T2 item2, T3 item3)
    {
        return new Tuple<T1, T2, T3>(item1, item2, item3);
    }
}

public class Tuple<T1, T2>
{

    public Tuple(T1 item1, T2 item2)
    {
        Item1 = item1;
        Item2 = item2;
    }

    public T1 Item1 { get; set;}
    public T2 Item2 { get; set;}

    public override string ToString()
    {
        return string.Format("Item1: {0}, Item2: {1}", Item1, Item2);
    }

}

public class Tuple<T1, T2, T3> : Tuple<T1, T2>
{
    public T3 Item3 { get; set; }

    public Tuple(T1 item1, T2 item2, T3 item3) : base(item1, item2)
    {
        Item3 = item3;
    }

    public override string ToString()
    {
        return string.Format(base.ToString() + ", Item3: {0}", Item3);
    }
}

I was quite impressed with Dominik's building an expression to lazily create the Tuple as we iterate over the IEnumerable, but my situation called for me to use some of his concepts in a different way.

I want to load the data from a DataReader into a Tuple with only knowing the data types at run time. To this end, I created the following class:

Public Class DynamicTuple

Public Shared Function CreateTupleAtRuntime(ParamArray types As Type()) As Object
    If types Is Nothing Then Throw New ArgumentNullException(NameOf(types))
    If types.Length < 1 Then Throw New ArgumentNullException(NameOf(types))
    If types.Contains(Nothing) Then Throw New ArgumentNullException(NameOf(types))

    Return CreateTupleAtRuntime(types, types.Select(Function(typ) typ.GetDefault).ToArray)
End Function

Public Shared Function CreateTupleAtRuntime(types As Type(), values As Object()) As Object
    If types Is Nothing Then Throw New ArgumentNullException(NameOf(types))
    If values Is Nothing Then Throw New ArgumentNullException(NameOf(values))
    If types.Length < 1 Then Throw New ArgumentNullException(NameOf(types))
    If values.Length < 1 Then Throw New ArgumentNullException(NameOf(values))
    If types.Length <> values.Length Then Throw New ArgumentException("Both the type and the value array must be of equal length.")

    Dim tupleNested As Object = Nothing
    If types.Length > 7 Then
        tupleNested = CreateTupleAtRuntime(types.Skip(7).ToArray, values.Skip(7).ToArray)
        types(7) = tupleNested.GetType
        ReDim Preserve types(0 To 7)
        ReDim Preserve values(0 To 7)
    End If
    Dim typeCount As Integer = types.Length

    Dim tupleTypeName As String = GetType(Tuple).AssemblyQualifiedName.Replace("Tuple,", "Tuple`" & typeCount & ",")
    Dim genericTupleType = Type.[GetType](tupleTypeName)
    Dim constructedTupleType = genericTupleType.MakeGenericType(types)

    Dim args = types.Select(Function(typ, index)
                                If index = 7 Then
                                    Return tupleNested
                                Else
                                    Return values(index)
                                End If
                            End Function)
    Try
        Return constructedTupleType.GetConstructors().First.Invoke(args.ToArray)
    Catch ex As Exception
        Throw New ArgumentException("Could not map the supplied values to the supplied types.", ex)
    End Try
End Function

Public Shared Function CreateFromIDataRecord(dataRecord As IDataRecord) As Object
    If dataRecord Is Nothing Then Throw New ArgumentNullException(NameOf(dataRecord))
    If dataRecord.FieldCount < 1 Then Throw New InvalidOperationException("DataRecord must have at least one field.")

    Dim fieldCount = dataRecord.FieldCount
    Dim types(0 To fieldCount - 1) As Type
    Dim values(0 To fieldCount - 1) As Object
    For I = 0 To fieldCount - 1
        types(I) = dataRecord.GetFieldType(I)
    Next
    dataRecord.GetValues(values)

    Return CreateTupleAtRuntime(types, values)
End Function

End Class

Some of the differences from Dominik's solution:

1) No lazy loading. Since we would be using one record of an IDataRecord from a IDataReader at a time, I did not see an advantage in lazy loading.

2) No IQueryable, instead it outputs an Object. This could be seen as a disadvantage since you are losing type safety, but I have found that how I am using it does not really disadvantage you. If you executed a query to get the DataRecord you might know what the pattern of types are and so you can cast it directly into a strongly typed Tuple immediately after the Object return.

For another use case that I am working on (code not posted because it is still in flux), I wanted a few returned tuples to represent multiple objects being built out of a select query with multiple joins. Sometimes processing a multi-line query result into an immutable object has an impedance mismatch because you are populating an array of subtypes as you are iterating over the DataReader. I have solved this in the past by having a private mutable class while building, then creating an immutable object when the populating is done. This DynamicTuple is letting me abstract that concept that I use on several different queries to a general-purpose function to read an arbitrary joined query, build it into a List(of DynamicTuples) instead of dedicated private classes, then use that to construct the immutable data object.