WCF Data Services Expand behaves unexpectedly when certain entity page sizes are set

While getting our WCF Data Service ready for production we encountered an issue with the behaviour of the expand operator when paging is enabled.

With paging disabled, expand works as expected. But when I enable paging on any of the expanded entity sets, no matter what the page sizes, the expanded entities appear to page with a size of 1.

[UPDATE]

In the absence of any further input from here or the MSDN forums I've created a bug on Connect. Maybe someone over the wall will get to the bottom of it!

For example, supposed I have the following simple model:

It's running on a generated SQL database with some sample data:

INSERT INTO [dbo].[Towns] (Name) VALUES ('Berlin');
INSERT INTO [dbo].[Towns] (Name) VALUES ('Rome');
INSERT INTO [dbo].[Towns] (Name) VALUES ('Paris');

INSERT INTO [dbo].[Gentlemen] (Id, Name) VALUES (1, 'Johnny');

INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Frieda', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Adelita', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Milla', 'Berlin', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Georgine', 'Paris', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Nannette', 'Paris', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Verona', 'Rome', 1);
INSERT INTO [dbo].[Ladies] (Name, Town_Name, Gentleman_Id) VALUES ('Gavriella', 'Rome', 1);

The Data Service is straightforward (note that here paging is disabled):

namespace TestWCFDataService
{
    public class TestWCFDataService : DataService<TestModel.TestModelContainer>
    {
        // This method is called only once to initialize service-wide policies.
        public static void InitializeService(DataServiceConfiguration config)
        {
            config.SetEntitySetAccessRule("Ladies", EntitySetRights.AllRead);
            config.SetEntitySetAccessRule("Gentlemen", EntitySetRights.AllRead);
            config.SetEntitySetAccessRule("Towns", EntitySetRights.AllRead);

            //config.SetEntitySetPageSize("Ladies", 10);
            //config.SetEntitySetPageSize("Gentlemen", 10);
            //config.SetEntitySetPageSize("Towns", 10);

            config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
        }
    }
}

Now, my user wants to find every Lady whose Town is "Berlin" and also who their Gentleman is.

The query in question is:

http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies/Gentleman

When I run this query (JSON because the Atom version is gigantic) I get the expected output; a town with three ladies, all of whom have Johnny as their gentleman.

var result = {
        "d": {
            "__metadata": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')", "type": "TestModel.Town"
            }, "Name": "Berlin", "Ladies": [
    {
        "__metadata": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)", "type": "TestModel.Lady"
        }, "Id": 1, "Name": "Frieda", "Gentleman": {
            "__metadata": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
            }, "Id": 1, "Name": "Johnny", "Ladies": {
                "__deferred": {
                    "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
                }
            }
        }, "Town": {
            "__deferred": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)/Town"
            }
        }
    }, {
        "__metadata": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(2)", "type": "TestModel.Lady"
        }, "Id": 2, "Name": "Adelita", "Gentleman": {
            "__metadata": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
            }, "Id": 1, "Name": "Johnny", "Ladies": {
                "__deferred": {
                    "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
                }
            }
        }, "Town": {
            "__deferred": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(2)/Town"
            }
        }
    }, {
        "__metadata": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(3)", "type": "TestModel.Lady"
        }, "Id": 3, "Name": "Milla", "Gentleman": {
            "__metadata": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
            }, "Id": 1, "Name": "Johnny", "Ladies": {
                "__deferred": {
                    "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
                }
            }
        }, "Town": {
            "__deferred": {
                "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(3)/Town"
            }
        }
    }
    ]
        }
    }

There are going to be many Towns eventually so I enable paging for Town.

...
            config.SetEntitySetPageSize("Towns", 10);
...

The query continues to function as expected. But there are also going to be a lot of Ladies and Gentlemen so I want to be able to limit the number of results that are returned:

...
            config.SetEntitySetPageSize("Ladies", 10);
            config.SetEntitySetPageSize("Gentlemen", 10);
...

But when I set a page size on either the Ladies entity set or the Gentlemen entity set (or both) the results of my query change unexpectedly:

var result = {
    "d": {
        "__metadata": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Towns('Berlin')", "type": "TestModel.Town"
        }, "Name": "Berlin", "Ladies": {
            "results": [
{
    "__metadata": {
        "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)", "type": "TestModel.Lady"
    }, "Id": 1, "Name": "Frieda", "Gentleman": {
        "__metadata": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)", "type": "TestModel.Gentleman"
        }, "Id": 1, "Name": "Johnny", "Ladies": {
            "__deferred": {
                "开发者_StackOverflow中文版uri": "http://localhost:62946/TestWCFDataService.svc/Gentlemen(1)/Ladies"
            }
        }
    }, "Town": {
        "__deferred": {
            "uri": "http://localhost:62946/TestWCFDataService.svc/Ladies(1)/Town"
        }
    }
}
]
        }
    }
}

The expand only includes one of the Lady objects (although at least her Gentleman is included). It doesn't matter how large the page size is set to, the query still only returns one object in the expanded collection.

It also does not matter whether or not the page size is set on one or both of the expanded entities, as long as one of them has a page size set then only one of the Lady objects will be eagerly loaded.

This behaviour smells buggy to me, as according to the OData Specification:

"A URI with a $expand System Query Option indicates that Entries associated with the Entry or Collection of Entries identified by the Resource Path section of the URI must be represented inline (i.e. eagerly loaded)."

Am I misreading the spec? Should I have expected this behaviour? I just want to be able to limit the page size of the entity sets when accessed directly but also have them eagerly loadable.

Is it a bug in WCF Data Services? (or my code? or my brain?)

[EDIT]

More info: the documentation for WCF Data Services states that:

"Also, when paging is enabled in the data service, you must explicitly load subsequent data pages from the service."

But I can't find an explanation of why the page size for the related entity sets seems to default to 1 no matter what page size is specified.

[EDIT]

Yet more info: the version in question is on .NET 4 version 4.0.30319 with System.Data.Services version 4.0.0.0. It's the version that comes in the box with Visual Studio 2010 (with SP1 installed).

[EDIT]

A sample solution showing the behaviour is now up in a github repository. It's got paging turned on in the InitializeService method and a DB creation script that also adds some sample data so that we're on the same page.

It took a few months but this is apparently going to be fixed in the next version:

Posted by Microsoft on 12/15/2011 at 8:08 AM

Thank you for reporting this issue. We have confirmed that a bug in the Entity Framework is causing this issue. The fix required changes to the core Entity Framework components that ship in the .NET Framework. We have fixed the bug, the fix will be included in the next release of the .NET Framework.

What version of WCF Data Services are you using? There is a bug that I found relating to using Expand with server-driven paging in .NET Framework 4, but I thought that it only affected entities with compound keys and when using the OrderBy option, neither of which seem to apply here. Still it definitely sounds like a bug.

Have you tried using Atom instead of JSON, and if so are the entities in the expand still missing?

Your query: localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies/Gentleman

doesn't expand Ladies, only the Gentelman.

The query should look like:

localhost:62946/TestWCFDataService.svc/Towns('Berlin')?$expand=Ladies,Ladies/Gentleman.

Hope this helps!

Monica Frintu