Entity Framework 4.1 Code First: Single column foreign-key to multiple entities

I've been trying to create model in EF 4.1 to represent a database schema with a single table and column holding foreign keys from two other tables, but have had little luck with both annotations and the fluent API. A sample model is shown here:

public class User
{
    ...
    public virtual ExtendedAttribute ExtendedAttributes { get; set; }
}    

public class Account
{
    ...
    public virtual ExtendedAttribute ExtendedAttributes { get; set; }
}

public class ExtendedAttribute
{
    public Guid Id {get; set;}
    public Guid ItemId {get; set;} // both Account.Id and User.Id stored here
    public string Value { get; set; }
}

Currently the configuration for these entities looks something like this for both User and Account modelBuilders:

this.HasOptional(u => u.ExtendedAttributes).WithRequired();

Any thoughts on how to do achieve? Many thanks.

It is even not possible with the database itself and EF will not put any abstraction for that. You must have separate column and navigation property for each entity.