How can I better diagnose a WCF service when an operation returns the meta page html?

I have a WCF service application (wsHttpBinding) hosted in IIS7.5 (.NET 4) and I can navigate the .svc file in the browser and the meta data page successfully shows up and clients can be generated from wsdl.

I have added my own custom service behavior to log unhandled exceptions ("ErrorHandling" below) and I'm using MembershipProvider for service credentials. Here is my full system.serviceModel configuration:

<system.serviceModel>
  <diagnostics wmiProviderEnabled="true">
    <messageLogging
      logEntireMessage="true"
      logMalformedMessages="true"
      logMessagesAtServiceLevel="true"
      logMessagesAtTransportLevel="true"
      maxMessagesToLog="3000"
      maxSizeOfMessageToLog="-1" />
  </diagnostics>
  <extensions>
    <behaviorExtensions>
      <add name="ErrorHandling" type="MyCompany.MyProduct.Services.ErrorHandlerBehavior, MyCompany.MyProduct.Services" />
    </behaviorExtensions>
  </extensions>
  <services>
    <service name="MyCompany.MyProduct.Services.ApplicationService" behaviorConfiguration="MyProductServiceBehavior">
      <endpoint bindingConfiguration="MyProductWsHttpBinding" binding="wsHttpBinding" contract="MyCompany.MyProduct.Services.IApplicationService" />
    </service>
    <service name="MyCompany.MyProduct.Services.AccountService" behaviorConfiguration="MyProductServiceBehavior">
      <endpoint bindingConfiguration="MyProductWsHttpBinding" binding="wsHttpBinding" contract="MyCompany.MyProduct.Services.IAccountService" />
    </service>
    <service name="MyCompany.MyProduct.Services.InvoiceService" behaviorConfiguration="MyProductServiceBehavior">
      <endpoint bindingConfiguration="MyProductWsHttpBinding" binding="wsHttpBinding" contract="MyCompany.MyProduct.Services.IInvoiceService" />
    </service>
    <service name="MyCompany.MyProduct.Services.ReportService" behaviorConfiguration="MyProductServiceBehavior">
      <endpoint bindingConfiguration="MyProductWsHttpBinding" binding="wsHttpBinding" contract="MyCompany.MyProduct.Services.IReportService" />
    </service>
    <service name="MyCompany.MyProduct.Services.PaymentService" behaviorConfiguration="MyProductServiceBehavior">
      <endpoint bindingConfiguration="MyProductWsHttpBinding" binding="wsHttpBinding" contract="MyCompany.MyProduct.Services.IPaymentService" />
    </service>
  </services>
  <bindings>
    <wsHttpBinding>
      <binding name="MyProductWsHttpBinding">
        <security mode="Message">
          <message clientCredentialType="UserName" establishSecurityContext="true" negotiateServiceCredential="true" />
        </security>
      </binding>
    </wsHttpBinding>
  </bindings>
  <behaviors>
    <serviceBehaviors>
      <behavior name="MyProductServiceBehavior">
        <serviceMetadata httpGetEnabled="true" />
        <serviceDebug includeExceptionDetailInFaults="true" />
        <serviceCredentials>
          <serviceCertificate findValue="MyProductServices" storeLocation="CurrentUser" storeName="My" x509FindType="FindBySubjectName" />
          <userNameAuthentication userNamePasswordValidationMode="MembershipProvider" membershipProviderName="AspNetSqlMembershipProvider" />
        </serviceCredentials>
        <ErrorHandling />
        <serviceSecurityAudit auditLogLocation="Application" suppressAuditFailure="false" serviceAuthorizationAuditLevel="SuccessOrFailure" messageAuthenticationAuditLevel="SuccessOrFailure" />
      </behavior>
    </serviceBehaviors>
  </behaviors>
  <serviceHostingEnvironment multipleSiteBindingsEnabled="true" aspNetCompatibilityEnabled="true" />
</system.serviceModel>

You can see that I also have security auditing enabled, however nothing shows up in the event log except for the message saying that message logging has been turned on. So it seems when the client calls the operation it never gets to the authentication phase.

I also have tracing enabled as follows:

<system.diagnostics>
  <sources>
    <source name="System.ServiceModel"
            switchValue="All"
            propagateActivity="true" >
      <listeners>
        <add name="xml"/>
      </listeners>
    </source>
    <source name="System.ServiceModel.MessageLogging" switchValue="All">
      <listeners>
        <add name="xml"/>
      </listeners>
    </source>
    <source name="myUserTraceSource"
            switchValue="Information, ActivityTracing">
      <listeners>
        <add name="xml"/>
      </listeners>
    </source>
  </sources>
  <sharedListeners>
    <add name="xml"
         type="System.Diagnostics.XmlWriterTraceListener"
               initializeData="C:\Users\Public\Documents\services.svclog" />
  </sharedListeners>
</system.diagnostics>

When the client calls the main operation in the primary service it gets a protocol exception with this message:

The content type text/html; charset=UTF-8 of the response message does not match the content type of the binding (application/soap+xml; charset=utf-8). If using a custom encoder, be sure that the IsContentTypeSupported method is implemented properly. The first 1024 bytes of the response were: '#content{ FONT-SIZE: 0.7em; PADDING-BOTTOM: 2em; MARGIN-LEFT: 30px}BODY{MARGIN-TOP: 0px; MARGIN-LEFT: 0px; COLOR: #000000; FONT-FAMILY: Verdana; BACKGROUND-COLOR: white}P{MARGIN-TOP: 0px; MARGIN-BOTTOM: 12px; COLOR: #000000; FONT-FAMILY: Verdana}PRE{BORDER-RIGHT: #f0f0e0 1px solid; PADDING-RIGHT: 5px; BORDER-TOP: #f0f0e0 1px solid; MARGIN-TOP: -5px; PADDING-LEFT: 5px; FONT-SIZE: 1.2em; PADDING-BOTTOM: 5px; BORDER-LEFT: #f0f0e0 1px solid; PADDING-TOP: 5px; BORDER-BOTTOM: #f0f0e0 1px solid; FONT-FAMILY: Courier New; BACKGROUND-COLOR: #e5e5cc}.heading1{MARGIN-TOP: 0px; PADDING-LEFT: 15px; FONT-WEIGHT: normal; FONT-SIZE: 26px; MARGIN-BOTTOM: 0px; PADDING-BOTTOM: 3px; MARGIN-LEFT: -30px; WIDTH: 100%; COLOR: #ffffff; PADDING-TOP: 10px; FONT-FAMILY: Tahoma; BACKGROUND-COLOR: #003366}.intro{MARGIN-LEFT: -15px}ApplicationService Service'.

Which is the html for the meta data page you see when you browse to .svc file. So it appears as if the service is returning that page as the response to the operation.

Viewing services.svclog tells the same story, in fact the log is malformed xml because this response is included in it without the tags being escaped. There's no helpful information what so ever in the trace.

There are no exceptions thrown on the server, the logs have nothing. This all works on IIS6 on another server I setup just to te开发者_运维知识库st it out and the error logging works there as well.

What tools and techniques can help me figure out what the root problem is?

UPDATE: Here is the svclog from server side tracing done for a single request from a newly generated console app client (scrubbed to remove personal information), notice how it is truncated at the end: http://pastebin.com/mksL0FFY

Since the event log is empty, you should check the IIS log to make sure the call is succeeding. I've experienced similar error messages and the problem was usually related to permissions. Check that the account(s) used for the app pool and website have sufficient priveleges. I think you'd see an event log message if there was a problem authenticating with the membership provider. Also, try calling the method with the WcfTestClient app so you can see the XML being transmitted each way.

Check the URL Rewrite rules in IIS for the site. If one of them is redirecting it could cause clients to receive the metadata page as the response to the operation. I've seen that happen before, it can be misleading.