Filter a SQL Server table dynamically using multiple joins

I am trying to filter a single table (master) by the values in multiple other tables (filter1, filter2, filter3 ... filterN) using only joins.

I want the following rules to apply:

(A) If one or more rows exist in a filter table, then include only those rows from the master that match the values in the filter table.

(B) If no rows exist in a filter table, then ignore it and return all the rows from the master table.

(C) This solution should work for N filter tables in combination.

(D) Static SQL using JOIN syntax only, no Dynamic SQL.

I'm really trying to get rid of dynamic SQL wherever possible, and this is one of those places I truly think it's possible, but just can't quite figure it out. Note: I have solved this using Dynamic SQL already, and it was fairly easy, but not particularly efficient or elegant.

What I have tried:

1. Various INNER JOINS between master and filter tables - works for (A) but fails on (B) because the join removes all records from the master (left) side when the filter (right) side has no rows.

2. LEFT JOINS - Always returns all records from the master (left) side. This fails (A) when some filter tables have records and some do not.

What I really need:

It seems like what I need is to be able to开发者_C百科 INNER JOIN on each filter table that has 1 or more rows and LEFT JOIN (or not JOIN at all) on each filter table that is empty.

My question: How would I accomplish this without resorting to Dynamic SQL?

In SQL Server 2005+ you could try this:

WITH
  filter1 AS (
    SELECT DISTINCT
      m.ID,
      HasMatched = CASE WHEN f.ID IS NULL THEN 0 ELSE 1 END,
      AllHasMatched = MAX(CASE WHEN f.ID IS NULL THEN 0 ELSE 1 END) OVER ()
    FROM masterdata m
      LEFT JOIN filtertable1 f ON join_condition
  ),
  filter2 AS (
    SELECT DISTINCT
      m.ID,
      HasMatched = CASE WHEN f.ID IS NULL THEN 0 ELSE 1 END,
      AllHasMatched = MAX(CASE WHEN f.ID IS NULL THEN 0 ELSE 1 END) OVER ()
    FROM masterdata m
      LEFT JOIN filtertable2 f ON join_condition
  ),
  …
SELECT m.*
FROM masterdata m
  INNER JOIN filter1 f1 ON m.ID = f1.ID AND f1.HasMatched = f1.AllHasMatched
  INNER JOIN filter2 f2 ON m.ID = f2.ID AND f2.HasMatched = f2.AllHasMatched
  …

My understanding is, filter tables without any matches simply must not affect the resulting set. The output should only consist of those masterdata rows that have matched all the filters where matches have taken place.

SELECT *
FROM master_table mt
WHERE (0 = (select count(*) from filter_table_1)
      OR mt.id IN (select id from filter_table_1)
  AND (0 = (select count(*) from filter_table_2)
      OR mt.id IN (select id from filter_table_2)
  AND (0 = (select count(*) from filter_table_3)
      OR mt.id IN (select id from filter_table_3)

Be warned that this could be inefficient in practice. Unless you have a specific reason to kill your existing, working, solution, I would keep it.

Do inner join to get results for (A) only and do left join to get results for (B) only (you will have to put something like this in the where clause: filterN.column is null) combine results from inner join and left join with UNION.

Left Outer Join - gives you the MISSING entries in master table ....

SELECT * FROM MASTER M INNER JOIN APPRENTICE A ON A.PK = M.PK LEFT OUTER JOIN FOREIGN F ON F.FK = M.PK

If FOREIGN has keys that is not a part of MASTER you will have "null columns" where the slots are missing

I think that is what you looking for ...

Mike

First off, it is impossible to have "N number of Joins" or "N number of filters" without resorting to dynamic SQL. The SQL language was not designed for dynamic determination of the entities against which you are querying.

Second, one way to accomplish what you want (but would be built dynamically) would be something along the lines of:

Select ...
From master
Where Exists    (
                Select 1
                From filter_1
                Where filter_1 = master.col1
                Union All
                Select 1
                From ( Select 1 )
                Where Not Exists    (
                                    Select 1
                                    From filter_1
                                    )
                Intersect
                Select 1
                From filter_2
                Where filter_2 = master.col2
                Union All
                Select 1
                From ( Select 1 )
                Where Not Exists    (
                                    Select 1
                                    From filter_2
                                    )
                ...
                Intersect
                Select 1
                From filter_N
                Where filter_N = master.colN
                Union All
                Select 1
                From ( Select 1 )
                Where Not Exists    (
                                    Select 1
                                    From filter_N
                                    )
                        )

I have previously posted a - now deleted - answer based on wrong assumptions on you problems.

But I think you could go for a solution where you split your initial search problem into a matter of constructing the set of ids from the master table, and then select the data joining on that set of ids. Here I naturally assume you have a kind of ID on your master table. The filter tables contains the filter values only. This could then be combined into the statement below, where each SELECT in the eligble subset provides a set of master ids, these are unioned to avoid duplicates and that set of ids are joined to the table with data.

    SELECT * FROM tblData INNER JOIN 
    (
        SELECT id FROM tblData td 
            INNER JOIN fa on fa.a = td.a
        UNION 
        SELECT id FROM tblData td 
            INNER JOIN fb on fb.b = td.b
        UNION 
        SELECT id FROM tblData td 
            INNER JOIN fc on fc.c = td.c
    ) eligible ON eligible.id = tblData.id

The test has been made against the tables and values shown below. These are just an appendix.

    CREATE TABLE tblData (id int not null primary key identity(1,1), a varchar(40), b datetime, c int)

    CREATE TABLE fa (a varchar(40) not null primary key)
    CREATE TABLE fb (b datetime  not null primary key)
    CREATE TABLE fc (c int not null primary key)

Since you have filter tables, I am assuming that these tables are probably dynamically populated from a front-end. This would mean that you have these tables as #temp_table (or even a materialized table, doesn't matter really) in your script before filtering on the master data table.

Personally, I use the below code bit for filtering dynamically without using dynamic SQL.

SELECT *
FROM [masterdata] [m]
INNER JOIN
    [filter_table_1] [f1]
ON
    [m].[filter_column_1] = ISNULL(NULLIF([f1].[filter_column_1], ''), [m].[filter_column_1])

As you can see, the code NULLs the JOIN condition if the column value is a blank record in the filter table. However, the gist in this is that you will have to actively populate the column value to blank in case you do not have any filter records on which you want to curtail the total set of the master data. Once you have populated the filter table with a blank, the JOIN condition NULLs in those cases and instead joins on itself with the same column from the master data table. This should work for all the cases you mentioned in your question.

I have found this bit of code to be faster in terms of performance.

Hope this helps. Please let me know in the comments.