Add parenthesis to check filter

[abargar]

What this does

Fixes the following bug: #2021

The bug

OR conditions in a check-level filter are not properly encapsulated, which can cause one of the OR predicates to override the other conditions in a SQL query. See the attached issue for examples: #2021

It affects:

• sampling SQL queries for checks
• the DuplicateQuery used for both sampling and metric calculation in duplicate_count checks.

The fix

I wrap parentheses around the check-level filter, e.g. a OR b becomes (a OR b).

I focused on the following checks based on the documentation:

• [-] reference check
  This check does not permit check-specific filters, so it is not affected by the bug.
• missing_count check
• missing_percent check
• invalid_count check
• invalid_percent check
• duplicate_count check
• duplicate_percent check

Testing

Test - Initial Behavior

check	failure count	sample count	sample ids
missing_count	1	7	['ID1', 'ID2', 'ID3', 'ID4', 'ID5', 'ID6', 'ID7']
missing_percent	1	7	['ID1', 'ID2', 'ID3', 'ID4', 'ID5', 'ID6', 'ID7']
invalid_count	2	6	['ID3', 'ID4', 'ID7', 'ID8', 'ID9', None]
invalid_percent	2	6	['ID3', 'ID4', 'ID7', 'ID8', 'ID9', None]
duplicate_count	2	3	['ID1', 'ID2', None]
duplicate_percent	2	3	['ID1', 'ID2', None]

Test - Final Behavior

check	failure count	sample count	sample ids
missing_count	1	1	['ID7']
missing_percent	1	1	['ID7']
invalid_count	2	2	['ID3', 'ID4']
invalid_percent	2	2	['ID3', 'ID4']
duplicate_count	3	3	['ID1', 'ID2', None]
duplicate_percent	3	3	['ID1', 'ID2', None]

[CLAassistant]

All committers have signed the CLA.

[abargar]

Note that the quotations-around-column-name formatting caused inconsistent behavior with the SparkDF DataSource. This was easily fixable in this case, but might be good to note in the following doc:

The test formatting was fixed in the commit dda387f, so this is ready for a review and rerunning the data source tests.

[abargar]

Hey! Just checking in. Anything I need to do to help see this PR through? :)

[m1n0]

hi @abargar , isn't the same issue present with the metric filter as well here?

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud