Skip to content

Bracket-quote the database name in SQL Server catalog queries - #14

Open
maharanay22 wants to merge 1 commit into
mindsdb:mainfrom
maharanay22:fix-mssql-bracket-quote-database
Open

maharanay22 wants to merge 1 commit into
mindsdb:mainfrom
maharanay22:fix-mssql-bracket-quote-database

Conversation

@maharanay22

Copy link
Copy Markdown

Description

The SQL Server handler's get_tables() and all of the data catalog methods (meta_get_tables, meta_get_columns, meta_get_column_statistics, meta_get_primary_keys, meta_get_foreign_keys) put the configured database name into the query as a bare prefix of a three-part name:

FROM sqldb-prod-eastus.INFORMATION_SCHEMA.TABLES

A bare T-SQL identifier can't contain -, spaces and most other special characters. Hyphenated names are very common for Azure SQL databases (for example sqldb-prod-eastus), and SQL Server rejects these queries with a syntax error. So for those databases, listing tables (for example SHOW TABLES) and loading the data catalog fail, while regular queries (which don't use the prefix) keep working

This change quotes the database name as a T-SQL bracket identifier ([sqldb-prod-eastus]) in all of those queries, escaping ] as ]] so that a name containing ] can't break out of the quotes. Brackets work for every database name, including plain ones like example_db, so nothing changes for databases whose names were already valid bare identifiers

Fixes: no linked issue, because issues are turned off for this repository. The bug is described in full above

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Verification Process

To ensure the changes are working as expected:

  • Test Location: tests/unit/handlers/test_mssql.py, new test TestMSSQLHandler::test_database_name_is_bracket_quoted (covers all six methods) and updated test_get_tables
  • Verification Steps: capture the SQL each method sends for the database sqldb-prod-eastus and parse it with sqlglot's T-SQL dialect. I didn't have a SQL Server instance, so sqlglot stands in for the server's parser. The same queries with example_db parse fine on main, so the failures below come only from the database name

Before (current main):

get_tables                   'FROM sqldb-prod-eastus.INFORMATION_SCHEMA.TABLES WHERE TABLE'   ParseError: Invalid expression / Unexpected token. Line 6, Col: 23.
meta_get_tables              'FROM sqldb-prod-eastus.INFORMATION_SCHEMA.TABLES t LEFT JOIN'   ParseError: Invalid expression / Unexpected token. Line 8, Col: 23.
meta_get_columns             'FROM sqldb-prod-eastus.INFORMATION_SCHEMA.COLUMNS c LEFT JOI'   ParseError: Invalid expression / Unexpected token. Line 9, Col: 23.
meta_get_column_statistics   'FROM sqldb-prod-eastus.sys.tables t INNER JOIN sqldb-prod-ea'   ParseError: Invalid expression / Unexpected token. Line 11, Col: 23.
meta_get_primary_keys        'FROM sqldb-prod-eastus.INFORMATION_SCHEMA.TABLE_CONSTRAINTS '   ParseError: Invalid expression / Unexpected token. Line 7, Col: 23.
meta_get_foreign_keys        'FROM sqldb-prod-eastus.sys.foreign_keys fk INNER JOIN sqldb-'   ParseError: Invalid expression / Unexpected token. Line 8, Col: 23.

After:

get_tables                   'FROM [sqldb-prod-eastus].INFORMATION_SCHEMA.TABLES WHERE TAB'   parses OK
meta_get_tables              'FROM [sqldb-prod-eastus].INFORMATION_SCHEMA.TABLES t LEFT JO'   parses OK
meta_get_columns             'FROM [sqldb-prod-eastus].INFORMATION_SCHEMA.COLUMNS c LEFT J'   parses OK
meta_get_column_statistics   'FROM [sqldb-prod-eastus].sys.tables t INNER JOIN [sqldb-prod'   parses OK
meta_get_primary_keys        'FROM [sqldb-prod-eastus].INFORMATION_SCHEMA.TABLE_CONSTRAINT'   parses OK
meta_get_foreign_keys        'FROM [sqldb-prod-eastus].sys.foreign_keys fk INNER JOIN [sql'   parses OK

A name with ] in it, such as odd]name, renders as [odd]]name], and sqlglot reads the catalog name back as odd]name

Unit tests. The new test fails on main for all six methods and passes with the fix:

# main
E               AssertionError: '[sqldb-prod]]eastus].' not found in "... FROM sqldb-prod]eastus.INFORMATION_SCHEMA.TABLES ..."
FAILED tests/unit/handlers/test_mssql.py::TestMSSQLHandler::test_database_name_is_bracket_quoted
7 failed, 31 passed, 16 subtests passed   (6 subtests of the new test, plus the updated test_get_tables)

# this branch
32 passed, 22 subtests passed

ruff check and ruff format --check (0.11.11) pass on the changed files. No dependencies changed

Additional Media:

  • I have attached a brief loom video or screenshots showcasing the new functionality or change.

I didn't record a video or take screenshots. The before and after terminal output above shows the change instead

Checklist:

  • My code follows the style guidelines(PEP 8) of MindsDB.
  • I have appropriately commented on my code, especially in complex areas.
  • Necessary documentation updates are either made or tracked in issues.
  • Relevant unit and integration tests are updated or added.

No documentation change is needed

I used an AI assistant to help investigate this and draft the change and tests, and I reviewed and tested it myself

get_tables() and the meta_get_* methods put the configured database
name into the query as a bare three-part-name prefix, for example
FROM sqldb-prod-eastus.INFORMATION_SCHEMA.TABLES. SQL Server can't
parse a bare name that contains '-' (common for Azure SQL databases),
spaces or other special characters, so listing tables and loading the
data catalog failed for those databases.

Quote the name as a T-SQL bracket identifier, escaping ']' as ']]'.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant