FEAT: Support for Complex Data Type- sql_variant

[gargsaumya]

Work Item / Issue Reference

AB#42724

GitHub Issue: #<ISSUE_NUMBER>

---

Summary

This pull request adds support for the sql_variant SQL Server type to the Python MSSQL driver, ensuring that sql_variant columns are fetched and mapped correctly to their underlying types. The changes introduce new constants, update type mappings, and enhance the fetch logic to detect and process sql_variant columns using their native types, improving compatibility and correctness when handling complex data.

Support for sql_variant type

• Added the SQL_SS_VARIANT constant and related attribute constants in both mssql_python/constants.py and the C++ binding layer to enable recognition and handling of the sql_variant type. [1] [2]
• Included SQL_SS_VARIANT in the set of valid types for type validation logic.

Type mapping and fetch logic

• Updated the C type mapping in cursor.py so that SQL_SS_VARIANT is mapped to SQL_C_BINARY, allowing binary transfer of the variant data.
• Implemented a helper function in the C++ layer to map a sql_variant's underlying C type to the appropriate SQL data type, enabling the fetch logic to reuse existing code for each possible underlying type.
• Enhanced the fetch routines (SQLGetData_wrap, FetchMany_wrap, and FetchAll_wrap) to detect sql_variant columns, determine their true data types at runtime, and handle them with the correct logic. This includes always using the streaming fetch path for sql_variant columns to preserve native type fidelity. [1] [2] [3]

Other improvements

• Improved error logging and debug output for easier troubleshooting and visibility into how sql_variant columns are processed. [1] [2] [3] [4]

These changes collectively ensure that the driver can now fully support reading sql_variant columns, mapping them to their actual types, and handling them efficiently.

[Copilot]

Pull request overview

Adds sql_variant read support to the MSSQL Python driver by introducing the SQL Server-specific type constant and enhancing the C++ fetch path to resolve and deserialize sql_variant values as their underlying base types, with a new test suite to validate behavior across fetch methods.

Changes:

• Added SQL_SS_VARIANT (-150) to Python constants and allowed it in SQLTypes.get_valid_types().
• Updated the C++ fetch path to detect sql_variant, map its underlying type, and route decoding through existing type handlers; adjusted LOB detection to force streaming for sql_variant.
• Added a dedicated pytest suite covering many base types and fetch patterns for sql_variant.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
mssql_python/constants.py	Introduces SQL_SS_VARIANT and includes it in valid SQL types for setinputsizes.
mssql_python/cursor.py	Adds a SQL→C type mapping entry for SQL_SS_VARIANT.
mssql_python/pybind/ddbc_bindings.cpp	Implements sql_variant type detection/mapping in SQLGetData_wrap and routes sql_variant through the streaming/LOB path for fetchmany/fetchall.
tests/test_019_sql_variant.py	New pytest coverage for fetching sql_variant across base types and fetch APIs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 86%

🎯 Overall Coverage 77%

📈 Total Lines Covered: 5730 out of 7409 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

• mssql_python/constants.py (100%)
• mssql_python/pybind/ddbc_bindings.cpp (86.4%): Missing lines 2948-2949,2961,2965,2971,3034-3038,3048-3051,4336

Summary

• Total: 111 lines
• Missing: 15 lines
• Coverage: 86%

mssql_python/pybind/ddbc_bindings.cpp

Lines 2944-2953

  2944             return SQL_WVARCHAR;
  2945         case SQL_C_DATE:
  2946         case SQL_C_TYPE_DATE:
  2947             return SQL_TYPE_DATE;
! 2948         case SQL_C_TIME:
! 2949         case SQL_C_TYPE_TIME:
  2950         case SQL_SS_VARIANT_TIME:
  2951             return SQL_TYPE_TIME;
  2952         case SQL_C_TIMESTAMP:
  2953         case SQL_C_TYPE_TIMESTAMP:

---

Lines 2957-2969

  2957         case SQL_C_GUID:
  2958             return SQL_GUID;
  2959         case SQL_C_NUMERIC:
  2960             return SQL_NUMERIC;
! 2961         case SQL_C_TINYINT:
  2962         case SQL_C_UTINYINT:
  2963         case SQL_C_STINYINT:
  2964             return SQL_TINYINT;
! 2965         default:
  2966             // Unknown C type code - fallback to WVARCHAR for string conversion
  2967             // Note: SQL Server enforces sql_variant restrictions at INSERT time, preventing
  2968             // invalid types (text, ntext, image, timestamp, xml, MAX types, nested variants,
  2969             // spatial types, hierarchyid, UDTs) from being stored. By the time we fetch data,

---

Lines 2967-2975

  2967             // Note: SQL Server enforces sql_variant restrictions at INSERT time, preventing
  2968             // invalid types (text, ntext, image, timestamp, xml, MAX types, nested variants,
  2969             // spatial types, hierarchyid, UDTs) from being stored. By the time we fetch data,
  2970             // only valid base types exist. This default handles unmapped/future type codes.
! 2971             return SQL_WVARCHAR;
  2972     }
  2973 }
  2974 
  2975 // Helper function to check if a column requires SQLGetData streaming (LOB or sql_variant)

---

Lines 3030-3042

  3030             //    Without this probe call, SQLColAttribute returns incorrect type codes.
  3031             SQLLEN indicator;
  3032             ret = SQLGetData_ptr(hStmt, i, SQL_C_BINARY, NULL, 0, &indicator);
  3033             if (!SQL_SUCCEEDED(ret)) {
! 3034                 LOG_ERROR("SQLGetData: Failed to probe sql_variant column %d - SQLRETURN=%d", i,
! 3035                           ret);
! 3036                 row.append(py::none());
! 3037                 continue;
! 3038             }
  3039             if (indicator == SQL_NULL_DATA) {
  3040                 row.append(py::none());
  3041                 continue;
  3042             }

---

Lines 3044-3055

  3044             SQLLEN variantCType = 0;
  3045             ret =
  3046                 SQLColAttribute_ptr(hStmt, i, SQL_CA_SS_VARIANT_TYPE, NULL, 0, NULL, &variantCType);
  3047             if (!SQL_SUCCEEDED(ret)) {
! 3048                 LOG_ERROR("SQLGetData: Failed to get sql_variant underlying type for column %d", i);
! 3049                 row.append(py::none());
! 3050                 continue;
! 3051             }
  3052             effectiveDataType = MapVariantCTypeToSQLType(variantCType);
  3053             LOG("SQLGetData: sql_variant column %d has variantCType=%ld, mapped to SQL type %d", i,
  3054                 (long)variantCType, effectiveDataType);
  3055         }

---

Lines 4332-4340

  4332             ret = SQLFetch_ptr(hStmt);
  4333             if (ret == SQL_NO_DATA)
  4334                 break;
  4335             if (!SQL_SUCCEEDED(ret))
! 4336                 return ret;
  4337 
  4338             py::list row;
  4339             SQLGetData_wrap(StatementHandle, numCols, row, charEncoding,
  4340                             wcharEncoding);  // <-- streams LOBs correctly

---

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.hpp: 58.8%
mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.pybind.ddbc_bindings.h: 67.8%
mssql_python.pybind.ddbc_bindings.cpp: 70.1%
mssql_python.row.py: 70.5%
mssql_python.pybind.connection.connection.cpp: 75.3%
mssql_python.__init__.py: 77.1%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.connection.py: 85.2%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sumitmsft]

@gargsaumya This is reviewed. Please make sure that we document a performance aspect of using sql variant types on the queries so that customer are aware of the fact that they may see slight degradation.