dataframe_from_result_table does not work with pandas 3.0 on a column with all empty datetime elements

[amarianne]

Code Sample, a copy-pastable example if possible

from azure.kusto.data import KustoClient
from azure.kusto.data.helpers import dataframe_from_result_table

kusto_client = KustoClient(...)

query =  """
datatable (timestamp:datetime, value:real )[
    datetime(null), 10,
    datetime(null), 11
]
"""
dataframe_from_result_table(kusto_client.execute(database="test_db", query=query).primary_results[0])

Problem description

dataframe_from_result_table does not work with pandas 3.0 on a column with all empty datetime elements. The code crashes. Works fine with pandas 2.3.3.

It seems like the datetime parsing does not work. If any of the values are not datetime(null), the dataframe is returned as expected.

If query related, does it happen on other platforms (Kusto Web UI, Kusto Explorer)?

No

Output of pip freeze

Details

artifacts-keyring==1.0.0 asttokens==3.0.1 azure-core==1.38.0 azure-identity==1.25.1 azure-kusto-data==6.0.1 backports.tarfile==1.2.0 certifi==2026.1.4 cffi==2.0.0 charset-normalizer==3.4.4 colorama==0.4.6 comm==0.2.3 cryptography==46.0.4 debugpy==1.8.20 decorator==5.2.1 executing==2.2.1 idna==3.11 ijson==3.4.0.post0 importlib_metadata==8.7.1 ipykernel==7.2.0 ipython==9.10.0 ipython_pygments_lexers==1.1.1 jaraco.classes==3.4.0 jaraco.context==6.1.0 jaraco.functools==4.4.0 jedi==0.19.2 jupyter_client==8.8.0 jupyter_core==5.9.1 keyring==25.7.0 matplotlib-inline==0.2.1 more-itertools==10.8.0 msal==1.35.0b1 msal-extensions==1.3.1 nest-asyncio==1.6.0 numpy==2.4.2 packaging==26.0 pandas==3.0.0 parso==0.8.5 platformdirs==4.5.1 prompt_toolkit==3.0.52 psutil==7.2.2 pure_eval==0.2.3 pycparser==3.0 Pygments==2.19.2 PyJWT==2.11.0 python-dateutil==2.9.0.post0 pytz==2025.2 pywin32-ctypes==0.2.3 pyzmq==27.1.0 requests==2.32.5 six==1.17.0 stack-data==0.6.3 tornado==6.5.4 traitlets==5.14.3 typing_extensions==4.15.0 tzdata==2025.3 urllib3==2.6.3 wcwidth==0.6.0 zipp==3.23.0

[larstellemannsaether]

Direct cause of error seems to be helpers.parse_datetime() contains conditional parsing logic checking specifically/only for pandas 2 - making pandas 3 use old pandas <2 logic

Currently:

    # Pandas before version 2 doesn't support the "format" arg
    import pandas as pd

    args = {}
    if (force_version or pd.__version__).startswith("2."):
        args = {"format": "ISO8601", "utc": True}
    else:
        # if frame contains ".", replace "Z" with ".000Z"
        # Using bitwise NOT (~) on the boolean Series is the idiomatic pandas way to negate the mask
        contains_dot = frame[col].str.contains("\\.")
        frame.loc[~contains_dot, col] = frame.loc[~contains_dot, col].str.replace("Z", ".000Z")
    frame[col] = pd.to_datetime(frame[col], errors="coerce", **args)
    return frame[col]

Could possibly be rewritten to:

    # Pandas before version 2 doesn't support the "format" arg
    import pandas as pd

    args = {}
    version_string = force_version or pd.__version__
    major_version = int(version_string.split(".")[0])
    
    if major_version >= 2:
        args = {"format": "ISO8601", "utc": True}
    else:
        # if frame contains ".", replace "Z" with ".000Z"
        # Using bitwise NOT (~) on the boolean Series is the idiomatic pandas way to negate the mask
        contains_dot = frame[col].str.contains("\\.")
        frame.loc[~contains_dot, col] = frame.loc[~contains_dot, col].str.replace("Z", ".000Z")
    frame[col] = pd.to_datetime(frame[col], errors="coerce", **args)
    return frame[col]