Skip to content

fix(datasource): convert CSV datetime columns after parsing - #1471

Merged
XiaJunjie2020 merged 1 commit into
dataease:mainfrom
pcbeingused333:fix/c3648-csv-datetime
Nov 17, 2026
Merged

XiaJunjie2020 merged 1 commit into
dataease:mainfrom
pcbeingused333:fix/c3648-csv-datetime

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Importing a CSV datasource with a column set to the time (datetime) type in the import dialog returns HTTP 500, because pandas' CSV parser does not accept datetime64[ns] in its dtype argument. The exception occurs before the import can process otherwise valid dates.

Read explicitly selected CSV datetime columns as strings, then convert them to the existing datetime dtype after parsing. This preserves date and time values and missing cells, rejects invalid dates, and leaves Excel reading and integer rounding unchanged.

Testing and validation

Five offline regression tests exercise the actual import function with dates, empty cells, time components, invalid dates, explicitly selected strings, an empty CSV, and integer rounding alongside datetime conversion. Before the fix, three tests error with the unsupported datetime dtype; after the fix, all five pass with Python 3.11.18 and pandas 2.3.3:

python backend/tests/test_csv_datetime_import.py

HTTP, database persistence, browser interactions and deployment were not exercised.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants