Importing CSV: the csv has a date column but it appears in the database as a string

[Kamilooosx]

Bug description

CSV import loads string type every time, instead of date. I check the Columns to be parsed as dates option but it still doesn't work. I tried
Database:File Uploads
Schema: main
I checked all data types in CSV for example YYYY-MM-DD or 2024-07-09 09:15:05
Version: 4.1.0.3

Screenshots/recordings

No response

Superset version

4.1.1

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Hey @Kamilooosx! I'm here to help you with any bugs, questions, or contributions you have for Superset. Let's squash this bug together!

I found a similar unsolved discussion titled Error message: data type 'Datetime' not understood which might be related to your issue ^[1].

Regarding your issue with the CSV import feature not recognizing date formats, ensure that the 'Columns to be parsed as dates' option is correctly set. The expected behavior is for this option to allow you to specify which columns should be interpreted as dates ^[2].

If the date parsing is failing, it might be due to incorrect settings in the CSV import process. Key settings to check include:

• The parse_dates option, which specifies which columns should be parsed as dates. Ensure the column names or indices match those in your CSV file.
• The dayfirst option, which affects how dates are interpreted. Make sure this is set according to your date format ^[3].

If these settings are correct and the issue persists, it might be a limitation or bug in the current version.

To continue talking to Dosu, mention @dosu.

[sfirke]

Did the same file + upload choices successfully get turned into a date using Superset version 4.0.2?

[Kamilooosx]

I hadn't used the CSV import feature before updating to version 4.1.0.3.
When I uploaded the CSV file to BigQuery and connected it to the database, everything worked correctly.
I am working with Preset.

[sfirke]

I can't replicate this in Superset 4.1.1. I created a CSV with a column called "datetime" and a value of 2024-07-09 09:15:05. I then uploaded it into Superset, specifying that the column called "datetime" was a date time column. It loaded into my data warehouse with the correct column type and I was able to make time series charts in Superset with it.

My only guess is that there's some invalid date-time value lurking in that column and causing it to be a string. I once had this problem and finally figured out that a user had inputted some invalid date, like 2023-02-29 when that year wasn't a leap year for instance. Or an invalid time like 25:00. Once I deleted those few rows, it worked correctly.

If you are able to create a reproducible minimal example that indicates this is indeed a problem with Superset, please share it on this issue.

[Kamilooosx]

I created a CSV file with random data, where the date format is similar to yours, but the bug still persists.





health_center_data (1).csv

[yousoph]

Hi @Kamilooosx , once the upload has finished, if you're having trouble with the column type, you should be able to go into your dataset under the Columns tab and mark the time column as a temporal column to use it as a time column in charts

[Kamilooosx]

Yeah ! It works ! But when i want to filter by Date it's generate error :C

[Kamilooosx]

With preset support i found a solution.
We must create SQL Lab queries with that function

datetime("") AS
Do note that you would still need to change the date column generated by the function above to a temporal field and then use it as a filter after that.