Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oracle Query Causes a Type Error #3096

Open
jnjimmy1 opened this issue Feb 16, 2025 · 0 comments
Open

Oracle Query Causes a Type Error #3096

jnjimmy1 opened this issue Feb 16, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@jnjimmy1
Copy link

jnjimmy1 commented Feb 16, 2025

Describe the bug

When querying a table in an Oracle database, the module runs into a TypeError when trying to determine whether the result is a decimal data type or not. It appears that additional None handling is required when iterating through an oracledb.Cursor's description field.

Error:

File c:\Users\Jimmy\miniconda3\envs\work\lib\site-packages\awswrangler\oracle.py:306, in read_sql_query(sql, con, index_col, params, chunksize, dtype, safe, timestamp_as_object, dtype_backend)
    [259](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:259) """Return a DataFrame corresponding to the result set of the query string.
    [260](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:260) 
    [261](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:261) Parameters
...
--> [610](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:610)         if row[1] == oracledb.DB_TYPE_NUMBER and row[5] > 0:
    [611](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:611)             dtype[row[0]] = pa.decimal128(row[4], row[5])
    [613](file:///C:/Users/Jimmy/miniconda3/envs/work/lib/site-packages/awswrangler/oracle.py:613) _logger.debug("decimal dtypes: %s", dtype)

TypeError: '>' not supported between instances of 'NoneType' and 'int'

Link to the method producing the error: https://github.com/aws/aws-sdk-pandas/blob/main/awswrangler/oracle.py#L603
Links to various Cursor.description documentation:
https://python-oracledb.readthedocs.io/en/latest/api_manual/cursor.html#Cursor.description
https://peps.python.org/pep-0249/#description

How to Reproduce

Create the following table in an Oracle database

CREATE TABLE TESTDB.TESTTABLE1 (
	VARCOL VARCHAR2(200),
	INTCOL INTEGER,
	NUMCOL NUMERIC(38, 10)
)

Ran the following Python code:

import awswrangler
import oracledb

ora_conn = oracledb.connect(
    dsn="localhost:5432/FREEPDB1", user="SYSTEM", password="oracle"
)
df = awswrangler.oracle.read_sql_query(
    "SELECT * FROM all_tab_columns WHERE table_name = 'TESTTABLE1'", ora_conn
)

display(df)

Expected behavior

The awswrangler.oracle.read_sql_query(...) call should return a DataFrame instead of raising a TypeError.
The DataFrame should have similar results to the below screenshot.

Image

Your project

No response

Screenshots

No response

OS

Windows 11

Python version

3.9.21

AWS SDK for pandas version

3.11.0

Additional context

Changing https://github.com/aws/aws-sdk-pandas/blob/main/awswrangler/oracle.py#L610 to check for None allows the above query to run. However, I'm not sure if that is a robust enough solution.
Changed Code:

# if row[1] == oracledb.DB_TYPE_NUMBER and row[5] > 0:
if row[1] == oracledb.DB_TYPE_NUMBER and row[5] is not None and row[5] > 0:

Results:
Image

@jnjimmy1 jnjimmy1 added the bug Something isn't working label Feb 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant