Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Case on Large Number Being Captured As Scientific Notation #771

Open
1 task done
DamonYip9891 opened this issue Aug 21, 2023 · 0 comments
Open
1 task done

New Case on Large Number Being Captured As Scientific Notation #771

DamonYip9891 opened this issue Aug 21, 2023 · 0 comments

Comments

@DamonYip9891
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

The Excel file (".xlsx" format) contains 3 columns. The issue, the sample image and the attachment are as follows:
The first column is being read as string format correctly.
The second column displayed in General format, as mentioned in issue #126 , and it can be read correctly by adjusting the parameter "usePlainNumberFormat=true".
In the third column, there is a value displayed in scientific notation, but in the formula bar, it shows as "230714073456". The display format for other values in this column is General. Regardless of parameter adjustments, it is not possible to read this value correctly.

image

My PySpark codes:
df = spark.read.format("com.crealytics.spark.excel")\ .option("header", "true")\ .option("dataAddress","""'sheet1'!A1""")\ .option("usePlainNumberFormat", "true")\ .load(file_path)

sample.xlsx

Expected Behavior

The third column of the table should be read as a string type into the dataframe. The value "230714073456", which displayed in scientific notation, should be read in its entirety.

Steps To Reproduce

No response

Environment

- Spark version:3.2.1
- Spark-Excel version:com.crealytics:spark-excel_2.12:3.4.1_0.19.0
- OS:Databricks on AWS
- Cluster environment:64GB, 8 Cores. DBR 10.4 LTS aarch64, Spark 3.2.1, Scala 2.12

Anything else?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant