Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Databricks JDBC + Driver #41

Open
Nerus87 opened this issue Mar 20, 2024 · 2 comments
Open

Azure Databricks JDBC + Driver #41

Nerus87 opened this issue Mar 20, 2024 · 2 comments

Comments

@Nerus87
Copy link

Nerus87 commented Mar 20, 2024

Azure cloud
Driver: DatabricksJDBC42.jar
Download: https://www.databricks.com/spark/jdbc-drivers-download
Guide: https://docs.databricks.com/en/_extras/documents/Databricks-JDBC-Driver-Install-and-Configuration-Guide.pdf

Hello, I have a problem fetching data, DB can be changed and table columns are shown but all the data are missing.
I've tried even with Dapper but it is not working.

@Nerus87
Copy link
Author

Nerus87 commented Mar 21, 2024

Additional info, logs from connection:

Mar 21 10:07:43.349 WARN  31 com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run: Caught Throwable in BackgroundFetcher.run() while the thread is running: null
Mar 21 10:07:43.354 ERROR 27 com.databricks.client.exceptions.ExceptionConverter.toSQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
java.sql.SQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
	at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
Caused by: com.databricks.client.support.exceptions.GeneralException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
	... 5 more
Caused by: java.lang.ExceptionInInitializerError
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:229)
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:224)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:728)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
	at com.databricks.client.spark.arrow.ArrowBuffer.handleInitializeBuffer(Unknown Source)
	at com.databricks.client.hivecommon.api.HiveServer2BaseBuffer.initializeBuffer(Unknown Source)
	at com.databricks.client.spark.jdbc.ResultFileDownloadManager.getNextDownloadedFile(Unknown Source)
	at com.databricks.client.spark.jdbc.DownloadableFetchClient.fetchNRows(Unknown Source)
	at com.databricks.client.hivecommon.api.HS2Client.fetchRows(Unknown Source)
	at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Failed to initialize MemoryUtil.
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:136)
	... 17 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @1c655221
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:84)
	... 17 more

Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list
Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list

@idunnoaiii
Copy link

I'm try clone the source and add this --add-opens java.base/java.nio=ALL_UNNAMED in the JdbcBridge before execute command java and it dose working. Waiting the author add this config to the java package.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants