[C++][Dataset] Fix DatasetWriter deadlock on writting batch greater than max_rows_queued

[gitmodimo]

Rationale for this change

DatasetWriter deadlocks when batch written is longer than max_rows_queued. This also causes deadlock in acero and propagates paused state.

What changes are included in this PR?

Changed throttle threshold calculation to allow batch.lenght>max_rows_queued

Are these changes tested?

Provoking test added.

Are there any user-facing changes?

No.

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview