zvol: Fix blk-mq sync

[tonyhutter]

Motivation and Context

Fixes: #17761

Description

The zvol blk-mq codepaths would erroneously send FLUSH and TRIM commands down the read codepath, rather than write. This fixes the issue, and updates the zvol_misc_fua test to verify that sync writes are actually happening.

How Has This Been Tested?

Updated test case. Ran test case on master and saw it fail to send enough sync writes to the log device with blk-mq. Re-ran with this PR and saw it send the expected number of sync writes to log with blk-mq.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Quality assurance (non-breaking change which makes the code more robust against bugs)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[tonyhutter]

I believe this is ready to go. Reviewers - please take a look when you get a chance.

[amotin]

Considering io_data_dir() is used in only one place, I'd probably prefer it inlined and be brought in sync with the surrounding code. For example, it explicitly verifies for op_is_discard(), but not for io_is_secure_erase(). Though both are probably covered by op_is_write().

[tonyhutter]

@amotin my latest push inlines io_data_dir().

[amotin]

my latest push inlines io_data_dir().

@tonyhutter You sure inlined it, but haven't synchronized with the surrounding code as I asked.

[tonyhutter]

@amotin sorry I jumped the gun. I think you've inadvertently raised another issue - I'm not 100% convinced we are advertising that our zvols can do secure erase. I'm not seeing us set max_secure_erase_sectors or set QUEUE_FLAG_SECERASE (depending on kernel version). That's probably a good thing, since secure erase doesn't really make sense in the context of zvols. We should probably disable any io_is_secure_erase() codepaths. I can open a separate issue after I've looked into it more.

[amotin]

@tonyhutter You might be right about io_is_secure_erase(), but (op_is_sync(rq->cmd_flags) && req_op(rq) != REQ_OP_READ) in its uncertainty makes me shiver also.

[tonyhutter]

(op_is_sync(rq->cmd_flags) && req_op(rq) != REQ_OP_READ) in its uncertainty makes me shiver also.

Yea, the kernel macros are somewhat scattershot in what they check for. There's an op_is_write() but not an op_is_read(), for example. This bubbled up as the least bad option for checking sync writes.

[tonyhutter]

Here's the follow-up with the secure erase updates: #17803