useAnimatedKeyboard fixes (iOS)

[mhoran]

Summary

fix: return correct height for detached keyboard on iOS (fixes #5584, #6440)
fix: don't clobber keyboard height when keyboard did not animate on iOS
fix: convert keyboard position to window coordinate space on iOS (Stage Manager compatibility)
fix: don't prematurely update keyboard height during dismissal on iOS
fix: don't jump to height when iOS keyboard is rapidly opened/closed (fixes #6727)
fix: correct height when keyboard opened from shortcuts bar on iOS

Test plan

Can be tested with the useAnimatedKeyboard example.

1. Enable iPad for the example project.
2. Launch the useAnimatedKeyboard example.
3. Open the keyboard, then undock or float the keyboard. The view should not transform when undocked or floating.
4. Dock the keyboard
5. Hide the keyboard, note that the view should now be transformed by the keyboard height
6. Enable Reduce Motion in Accessibility, and Prefers Cross-Fade Transitions
7. Open the keyboard, note that he transform should work as expected

[tomekzaw]

Hi @mhoran, thanks for another PR! Someone from Reanimated team will review this PR shortly.

Would you mind giving us a bit more context on why exactly some of the changes were made (e.g. what's the reason to change UIKeyboardWillChangeFrameNotification to UIKeyboardWillShowNotification)? This would facilitate our review process very much. Thanks in advance!

[mhoran]

Sure thing! I tried to leave detailed notes in my commit messages for posterity, but happy to provide further context as well.

Regarding UIKeyboardWillChangeFrameNotification vs show/hide notifications, initially I tried to take an approach similar to the one in #5710. While I did find a working solution (checking the endFrame only, which worked with interactive dismissal), I ran into issues with this approach in split view as well as Stage Manager. Dealing with all the different coordinate systems was a real pain, and I figured the code would be so complex that it would end up regressing in the future.

I found that the most reliable way to determine if the keyboard has been detached is to listen for UIKeyboardWillHideNotification. This required reworking things a bit to use UIKeyboardWillShowNotification and UIKeyboardWillHideNotification instead of UIKeyboardWillChangeFrameNotification, due to ordering (UIKeyboardWillChangeFrameNotification fires before the show / hide notifications.)

However, this approach also has the benefit of simplifying the logic in the change handler. I was able to remove the logic that tried to guess whether the keyboard was opening or closing -- which I found to be inaccurate at times, especially when a hardware keyboard is attached (due to the autosuggest dock.) The OPENING and CLOSING states are now set only in the appropriate notification handlers (or overridden in updateKeyboardHeightDuringInteractiveDismiss -- more on that later.)

I fixed a few more issues along the way.

1. There was an issue with the hardware keyboard autosuggest bar not being respected, due to the notification handler expecting there to be a measurement frame with accurate height at all times. However, that's not true if the event handler doesn't happen to run at the exact moment a keyboard animation is running. I found this to be unnecessary complexity, since _targetKeyboardHeight can be used instead.
2. I fixed an issue with the calculated height when the user has enabled Reduced Motion (Accessibility) and Prefers Cross-Fade Transitions. The implemented fix is the same that is in React Native.
3. The coordinate space of keyboard frame notifications are in the screen's coordinate space. These need to be converted to the window's coordinate space to work correctly with Stage Manager. Without 3c2d160, the height returned by useAnimatedKeyboard will only be correct if an app is full screen when Stage Manager is enabled. Otherwise, the height will not take into consideration the position of the window within the stage.
4. I found that sometimes the wrong height would be returned -- often a quick jump to 0, then back to the correct height -- during animations, due to an issue with updateKeyboardHeightDuringInteractiveDismiss. I added a guard to make sure that the keyboard did not immediately go to 0 (this will be handled by keyboardWillHide) and corresponds to the user dismissing the keyboard with the close button or blurring the input. I also added a check to determine if the keyboard is floating (keyboard width != window width). These are not perfect solutions, but do make the animation less janky. I think someone with more iOS experience could come up with a more bullet proof solution (e.g. check if there is a touch somewhere in the main view), but I'm not sure how to do that myself.

There is a new API, Keyboard Layout Guide, which will make this all a lot simpler in the future. However, it was introduced in iOS 15. So we will have to live with this complexity until RN 0.76 is the oldest supported version for Reanimated.

Note that there remain some issues with undocked and split keyboard modes, due to incorrect start frame positions sent in notifications. Apple has removed these modes in new devices, so I didn't try to implement workarounds.

[mhoran]

Here is a screen recording showing the improvements: https://drive.google.com/file/d/1EFDhYZWNMXhJIwNlj1_ugeVwe3FqZqVF/view?usp=drivesdk.

[mhoran]

It seems I may also have fixed #6727 as I was cleaning things up in 9f6b546. When a hardware keyboard is connected and the autocorrect bar is on screen, the keyboard is considered "open". But when you show the on screen keyboard, it rapidly hides and then shows the keyboard -- inadvertently triggering the interactive dismissal logic. Cleaning up the interactive dismissal flags fixed this for the hardware keyboard case, and I believe it should fix the scenario described in #6727 as well.

[janicduplessis]

I tested this in my app and can confirm it does fix the Prefer Cross Fade Transition issue (#6440)

[mhoran]

There is an issue with iOS 18.2 when opening the on-screen keyboard from the autosuggest bar when a hardware keyboard is attached. Opening the on-screen keyboard in this way will not properly update the height returned by useAnimatedKeyboard. However, the issue is not related to these changes (I can reproduce without them.) The keyboard handlers receive bogus end frame information in the notification, so we do not know where the keyboard actually is.

[mhoran]

I managed to fix the iOS 18.2 hardware keyboard issue by updating the keyboard height in response to UIKeyboardDidShowNotification and UIKeyboardDidHideNotification notifications. The keyboard does jump to the target height instead of animating when opening the software keyboard with a hardware keyboard attached, so hopefully we'll get a proper fix from Apple.

This has cleaned up keyboard state handling as well. Now state changes are handled only by the corresponding notification handlers, and updateKeyboardFrame is only responsible for animation tracking. That should be a bit less error prone in the future.

On a side note, I managed to get the keyboardLayoutGuide working with React Native, sort of. Unfortunately it relies heavily on constraints, and I think this may be fundamentally incompatible with React Native's layout model. Happy to share my learnings with anyone on the Reanimated team who is interested.

[tomekzaw]

@mhoran Thanks again for your work on this PR. We're quite focused on other areas but we'll try to find some availability to review and thoroughly test your changes.

[szydlovsky]

Overall, an incredible PR and I wish more folks from community made such impressive contributions!
I've left some rather stylistic comments to make sure the code is anyhow readable in the future, since most of the stuff we do here is already pretty hacky.
I am yet to test everything I need on a physical iPad, I'll get that done tomorrow most likely.

Nonetheless, great work!

[szydlovsky]

Also, I found that the height values emitted when undocking the keyboard are some quite rough jumps.
See the vid below, the values in my case were:

• 340 (open)
• sudden jump to 54 (closing)
• a second later, jump to 0 (closed)

floating_jumps.mp4

I wonder if we could maybe force a animation, just like you did with the canceled interactive dismiss.
It would probably need a way to determine if incoming keyboardWillHide notification hides it for a floating keyboard. I think I kinda had it in my own attempt that you mentioned: #5710

[mhoran]

Thanks again for the review @szydlovsky. I believe I have addressed all your concerns.

I was able to fix the floating keyboard animation by always animating if the keyboard event has a duration and adding another check to the interactive dismiss logic to not update height if the state is CLOSED (I wouldn't expect this to be needed since the observer should have been detached, but apparently there is an event that makes it through.)

[szydlovsky]

For the record, the changes regarding jumping midpoints when transitioning from attached to detached keyboard turned out to be inconsistent. On some devices they fixed stuff, on others did nothing and we believe it just added unnecessary complexity. Thus, the logic from that commit is gonna be rolled back, but the comments stay here

[szydlovsky]

Unfortunately, I found a more severe mismatch with interactive dismiss emitted values - they seem to be quite a bit off 😢
EDIT: I checked and it was already present before the PR. If we manage to do something about it - would be great but it's not essential

mismatch.mp4

[szydlovsky]

After quite a bit of testing, looks and works great, fixes exactly what it is supposed to and introduces no regressions. Beautiful 🎉

[tomekzaw]

@mhoran Thanks again for the PR!