Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom screen object to render effects on top of the stream #1552

Closed

Conversation

levs42
Copy link
Contributor

@levs42 levs42 commented Aug 28, 2024

Description & motivation

  • Feature: add dedicated video effects screen object

This PR introduces a new screen object that allows to render effects on top of the other screen objects. By default IOVideoMixer uses videoTrackScreenObject to render effects. As result, multicam covers the effects during the rendering. Having custom object allows to add it as a screen child and display extra elements on top of the stream.

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots:

Spoiler

Screenshot 2024-08-28 at 3 15 06 PM

Copy link
Owner

@shogo4405 shogo4405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a maintenance perspective, I want to avoid increasing the number of ScreenObjects optimized for specific needs.

If place it on the HaishinKit side, I would like to make it more general so that other users can also utilize it easily. It can be accepted with the following changes:

  1. Allow ImageScreenObject to be set from the CIImage library side since there is currently no CIImage-based ImageScreenObject.
  2. Rename it to CIImageScreenObject.
public class CIImageScreenObject: ScreenObject {
    public var image: CIImage?
}

@levs42
Copy link
Contributor Author

levs42 commented Sep 6, 2024

I've updated the PR, please take a look. I also added an extra check to ScreenRendererByCPU to avoid the situation where screen object was rendered for some time, than it became invisible, but the rendered still has an image in the images array and renders this cached image.

@shogo4405
Copy link
Owner

I believe this is a class that should be placed in the app layer as a unique extension on the app side. There is no benefit in placing it on the HK side, so please position it in the app layer.

@levs42
Copy link
Contributor Author

levs42 commented Sep 6, 2024

Sounds good. How about ScreenRendererByCPU cached image issue, is the extra check something that can be merged?

@shogo4405
Copy link
Owner

I cannot make a decision because there is no material (such as tests or reproducible code) to judge from. Let’s close this as a PR. When you have time, please provide reproducible code and create an issue. I will take care of the fix on my end.

From a design perspective, that check should be on the ScreenObjectContainer side, not on the ScreenRenderer side. https://github.com/shogo4405/HaishinKit.swift/blob/4adecd1a89ecde9bf873fe28911c17a0c93aa4ec/Sources/Screen/ScreenObjectContainer.swift#L60-L7

If I place it on the ScreenRenderer side, it could potentially cause other issues, or it might not. That’s why I’m being cautious.

@levs42 levs42 closed this Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants