Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GSoC Infragram.org full-screen UI and video upload Discussion and Planning #418

Closed
13 of 22 tasks
Forchapeatl opened this issue Jun 14, 2022 · 16 comments · Fixed by #442, #445, #446, #449 or #453
Closed
13 of 22 tasks

GSoC Infragram.org full-screen UI and video upload Discussion and Planning #418

Forchapeatl opened this issue Jun 14, 2022 · 16 comments · Fixed by #442, #445, #446, #449 or #453
Labels
discussion ideas to discuss and refine before deciding whether to do them or not

Comments

@Forchapeatl
Copy link
Collaborator

Forchapeatl commented Jun 14, 2022

Hi, everyone! This is the Planning Issue for the Infragram.org: full-screen UI and video upload for infragram.org. The project is in collaboration with the Outreachy proposal: Design a new full-screen user interface for Infragram.org . All are open for discussion and improvement.

Process Video Locally

  • Allow conversion of entire video.
  • Implement video controls on canvas.
  • Include another Canvas Element ( second Canvas ).
  • Record video on process at second canvas.
  • Save processed video file.
  • Integrate the above functionalities to with UI of outreachy collaborator. @stephaniequintana

Allow selecting multiple output resolutions as well

  • Dynamically change the size of second canvas to fit required resolution.
  • Integrate the above functionalities to with UI of outreachy collaborator.

Integrate Teamsys latest camera selection API
Issue #39

  • Download Temasys/AdapterJS library.
  • Change all instances of the word webRtcOptions.context to webRtcDetectedType
  • Remove code if (webRtcOptions.context === "flash")
    Reference 1 Reference 2

Accept multiple webcam resolutions

  • Maintain css canvas size.
  • Change HTML canvas size to fit requested resolution.
  • Integrate the above functionalities to with UI of outreachy collaborator.

Write UI tests for new Infragram UI Jest

  • write test for infragram UI functionalities.

Allow zooming/panning of video

  • Zooming : Add uScale of uniform vect2 to shader.frag and shader.vert.
  • Panning : Add uPan of uniform vect2 to shader.frag and shader.vert .
  • Implement above functionalities to infragram drawscreen function.
  • Integrate the above functionalities to with UI of outreachy collaborator.

Miscellaneous

  • Add option to close webcam input
  • Enable image and video stream mirroring
  • Add file filters on input to optimize infrared analysis
@TildaDares TildaDares added the discussion ideas to discuss and refine before deciding whether to do them or not label Jun 14, 2022
@TildaDares
Copy link
Member

This looks great @Forchapeatl🎉. You're off to a great start!

@TildaDares
Copy link
Member

I've created labels for planning and gsoc, so you can add them if you want. I love seeing colors on Github so I like to add labels wherever I can😅.

@jywarren
Copy link
Member

Hi @Forchapeatl - this is looking great. I wanted to ask for some clarification on the video portion - are we equally supporting uploaded video (through drag and drop) and live recorded video? Do they use almost all the same functions? If you can just give me a quick summary of this I'd love that!

Also I wanted to say, i know the panning has been challenging. If you want to take a break from it that's fine! The Temasys adapter is a high priority for us, so if you wanted to try taking that one on i think it'd be wonderful -- it'll unlock Safari/iOS functionality, for one -- and should allow for other improvements. What do you think? And/or, am I behind on things at all regarding this? Thank you!

@Forchapeatl
Copy link
Collaborator Author

Hello @jywarren . Thank you . Yes it works on recorded videos and live videos.
Please have a look at it here https://forchapeatl.github.io/infragram/index3.html#

Okay . I will commence with Temasys immediately . Thank you.

@RaviAnand111
Copy link

Hey @Forchapeatl @jywarren , If there is anything can I help you too ?

@jywarren
Copy link
Member

Sounds great thank you Forcha!

@RaviAnand111 so helpful, thanks! Would you like to test out the version Forcha linked w both uploaded and live videos and let us know how it works?

@Forchapeatl
Copy link
Collaborator Author

Forchapeatl commented Jul 29, 2022

Friday 29th July 2022.
Hi @jywarren.
Please here are a few of my worries

  1. I have made analysis on this issue Update to Use Temasys/AdapterJS for getUserMedia (brings iOS + Safari compatibility) spectral-workbench.js#172 I tested the Publiclab Spectral Workbench on an Apple phone (IOS 11). (chrome and safari). but all I got was a blank webcam stream. Please am I missing something ?

IMG-20220729-WA0046
IMG-20220729-WA0047

On Chrome (IOS)
IMG-20220729-WA0048

@RaviAnand111
Copy link

hey @Forchapeatl @jywarren , I tried to open the link @Forchapeatl gave https://forchapeatl.github.io/infragram/index3.html# , but I am unable to get any video, on uploading I am neither able to see an image nor video. In live video function as well, there is no video on screen just black screen.
I am using firefrox and Manjaro Arch Linux OS.
Thanks

@Forchapeatl
Copy link
Collaborator Author

hello @RaviAnand111 you are off to a great start. It should work fine on Chrome.

@jywarren
Copy link
Member

Oh hmm, @Forchapeatl is this possibly the error you're seeing on Safari? (BTW we may want to move to a different more specifically named issue for this?) publiclab/spectral-workbench.js#266

I believe it had been working in Safari as of publiclab/spectral-workbench.js#172 (review), so I hope we can figure out what happened if it is no longer!

@jywarren
Copy link
Member

@RaviAnand111 can you confirm that basic webcam access is working on your system with https://webrtc.github.io/samples/src/content/getusermedia/gum/ ? Thank you!

@Forchapeatl
Copy link
Collaborator Author

@jywarren . Please have a look at this comment #39 (comment) .

@RaviAnand111
Copy link

hey @jywarren, I tried to open this link https://webrtc.github.io/samples/src/content/getusermedia/gum/ and it seems to work neither on chrome nor on firefox. I tried to experiment with dev tools but got confused myself.

Hey @Forchapeatl, I tried this link https://forchapeatl.github.io/infragram/index3.html# to run on chrome and it is working fine n chrome, the webcam is visible there, but I guess we would need to work on it for firefox.
Thanks

@jywarren
Copy link
Member

jywarren commented Aug 2, 2022

Hmm, its strange that the webrtc demo/example isn't running -- but if https://forchapeatl.github.io/infragram/index3.html is working then your webcam drivers are probably OK. But the webrtc.github.io implementation should be the canonical/official correct usage, so that's concerning. I wonder if we should hold off for now on testing on Arch Linux, as it's not as common a platform? The original question here was Safari support, as well.

Alternatively @RaviAnand111 if you can find an implementation that is working on chrome and firefox from an official source or a commonly used library, we can try using their syntax. For example, https://p5js.org/examples/dom-video-capture.html is what I had suggested Forcha look at as an example. Does that work for you in multiple browsers @RaviAnand111?

@RaviAnand111
Copy link

Hey @jywarren, I tried https://p5js.org/examples/dom-video-capture.html and it is working fine on firefox but not on chrome and chromium browser. I don't know what to conclude from this.

I guess I should try to find some official library which can be used for firefox and chrome on Arch Linux.
I think you are right, Arch Linux (Manjaro) is not a common OS and is used by very few so we can hold off its testing and work on mainstream OS, and by the sidelines I can try to find some resources for Arch Linux and inform you guys if I find any.
how does that sound ?
Thanks

@Forchapeatl
Copy link
Collaborator Author

Hello @RaviAnand111 . No worries. This https://p5js.org/examples/dom-video-capture.html example uses invert filters. It appears the filter intensity styles differently on various browsers. I send gratitude.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment