Expose dots which haven't been matched to a paper as markers #43
Conversation
Cool idea, when ready we can look at the square detection code here
👍 |
client/camera/detectPrograms.js
Outdated
| // add markers to programs | ||
| programsToRender.forEach(programToRender => { | ||
| programToRender.markers = markers; | ||
| }); |
There was a problem hiding this comment.
Exposing the markers as data on every paper seems a bit strange to me. Maybe allow access to them via paper.get('markers');? Or only expose the markers that are contained within a paper on the paper (similarly to how it works on DL)?
There was a problem hiding this comment.
Maybe having both would be valuable?
myPaper.markersexposes markers onmyPaperpaper.get('markers')exposes all markers that are not on any piece of paper.
There was a problem hiding this comment.
I think the most flexible way would be to receive all markers with paper.get('markers'). I thought about having both APIs like @sgwilym suggested, but then it get's complicated for some case for example If you want to get all markers. Instead we could add a paperId property and positionOnPaper property if the marker is on a paper.
Marker Object:
{
position: {x, y}, // global position
color: [r, g, b]
colorName // "red", "green", "blue" or "black"
// these properties will be null if the marker is not on a paper
paperId,
positionOnPaper: {x, y} // position converted to coordinate system of the paper
}
client/camera/detectPrograms.js
Outdated
| .filter(({ matchedShape }) => !matchedShape) | ||
| .map(({ colorIndex, avgColor, pt, size }) => { | ||
| const { x, y } = projectPointToUnitSquare(pt, videoMat, config.knobPoints); | ||
| return { size, position: { x, y }, avgColor, colorIndex }; |
There was a problem hiding this comment.
colorIndex might not be super meaningful to programs. How about omitting that and changing avgColor to simply color?
There was a problem hiding this comment.
Ah I see how you used the colorIndex in the curling game and how that was convenient. Hm, I could get on board with that. Though might be nicer then to convert it to a colour name ("red", "green", etc)?
There was a problem hiding this comment.
Agree with you, I'll change that.
|
Great start! We played with this, and drew circles over all the markers:
Looks like the markers aren't attached to papers yet, so I'll remove that from the docs. |
|
Yes, I've started working on that but haven't finished it yet. |
I wanted to have markers which are tracked independently of the papers, similar to issue #42. I thought the fastest implementation would be to reuse the existing circle detection of the paper recognition. Any point which isn't part a corner shape is exposed as a marker on each paper object
I've build a little curling game with it:
https://www.youtube.com/watch?v=0wXpnWLRP1Y
The only drawback I've noticed is that if a corner of a paper is partially covered the remaining dots are no longer part of a shape and will be recognized as markers. In my demo, I've dealt with this by using bigger circles for the markers and filter for the size in my paper program. The implementation isn't complete right now; I would also like to add the information which of the markers are on the paper and their relative position to the paper. I'm interested to hear your feedback.