Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement a spark line counter using Accumulators #279

Closed
mibrahim opened this issue Feb 18, 2016 · 2 comments
Closed

Implement a spark line counter using Accumulators #279

mibrahim opened this issue Feb 18, 2016 · 2 comments
Assignees
@mibrahim
Labels
enhancement
Milestone
DG2.3

Comments

@mibrahim
Copy link
Member

May be that can provide a solution:
http://spark.apache.org/docs/latest/programming-guide.html#accumulators-a-nameaccumlinka
@mibrahim mibrahim self-assigned this Feb 18, 2016
@mibrahim mibrahim added this to the DG2.2 milestone Feb 18, 2016
@mibrahim
Copy link
Member Author

And research the operation of the searchExitFlag in the spark disttributor.

@mibrahim mibrahim modified the milestones: DG2.3, DG2.2 Feb 19, 2016
@mibrahim
Copy link
Member Author

mibrahim commented Apr 1, 2016

The outcome of @mpeter28 's research is that this is not possible and the user will need to design their model to scale to the correctly needed line/vector count.

@mibrahim mibrahim closed this as completed Apr 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mibrahim mibrahim

Labels
enhancement
Projects
None yet
Milestone
DG2.3
Development

No branches or pull requests
1 participant
@mibrahim