New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Question on challenge #3

Open

yaravind opened this issue May 18, 2020 · 1 comment

Labels

question

yaravind commented May 18, 2020 •

edited

Loading

Thank you for the example. Learned quite a bit! I have following questions if you can help!

Why do we have to sort the department DF? What advantage does it give?
Why to sort in this specific order: 'id, 'assigned_date, 'company_id, 'factory_id' ?
What is the intent of the query in the code? Count of all users whose joined a department on the same day their birth date?

Member

vitalyte commented May 29, 2020

@yaravind you are welcome. I think our detail description in blog could help you in research https://dataengi.com/2019/02/06/spark-data-skew-problem/

vitalyte added the question label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment