PROJECT 2: Dhruv Karthik #27
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Github link
Due to critical paper submissions, started this assignment on Thursday & took 3 late days :( I was really looking forward to doing a ton of extra credit on this assignment, it is unfortunate that it came during a very busy week of the semester. I hope on exploring implementations of NN's through the rest of the course.
Character Recognition: XOR converges if you initialize nicely. Interestingly, it does not converge if you feed it as an entire batch (all 4 samples), as your loss doesn't change. Implemented batched inputs and an MLP Builder that allows you to arbitrarily specify input dimensions and 'number of samples' per batch for batch gradient descent. Did not get the time to try the full character recognition portion of it. Would love to keep working on it.
Feedback: Not that far from a CIS 680 assignment or a CIS 520 assignment (which each required some linear algebra prereq and took one lecture). Good practice to implement NN's from scratch, you learn something new every time. This time, I learnt about the importance of initialization and 'batch construction'. With very large datasets, you can randomly sample inputs to form a batch, but with the XOR problem, you should be more careful.
Stream Compaction: Did the GPU vs CPU extra credit (ended non-working threads early). Surprisingly, my WEF scan did better than thrust on some array sizes.