-
Notifications
You must be signed in to change notification settings - Fork 643
[archive] Course projects
Below are some regulations and recommendations on picking and developing a course project for Deep Learning @HSE
The best possible scenario is when you already have an idea deep learning project, awaiting only the opportunity, e.g.:
- improving the model in that research you conduct at your department
- reproducing that article that you are excited about, but haven't found any reasonable implementation yet
When you converge on the project idea, please do tell us about that and do so ASAP. Working on your projects in secret and suddenly approaching us with them a few weeks before course end is a bad practice.
Ideally, you should tell us of your pick by the week [lecture] #6.
When you converged on the project idea, it is time to analyse existing work[must be approved] and formulate the initial milestone[must be approved].
This milestone is either about reproducing some baseline or creating a minimal working prototype of your idea, depending on project complexity and contents.
Upon reaching the milestone, if you still have time and enthusiasm, you begin a new iteration of approximately the same size. The new iteration may be "trying out this new idea" or "Applying the same idea to several experiments to find out if it reproduces well".
Finally, a project should be concluded with some kind public activity. At the very least, this means reporting the results to other fellow students. However, if the results are appropriate, you are encouraged to instead pick something of greater impact (for bonus points ofc):
- Writing a blog post and making sure people read it
- Publishing article [at least arxiv] and making sure people read it
- Creating a demo[e.g. http://likemo.net/] or a repository[e.g. https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner] and making sure people see it
- Pull-requesting the modification (if you contribute to opensource libraries)
- Giving a talk at some topic-related workshop (e.g. Vision workshop @ ysda or anything)
The only possible exception is if your project is as deep in NDA as a project can get.
The projects may be of varied total complexity, but the scoring system is oriented on evening that out. The rewards are given for iterations of roughly same size, so if simple image classification-based project would have the first milestone about "finding out whether some particular idea beats the baseline", for a more complicated project about, say, deep reinforcement learning, merely reproducing the baseline is already a milestone.
The base rewards are:
- 10 points for preliminary analysis: reading articles, posts, formulating ideas, reporting.
- 20 points for a basic prototype:
- for research projects: an basic experiment that implements and supports/rejects the initial hypothesis.
- for development projects: implementing core functionality in an MVP way.
- 10 points per each additional iteration
- research: extending/adjusting hypothesis,
- development: additional majore feature/set
- Must be agreed with us to prevent abusing.
- Rule of thumb: at least half as hard as a homework assignment
- [optional] Bonus points for public activity
- base 10 points for blog posts / arxiv article / minor impact talk (+10 if >500 viewers)
- base 10 points for a user-friendly demo (+10 if >50 users, many other ways to get +10)
- base 5 points for each accepted pull request (+5 if library has >500 users or PR had complications, more in complex cases)
- base 5 points for library/toolkit open-source release (+5 per 25 stars)
So in the worst case(idea proven wrong after a few iterations), you can expect your project to yield a total of 40 points or more.
Some topics have higher entry barrier than others. For this reason, topics like
- Machine translation
- Speech recognition
- Image captioning
are not present in project examples and generally discouraged UNLESS you have a prior experience in the domain area or have strong reasons to proceed. All these topics generally REQUIRE sizeable effort and hardware to even start iterating, training times may reach several weeks and potential opportunities to shoot oneself in the foot are virtually infinite.
Technically there are no restrictions on the framework you use for your project, you may even write it from scratch. However, there is no guarantee we will be able to aid you with any issues outside numpy/theano/lasagne stack.
As a general rule, make sure that you are either a confident user of the framework you chose or have a regular access to someone who is.
Also, please make sure that your framework is compatible with all the project steps you planned and they take reasonable amount of time/effort.
Teams are allowed but only as far as both teammates have a pre-determined role in the project. E.g.
- your team is synthesising fonts; One person is trying the autoencoders, the other tries GAN.
- your team is building a neural conversation model; One person optimizes the NN architecture, the other tries out various improved sampling approaches at the decoder (e.g. beam search)