Refactor Dojo Creation Process

[maxtrap]

Summary

This PR is a refactor for all code related with dojo creating/updating. I moved large chunks of the code from dojo.py into separate files and did a complete refactor of dojo_from_spec. My intention behind this PR is to increase the readability and scalability of the dojo creation process by decoupling its parts into separate files.

Changes

I separated the majority of utils/dojo.py into a directory dojo_creation, containing five files as follows:

• dojo_initializer.py:
  Handles github cloning, loading the yamls, creating the dojo file structure, etc. I basically just copied and pasted large chunks from dojo.py. This shouldn't introduce any problems because it is mostly existing code. The one small change I did here was I took the final lines of dojo_from_spec and separated them into two separate functions: validate_challenge_paths and initialize_course.

• dojo_builder.py:
  Contains DOJO_SPEC and dojo_from_spec. Handles initializing everything defined at the top level inside dojo.yml. This is where the DOJO_SPEC and dojo_from_spec functions were relocated. I was able to reduce DOJO_SPEC from 149 to 33 lines of code and dojo_from_spec from 164 to 21 lines.

• module_builder.py:
  Contains MODULE_SPEC and module_from_spec. Handles initializing anything inside of the module.yml or at the module level inside of dojo.yml. This includes initializing all of the module resources.

• challenge_builder.py:
  Contains CHALLENGE_SPEC and challenge_from_spec. You get the idea.

• builder_utils.py:
  Contains all helper functions that are used by the *_builder.py files.

Additional Changes:

• models/__init.py__:
  This file had logic relating to importing modules, challenges, and resources. I felt like this functionality was misplaced, so I instead put import functionality into the builder files. The only changes in this file are the deletion of handling the default keyword argument in the DojoModules, DojoChallenges and DojoResources constructors. These classes are not initialized anywhere besides the dojo init process, so there shouldn't be any side effects from this change.

Attribute Inheritance

The attribute inheritance functionality is preserved. If a field is not present in a lower layer of the process (say the challenge layer), it will check all layers above for the field (the module, then the dojo layer).

Imports

The way I structured imports is similar to how it was previously, but not quite the same. The inheritance hierarchy is still there, meaning if a challenge import doesn't specify the module or dojo, it will check the imports in the higher layers. The thing that is different is if importing at a higher layer, like the dojo layer. If the next layer isn't defined, meaning the module list is empty, it will copy over the module list into the dojo data to be handled at the module layer. The module layer then does the same with challenges, with each individual challenge then being imported at the challenge layer. This enables seamless importing of entire dojos.

Consider the following dojo.yml:

import:
  dojo: example

This would now essentially clone the entire dojo, instead of creating an empty dojo. You can further specify specific modules of the import as usual:

import:
  dojo: example
modules:
  - import:
      module: hello

Which only imports the hello module, which is essentially the same as just:

id: example
modules:
  - import:
      dojo: example
      module: hello

Bug fixes

This PR fixes the following bugs:

• Relative path being defined as absolute path inside dojo_initialize_files is now fixed.
• Error messages during challenge transfer inside get_challenge were fixed to be more descriptive.

I know this is a very big PR, so please let me know if there are any questions or concerns!

[zardus]

Can we resolve the conflicts here and get it merged ASAP? The scale of this PR is going to ensure a conflict almost any time any dojo code gets touched (case in point, I checked to see the mergeability of this because I am touching one of the files right now).

[maxtrap]

Resolved! I integrated the changes from f2cc10a during my merge. Also, #740 would need to be modified as a result of this PR merge.