Sampling parameters, generalize batch config.

[jlamypoirier]

✨ Description

Deal with some structural issues and technical debt causing trouble for ongoing work.

• Make a SamplingParameter structure for holding the data sampling parameters that come from the trainer (batch config, model, etc.). This is needed as an alternative to the fast-growing argument list to GPTData.__init__ and the associated bloat. (ex. optionally prevent cross-document attention #177, improvements to MTP implementation #218, DPO #223)
• Generalize BatchConfig, extract the model-specific parameters into GPTBatchConfig.
• Rename num_micro_sequences -> micro_batch_splits. Since the generic batch config and schedule runner shouldn't have to know about model-specific sequences.
• Move use_loss_masking_spans to the batch config. This will make it easier to know if loss masking is enabled, ex. to prevent it in Knowledge distillation, fix and improve cross-entropy #229). @sohamparikh this may require some config changes. I added backward compatibility, but it will only work if set globally (data.sampling.use_loss_masking_spans)
• Trying out cached_property in a few places following the discussion in Make the specified config parameters update the pretrained config #211. If it work we can use it all over the place to simplify derived fields.

🔍 Type of change

Select all that apply:

• 🐛 Bug fix (non-breaking change that addresses a specific issue)
• 🚀 New feature (non-breaking change that adds functionality)
• ⚠️ Breaking change (a change that could affect existing functionality)
• 📈 Performance improvement/optimization (improves speed, memory usage, or efficiency)
• 🛠️ Code refactor (non-functional changes that improve code readability, structure, etc.)
• 📦 Dependency bump (updates dependencies, including Dockerfile or package changes)
• 📝 Documentation change (updates documentation, including new content or typo fixes)
• 🔧 Infrastructure/Build change (affects build process, CI/CD, or dependencies)

[tscholak]

I think the most interesting and valuable change here is actually that with this we can now set these so-called sampling parameters differently for different datasets.

[tscholak]

LGTM