v0.7.0: Supporting arbitrarily long docs and various new tasks

✨ New features and improvements

• NEW: Support arbitrarily long docs leveraging map-reduce and document sharding. Read more here (#332)
• NEW: New tasks -
  • EntityLinkingTask: use LLMs to link entity mentions in texts to entities in a custom knowledge base. Read more here (#241)
  • TranslationTask: translate documents from and to arbitrary languages. Read more here (#396)
  • RawTask: prompt models with only doc content, no prompt scaffolding, for a maximum of flexibility. Read more here (#395)
• Allow custom endpoints for all models to support usage of fine-tuned models (#390)

🔴 Bug fixes

• Fix issues with previously incomplete integration of LangChain objects (#417)

⚠️ Backwards incompatibilities

No backwards incompatibilities w.r.t. v0.6.x.

📖 Documentation and examples

• Descriptions of EntityLinkingTask, TranslationTaskand RawTask (explosion/spaCy#12988, explosion/spaCy#13183, explosion/spaCy#13180)
• Introduction to and description of doc sharding/map-reduce for the support of infinitely long docs (explosion/spaCy#13214)

👥 Contributors

@honnibal, @ines, @rmitsch, @svlandeg