Skip to content

Adding Parakeet Export and Runtime - #136

Merged
kevchengcodes merged 24 commits into
apple:mainfrom
kevchengcodes:add_parakeet_export
Aug 7, 2026
Merged

Adding Parakeet Export and Runtime#136
kevchengcodes merged 24 commits into
apple:mainfrom
kevchengcodes:add_parakeet_export

Conversation

@kevchengcodes

Copy link
Copy Markdown
Contributor

Summary

This PR adds the export and runtime for parakeet-tdt-0.6b-v3. It also does some light cleanup of the existing speech recognition Swift runtime, and renames all speech-runner-esque terminology to speech-recognizer. This is why you see some files deleted (they were renamed but with large changes).

Closes #7

Changes

  • new Python file and README for export
  • Swift code to differentiate between Whisper and Parakeet TDT (such as for bundle loading)
  • Code addition to MelSpectogram.swift, such that Whisper is unaffected but Parakeet can be supported
  • Parakeet TDT decoder

Testing

Using this jfk.wav file as an example:

uv run models/parakeet/export.py --overwrite --dynamic
swift run -c release speech-recognizer exports/parakeet-tdt-0.6b-v3_float32_dynamic/ jfk.wav --verbose --warmup

Comment thread swift/Sources/CoreAIShared/Runtime/NDArray+Helpers.swift
Comment thread swift/Sources/CoreAISpeech/MelSpectrogram.swift
Comment thread swift/Sources/CoreAISpeech/ParakeetTDTDecoder.swift
Comment thread models/parakeet/export.py Outdated
Comment thread swift/Sources/CoreAISpeech/ParakeetTDTDecoder.swift
Comment thread swift/Sources/CoreAISpeech/ParakeetTDTDecoder.swift Outdated

@carinapeng carinapeng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with @stikves that we might want to iterate a bit more. To me it'd be great to include in this PR the decoder advancing on blank frames needs more checking, and more testing. The other things flagged are worth considering but could be follow up PRs :)

Comment thread swift/Sources/CoreAISpeech/SpeechDecoder.swift Outdated
Comment thread swift/Sources/CoreAIShared/Runtime/NDArray+Helpers.swift

@stikves stikves left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, thank you

@carinapeng carinapeng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! :)

@kevchengcodes
kevchengcodes merged commit 82a0e5e into apple:main Aug 7, 2026
3 checks passed
msnabiel added a commit to msnabiel/coreai-models that referenced this pull request Aug 11, 2026
Resolves conflicts:
- Removed ModelShapeConfig.swift (deleted upstream in apple#148)
- Updated NDArray+Helpers.swift documentation for stride-aware indexing

Pulls in upstream features:
- Parakeet speech recognition export and runtime (apple#136)
- SyncInputHandler protocol and InputContext (apple#147)
- Zero-copy state binding with bind(into:) (apple#156)
- Additional speech tests and config improvements
- Bug fixes for forced_decoder_ids parsing, topK normalization, logits token count

Preserves local iOS optimizations:
- int8 KV cache quantization (2× memory reduction)
- 16k default context with unreachable shape pruning
- CoreAI simulator availability guards

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model request: NVIDIA Parakeet TDT 0.6B v3 (multilingual streaming ASR)

4 participants