Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 22 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,19 @@ rewrites the timeline to keep only the good segments.
project.tscproj while a document is open, so the edit cycle is
`close (saves) → edit JSON → open (reloads)`. Throws on other platforms.
- **@camkit/cli** — the `camkit` binary: info, clips, sources, rebuild,
export-audio, captions, silences, transcribe, status, close, open, docs.
export-audio, export-video, captions, silences, transcribe, status, close,
open, docs.
Rebuild always backs up to `.bak` and refuses to run with a
`~project.tscproj` lock or an existing backup unless `--force`. Always
`--dry-run` first. `export-audio` flat-mixes the timeline's audio to one file
(m4a/wav/flac/…) for cleanup in Audacity/Auphonic — pure ffmpeg, honours track
mute and per-clip gain (`--raw` to bypass). `captions` injects an animated
Dynamic Caption track straight into the project from a transcript (same
backup/lock safety as rebuild).
backup/lock safety as rebuild). `export-video` renders the current timeline to
a ProRes 422 `.mov` by driving Camtasia's GUI export (Camtasia stays the
renderer — effects, transitions, Dynamic Captions and `.trec` video streams
need its engine, no ffmpeg re-encode). macOS-only; needs Accessibility
permission. Verified on Camtasia 2026.1.3.
- **@camkit/mcp** — placeholder; will wrap core later.

## Prerequisites
Expand All @@ -44,48 +49,21 @@ rewrites the timeline to keep only the good segments.
- **A transcription engine** — required by `camkit transcribe` only; see
*Transcription engines* below. Either `OPENAI_API_KEY` (cloud) or
`whisper-cpp` (local, `brew install whisper-cpp`).
- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` only;
everything else is cross-platform

## Transcription engines

`camkit transcribe` resolves an engine by precedence (highest wins): an
explicit `--engine openai|whisper-cpp` flag, then environment, then the
`auto` default. `auto` picks:

1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is
pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models
don't return the word-level timestamps the rebuild step needs.
2. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the
`ggml` model Camtasia downloads to
`Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized —
fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a
larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`.
3. **Neither → an error** telling you to set `OPENAI_API_KEY` or run
`brew install whisper-cpp`. camkit never auto-installs (no silent `brew`).

camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib`
(private, code-signed, undocumented ABI) — you bring your own `whisper-cli`
runner. The tiny local model has coarser word timestamps, so cross-checking
with `camkit silences` matters even more on the local path.

## Captions in the Camtasia UI

To get higher-quality captions than Camtasia's built-in tiny model: transcribe
with camkit (OpenAI or a larger local model), then either bring the result into
Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to
inject an animated **Dynamic Caption** track straight into the project. Do
**not** swap Camtasia's bundled model file — it's redownloaded on update and
unsupported.

`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"`
writes the word-level stream onto the source and adds a styled caption track via
the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The
style comes from a Camtasia Dynamic Caption preset, resolved on demand from
Camtasia's app-support dir — list them with `camkit captions --list-presets`,
including any custom presets you've saved. Classic (non-animated) captions
aren't supported; they can't do the per-word highlight, and you can promote a
Dynamic track's styling further in Camtasia's UI.
- **macOS + Camtasia** — required by `status`/`close`/`open`/`docs` and
`export-video`; everything else is cross-platform
- **Accessibility permission** — required by `camkit export-video` only. It
drives Camtasia's export GUI via System Events, so grant your terminal under
System Settings ▸ Privacy & Security ▸ Accessibility.

## Docs

- [Transcription engines](docs/transcription.md) — engine precedence, model
overrides, OpenAI vs local whisper.cpp.
- [Captions in the Camtasia UI](docs/captions.md) — higher-quality captions,
Dynamic Caption presets, SRT import.
- [Why `export-video` drives the GUI](docs/export-video.md) — why rendering
can't be done from JSON, the export approaches tried, and why UI scripting is
the only working path.

## Use

Expand Down
17 changes: 17 additions & 0 deletions docs/captions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Captions in the Camtasia UI

To get higher-quality captions than Camtasia's built-in tiny model: transcribe
with camkit (OpenAI or a larger local model), then either bring the result into
Camtasia via SRT import (File ▸ Import ▸ Captions), or use `camkit captions` to
inject an animated **Dynamic Caption** track straight into the project. Do
**not** swap Camtasia's bundled model file — it's redownloaded on update and
unsupported.

`camkit captions --from take.transcript.json --preset "Bebas 3 Line Word Red"`
writes the word-level stream onto the source and adds a styled caption track via
the same `close → edit → open` cycle as rebuild (with a `.bak` backup). The
style comes from a Camtasia Dynamic Caption preset, resolved on demand from
Camtasia's app-support dir — list them with `camkit captions --list-presets`,
including any custom presets you've saved. Classic (non-animated) captions
aren't supported; they can't do the per-word highlight, and you can promote a
Dynamic track's styling further in Camtasia's UI.
50 changes: 50 additions & 0 deletions docs/export-video.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Why `export-video` drives the GUI

Every other camkit command works on `project.tscproj` directly — that file
*describes* the timeline, and a description is just data we can read and
rewrite. **Rendering is different**: turning that description into a `.mov`
means compositing every track, effect, transition and Dynamic Caption,
decoding the `.trec` `tscc2` screen-recording stream (which ffmpeg can't), and
encoding ProRes. Only Camtasia's engine does that. No file you can write makes
the rendered video appear — something has to run the renderer. So `export-video`
has to invoke Camtasia, not edit JSON.

That left the question of *how* to invoke it. Options tried, in order of
preference, verified against **Camtasia 2026.1.3**:

1. **AppleScript `export` verb — the right way. Dead.** `sdef
/Applications/Camtasia.app` shows `project` only `responds-to "export"`
(`exportToFile:`) — there is no `<command name="export">` block, so the verb
declares **no parameters** (no file, no codec, no preset). Tested live:
`export (project 1 …) file "…"` → syntax error (destination has no grammar
to attach to); `tell (project 1 …) export file "…"` → `doesn't understand
"export" (-1708)`; bare `export (project 1 …)` → dispatches then dies `Can't
continue export. (-1708)`. It routes to `exportToFile:` but there's no way to
supply the file argument, and it won't run without one. Consistent with the
already-documented broken suite (`-10000` on media elements). Re-check this
each Camtasia release — if TechSmith fixes the suite, it should replace UI
scripting.

2. **A headless render/CLI binary in the app bundle.** Ruled out:
`Camtasia.app/Contents/MacOS` ships only `Camtasia` and `CamtasiaSupport.app`
— no standalone exporter. The render code is internal dylibs
(`libCSRenderLib.dylib`, `libCSEncodeLib.dylib`) driven by GUI-only Obj-C
controllers (`ExportMenuControllerProtocol`, …). No documented command-line
entry point.

3. **Re-encode the `.tscproj` ourselves with ffmpeg.** Rejected by design — see
above: `.trec` `tscc2` is undecodable and Camtasia's effects/transitions
aren't reproducible. camkit's principle is "Camtasia stays the final
renderer."

4. **UI scripting via macOS Accessibility (what shipped).** Drive the real
on-screen export UI through `osascript` → `System Events`: Export ▸ Local
File → File format *QuickTime Movie* → Options → Compression Type *Apple
ProRes 422* → set name/folder → Export. This is the **only** path that
actually produces a file today.

Consequences of (4): **not headless.** It needs Camtasia running and frontmost
on a logged-in GUI session (no `ssh`/daemon), the controlling terminal must
have **Accessibility** permission (System Settings ▸ Privacy & Security ▸
Accessibility), and the render blocks the GUI while it runs. It's also brittle
across Camtasia versions — the element paths are verified only on 2026.1.3.
21 changes: 21 additions & 0 deletions docs/transcription.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Transcription engines

`camkit transcribe` resolves an engine by precedence (highest wins): an
explicit `--engine openai|whisper-cpp` flag, then environment, then the
`auto` default. `auto` picks:

1. **`OPENAI_API_KEY` set → OpenAI `whisper-1`** (best quality). Note: this is
pinned to `whisper-1`, not a "newer" model — the `gpt-4o-transcribe` models
don't return the word-level timestamps the rebuild step needs.
2. **Else `whisper-cli` on PATH → local whisper.cpp.** By default it reuses the
`ggml` model Camtasia downloads to
`Camtasia.app/Contents/Resources/models/speechToText/` (tiny/quantized —
fast, lower fidelity). Override with `CAMKIT_WHISPER_MODEL` (path to a
larger `ggml-*.bin`) or `CAMKIT_WHISPER_BIN`.
3. **Neither → an error** telling you to set `OPENAI_API_KEY` or run
`brew install whisper-cpp`. camkit never auto-installs (no silent `brew`).

camkit reuses Camtasia's *model file* but not its bundled `libwhisper.dylib`
(private, code-signed, undocumented ABI) — you bring your own `whisper-cli`
runner. The tiny local model has coarser word timestamps, so cross-checking
with `camkit silences` matters even more on the local path.
32 changes: 31 additions & 1 deletion packages/cli/src/camkit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import {
resolveProjectPath,
type KeepSeg,
} from "@camkit/core";
import { camtasiaDocPaths, camtasiaDocs, closeProject, openProject, projectStatus } from "@camkit/darwin";
import { camtasiaDocPaths, camtasiaDocs, closeProject, exportVideo, openProject, projectStatus } from "@camkit/darwin";
import { exportAudio, runSilencedetect, transcribeRecording } from "./media.ts";
import { listPresets, resolvePreset } from "./presets.ts";
import { version } from "../package.json";
Expand Down Expand Up @@ -123,6 +123,24 @@ const HELP: Record<string, { usage: string; about: string[] }> = {
" --raw ignore mute/solo/gain — dry unity sum of every clip",
],
},
"export-video": {
usage: "camkit export-video [--project PATH] [--out FILE] [--codec prores422]",
about: [
"Render the CURRENT timeline to a ProRes 422 .mov by driving Camtasia's",
"GUI export (Export ▸ Local File ▸ QuickTime ▸ Apple ProRes 422). Camtasia",
"stays the renderer — effects, transitions and Dynamic Captions need its",
"engine, and .trec screen recordings carry a video stream ffmpeg can't",
"decode. Run after rebuild to render the edited cut.",
"",
"macOS-only and needs Accessibility permission for your terminal (System",
"Settings ▸ Privacy & Security ▸ Accessibility). The project must be open",
"in Camtasia (camkit open) — export renders the front document. Verified",
"on Camtasia 2026.1.3; the GUI path is brittle across versions.",
"",
" --out FILE output path (default ./<project>.mov)",
" --codec NAME only prores422 for now (default)",
],
},
captions: {
usage:
"camkit captions [--project PATH] --from FILE.transcript.json (--preset NAME | --preset-file PATH) [--list-presets] [--src ID] [--strip-punctuation] [--strip-apostrophes] [--dry-run] [--force]",
Expand Down Expand Up @@ -219,6 +237,7 @@ function printHelp(cmd?: string): void {
sources: "list media-bin sources, placed or not",
rebuild: "rewrite timeline to kept segments (rough cut)",
"export-audio": "mix the timeline's audio to one file (m4a/wav/…)",
"export-video": "render the timeline to a ProRes 422 .mov via Camtasia",
captions: "inject an animated Dynamic Caption track from a transcript",
silences: "ffmpeg silencedetect on a recording",
transcribe: "word-level Whisper transcript of a recording",
Expand Down Expand Up @@ -344,6 +363,16 @@ async function cmdExportAudio(argv: string[]) {
await exportAudio({ segs, projectPath: path, out, durationSeconds });
}

function cmdExportVideo(argv: string[]) {
const path = camtasiaDocPaths()[0]?.path ?? resolve(flag(argv, "--project") ?? ".");
const base = bundleName(path).replace(/\.(cmproj|tscproj)$/, "");
const out = flag(argv, "--out") ? resolve(flag(argv, "--out")!) : resolve(`${base}.mov`);
const codec = flag(argv, "--codec") ?? "prores422";
console.log(`Rendering timeline → ${out} (${codec}) via Camtasia…`);
exportVideo({ out, codec });
console.log(`✓ exported ${out}`);
}

function cmdCaptions(argv: string[]) {
if (has(argv, "--list-presets")) {
const presets = listPresets();
Expand Down Expand Up @@ -494,6 +523,7 @@ const COMMANDS: Record<string, (argv: string[]) => void | Promise<void>> = {
sources: cmdSources,
rebuild: cmdRebuild,
"export-audio": cmdExportAudio,
"export-video": cmdExportVideo,
captions: cmdCaptions,
silences: cmdSilences,
transcribe: cmdTranscribe,
Expand Down
105 changes: 105 additions & 0 deletions packages/darwin/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,108 @@ export function openProject(projectPath: string): void {
open POSIX file "${bundle}"
end tell`);
}

/** Map a camkit --codec value to Camtasia's "Compression Type" menu label.
* Only prores422 is required for v1; add rows here for prores4444/h264. */
const CODEC_LABELS: Record<string, string> = {
prores422: "Apple ProRes 422",
};

/**
* Render the front Camtasia document's timeline to a .mov via UI scripting.
*
* The AppleScript suite has NO working export verb on 2026.1.3 (export is a
* bare exportToFile: stub with no file/codec parameters — see issue #10 spike),
* so this drives the GUI: Export ▸ Local File… → File format "QuickTime Movie"
* → Options… → Compression Type "Apple ProRes 422" → set name+folder → Export.
* Verified against Camtasia 2026.1.3. Brittle across versions and needs
* Accessibility permission for the controlling terminal (System Events).
*
* ponytail: UI scripting is the only path Camtasia exposes; no abstraction
* layer until a second codec/version actually needs different element paths.
*/
export function exportVideo(opts: { out: string; codec?: string }): void {
assertDarwin();
const codec = opts.codec ?? "prores422";
const label = CODEC_LABELS[codec];
if (!label) {
throw new Error(`Unsupported --codec "${codec}". Supported: ${Object.keys(CODEC_LABELS).join(", ")}.`);
}
if (camtasiaDocs().length === 0) {
throw new Error("No project open in Camtasia. Open it first (camkit open) — export renders the front timeline.");
}
const dir = dirname(opts.out);
const base = opts.out.split("/").pop() ?? opts.out;

osascript(`
tell application "Camtasia" to activate
delay 0.5
tell application "System Events"
if not (UI elements enabled) then error "Accessibility permission required — grant your terminal under System Settings ▸ Privacy & Security ▸ Accessibility."
tell process "Camtasia"
click menu item "Local File..." of menu 1 of menu bar item "Export" of menu bar 1
delay 1.5
-- the save sheet attaches to whichever window; find it
set swin to 0
repeat with i from 1 to (count of windows)
try
if (count of sheets of window i) > 0 then set swin to i
end try
end repeat
if swin = 0 then error "Export dialog did not appear."
set sg to splitter group 1 of sheet 1 of window swin

-- File format ▸ QuickTime Movie (.mov)
set fp to pop up button 2 of sg
click fp
delay 0.4
click menu item "Export to QuickTime Movie (.mov)" of menu 1 of fp
delay 0.6

-- Options… ▸ Advanced Export Options ▸ Compression Type ▸ ${label}
click (item 1 of (buttons of sg whose name is "Options..."))
delay 1.2
set adv to window "Advanced Export Options"
set picked to false
repeat with p in (pop up buttons of group "Video" of adv)
if not picked then
try
click p
delay 0.3
if (exists menu item "${label}" of menu 1 of p) then
click menu item "${label}" of menu 1 of p
set picked to true
else
key code 53
end if
end try
end if
end repeat
if not picked then
click button "Cancel" of adv
error "Compression Type \\"${label}\\" not found in Advanced Export Options."
end if
click button "OK" of adv
delay 0.6

-- filename + destination folder
set value of text field "Export As:" of sg to "${base}"
delay 0.2
keystroke "g" using {command down, shift down}
delay 0.5
keystroke "${dir}"
delay 0.3
keystroke return
delay 0.6

click button "Export" of sg
delay 0.6
-- overwrite confirmation, if the file already exists
try
if (count of sheets of window swin) > 0 then
click button "Replace" of sheet 1 of window swin
end if
end try
end tell
end tell`);
}