Repository navigation
Meetings: resample both lanes before combining; repair the wrong-rate meeting.m4a files - #34
Merged
Merged
Conversation
The combine copied mic.m4a and system.m4a frame for frame at the microphone's rate. With a 16 kHz headset the other side played three times too slow and meeting.m4a lasted three times the meeting (x1.088 with a 44.1 kHz MacBook array). Each lane is now read as mono and resampled to the higher of the two rates before they are interleaved, in chunks (two passes: loudness, then conversion), so memory no longer grows with the meeting. MeetingAudioLanes (KleothCore) says how long a combined file must last and how long the old combine made it; MeetingStore can correct the cost duration stored from the stretched file.
MeetingAudioRepair rebuilds a meeting.m4a only when it matches what the old combine made of its lanes: written beside the old file, checked to last as long as the lanes, renamed over it; mic.m4a and system.m4a are only read, the folder keeps its date, a crash leaves the old file. The app runs it once per launch on a utility task of its own (not the pipeline queue), one meeting at a time, skipping a meeting that records or processes; the stored length goes back on the main actor. Never on a demo launch. recombine [--dry-run] runs the same repair from a shell.
…ling after resampling The old combine wrote at the microphone's rate, so a combined file below the higher lane's rate is an old one whatever its length (a microphone lane with more frames gave the right length and the wrong speed). Every combine now marks its file (xattr dev.kleoth.combine), which also tells the one case rate cannot: a microphone above the system lane's rate. An existing meeting.m4a that does not open or holds no audio is rebuilt. The new file is flushed with F_FULLFSYNC before it is renamed over the old one, the folder date is restored on failure too, and a cancelled task stops the combine between chunks. The scan opens nothing for a marked file and never opens a file that is not downloaded. The loudness gain and its 0.97 ceiling now apply after the rate change: clipped first and resampled after, a boosted lane overshot past 0.97. Tests use synthetic values only.
The pass state (UserDefaults dev.kleoth.meetingAudioRepair) keeps a finished pass per output folder, the meetings left pending and those whose rebuild failed, so later launches look only at the pending ones and never retry a failure (recombine still repairs it). The repair waits while a meeting or a screen recording runs and stops a rebuild under way when one starts; a dictation does not hold it up. rebuildCombinedAudioIfMissing treats an unreadable meeting.m4a as missing and writes through the same safe swap. An open meeting page reloads its player when the file on disk was replaced, at the same position. recombine refuses to write while Kleoth runs.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Recorder.combinedecodedmic.m4aandsystem.m4ato mono at each file's own rate, then wrote both channels frame for frame at the microphone's rate. It never resampled. The system lane is 48 kHz.meeting.m4alasted 3× the meeting. History reads the duration from that file.The lane files were always correct.
The fix
aacSettingsclamp.AVAudioConverter2→1 withdownmixoff (the default) gave 0.5 for L=0.5 / R=0.1 (probed). I found no sign this was deliberate; the pill's system meter already measures both channels. So the system lane's RMS, and its gain, can differ a little from before.ChannelAudio.mixToMonoandenvelope(the Scribe path) still keep the left channel. This is documented there, not changed.dev.kleoth.combine = resampledon its output.Audit: other code that lines up the two lanes
ChannelAudio.mixToMono(mono mix sent to Scribe, dictation upload)ChannelAudio.envelopehopSeconds × that file's own rate, so envelope index i is a time.ChannelAttribution(energy-based You/Them mapping)word.start / hopSeconds, on the per-lane files.LocalTranscriberper channelMeetingPipelinecostmeeting.m4a, someta.jsoncost.audio_duration_secsis stretched. A cloud transcript'stranscription_costis priced on it, while Scribe itself billed the correctly resampled mono mix.kleoth summarizecost,localtranscribemeeting.m4a. They are right again after the rebuild.MeetingAudioPlayermeeting.m4a.transcript.json(Scribe's own duration is right), word timestamps,summary.*and the Markdown hold no wrong duration.Repair of meetings already recorded
Detection (
MeetingAudioLanes, pure)meeting.m4abelow the higher lane's rate is an old file, whatever its length.meeting.m4athat exists but does not open or holds no audio (a combine cut short) is rebuilt. A missing one is left torebuildCombinedAudioIfMissing. That function now also treats an unreadable file as missing and writes through the same safe swap.Safe write (
MeetingAudioRepair.writeCombinedFile)F_FULLFSYNC(falling back tofsync) and thenrename(2)'d over the old file.defer, so it is kept on failure too.Stored length
cost.audio_duration_secsandtranscription_costinmeta.jsonandvariants/*/variant.jsonare corrected only when the stored length equals what the old combine produced. The folder date is kept.The app's launch pass (
RecordingController.repairCombinedAudioAtLaunch)dev.kleoth.meetingAudioRepair: the finished pass, the pending folders and the failed ones.getxattr. An old file costs three header opens, once.recombinestill repairs them. Progress is saved after each meeting.SF_DATALESSset, or a ubiquitous item that is.notDownloaded, is never opened. The folder stays pending.meta.jsonrewrite runs on the main actor, and only while no run owns the folder.Player
An open meeting page reloads its player when the file on disk was replaced. It compares
statidentity: device, inode, size and mtime. It keeps the position and play state. On acontentRevisionbump an unaffected meeting costs onestatand keeps playing. A page mounted later loads the new file inonAppear, so it does not rely ononChangeat mount.recombine [--dry-run] <meeting-dir | folder>It runs the same repair from a shell and looks at every folder it is given, including the app's failed ones. It refuses to write while Kleoth runs (exit 3, with a message), skips files that are not downloaded, and only reads with
--dry-run.Verification
AVAudioConverter2→1 keeps the left channel.swift build && swift test: 812 tests pass.swift build --package-path app: passes.swift test --package-path app: 18 tests in 3 suites pass.recombinerefused (exit 3). The repair then ran through a throwaway program calling the sameMeetingAudioRepairAPI.system.m4aat r = 1.0000, lag 0, against about 0 for the original.Not verified