Skip to main content

Thumb piano

Template by nthState

Graph · AVAudioEngine · 10 nodes · 0:06 preview

Preview

The app rendered this preview offline from the template’s own graph.

Download Thumb piano.phonique

A .phonique document is a folder, so it comes as a ZIP archive, 0.2 MB. Unzip it and open the folder in Phonique.

Opening a template straight from this page waits until the app can open one; until then the download is the way in.

What’s inside

10 nodes and 9 cables, read from the template’s own document.

The Swift

What Phonique exports from this template: a Swift package, ThumbPiano, that needs nothing beyond Apple’s frameworks. Its README’s “Use it”:

import ThumbPiano

let graph = try ThumbPianoGraph()
try graph.start()
// …
graph.stop()
The whole of ThumbPianoGraph.swift, 591 lines
//
//  ThumbPianoGraph.swift
//  Generated by Phonique from “Thumb piano”.
//
//  Phonique writes this file afresh each time it exports the graph, so a
//  change made here is gone after the next export.
//

import AVFAudio
import Foundation
import Synchronization

/// “Thumb piano”, as Phonique plays it: its nodes on an `AVAudioEngine`, connected as
/// the graph's cables connect them.
///
/// Make one, `start()` it to play through the device's output, and `stop()` it; it plays
/// again when it is started again. It is isolated to the main actor, where its engine is
/// driven: make it, start it and stop it there. Everything that can fail throws —
/// `Failure` for what this code checks itself, and AVFAudio's own errors — and nothing here
/// stops the app.
@MainActor
public final class ThumbPianoGraph {
    /// Why the graph could not be built or started, besides the errors AVFAudio throws.
    public nonisolated enum Failure: Error, Hashable, Sendable {
        /// The device has no audio output to play through: the engine's output node has no
        /// format.
        case outputUnavailable
        /// The device has no audio input to listen to: the graph uses the microphone, and the
        /// engine's input node has no format.
        case inputUnavailable
        /// A node could not be made: its name in the graph.
        case nodeNotMade(String)
        /// One of the package's audio files could not be found or read: its name.
        case audioFileUnreadable(String)
    }

    /// The graph's Triggers: `press(_:)` holds one's button down, and `release(_:)` lets it go.
    public nonisolated enum Trigger: Hashable, Sendable {
        /// Trigger “Key C”.
        case keyC
        /// Trigger “Key E”.
        case keyE
        /// Trigger “Key G”.
        case keyG
    }

    /// The engine the graph plays on.
    public let engine: AVAudioEngine

    /// The render time of the control nodes' last step, in frames at the output's sample rate;
    /// nil before their first.
    private var lastStepTime: AVAudioFramePosition?
    /// The render time the first step after a start saw, which the seconds count from: the
    /// output's sample time need not start at zero.
    private var stepOrigin: AVAudioFramePosition?
    /// The render time of the control nodes' last step, in seconds since the graph started,
    /// which a Timed Trigger works its presses out from; zero at the first.
    private var renderTime = 0.0
    /// Steps the control nodes while the engine runs.
    private var stepper: Task<Void, Never>?

    /// Sampler “Pluck C”.
    private let pluckC: SamplerVoice

    /// Sampler “Pluck E”.
    private let pluckE: SamplerVoice

    /// Sampler “Pluck G”.
    private let pluckG: SamplerVoice

    /// Mixer “Mix bus”.
    private let mixBus: AVAudioMixerNode

    /// Delay “Echo”.
    private let echo: AVAudioUnitDelay

    /// Recorder “Take”.
    private let take: RecordingTap

    /// Builds the graph on `engine`, ready to `start()`: makes each node, attaches it to the
    /// engine, sets its values and connects the cables, each on the bus Phonique connects it on.
    ///
    /// To render the graph offline, give it an engine already in manual rendering mode
    /// (`enableManualRenderingMode(_:format:maximumFrameCount:)`), which has to come before
    /// anything asks for the engine's output node.
    ///
    /// - Parameter engine: The engine to build on: a new one, unless you give your own.
    /// - Throws: `Failure`, or the error AVFAudio throws when it refuses a cable or a file.
    public init(engine: AVAudioEngine = AVAudioEngine()) throws {
        // Every cable carries one format: two channels of 32-bit float at the output's sample rate.
        let output = engine.outputNode.outputFormat(forBus: 0)
        guard output.sampleRate > 0, output.channelCount > 0,
              let format = AVAudioFormat(standardFormatWithSampleRate: output.sampleRate, channels: 2)
        else {
            throw Failure.outputUnavailable
        }

        // Make each node.
        let pluckC = SamplerVoice()
        pluckC.sourceMode = .pointSource
        pluckC.renderingAlgorithm = .HRTFHQ
        let pluckE = SamplerVoice()
        pluckE.sourceMode = .pointSource
        pluckE.renderingAlgorithm = .HRTFHQ
        let pluckG = SamplerVoice()
        pluckG.sourceMode = .pointSource
        pluckG.renderingAlgorithm = .HRTFHQ
        let mixBus = AVAudioMixerNode()
        let echo = AVAudioUnitDelay()
        let take = RecordingTap(name: "Take")

        // Attach each to the engine.
        engine.attach(pluckC)
        engine.attach(pluckE)
        engine.attach(pluckG)
        engine.attach(mixBus)
        engine.attach(echo)

        // Set each node's values.
        pluckC.note = 60.0
        pluckC.velocity = 100.0
        pluckC.overallGain = 0.0
        pluckC.stereoPan = 0.0
        pluckC.globalTuning = 0.0
        pluckC.gate = false
        pluckC.position = AVAudio3DPoint(x: 0.0, y: 0.0, z: 0.0)
        pluckE.note = 64.0
        pluckE.velocity = 100.0
        pluckE.overallGain = 0.0
        pluckE.stereoPan = 0.0
        pluckE.globalTuning = 0.0
        pluckE.gate = false
        pluckE.position = AVAudio3DPoint(x: 0.0, y: 0.0, z: 0.0)
        pluckG.note = 67.0
        pluckG.velocity = 100.0
        pluckG.overallGain = 0.0
        pluckG.stereoPan = 0.0
        pluckG.globalTuning = 0.0
        pluckG.gate = false
        pluckG.position = AVAudio3DPoint(x: 0.0, y: 0.0, z: 0.0)
        mixBus.outputVolume = 0.71
        echo.bypass = false
        echo.delayTime = 0.375
        echo.feedback = 40.0
        echo.lowPassCutoff = 4000.0
        echo.wetDryMix = 30.0

        // Connect the cables, in the order they were made: a mixer's input takes its next free bus.
        try engine.connectNode(pluckC, to: mixBus, fromBus: 0, toBus: mixBus.nextAvailableInputBus, format: format)
        try engine.connectNode(pluckE, to: mixBus, fromBus: 0, toBus: mixBus.nextAvailableInputBus, format: format)
        try engine.connectNode(pluckG, to: mixBus, fromBus: 0, toBus: mixBus.nextAvailableInputBus, format: format)
        try engine.connectNode(mixBus, to: echo, fromBus: 0, toBus: 0, format: format)
        try engine.connectNode(echo, to: engine.outputNode, fromBus: 0, toBus: 0, format: format)

        // What each node does once it is connected.
        try pluckC.loadSample(Self.decodedAudio(at: Self.audioFile("pluck.wav"), into: pluckC.outputFormat(forBus: 0)))
        try pluckE.loadSample(Self.decodedAudio(at: Self.audioFile("pluck.wav"), into: pluckE.outputFormat(forBus: 0)))
        try pluckG.loadSample(Self.decodedAudio(at: Self.audioFile("pluck.wav"), into: pluckG.outputFormat(forBus: 0)))
        // Recorder “Take” listens to echo's output through a tap, which copies each buffer into
        // a slot made here and returns; a task writes the slots to the file.
        try take.prepare(format: echo.outputFormat(forBus: 0))
        try echo.installAudioTap(onBus: 0, bufferSize: take.tapBufferSize, format: nil, tapProvider: take.tapProvider())

        self.engine = engine
        self.pluckC = pluckC
        self.pluckE = pluckE
        self.pluckG = pluckG
        self.mixBus = mixBus
        self.echo = echo
        self.take = take
    }

    /// Starts the engine, then each node that plays once the engine has started. If a node
    /// does not start, the graph stops again and the error is thrown.
    public func start() throws {
        // The control nodes' first step is at render time zero, before the engine starts, so what
        // they set is there from the first frame.
        lastStepTime = nil
        stepOrigin = nil
        renderTime = 0
        stepControls(elapsed: 0)
        try engine.start()
        do {
            try startNodes()
        } catch {
            stop()
            throw error
        }
        startStepping()
    }

    /// Stops the graph: each node that has something to do first, then the engine. `start()`
    /// plays it again.
    public func stop() {
        stepper?.cancel()
        stepper = nil
        lastStepTime = nil
        stepOrigin = nil
        renderTime = 0
        pluckC.end()
        pluckE.end()
        pluckG.end()
        keyCIsOn = false
        pluckC.gate = false
        keyEIsOn = false
        pluckE.gate = false
        keyGIsOn = false
        pluckG.gate = false
        take.stopWriting()
        engine.stop()
    }

    /// What each node does once the engine has started.
    private func startNodes() throws {
        pluckC.begin()
        pluckE.begin()
        pluckG.begin()
        try take.startWriting(named: "Take", in: Self.recordingsFolder)
    }

    /// The package's audio file `name`, from the package's resources: “Always use
    /// `Bundle.module` when you access resources” (Bundling resources with a Swift package).
    private static func audioFile(_ name: String) throws -> URL {
        guard let url = Bundle.module.url(forResource: name, withExtension: nil) else {
            throw Failure.audioFileUnreadable(name)
        }
        return url
    }

    /// Steps the graph's control nodes — its LFOs, Triggers and Listeners — to render time `time`,
    /// in frames at the output's sample rate: each moves on by the render time since the last
    /// step, and sets the value it moves. While the graph plays, `start()` does this every 5 ms
    /// from the output node's `lastRenderTime`; an app that renders the graph offline, in manual
    /// rendering mode, calls it instead at each render time it reaches, every 256 frames, say.
    public func step(to time: AVAudioFramePosition) {
        let sampleRate = engine.outputNode.outputFormat(forBus: 0).sampleRate
        var elapsed = 0.0
        if let lastStepTime, sampleRate > 0 {
            elapsed = Double(time - lastStepTime) / sampleRate
        }
        lastStepTime = time
        let origin = stepOrigin ?? time
        stepOrigin = origin
        if sampleRate > 0 {
            renderTime = max(0, Double(time - origin) / sampleRate)
        }
        stepControls(elapsed: max(0, elapsed))
    }

    /// Looks for a new render time every 5 ms, on the main actor, and steps the control nodes to
    /// it: the look is only how often; the render time says how far they move.
    private func startStepping() {
        stepper = Task { [weak self] in
            while !Task.isCancelled {
                try? await Task.sleep(for: .milliseconds(5))
                guard let self, !Task.isCancelled else { return }
                if let time = engine.outputNode.lastRenderTime, time.isSampleTimeValid {
                    step(to: time.sampleTime)
                }
            }
        }
    }

    /// One step of every control node, `elapsed` seconds of render time after the last, at
    /// `renderTime` seconds since the graph started.
    private func stepControls(elapsed: Double) {
        // Trigger “Key C” sets Sampler “Pluck C”'s gate to true while its button is held; let go, the value is the document's.
        if keyCIsOn {
            pluckC.gate = true
        } else {
            pluckC.gate = false
        }
        // Trigger “Key E” sets Sampler “Pluck E”'s gate to true while its button is held; let go, the value is the document's.
        if keyEIsOn {
            pluckE.gate = true
        } else {
            pluckE.gate = false
        }
        // Trigger “Key G” sets Sampler “Pluck G”'s gate to true while its button is held; let go, the value is the document's.
        if keyGIsOn {
            pluckG.gate = true
        } else {
            pluckG.gate = false
        }
    }

    /// Presses `trigger`'s button, as holding it down in Phonique does, and steps the control nodes
    /// at once, so what it sets lands before the next render cycle.
    public func press(_ trigger: Trigger) {
        switch trigger {
        case .keyC:
            keyCIsOn = true
        case .keyE:
            keyEIsOn = true
        case .keyG:
            keyGIsOn = true
        }
        stepControls(elapsed: 0)
    }

    /// Releases `trigger`'s button, and steps the control nodes at once.
    public func release(_ trigger: Trigger) {
        switch trigger {
        case .keyC:
            keyCIsOn = false
        case .keyE:
            keyEIsOn = false
        case .keyG:
            keyGIsOn = false
        }
        stepControls(elapsed: 0)
    }

    /// A single held note on Apple's Sampler. All state and MIDI calls are main-actor
    /// isolated; AVFAudio owns its render callback. Values arrive checked by the definition.
    private nonisolated final class SamplerVoice: AVAudioUnitSampler, @unchecked Sendable {
        @MainActor var note = 0.0 { didSet { updateNote() } }
        @MainActor var velocity = 0.0
        @MainActor var gate = false { didSet { updateNote() } }
        @MainActor private(set) var activeNote: UInt8?
        @MainActor private var loaded = false
        @MainActor private var running = false

        /// A canonical, single-zone sample. Bounded decoding happens before this method.
        /// Float CAF/WAV loading dropped blocks on macOS 27.0.1; integer PCM did not.
        /// See #301 and spec/297-sampler.md. The original document audio is unchanged.
        @MainActor func loadSample(_ buffer: AVAudioPCMBuffer) throws {
            // Keep UUID digits out of the basename: they can alter Sampler key mapping.
            let folder = FileManager.default.temporaryDirectory.appending(path: "sampler-\(UUID().uuidString)", directoryHint: .isDirectory)
            try FileManager.default.createDirectory(at: folder, withIntermediateDirectories: true)
            defer { try? FileManager.default.removeItem(at: folder) }
            let url = folder.appending(path: "sample.wav")
            var settings = buffer.format.settings
            settings[AVLinearPCMBitDepthKey] = 32
            settings[AVLinearPCMIsFloatKey] = false
            settings[AVLinearPCMIsNonInterleaved] = false
            do {
                let file = try AVAudioFile(forWriting: url, settings: settings)
                try file.write(from: buffer)
                file.close()
            }
            // Apple's host contract: an offline unit may wait for its secondary work.
            // https://developer.apple.com/documentation/audiotoolbox/auaudiounit/isrenderingoffline
            let offline = engine?.isInManualRenderingMode == true && engine?.manualRenderingMode == .offline
            withAUAudioUnit { $0.isRenderingOffline = offline }
            try loadAudioFiles(at: [url])
            loaded = true
        }

        @MainActor func begin() {
            running = true
            updateNote()
        }

        @MainActor func end() {
            running = false
            releaseNote()
            // MIDI All Sound Off also clears a released note's tail before a restart.
            sendController(120, withValue: 0, onChannel: 0)
        }

        @MainActor private func updateNote() {
            let next = loaded && running && gate ? UInt8(note.rounded()) : nil
            guard next != activeNote else { return }
            releaseNote()
            if let next {
                startNote(next, withVelocity: UInt8(velocity.rounded()), onChannel: 0)
                activeNote = next
            }
        }

        @MainActor private func releaseNote() {
            if let activeNote { stopNote(activeNote, onChannel: 0) }
            activeNote = nil
        }
    }

    /// The whole of the audio file at `url`, decoded to loop in `format`: read until the file
    /// ends, then converted to its sample rate and its channels — a mono file into every
    /// channel, a file with more channels mixed down — as Phonique's File Player reads its file.
    /// A file that would decode to more than 512 MiB is refused before a byte is decoded,
    /// and one that ends before its length is unreadable, never a loop missing its end.
    private static func decodedAudio(at url: URL, into format: AVAudioFormat) throws -> AVAudioPCMBuffer {
        let name = url.lastPathComponent
        let file = try AVAudioFile(forReading: url)
        let source = file.processingFormat
        guard file.length > 0, source.sampleRate > 0, source.channelCount > 0, format.sampleRate > 0 else {
            throw Failure.audioFileUnreadable(name)
        }
        let largest = Double(512 * 1024 * 1024)
        let frames = (Double(file.length) * format.sampleRate / source.sampleRate).rounded(.up)
        let bytesPerFrame = Double(max(source.channelCount, format.channelCount)) * Double(MemoryLayout<Float>.size)
        guard frames * bytesPerFrame <= largest,
              Double(file.length) * Double(source.channelCount) * Double(MemoryLayout<Float>.size) <= largest,
              let whole = AVAudioPCMBuffer(pcmFormat: source, frameCapacity: AVAudioFrameCount(file.length))
        else {
            throw Failure.audioFileUnreadable(name)
        }
        // One read "attempts to fill the buffer to its capacity" and can come back short with no
        // error, so what is left is read into a buffer of its own and copied on, until the file's
        // position reaches its length. A file that ends before its length is an error, never a
        // buffer missing its end: AVFAudio's error for a read past the end, or this one for a
        // read that gives no frames.
        try file.read(into: whole)
        while file.framePosition < file.length {
            let left = AVAudioFrameCount(file.length - file.framePosition)
            guard let rest = AVAudioPCMBuffer(pcmFormat: source, frameCapacity: left) else {
                throw Failure.audioFileUnreadable(name)
            }
            try file.read(into: rest)
            guard rest.frameLength > 0 else {
                throw Failure.audioFileUnreadable(name)
            }
            // Onto the end of the whole, in the one format both are in: each of its buffers — one
            // a channel deinterleaved, one for all of them interleaved — at the format's bytes a
            // frame.
            let bytesPerFrame = Int(source.streamDescription.pointee.mBytesPerFrame)
            let read = UnsafeMutableAudioBufferListPointer(UnsafeMutablePointer(mutating: rest.audioBufferList))
            let kept = UnsafeMutableAudioBufferListPointer(whole.mutableAudioBufferList)
            for (from, to) in zip(read, kept) {
                guard let from = from.mData, let to = to.mData else { continue }
                (to + Int(whole.frameLength) * bytesPerFrame).copyMemory(from: from, byteCount: Int(rest.frameLength) * bytesPerFrame)
            }
            whole.frameLength += rest.frameLength
        }
        if source == format {
            return whole
        }
        guard let converter = AVAudioConverter(from: source, to: format) else {
            throw Failure.audioFileUnreadable(name)
        }
        converter.downmix = true
        converter.sampleRateConverterQuality = AVAudioQuality.max.rawValue
        let capacity = AVAudioFrameCount((Double(whole.frameLength) * format.sampleRate / source.sampleRate).rounded(.up)) + 1
        guard let converted = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: capacity) else {
            throw Failure.audioFileUnreadable(name)
        }
        // The whole file, once, then the end of the stream: the converter calls this on the
        // calling thread, inside `convert(to:error:withInputFrom:)`.
        var given = false
        var error: NSError?
        let status = converter.convert(to: converted, error: &error) { _, inputStatus in
            if given {
                inputStatus.pointee = .endOfStream
                return nil
            }
            given = true
            inputStatus.pointee = .haveData
            return whole
        }
        if let error {
            throw error
        }
        guard status == .haveData || status == .endOfStream, converted.frameLength > 0 else {
            throw Failure.audioFileUnreadable(name)
        }
        return converted
    }

    /// Trigger “Key C”: whether it holds its value.
    private var keyCIsOn = false

    /// Trigger “Key E”: whether it holds its value.
    private var keyEIsOn = false

    /// Trigger “Key G”: whether it holds its value.
    private var keyGIsOn = false

    /// The file Recorder “Take” wrote the last time the graph stopped — `Take.caf`, linear PCM at the
    /// graph's sample rate — or nil before it has. It is in the app's temporary directory: move or copy
    /// it where your app keeps it.
    public var takeRecording: URL? {
        take.recording
    }

    /// Where the Recorders' files go: `ThumbPiano Recordings` in the app's temporary
    /// directory, a new folder for each take.
    private static var recordingsFolder: URL {
        FileManager.default.temporaryDirectory.appending(path: "ThumbPiano Recordings", directoryHint: .isDirectory)
    }

    /// Records what a tap hears into a Core Audio Format file, off the tap's thread. The tap copies
    /// each buffer into one of the slots made before the engine started and returns: nothing on
    /// the tap's thread allocates, takes a lock or waits. A task writes the slots to the file every
    /// 20 ms, and `stopWriting()` writes what is left and closes it. The slots hold two seconds; a
    /// buffer that finds no room is dropped.
    private nonisolated final class RecordingTap: @unchecked Sendable {
        /// The Recorder's name in the graph, for an error.
        private let name: String
        /// The bus's format, once the graph is connected.
        private var format: AVAudioFormat?
        /// The slots the tap copies into, 100 ms each.
        private var slots: [AVAudioPCMBuffer] = []
        /// How many buffers the tap has copied, and how many the writer has written.
        private let copied = Atomic<Int>(0)
        private let drained = Atomic<Int>(0)
        /// The file being written, under the writer's lock, which the tap never takes.
        private let lock = NSLock()
        private var file: AVAudioFile?
        private var writer: Task<Void, Never>?
        /// The last file written, once the graph has stopped.
        private(set) var recording: URL?

        init(name: String) {
            self.name = name
        }

        /// 100 ms a buffer: the shortest a tap delivers, so no slot is ever too small.
        var tapBufferSize: AVAudioFrameCount {
            AVAudioFrameCount((format?.sampleRate ?? 0) / 10)
        }

        /// Makes the slots for `format`, the format of the bus the tap is on.
        func prepare(format: AVAudioFormat) throws {
            self.format = format
            let frames = AVAudioFrameCount(format.sampleRate / 10)
            guard frames > 0 else {
                throw Failure.nodeNotMade(name)
            }
            slots = try (0 ..< 20).map { _ in
                guard let slot = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: frames) else {
                    throw Failure.nodeNotMade(name)
                }
                return slot
            }
        }

        /// The tap: copies the buffer into the next slot, if one is free, and returns.
        func tapProvider() -> @Sendable (AVReadOnlyAudioPCMBuffer, AVAudioTime) -> Void {
            { [self] buffer, _ in
                let count = copied.load(ordering: .acquiring)
                guard count - drained.load(ordering: .acquiring) < slots.count else { return }
                let slot = slots[count % slots.count]
                slot.frameLength = AVAudioFrameCount(min(buffer.frameLength, Int(slot.frameCapacity)))
                buffer.withUnsafeAudioBufferList { source in
                    let from = UnsafeMutableAudioBufferListPointer(UnsafeMutablePointer(mutating: source))
                    let into = UnsafeMutableAudioBufferListPointer(slot.mutableAudioBufferList)
                    for channel in 0 ..< min(from.count, into.count) {
                        guard let data = from[channel].mData, let room = into[channel].mData else { continue }
                        memcpy(room, data, Int(min(from[channel].mDataByteSize, into[channel].mDataByteSize)))
                    }
                }
                copied.store(count + 1, ordering: .releasing)
            }
        }

        /// Opens `<name>.caf` in a new folder under `folder` and starts writing to it.
        func startWriting(named name: String, in folder: URL) throws {
            guard let format else { return }
            let url = folder.appending(path: UUID().uuidString, directoryHint: .isDirectory).appending(path: name + ".caf")
            try FileManager.default.createDirectory(at: url.deletingLastPathComponent(), withIntermediateDirectories: true)
            let opened = try AVAudioFile(forWriting: url, settings: format.settings, commonFormat: .pcmFormatFloat32, interleaved: false)
            lock.withLock {
                file = opened
                recording = nil
            }
            writer = Task.detached(priority: .userInitiated) { [self] in
                while !Task.isCancelled {
                    try? await Task.sleep(for: .milliseconds(20))
                    drain()
                }
            }
        }

        /// Writes every slot the tap has filled since the last time, under the writer's lock.
        private func drain() {
            lock.withLock {
                guard let file else { return }
                var count = drained.load(ordering: .acquiring)
                while count < copied.load(ordering: .acquiring) {
                    try? file.write(from: slots[count % slots.count])
                    count += 1
                    drained.store(count, ordering: .releasing)
                }
            }
        }

        /// Writes what is left, closes the file and keeps its URL as `recording`.
        func stopWriting() {
            writer?.cancel()
            writer = nil
            drain()
            lock.withLock {
                file?.close()
                recording = file?.url
                file = nil
            }
        }
    }
}

Tags

  • Graph · AVAudioEngine
  • Sources
  • Effects
  • Control
  • Outputs

Licence

Free to use in your own apps, commercial or not, with no credit needed. The terms of use carry the licence in full.