Skip to main content

Four corners (delays)

Template by nthState

Graph · AVAudioEngine · 6 nodes · 0:32 preview

Preview

Binaural preview · wear headphones. The app rendered it offline through AVAudioEnvironmentNode: it is a preview, not PHASE’s own output.

Download Four corners (delays).phonique

A .phonique document is a folder, so it comes as a ZIP archive, 0.0 MB. Unzip it and open the folder in Phonique.

Opening a template straight from this page waits until the app can open one; until then the download is the way in.

What’s inside

6 nodes and 5 cables, read from the template’s own document.

The Swift

What Phonique exports from this template: a Swift package, FourCornersDelays, that needs nothing beyond Apple’s frameworks. Its README’s “Use it”:

import FourCornersDelays

let graph = try FourCornersDelaysGraph()
try graph.start()
// …
graph.stop()
The whole of FourCornersDelaysGraph.swift, 321 lines
//
//  FourCornersDelaysGraph.swift
//  Generated by Phonique from “Four corners (delays)”.
//
//  Phonique writes this file afresh each time it exports the graph, so a
//  change made here is gone after the next export.
//

import AVFAudio
import Foundation

/// “Four corners (delays)”, as Phonique plays it: its nodes on an `AVAudioEngine`, connected as
/// the graph's cables connect them.
///
/// Make one, `start()` it to play through the device's output, and `stop()` it; it plays
/// again when it is started again. It is isolated to the main actor, where its engine is
/// driven: make it, start it and stop it there. Everything that can fail throws —
/// `Failure` for what this code checks itself, and AVFAudio's own errors — and nothing here
/// stops the app.
@MainActor
public final class FourCornersDelaysGraph {
    /// Why the graph could not be built or started, besides the errors AVFAudio throws.
    public nonisolated enum Failure: Error, Hashable, Sendable {
        /// The device has no audio output to play through: the engine's output node has no
        /// format.
        case outputUnavailable
        /// The device has no audio input to listen to: the graph uses the microphone, and the
        /// engine's input node has no format.
        case inputUnavailable
        /// A node could not be made: its name in the graph.
        case nodeNotMade(String)
        /// One of the package's audio files could not be found or read: its name.
        case audioFileUnreadable(String)
    }

    /// The engine the graph plays on.
    public let engine: AVAudioEngine

    /// File Player “Top left”.
    private let topLeft: AVAudioPlayerNode

    /// File Player “Top right”.
    private let topRight: AVAudioPlayerNode

    /// File Player “Bottom right”.
    private let bottomRight: AVAudioPlayerNode

    /// File Player “Bottom left”.
    private let bottomLeft: AVAudioPlayerNode

    /// 3D Environment.
    private let node3DEnvironment: AVAudioEnvironmentNode

    /// Builds the graph on `engine`, ready to `start()`: makes each node, attaches it to the
    /// engine, sets its values and connects the cables, each on the bus Phonique connects it on.
    ///
    /// To render the graph offline, give it an engine already in manual rendering mode
    /// (`enableManualRenderingMode(_:format:maximumFrameCount:)`), which has to come before
    /// anything asks for the engine's output node.
    ///
    /// - Parameter engine: The engine to build on: a new one, unless you give your own.
    /// - Throws: `Failure`, or the error AVFAudio throws when it refuses a cable or a file.
    public init(engine: AVAudioEngine = AVAudioEngine()) throws {
        // Every cable carries one format: two channels of 32-bit float at the output's sample rate,
        // but one channel into a node that places a sound in space, and through every effect before
        // it.
        let output = engine.outputNode.outputFormat(forBus: 0)
        guard output.sampleRate > 0, output.channelCount > 0,
              let format = AVAudioFormat(standardFormatWithSampleRate: output.sampleRate, channels: 2)
        else {
            throw Failure.outputUnavailable
        }
        let monoFormat = AVAudioFormat(standardFormatWithSampleRate: format.sampleRate, channels: 1) ?? format

        // Make each node.
        let topLeft = AVAudioPlayerNode()
        topLeft.sourceMode = .pointSource
        topLeft.renderingAlgorithm = .HRTFHQ
        let topRight = AVAudioPlayerNode()
        topRight.sourceMode = .pointSource
        topRight.renderingAlgorithm = .HRTFHQ
        let bottomRight = AVAudioPlayerNode()
        bottomRight.sourceMode = .pointSource
        bottomRight.renderingAlgorithm = .HRTFHQ
        let bottomLeft = AVAudioPlayerNode()
        bottomLeft.sourceMode = .pointSource
        bottomLeft.renderingAlgorithm = .HRTFHQ
        let node3DEnvironment = AVAudioEnvironmentNode()

        // Attach each to the engine.
        engine.attach(topLeft)
        engine.attach(topRight)
        engine.attach(bottomRight)
        engine.attach(bottomLeft)
        engine.attach(node3DEnvironment)

        // Set each node's values.
        topLeft.position = AVAudio3DPoint(x: -2.0, y: 0.0, z: -2.0)
        topRight.position = AVAudio3DPoint(x: 2.0, y: 0.0, z: -2.0)
        bottomRight.position = AVAudio3DPoint(x: 2.0, y: 0.0, z: 2.0)
        bottomLeft.position = AVAudio3DPoint(x: -2.0, y: 0.0, z: 2.0)
        node3DEnvironment.listenerAngularOrientation = AVAudio3DAngularOrientation(yaw: 0.0, pitch: 0.0, roll: 0.0)

        // Connect the cables, in the order they were made: a mixer's input takes its next free bus.
        try engine.connectNode(topLeft, to: node3DEnvironment, fromBus: 0, toBus: node3DEnvironment.nextAvailableInputBus, format: monoFormat)
        try engine.connectNode(topRight, to: node3DEnvironment, fromBus: 0, toBus: node3DEnvironment.nextAvailableInputBus, format: monoFormat)
        try engine.connectNode(bottomRight, to: node3DEnvironment, fromBus: 0, toBus: node3DEnvironment.nextAvailableInputBus, format: monoFormat)
        try engine.connectNode(bottomLeft, to: node3DEnvironment, fromBus: 0, toBus: node3DEnvironment.nextAvailableInputBus, format: monoFormat)
        try engine.connectNode(node3DEnvironment, to: engine.outputNode, fromBus: 0, toBus: 0, format: format)

        // What each node does once it is connected.
        try topLeft.scheduleBuffer(
            Self.repeatedAudio(
                Self.decodedAudio(at: Self.audioFile("woodblock.wav"), into: topLeft.outputFormat(forBus: 0)),
                every: 4.0,
                node: "Top left"
            ),
            at: nil,
            options: .loops,
            completionHandler: nil
        )
        try topRight.scheduleBuffer(
            Self.repeatedAudio(
                Self.decodedAudio(at: Self.audioFile("woodblock.wav"), into: topRight.outputFormat(forBus: 0)),
                every: 4.0,
                node: "Top right"
            ),
            at: Self.playerTime(after: 1.0, in: topRight.outputFormat(forBus: 0)),
            options: .loops,
            completionHandler: nil
        )
        try bottomRight.scheduleBuffer(
            Self.repeatedAudio(
                Self.decodedAudio(at: Self.audioFile("woodblock.wav"), into: bottomRight.outputFormat(forBus: 0)),
                every: 4.0,
                node: "Bottom right"
            ),
            at: Self.playerTime(after: 2.0, in: bottomRight.outputFormat(forBus: 0)),
            options: .loops,
            completionHandler: nil
        )
        try bottomLeft.scheduleBuffer(
            Self.repeatedAudio(
                Self.decodedAudio(at: Self.audioFile("woodblock.wav"), into: bottomLeft.outputFormat(forBus: 0)),
                every: 4.0,
                node: "Bottom left"
            ),
            at: Self.playerTime(after: 3.0, in: bottomLeft.outputFormat(forBus: 0)),
            options: .loops,
            completionHandler: nil
        )

        self.engine = engine
        self.topLeft = topLeft
        self.topRight = topRight
        self.bottomRight = bottomRight
        self.bottomLeft = bottomLeft
        self.node3DEnvironment = node3DEnvironment
    }

    /// Starts the engine, then each node that plays once the engine has started. If a node
    /// does not start, the graph stops again and the error is thrown.
    public func start() throws {
        try engine.start()
        do {
            try startNodes()
        } catch {
            stop()
            throw error
        }
    }

    /// Stops the graph: each node that has something to do first, then the engine. `start()`
    /// plays it again.
    public func stop() {
        topLeft.pause()
        topRight.pause()
        bottomRight.pause()
        bottomLeft.pause()
        engine.stop()
    }

    /// What each node does once the engine has started.
    private func startNodes() throws {
        try topLeft.playAudio()
        try topRight.playAudio()
        try bottomRight.playAudio()
        try bottomLeft.playAudio()
    }

    /// The package's audio file `name`, from the package's resources: “Always use
    /// `Bundle.module` when you access resources” (Bundling resources with a Swift package).
    private static func audioFile(_ name: String) throws -> URL {
        guard let url = Bundle.module.url(forResource: name, withExtension: nil) else {
            throw Failure.audioFileUnreadable(name)
        }
        return url
    }

    /// The whole of the audio file at `url`, decoded to loop in `format`: read until the file
    /// ends, then converted to its sample rate and its channels — a mono file into every
    /// channel, a file with more channels mixed down — as Phonique's File Player reads its file.
    /// A file that would decode to more than 512 MiB is refused before a byte is decoded,
    /// and one that ends before its length is unreadable, never a loop missing its end.
    private static func decodedAudio(at url: URL, into format: AVAudioFormat) throws -> AVAudioPCMBuffer {
        let name = url.lastPathComponent
        let file = try AVAudioFile(forReading: url)
        let source = file.processingFormat
        guard file.length > 0, source.sampleRate > 0, source.channelCount > 0, format.sampleRate > 0 else {
            throw Failure.audioFileUnreadable(name)
        }
        let largest = Double(512 * 1024 * 1024)
        let frames = (Double(file.length) * format.sampleRate / source.sampleRate).rounded(.up)
        let bytesPerFrame = Double(max(source.channelCount, format.channelCount)) * Double(MemoryLayout<Float>.size)
        guard frames * bytesPerFrame <= largest,
              Double(file.length) * Double(source.channelCount) * Double(MemoryLayout<Float>.size) <= largest,
              let whole = AVAudioPCMBuffer(pcmFormat: source, frameCapacity: AVAudioFrameCount(file.length))
        else {
            throw Failure.audioFileUnreadable(name)
        }
        // One read "attempts to fill the buffer to its capacity" and can come back short with no
        // error, so what is left is read into a buffer of its own and copied on, until the file's
        // position reaches its length. A file that ends before its length is an error, never a
        // buffer missing its end: AVFAudio's error for a read past the end, or this one for a
        // read that gives no frames.
        try file.read(into: whole)
        while file.framePosition < file.length {
            let left = AVAudioFrameCount(file.length - file.framePosition)
            guard let rest = AVAudioPCMBuffer(pcmFormat: source, frameCapacity: left) else {
                throw Failure.audioFileUnreadable(name)
            }
            try file.read(into: rest)
            guard rest.frameLength > 0 else {
                throw Failure.audioFileUnreadable(name)
            }
            // Onto the end of the whole, in the one format both are in: each of its buffers — one
            // a channel deinterleaved, one for all of them interleaved — at the format's bytes a
            // frame.
            let bytesPerFrame = Int(source.streamDescription.pointee.mBytesPerFrame)
            let read = UnsafeMutableAudioBufferListPointer(UnsafeMutablePointer(mutating: rest.audioBufferList))
            let kept = UnsafeMutableAudioBufferListPointer(whole.mutableAudioBufferList)
            for (from, to) in zip(read, kept) {
                guard let from = from.mData, let to = to.mData else { continue }
                (to + Int(whole.frameLength) * bytesPerFrame).copyMemory(from: from, byteCount: Int(rest.frameLength) * bytesPerFrame)
            }
            whole.frameLength += rest.frameLength
        }
        if source == format {
            return whole
        }
        guard let converter = AVAudioConverter(from: source, to: format) else {
            throw Failure.audioFileUnreadable(name)
        }
        converter.downmix = true
        converter.sampleRateConverterQuality = AVAudioQuality.max.rawValue
        let capacity = AVAudioFrameCount((Double(whole.frameLength) * format.sampleRate / source.sampleRate).rounded(.up)) + 1
        guard let converted = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: capacity) else {
            throw Failure.audioFileUnreadable(name)
        }
        // The whole file, once, then the end of the stream: the converter calls this on the
        // calling thread, inside `convert(to:error:withInputFrom:)`.
        var given = false
        var error: NSError?
        let status = converter.convert(to: converted, error: &error) { _, inputStatus in
            if given {
                inputStatus.pointee = .endOfStream
                return nil
            }
            given = true
            inputStatus.pointee = .haveData
            return whole
        }
        if let error {
            throw error
        }
        guard status == .haveData || status == .endOfStream, converted.frameLength > 0 else {
            throw Failure.audioFileUnreadable(name)
        }
        return converted
    }

    /// `audio` made `seconds` long, to loop from one start of the file to the next, as Phonique's
    /// File Player repeats its file: the audio, then silence, when it is shorter; its first
    /// `seconds`, when it is longer. A time shorter than a frame is `audio` as it is, which loops
    /// end to end. Made once the graph is built, never on the render thread.
    private static func repeatedAudio(_ audio: AVAudioPCMBuffer, every seconds: Double, node: String) throws -> AVAudioPCMBuffer {
        let format = audio.format
        let frames = AVAudioFramePosition((seconds * format.sampleRate).rounded())
        guard frames > 0 else {
            return audio
        }
        let length = AVAudioFrameCount(clamping: frames)
        guard let repeated = AVAudioPCMBuffer(pcmFormat: format, frameCapacity: length) else {
            throw Failure.nodeNotMade(node)
        }
        repeated.frameLength = length
        // Each of its buffers — one a channel deinterleaved, one for all of them interleaved — is
        // zeroed, and the audio copied in at the format's bytes a frame.
        let bytesPerFrame = Int(format.streamDescription.pointee.mBytesPerFrame)
        let kept = Int(min(audio.frameLength, length)) * bytesPerFrame
        let from = UnsafeMutableAudioBufferListPointer(UnsafeMutablePointer(mutating: audio.audioBufferList))
        let to = UnsafeMutableAudioBufferListPointer(repeated.mutableAudioBufferList)
        for (source, destination) in zip(from, to) {
            guard let destination = destination.mData else { continue }
            memset(destination, 0, Int(length) * bytesPerFrame)
            if let source = source.mData {
                destination.copyMemory(from: source, byteCount: kept)
            }
        }
        return repeated
    }

    /// `seconds` into a player's own timeline, which begins at zero when the player starts, in
    /// frames of `format`: when its loop first plays, as Phonique's File Player waits its start
    /// delay. Nil, as for no delay, when it rounds to no frames.
    private static func playerTime(after seconds: Double, in format: AVAudioFormat) -> AVAudioTime? {
        let frames = AVAudioFramePosition((seconds * format.sampleRate).rounded())
        return frames > 0 ? AVAudioTime(sampleTime: frames, atRate: format.sampleRate) : nil
    }
}

Tags

  • Graph · AVAudioEngine
  • Sources
  • Spatial
  • Outputs

Licence

Free to use in your own apps, commercial or not, with no credit needed. The terms of use carry the licence in full.