Audio

Audio objects

Audio objects are connectable objects i.e. objects which output can be connected to the input of another audio object. In the INScore model, the following objects are audio objects: audio, video, faust (see section faustObjects) and audioio (see section audioioObjects).

Audio objects support the following messages:

audioMsgs
  • connect: connect the outputs of the object to the destination inputs. destination must be another audio object. Without numeric prefix and suffix [1], the connexion may be subject of up or down mixing as specified by the Web Audio API. The destination supports regular expressions. The second form [2] connects an output channel to the a destination input channel. The first number refers to the target object channel, the second one to the destination channel. Channels are indexed from 1, using 0 is equivalent to all channels (that are then mixed together). See the examples below.
  • disconnect: disconnect the outputs of the object from the destination inputs. Note that errors (e.g. no existing connection with the destination) are silently ignored.

Example Connect the first channel of an audio object to the second channel of the audio output :

/ITL/scene/obj connect '1:audioOutput:2';

Connect all channels of an audio object to the first channel of the audio output :

/ITL/scene/obj connect '0:audioOutput:1';

Note: The messages /ITL/scene/obj connect '0:dest:0'; is equivalent to /ITL/scene/obj connect dest;

Note: The connect message assumes that the source and destination are located in the same hierarchy (i.e. they have the same parent).

audiogetMsgs
  • in: gives the number of inputs of the audio object
  • out: gives the number of outputs of the audio object

Faust objects

Faust is a functional programming language for sound synthesis and audio processing. Faust objects are available, providing that the Faust library has been loaded.

Warning A page containing Faust objects require an https connection, unless it runs on localhost.

Creating a Faust object

The faust or faustf types must be used to create a Faust object.

Note: The faust type exists with the native version but to load a pre-compiled DSP. faustdsp and faustdspf types are not supported.

setFaust

The expected arguments of the set message are:

  • an optional integer that indicates a number of voices used to create a polyphonic DSP (see section webFaustPoly). Note that when present, a polyphonic DSP is created even if equal to 1.
  • 1: Faust DSP code (see the Faust language for more information).
  • 2: a Faust DSP file.
  • 3: a Faust pre-compiled wasm file and the associated json file. Note that theses files can be generated using the get wasm message (see section webFaustMsgs).

By default, a Faust DSP appears as a browsable block diagram.

Note: The Faust language uses characters that have a special meaning in HTML: for example, with the split operator <:, the '<' character will be interpreted as an opening HTML tag and you should use HTML escapes (e.g. &lt; instead of <). This is necessary for inline DSP code only, DSP files are not concerned.

Faust messages

Faust objects are active by default, i.e. the output signals is computed whatever the audio connection state. However, it is possible to disable the signals computation when it is not needed (e.g. for objects that are only temporarily active), which allows a large number of faust objects to be embedded in a score, while saving a significant CPU. The compute message is provided in this intend.

faustMsgs
  • compute: start or stop computing the audio signals. The parameter is a boolean value. Default value is 1.
  • autoOff: automatically switch buttons Faust UI elements back to off. The parameter is a boolean value. Default value is 0.

Faust objects support the audio objects messages plus additional query messages:

faustgetMsgs
  • ui: gives the Faust processor parameters (see section webFaustParams) i.e. for each parameter: its OSC address followed by the parameter UI type, a label, the minimum and maximum values and the parameter step.
  • wasm: generates the Faust dsp as a precompiled wasm file with the associated json file. The Faust object name is used for the files name. On output, the files xxx.wasm and xxx.json should be present in the local download folder (where xxx is the Faust object name).

Faust objects parameters

A Faust DSP code can declare UI elements that are used by architecture files to build controllers providing users with dynamic control of the DSP parameters. In INScore, DSP UI elements are used to extend the Faust object address space. For example, when a DSP code declares a UI element named 'Volume', a Faust object which address is /ITL/scene/dsp is extended as /ITL/scene/dsp/Volume.

Faust parameters support two types of messages:

faustParamMsgs
  • 1: set the parameter value
  • 2: gives the parameter value

Polyphonic objects

Polyphonic objects (i.e. Faust objects created using the optional voice number) support additional messages:

faustPolyMsgs
  • keyOn and keyOff messages take 3 integer parameters: a MIDI channel, the note pitch and the velocity.
  • allNotesOff: similar to MIDI all notes off message

Audio Input / Output

Audio input / output objects are provided as audio object (see section audioObjects) to represent the physical audio inputs and outputs, in order to homogenize the connection process. An audio input / output type is audioio. It is created with a number of inputs and outputs as arguments.

audioioMsgs

Note: The current Javascript implementation automatically creates an audio input objet named audioInput and an audio output object named audioOutput. Thus the names audioInput and audioOutput are reserved and you must not use them. audioOutput and audioInput are created on audio request only, i.e. when an audio object is created (audio, video or faust).

Note: About I/O OSC addresses Since audio connections can only be made between objects that are in the same hierarchy, audioOutput and audioInput objects are created in all hierarchies that contain audio objects. For example, by creating an faust object in the /ITL/scene/folder hierarchy, audio I/O will be created at /ITL/scene/folder/audioInput and /ITL/scene/folder/audioOutput. Thus audio I/O objects may appear at different levels of a scene.

Audio Input

By default, the audioInput object is created with 0 inputs. It must be explicitly initialized using the init message. On success, the ready event is triggered and the number of input channels is set, otherwise an error event is triggered. This initialization is intended to avoid capturing the audio input when not used.

Example

/ITL/scene/audioInput watch ready ( do something to use the audio input ); 
/ITL/scene/audioInput watch error ( do something else ); 
/ITL/scene/audioInput init;