nxu/audio-v2.md

# UXN Audio Proposal (v2)

*(Updated with input from bd and neauoire)*

## Problems

Currently the UXN audio device doesn't work very well for playing
complex music. There are a few reasons for this:

 * Note duration is conflated with envelope shape
 * Envelope resolution (67ms) limits tempos/subdivisions
 * Using audio callback requires scheduling pauses/silence
 
## Proposal outline

One way to improve the situation is to disentangle the envelope
specification from the note duration, and more generally make it
easier to specify things that a composer will frequently need to
change (pitch, articulation, duration) without having to change the
underlying voice (waveform/envelope settings).

This proposal makes four changes:

 1. Add a two-byte `duration` port that configures a note's duration
    in milliseconds. The longest possible note is about 66 seconds.

 2. Double the size of the `adsr` port. This means replacing the
    existing two-byte port with four one-byte ports for `attack`,
    `decay`, `sustain`, and `release`. Since we have 4 extra bits per
    stage, we will reduce the resolution of each stage from 66ms to
    10ms (so 0x01 means 10ms). The longest envelope stage is now about
    2.6s (up from 1s previously). We special-case `sustain` and
    instead treat its value as a fraction `x/255` (i.e. 0.0 to 1.0).
    
 3. Move various ports around, both to improve the layout and prepare
    for future additions. In particular an `expansion` port for
    possible MIDI operations and a `detune` port for microtonal music
    are likely (but are left unspecified by this proposal).

 4. Recommends that emulators use a separate `wst` and `rst` for
    evaluating the audio vector (when possible). Code run from the
    audio vector should not expect to read existing values from `wst`
    or `rst` (and should not leave values behind). This allows
    emulators to use a separate audio thread for evaluating callbacks
    without needing to pause other execution.

## Note duration and tempo

The `duration` and `vector` ports precisely specify the audio device
behavior. The given note should be played for a number of milliseconds
specified by `duration`, at which point the `vector` should be called
to play the next note (or next silence).

## More flexible envelope settings

The ADSR ports determine how loud the pitch should be at any given
moment. The ADR ports (`attack`, `decay`, and `release`) are all
specified in 10ms increments (e.g. `0x03` is 30ms). The S port for
`sustain` behaves differently: it specifies what how much of the
"leftover" duration to use before the release as a fraction `x/255`.
So with a value of `0xff` the note would hold as long as possible, and
with `0x00` the release would occur just after the decay ends.

Since each component has its own port, it's also much easier to adjust
one without having to fiddle with bit masks, shifting, etc.

## Appending A: proposed specification:

ADDR  SIZE     NAME      DESCRIPTION
0x30  2 bytes  vector    callback address to use when note finishes playing
0x32  2 bytes  duration  duration to play sound in fractional seconds (1ms resolution)
0x34  1 byte   attack    envelope: attack duration (vol 0-100%, 10ms resolution)
0x35  1 byte   decay     envelope: decay duration (vol 100-50%, 10ms resolution)
0x36  1 byte   sustain   envelope: sustain duration (vol 50%, 10ms resolution)
0x37  1 byte   release   envelope: release duration (vol 50-0%, 10ms resolution)
0x38  2 bytes  addr      address to read waveform data from
0x3a  2 bytes  length    length of waveform data to read (in bytes)
0x3c  1 byte   volume    4-bit volumes for left/right channels (6.7% resolution)
0x3d  1 byte             (unused - reserved for expansion)
0x3e  1 byte   pitch     1-bit loop and 7-bit MIDI note (0x00 gives silence)
0x3f  1 byte             (unused - reserved for detune)

## Appendix B: existing specification

ADDR  SIZE     NAME      DESCRIPTION
0x30  2 bytes  vector    callback address to use when note finishes playing
0x32  2 bytes  position  read current position in sample
0x34  1 byte   output    read envelope loudness at this moment (0x000 to 0x888)
0x35                     (unused)
0x36                     (unused)
0x37                     (unused)
0x38  2 bytes  adsr      four 4-bit envelope values (attack/decay/sustain/release)
0x3a  2 bytes  length    length of waveform data to read in bytes
0x3c  2 bytes  addr      address to read waveform data from
0x3e  1 byte   volume    4-bit volumes for left/right channels
0x3f  1 byte   pitch     1-bit loop and 7-bit MIDI note