Stream

Handles the management of the SuperCollider instances and their respective WebRTC streams. See also OSC Server for the communication from each SuperCollider instance to our backend.

Graph

Model graph for the stream app.

Models

class stream.models.AudioFile(*args, **kwargs)

Represents a local audio file on the server. As SuperCollider and Django are running on the same server we can pass these files to the SuperCollider instances as they are mounted within each service.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• name (CharField) – Name. Acts as an identifier for humans

• auto_generated (BooleanField) – Auto generated. Allows to separate automatic generated audio files speech to text and user uploads

• file (FileField) – File

• description (TextField) – Additional description

Reverse relationships:

Parameters:

• audio_cells (Reverse ForeignKey from AudioCell) – All audio cells of this Audio file (related name of audio_file)

• texttospeech (Reverse ForeignKey from TextToSpeech) – All Text to speech jobs of this Audio file (related name of audio_file)

class stream.models.Stream(*args, **kwargs)

Assigns a StreamPoint to a user/client. This allows us to see which streams are currently in use and also by which user. It also allows us to trace past streams.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• num_listeners (IntegerField) – Number of listeners. Used as a garbage collection. If multiple users share the same stream we need to know when we can release the stream which happens if listener counter is 0. It starts with a default of 0 because this allows us to count stateless.

Relationship fields:

Parameters:

• stream_point (ForeignKey to StreamPoint) – Associated instance (related name: streams)

• graph (ForeignKey to Graph) – Graph (related name: stream)

Reverse relationships:

Parameters:

• variables (Reverse ForeignKey from StreamVariable) – All variables of this Stream (related name of stream)

• streamlog (Reverse ForeignKey from StreamLog) – All Stream logs of this Stream (related name of stream)

class stream.models.StreamInstruction(*args, **kwargs)

Instruction for a StreamPoint, most likely to be created from a ScriptCell.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• instruction_text (TextField) – Instruction that gets transmitted via OSC

• state (TextField) – Instruction state

• return_value (TextField) – Return value from statement

Relationship fields:

Parameters:

stream_point (ForeignKey to StreamPoint) – Stream point (related name: instructions)

class InstructionState(value)

Possible states of our instruction.

See also

See also SCAcknowledgeMessage.

classmethod from_sc_string(sc_string)

Converts a string from SuperCollider to our typed state choices.

Todo

return type

class stream.models.StreamLog(uuid, created_date, modified_date, stream_point, stream, origin, level, message, name)

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• origin (TextField) – Origin

• level (IntegerField) – Level

• message (TextField) – Message

• name (TextField) – Name

Relationship fields:

Parameters:

• stream_point (ForeignKey to StreamPoint) – Stream point (related name: streamlog)

• stream (ForeignKey to Stream) – Stream (related name: streamlog)

class LogLevel(value)

Taken from logging module but omitting FATAL and WARN.

class Origin(value)

States from which module the current logging occurs

class stream.models.StreamPoint(*args, **kwargs)

Stores metadata for each SuperCollider/Janus instance and how we can interact with this instance.

Every SuperCollider instance that send a beacon to us via the OSC Server will be a StreamPoint. Consider last_live to filter out non-live from live instances.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• host (CharField) – SuperCollider host

• port (IntegerField) – SuperCollider port

• use_input (BooleanField) – Use input. Accepts to send audio input

• janus_in_port (IntegerField) – Jauns in port. RTP port where Janus streams the audio its received from user

• janus_out_port (IntegerField) – Janus out port. RTP port where SuperCollider/gstreamer streams its audio to Janus

• janus_in_room (IntegerField) – Janus in room. Audiobridge room ID under which Janus can send audio to SuperCollider

• janus_out_room (IntegerField) – Jauns out room. Streaming room ID under which Janus serves audio from SuperCollider

• janus_public_ip (CharField) – Janus public IP. IP or Hostname under which the janus instance is reachable

• sc_name (CharField) – SuperCollider name. Internal name of the SuperCollider instance on the host, necessary for gstreamer

• last_live (DateTimeField) – Last live signal. Last live signal from SuperCollider server

Reverse relationships:

Parameters:

• streams (Reverse ForeignKey from Stream) – All streams of this Stream Endpoint (related name of stream_point)

• instructions (Reverse ForeignKey from StreamInstruction) – All instructions of this Stream Endpoint (related name of stream_point)

• streamlog (Reverse ForeignKey from StreamLog) – All Stream logs of this Stream Endpoint (related name of stream_point)

send_stream_instruction(instruction, manual_finish=False)

Sends an OSC message with an instruction to our SuperCollider server. Check the function instructionReceiver in GenCaster.sc which will accept the send message.

Parameters:

• instruction (StreamInstruction) – The given instruction on the Server.

• manual_finish (bool) – If set to True, one has to take care self of sending the unlocking finished message. This is helpful in cases of async function calls, e.g. on a playback of a sample. Defaults to False.

Return type:

None

speak_on_stream(ssml_text)

Speaks on the stream

Parameters:

ssml_text (str) – See https://cloud.google.com/text-to-speech/docs/ssml

Return type:

StreamInstruction

class stream.models.StreamVariable(*args, **kwargs)

Allows to store variables in a stream session as a key/value pair.

Warning

Due to database constraints all keys and values will be stored as a string, so parsing a float, int or boolean requires type conversion.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• key (CharField) – Key

• value (TextField) – Value

• stream_to_sc (BooleanField) – Stream to SuperCollider. Stream values to SC as control rate Ndef

Relationship fields:

Parameters:

stream (ForeignKey to Stream) – Stream (related name: variables)

send_to_sc()

Makes the stream variable available on the scsynth server under the same name as an Ndef.

Note

This used to be solved with

Ndef(\foo, {val});

but this introduced a clicking noise on each update. The solution seems to be to use instead

Ndef(\foo, val);

without the surrounding curly brackets for the value.

Return type:

StreamInstruction

class stream.models.TextToSpeech(*args, **kwargs)

Handles the conversion of text to speech by using external APIs.

Parameters:

• uuid (UUIDField) – Primary key: Uuid

• created_date (DateTimeField) – Created date

• modified_date (DateTimeField) – Modified date

• text (TextField) – Input text in SSML format

• voice_name (CharField) – Name of voice used to generate

Relationship fields:

Parameters:

audio_file (ForeignKey to AudioFile) – Audio file (related name: texttospeech)

class VoiceNameChoices(value)

See here.

The first 5 characters need to be the language code

classmethod create_from_text(ssml_text, voice_name=VoiceNameChoices.DE_NEURAL2_C__FEMALE, force_new=False)

Creates a new instance for a given text by calling the Google Cloud. We will not call the API if we find the exact same text in our database, in which case we will return the object from the database. This caching behavior can be controlled via force_new.

See also

Copied from google examples

Parameters:

• ssml_text (str) – SSML text to convert to audio

• voice_name (str) – Voice name to use

• force_new (bool) – If new we will not search for existing objects with the same text.

Return type:

TextToSpeech

Frontend Types

These types are not reflecting any database content but are for triggering functionality in the frontend from within a :class`~story_graph.models.Graph`.

These types can be used within a Python ScriptCell as they are also available within the Engine.

The stream subscription makes it possible to yield the

class stream.frontend_types.Button(*, text, value, key='button', button_type=ButtonType.DEFAULT, callback_actions=<factory>)

A button which can also trigger a set of functionality.

__init__(*, text, value, key='button', button_type=ButtonType.DEFAULT, callback_actions=<factory>)

classmethod cancel(text='Cancel', value='cancel', button_type=ButtonType.WARNING, callback_actions=[<CallbackAction.SEND_VARIABLE: 'send_variable'>], **kwargs)

Constructor for a cancel button which will simply close the dialog and set the StreamVariable button to 'cancel'.

classmethod ok(text='OK', value='ok', button_type=ButtonType.PRIMARY, callback_actions=[<CallbackAction.SEND_VARIABLE: 'send_variable'>, <CallbackAction.SEND_VARIABLES: 'send_variables'>], **kwargs)

Constructor for a OK button which will set the StreamVariable button to OK.

class stream.frontend_types.ButtonType(value)

Derived from ElementPlus framework, see `https://element-plus.org/en-US/component/button.html`_.

class stream.frontend_types.CallbackAction(value)

Allows to add a pre-defined JavaScript callback to a button or a checkbox.

ACTIVATE_GPS_STREAMING Activates streaming of GPS coordinates

as StreamVariable. If the GPS request succeeds the dialog will be closed, if not it the user will be forwarded to an error page which describes the setup procedure for the OS.

SEND_VARIABLES Send all variables of the form / dialog to

the server.

SEND_VARIABLE Sends a single StreamVariable

with the key/value of the where the callback is attached to.

class stream.frontend_types.Checkbox(*, key, label, checked=False, callback_actions=<factory>)

A classic <checkbox> whose state (true/false) will be saved as a string under key in a StreamVariable.

__init__(*, key, label, checked=False, callback_actions=<factory>)

class stream.frontend_types.Dialog(*, title, content, buttons)

Triggers a popup on the frontend of the listener.

__init__(*, title, content, buttons)

class stream.frontend_types.Input(*, key, label='input', placeholder='Please input')

A classic <inptut> which will save its content under the key as a StreamVariable.

__init__(*, key, label='input', placeholder='Please input')

class stream.frontend_types.Text(*, text)

Displays plain text.

__init__(*, text)

Exceptions

Exceptions

exception stream.exceptions.InvalidAudioFileException

Invalid audio file. This is necessary as the API accepts empty files so we can not use the data model validation but need to implement a validation by ourselves.

exception stream.exceptions.NoStreamAvailableException

No stream available