Events and Interaction

Interaction messages are user defined messages associated to events and triggered when these events occur. These messages accept variables as message arguments.

Events are typical user interface events (mouse or touch events), extended in the time domain and to specific objects or engine properties. Starting from INScore version 1.20, the modification of any object attribute may be viewed as an event and user defined events have also been introduced (see section attributeevents for more details).

The general form of the message to watch an event is the following:

what represents the event to watch and message is a list of associated messages, separated by a comma. A colon (':') is also supported as separator (to avoid issues with comma in Max).

• [1]: clear all the messages for all the events.
• [2]: clear the messages associated to the what event.
• [3]: associate a list of messages to the what event. With watch, the messages replace previously associated messages. Using watch+, the messages are appended to the messages currently associated to the event.
• [4]: associate or add a single message to the what event. This form is provided for compatibility with previous versions.

Note: The [1] and [2] form has no effect with the watch+ message. In some environments, the comma has a special meaning, making tricky to use it as a message separator. This is why ':' is also accepted as separator in OSC messages lists. The get watch message gives all the watch messages of a node, but doesn't take any argument.

The associated messages are any valid OSC message (not restricted to the INScore message set), with an extended address scheme, supporting IP addresses or host names and udp port number to be specified as OSC addresses prefix. The message parameters are any valid OSC type or variable (see section interactvar).

Example An extended address to send messages to localhost on port 12000:

localhost:12000/your/osc/address;

Internal events

Internal events are triggered by the user interaction (mouse or touch events) or by the engine internal functioning.

Mouse events

User interface events are typical mouse events:

Example Triggering a message on mouse down:

/ITL/scene/myObject watch mouseDown (/ITL/scene/myObject show 0);

the object hides itself on mouse click. Triggering a message on mouse down but addressed to another host on udp port 12100:

/ITL/scene/myObject watch mouseDown (host.domain.org:12100/an/address start); 

Note: UI events are not supported by objects that are synchronized as slave.

Mouse events can be simulated using a event message:

where mouseEvent is one of the events described above, x and y are integer values giving the click position, expressed in pixels and relative to the target object.

Example Simulating a mouse down at position 10, 10 :

/ITL/scene/myObject event mouseDown 10 10;

Touch events

Depending on the display device, multi-touch events are supported by INScore :

A typical sequence of generated events consists in a touchBegin event, followed by touchUpdate events and closed by a touchEnd.

Time events

Events are also defined on the time domain:

Each event takes a time interval as parameter, defined by two time specifications (see section time for the time format)

• timeEnter, timeLeave are triggered when an object date is moved to or out of a watched time interval,
• durEnter, durLeave are triggered when an object duration is moved to or out of a watched time interval.

Example An object that moves a score to a given page number when it enters its time zone.

/ITL/scene/myObject watch timeEnter 10/1 18/1 (/ITL/scene/score page 2);

Keyboard events

Keyboard events are triggered by pressing or releasing a key on the keyboard.

• 1) only the specified char will trigger the event,
• 2) charlist is a string interpreted like the bracket expression of regular expressions. For example, using [abc], 'a' or 'b' or 'c' will trigger a key event, using [1-9], any numeric from 1 to 9 will trigger a key event.
• 3) any key triggers a key event.
• 4) handles special keys to trigger the key event. A literal description of the key must be provided (e.g. ArrowLeft).

Note: Unlike standard applications, there is no notion of focus regarding key events: when several objects request keyboard input, all of them will receive key events, provided they are watching the same input.

MIDI events

MIDI events are supported, but only by the Web version. MIDI events are supported by the native version for compatibility reasons but they are inactive (no MIDI input). See INScore Web documentation for more details.

URL events

url objects (i.e. intermediate objects for URL based objects (see section filebasedrsrc) support specific events, intended to reflect the transaction state:

• success is triggered when the data have been downloaded,
• error is triggered when an error has occurred during the download,
• cancel is triggered when the target url or the object type is changed while downloading.

Example Triggering an error message in case of failure :

/ITL/scene/score set gmnf "http://ahost.adomain.org/score.gmn";
/ITL/scene/score watch error(
  /ITL/scene/status set txt "Failed to download file"
);

Miscellaneous events

• the export event is supported by all the components. It is triggered after an export message has been handled and could be used to simulate synchronous exports.
• the newData event is supported by all the components. It is triggered when the object specific data are modified (typically using the set message).

Type specific events

• the pageCount event is supported by all the symbolic score components (gmn(f), gmnstream, musicxml(f)). It is triggered when the page count of the score changes. It is mainly intended to manage variable scores like gmnstream.
• the newElement event is supported at scene and application (/ITL) level and triggered when a new element is added to the scene or when a new scene is created.
• the endPaint event is supported at scene level only and triggered after a scene has been painted.
• the error event is supported at application level and triggered when an error occurs while receiving messages. Typically you can use of this event to open the log window (/ITL/log show 1)
• the ready event is supported by media objects (audio and video). It is triggered when media data (duration, graphic dimensions) are available.
• the end event is supported by media objects (audio and video). It is triggered when a media is playing and reaches the end of the media. In this case, the object play status is automatically switched to 0 to reflect the actual player state.

Example Displaying a welcome message to new elements:

/ITL/scene watch newElement (/ITL/scene/msg set txt "Welcome");

Attribute based events

Attribute based events includes the whole set of messages that are supported by an object: x, y, color, etc. but also type specific messages. These events are triggered when a message has been succesfully processed. However, you shouldn't assume that the attribute value has been changed: when a message sets an attribute to it's current value, it is succesully processed and the corresponding event - if any - is triggered.

Example Watching an object x coordinate change:

/ITL/scene/myObject watch x (/ITL/scene/msg set txt "myObject moved");

Note: Watching the newData event is equivalent to watch the set attribute. However, the newData event is triggered only when the object state is changed. Although not an object attribute, the del message can be watched and triggered when an object is deleted.

Warning With the event's generalization to any object attribute, a one tick delay has been introduced to all events. Thus the associated messages are not processed synchronously to the event but posted to be processed by the next time task. This delay has been introduced to avoid freezing the system in case of loops. However, it introduces also a pitfall in interaction design, when message based variables are used (see section msgvar): message based variables are evaluated at the event time while messages are processed by the next time task, thus the following messages won't produce the expected result:

/ITL/scene/myObject watch x ( 
   /ITL/scene/A x '$(/ITL/scene/myObject get x)', 
   /ITL/scene/B x '$(/ITL/scene/A get x)' 
);

actually, when the $(/ITL/scene/A get x) variable is evaluated, the preceding message that sets the x attribute of A has not been already processed. One workaround consists in splitting the interaction in several parts, making sure that the messages are processed e.g.

/ITL/scene/myObject watch x ( /ITL/scene/A x '$(/ITL/scene/myObject get x)'); 
/ITL/scene/A watch x ( /ITL/scene/B x '$(/ITL/scene/A get x)' );

User defined events

INScore events supports user defined events. The name of user defined events must start with a capital letter and be composed of capital letters and/or numbers. Other characters are reserved for INScore use.

Messages attached to user defined events accept regular variables (although the position variables are useless), but they accept also any number of a variables which names are $1, ... $i and which values are taken from the event call arguments (see section udevar).

User defined events can only be triggered using the event message (see section eventMsg).

Example Watching and triggering a user defined event:

/ITL/scene/myObject watch MYEVENT (/ITL/scene/msg set txt "MYEVENT occured!");
/ITL/scene/myObject event MYEVENT;

Defining high level abstractions:

/ITL/scene/myObject watch MOVEABC ( 
  /ITL/scene/a x $1,
  /ITL/scene/b x $2,
  /ITL/scene/C x $3
); 
/ITL/scene/myObject event MOVEABC -0.4 0.1 0.6;

The 'event' message

The event message may be used to triggered events. It's the only way to trigger user defined events. It may be used also to simulate user interface events (like mouse events).

• 1: this form is intended to simulate mouse or touch event. It must be followed by the x and y coordinates of the interaction point. Coordinates express a position in pixels (the top left corner of the object is in [0,0]).
• 2: triggers a user defined event. It accepts any number of arguments, that are then expanded in place of the variables $1 ... $i. User defined events may be viewed as functions with arbitrary parameters; however parameters count and type is not checked.
• 3: triggers any other event.

Note: Time events are excluded from the event message supported events: to trigger a time event, you can use a date message.

Variables

Variables are values computed when an event is triggered. These values are send in place of the variable. A variable name starts with a '$' sign.

Position variables

Position variables reflects the current mouse position for mouse events or the current touch position for touch events. For attribute based events, the x and y variables are set to the target object current position and the other variables are undefined. For other events, the position variables are set to 0.

where

• $x $y: denotes the mouse pointer position at the time of the event. The values are in the range [0,1] where 1 is the object size in the x or y dimension. The value is computed according to the object origin: it represents the mouse pointer distance from the object x or y origin (see origin). $x and $y variables support an optional range in the form [low, high] that transforms the [0,1] values range into the [low, high] range.

• $absx $absy: denotes the mouse pointer absolute position at the time of the event. The values represent a pixel position relative to the top-left point of the target object. Note that this position is unaffected by scale. Note also that the values are not clipped to the object dimensions and could exceed its width or height or become negative in case of mouse move events.

• $sx $sy: denotes the mouse pointer position in the scene coordinates space.

Example An object that follows mouse move.

/ITL/scene/myObject watch mouseDown ( 
  /ITL/scene/myObject x '$sx', 
  /ITL/scene/myObject y '$sy' );

Sensor variables

Sensors values are available using the $x, $y and $z variables, for events generated in the context of a sensor.

Note that depending on the sensor type, the $y and $z variables may be useless.

Time variables

Time variables reflects the date corresponding to the current mouse position for mouse events. For attribute based events, the time variables are set to the target object current time position. They are set to 0 for the other events.

• $date: denotes the object date corresponding to the mouse pointer position at the time of the event. It is optionally followed by a colon and the name of the mapping to be used to compute the date. The $date variable is replaced by its rational value (i.e. two integers values). The optional rational enclosed in brackets may be used to indicate a quantification: the date value is rounded to an integer count of the specified rational value. The optional %f may be used to get the date delivered as a float value.
• $rdate: is similar to $date but ignores the target current date: the date is relative to the object mapping only.
• %f or %a: request the date value as a float value or as an absolute time value. Absolute time is expressed in seconds, assuming a tempo value of 60 bpm. Actually, $date%a = $date%f / 4

Note: A variable can be used several times in a message, but several $date variables must always refer to the same mapping.

Example Sending the current date as a float value to an external application:

/ITL/scene/myObject watch mouseDown ( targetHost:12000/date '$date%f' );

Miscellaneous variables

• $name is replaced by the target object name.
• $scene is replaced by the target object scene name.
• $address is replaced by the target object OSC address.

Note: For the newElement event, the target object is the new element.

Example Using an object name:

/ITL/scene watch newElement (/ITL/scene/welcome set txt "Welcome" '$name');

Message based variables

A message based variable is a variable containing an OSC message which will be evaluated at the time of the event. They are supported by all kind of events. Like the variables above, a message based variable starts with a '$' sign followed by a valid 'get' message enclosed in parenthesis:

The evaluation of a 'get' message produces a message or a list of messages. The message based variable will be replaced by the parameters of the messages resulting from the evaluation of the 'get' message. Note that all the 'get' messages attached to an event are evaluated at the same time.

Example An object that takes the x position of another object on mouse down:

/ITL/scene/myObject watch mouseDown 
  (/ITL/scene/myObject x '$(/ITL/scene/obj get x)');

Variables for user defined events

Messages associated to user defined events accept any number of a variables which names are $1, ... $i and which values are taken from the event call arguments. These events may be viewed as functions with arbitrary parameters; however parameters count and type is not checked: arguments in excess are ignored and variables without corresponding argument (e.g. $3 when only 2 arguments are available) are left unexpanded.

OSC address variables

The OSC address of a message associated to an event supports the following variables:

• $self: replaced by the object name.
• $scene: replaced by the scene name.

Example Requesting a set of objects to send a message to themselves on a mouse event:

/ITL/scene/* watch mouseDown   # request all the objects of the scene 
  (/ITL/scene/$self x '$sx'); # to send a message to themselves

Interaction state management

For a given object, its interaction state (i.e. the watched events and the associated messages) can be saved and restored.

Interaction states are managed using a stack where the states are pushed to or popped from.

• push: push the current interaction state on top of the stack.
• pop: replace the current interaction state with the one popped from the top of the stack.

The different states stored in this stack can be obtain with the message :

Note: The effect of a pop message addressed to an object with an empty stack is to clear the object current interaction state.

File watcher

The fileWatcher is a static node of a scene that is intended to watch file modifications. It receives messages at the address /ITL/scene/fileWatcher.

The fileWatcher support the watch and watch+ messages as described in section interaction with a file name used in place of the what parameter.

Example Reload an INScore script on file modification:

/ITL/scene/fileWatcher watch 'myScript.inscore' 
  ( /ITL/scene load 'myScript.inscore' );

Note: Some text editors delete the target file before saving. In this case, it breaks the file watching system and thus, it doesn't work as expected.