MT (MIDI Trigger) GUI: case study in model/view design

[dewdrop_world]

This is just a start... I'm not on a machine where I can post code, so I'll start with an overview of my MIDI trigger GUI and follow up with the code later.

The main points to take away:

• a model/view relationship where the models can be hot swapped without changing the "physical" GUI object. The model is the "real object" that does work. The view only represents the model in the interface.
• using object dependencies for communication from the model to the view. I learned this from the crucial library GUI structures. Each view knows who its model is so that manipulating the view can send the action back to the model. You might want to have multiple views of the same model. An example in OSX would be two folder windows in the Finder both showing the same folder. If you rename or delete a file in one window, the other window should update also. In that case, the execution flow is (roughly):

• the folder view tells the operating system to rename the file
• the OS notifies all the dependents of the folder that a file name changed
• each dependent can then decide whether this is a changed it can represent and update itself accordingly

[li]a nifty trick to make a GUI widget behave like a drag source (drag things out of it), a drag sink (drag things into it) and a clickable button[/li][/list]

My requirements for this interface component were:

• GUI objects that would represent keys on a MIDI keyboard, so that playing the note on the keyboard would trigger (play or stop) the sequencing process assigned to the GUI. The relationships between the widgets and the MIDI keys would be fixed, so I could look at the screen and know which MIDI key to press.
• easy assignment and reassignment of sequencing processes -- that is, the relationship between the GUI and the processes (the models) would be flexible.
• showing the state of the sequencing process (playing or stopped). The state might change programmatically without using the GUI, so there has to be a mechanism so that the process can tell the GUI that it changed.

So I designed it into this structure:

MT object -- manages the MIDI triggers for an entire MIDI channel. MT holds an IdentityDictionary mapping a MIDI note number to a MTNoteInfo object. If there is no MTNoteInfo for particular note number, then the GUI should be empty for that number.

MTNoteInfo -- the model for each GUI slot. Contains a link to the sequencing process, the MTNoteInfo's state (ready, playing or idle), the note number, and a link to the MT object who owns it.

MTGui -- maintains all the views.

On top of that, we have a couple of levels of object dependencies: BP (sequencing process or "bound process") has a dependent MTNoteInfo. MTNoteInfo doesn't have "dependents" using sc's dependency protocol, but it knows who the owning MT is, which is a one-to-one dependency. The MT then has a dependent, MTGui.

BP --> MTNoteInfo --owner variable--> MT --> MTGui --> gui widget

That covers all the cases I needed:

• If something happens to the sequencing process that will affect the MTGui, the process broadcasts a message to its dependents "I started playing" or "I stopped." The MTNoteInfo recognizes the message and tells the MT to notify its dependents that this noteinfo object changed. The notification goes to the MTGui, which updates the graphic widget onscreen.
• If you trigger the process using the MIDI keyboard, the MIDI responder passes the message to the MT object which looks up the MTNoteInfo in the dictionary. It starts or stops the process (BP), then notifies the MTGui that this slot changed. Then the update happens as above.
• If you trigger the process by clicking the GUI, the GUI knows which slot was clicked so it can tell the MT object which MTNoteInfo to trigger. Then the GUI state update cascades back through the dependencies as noted.

These execution flows also go for changing the MTNoteInfo in a slot, which is how reassignment takes place. The code releases the MTNoteInfo that was in the slot, then creates a new one. Then it passes the GUI updates along. It's all seamless to the user.

What I'm really trying to get across is the idea that the basic level of GUI writing:

Code:

slider = SCSlider(window, Rect(...)).action_({ |view| someObject.value = view.value });

... is easy to understand but is inherently limiting. If slider and someObject are local variables, then the GUI relationship is unidirectional: moving the slider can change something's state (gui --> model), but if that thing's state changes independently of the GUI, you might not have programmatic access to update the GUI (model --/////--> GUI, no dice).

Part of the trick to programming in SuperCollider (or any object-oriented language) is to have a clear idea of what you want the interface to do, and then design objects that support the requirements. In my case, I wanted multiple entry points: programmatic manipulation (writing code against the objects), MIDI manipulation (which is a special case of programmatic manipulation), and GUI manipulation. By breaking up the functions into objects that represent the behaviors, I'm able to represent the behaviors in terms of relationships and have code that is easier to add functionality to later. For instance, I need to add another state for processes, and I have a very clearly defined place to put those changes.

More later. I'll highlight the clickable drag object trick when I post some code.

Side note: I'm not calling it "model-view-controller" (MVC, one of the classic OOP design patterns) because I never could figure out what the controller does   -- models and views seem to do the job well enough for me.

hjh

[dewdrop_world]

Oh, forgot to mention... PLEASE ASK QUESTIONS! It's the only way I can refine this into a document for my website that will be comprehensible to other people 

hjh

[vesaflex]

Hi James,
Thanks for posting this.
I guess what your saying here is with your model, you could perhaps have a routine controling a synth, as well as a slider?
Could you post some basic code to show how this might occur?

[dewdrop_world]

OK, here's the code from the classes, lightly annotated. Reviewing the code I can see a couple of loose ends but I won't call attention to them

Code:

// Midi Trigger -- corresponds to a socket in the MIDI hierarchy
// contains a dictionary of note nums -> processes
MT : AbstractChuckNewDict {
classvar <>default; // used in ChuckBrowserKeyController
classvar <>readyThreshold = 5; // how long to hold a process in ready state before clearing
classvar <>defaultMinNote = 48, <>defaultMaxNote = 72;  // integer note numbers
var <lastBP, noteAllocator;

var <socket, <>minNote, <>maxNote; // socket is midi responder, calls my noteOn method

// specific to the chucklib framework, not really relevant to the topic
//
// all indices have to be converted to channel objects
*prNew { |index|
var temp;
this.put(index = index.asChannelIndex, temp = super.prNew(index));
^temp
}

*new { |index|
var collTemp;
^collection[this.name][index = index.asChannelIndex] ?? { this.prNew(index) }
}

// needed b/c MIDIChannelIndices can be == but not ===
*collectionType { ^Dictionary }
//
//////

init {
value = IdentityDictionary.new; // note num -> MTNoteInfo
minNote = defaultMinNote;
maxNote = defaultMaxNote;
socket = MTSocket(collIndex, this);
noteAllocator = ContiguousBlockAllocator(maxNote+1, minNote);
default = this;
}

free {
this.changed(\free);
socket.free;
value.do(_.free);
this.class.collection.removeAt(collIndex);
collIndex = value = socket = noteAllocator = minNote = maxNote = nil;
this.releaseDependants; // remove from dependants dictionary
this.removeFromCollection;
}

// chucklib methods to populate slots
bindPR { |pr, adverb|
BP(pr.collIndex).free;
this.add(pr => BP(pr.collIndex), adverb);
}

bindBP { |bp, adverb|
this.add(bp, adverb);
}

bindFact { |fact, adverb|
(fact.isBP).if({
this.add(fact.makev, adverb)
}, {
"%'s subtype is %; must be 'bp' to chuck into MT.".format(fact, fact.v[\type]).warn;
});
}

// support for BP([\pr1, \pr2, \pr3, \pr4]) => MT(0) syntax
// you give up control over where they go
bindArray { |ar|
ar.do({ |bp| bp => this });
}

// this does the work of filling a slot
add { |bp, adverb, updateGUI = true|
var new, nextAddNote;
lastBP = bp;
nextAddNote = this.convertAdverb(adverb); // get the number to use
this.adverbIsValidNote(adverb).if({
noteAllocator.reserve(adverb);
});
value[nextAddNote].notNil.if({ value[nextAddNote].free });
// create the MTNoteInfo object and add to the collection
value.put(nextAddNote, new = MTNoteInfo(bp, false, nextAddNote, this));
// .changed calls .update on all my dependents -- see MTGui.update
updateGUI.if({ this.changed(new) }); // tell the gui
}

// release a slot
removeAt { |notenum|
noteAllocator.free(notenum);
this.changed(notenum); // tells the dependents this slot is empty
^value.removeAt(notenum).free;
}

adverbIsValidNote { |adverb|
(adverb = adverb.tryPerform(\asInteger)).notNil
and: { adverb.inclusivelyBetween(minNote, maxNote) }
}

convertAdverb { |adverb|
(this.adverbIsValidNote(adverb)).if({
^adverb.asInteger
}, {
^noteAllocator.alloc(1)
});
}

// handle an incoming midi message
// .ready is an insurance policy -- you have to hit the midi key twice within 5 secs
// to get the action -- it protects against accidentally starting or stopping the wrong process
noteOn { |num|
var entry, clock;
// entry must exist
(entry = value[num]).notNil.if({
// if ready to fire, do play
entry.ready.if({
entry.ready = false;
entry.bp.isPlaying.if({
entry.bp.stop;
}, {
entry.bp.play;
});
this.changed(entry);
}, { // else make ready and schedule check for non-ready
entry.ready = true;
AppClock.sched(readyThreshold, {
entry.ready = false;
this.changed(entry);
});
this.changed(entry);
});
});
}

guiClass { ^MTGui }
}

MTNoteInfo {
var <>bp, <>ready, <>noteNum, <owner;

*new { |bp, ready, noteNum, owner|
var new;
new = super.newCopyArgs(bp, ready, noteNum, owner);
// part of the dependency chain: the BP object can notify any MTNoteInfo objects
// corresponding to it using .changed
bp.addDependant(new);
^new
}

asString { ^(noteNum.asMIDINote ++ ": " ++ bp.collIndex) }

// MTGui uses this to determine the color of the slot
playState {
^case { ready == true } { \ready } // ready takes precedence
{ bp.isDriven } { \driven }
{ bp.isPlaying } { \playing }
{ \idle }
}

free { bp.removeDependant(this) }

// respond to notifications from BP
update { |obj, changer|
(changer == \free).if({
owner.removeAt(noteNum);
});
#[\play, \stop, \driven].includes(changer).if({  // ignore other messages
owner.changed(this); // tell the owner I changed--owner's dependent is the MTGui
});
}
}


MTGui : HJHObjectGui {
classvar <>dragWidth = 80, <>dragHeight = 15, <>rows = 12,
<>gap = 2;
classvar <>font, <backColors, blackKeys;

var <mainView,
<chanText, // which channel?
<dragViews, // one per key
minNoteFloor, maxNoteCeil; // to position views correctly

*initClass {
font = Font("Helvetica", 10); // nice and small
backColors = (
ready: [Color.new255(181, 196, 255), Color.new255(136, 147, 191)],  // baby blue
playing: [Color.new255(69, 255, 23), Color.new255(52, 191, 17)],  // green
late: [Color.yellow, Color.yellow(0.75)], // not currently used
driven: [Color.new255(214, 191, 255), Color.new255(161, 143, 191)],  // lavender
idle: [Color.gray(0.75), Color.gray(0.5)]
);
// so that c#, d# etc. are shaded slightly darker: see populateView
blackKeys = #[1, 3, 6, 8, 10];
}

// this initializes the views
guiBody { |lay|
var size, cols, myRows, width, height, temp;
layout = lay;

// compute bounds
minNoteFloor = model.minNote.trunc(rows);
((maxNoteCeil = model.maxNote.trunc(rows)) != model.maxNote).if({
maxNoteCeil = maxNoteCeil + rows;
});
size = maxNoteCeil - minNoteFloor + 1;
myRows = rows;
cols = (size / rows).floor; // one column per octave
// octave + 1 should not create a new column, but it needs an extra row
// if > octave+1, need an extra column
temp = (size - (cols*rows + 1)).sign;
// case here ignores temp == -1
case { temp == 0 } { myRows = myRows + 1 }
{ temp == 1 } { cols = cols + 1 };
width = dragWidth * cols + gap + (gap*cols);
// include an extra row for the header
height = dragHeight * (myRows+1) + (gap*2) + (gap*myRows);

mainView.isNil.if({ // if this gui is already open, don't recreate
{ mainView = SCCompositeView(layout, argBounds ?? { Rect(0, 0, width, height) })
.onClose_({ this.remove });

chanText = SCStaticText(mainView,
Rect(gap+mainView.bounds.left, gap+mainView.bounds.top, width-gap, dragHeight))
.string_("MT(" ++ model.collIndex.asShortString ++ ")")  // midi channel index
.font_(font).align_(\center);

// like mixingboard -- create them and index them, then place them
dragViews = Array.fill(model.maxNote - model.minNote + 1, { |i|
this.emptyView(this.makeDragView, i);
});

this.positionViews(myRows, cols)
.populateViews;
}.defer;
});
}

// create the slot's view onscreen -- SCDragBoth allows dragging both in and out
makeDragView {
^SCDragBoth(mainView, Rect(0, 0, dragWidth, dragHeight))
.font_(font)
.beginDragAction_({ |drag| drag.object })
.action_({ |sink|
var index;
// dragging an object from a dragboth into itself is like clicking on it
// this should trigger the model
index = dragViews.indexOf(sink) + model.minNote;

Here's the trick for clickable drag sinks. Because the model holds the object to which the slot refers, I can check for identity:
model.v[index].bp === sink.object -- if you click on the dragboth without dragging, this condition will be true!

Code:

(model.v[index].notNil and: { model.v[index].bp === sink.object }).if({
model.v[index].ready = true;
model.noteOn(index);  // trigger
}, {
// else reassign

I use polymorphism to handle dragging a new object in. Instead of doing complex and ugly type checking here, I see if the object understands how to get dragged into the gui. If it's an invalid object, tryPerform does nothing (correctly).

Code:

(sink.object.tryPerform(\draggedIntoMTGui, this, index) != true).if({
// if inappropriate class, reset view
model.v[index].isNil.if({
this.emptyView(sink, index - model.minNote);
}, {
this.populateView(model.v[index])
});
});
});
})
}

   
Sometimes it's easier to separate the logic for creating views from that for positioning them. I create the views with an origin of (0, 0), then use Rect-moveTo to apply the right origin later.

Code:

positionViews { |r, c|
var x, y, i, j, top, left;
top = mainView.bounds.top;
left = mainView.bounds.left;
i = 0;
j = model.minNote - minNoteFloor;
x = gap;
// if minNote is not a multiple of 12, first y position needs to be adjusted
y = gap*2 + dragHeight + ((dragHeight + gap) * j);
{
dragViews.do({ |view|
view.bounds_(view.bounds.moveTo(x + left, y + top));
((y = y + dragHeight + gap; j = j+1) >= rows).if({
((i = i+1) < c).if({
// advance to next col, if we haven't hit the end of the cols
// otherwise, leave position as is (oct+1)
y = gap*2 + dragHeight; x = x + dragWidth + gap; j = 0;
});
});
});
}.defer;
}

// methods to update views
populateViews {
model.value.keysValuesDo({ |k, v|
this.populateView(v);
});
}

populateView { |entry|
{ dragViews[entry.noteNum - model.minNote].object_(entry.bp)
.string_("  " ++ entry.asString)
.background_(backColors[entry.playState]
[blackKeys.includes(entry.noteNum % 12).binaryValue])
}.defer;
}

emptyView { |view, index|
^view.string_("  " ++ (index + model.minNote).asMIDINote)
.background_(backColors[\idle]
[blackKeys.includes(index % 12).binaryValue])
}

// handles changes from the model (the MT object)
update { |mt, mtEntry|
case { mtEntry == \free } { this.remove; }
{ mtEntry.isNumber }
{ this.emptyView(dragViews[mtEntry-model.minNote], mtEntry-model.minNote) }
{ this.populateView(mtEntry); };
}

writeName {} // from cruxxial: don't want it, don't need it

// drop all the views on close
remove {
mainView.notNil.if({
mainView.notClosed.if({ // check for window closed
{ mainView.remove; }.defer;
});
mainView = chanText = dragViews = nil;
model.removeDependant(this);
});
}
}

Vesaflex, I'll answer your question tomorrow...
hjh

[dewdrop_world]

Hi James,
Thanks for posting this.
I guess what your saying here is with your model, you could perhaps have a routine controling a synth, as well as a slider?
Could you post some basic code to show how this might occur?

Not sure I follow you here. In model-view-controller GUI design, the model represents what you're working with, the view presents the model to the user on screen, and the controller manages the views. So you can always have a routine controlling a synth, but that isn't model-view-controller.

For what you're thinking, how does the routine relate to the GUI?

... thinking further ... maybe you mean a situation where a routine is controlling a synth and the routine's activities are also reflected in the slider. In that case, the model would communicate with the synth and have GUIs as dependents. The routine would update the model. In response, the model would send the update to the synth on the server and also to its dependents.

A hacked up version of this design might look like this. Note that the model has no variable referring to the gui. The model handles it by creating an Updater, which sets itself as a dependent of the model.

Code:

s.boot;

(
~model = (synthnode: { |freq = 440| SinOsc.ar(freq, 0, 0.2) ! 2 }.play,
freq: 440,
updateFreq: { |self, newFreq|
self.use({
~freq = newFreq;
~synthnode.set(\freq, ~freq);
});
self.changed;
}
);

~gui = (
prepare: { |self, model|
self.use({
~spec = \freq.asSpec;
~model = model;
// Updater makes a basic dependent of an object
// which runs a function on .update
// this is for the model -> view communication
~updater = Updater(model, { |changer|
{ self[\slider].value_(self[\spec].unmap(changer.freq)) }.defer;
});
~flowview = FlowView.new;
~slider = SCSlider(~flowview, 100@20)
.value_(self[\spec].unmap(model.freq))
// this is the view -> model communication
.action_({ |sl| self[\model].updateFreq(self[\spec].map(sl.value)) });
});
}
);

~gui.prepare(~model);
)

// move the slider to see that the gui is connected

// run the routine -- the model controls the gui via self.changed;
// you could create multiple guis from the same model and they would all change
// without changing the model's code

(
r = Routine({
loop {
~model.updateFreq(rrand(200, 1000));
0.125.wait;
}
}).play;
)

r.stop;

// when done: drop the synth and break the dependency

~model.synthnode.free; ~model.releaseDependants;

... the point being that programmatic updates should happen to the MODEL and they should bubble down to the views.

Damn. I wish the guis in my library were better designed like this but I didn't understand this so well when I was first writing it.

hjh

[vesaflex]

Hi James,
Yes, thats precisely what I meant! Although in my head, I envisioned the slider controlling the actual parameters in the routine:

like you would have a Pseq or whatever:
Pseq(#[1,2,3,4], inf);

and move the slider up, would make the sequence now:
Pseq(#[3,4,5,6], inf);

I'm sure it's been discussed on the list before, I just have to search through the years of useful archive! 
Anyway, thanks again for your great work and dedication to learning!!

/vx..

[vesaflex]

oh, I meant to ask as well, what is this syntax?

Code:

~gui = (
prepare: { |self, model|
self.use({
~spec = \freq.asSpec;
~model = model;
// Updater makes a basic dependent of an object
// which runs a function on .update
// this is for the model -> view communication
~updater = Updater(model, { |changer|
{ self[\slider].value_(self[\spec].unmap(changer.freq)) }.defer;
});
~flowview = FlowView.new;
~slider = SCSlider(~flowview, 100@20)
.value_(self[\spec].unmap(model.freq))
// this is the view -> model communication
.action_({ |sl| self[\model].updateFreq(self[\spec].map(sl.value)) });
});
}
);

like the prepare: etc. It would seem from the outset that it is somesort of dynamic class creation?  (because it seems you have called a prepare() method?). I admit this syntax looks alot cleaner than the already established class structure.

/vx..

[dewdrop_world]

oh, I meant to ask as well, what is this syntax?

http://www.create.ucsb.edu/pipermail/sc-users/2003-December/007200.html

like the prepare: etc. It would seem from the outset that it is somesort of dynamic class creation?  (because it seems you have called a prepare() method?). I admit this syntax looks alot cleaner than the already established class structure.

Actually, syntactically it's messier. "Methods" have to have a "self" argument first in the arg list. To access "instance variables" or "methods" inside the ad hoc object, you have to use "self.use({  })" or write the variables as self[\varname].

Further, any methods (real ones) implemented by Object, Collection, Set, Dictionary, IdentityDictionary, and Environment become reserved method names. You could not include an asString method inside the environment -- well, you could include it, but it will never get called because .asString will go to Object-asString instead. That can be pretty inconvenient sometimes.

For my own work, I devised AdhocClass which works similarly but is a little easier, I think.

Despite the hassles, it has the advantage of allowing dynamic prototyping of new code. At this point, I put core, support code into true classes, but anything that is specific to a given composition, or that works in a specific way with musical material, goes into AdhocClass prototypes. That's so I can test and change things without having to recompile the library.

like you would have a Pseq or whatever:
Pseq(#[1,2,3,4], inf);

and move the slider up, would make the sequence now:
Pseq(#[3,4,5,6], inf);

Boy, I really do need to document chucklib! It was designed exactly for this, among other things.

The short version is that you want something like:

Pseq(#[1,2,3,4], inf) + Pfunc({ sliderModel.value })

... chucklib lets you package everything into self-contained units. You can get by without it, but I find I can write more quickly and reliably when I have meaningful places to put things.

Note that you shouldn't try to get the slider value directly from the GUI inside the Pfunc -- the old routine/GUI problem again. That's another reason why it's better to have a model that the GUI updates.

hjh