Hi All,
I spent a lot of time last year working out (with the help of @Sylvain) on how to use Mackie protocol to update expression maps in OSC and use the custom module to keep them up to date automatically. To be clear, this is without the need to add plugins to your tracks or midisends, and can be used on any type of cubase track as long as you follow some simple track naming rules and keep your track names limited to below 29 characters.
Some Extracts from My OSC Panel
I'm still getting a lot of questions offline that ask me how I've set all this up in my cubase template so I thought I'd write a quick tutorial...
So here goes:
-
I assume you’re using Cubase.
-
Set up some expression maps? You need to make sure that they use program change to work,
Not note key switches - that way you can standardise OSC output messages and do any the translation in the map in cubase.
-
OSC Buttons widgets need to be set to "tap".
-
You use my method you’ll need to have OSC set up to receive Mackie Control messages and also sysex, so make sure your midi settings are as follows:
-
Create 2 new loopMIDI ports, one to receive the articulation control and one to send and receive the mackie sysex.
-
install MS Visual studio onto your machine so that you can write the custom module easily
-
Install node.js and npm Here's how to do it
-
When you have installed these basic bits you also need to install some additional nodes so that you can read the xml and watch for changes to expression maps in your custom module.
You will needxml2jsandchokidar
To do this you will need to put these modules into your custom modules directory so you need to create a directory
C:\Users\YourUserName\Open Stage Control\Custom Modules -
Then open a command prompt from that directory and type
npm install xml2js
more info here: xml2js node -
do the same with chokidar
npm install chokidar -
So now it's time to create a custom module.
First create a blank .js document and name it something useful likebasicCM.jsand add the following into it:
When you have got that working amend your custom module like this.
When you have got that working amend your custom module like this.
JavaScript:
First Custom Module for testing
//set a variable for the midi port of your generic remote
var genericRemoteMIDIport = 'OSCMIDICONTROL'; //BASE MIDI PORT FOR showing all tracks in cubase
var midiPort_MCU_From_OSC = 'MCU_From_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
var midiPort_MCU_To_OSC = 'MCU_To_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
//MIDI PORTS FOR FUNCTIONS FROM OSC
send('midi', midiPort_MCU_From_OSC, '/note', 1, 44, 127); //Send MCU command to swicth to track name sending
module.exports = {
oscInFilter: function (data) { // this filters data from Cubase or midi device into OSC
console.log('MIDI message recieved into OSC from Cubase')
var { address, args, host, port } = data
mcuToOsc(host, port, address, args)
return { address, args, host, port }
},
oscOutFilter: function (data) { // Filter incomming osc messages from OSC and gets them out to Cubase
var { address, args, host, port } = data
if (address == '/testAddress') {
console.log('You have sent a message to custom module from OSC')
}
return data //End of OSC out Filter here
}
}
//These are your functions that do the work in JS
function mcuToOsc(host, port, address, args) {
console.log('fired')
if (host !== 'midi' || port !== midiPort_MCU_To_OSC) return
// SYSEX
if (address === '/sysex') {
console.log('You have successfully sent sysex into your custom module')
}
}
-
Now create a blank OSC canvas and create a button. You should set its property to tap.
This is just a test at the moment to check everything is working as it should. -
Go into the address and change it from auto to /testAddress then save the panel.
-
Then open cubase and set up a couple of tracks with unique names.
-
In the cubase MCU settings you should set those to your MCU virtual ports.
You don’t need to connect you cubase tracks to anything other than an instrument or vsti etc.
-
Stop the OSC server and start it again.
-
Now you should get messages in your console when you either tap the button or change tracks in cubase
-
Now you have to get OSC listeners into your button that you've created so set this up like this and amend the target to the other virtual port you have created (not the MCU ones):
-
Just as a check, make sure your Cubase midi device panel isn't set with the MCU ports to “all midi”. The check box to the right of “active” should be unchecked.
-
Add a text widget that will display the track name so we can test functionality so do that now with the following properties:
-
When this is all working you now need to grab just the sysex that is the track name from MCU so you can then decide what labels to use. But there are loads of bits sent with the track name that mean that you need to strip out and all sorts so I'm going to first demo how to amend the labels without the complexity of different labels for articulations and then after that will do the complicated stuff. We will first just add a trackname to your OSC panel.
-
So change your custom module to the following - I have tried to add commentary to the javascript to try to allow you to follow it:
Update your Custom Module
//define the variables for the track name function
var trackName = ""
var pos = 72
var rootPosLCD = 72
var keepStringVal = 0
var trackNameJoined = ""
var bufferTrackName = ""
var posBuffer = 72
//set a variable for the midi port of your generic remote
var genericRemoteMIDIport = 'OSCMIDICONTROL'; //BASE MIDI PORT FOR showing all tracks in cubase
var midiPort_MCU_From_OSC = 'MCU_From_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
var midiPort_MCU_To_OSC = 'MCU_To_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
//MIDI PORTS FOR FUNCTIONS FROM OSC
send('midi', midiPort_MCU_From_OSC, '/note', 1, 44, 127); //Send MCU command to swicth to track name sending
module.exports = {
oscInFilter: function (data) { // this filters data from Cubase or midi device into OSC
console.log('MIDI message recieved into OSC from Cubase')
var { address, args, host, port } = data
mcuToOsc(host, port, address, args)
return { address, args, host, port }
},
oscOutFilter: function (data) { // Filter incomming osc messages from OSC and gets them out to Cubase
var { address, args, host, port } = data
if (address == '/testAddress') {
console.log('You have sent a message to custom module from OSC')
}
return data //End of OSC out Filter here
}
}
//These are your functions that do the work in JS
function mcuToOsc(host, port, address, args) {
console.log('fired')
if (host !== 'midi' || port !== midiPort_MCU_To_OSC) return
//added this to fix error
var inArgs = args.map(x=>x.value),
outArgs = [],
action = ''
// SYSEX
if (address === '/sysex') {
var [value] = inArgs
if (value.includes("f0 00 00 66 14 12")) { // mackie lcd text < this is the sysex identifier for the track name
console.log(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>sysex value recieved")
let sysExVal = args[0].value
//Use the function getTrackName to do all the cool stuff to get the fullTrackName
let fullTrackName = getTrackName(sysExVal)
console.log("You have Grabbed the track Name using the function getTrackName = " + fullTrackName)
//now you need to Send the Full Track Name to OSC Panel using another function
sendTrackName(fullTrackName)
}
}
}
function getTrackName(sysExVal) {
var nameDone = false
var d = sysExVal.split(" ").slice(6).map(x => parseInt(x, 16)) //this devides up using .split, then slices off the front 6 elements, and x => parseInt(x, 16) converts the rest from Hex to Int then .map creates the array d[]
pos = d[0] // first byte -> position // hex to int
console.log("position = " + pos)
//console.log("sysExVal = " + sysExVal)
text = d.slice(1).map(x => String.fromCharCode(x))// these are the new characters which are updated on the console, the rest -> updated characters
if (pos < 29) {
return trackNameJoined
}
text.pop() // drop sysex closing byte
trackName = text.join('')
//MCU only sends what it needs so you need to buffer the previous name and use parts of that
//Check the length of the new name vs the buffer
nameLengthCheck = bufferTrackName.length - trackName.length
//MCU sends some sysex data to tell the screen where to start showing the new characters
//This is related to the root position on the screen which is 72
//Check if root position matches the position where the characters are to be placed
charFromStart = pos - rootPosLCD
var lengthCheck = charFromStart + trackName.length
if (lengthCheck < 29) {
let newEndLength = 29 - charFromStart - trackName.length
newEnd = bufferTrackName.substring(bufferTrackName.length - newEndLength)
} else { newEnd = "" }
if (pos == 72) { //Full length name recieved
trackNameJoined = trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
// console.log('TrackName Joined = ' + trackNameJoined)
nameDone = true
} else if (pos > posBuffer && posBuffer == 72 && nameDone == false) {
keepStringVal = pos - posBuffer //new name follows a full string text
var prevTrackKeep = bufferTrackName.substring(0, keepStringVal)
trackNameJoined = prevTrackKeep + trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
nameDone = true
} else {
keepStringVal = pos - rootPosLCD //new name follows a full string text
var prevTrackKeep = bufferTrackName.substring(0, keepStringVal)
trackNameJoined = prevTrackKeep + trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
nameDone = true
}
//MCU will sometimes send the characters (MIDI) as part of the track name
//so you need to strip these out
const findMidiTag = '(';
var posMidiTag = trackNameJoined.search("\\(M");
var posBrackTag = trackNameJoined.search("\\(");
var trackNameJoinedTrim = ""
if (posBrackTag == trackNameJoined.length - 1) {
trackNameJoinedTrim = trackNameJoined.substring(0, (posBrackTag - 1))
trackNameJoined = trackNameJoinedTrim
}
if (posMidiTag > -1) {
trackNameJoinedTrim = trackNameJoined.substring(0, (posMidiTag - 1))
trackNameJoined = trackNameJoinedTrim
}
//trim off all the spaces at the end
trackNameJoined = trackNameJoined.trimEnd();
return trackNameJoined
}
function sendTrackName(fullTrackName) {
//set the address in OSC of the full Name Label
var trackNameLabel = { 'trackName': '/trackName/Label' }
//console.log("full trackName = " + fullTrackName )
receiveOsc({
address: Object.values(trackNameLabel), //this populates the labels on the buttons based on the array
args: [
{ type: 's', value: fullTrackName }
]
})
return
}
-
So now this is the final step to get the articulations showing. You need to make sure you have set up everything correctly in your OSC map and directory structure. I have assumed you have 64 button widgets to show the articulations - if you have a different number there is a note in the code above where to change this.
-
You also have to have all your expression maps named with the track tag that reflects the same tagging system you have used in your cubase template - the tag should be in the format [xxxxxx] where x are the characters. For example:
Now change your custom module to this, remembering to update the path in the script to where you keep your xml expression maps.
Final Custom Module
//define the variables for the track name function
var trackName = ""
var pos = 72
var rootPosLCD = 72
var keepStringVal = 0
var trackNameJoined = ""
var bufferTrackName = ""
var posBuffer = 72
const fs = nativeRequire('fs');
const xml2js = nativeRequire('xml2js');
const glob = nativeRequire('glob');
//set a variable for the midi port of your generic remote
var genericRemoteMIDIport = 'OSCMIDICONTROL'; //BASE MIDI PORT FOR showing all tracks in cubase
var midiPort_MCU_From_OSC = 'MCU_From_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
var midiPort_MCU_To_OSC = 'MCU_To_OSC' //this is the MCU port you've set up in cubase to send and recieve MCU messages
//Create variables for each of the OSC watchers in the articulation buttons you have created on your template
//Set the path to your expression Maps
var mapFiles = glob.sync('E:/01 - Composing Data Drive/01 - Music Files/01 - Patches and Maps/01 - Cubase Mapings/01 - Expression Maps/Rebuilt Set 12-11-21/final patch numbers/*.expressionmap')
//Set the address of the library buttons
var artButtons = {
'b1': '/b1/show', 'b2': '/b2/show', 'b3': '/b3/show', 'b4': '/b4/show', 'b5': '/b5/show',
'b6': '/b6/show', 'b7': '/b7/show', 'b8': '/b8/show', 'b9': '/b9/show', 'b10': '/b10/show',
'b11': '/b11/show', 'b12': '/b12/show', 'b13': '/b13/show', 'b14': '/b14/show', 'b15': '/b15/show',
'b16': '/b16/show', 'b17': '/b17/show', 'b18': '/b18/show', 'b19': '/b19/show', 'b20': '/b20/show',
'b21': '/b21/show', 'b22': '/b22/show', 'b23': '/b23/show', 'b24': '/b24/show', 'b25': '/b25/show',
'b26': '/b26/show', 'b27': '/b27/show', 'b28': '/b28/show', 'b29': '/b29/show', 'b30': '/b30/show',
'b31': '/b31/show', 'b32': '/b32/show', 'b33': '/b33/show', 'b34': '/b34/show', 'b35': '/b35/show',
'b36': '/b36/show', 'b37': '/b37/show', 'b38': '/b38/show', 'b39': '/b39/show', 'b40': '/b40/show',
'b41': '/b41/show', 'b42': '/b42/show', 'b43': '/b43/show', 'b44': '/b44/show', 'b45': '/b45/show',
'b46': '/b46/show', 'b47': '/b47/show', 'b48': '/b48/show', 'b49': '/b49/show', 'b50': '/b50/show',
'b51': '/b51/show', 'b52': '/b52/show', 'b53': '/b53/show', 'b54': '/b54/show', 'b55': '/b55/show',
'b56': '/b56/show', 'b57': '/b57/show', 'b58': '/b58/show', 'b59': '/b59/show', 'b60': '/b60/show'
}
// These correspond to the label within OSC #{OSC{/b1/label, , false}}
// Set The variable when populated is the actual text on the button
var artLabels = {
'b1': '/b1/label', 'b2': '/b2/label', 'b3': '/b3/label', 'b4': '/b4/label', 'b5': '/b5/label',
'b6': '/b6/label', 'b7': '/b7/label', 'b8': '/b8/label', 'b9': '/b9/label', 'b10': '/b10/label',
'b11': '/b11/label', 'b12': '/b12/label', 'b13': '/b13/label', 'b14': '/b14/label', 'b15': '/b15/label',
'b16': '/b16/label', 'b17': '/b17/label', 'b18': '/b18/label', 'b19': '/b19/label', 'b20': '/b20/label',
'b21': '/b21/label', 'b22': '/b22/label', 'b23': '/b23/label', 'b24': '/b24/label', 'b25': '/b25/label',
'b26': '/b26/label', 'b27': '/b27/label', 'b28': '/b28/label', 'b29': '/b29/label', 'b30': '/b30/label',
'b31': '/b31/label', 'b32': '/b32/label', 'b33': '/b33/label', 'b34': '/b34/label', 'b35': '/b35/label',
'b36': '/b36/label', 'b37': '/b37/label', 'b38': '/b38/label', 'b39': '/b39/label', 'b40': '/b40/label',
'b41': '/b41/label', 'b42': '/b42/label', 'b43': '/b43/label', 'b44': '/b44/label', 'b45': '/b45/label',
'b46': '/b46/label', 'b47': '/b47/label', 'b48': '/b48/label', 'b49': '/b49/label', 'b50': '/b50/label',
'b51': '/b51/label', 'b52': '/b52/label', 'b53': '/b53/label', 'b54': '/b54/label', 'b55': '/b55/label',
'b56': '/b56/label', 'b57': '/b57/label', 'b58': '/b58/label', 'b59': '/b59/label', 'b60': '/b60/label',
}
send('midi', midiPort_MCU_From_OSC, '/note', 1, 44, 127); //Send MCU command to swicth to track name sending
module.exports = {
oscInFilter: function (data) { // this filters data from Cubase or midi device into OSC
console.log('message recieved into custom module')
var { address, args, host, port } = data
mcuToOsc(host, port, address, args)
return { address, args, host, port }
},
oscOutFilter: function (data) { // Filter incomming osc messages from OSC and gets them out to Cubase
var { address, args, host, port } = data
if (address == '/testAddress') {
console.log('You have sent a message to custom module from OSC')
}
if (address == '/resetMCU') {
console.log('MCU Reset message sent')
send('midi', midiPort_MCU_From_OSC, '/note', 1, 44, 127); //Send MCU command to swicth to track name sending
}
return data //End of OSC out Filter here
}
}
//These are your functions that do the work in JS
function mcuToOsc(host, port, address, args) {
if (host !== 'midi' || port !== midiPort_MCU_To_OSC) return
var inArgs = args.map(x => x.value),
outArgs = [],
action = ''
// SYSEX
if (address === '/sysex') {
var [value] = inArgs
if (value.includes("f0 00 00 66 14 12")) { // mackie lcd text < this is the sysex identifier for the track name
console.log(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>sysex value recieved")
let sysExVal = args[0].value
//Use the function getTrackName to do all the cool stuff to get the fullTrackName
let fullTrackName = getTrackName(sysExVal)
console.log("You have Grabbed the track Name using the function getTrackName = " + fullTrackName)
//now you need to Send the Full Track Name to OSC Panel using another function
sendTrackName(fullTrackName)
//Populate the Map using the below but make sure your track names follow the rules on expression map tagging
//limitatation - track tag has to be in the last 6 characters of the name and must be bounded by []
//Strip out the trackMapTag from the full name
//To avoid complications also strip off the square brackets after you've pulled them into the OSC world
trackMapTag = (fullTrackName.substring(fullTrackName.length - 6));
trackMapTag = trackMapTag.replace('[', '')
trackMapTag = trackMapTag.replace(']', '')
console.log("Track TAG = " + trackMapTag)
// Build the articulation map and populate the buttons on the expression map
buildMap(trackMapTag)
}
}
}
function getTrackName(sysExVal) {
var nameDone = false
var d = sysExVal.split(" ").slice(6).map(x => parseInt(x, 16)) //this devides up using .split, then slices off the front 6 elements, and x => parseInt(x, 16) converts the rest from Hex to Int then .map creates the array d[]
pos = d[0] // first byte -> position // hex to int
console.log("position = " + pos)
//console.log("sysExVal = " + sysExVal)
text = d.slice(1).map(x => String.fromCharCode(x))// these are the new characters which are updated on the console, the rest -> updated characters
if (pos < 29) {
return trackNameJoined
}
text.pop() // drop sysex closing byte
trackName = text.join('')
//MCU only sends what it needs so you need to buffer the previous name and use parts of that
//Check the length of the new name vs the buffer
nameLengthCheck = bufferTrackName.length - trackName.length
//MCU sends some sysex data to tell the screen where to start showing the new characters
//This is related to the root position on the screen which is 72
//Check if root position matches the position where the characters are to be placed
charFromStart = pos - rootPosLCD
var lengthCheck = charFromStart + trackName.length
if (lengthCheck < 29) {
let newEndLength = 29 - charFromStart - trackName.length
newEnd = bufferTrackName.substring(bufferTrackName.length - newEndLength)
} else { newEnd = "" }
if (pos == 72) { //Full length name recieved
trackNameJoined = trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
// console.log('TrackName Joined = ' + trackNameJoined)
nameDone = true
} else if (pos > posBuffer && posBuffer == 72 && nameDone == false) {
keepStringVal = pos - posBuffer //new name follows a full string text
var prevTrackKeep = bufferTrackName.substring(0, keepStringVal)
trackNameJoined = prevTrackKeep + trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
nameDone = true
} else {
keepStringVal = pos - rootPosLCD //new name follows a full string text
var prevTrackKeep = bufferTrackName.substring(0, keepStringVal)
trackNameJoined = prevTrackKeep + trackName + newEnd
bufferTrackName = trackNameJoined
posBuffer = pos
nameDone = true
}
//MCU will sometimes send the characters (MIDI) as part of the track name
//so you need to strip these out
const findMidiTag = '(';
var posMidiTag = trackNameJoined.search("\\(M");
var posBrackTag = trackNameJoined.search("\\(");
var trackNameJoinedTrim = ""
if (posBrackTag == trackNameJoined.length - 1) {
trackNameJoinedTrim = trackNameJoined.substring(0, (posBrackTag - 1))
trackNameJoined = trackNameJoinedTrim
}
if (posMidiTag > -1) {
trackNameJoinedTrim = trackNameJoined.substring(0, (posMidiTag - 1))
trackNameJoined = trackNameJoinedTrim
}
//trim off all the spaces at the end
trackNameJoined = trackNameJoined.trimEnd();
return trackNameJoined
}
function sendTrackName(fullTrackName) {
//set the address in OSC of the full Name Label
var trackNameLabel = { 'trackName': '/trackName/Label' }
console.log("full trackName = " + fullTrackName )
receiveOsc({
address: Object.values(trackNameLabel), //this populates the labels on the buttons based on the array
args: [
{ type: 's', value: fullTrackName }
]
})
return
}
// Get the articulations from the *.xml expression maps in cubase
async function buildMap(trackMapTag) {
const artArr = [];
const artColor = [];
for (const mapFile of mapFiles) {
let mapName = "Not Defined"
//console.log('Selected Map: ' + mapFile);
if (mapFile.includes(trackMapTag)) { //trackID comes from sysex track naming routine
const parser = new xml2js.Parser({
explicitArray: false,
mergeAttrs: true
}); // 'mergeAttrs: true' was essential
//console.log('Selected Map: ' + mapFile);
const data = await fs.promises.readFile(mapFile);
const result = await parser.parseStringPromise(data);
//let art = await result.InstrumentMap.member[1].list.obj;
//
mapName = result.InstrumentMap.string.value
console.log('>>>>>>>>>>>>>>>>>>>>>>>>>>>>Map Name = ' + mapName)
const art = result.InstrumentMap.member[1].list.obj;
// NOTE: this code assumes that this part only ever runs for
// one mapFile, that all the others don't match the trackID test
// because if you run this more than once, it will just
// overwrite the members of artArr and artColor
//Add label to Expression Map Label
addExpMapLabel(mapName)
for (let i = 0, len = art.length; i < len; i++) {
artArr[i] = art[i].member[1].string.value
artColor[i] = art[i].int.value
}
//Now call the function to label the map template you have created in OSC
addExpBtnLabels(artArr)
break;
}
}
return { artArr, artColor };
}
//Adds Label to the expression map buttons called from function labelMap - this is set up to have a grid of 64 buttons
function addExpBtnLabels(artArr) {
var artButtonGrid = artArr.length //number of buttons based on number of articulations in the map
//Populate the lables on the grid
for (i = 0; i < artButtonGrid; i++) {
var artText = artArr[i] //collects each label from the object labeltexts that has just been created at position [i]
receiveOsc({
address: Object.values(artButtons)[i], //this controls the visibility of the buttons
args: [
{ type: 'i', value: 1 }
]
})
receiveOsc({
address: Object.values(artLabels)[i], //this populates the labels on the buttons based on the array
args: [
{ type: 's', value: artText }
]
})
}
//Remove remaining buttons on articulation grid
for (i = artButtonGrid; i < 64; i++) { // amend the 64 to the number of buttons you have in total on your grid
receiveOsc({
address: Object.values(artButtons)[i], //continues through the remaining buttons and hides them
args: [
{ type: 'i', value: 0 }
]
})
}
}
And that's it. Should all work now for you...
