This is just a general question, so if you don't have time to respond, I totally understand as it's not really Open Stage Control related. Basically, I was wondering from a threading perspective how you implement your OSC (protocol) server. I've implemented an OSC server/client in Ableton using Python, but it's much slower than MIDI for things like xy pads and what not, but I think it's due to my implementation.
What i've done is the following: a single worker thread is dedicated to receiving the OSC messages, and then a timer object on the main thread checks a message queue every 5 ms and does the work. I can't put the socket listener on the main thread, because it blocks operation of Ableton. This works just fine unless I'm sending a bunch of messages, and then the queue just gets totally bogged down.
One thought is that I could create a thread pool for the incoming messages, but then i'm still limited by the bottleneck of that main thread timer. I couldn't think of anyone else to ask about this.... 
I'd be happy to pay you for a phone consultation or something of the sort!
Hey,
Parsing 1 message every 5ms is definitely not enough, python can go much faster ! Usually in python I parse all the messages in the queue every 1ms and things run fine:
import time
while true:
# main loop
while message_in_queue():
parse_message():
time.sleep(0.001)
Just lowered to 1ms and it's definitely better, but still pretty sluggish with xy pads. Would you be open to taking a peek at my python code? I know it's totally out of the scope of this project but I've been at it for days and don't really know who else to ask.
Code here:
https://bitbucket.org/datalooper/liveoscbridge/src/master/
Ok first of all I did a quick test and the python osc lib you're using is perfectly capable of parsing thousands of messages per second. Now I think you could simplify the LiveOSCBridge class a lot, using a python queue here doesn't make sense, there's already a queue handled by the UDP backend; also the osc helper doesn't really help, let's remove it; and finally, why not using a simple threaded loop to read incoming messages instead of relying on live's timer ?
import Live
from _Framework.ControlSurface import ControlSurface
from .constants import *
from .ActionHandler import ActionHandler
from .OSC3 import OSCServer
from threading import Thread
from time import sleep
class LiveOSCBridge(ControlSurface):
def __init__(self, c_instance):
super(LiveOSCBridge, self).__init__(c_instance)
self.__c_instance = c_instance
with self.component_guard():
# use OSCServer class directly
self.server = OSCServer(('127.0.0.1', 9001))
# bind callback
self.server.addMsgHandler('default', self._osc_hande_messages)
self.__action_handler = ActionHandler(self, self.song())
# only define that once, not at each incoming message
# should probably be defined in ActionHandler.py instead
# to keep thing clean
self.action_map = {
'/live/metronome' : 'metronome',
'/live/tempo' : 'change_tempo',
}
# threaded loop to handle incoming messages
self.running = True
self.thread = Thread(target=self._osc_main_loop)
self.thread.start()
def connect(self):
self.log_message("LiveOSCBridge Connected")
def disconnect(self):
self.running = False
def _osc_main_loop(self):
# run until disconnect()
while self.running:
self.server.handle_request()
sleep(0.001)
def _osc_hande_messages(self, addr, tags, data, source):
# no queue, just dispatch the message to the appropriate callback
if self.action_map[addr]:
getattr(self.__action_handler, self.action_map[addr])(tags, data, source)
I could not test it so there may be some errors but it may help.
Hmm I implemented this (with a few minor changes to close the server on disconnect), and it's actually significantly slower than the other architecture.
As an experiment, I tried reducing the work done to just moving the master fader based on the value, eliminating any lookups, and it's still slow as hell...like turtle speed when implemented this way.
I then put back the Ableton timer object (using the updated architecture), and that got me back to where my speed was beforehand, which tells me that the issue is in how Ableton's canned implementation of Python prioritizes threads. So...I hammered my head against the wall and tried a bunch of things and I do believe I have a working thing.
import Live
from _Framework.ControlSurface import ControlSurface
from .ActionHandler import ActionHandler
from .OSC3 import OSCServer
import threading
class LiveOSCBridge(ControlSurface):
def __init__(self, c_instance):
ControlSurface.__init__(self, c_instance)
with self.component_guard():
# use OSCServer class directly
self.server = -1
# bind callback
self.message_timer = Live.Base.Timer(callback=self._osc_main_loop, interval=1, repeat=True)
self.queue = []
self.start_server()
self.__action_handler = ActionHandler(self, self.song())
def connect(self):
self.log_message("LiveOSCBridge Connected")
def start_server(self):
if self.server == -1:
self.log_message("server started")
addr = ('127.0.0.1', 9001)
self.server = OSCServer(addr)
self.server.addMsgHandler('default', self._osc_handle_messages)
self.server.socket.setblocking(0)
self.message_timer.start()
def disconnect(self):
self.message_timer.stop()
if self.server != -1:
self.server.close()
def _osc_main_loop(self):
self.server._handle_request_noblock()
def _osc_handle_messages(self, addr, tags, data, source):
if self.__action_handler.action_map[addr]:
getattr(self.__action_handler, self.__action_handler.action_map[addr])(tags, data,
source)