The Human Interface Device standard for USB devices is great. The obvious appeal of using them as controllers: there are a huge number of cheaply available keyboard, mice, trackpads, joysticks, jog wheels, etc, all of which can be understood without individual drivers. However using them as controllers is not straightforward because most audio software does not accept joystick input, and mice and keyboards just duplicate the normal mouse and keyboard.
In Linux, I wrote a GDAM input plugin for reading USB HID events. USB events show up on /dev/input/event* (if you have your kernel configured correctly, but these days it seems to be the default) The event interface is also great: you get 3 bytes, the first is event type (button vs axis vs relative etc) second is index (256 of each type max per device) 3rd is value. Implementing the OSX version involved many lines of dynamically allocating objects based on how big they were in the current version of OS.... linux was dead simple.
Recently revisiting some HID controllers, they were less convenient than MIDI. In particular, if I unplug the device /dev/input/eventN disappears and I get read errors. If I replug I have to reopen device. So I made a HID -> MIDI driver for linux. It hotplugs, and any USB HID device plugged in becomes a virtual ALSA MIDI port. So far I'm translating key and button presses, and absolute values (faders, analog joysticks) into MIDI notes and CC's. I haven't decided the best way to translate relative (mouse, touchpad) events into MIDI, there is no clear match. I refuse to convert to absolute by driving a virtual CC because that would be limiting. I may send note on with different velocities, or set CC value for each relative event but those are non-standard uses.
Now, using qjackctl's patchbay to automatically restore MIDI routing, I can replug my HID controllers at will without doing a thing in software. Instead of working through a plugin specific to my software, the MIDI can be used with any software or even sent over a MIDI cable to drive hardware.
I made a quick GDAM binding file to make the MIDI notes correspond to qwerty layout. My laptop keyboard may or may not be implemented via USB, but it does show up in /dev/input/event* so I can run my driver on it. Now all my typing is rendered rhythmically and melodically. I've enjoyed some of the riffs that emerged while I code, newlines and braces resolve nicely.
Wednesday, February 28, 2007
GDAM: scratching
Scratching in GDAM is based on the Bender plugin - an inline filter which keeps a large buffer of audio and allows playback at any speed, backward or forward, from any point in the buffer. If you are familiar with the Buffer Override VST filter, I am told it uses this technique.
Bender plugin can manage the input to the buffer in various ways; it can keep the input at full speed, advance only when needed, or even drop bars of music to keep playing in beat, without making progress through the song.
Scratching in GDAM allows the user to set certain cuepoints in the audio stream, and jump to them, while playing backward or forward at any pitch. The ability to teleport between point in the audio stream breaks a major limitation of vinyl scratching: to get from one part of a sample to another, the vinyl DJ has to move the needle along the groove. To get there fast, the record spins fast and the sample plays at a higher pitch. The scratch DJ may selectively mute parts of the scratch to obscure some of the "incidental" pitch, but there is noneless a very rigid relationship between where in the sample you can be, how fast you are moving through the sample and what the pitch is at any moment.
GDAM's ability to jump to marks within the sample allows different parts of a sample to be played back-to-back in a way that is impossible in vinyls. On top of Bender, it is even possible to switch seamlessly into timestretching and pitch shifting, breaking the speed/pitch relationship. In theory a bit like going from pushing a car on a flat track to flying uninhibited through 3d space.
Another technique is to take timestretching to the extreme that position does not progress forward or backward on its own. It plays a microloop of the current position, and you can drag the playhead back and forth while changing the pitch independently. You can do a record stall in place by fading pitch down to 0, scratch back and forth at any consistent pitch, and all kinds of other tricks impossible with vinyl.
For MIDI control for scratching I like to use the pitchbend wheel - pulling it towards me to rewing and pushing it away from me to play > 100% speed. A nice thing about the pitch bend wheel is that it returns to its detente position when you remove your hand, in the same way that a record returns to full turntable speed when you remove your hand from it.
So of course pitchbend 0 always mapped to 100% speed. But I have a number of different mappings for the rest of pitchbend range. The two most common are 1) pulling back from detent slows from 100% to stalled then increases to -200% or -400% at pitchwheel full back 2) pulling back from detente instantly trigger 0% and from there increases to -400%. 1) gives a snappier scratch, 2) is useful if you want to stall a record but GDAM also offers plugins which do this perfectly so it didn't get as much use.
Although I've often dropped a bit of scratching into sets and demos, I never really spent time developing routines and pushing the limits of what is possible.
Bender plugin can manage the input to the buffer in various ways; it can keep the input at full speed, advance only when needed, or even drop bars of music to keep playing in beat, without making progress through the song.
Scratching in GDAM allows the user to set certain cuepoints in the audio stream, and jump to them, while playing backward or forward at any pitch. The ability to teleport between point in the audio stream breaks a major limitation of vinyl scratching: to get from one part of a sample to another, the vinyl DJ has to move the needle along the groove. To get there fast, the record spins fast and the sample plays at a higher pitch. The scratch DJ may selectively mute parts of the scratch to obscure some of the "incidental" pitch, but there is noneless a very rigid relationship between where in the sample you can be, how fast you are moving through the sample and what the pitch is at any moment.
GDAM's ability to jump to marks within the sample allows different parts of a sample to be played back-to-back in a way that is impossible in vinyls. On top of Bender, it is even possible to switch seamlessly into timestretching and pitch shifting, breaking the speed/pitch relationship. In theory a bit like going from pushing a car on a flat track to flying uninhibited through 3d space.
Another technique is to take timestretching to the extreme that position does not progress forward or backward on its own. It plays a microloop of the current position, and you can drag the playhead back and forth while changing the pitch independently. You can do a record stall in place by fading pitch down to 0, scratch back and forth at any consistent pitch, and all kinds of other tricks impossible with vinyl.
For MIDI control for scratching I like to use the pitchbend wheel - pulling it towards me to rewing and pushing it away from me to play > 100% speed. A nice thing about the pitch bend wheel is that it returns to its detente position when you remove your hand, in the same way that a record returns to full turntable speed when you remove your hand from it.
So of course pitchbend 0 always mapped to 100% speed. But I have a number of different mappings for the rest of pitchbend range. The two most common are 1) pulling back from detent slows from 100% to stalled then increases to -200% or -400% at pitchwheel full back 2) pulling back from detente instantly trigger 0% and from there increases to -400%. 1) gives a snappier scratch, 2) is useful if you want to stall a record but GDAM also offers plugins which do this perfectly so it didn't get as much use.
Although I've often dropped a bit of scratching into sets and demos, I never really spent time developing routines and pushing the limits of what is possible.
STC-1000
Another promising device, a pressure-sensitive Kaoss Pad without the audio circuitry or flashing lights. Elegant design. Has MIDI but no USB - less convenient but it is guaranteed to be standard compliant (kinda...)
When I received the unit in the mail, I eagerly hooked it into GDAM, dumped the MIDI, and made a binding file linking the MIDI signals to some effect parameters. But playing with it, something was not right... instead of sending straight X/Y/Z (pressure) coordinates, it sent relative values! This may help to adjust parameters smoothly, but it prevents the obvious application of tapping the center of the X axis to get 50% on parameter, far left to get 0%, etc. Again, relative motion with different levels of sensitivity, soft pickup, and all kinds of other behaviour would have been possible in standard software if they had just sent the pure XYZ coordinates as CC or Pitch Wheel (touchscreen resolution is greater than 7 bit CC value).
The company promised applications for configuing the unit, and the pure coordinates were packed into MIDI sysex data, but as I had no immediate need for the controller I didn't write a driver. Judging by the response to my post to their message board, and other posts, I was far from the only person disappointed by this. When I visited their site recently they seemed to provide software for programming the unit.
When I received the unit in the mail, I eagerly hooked it into GDAM, dumped the MIDI, and made a binding file linking the MIDI signals to some effect parameters. But playing with it, something was not right... instead of sending straight X/Y/Z (pressure) coordinates, it sent relative values! This may help to adjust parameters smoothly, but it prevents the obvious application of tapping the center of the X axis to get 50% on parameter, far left to get 0%, etc. Again, relative motion with different levels of sensitivity, soft pickup, and all kinds of other behaviour would have been possible in standard software if they had just sent the pure XYZ coordinates as CC or Pitch Wheel (touchscreen resolution is greater than 7 bit CC value).
The company promised applications for configuing the unit, and the pure coordinates were packed into MIDI sysex data, but as I had no immediate need for the controller I didn't write a driver. Judging by the response to my post to their message board, and other posts, I was far from the only person disappointed by this. When I visited their site recently they seemed to provide software for programming the unit.
Beatmatching in GDAM
At the end of the last century, as we developed GDAM, we got a lot of questions about automatic beatmatching. Non-DJ's wanted a solution to let them string songs together without error. And there was a growing body of academic work on automated pitch and rhythm tracking.
However, by that time I was listening to music complex enough that auto detection is, even today, far from sufficient to track it. Also, I wanted to have complete control over the music, and was willing to spend a bit of prep time on the music I was going to play. So I decided that regardless of any automated beatmatching, my software needed an ironclad solution for defining the rhythm of music which was too complex to rely on automatic beat detection.
The solution I came up with was the Beat Calculator a tool which assisted the user in
manually mapping out the beat for each song once, and a Song Database which stores and retrieves the beatmatching data for each file. Once you know tempo and phase of music, you have the ability to make perfect loops, synchronize effects, and even jump around in the song without dropping the beat. In fact, I'm shocked that I've still not seen another piece of DJ software which allows the user complete freedom to reorganize the song in real-time without ever dropping the beat. I get a lot of dropped jaws when I show this feature, but it is the simplest little bit of math and logic, which we've employed for over a decade.
We did at one point also support a bit of automated beat-tracking software, if i recall correctly it was called pitchtrack and was part of some grad student's paper. However by this point I had a huge library of perfectly-beatmapped files, and it was clear that automated beatmatching may be a good bullet point for selling an app to non-DJ's, but my simple assisted manual solution offered far better results.
USB A
The worst plug design I've come across. The team that approved the design should be ashamed. It is easy to try to plug in upside-down, and impossible to tell the difference in the dark or if you are pluggin in by touch.
USB B is much better, the difference between square and cut corners is easy to feel in both socket and plug, and it doesn't almost go in if you have it wrong.
Why are A/B different to begin with? Allowing for different form factors allow you to keep up with the times, eg mini-usb ports on cell phones and other tiny devices. And certainly A is an easier fit into the edge of a powerbook than B. But I suspect the standard of A on hosts and B on peripherals was adopted so that people were less likely to plug in hubs the wrong way. If the host, hub, and peripherals all used B then the user would have to pay attention to the routing, the ins and outs. But because in/out ports are differentiated by shape, it is impossible to connect peripherals through a hub incorrectly.
USB B is much better, the difference between square and cut corners is easy to feel in both socket and plug, and it doesn't almost go in if you have it wrong.
Why are A/B different to begin with? Allowing for different form factors allow you to keep up with the times, eg mini-usb ports on cell phones and other tiny devices. And certainly A is an easier fit into the edge of a powerbook than B. But I suspect the standard of A on hosts and B on peripherals was adopted so that people were less likely to plug in hubs the wrong way. If the host, hub, and peripherals all used B then the user would have to pay attention to the routing, the ins and outs. But because in/out ports are differentiated by shape, it is impossible to connect peripherals through a hub incorrectly.
GDAM
GDAM is audio software I've been developing for almost a decade, and provides the foundation for a lot of my experiments in audio and control.
Dave Benson, a brilliant fellow who had ten times my programming experience, started the project with me around 1997. It began as an engine for crossfading playlists controlled by clients across the network, but very soon grew to encompass live mixing. The early GUI was written in JAVA, this was rewritten in GTK in 1998.
When we started, there weren't many options for digital DJ mixing software... if I recall, VTT by carrot interactive, virtual 1200s, maybe an early version of PCDJ. Early DJ software was either winamp with crossfades, or an attempt to clone vinyl turntables or CD decks. The field was young, there wasn't existing software to learn from, these were the first attempts. There was TerminatorX on linux, focused on scratching, but nothing which really treated the problem in terms of manipulating audio files with rhythmic content.
Although I had a turntable and sample, beat, and hip-hop records, I didn't have a large body of experience tied to the turntable interface. I wanted a tool geared towards arranging and presenting a mix of digital music files.
By 1999 we had a tool with a database of beatmaps for song files, sample-accurate beatmatching, and different techniques for arranging song files in real time. A number of GDAM's groundbreaking features have made their way into modern apps, but many still seem to be unique to GDAM.
At first we talked about "going beta" and the version number crept to 0.9, 0.912, 0.945... I think we were both interested in working on it for our own reasons, and didn't focus the effort on a single bugfree release with all the niceties of a finished product. GDAM is available in projects such as planetCCRMA and on bootable discs such as dynebolic. I've kept extending GDAM to accomodate my experiments and music projects. Although I haven't made a tarball release in years, I still commit new features to CVS and it keeps getting more powerful. There was an OSX version for a while, but as OSX grew from 10.0 to 10.1 and later versions, every minor update was breaking the install and I didn't feel it was worth the effort to keep up. Lately, GTK-based apps such as ARDOUR have been ported to native OSX versions, so an OSX install may be easy in the near future. I do still have instructions for compiling on OSX for the brave ones willing to type in terminal, these worked at least as recently as OSX 10.4.
Other posts detailing aspects of GDAM:
Beatmatching
Time Access
Dave Benson, a brilliant fellow who had ten times my programming experience, started the project with me around 1997. It began as an engine for crossfading playlists controlled by clients across the network, but very soon grew to encompass live mixing. The early GUI was written in JAVA, this was rewritten in GTK in 1998.
When we started, there weren't many options for digital DJ mixing software... if I recall, VTT by carrot interactive, virtual 1200s, maybe an early version of PCDJ. Early DJ software was either winamp with crossfades, or an attempt to clone vinyl turntables or CD decks. The field was young, there wasn't existing software to learn from, these were the first attempts. There was TerminatorX on linux, focused on scratching, but nothing which really treated the problem in terms of manipulating audio files with rhythmic content.
Although I had a turntable and sample, beat, and hip-hop records, I didn't have a large body of experience tied to the turntable interface. I wanted a tool geared towards arranging and presenting a mix of digital music files.
By 1999 we had a tool with a database of beatmaps for song files, sample-accurate beatmatching, and different techniques for arranging song files in real time. A number of GDAM's groundbreaking features have made their way into modern apps, but many still seem to be unique to GDAM.
At first we talked about "going beta" and the version number crept to 0.9, 0.912, 0.945... I think we were both interested in working on it for our own reasons, and didn't focus the effort on a single bugfree release with all the niceties of a finished product. GDAM is available in projects such as planetCCRMA and on bootable discs such as dynebolic. I've kept extending GDAM to accomodate my experiments and music projects. Although I haven't made a tarball release in years, I still commit new features to CVS and it keeps getting more powerful. There was an OSX version for a while, but as OSX grew from 10.0 to 10.1 and later versions, every minor update was breaking the install and I didn't feel it was worth the effort to keep up. Lately, GTK-based apps such as ARDOUR have been ported to native OSX versions, so an OSX install may be easy in the near future. I do still have instructions for compiling on OSX for the brave ones willing to type in terminal, these worked at least as recently as OSX 10.4.
Other posts detailing aspects of GDAM:
Beatmatching
Time Access
Subscribe to:
Posts (Atom)