Creating Audio Unit plug-ins is an opportunity for developers with a love of audio
to get involved in an exciting and growing market. Music and sound applications
are at the heart of creative work in film and music production studios,
recording studios, and home studios all over the world. In all these venues,
there is a robust demand for high-quality plug-ins that can work with and
enhance the applications.
In fact, it seems people who use music or sound
programs are always hungry for more plug-ins. Once they get a platform to work on, they
want more and more effects, more and more instruments.
This large market got even larger in 2004 when Apple began bundling GarageBand with
every Mac shipped. For the first time, a fully-capable music application was in the
hands of every buyer of a new Mac—now numbering in the millions. As a result, retail
music outlets are seeing new store traffic and new growth in
their computer music and peripherals business. And music plug-in developers, who
traditionally focused on the pro market, are now starting to offer consumer
versions of their software.
This Audio Units community is growing, and it offers you an opportunity to
get involved if you have the programming skills and an interest in audio. In this article, we'll highlight the tools
and technologies involved, talk with one successful developer to see how he does it, and show you how
to get started. We'll also fire up a couple of sample audio unit plug-ins for you to try, and discuss
some of the issues you'll want to consider before you get started.
Advantage Audio Units
The Audio Unit framework built into Mac OS X as part of Core Audio makes it a
lot easier to pursue this opportunity, whether you're a solo developer or a
30-person company. First, the framework provides a complete foundation
for writing robust audio plug-ins, because it lives at the OS level rather than
the applications level. In addition, it frees you from worrying about applications
compatibility; most any music program running on the Mac can host your Audio Unit
plug-in, and many in fact have this capability already. That great custom sound effect or unique digital
instrument will work seamlessly with GarageBand, Logic Express, Logic Pro,
Final Cut Express, Final Cut Pro, Soundtrack Pro, Motu Digital Performer, BIAS Peak
and many others.
Since Apple maintains the Audio Unit framework, you can be sure that it works
well with everything else in the operating system, now and in the future. In addition,
Apple will continue to evolve and extend its features. With the release of
Tiger, for example, two new audio units shipped: AUNetSend and AUNetReceive.
(Note: These new units let you distribute sound processing or sound generation over
the network—a useful capability in a recording studio, for example, with a lot
of Macs on hand.)
The Markets
Today, the market segments for Audio Unit plug-ins
include consumer, aspiring professional, and professional. Many developers are
now creating plug-ins for all these segments.
A good example is SoundSoap and SoundSoap Pro by BIAS. SoundSoap is lighter on features than
SoundSoap Pro and offers a much simplified UI. But it has some of the key
SoundSoap Pro capabilities, which leaves an ideal upgrade opportunity as the
consumer finds they want more functionality.
Figure 1: SoundSoap—features and UI for the consumer market.
Writing a plug-in is a smaller project than writing a full-blown music
application. So it's a natural place to start. With acute ears, some sonic
creativity, and good programming skills, you can end up with a viable business.
Figure 2: SoundSoap Pro—features and UI for the professional market..
A Successful Independent Audio Developer
Urs Heckmann has done just that. Based in Berlin, Germany, Urs has built a
worldwide reputation for outstanding work in the audio plug-in market. To see
his innovative products visit Urs's website.
Figure 3: Filterscape—one of the newest plug-in products from Urs Heckmann.
Urs is now making a living with work he loves. "After I released my first
commercial plug-in," he says, "everything went so well that I could quit my
former day job. I've become a full time developer and, so far, I've always been
able to pay my rent. It's the job that I always wanted. I definitely work hard,
but it's also maximum fun."
As an independent developer, Urs sees some real advantages. "I can keep in
touch with my customers. I'd say I have 99% satisfied buyers of my software,
which is pretty good. The way I develop something is to start with a small and
simple idea, and then work with musicians to evolve the idea. Filterscape, for
example, started out as a simple parametric equalizer with eight knobs.
Eventually it became a plug-in with almost 700 parameters. Typically I build a
prototype and send it out to some musicians so that we can discuss what's useful
and what's not. Then I come up with a new version. It's very much an iterative
process. There's not really a fully detailed concept in the beginning. It's more
like "raising" a plug-in until it becomes mature enough to bring it to market."
The Audio Unit technology caught Urs' attention from the beginning. "When
the framework first came out on the Mac in 2002," he says, "I saw some technical benefits.
For instance, the idea of metadata for plug-in parameters and the strict
separation of DSP processes from the user interface sounded intriguing. Now the
Audio Unit technology has become a critical part of my work. It's scalability
allows me to evolve plug-ins in a way that fits my needs. Also, I love the way
Audio Unit plug-ins are integrated into the system—with standardized file
locations, the events scheme, and so on."
As well as larger companies, independent developers such as Urs play a role
in providing feedback and suggestions to Apple during Apple's on-going
development of the Audio Unit framework.
Trying Out an Audio Unit Plug-in
How can you try out an Audio Unit plug-in to see what it does? One way is to use
AU Lab, a free digital mixing application that hosts Audio Unit effects and that
ships with the Apple Developer tools.
To check out the Audio Unit plug-ins that Apple ships with every system, try the following:
- Launch the AU Lab application
found in:
/Developer/Applications/Audio.
- In the Create New Document dialog, click on the Inputs tab.
- Click Add Input (click the plus sign).
- Then select OK.
- You should end up with a AU Lab document with one input and one output, as shown in Figure 4 (see the righthand column).
- Play some music or audio through the audio-in jack on your Mac. AU Lab processes the sound.
- To try an effect, click in the Effects pane and select from the Apple
submenu. Try AUMatrixReverb—it should provide a dramatic sound.
- This effect has a custom user interface that allows you to select multiple bands and then
change them as a group, as shown in Figure 5 (below). It also has generic UI, which you
can see by selecting Generic View from the upper right control. If you don't
provide a custom view for your plug-in, Core Audio can automatically provide a
generic view.
- Try other Apple plug-ins checking to see which have custom UI and which don't.
You can look to the Web for interesting effects you'd like to try. Here are some sample web sites that provide information and free plug-ins:
Figure 5: Custom UI for the AUMaxtrixReverb plug-in.
Developing an Audio Unit Plug-in
Typically, a developer creates an Audio Unit plug-in that either processes audio
(an effect) or produces audio (an instrument). The Audio Unit plug-in is
packaged as a component and usually has some sort of custom UI that lets the
user tweak the plug-in's parameters.
If you are setting out to create an Audio Unit plug-in, there are five phases of
the work to think about:
- Start with an idea;
- Write the DSP algorithms for the effect, the instrument, or whatever you're creating;
- Test and debug;
- Optimize performance;
- Write the user interface.
Let's look at each of these phases individually. Your workflow may be different, of course, but this is a good way to start.
1. Start with an idea.
Don't hesitate to start with something simple. A simple idea can quickly lead to
something richer and more complex—as Urs Heckmann testifies.
2. Write the DSP algorithms.
The key starting point for this work is the Core Audio SDK. This SDK
is a collection of C++ classes. By subclassing from these classes to build your
plug-in, you can be assured that the your engineering foundations are solid.
For an essential and useful hands-on overview of the basics of building and
validating a simple Audio Unit plug-in, see the Audio Unit Programming Guide, and the ADC article Turning Up the Volume with Audio Units.
The basic goal in this phase is to get the effect or instrument sounding the way
you want it to sound.
3. Test and debug
Equally important is checking your work with the command-line AU Validation
Tool. (You can invoke this tool from the command line with the command auval.)
Passing this validation test means that your plug-in is in conformance with the
AU Specification and should be operationally correct when used in host
applications. You can be confident that your code is well-behaved—and not
likely to get into trouble in the future.
Also note that this validation test is required if you want to use the official
Apple AU logo on your software when you take it to market. (See Software
Licensing & Trademark Agreements: Audio Units for the details). As well, some
host applications such as Logic or Digital Performer actually run the AU
Validation Tool on any new plug-in before they accept it and make it available
to the user.
So there are a lot of reasons to use the AU Validation Tool all along the way.
4. Optimize Performance.
This is a critical step. No matter how great the effect or cool the instrument,
it won't get far in the marketplace if it chews up too much CPU time and
compromises performance of the host application.
A first step in optimizing your DSP code is to profile your routines and
identify that are most CPU-intensive. Standard performance tools such as Shark can help do
this.
Once you know where to focus, you can rewrite the C or C++ code in more
efficient but less readable C or C++.
Alternatively, you can think about "vectorizing" your code, accelerating the
vector operations that are common in DSP routines. Traditionally, this has meant
using assembly language—in the case of the PowerPC-based Macintosh computers, the AltiVec
instruction set; in the case of the upcoming Intel-based Macintosh computers, the SSE instruction set.
With the advent of Intel-based Macintosh computers, many audio unit developers
may speed up their code using the AltiVec instructions, and then port these
routines to SSE to create a fully bilingual plug-in. (At run time, you simply
check for the type of CPU and branch accordingly.)
But Apple provides an appealing alternative: the Accelerate Framework. Made up of
high-performance, vector-accelerated libraries that create a new layer of
abstraction, this framework lets you access vector-based code without writing
vector instructions. And you don't need to worry about processor dependencies. The system automatically invokes the appropriate
instruction set. For details, see the Universal Binary
Programming Guidelines.
5. Write the User Interface.
Once your DSP algorithms are in place, you can turn your attention to the UI.
Optionally, you can do no custom UI and let Core Audio create generic UI at run
time.
For instance, some people would write the UI first. Others (like Urs), would
have the UI actually written or designed by someone else, before or while the
DSP is being written.
But if you want custom UI, use the Cocoa frameworks to do the work.
An important UI technology supported in Cocoa is OpenGL, which is makes it easy
to present compelling visual representations of audio data or visually-rich user
feedback.
In addition, there are two important checks you should
make in this phase of the work: CPU usage and validity.
The CPU usage of your unit is directly related to optimization (phase 4). You
want to make sure that even your unoptimized code is reasonably efficient from
the get go. That way final optimization won't involve a massive rework of your
efforts. To do this, you can use the CPU Load view in AU Lab (selected from the
Window menu).
A Great Example
A great way to learn is to look at a great example.
The FilterDemo sample plug-in in the Core Audio SDK version 1.4.2 fits the
profile. It provides a clearly written, fully functional set of DSP routines. It
also has Cocoa-based custom graphic UI that interacts with the DSP processing in
real time.
You might want to build FilterDemo first and see it in action before you look at the source code. Do this:
- First, look for the Filter Demo at the location:
/Developer/Examples/CoreAudio/AudioUnits/FilterDemo
- If you don't find it there, make sure you have upgraded your developer tools to Xcode 2.1 (or later). If necessary, install
the latest Core Audio SDK (version 1.4.2 or later) available from the Development Kits page.
- Launch Xcode and choose Open from the File menu.
- Navigate to
/Developer/Examples/CoreAudio/AudioUnits/FilterDemo and click Open.
- Click Build.
- When the build is complete, find the FilterDemo.component in the Development
folder in the designated directory for your build products.
- Move or copy this component to the directory
/Library/Audio/Plug-ins/Components.
- Now relaunch AU Lab.
- Play some audio through AU Lab and then open the Effects menu. You'll notice the
new Apple Demo item. Choose Filter from its submenu.
Figure 6: Apple Demo Filter.
You can change both plug-in parameters interactively using the cross-hairs.
(You can also switch to a generic view using the menu on the upper right.)
When you're ready, take a look at the source code for FilterDemo.
One important point—the DSP routines in FilterDemo are not optimized.
What you have is an example of a project with a complete phase 1 and phase 2.
Phase 3 is TBD, or, as they say in the textbooks, left as an exercise for the
reader.
For More Information
As you investigate what it takes to do Audio Unit plug-in development,
here are some resources to know about:
Web sites with information and free plug-ins:
Updated: 2006-08-21
|
