Creating and Programming Synths, Samplers, and VSTs

Photo by Pixabay on Pexels.com

Programming can be difficult – no doubt about that. It is a whole new language to learn and I am just starting. But where to start?

First, I think you need to know where you are going. I am considering programming audio plugins and here are some things I’ve found.

Doing some research, I found that Steve Duda, who made Serum at Xfer Records, had this to say when asked:

I don’t use JUCE (well, not exactly, I use it for a custom test host for debugging, I may implement more bits of it).   I use VSTGUI and a few other libraries such as Boost and SQLite.  JUCE is a fine way to go however.  I wouldn’t recommend taking my advice, I often do things the hard way, but either way, it’s still years of dedication and work, and ultimately “what you use” doesn’t matter much (just like the DAW or OS debates).  JUCE is probably the right choice, as it’s free if you keep things open source, should you make a commercial plug-in the price tag really isn’t bad considering what you get.
As for other knowledge, it’s just so many sources I wouldn’t dare try to list them.   There’s no shortcuts as far as I’m concerned, eventually you want or need to understand things at the lowest levels, so in some ways it makes sense to start with nothing as well.  
But with that said here’s a good starting point, I would recommend learning about everything mentioned in the first post before moving forward 
http://www.kvraudio.com/forum/viewtopic.php?f=33&t=329696 
(I spent months in Reaktor literally, months in Max/MSP, CSound, etc and wouldn’t have ever had a clue what do do in C if I didn’t have that fluency first).   
Hope this helps,  
-Steve

Steve Duda on the Xfer Records forum:
https://xferrecords.com/forums/general/programming-my-own-plugins

The KVR link he mentions has a lot of information in it.

There seems to have been a few “how do I…” threads lately. Perhaps a sticky is needed? 

The text below contains additions and edits from several people. Ideally it would be a wiki and publicly editable. Please suggest additions/edits/updates etc.

*** 

Learn How to Make VST Plugins.

Learning how to make VST plugins from scratch isn’t easy. Making anything non-trivial requires knowledge and skills in several areas. Additionally, there isn’t a single “How To…” guide that will take you through step by step. You’ll need to piece together information from different sources depending on your own specific goals.

Before learning how to code VST plugins you should check out:
– SynthEdit
– SynthMaker
– Reaktor
– Max/MSP
– PureData
– CSound
– SuperCollider
– Bidule
– Usine 
These environments allow you to build something unique without the pain of writing low-level code. If you absolutely need to build a VST plugin, these environments may be useful for prototyping.

Depending on exactly what you want to do, you will require knowledge in a few different areas. If you’re lucky you’ll already have studied some of this and only need to fill in gaps. If not, you’ve a long road ahead. Don’t get overly discouraged, a surprising amount can be achieved with a limited understanding of the basic principles. 


Audio Basics

Before building a VST plugin it would be useful to have an understanding of exactly what sound is and how sound is represented in the digital domain. 

Introduction to Sound Processing by Davide Rocchesso [PDF] Discrete time systems, sampling theorem, audio DSP, maths, psychoacoustics, sound analysis and sound modelling. 

Signals, Sound, and Sensation by William M. Harmann “Designed to follow an introductory text on psychoacoustics, this book takes readers through the mathematics of signal processing from its beginnings in the Fourier transform to advanced topics in modulation, dispersion relations, minimum phase systems, sampled data, and nonlinear distortion. “


Programming

Most professional VST plugins are written in C++. Alternative languages can be used. They each have pros and cons. If you are unsure what language to use, C++ is probably the safest bet. If you don’t already know how to program, go to your local bookstore, browse through all the programming books and choose the one that makes the most sense. Consider enrolling in a class.

Be aware, learning how to program isn’t easy. Learning how to develop VST plugins at the same time will make the task more difficult again. Most people would recommend learning how to program before developing VST plugins.

Some understanding of basic computer science stuff (especially algorithmic complexity, maybe state machines too) is probably useful as well… though that isn’t very hard.

The Audio Programming Book by Richard Boulanger might be useful. Its one of the few books which covers audio plugin formats. 

BasicSynth by Daniel Mitchell “BasicSynth […] shows you how to create a custom synthesizer in software using the C++ programming language.”

For many more programming resources see What are the most important parts of C++ for coding plug-ins?

Recommend a book for learning C++


Maths

Basic engineering math (linear algebra, complex analysis, etc) should be more or less sufficient (ie pick any university textbook, it’ll have more or less the right stuff); even less if you’re not planning on doing any filter design or other “fancy” stuff. Mostly you just have to be fairly confident with your math, since most of the time “try random things” doesn’t exactly work very well (as if we didn’t do it anyway) when it comes to math. 

www.PurpleMath.com Practical algebra lessons.

Digital Signal Processing

You will benefit from some digital signal processing (DSP) knowledge. You can probably get by without knowing how to write your own FFT routines, but you should know what a FFT is and why it is useful.

Much of what is covered in introductory DSP textbooks will not be immediately useful for developing plugins, but advanced texts focusing on audio usually require at least a conversational level DSP understanding.

Online and Free:
The Scientist & Engineer’s Guide to Digital Signal Processing

Print:
Understanding Digital Signal Processing by Richard G. Lyons


Audio Digital Signal Processing

Audio DSP extends on core DSP concepts to include the way digital signal processes apply to digital audio. It includes subjects such as audio filters, delays, non-linear effects (think compression) and much more. 

DAFX by Udo Zolzer is a good introductory textbook covering many audio DSP techniques. It sometimes reads as a summary. For more detailed material you may need to seek specific papers. DAFX has many references listed and provides a good starting point for exploring.

The Art of VA Filter Design Zero-delay feedback filter design. No preliminary filter design knowledge required. 

DSP Audio Classics

DSP Audio Algorithm Notes by XOXOS


Previous Threads Asking a Similar Question
Adivce for someone with ZERO experience
How do I go about learing to develop synths?
Developing a Vst Effect Plugin Where To Start?
What is your development setup?

Any more threads that should be linked here?


Books… 
The books listed here aren’t the only books on these topics. They are listed here as a starting point for your own research. Please look at alternatives before purchasing anything. 


Audio Plugin Frameworks

JUCE is an all-encompassing C++ class library for developing cross-platform software. JUCE includes components for VST, AU and RTAS. JUCE is often highly recommended. Definitely check it out if you use C++. 

IPlug is a C++ framework for developing audio plugins and GUIs.

Enhanced version of Cockos’ IPlug A simple-to-use C++ framework for developing cross platform audio plugins and targeting multiple plugin APIs with the same code. VST / VST3 / Audiounit / RTAS / AAX (Native) formats supported. 

VST.NET allows VST Plugin developers to write Plugins in any .NET language. It makes the transition between the C++ and .NET world smooth and easy. The Framework built on top of the interop layer provides a clear and structured architecture accelerating development considerably.

Delphi ASIO and VST A Delphi library for creating VST plugins, VST hosts and ASIO applications. Includes algorithms for filters and dynamics.

KVRAudio forum post:
https://www.kvraudio.com/forum/viewtopic.php?f=33&t=329696

Csound

Csound is mentioned and is a sound and music computing system.
https://csound.com/

Partikkel Audio has made a granular synth named Hadron by using Csound.
https://www.partikkelaudio.com/

Juce

Juce is mentioned also.
https://juce.com/

C++

C++ has many IDEs and some of them are C++ Builder or Visual Studio C++.