Resynthesis on the Synclavier Audio System
The Synclavier Resynthesis System provides the tools for a powerful new approach to sound synthesis that until now had been beyond the capability of ANY synthesizer. Using the SFM command SYNTHESIZE, you will be able to create life like acoustic instrument timbres as well as an unlimited range of new timbres unrealisable on any other digital sampler.
Resynthesis may be likened to the process of filming a scene with a movie camera and playing it back through a projector. The camera records a series of static images, or frames, which are pictures of the scene filmed at distinct moments in time. When played back through the movie projector, the different frames combine to produce the illusion of smooth movement and convey a realistic representation of the original scene. Similarly, the Resynthesis program captures the acoustic properties of a sound at a number of discrete moments in time, which you mark with a special label.
These sound 'pictures' are then spliced together into a series of timbre frames that crossfade from one to the next. (A timbre frame could be thought of as an individual partial timbre with it's own unique set of harmonics, volume, possibly it's own pitch, and several other parameters which you will see as you use the software). The enormous power for RESYNTHESIS then places these automatically computed timbre frames one after another, giving you many, many sets of harmonics in succession as you play the note. In some cases it would be as though you had as many as 64 different partial timbres flying by as you play each note.
To make such complex sound easy to use, all of these automatically computed timbre frames are placed on your Partial Timbre Button 1. You may treat this COMPLEX PARTIAL TIMBRE 1 in the same manner as you have done before. You may BOUNCE it, change the TUNING, change the VOLUME etc., etc. (Note: FM - Frequency Modulation, is not used at all in the Resynthesis process)
Now we know what Synthesis does, let's go through the process from sampled sound to resynthesised sound. Begin by recording a sample you wish to resynthesise, or use one you have already sampled. For best results, start with a sample that has a constant harmonic spectrum, such as a trumpet, trombone, clarinet or flute.
First start SFM by typing SFM from the Ready > prompt or by selecting it from the main menu. Call up your file with the command OLD followed by the name of your stored sample. e.g.: OLD TRUMBF#3 (Type DIR to see a list of what is available on you Hard Disk - mono files only can be used, type MENU to see a listing of all available commands). You will see immediately the length of your sample in seconds. What you do now is first get a general view of the shape of the volume from beginning to end. Type DISPLAY and change the horizontal scale of the SIGNAL DISPLAY so that the entire sound file is displayed by typing SET HOR (izontal) followed by the length of the file in seconds. . e.g.: SET HOR 1.6
You will see the entire sound file displayed on the screen, making the points of major change of volume relatively obvious. Label these points of change by moving your cursor (using the arrow keys) and as general points of reference you might label them Attack, Decay, Final Decay etc. Type LABEL followed by your own choice of a label name. These are general labels to help you in placing the analysis labels, which you will do next.
Now return the horizontal scale to it's default setting by typing SET HOR .01. You will see 1/100th of a second of the sample. Find the start of the sample by typing DISPLAY 0. If your sample does not begin at 0 seconds you could find the start by typing SEARCH .2 This will find the first occurrence of a very small level of the signal (2/10 of a volt). This is just a suggested level but it works in most cases. You will now be looking at the first 10 milliseconds (1/100 second) of the attack portion of the sound.
Before using the analysis program, the sound file must be prepared by adding special labels at the points of time you consider to be important to capture the realism of the sound. It is important to consider REAL SOUNDS at this point. Your sampling system captures the REAL SOUND by recording 'images' of the sound at up to 50 thousand times a second (resynthesis only recognises mono sounds up to 50khz sample rate). Most of these 'images' are almost entirely identical, in the case of a held note.
For example, when a brass instrument is played, the sound changes quite a bit at the beginning of the note. The lips start the vibration in the horn, there is some noise produced which very rapidly disappears, and then what is called the 'steady state' of the sound is heard - the sustained note. It is this steady state portion of the sound that contains much of the same information.
With careful placing of these special labels at the right moments in time in the recorded sample, - let's say a lot of labels at the beginning to catch the fast changes in the attack portion, and fewer labels later on to capture the held note or 'steady state' portion of the sample - authentic acoustic sounds will result. Get it wrong and you can still discover wild new sounds so do not despair. A crossfade time between the resulting timbre frames will be automatically computed and will match the times you marked in the original sample where you placed these special labels.
The idea is to create just enough timbre frames to capture the essence of the sound and no more. If you use too many labels, the efficiency of the RTP will be reduced, if you use too few labels the resynthesised sounds will not be authentic. By placing labels at the locations in the sound that you deem important to the identity of the sound, you can maintain precise control over the analysis.
These special analysis labels are letters or words followed by the characters '_1' and occasionally '_2'. Most of the time you will only use '_1'. '_2' is not used very often when analysing a steady pitch. So to start lets ignore '_2' and analyse a steady pitch of some kind. (See the Sample To Disk manual for more information on the '_2' label).
Now let's look at the sound you've chosen to analyse. Some sections of the waveform will either be periodic (a waveform that is composed of a series of like or very similar cycles), or it will be aperiodic (it does not contain primarily repetitive cycles). You will find aperiodic waveforms will occur at the beginning, during the attack portion of the sample, and the periodic waveforms during the steady state portion. First you can type the command LINE to connect all the dots of the waveform to make it more readable.
Use the right and left arrow to move your cursor to the beginning of the sample. Type LAB A_1 You can use can use any name up to 6 letters but it must end with _1 It is worth noting at this point that the process of marking labels is different with each TYPE of sample you resynthesize - and the first few times you try it you may not realise any success at all. If you start with a well played note from a brass instrument, you should have a minimum of problems during your first few attempts.
You may at times want to change the amount of time you are looking at the SIGNAL DISPLAY to get a better view of the waveforms. The PF3 and the PF4 keys (on an extended Mac keyboard these are the '/' and the '*' keys above the numeric keypad) respectively half or double the amount of time you are looking at. Also the up and down arrow keys move you to the next page of waveform display.
At the beginning of a brass instrument attack, you will see a small waveform gradually getting larger and changing it's shape into the steady state waveform. Place each of your labels, as a suggestion at maybe every fourth or fifth of these first waveforms. Place your label as close to the zero crossing point of the waveform as possible. This method should capture the attack variations fairly well. Now with UP arrow key view the next portion of the sample.
Now as you see the waveforms gradually becoming more of the same shape, place your labels farther and farther apart. The crossfade times mentioned earlier will be the ATTACK times of each timbre frame matching the actual times between each of your labels. Remember that each timbre frame created can be thought of as a partial timbre with it's own unique set of harmonics, volume, etc. Start your first analysis with no more than fifteen or twenty labels and experiment from there. Extremely good timbres have resulted from even fewer labels.
When you have finished placing your labels the next step is to calculate the exact pitch of the sample. You will use the spectral display to get this information. To prepare the spectral display you will need to change a few parameters in the SET MENU. Type SET and you will see many parameters. All you need to change is the number of FFT's, the window LENGTH and the RANGE of frequency you will be looking at. The first two values will always be the same. Type SET FFT 9000 Type SET LENGTH .2 These values are larger than required and the computer will set the values to a value less than you type in automatically. Setting the RANGE to 1 or 2 is up to you. 1 will display zero to one thousand, and 2 will display zero to two thousand hertz.
Type ENVELOPE to see the overall sample shape. Move the cursor to the centre of the notes envelope and type SPECTRAL . After a short while you will see several peaks drawn in the display. When you see 'OK' printed at the top of the screen, move the cursor to the top of the first peak. In the box to the right there will be a number in parenthesis. This is called the pitch class of the note. Type SET OCTAVE and this number. This number tells the resynthesis program the pitch of the sample.
Now type SYNTHESIZE. You will see the Analysis Program Menu come to the screen. You may change many things here, but it is not necessary for this first attempt. If there is vibrato in the sample you can choose to have it followed by changing NO to YES at the line that reads USE RAW PITCH by hitting the space bar. You will see that the keypad at the right of the keyboard is used here - the instructions for using this program are displayed on the screen. You'll see that if you hit the number 1 key (called KP1 for keypad #1) the computer will then resynthesize the sample. You will see the various displays during the analysis, and when it is complete (faster for higher notes), the timbre will be placed on the VK keyboard ready for you to save to disk.
Good luck and happy experimenting. That is what the Synclavier is all about.
In Brief:
SFM
DIR or CAT (use ent subcat: to change catalogs)
MENU
OLD
TRUMBF#3
DISPLAY
SET HOR 1.6 (length of the file called up)
LABEL
ATTACK Add marker labels; attack, sustain, decay etc.
SET HOR .01
DISPLAY 0
LINE
LAB A_1
LAB B_1
LAB C_1
etc. etc.
SET FFT 9000
SET LENGTH .2
ENVELOPE
SPECTRAL
SET OCTAVE +value
taken reading from first peak
SYNTHESIZE
KP1
Now name and save
your timbre on the A Page
