Using the new sound engine
Things you'll need:
*A schema will let you know where there are mistakes, and will enable autocomplete to suggest what to type.
Fundamental Design
The new sound system is event based, meaning that experience with other popular middleware such as FMOD or Wwise can be useful. A vehicle loads a number of events and identifies them by name.
What these events do is entirely up to the author of the content. These events are stored as a JSON file and can be validated using the rFactor 2 SFX JSON Schema in any supported JSON editor.
This will also enable useful hints and autocomplete for near automated authoring of events.
There are different types of events, and each type can contain other events. The event type determines how the contained events get played back (examples follow). This leads to an event hierarchy, where each event inherits properties from its parents.
As an example, this allows an event to contain several loops that are played back at the same time and inherit the event's position and effect settings, while each loop separately controls pitch and volume.
An example of what this engine event looks like, on a higher level:
And as JSON:
{
"multi": {
"comment": "multi represents the event type. The name variable below represents the name this event is identified with by the game. The other keys are event properties that affect playback of all children",
"name": "engine",
"volume": {
"key": "engineRPM",
"value": [[10,0,0],[1850,0.7,0.2],[6000,1,0],[9400,1.5,0.2]]
"comment": "Here we scale the entire engine volume by RPM to create a ramping effect of a louder engine at higher RPM without having to mess with individual samples.
Each bracket represents one key, where the first number is the RPM where it applies, the second number is the volume, and the third number is the shape (where 0 is linear, 1 and -1 are a curve).",
},
"position": [0,0.3,1.0],
"effect": "engine",
"filter": "engine",
"commentExtra": "effect and filter apply sound effects and direct filters respectively. They use a name to reference a filter or effect that is defined elsewhere in the file. Position is local to the car.",
"events": [
{
"multi": {
"name": "power",
"comment": "The name here is purely for our own knowledge since this event's parent plays back all children anyway. And so does this event! It applies a slightly curved throttle ramp between 10% and 70% throttle, applying to all children, which should be samples used for being on power similar to the old sound system.",
"volume": {
"key": "throttle",
"comment": "The key is a reference to a variable the game will read that correlates to the first value of each key. A list of possible keys, and a visual example, will be provided later"
"value": [[0.1,0,0],[0.7,1,0.3]]
},
"events": [
{
"loop": {
"comment": "Instead of the event type multi, we're looking at the type loop. This type loads a sample and plays it back in a loop whenever the event is fired by its parent. We use the engineRPM key again to define when this particular sample fades in our out, and the same exact system for the sample's pitch. While the volume curve, ideally, is a curve, the pitch should be linearly proportional to RPM. The key used at 1.0 pitch should correspond to the natural RPM the sample was recorded with. This event supports advanced features such as loop points, allowing a sample to loop between specific points so that initial playback won't have to sound as harsh. This can also allow fake engine wobble effects coming on throttle, which adds a lot of immersion! The other loop events below make up part of an engine's Power sound effects.
Note: Samples can be referenced relative to the vehicle's directory now. They no longer have to be in a MAS file in moddev.",
"sample": "sounds\\idle_int.wav",
"volume": {
"key": "engineRPM",
"value": [[10,0,0],[250,1,0.2],[2000,1,0],[2400,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[1850,1,0],[3700,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\1900_on_int.wav",
"volume": {
"key": "engineRPM",
"value": [[2000,0,0],[2400,1,0.2],[2500,1,0],[3000,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[1993,1,0],[3986,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\3800_on_int.wav",
"volume": {
"key": "engineRPM",
"value": [[2500,0,0],[3000,1,0.2],[3700,1,0],[4500,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[3800,1,0],[7600,2,0]]
}
}
}
]
}
},
{
"multi": {
"name": "coast",
"comment": "This is essentially the Coast equivalent of the previous event group. Same thing, but for coast!",
"volume": {
"key": "throttle",
"value": [[0.1,0.75,0],[0.7,0,0.3]]
},
"events": [
{
"loop": {
"sample": "sounds\\idle_int.wav",
"volume": {
"key": "engineRPM",
"value": [[10,0,0],[250,1,0.2],[2000,1,0],[2400,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[1850,1,0],[3700,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\3400_off_int.wav",
"volume": {
"key": "engineRPM",
"value": [[2000,0,0],[2400,1,0.2],[3600,1,0],[4800,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[3566,1,0],[7132,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\5800_off_int.wav",
"volume": {
"key": "engineRPM",
"value": [[3600,0,0],[4800,1,0.2],[5700,1,0],[6400,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[6083,1,0],[12166,2,0]]
}
}
}
]
}
}
]
}
}
Since events and their properties are now logically grouped together, a lot of indentation is used. This bloats the file size a bit. Collapsing single events that aren't being worked on can help focus on the important bits!
Due to this logical grouping, a visual editor is possible. However, currently there are no immediate plans to create one.
Testing Effects Ingame
A new debugger was added to ModDev. This debugger can be used to manually fire specific events, and both read out and override variables an event might be using, in order to simulate any situation in a controlled environment.
In order to activate this debugger, load a car using a JSON based sound file and hit the "Enter Sound Debugger" button, which presents this window:
Here you can click on Add Variable or Add Event in order to select which event or variable you would like to debug. While Edit Mode is inactive, this will show the current state of the event or variable. If edit mode is active, the event's state and variable can be freely changed and the effects apply in realtime.
The Reload JSON button will immediately reload the file and any changes will be applied on the spot. This may require sounds to be stopped for a moment in case of bigger modifications.
Using this tool, a car's sound can be fully tested and quickly iterated upon without the need to drive around, using the ingame sound stage and effect systems, for perfect representation of what the user will end up hearing.
Event Types
Several types of events exist. Any event type can be used for any given event. The type only determines what the event actually does when fired, and some other event specific properties.
Some event types extend other event types, in which case they will have all the same properties plus a few additional ones. One such example is a loop event which is similar to a burst event, but has additional properties and behaves differently.
Note that due to the way the event tree works, giving an event a property it does not use will still mean any events it contains that do use the property will inherit it.
The following event types exist:
| Type Identifier | Property | Description | Inherits from |
|---|---|---|---|
| bank | Base event. All other types inherit all properties from it. If it contains any other events, when fired, it will fire those events passing them the same event name it was fired with. Generally unused. | - | |
| events | An array containing other events this event can fire and interact with. | ||
| volume | (optional) Either a fixed number or a curve (see doc "Volume and Pitch Curves"). Special property: Applies to all children ontop of their own volume. | ||
| pitch | (optional) Same as volume. | ||
| position | (optional) An array with 3 elements containing the local position of this emitter relative to its parent. If no parent exists or all parents have the offset 0,0,0, this is relative to the car's pivot point. | ||
| direction | (optional) An array with 3 elements defining the euler angles of the emitter. This is only useful if an event related to this event uses a cone shape (see burst type) | ||
| name | Identifier. Used to fire the event. This only has to exist if this event's parent is a type that fires by name, otherwise it may be used for identification but is not required by the system. | ||
| multibank | Contains multiple events and only ever plays one at a time. Commonly used to separate between internal and external sounds, or different types of terrain for road noise. When an event of this type gets fired, the name of the event that gets fired will be passed on to the active child of it. | bank | |
| bankKey | String containing the name of the property that will be used to determine which event to play. For example, "camera-in-out" will lead to either "inside" or "outside" being played depending on where the camera is. When the camera changes position, the property will automatically update this event causing it to switch the event it plays back - which could be an event containing the whole spectrum of events needed to represent a full car for one of the camera perspectives in this case. | ||
| allowEmpty | (optional) If true, when the bankKey refers to an event name that does not exist, it will stop playing until the bankKey refers to an existing event again. Otherwise it will simply continue playing the previously valid event. | ||
| crossfade | This event is identical to multibank, except it crossfades between events when it switches, causing a smoother transition. | multibank | |
| time | Time in seconds for the crossfade to complete. | ||
| exponent | Shape of the fade. 0 is linear, 1 is exponential. Anything in-between is valid. | ||
| multi | Simply plays back every event it contains. If called as part of a loop, all contained events must be loops. If called as a burst, they must be burst. | bank | |
| random | Randomly picks a contained event to play it back each time it gets fired. In case of loops, this selection only changes when the loop stops. Contained events may be empty or with invalid samples in order to create an event that only plays sometimes. | bank | |
| burst | Plays a single burst sound. | bank | |
| cone | (optional) Array with 3 elements specifying radius at which to start fading out, radius at which to end fading out, gain when fully faded out. Used in combination with direction to create sounds that can only be heard from certain directions. Very useful for separating exhaust and intake engine samples. | ||
| sample | Path to sample this event will play back. This path can be local to the vehicle's subdirectory in moddev and the sample does not have to be in a MAS archive. Presently, only 16-bit PCM WAV files are officially supported, but certain other PCM formats might work. Stereo samples will be downmixed to mono, so ideally just use mono mixes. This is necessary for attenuation and panning. | ||
| randomSample | (optional) If bigger than 0, it determines the random playback offset each time this event gets fired. | ||
| refDistance | (optional) Distance (in m) where gain will be 1.0. Reasonable defaults have been chosen, so this is not required. If used, try applying it to the whole car for consistency. Automatic inheritance should make that a two line change to the file. | ||
| rolloff | (optional) Rolloff. Works together with refDistance. Lower numbers mean higher attenuation. | ||
| doNotRestart | (optional) If true, will prevent the sample from being fired again when it's still in the middle of playing. | ||
| effect | (optional) String referring to the name of the environment effect used for this event. | ||
| filter | (optional) String referring to the name of the direct path filter used for this event. Combine with effects to achieve a sensation of depth and a wide sound stage! | ||
| loop | Plays a looping sound | burst | |
| loopSample | (optional) If defined, specifies offset in samples where the sound will loop when it reaches the end. This can be used to add fake transmission wobble or more pleasant back-on-throttle or post-shift effects without hearing them in a loop, and can dramatically reduce the length of a sample used for loops compared to the old system before it inevitably repeats in an immersion-breaking manner. | ||
| template | This is a special type. It does not inherit from anything, but is used to create more events from a single input to avoid lots of copy-paste work for easy iteration. Useful for defining only one wheel, but applying the info to all four wheels instead. See Templates section for more info! |
List of Events and Variables
While the JSON Schema should automatically suggest a full list of events and variables available to use, a full list of available events and variables can always be found in the Sound Debugger window's dropdown menus.
Event/Variable names should suggest what they represent. A more detailed list with full descriptions will be added here at a later date.
Volume and Pitch Curves
Pitch and volume definitions can either be a static number or use a keying system that allows these to change in realtime based on game parameters.
Use of the keying system can look like this:
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[6083,1,0],[12166,2,0]]
}
Let's take this apart a little. The pitch has a key and a value. The key represents the name of the ingame parameter that will be used by the sound engine (see list in the sound debugger!).
The value is an array of keys, each key being an array with 3 elements too. There is no limit for how many keys can be used here as long as it's a minimum of 2.
Each key consists of the following numbers: ref value, pitch (or volume) at ref, shape. The first value in our example represents an engine RPM at which the key is valid. For example, at an engine RPM below 6083, the system will interpolate between the first two keys. If the value falls outside of the boundaries of available keys, it will simply clamp to the nearest one.
In that example, the shape is set to 0 (linear), so it will be linearly interpolated. So at an engine RPM of 3041, we would be at a pitch of about 0.5. This shape value is always taken from the next key, so in case of our example interpolating between keys 1 and 2, it would take key 2's shape.
If the third value, the shape, wasn't 0, this would not be linear. Instead, it can be anywhere between -1 (red, pushed down) and 1 (blue, pushed up) in the following visual demonstration:
As an example, this can be useful for example for wind effects, which are not linear by nature as drag increases exponentially with velocity. So in that case, the shape value of a wind's volume (or maybe also pitch!) would be a negative number.
None of this is restricted to specific events. Any key can be used in any event. It is completely possible to use wheel load to change how the engine sounds if that is desired.
Note: Volume and pitch aren't just inherited from parent events, but multiplied. So if we have an event that has a volume of 0.5, and that contains an event with a volume of 0.6, the final playback volume will be 0.3!
Templates
Templates are a special type of event type. They get processed at load time, and will turn one event into multiple. This can be used in place of copy-pasting events that repeat a lot, such as events that affect all four wheels and would be identical aside from their playback positions.
A template defines the following information:
- find: A string to find in the contained event
- replace: an array of strings to replace the string above with. The more elements the array contains, the more copies will be made
- event: The actual event object to process. Can be of any other type, and strings will be found and replaced recursively inside of it
The following template:
{
"template": {
"find": "CORNER",
"replace": ["FL","FR","RL","RR"],
"event": {
"loop": {
"sample": "sounds\\wheel_noise_CORNER.wav",
"position": "wheelCORNER",
"volume": {
"key": "wheelRPM_CORNER",
"value": [[0,0,0],[500,1,-0.3]]
}
}
}
}
}
Will be resolved to the following events:
{
"loop": {
"sample": "sounds\\wheel_noise_FL.wav",
"position": "wheelFL",
"volume": {
"key": "wheelRPM_FL",
"value": [[0,0,0],[500,1,-0.3]]
}
}
},
{
"loop": {
"sample": "sounds\\wheel_noise_FR.wav",
"position": "wheelFR",
"volume": {
"key": "wheelRPM_FR",
"value": [[0,0,0],[500,1,-0.3]]
}
}
},
{
"loop": {
"sample": "sounds\\wheel_noise_RL.wav",
"position": "wheelRL",
"volume": {
"key": "wheelRPM_RL",
"value": [[0,0,0],[500,1,-0.3]]
}
}
},
{
"loop": {
"sample": "sounds\\wheel_noise_RR.wav",
"position": "wheelRR",
"volume": {
"key": "wheelRPM_RR",
"value": [[0,0,0],[500,1,-0.3]]
}
}
}
Templates can be used this way to create unlimited amounts of events.
References
The JSON system supports references. These allow objects from elsewhere to be copied by simply referring to them. It follows this specification.
Currently it does not support loading resources outside of the current vehicle's sound json files and a collection of shared resources that are shipped with the base game. An example of a reference to a shared resource:
"$ref": "rfsnd:global/effects/examples/ambient"
This example will replace itself with a full effect definition for an ambient sound effect, used for external sounds.
Common Techniques
Engine Blending
In the old sound system, engine blending worked by defining all used engine samples, then entering their reference RPMs. Since the new system does not hardcode this, engine blending will be done the same as any other blending, through the use of a "multi"-event:
{
"multi": {
"name": "engine",
"volume": {
"key": "engineRPM",
"value": [[10,0,0],[1800,1.0,0.2],[4000,1.2,0],[6000,1.3,0],[7000,1.45,0.2]]
},
"position": [0,0.5,-1.0],
"effect": "engine",
"filter": "engine",
"value": "engineRPM",
"events": [
{
"loop": {
"sample": "sounds\\Engine_int_idle_01466.wav",
"volume": {
"key": "engineRPM",
"value": [[10,0,0],[250,1.0,0.2],[1300,1.0,0],[2000,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[1240,1,0],[2480,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\Engine_in_3960_on.wav",
"volume": {
"key": "engineRPM",
"value": [[1300,0,0],[2000,1.0,0.2],[3100,1.0,0],[3500,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[3960,1,0],[7920,2,0]]
}
}
},
{
"loop": {
"sample": "sounds\\Engine_in_3800_on_in.wav",
"volume": {
"key": "engineRPM",
"value": [[3100,0,0],[3500,1.0,0.2],[4000,1.0,0],[4300,0,0.2]]
},
"pitch": {
"key": "engineRPM",
"value": [[0,0,0],[3800,1,0],[7600,2,0]]
}
}
}
]
}
The example blends the following samples:
Intake and Exhaust Sounds
Using the cone and direction features of the new sound system, it is possible to create intake and exhaust exclusive sound effects:
{
"multi": {
"name": "engine",
"events": [
{
"multi": {
"name": "rear",
"direction": [180,0,0],
"cone": [180,270,0],
"events": [
...
]
}
},
{
"multi": {
"name": "front",
"direction": [0,0,0],
"cone": [180,270,0],
"events": [
...
]
}
},
]
}
}
This example defines two multi-events, one for intake sounds, one for exhaust sounds. Both have a cone, meaning the sound is only audible within a certain cone shape. If it faces away from the camera, it becomes inaudible. The difference between these two events is their direction. One faces forward, the other faces backward. These events now get populated with the usual rev samples, except specific to intake and exhaust sounds.
The example left out the usual volume and pitch examples for demonstration purposes.
Looping
The new system can loop between specific points, rather than looping the whole buffer. This is useful for creating a short 2-5s loop without having to worry about it repeating from the start, which lead to 30s+ loops in the old system.
When a loop like this gets played back, it plays from the beginning. Once it reaches the end, it loops from a specific point. Which exact sample it loops from can be defined in "loopSample", the optional property of a loop event.
For example, adding
"loopSample": 92799,
to a loop event will make it loop at the 92799th sample of the buffer when it reaches the end.
This way of looping is somewhat unconventional, but certainly effective.