This is my thought process without reviewing existing techniques.
The goal is to approach the problem and consider how I might solve it by brainstorming potential solutions. I then plan to improve upon this by reviewing more of the research associated with it to make things even more efficient.
Occlusion
When a sound source is completely blocked from reaching the listener without propagating through an object.
For example: A sound is emitted in a room with a door closed. The listener is in the next room.
Obstruction
When a sound source is partially blocked from reaching the listener, but can do so by sound waves bouncing off surrounding objects.
For example: A sound source is obstructed by a massive stone in a room, with the listener on the other side of the stone.
Potential variables
- Propagation value per obstruction object
- Obstructed objects count from sound source to listener as a real time parameter control (RTPC)
- Number of rays reaching player
- Maximum obstructions allowed (integer value before sound is dropped)
Raycasting to determine dry/wet signal of a reverb and low pass attenuation from sound source
Calculating if an object is obstructed/occluded is computationally expensive, since sounds don’t emit in a single direction. Many different rays would normally be cast from the sound source in to the scene and reflected a set number of times to see how many reach the listener (if any), but we want to reduce the number of rays while maintaining some fidelity to the practice.
Instead, send one ray towards the listener to check for obstruction. And then send 4 rays at 45 degree angles from around the first ray to check for occlusion. The first ray would not reflect, and would instead calculate how many obstacles it passes through until it reaches the listener. The other 4 rays would cast in to their respective directions, and on their first collision be recast directly towards the listener. If they reach the listener then the check would pass. This reduces the number of rays and number of reflections required to approximate if the sound reaches the listener.
NOTE: there are cases where rays may cast continually without finding a collision. A maximum distance would be set for the rays before failing.
Factors affected when a sound is obstructed/occluded
Reverb
Obstructed sounds would have more emphasis on the wet signal caused by reflections from the surrounding environment, since the dry signal is obstructed.
Low Pass
The frequencies of the sound are altered, with more emphasis on the low pass for both obstruction and occlusion. The exception to this is the wet reverb signal containing more higher frequency information.
Percieved Sound Source
The sound may be perceived to be coming from the reflections rather than the original sound source.
NOTE: the following contains potential algorithms to determine sound attributes after these considerations. Some steps are repeated across each consideration but would be combined in actual implementation. Casting the rays for example would make calculations for both reverb and low pass, and would not have to be repeated for each.
Reverb Considerations
Case 1
Sound is not obstructed or occluded
- Proceed as normal
Case 2
Sound is obstructed
- Send sound source to reverb aux bus
- Cast 4 rays using the methodology in ray casting discussion above. Lower the decibel level by a set amount with an RTPC graph for each ray that is not able to reach the player.
- Create an RTPC that will track obstruction count from the ray that is cast directly from sound source to player. For each object that it passes through, lower the dry signal and increase the wet signal (The actual db level will be calculated using the other 4 rays along with the usual attenuation parameter)
Case 3
Sound is occluded
- Cast 4 rays using the methodology in ray casting discussion above. If no rays reach player, don’t apply wet/dry signal adjustments for sound source to player ray.
Low Pass Considerations
Case 1
Sound is not obstructed or occluded
- Proceed as normal
Case 2
Sound is obstructed
- Send sound source to low pass aux bus
- Create an RTPC that will track obstruction count from the ray that is cast directly from sound source to player. For each object that it passes through, grab propagation values of each object (if applicable) and lower the db level with an rtpc. (regular attenuation will be applied as well). We then divide propagation values with the calculated decibel level to get decibel level before attenuation. For each object it passes through, also ramp the low pass until maximum obstructions allowed is met.
For example:
The sound passes through 3 objects.
The level before attenuation is -6db.
We set the decibel level reduction for each obstructed object to be -5db.
The first object has a propagation value of 1.1. The second has a propagation value of 1.4, and the third a propagation value of 1.3.
The total decibel level with obstruction would be:
[-6 + -5(3)] x (1.1)(1.4)(1.7) = -42.042 db before attenuation
The low pass is simpler in that it ramps normally up until the maximum obstructed objects is reached.
Case 3
Sound is occluded
- Use obstruction calculations above.
Current Overall Limitations
- Perceived sound source location is currently not changed but could be considered
- Since the sound source requires 2 busses per sound source, it may not be realistic to split up reverb and low pass effects to act on the original signal individually. Perhaps a compromise between the ordering and relationships of reverb and low pass filtering to the object could be found.
- Each sound source would require its own set of RTPC’s, but we may be able to use only one set and reset them after sending their info for each source. For example. Calculate sound source one and set RTPC values. Send values. Reset values, calculate sound source two and set values. Send values. Reset values. And so on.