Getting the raw audio samples data

[kamen]

Hi Steven!

What do you think of an API extension which provides the data chunks of the currently played audio?

The GetSpectrumData only provides the FFT from the BASS library as far as I was able to explore.

It turned out other algorithms can be used for deriving the spectrum data, which provide better frequence consistency among the spectrum.

[TF3RDL]

pretty much the most necessary feature request for a new API function for analysis-oriented visualizations (oscilloscope, peak/RMS meters, etc.) as some of audio analysis algorithms (including certain spectral analysis algorithms like IIR filter bank and even non-power of two FFTs like 4800 samples long FFT) do require raw audio data (which is an array of floating-point numbers representing a short chunk of raw audio data before being put into FFT) in order to function properly

BTW unless the FFT data MusicBee currently have is complex-valued (which can have negative values as the magnitude is a hypotenuse of real and imaginary parts of FFT data, which can't have negative values), it is not possible to properly implement something like Brown-Puckette CQT/VQT (which uses a frequency kernel derived directly in frequency-domain to sharpen time-resolution at higher frequencies) and reassigned FFT (which I think needs complex-valued FFT) into CoolEdit Nostalgia visualizer

[TF3RDL]

Don't forget that IIR filter bank spectrum analyzer do require delta timing thing (to ensure samples don't overlap or miss) in-addition to an API to get floating-point sample data, since the display framerate is indeed variable (at least for realtime visualizations), so for 60fps display framerate and 48kHz audio samplerate, you need to get approximately 800 samples, and for 30fps, you'll need something like 1600 samples but even a one sample off the required length can cause clicks/pop artifacts to pop up on the filter bank analyzer even if you can't hear these since the visualization doesn't touch the actual audio data (besides reading sample data for audio analysis)

[kamen]

For those interested in using the raw audio samples, I can let you know that using a winamp dsp plugin is a working soultion.
There are several technical limitations such as:  buffer size (which depends on the audio device) and maximal frequency and bit depth (which are limited to 44100khz, 16bit).

[TF3RDL]

For those interested in using the raw audio samples, I can let you know that using a winamp dsp plugin is a working soultion.
There are several technical limitations such as:  buffer size (which depends on the audio device) and maximal frequency and bit depth (which are limited to 44100khz, 16bit).

Fair enough, it is a good workaround while we're waiting for Steven Mayall (the creator of MusicBee) finally adding an API function to get raw samples data (which can have any number of channels, samplerate can be anything, and it is in floating-point format unlike Winamp DSP plugins or the FLAC format) for parity with foobar2000's get_chunk_absolute() function (which is used by oscilloscope components and certain spectrum visualizations like foo_enhanced_spectrum_analyzer)

[Steven]

I have done a proof of concept to implement this, but before I spend more time implementing a proper solution is there anyone who is going to make use of this?
And if so, would you use C++ or C# ? (I recommend C++ for this type thing)
Your plugin would provide a function that MB directly supplies the raw PCM data to, post any DSP modifications, in real time

[TF3RDL]

I have done a proof of concept to implement this, but before I spend more time implementing a proper solution is there anyone who is going to make use of this?

Not only useful for oscilloscope/vectorscope-type visualizations but also certain spectrum analyzers (especially when it uses custom FFT routine or even ditches the FFT entirely like ones that uses IIR filter bank), loudness (LUFS), and even true peak meters (basically peakmeter but sinc-interpolated to detect possible intersample peak clipping)

Obviously, plugins using this would only work on latest MB versions starting from where raw samples data acquisition for visualizations is introduced right?

[TF3RDL]

True. But when you put a float32 WAVs (e.g. coming from bass boosted exports of any audio file including MP3 and other lossy sources, which contains +0dBFS peaks when decoded even before processing) while these Winamp DSPs aren't enabled, the output are limited rather than clipped/distorted right?

I'll have to dig deeper into the topic in future, but for now the float32s are transcoded into 16bits with the DSP (lesser bit depth). So is my understanding, and so is the behavior for the sample file you've prepared.
Also, both - dsp and bass - return values beyond the 0db. Whether limited in some way or not, I currently don't know. But the visual clipping to 0db comes from the CEN plugin.

Is using a Winamp DSP plugin just for working around the MB API currently not having a feature to get sample data, worth risking unintentionally "adulterating" (albeit inaudibly except on extreme bass boosted songs, assuming it is a float32 export in which there are peaks above 0dBFS line as the result of bass boosting process) the audio output by "transcoding" into int16 format (the very same reason behind the transcode warnings I've get on some WAVs when I've try converting uncompressed PCM into FLAC using foobar2000), so that the Windows' built-in audio peak limiter (that only appears when not using WASAPI exclusive output in any supported players) won't kick in because it was hardclipped to 0dB anyway?

BTW, CEN using Winamp DSP just for alternative IIR filter bank algorithm kinda reminds me of DSP Spectrum Tool plugin for Winamp using Winamp's DSP system to get around limitations of Winamp's visualization system (only 576 samples long, low bitdepth of 8-bit integer PCM data "transcoded" from higher bitdepths), though hopefully, MusicBee API adds a feature to get raw samples data, which makes the Winamp DSP as a workaround obsolete

[Steven]

though hopefully, MusicBee API adds a feature to get raw samples data, which makes the Winamp DSP as a workaround obsolete

all i said in the previous post above, i want someone to state that they will make use of it before I do the work to make it available

[kamen]

Hi Steven,

Thank your for taking the time to consider the above suggestions!

I have done a proof of concept to implement this, but before I spend more time implementing a proper solution is there anyone who is going to make use of this?
And if so, would you use C++ or C# ? (I recommend C++ for this type thing)
Your plugin would provide a function that MB directly supplies the raw PCM data to, post any DSP modifications, in real time

I can confirm that a C++ API is the way to go for such a thing, coming from my recent experience with the mentioned CEN DSP plugin. The later versions of C#/.NET also support SIMD acceleration, but cannot confirm whether it is on the same level as C++ in that regard.
Also a C++ API could help with the performance when marshaling the graphics data from the .NET to the native world (given that a plugin gets developed mainly in C++).

all i said in the previous post above, i want someone to state that they will make use of it before I do the work to make it available

I cannot say that the need for such an API is existential, but it unlocks the possibility for a bit more quality for the plugins in the future. Again from my experience with the CEN plugin, this would mean more suitable Fourier transformations for the specific purpose - spectrograms, vu-meters, etc. which brings a bit more quality to the visualizations and expands the options for diverse algorithms.

In short, I'll be happy to use your API, or at least to test it at this early stage.

[Steven]

get the latest v3.6 update and use this updated MusicBeeBass.dll
https://www.mediafire.com/file/sl0k311vrbbs66j/MusicBeeBass.dll/file

In your C++ plugin, export this function name "GetPCMRawData"

Code

__declspec(dllexport) void GetPCMRawData(float*, int);

// called in real time with a copy of current playing stream data, but MB calls this function in a separate thread to protect MB from potential interference from the plugin.
void GetPCMRawData(float* data, int length) {
}

MB will look for that function and use it to send data to it

[kamen]

Thank you Steven!

I managed to execute some initial tests and the GetPCMRawData gets triggered as desired during playback.
An exception when it doesn't get triggered however is when the option "on startup -> resume playback" is activated.

I'll report back when I test the whole dynamics of my plugin and the actual data values.

For my tests I used MB3.6 Patched from 2.09.2024 with the updated MusicBeeBass.dll.

[TF3RDL]

I'll report back when I test the whole dynamics of my plugin and the actual data values.

BTW,  are "proper" Mid/Side (M/S) mode only possible with new API to get raw samples data, or it is already done with built-in function to get spectrum data?

[kamen]

It seems that my post from two days ago is somehow lost during the forum down-time, so I'll try to reproduce it again:

@Steven:
To test the GetPCMRawData, I used a 1khz test signal at 44.1khz sampling rate and compared the results with the dsp api output.
So the results are:
-a couple chunks with normal values
-a couple chunks with too high values (100 to 2000)
-one or two chunks with zeros
The whole pattern repeats for the duration of the audio file.

The test was done with WSAPI shared output, but there is also a corruption pattern with ASIO too.

@TF3RDL: The comparison between the APIs for a specific algorithm has to come at a later stage.

[Steven]

the link has been updated
https://www.mediafire.com/file/sl0k311vrbbs66j/MusicBeeBass.dll/file