Getting the raw audio samples data

[kamen]

Anomaly just noticed:  when using RawAPI 4K, occasionally there is a "blip" of small spectrum at the extreme right end of the kHz scale which seems unrelated to the source freqencies, i.e. not just an occasional very high frequency actually from the source.  This disappears when changing to MusicBee Native Fourier.  Using Linear for scale and interpolation, Normal dB, Time 75ms/60fps  Two shots from bars and regular:

I think you've found something important.

To test it further I set the resemple setting of the MB output to 96khz of a normal 48khz audio file. Then I observed the same artifacts in the zone above 24khz (where there isn't any audio). The artifacts are not present on the DSP source, although the same algorithm pipeline is used for both.

Here are the screenshots (first RAW API, then DSP):



It seems to me that the RAW API can have some kind of synchronization problem. But I'm speculating here. Maybe Steven has more to say on that.

[Steven]

MB just passes through the data that bass sends it. However there is some inter-thread interaction so as a temporary testing thing, this version passes the data directly from the bass callback and not via a separate thread. So if it still has unexpected behavior then its not MB

musicbeebass.dll test file:
https://mega.nz/file/00dkTSTB#6VpGC9YPu2JGKhUhxH_71xefu_7BvMR3EhXoyKC2gT8

[kamen]

Good idea. I couldn't use the test bass, because it crashes for some reason, when I click play.

Through debugging, I've isolated the problem to a floating point rounding issue, which presents very small fractional numbers as 0.
For the silent signal part of the FFT spectrogram the values for rawapi are about e-10 and some occasional zeros, where for the the DSP e-5 and no zeros (due to the higher silence level).
Those zeros then convert into the decibel scale improperly as blips.

So it is in the end a conversion error on my side.
Should it have been a threading problem, some artifacts should be visible also elsewhere on the spectrum (not only the silent part).

[sveakul]

Good idea. I couldn't use the test bass, because it crashes for some reason, when I click play.

Through debugging, I've isolated the problem to a floating point rounding issue, which presents very small fractional numbers as 0.
For the silent signal part of the FFT spectrogram the values for rawapi are about e-10 and some occasional zeros, where for the the DSP e-5 and no zeros (due to the higher silence level).
Those zeros then convert into the decibel scale improperly as blips.

So it is in the end a conversion error on my side.
Should it have been a threading problem, some artifacts should be visible also elsewhere on the spectrum (not only the silent part).

kamen, will you be releasing an incremental update to CoolEdit Nostalgia 5 soon, with those fixes to the RawAPI Bridge mode in place?  The artifacts the error produces in the spectrum, albeit slight, get distracting once you know they are there!

[kamen]

kamen, will you be releasing an incremental update to CoolEdit Nostalgia 5 soon, with those fixes to the RawAPI Bridge mode in place?  The artifacts the error produces in the spectrum, albeit slight, get distracting once you know they are there!

Yes, I want to make a small update together with some other things I'm currently testing. I'll write about it on the plugin's thread when ready.

[sveakul]

Great, thanks!

[TF3RDL]

BTW, the RawAPI thing is not complete without an oscilloscope or even a realtime LUFS meter as visualization plugin for MB

[Mayibongwe]

BTW, the RawAPI thing is not complete without an oscilloscope or even a realtime LUFS meter as visualization plugin for MB

Looks cool. But these features, are they something that you wish to incorporate into a new MusicBee plugin that you are currently developing?
If so, what specific info are you looking to get from the MB API that you cannot currently access?

or are they something that you'd like to see in one of the existing MB plugins (I'm guessing Kamen and BoringName's add-ons)?
If the latter, then you'd first have to check with those developers to see if it's something they are interested in building to their plugins.

If they wouldn't be interested, I don't think it'd be reasonable to ask Steven to spend his limited time enhancing API calls that no user has plans to make use of.

[TF3RDL]

Looks cool. But these features, are they something that you wish to incorporate into a new MusicBee plugin that you are currently developing?
If so, what specific info are you looking to get from the MB API that you cannot currently access?

An API to get raw sample data with arbitrary length should be enough to implement almost anything other than oscilloscope visualization, something like an LUFS meter

or are they something that you'd like to see in one of the existing MB plugins (I'm guessing Kamen and BoringName's add-ons)?
If the latter, then you'd first have to check with those developers to see if it's something they are interested in building to their plugins.

Perhaps either kamen or BoringName (or even someone else who interested in developing plugins for MB and have some needed expertise on DSP especially the LUFS metering) could implement either the realtime LUFS meter (with a reasonable amount of accuracy and compliance with ITU-R BS.1770 standard), the oscilloscope visualization (of course with multichannel support to visualize waveforms of surround sound), or even both

If they wouldn't be interested, I don't think it'd be reasonable to ask Steven to spend his limited time enhancing API calls that no user has plans to make use of.

Yeah, what I meant is that an API to get raw samples data is more than enough and adding a realtime LUFS metering capabilities should be left to plugin developers rather than an API function to fetch LUFS of currently-playing song, which isn't needed as it can be done just by using the waveform data and transform into something that makes sense

[kamen]

For the next update of CEN there will be a simple waveform viewer integrated.

@Steven

While testing various things I came to two issues that relate to the raw api, which are worth mentioning:
1) The raw api skips chunks/calls when the output device uses chunks less than 512 samples ( roughly 20% of the chunks are missing). The DSP however works well, and the sound isn't affected. I used FlexASIO for the tests, as it has configurable latency.
2) The time delivered by Player_GetPosition() seems to be very inaccurate compared to the time interval between the chunks (sometimes showing the same time for whole series of chunks). It would be nice if it can be improved a bit.

[Steven]

this test version directly calls the sample data api without using a background thread
https://www.mediafire.com/file/g8hfzwfvi40e0nn/MusicBeeBass.zip/file

note that if the processing on side is too slow it will cause playback to stutter and if you modify the values in the data array it will change what is sent to the output device

[kamen]

note that if the processing on side is too slow it will cause playback to stutter and if you modify the values in the data array it will change what is sent to the output device

Thank you for that. It turns out the processing time of my callback + any time jitter (due to MB thread, windows, etc.) is about 5-6ms, which obviously becomes critical to maintain at below 512 samples.

I tested that by adding extra sleep time into the callback.Your test MusicBeeBass is only working for C#. It crashes on a C++ plugin, even with empty callback function, so I wasn't able to test it directly on my code.

[Steven]

I am uncertain how the C++ plugin would ever have worked based on what i told you previously. GetPCMRawData needs to be defined

Code

	__declspec(dllexport) void (__stdcall GetPCMRawData(float*, int));
void __stdcall GetPCMRawData(float* data, int length)

[kamen]

With __stdcall the function does not get called at all and for me it makes sense.

All functions that MB calls from the plugin are __cdecl - such as SaveSettings, Close, Uninstall... GetPCMRawData too. (callbacks)
All functions that the plugin calls from MB are __stdcall - such as Player_GetPosition, Player_GetVolume, etc. (the api interface functions)

[Steven]

With __stdcall the function does not get called at all and for me it makes sense.

All functions that MB calls from the plugin are __cdecl - such as SaveSettings, Close, Uninstall... GetPCMRawData too. (callbacks)
All functions that the plugin calls from MB are __stdcall - such as Player_GetPosition, Player_GetVolume, etc. (the api interface functions)

its working fine here as long as you use __stdcall in the GetPCMRawData  definition
the difference with the other examples is they called from within MB itself, so the call can be decorated as a C or std function depending on whether the plugin is a .net or a c++ dll
whereas GetPCMRawData is called from within MusicBeeBass which is a C++ program and could be calling a C# version or a C++ version, so its standardised as a std function call

if you still have issues with the test files, i can have a look if you send me a zip