lba.gitea/WLED
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

audioreactive usermod update (align with MoonMod code) (#2907)

Browse Source 
* audioreactive driver update

- Better handling of PDM and I2S Line-in
- Bugfixes for ES7243 (allocateMultiplePins)
- More error messages for ES7243
- sample scaling (needed for sources that use full scale of samples)

* audiorective update

* align SR_DEBUG with WLED_DEBUG
* optional bandpass filter (needed for PDM mics)
* sample scaling for PDM and Line-In
* small improvements for analog input
* bugfixes and small performance improvements
* code for FFT task refactored, for better readablity. Introduces separate functions for filtering and post-processing
* small improvement for beat detection
* default mic settings can be configured at compile time
* correct mic type if MCU does not support PDM or ADC
* hide analog PIN config if not supported by MCU

* audioreactive updates

- minor updates to source code (see discussion in PR #2907)
- usermod readme improvements

* small readme update

* one think I overlooked

* ok, another edit. Now its final. Hopefully.

* small upps

wrong parameter order in debug message.
This commit is contained in:
Frank 2022-11-28 20:52:33 +01:00 committed by GitHub
parent f7004e7a7c
commit 98138a02e3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 412 additions and 173 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

381
usermods/audioreactive/audio_reactive.h
View File

@ -4,7 +4,7 @@
#include <driver/i2s.h>
#include <driver/adc.h>
#ifndef ESP32
#ifndef ARDUINO_ARCH_ESP32
#error This audio reactive usermod does not support the ESP8266.
#endif
@ -25,38 +25,46 @@
// #define FFT_SAMPLING_LOG // FFT result debugging
// #define SR_DEBUG // generic SR DEBUG messages
#ifdef SR_DEBUG
#define DEBUGSR_PRINT(x) Serial.print(x)
#define DEBUGSR_PRINTLN(x) Serial.println(x)
#define DEBUGSR_PRINTF(x...) Serial.printf(x)
#define DEBUGSR_PRINT(x) DEBUGOUT.print(x)
#define DEBUGSR_PRINTLN(x) DEBUGOUT.println(x)
#define DEBUGSR_PRINTF(x...) DEBUGOUT.printf(x)
#else
#define DEBUGSR_PRINT(x)
#define DEBUGSR_PRINTLN(x)
#define DEBUGSR_PRINTF(x...)
#endif
#if defined(MIC_LOGGER) || defined(FFT_SAMPLING_LOG)
#define PLOT_PRINT(x) DEBUGOUT.print(x)
#define PLOT_PRINTLN(x) DEBUGOUT.println(x)
#define PLOT_PRINTF(x...) DEBUGOUT.printf(x)
#else
#define PLOT_PRINT(x)
#define PLOT_PRINTLN(x)
#define PLOT_PRINTF(x...)
#endif
// use audio source class (ESP32 specific)
#include "audio_source.h"
constexpr i2s_port_t I2S_PORT = I2S_NUM_0;
constexpr int BLOCK_SIZE = 128;
constexpr SRate_t SAMPLE_RATE = 22050; // Base sample rate in Hz - 22Khz is a standard rate. Physical sample time -> 23ms
//constexpr SRate_t SAMPLE_RATE = 16000; // 16kHz - use if FFTtask takes more than 20ms. Physical sample time -> 32ms
//constexpr SRate_t SAMPLE_RATE = 20480; // Base sample rate in Hz - 20Khz is experimental. Physical sample time -> 25ms
//constexpr SRate_t SAMPLE_RATE = 10240; // Base sample rate in Hz - previous default. Physical sample time -> 50ms
#define FFT_MIN_CYCLE 21 // minimum time before FFT task is repeated. Use with 22Khz sampling
//#define FFT_MIN_CYCLE 30 // Use with 16Khz sampling
//#define FFT_MIN_CYCLE 23 // minimum time before FFT task is repeated. Use with 20Khz sampling
//#define FFT_MIN_CYCLE 46 // minimum time before FFT task is repeated. Use with 10Khz sampling
constexpr i2s_port_t I2S_PORT = I2S_NUM_0; // I2S port to use (do not change !)
constexpr int BLOCK_SIZE = 128; // I2S buffer size (samples)
// globals
static uint8_t inputLevel = 128; // UI slider value
static uint8_t soundSquelch = 10; // squelch value for volume reactive routines (config value)
static uint8_t sampleGain = 60; // sample gain (config value)
static uint8_t soundAgc = 0; // Automagic gain control: 0 - none, 1 - normal, 2 - vivid, 3 - lazy (config value)
#ifndef SR_SQUELCH
uint8_t soundSquelch = 10; // squelch value for volume reactive routines (config value)
#else
uint8_t soundSquelch = SR_SQUELCH; // squelch value for volume reactive routines (config value)
#endif
#ifndef SR_GAIN
uint8_t sampleGain = 60; // sample gain (config value)
#else
uint8_t sampleGain = SR_GAIN; // sample gain (config value)
#endif
static uint8_t soundAgc = 1; // Automagic gain control: 0 - none, 1 - normal, 2 - vivid, 3 - lazy (config value)
static uint8_t audioSyncEnabled = 0; // bit field: bit 0 - send, bit 1 - receive (config value)
static bool udpSyncConnected = false; // UDP connection status -> true if connected to multicast group
// user settable parameters for limitSoundDynamics()
static bool limiterOn = true; // bool: enable / disable dynamics limiter
@ -86,11 +94,13 @@ const float agcSampleSmooth[AGC_NUM_PRESETS] = { 1/12.f, 1/6.f, 1/16.f}; //
static AudioSource *audioSource = nullptr;
static volatile bool disableSoundProcessing = false; // if true, sound processing (FFT, filters, AGC) will be suspended. "volatile" as its shared between tasks.
static bool useBandPassFilter = false; // if true, enables a bandpass filter 80Hz-16Khz to remove noise. Applies before FFT.
// audioreactive variables shared with FFT task
static float micDataReal = 0.0f; // MicIn data with full 24bit resolution - lowest 8bit after decimal point
static float multAgc = 1.0f; // sample * multAgc = sampleAgc. Our AGC multiplier
static float sampleAvg = 0.0f; // Smoothed Average sample - sampleAvg < 1 means "quiet" (simple noise gate)
static float sampleAgc = 0.0f; // Smoothed AGC sample
// peak detection
static bool samplePeak = false; // Boolean flag for peak - used in effects. Responding routine may reset this flag. Auto-reset after strip.getMinShowDelay()
@ -106,6 +116,62 @@ static void autoResetPeak(void); // peak auto-reset function
// Begin FFT Code //
////////////////////
// some prototypes, to ensure consistent interfaces
static float mapf(float x, float in_min, float in_max, float out_min, float out_max); // map function for float
static float fftAddAvg(int from, int to); // average of several FFT result bins
void FFTcode(void * parameter); // audio processing task: read samples, run FFT, fill GEQ channels from FFT results
static void runMicFilter(uint16_t numSamples, float *sampleBuffer); // pre-filtering of raw samples (band-pass)
static void postProcessFFTResults(bool noiseGateOpen, int numberOfChannels); // post-processing and post-amp of GEQ channels
#define NUM_GEQ_CHANNELS 16 // number of frequency channels. Don't change !!
static TaskHandle_t FFT_Task = nullptr;
// Table of multiplication factors so that we can even out the frequency response.
static float fftResultPink[NUM_GEQ_CHANNELS] = { 1.70f, 1.71f, 1.73f, 1.78f, 1.68f, 1.56f, 1.55f, 1.63f, 1.79f, 1.62f, 1.80f, 2.06f, 2.47f, 3.35f, 6.83f, 9.55f };
// globals and FFT Output variables shared with animations
static float FFT_MajorPeak = 1.0f; // FFT: strongest (peak) frequency
static float FFT_Magnitude = 0.0f; // FFT: volume (magnitude) of peak frequency
static uint8_t fftResult[NUM_GEQ_CHANNELS]= {0};// Our calculated freq. channel result table to be used by effects
#if defined(WLED_DEBUG) || defined(SR_DEBUG)
static uint64_t fftTime = 0;
static uint64_t sampleTime = 0;
#endif
// FFT Task variables (filtering and post-processing)
static float fftCalc[NUM_GEQ_CHANNELS] = {0.0f}; // Try and normalize fftBin values to a max of 4096, so that 4096/16 = 256.
static float fftAvg[NUM_GEQ_CHANNELS] = {0.0f}; // Calculated frequency channel results, with smoothing (used if dynamics limiter is ON)
#ifdef SR_DEBUG
static float fftResultMax[NUM_GEQ_CHANNELS] = {0.0f}; // A table used for testing to determine how our post-processing is working.
#endif
// audio source parameters and constant
constexpr SRate_t SAMPLE_RATE = 22050; // Base sample rate in Hz - 22Khz is a standard rate. Physical sample time -> 23ms
//constexpr SRate_t SAMPLE_RATE = 16000; // 16kHz - use if FFTtask takes more than 20ms. Physical sample time -> 32ms
//constexpr SRate_t SAMPLE_RATE = 20480; // Base sample rate in Hz - 20Khz is experimental. Physical sample time -> 25ms
//constexpr SRate_t SAMPLE_RATE = 10240; // Base sample rate in Hz - previous default. Physical sample time -> 50ms
#define FFT_MIN_CYCLE 21 // minimum time before FFT task is repeated. Use with 22Khz sampling
//#define FFT_MIN_CYCLE 30 // Use with 16Khz sampling
//#define FFT_MIN_CYCLE 23 // minimum time before FFT task is repeated. Use with 20Khz sampling
//#define FFT_MIN_CYCLE 46 // minimum time before FFT task is repeated. Use with 10Khz sampling
// FFT Constants
constexpr uint16_t samplesFFT = 512; // Samples in an FFT batch - This value MUST ALWAYS be a power of 2
constexpr uint16_t samplesFFT_2 = 256; // meaningfull part of FFT results - only the "lower half" contains useful information.
// the following are observed values, supported by a bit of "educated guessing"
//#define FFT_DOWNSCALE 0.65f // 20kHz - downscaling factor for FFT results - "Flat-Top" window @20Khz, old freq channels
#define FFT_DOWNSCALE 0.46f // downscaling factor for FFT results - for "Flat-Top" window @22Khz, new freq channels
#define LOG_256 5.54517744f // log(256)
// These are the input and output vectors. Input vectors receive computed results from FFT.
static float vReal[samplesFFT] = {0.0f}; // FFT sample inputs / freq output - these are our raw result bins
static float vImag[samplesFFT] = {0.0f}; // imaginary parts
#ifdef UM_AUDIOREACTIVE_USE_NEW_FFT
static float windowWeighingFactors[samplesFFT] = {0.0f};
#endif
// Create FFT object
#ifdef UM_AUDIOREACTIVE_USE_NEW_FFT
// lib_deps += https://github.com/kosme/arduinoFFT#develop @ 1.9.2
#define FFT_SPEED_OVER_PRECISION // enables use of reciprocals (1/x etc), and an a few other speedups
@ -116,58 +182,20 @@ static void autoResetPeak(void); // peak auto-reset function
#endif
#include <arduinoFFT.h>
// FFT Output variables shared with animations
#define NUM_GEQ_CHANNELS 16 // number of frequency channels. Don't change !!
static float FFT_MajorPeak = 1.0f; // FFT: strongest (peak) frequency
static float FFT_Magnitude = 0.0f; // FFT: volume (magnitude) of peak frequency
static uint8_t fftResult[NUM_GEQ_CHANNELS]= {0};// Our calculated freq. channel result table to be used by effects
// FFT Constants
constexpr uint16_t samplesFFT = 512; // Samples in an FFT batch - This value MUST ALWAYS be a power of 2
constexpr uint16_t samplesFFT_2 = 256; // meaningfull part of FFT results - only the "lower half" contains useful information.
// These are the input and output vectors. Input vectors receive computed results from FFT.
static float vReal[samplesFFT] = {0.0f}; // FFT sample inputs / freq output - these are our raw result bins
static float vImag[samplesFFT] = {0.0f}; // imaginary parts
// the following are observed values, supported by a bit of "educated guessing"
//#define FFT_DOWNSCALE 0.65f // 20kHz - downscaling factor for FFT results - "Flat-Top" window @20Khz, old freq channels
#define FFT_DOWNSCALE 0.46f // downscaling factor for FFT results - for "Flat-Top" window @22Khz, new freq channels
#define LOG_256 5.54517744
#ifdef UM_AUDIOREACTIVE_USE_NEW_FFT
static float windowWeighingFactors[samplesFFT] = {0.0f};
#endif
// Try and normalize fftBin values to a max of 4096, so that 4096/16 = 256.
static float fftCalc[NUM_GEQ_CHANNELS] = {0.0f};
static float fftAvg[NUM_GEQ_CHANNELS] = {0.0f}; // Calculated frequency channel results, with smoothing (used if dynamics limiter is ON)
#ifdef SR_DEBUG
static float fftResultMax[NUM_GEQ_CHANNELS] = {0.0f}; // A table used for testing to determine how our post-processing is working.
#endif
#if defined(WLED_DEBUG) || defined(SR_DEBUG)
static uint64_t fftTime = 0;
static uint64_t sampleTime = 0;
#endif
// Table of multiplication factors so that we can even out the frequency response.
static float fftResultPink[NUM_GEQ_CHANNELS] = { 1.70f, 1.71f, 1.73f, 1.78f, 1.68f, 1.56f, 1.55f, 1.63f, 1.79f, 1.62f, 1.80f, 2.06f, 2.47f, 3.35f, 6.83f, 9.55f };
// Create FFT object
#ifdef UM_AUDIOREACTIVE_USE_NEW_FFT
static ArduinoFFT<float> FFT = ArduinoFFT<float>( vReal, vImag, samplesFFT, SAMPLE_RATE, windowWeighingFactors);
#else
static arduinoFFT FFT = arduinoFFT(vReal, vImag, samplesFFT, SAMPLE_RATE);
#endif
static TaskHandle_t FFT_Task = nullptr;
// Helper functions
// float version of map()
static float mapf(float x, float in_min, float in_max, float out_min, float out_max){
return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}
// compute average of several FFT resut bins
static float fftAddAvg(int from, int to) {
float result = 0.0f;
for (int i = from; i <= to; i++) {
@ -176,7 +204,9 @@ static float fftAddAvg(int from, int to) {
return result / float(to - from + 1);
}
//
// FFT main task
//
void FFTcode(void * parameter)
{
DEBUGSR_PRINT("FFT started on core: "); DEBUGSR_PRINTLN(xPortGetCoreID());
@ -213,6 +243,10 @@ void FFTcode(void * parameter)
xLastWakeTime = xTaskGetTickCount(); // update "last unblocked time" for vTaskDelay
// band pass filter - can reduce noise floor by a factor of 50
// downside: frequencies below 100Hz will be ignored
if (useBandPassFilter) runMicFilter(samplesFFT, vReal);
// find highest sample in the batch
float maxSample = 0.0f; // max sample from FFT batch
for (int i=0; i < samplesFFT; i++) {
@ -229,7 +263,7 @@ void FFTcode(void * parameter)
#ifdef SR_DEBUG
if (true) { // this allows measure FFT runtimes, as it disables the "only when needed" optimization
#else
if (sampleAvg > 0.5f) { // noise gate open means that FFT results will be used. Don't run FFT if results are not needed.
if (sampleAvg > 0.25f) { // noise gate open means that FFT results will be used. Don't run FFT if results are not needed.
#endif
// run FFT (takes 3-5ms on ESP32, ~12ms on ESP32-S2)
@ -273,7 +307,7 @@ void FFTcode(void * parameter)
} // for()
// mapping of FFT result bins to frequency channels
if (sampleAvg > 0.5f) { // noise gate open
if (fabsf(sampleAvg) > 0.5f) { // noise gate open
#if 0
/* This FFT post processing is a DIY endeavour. What we really need is someone with sound engineering expertise to do a great job here AND most importantly, that the animations look GREAT as a result.
*
@ -303,10 +337,22 @@ void FFTcode(void * parameter)
#else
/* new mapping, optimized for 22050 Hz by softhack007 */
// bins frequency range
fftCalc[ 0] = fftAddAvg(1,2); // 1 43 - 86 sub-bass
fftCalc[ 1] = fftAddAvg(2,3); // 1 86 - 129 bass
fftCalc[ 2] = fftAddAvg(3,5); // 2 129 - 216 bass
fftCalc[ 3] = fftAddAvg(5,7); // 2 216 - 301 bass + midrange
if (useBandPassFilter) {
// skip frequencies below 100hz
fftCalc[ 0] = 0.8f * fftAddAvg(3,4);
fftCalc[ 1] = 0.9f * fftAddAvg(4,5);
fftCalc[ 2] = fftAddAvg(5,6);
fftCalc[ 3] = fftAddAvg(6,7);
// don't use the last bins from 206 to 255.
fftCalc[15] = fftAddAvg(165,205) * 0.75f; // 40 7106 - 8828 high -- with some damping
} else {
fftCalc[ 0] = fftAddAvg(1,2); // 1 43 - 86 sub-bass
fftCalc[ 1] = fftAddAvg(2,3); // 1 86 - 129 bass
fftCalc[ 2] = fftAddAvg(3,5); // 2 129 - 216 bass
fftCalc[ 3] = fftAddAvg(5,7); // 2 216 - 301 bass + midrange
// don't use the last bins from 216 to 255. They are usually contaminated by aliasing (aka noise)
fftCalc[15] = fftAddAvg(165,215) * 0.70f; // 50 7106 - 9259 high -- with some damping
}
fftCalc[ 4] = fftAddAvg(7,10); // 3 301 - 430 midrange
fftCalc[ 5] = fftAddAvg(10,13); // 3 430 - 560 midrange
fftCalc[ 6] = fftAddAvg(13,19); // 5 560 - 818 midrange
@ -318,8 +364,6 @@ void FFTcode(void * parameter)
fftCalc[12] = fftAddAvg(70,86); // 16 3015 - 3704 high mid
fftCalc[13] = fftAddAvg(86,104); // 18 3704 - 4479 high mid
fftCalc[14] = fftAddAvg(104,165) * 0.88f; // 61 4479 - 7106 high mid + high -- with slight damping
fftCalc[15] = fftAddAvg(165,215) * 0.70f; // 50 7106 - 9259 high -- with some damping
// don't use the last bins from 216 to 255. They are usually contaminated by aliasing (aka noise)
#endif
} else { // noise gate closed - just decay old values
for (int i=0; i < NUM_GEQ_CHANNELS; i++) {
@ -329,9 +373,67 @@ void FFTcode(void * parameter)
}
// post-processing of frequency channels (pink noise adjustment, AGC, smooting, scaling)
for (int i=0; i < NUM_GEQ_CHANNELS; i++) {
postProcessFFTResults((fabsf(sampleAvg) > 0.25f)? true : false , NUM_GEQ_CHANNELS);
if (sampleAvg > 0.5f) { // noise gate open
#if defined(WLED_DEBUG) || defined(SR_DEBUG)
if (haveDoneFFT && (start < esp_timer_get_time())) { // filter out overflows
uint64_t fftTimeInMillis = ((esp_timer_get_time() - start) +5ULL) / 10ULL; // "+5" to ensure proper rounding
fftTime = (fftTimeInMillis*3 + fftTime*7)/10; // smooth
}
#endif
// run peak detection
autoResetPeak();
detectSamplePeak();
#if !defined(I2S_GRAB_ADC1_COMPLETELY)
if ((audioSource == nullptr) || (audioSource->getType() != AudioSource::Type_I2SAdc)) // the "delay trick" does not help for analog ADC
#endif
vTaskDelayUntil( &xLastWakeTime, xFrequency); // release CPU, and let I2S fill its buffers
} // for(;;)ever
} // FFTcode() task end
///////////////////////////
// Pre / Postprocessing //
///////////////////////////
static void runMicFilter(uint16_t numSamples, float *sampleBuffer) // pre-filtering of raw samples (band-pass)
{
// low frequency cutoff parameter - see https://dsp.stackexchange.com/questions/40462/exponential-moving-average-cut-off-frequency
//constexpr float alpha = 0.04f; // 150Hz
//constexpr float alpha = 0.03f; // 110Hz
constexpr float alpha = 0.0225f; // 80hz
//constexpr float alpha = 0.01693f;// 60hz
// high frequency cutoff parameter
//constexpr float beta1 = 0.75f; // 11Khz
//constexpr float beta1 = 0.82f; // 15Khz
//constexpr float beta1 = 0.8285f; // 18Khz
constexpr float beta1 = 0.85f; // 20Khz
constexpr float beta2 = (1.0f - beta1) / 2.0;
static float last_vals[2] = { 0.0f }; // FIR high freq cutoff filter
static float lowfilt = 0.0f; // IIR low frequency cutoff filter
for (int i=0; i < numSamples; i++) {
// FIR lowpass, to remove high frequency noise
float highFilteredSample;
if (i < (numSamples-1)) highFilteredSample = beta1*sampleBuffer[i] + beta2*last_vals[0] + beta2*sampleBuffer[i+1]; // smooth out spikes
else highFilteredSample = beta1*sampleBuffer[i] + beta2*last_vals[0] + beta2*last_vals[1]; // spcial handling for last sample in array
last_vals[1] = last_vals[0];
last_vals[0] = sampleBuffer[i];
sampleBuffer[i] = highFilteredSample;
// IIR highpass, to remove low frequency noise
lowfilt += alpha * (sampleBuffer[i] - lowfilt);
sampleBuffer[i] = sampleBuffer[i] - lowfilt;
}
}
static void postProcessFFTResults(bool noiseGateOpen, int numberOfChannels) // post-processing and post-amp of GEQ channels
{
for (int i=0; i < numberOfChannels; i++) {
if (noiseGateOpen) { // noise gate open
// Adjustment for frequency curves.
fftCalc[i] *= fftResultPink[i];
if (FFTScalingMode > 0) fftCalc[i] *= FFT_DOWNSCALE; // adjustment related to FFT windowing function
@ -401,36 +503,23 @@ void FFTcode(void * parameter)
}
fftResult[i] = constrain((int)currentResult, 0, 255);
}
#if defined(WLED_DEBUG) || defined(SR_DEBUG)
if (haveDoneFFT && (start < esp_timer_get_time())) { // filter out overflows
uint64_t fftTimeInMillis = ((esp_timer_get_time() - start) +5ULL) / 10ULL; // "+5" to ensure proper rounding
fftTime = (fftTimeInMillis*3 + fftTime*7)/10; // smooth
}
#endif
// run peak detection
autoResetPeak();
detectSamplePeak();
#if !defined(I2S_GRAB_ADC1_COMPLETELY)
if ((audioSource == nullptr) || (audioSource->getType() != AudioSource::Type_I2SAdc)) // the "delay trick" does not help for analog ADC
#endif
vTaskDelayUntil( &xLastWakeTime, xFrequency); // release CPU, and let I2S fill its buffers
} // for(;;)ever
} // FFTcode() task end
}
////////////////////
// Peak detection //
////////////////////
// peak detection is called from FFT task when vReal[] contains valid FFT results
static void detectSamplePeak(void) {
bool havePeak = false;
// Poor man's beat detection by seeing if sample > Average + some value.
// This goes through ALL of the 255 bins - but ignores stupid settings
// Then we got a peak, else we don't. The peak has to time out on its own in order to support UDP sound sync.
if ((sampleAvg > 1) && (maxVol > 0) && (binNum > 1) && (vReal[binNum] > maxVol) && ((millis() - timeOfPeak) > 100)) {
// This goes through ALL of the 255 bins - but ignores stupid settings
// Then we got a peak, else we don't. The peak has to time out on its own in order to support UDP sound sync.
havePeak = true;
}
if (havePeak) {
samplePeak = true;
timeOfPeak = millis();
udpSamplePeak = true;
@ -459,10 +548,11 @@ class AudioReactive : public Usermod {
#else
int8_t audioPin = AUDIOPIN;
#endif
#ifndef DMTYPE // I2S mic type
#ifndef SR_DMTYPE // I2S mic type
uint8_t dmType = 1; // 0=none/disabled/analog; 1=generic I2S
#define SR_DMTYPE 1 // default type = I2S
#else
uint8_t dmType = DMTYPE;
uint8_t dmType = SR_DMTYPE;
#endif
#ifndef I2S_SDPIN // aka DOUT
int8_t i2ssdPin = 32;
@ -526,7 +616,6 @@ class AudioReactive : public Usermod {
// variables for UDP sound sync
WiFiUDP fftUdp; // UDP object for sound sync (from WiFi UDP, not Async UDP!)
bool udpSyncConnected = false;// UDP connection status -> true if connected to multicast group
unsigned long lastTime = 0; // last time of running UDP Microphone Sync
const uint16_t delayMs = 10; // I don't want to sample too often and overload WLED
uint16_t audioSyncPort= 11988;// default port for UDP sound sync
@ -538,15 +627,14 @@ class AudioReactive : public Usermod {
// variables used by getSample() and agcAvg()
int16_t micIn = 0; // Current sample starts with negative values and large values, which is why it's 16 bit signed
double sampleMax = 0.0; // Max sample over a few seconds. Needed for AGC controler.
float micLev = 0.0f; // Used to convert returned value to have '0' as minimum. A leveller
double micLev = 0.0; // Used to convert returned value to have '0' as minimum. A leveller
float expAdjF = 0.0f; // Used for exponential filter.
float sampleReal = 0.0f; // "sampleRaw" as float, to provide bits that are lost otherwise (before amplification by sampleGain or inputLevel). Needed for AGC.
int16_t sampleRaw = 0; // Current sample. Must only be updated ONCE!!! (amplified mic value by sampleGain and inputLevel)
int16_t rawSampleAgc = 0; // not smoothed AGC sample
float sampleAgc = 0.0f; // Smoothed AGC sample
// variables used in effects
float volumeSmth = 0.0f; // either sampleAvg or sampleAgc depending on soundAgc; smoothed sample
float volumeSmth = 0.0f; // either sampleAvg or sampleAgc depending on soundAgc; smoothed sample
int16_t volumeRaw = 0; // either sampleRaw or rawSampleAgc depending on soundAgc
float my_magnitude =0.0f; // FFT_Magnitude, scaled by multAgc
@ -576,28 +664,28 @@ class AudioReactive : public Usermod {
if (disableSoundProcessing && (!udpSyncConnected || ((audioSyncEnabled & 0x02) == 0))) return; // no audio availeable
#ifdef MIC_LOGGER
// Debugging functions for audio input and sound processing. Comment out the values you want to see
Serial.print("micReal:"); Serial.print(micDataReal); Serial.print("\t");
Serial.print("volumeSmth:"); Serial.print(volumeSmth); Serial.print("\t");
//Serial.print("volumeRaw:"); Serial.print(volumeRaw); Serial.print("\t");
//Serial.print("DC_Level:"); Serial.print(micLev); Serial.print("\t");
//Serial.print("sampleAgc:"); Serial.print(sampleAgc); Serial.print("\t");
//Serial.print("sampleAvg:"); Serial.print(sampleAvg); Serial.print("\t");
//Serial.print("sampleReal:"); Serial.print(sampleReal); Serial.print("\t");
//Serial.print("micIn:"); Serial.print(micIn); Serial.print("\t");
//Serial.print("sample:"); Serial.print(sample); Serial.print("\t");
//Serial.print("sampleMax:"); Serial.print(sampleMax); Serial.print("\t");
//Serial.print("samplePeak:"); Serial.print((samplePeak!=0) ? 128:0); Serial.print("\t");
//Serial.print("multAgc:"); Serial.print(multAgc, 4); Serial.print("\t");
Serial.println();
PLOT_PRINT("micReal:"); PLOT_PRINT(micDataReal); PLOT_PRINT("\t");
PLOT_PRINT("volumeSmth:"); PLOT_PRINT(volumeSmth); PLOT_PRINT("\t");
//PLOT_PRINT("volumeRaw:"); PLOT_PRINT(volumeRaw); PLOT_PRINT("\t");
PLOT_PRINT("DC_Level:"); PLOT_PRINT(micLev); PLOT_PRINT("\t");
//PLOT_PRINT("sampleAgc:"); PLOT_PRINT(sampleAgc); PLOT_PRINT("\t");
//PLOT_PRINT("sampleAvg:"); PLOT_PRINT(sampleAvg); PLOT_PRINT("\t");
//PLOT_PRINT("sampleReal:"); PLOT_PRINT(sampleReal); PLOT_PRINT("\t");
//PLOT_PRINT("micIn:"); PLOT_PRINT(micIn); PLOT_PRINT("\t");
//PLOT_PRINT("sample:"); PLOT_PRINT(sample); PLOT_PRINT("\t");
//PLOT_PRINT("sampleMax:"); PLOT_PRINT(sampleMax); PLOT_PRINT("\t");
//PLOT_PRINT("samplePeak:"); PLOT_PRINT((samplePeak!=0) ? 128:0); PLOT_PRINT("\t");
//PLOT_PRINT("multAgc:"); PLOT_PRINT(multAgc, 4); PLOT_PRINT("\t");
PLOT_PRINTLN();
#endif
#ifdef FFT_SAMPLING_LOG
#if 0
for(int i=0; i<NUM_GEQ_CHANNELS; i++) {
Serial.print(fftResult[i]);
Serial.print("\t");
PLOT_PRINT(fftResult[i]);
PLOT_PRINT("\t");
}
Serial.println();
PLOT_PRINTLN();
#endif
// OPTIONS are in the following format: Description \n Option
@ -624,20 +712,21 @@ class AudioReactive : public Usermod {
if(fftResult[i] < minVal) minVal = fftResult[i];
}
for(int i = 0; i < NUM_GEQ_CHANNELS; i++) {
Serial.print(i); Serial.print(":");
Serial.printf("%04ld ", map(fftResult[i], 0, (scaleValuesFromCurrentMaxVal ? maxVal : defaultScalingFromHighValue), (mapValuesToPlotterSpace*i*scalingToHighValue)+0, (mapValuesToPlotterSpace*i*scalingToHighValue)+scalingToHighValue-1));
PLOT_PRINT(i); PLOT_PRINT(":");
PLOT_PRINTF("%04ld ", map(fftResult[i], 0, (scaleValuesFromCurrentMaxVal ? maxVal : defaultScalingFromHighValue), (mapValuesToPlotterSpace*i*scalingToHighValue)+0, (mapValuesToPlotterSpace*i*scalingToHighValue)+scalingToHighValue-1));
}
if(printMaxVal) {
Serial.printf("maxVal:%04d ", maxVal + (mapValuesToPlotterSpace ? 16*256 : 0));
PLOT_PRINTF("maxVal:%04d ", maxVal + (mapValuesToPlotterSpace ? 16*256 : 0));
}
if(printMinVal) {
Serial.printf("%04d:minVal ", minVal); // printed with value first, then label, so negative values can be seen in Serial Monitor but don't throw off y axis in Serial Plotter
PLOT_PRINTF("%04d:minVal ", minVal); // printed with value first, then label, so negative values can be seen in Serial Monitor but don't throw off y axis in Serial Plotter
}
if(mapValuesToPlotterSpace)
Serial.printf("max:%04d ", (printMaxVal ? 17 : 16)*256); // print line above the maximum value we expect to see on the plotter to avoid autoscaling y axis
else
Serial.printf("max:%04d ", 256);
Serial.println();
PLOT_PRINTF("max:%04d ", (printMaxVal ? 17 : 16)*256); // print line above the maximum value we expect to see on the plotter to avoid autoscaling y axis
else {
PLOT_PRINTF("max:%04d ", 256);
}
PLOT_PRINTLN();
#endif // FFT_SAMPLING_LOG
} // logAudio()
@ -753,7 +842,7 @@ class AudioReactive : public Usermod {
micIn = inoise8(millis(), millis()); // Simulated analog read
micDataReal = micIn;
#else
#ifdef ESP32
#ifdef ARDUINO_ARCH_ESP32
micIn = int(micDataReal); // micDataSm = ((micData * 3) + micData)/4;
#else
// this is the minimal code for reading analog mic input on 8266.
@ -770,13 +859,13 @@ class AudioReactive : public Usermod {
#endif
#endif
micLev = ((micLev * 8191.0f) + micDataReal) / 8192.0f; // takes a few seconds to "catch up" with the Mic Input
micLev += (micDataReal-micLev) / 12288.0f;
if(micIn < micLev) micLev = ((micLev * 31.0f) + micDataReal) / 32.0f; // align MicLev to lowest input signal
micIn -= micLev; // Let's center it to 0 now
// Using an exponential filter to smooth out the signal. We'll add controls for this in a future release.
float micInNoDC = fabsf(micDataReal - micLev);
expAdjF = (weighting * micInNoDC + (1.0-weighting) * expAdjF);
expAdjF = (weighting * micInNoDC + (1.0f-weighting) * expAdjF);
expAdjF = fabsf(expAdjF); // Now (!) take the absolute value
expAdjF = (expAdjF <= soundSquelch) ? 0: expAdjF; // simple noise gate
@ -794,6 +883,12 @@ class AudioReactive : public Usermod {
// keep "peak" sample, but decay value if current sample is below peak
if ((sampleMax < sampleReal) && (sampleReal > 0.5f)) {
sampleMax = sampleMax + 0.5f * (sampleReal - sampleMax); // new peak - with some filtering
// another simple way to detect samplePeak
if ((binNum < 10) && (millis() - timeOfPeak > 80) && (sampleAvg > 1)) {
samplePeak = true;
timeOfPeak = millis();
udpSamplePeak = true;
}
} else {
if ((multAgc*sampleMax > agcZoneStop[AGC_preset]) && (soundAgc > 0))
sampleMax += 0.5f * (sampleReal - sampleMax); // over AGC Zone - get back quickly
@ -1015,11 +1110,14 @@ class AudioReactive : public Usermod {
}
// Reset I2S peripheral for good measure
i2s_driver_uninstall(I2S_NUM_0);
i2s_driver_uninstall(I2S_NUM_0); // E (696) I2S: i2s_driver_uninstall(2006): I2S port 0 has not installed
#if !defined(CONFIG_IDF_TARGET_ESP32C3)
delay(100);
periph_module_reset(PERIPH_I2S0_MODULE); // not possible on -C3
#endif
delay(100); // Give that poor microphone some time to setup.
useBandPassFilter = false;
switch (dmType) {
#if defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C3) || defined(CONFIG_IDF_TARGET_ESP32S3)
// stub cases for not-yet-supported I2S modes on other ESP32 chips
@ -1048,14 +1146,15 @@ class AudioReactive : public Usermod {
break;
case 4:
DEBUGSR_PRINT(F("AR: Generic I2S Microphone with Master Clock - ")); DEBUGSR_PRINTLN(F(I2S_MIC_CHANNEL_TEXT));
audioSource = new I2SSource(SAMPLE_RATE, BLOCK_SIZE);
audioSource = new I2SSource(SAMPLE_RATE, BLOCK_SIZE, 1.0f/24.0f);
delay(100);
if (audioSource) audioSource->initialize(i2swsPin, i2ssdPin, i2sckPin, mclkPin);
break;
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3)
case 5:
DEBUGSR_PRINT(F("AR: I2S PDM Microphone - ")); DEBUGSR_PRINTLN(F(I2S_MIC_CHANNEL_TEXT));
audioSource = new I2SSource(SAMPLE_RATE, BLOCK_SIZE);
DEBUGSR_PRINT(F("AR: I2S PDM Microphone - ")); DEBUGSR_PRINTLN(F(I2S_PDM_MIC_CHANNEL_TEXT));
audioSource = new I2SSource(SAMPLE_RATE, BLOCK_SIZE, 1.0f/4.0f);
useBandPassFilter = true; // this reduces the noise floor on SPM1423 from 5% Vpp (~380) down to 0.05% Vpp (~5)
delay(100);
if (audioSource) audioSource->initialize(i2swsPin, i2ssdPin);
break;
@ -1079,7 +1178,11 @@ class AudioReactive : public Usermod {
if (enabled) disableSoundProcessing = false; // all good - enable audio processing
if((!audioSource) || (!audioSource->isInitialized())) { // audio source failed to initialize. Still stay "enabled", as there might be input arriving via UDP Sound Sync
#ifdef WLED_DEBUG
DEBUG_PRINTLN(F("AR: Failed to initialize sound input driver. Please check input PIN settings."));
#else
DEBUGSR_PRINTLN(F("AR: Failed to initialize sound input driver. Please check input PIN settings."));
#endif
disableSoundProcessing = true;
}
@ -1353,10 +1456,11 @@ class AudioReactive : public Usermod {
if (enabled) {
// Input Level Slider
if (disableSoundProcessing == false) { // only show slider when audio processing is running
if (soundAgc > 0)
if (soundAgc > 0) {
infoArr = user.createNestedArray(F("GEQ Input Level")); // if AGC is on, this slider only affects fftResult[] frequencies
else
} else {
infoArr = user.createNestedArray(F("Audio Input Level"));
}
uiDomString = F("<div class=\"slider\"><div class=\"sliderwrap il\"><input class=\"noslide\" onchange=\"requestJson({");
uiDomString += FPSTR(_name);
uiDomString += F(":{");
@ -1450,7 +1554,7 @@ class AudioReactive : public Usermod {
infoArr.add(" ms");
infoArr = user.createNestedArray(F("FFT time"));
infoArr.add(float(fftTime)/100.0f);
infoArr.add(float(fftTime)/100.0f);
if ((fftTime/100) >= FFT_MIN_CYCLE) // FFT time over budget -> I2S buffer will overflow
infoArr.add("<b style=\"color:red;\">! ms</b>");
else if ((fftTime/80 + sampleTime/80) >= FFT_MIN_CYCLE) // FFT time >75% of budget -> risk of instability
@ -1541,8 +1645,10 @@ class AudioReactive : public Usermod {
JsonObject top = root.createNestedObject(FPSTR(_name));
top[FPSTR(_enabled)] = enabled;
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3) && !defined(CONFIG_IDF_TARGET_ESP32S3)
JsonObject amic = top.createNestedObject(FPSTR(_analogmic));
amic["pin"] = audioPin;
#endif
JsonObject dmic = top.createNestedObject(FPSTR(_digitalmic));
dmic[F("type")] = dmType;
@ -1595,9 +1701,20 @@ class AudioReactive : public Usermod {
configComplete &= getJsonValue(top[FPSTR(_enabled)], enabled);
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3) && !defined(CONFIG_IDF_TARGET_ESP32S3)
configComplete &= getJsonValue(top[FPSTR(_analogmic)]["pin"], audioPin);
#else
audioPin = -1; // MCU does not support analog mic
#endif
configComplete &= getJsonValue(top[FPSTR(_digitalmic)]["type"], dmType);
#if defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C3) || defined(CONFIG_IDF_TARGET_ESP32S3)
if (dmType == 0) dmType = SR_DMTYPE; // MCU does not support analog
#if defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C3)
if (dmType == 5) dmType = SR_DMTYPE; // MCU does not support PDM
#endif
#endif
configComplete &= getJsonValue(top[FPSTR(_digitalmic)]["pin"][0], i2ssdPin);
configComplete &= getJsonValue(top[FPSTR(_digitalmic)]["pin"][1], i2swsPin);
configComplete &= getJsonValue(top[FPSTR(_digitalmic)]["pin"][2], i2sckPin);

143
usermods/audioreactive/audio_source.h
View File

@ -23,11 +23,15 @@
// see https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/chip-series-comparison.html#related-documents
// and https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/peripherals/i2s.html#overview-of-all-modes
#if defined(CONFIG_IDF_TARGET_ESP32C2) || defined(CONFIG_IDF_TARGET_ESP32C3) || defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C5) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32H2)
#if defined(CONFIG_IDF_TARGET_ESP32C2) || defined(CONFIG_IDF_TARGET_ESP32C3) || defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32C5) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32H2) || defined(ESP8266) || defined(ESP8265)
// there are two things in these MCUs that could lead to problems with audio processing:
// * no floating point hardware (FPU) support - FFT uses float calculations. If done in software, a strong slow-down can be expected (between 8x and 20x)
// * single core, so FFT task might slow down other things like LED updates
#if !defined(SOC_I2S_NUM) || (SOC_I2S_NUM < 1)
#error This audio reactive usermod does not support ESP32-C2, ESP32-C3 or ESP32-S2.
#else
#warning This audio reactive usermod does not support ESP32-C2, ESP32-C3 or ESP32-S2.
#endif
#endif
/* ToDo: remove. ES7243 is controlled via compiler defines
@ -76,11 +80,15 @@
#ifdef I2S_USE_RIGHT_CHANNEL
#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_LEFT
#define I2S_MIC_CHANNEL_TEXT "right channel only (work-around swapped channel bug in IDF 4.4)."
#define I2S_PDM_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_RIGHT
#define I2S_PDM_MIC_CHANNEL_TEXT "right channel only"
#else
//#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ALL_LEFT
//#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_RIGHT_LEFT
#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_RIGHT
#define I2S_MIC_CHANNEL_TEXT "left channel only (work-around swapped channel bug in IDF 4.4)."
#define I2S_PDM_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_LEFT
#define I2S_PDM_MIC_CHANNEL_TEXT "left channel only."
#endif
#else
@ -92,6 +100,9 @@
#define I2S_MIC_CHANNEL I2S_CHANNEL_FMT_ONLY_LEFT
#define I2S_MIC_CHANNEL_TEXT "left channel only."
#endif
#define I2S_PDM_MIC_CHANNEL I2S_MIC_CHANNEL
#define I2S_PDM_MIC_CHANNEL_TEXT I2S_MIC_CHANNEL_TEXT
#endif
@ -138,15 +149,17 @@ class AudioSource {
virtual I2S_datatype postProcessSample(I2S_datatype sample_in) {return(sample_in);} // default method can be overriden by instances (ADC) that need sample postprocessing
// Private constructor, to make sure it is not callable except from derived classes
AudioSource(SRate_t sampleRate, int blockSize) :
AudioSource(SRate_t sampleRate, int blockSize, float sampleScale) :
_sampleRate(sampleRate),
_blockSize(blockSize),
_initialized(false)
_initialized(false),
_sampleScale(sampleScale)
{};
SRate_t _sampleRate; // Microphone sampling rate
int _blockSize; // I2S block size
bool _initialized; // Gets set to true if initialization is successful
float _sampleScale; // pre-scaling factor for I2S samples
};
/* Basic I2S microphone source
@ -154,8 +167,8 @@ class AudioSource {
*/
class I2SSource : public AudioSource {
public:
I2SSource(SRate_t sampleRate, int blockSize) :
AudioSource(sampleRate, blockSize) {
I2SSource(SRate_t sampleRate, int blockSize, float sampleScale = 1.0f) :
AudioSource(sampleRate, blockSize, sampleScale) {
_config = {
.mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX),
.sample_rate = _sampleRate,
@ -195,18 +208,51 @@ class I2SSource : public AudioSource {
return;
}
} else {
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 2, 0)
#if !defined(SOC_I2S_SUPPORTS_PDM_RX)
#warning this MCU does not support PDM microphones
#endif
#endif
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3)
// This is an I2S PDM microphone, these microphones only use a clock and
// data line, to make it simpler to debug, use the WS pin as CLK and SD
// pin as DATA
// data line, to make it simpler to debug, use the WS pin as CLK and SD pin as DATA
// example from espressif: https://github.com/espressif/esp-idf/blob/release/v4.4/examples/peripherals/i2s/i2s_audio_recorder_sdcard/main/i2s_recorder_main.c
// note to self: PDM has known bugs on S3, and does not work on C3
// * S3: PDM sample rate only at 50% of expected rate: https://github.com/espressif/esp-idf/issues/9893
// * S3: I2S PDM has very low amplitude: https://github.com/espressif/esp-idf/issues/8660
// * C3: does not support PDM to PCM input. SoC would allow PDM RX, but there is no hardware to directly convert to PCM so it will not work. https://github.com/espressif/esp-idf/issues/8796
_config.mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_PDM); // Change mode to pdm if clock pin not provided. PDM is not supported on ESP32-S2. PDM RX not supported on ESP32-C3
_config.channel_format =I2S_PDM_MIC_CHANNEL; // seems that PDM mono mode always uses left channel.
_config.use_apll = true; // experimental - use aPLL clock source to improve sampling quality
#endif
}
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 2, 0)
if (mclkPin != I2S_PIN_NO_CHANGE) {
_config.use_apll = true; // experimental - use aPLL clock source to improve sampling quality, and to avoid glitches.
// //_config.fixed_mclk = 512 * _sampleRate;
// //_config.fixed_mclk = 256 * _sampleRate;
}
#if !defined(SOC_I2S_SUPPORTS_APLL)
#warning this MCU does not have an APLL high accuracy clock for audio
// S3: not supported; S2: supported; C3: not supported
_config.use_apll = false; // APLL not supported on this MCU
#endif
#if defined(ARDUINO_ARCH_ESP32) && !defined(CONFIG_IDF_TARGET_ESP32S3) && !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3)
if (ESP.getChipRevision() == 0) _config.use_apll = false; // APLL is broken on ESP32 revision 0
#endif
#endif
// Reserve the master clock pin if provided
_mclkPin = mclkPin;
if (mclkPin != I2S_PIN_NO_CHANGE) {
if(!pinManager.allocatePin(mclkPin, true, PinOwner::UM_Audioreactive)) return;
if(!pinManager.allocatePin(mclkPin, true, PinOwner::UM_Audioreactive)) {
DEBUGSR_PRINTF("\nAR: Failed to allocate I2S pin: MCLK=%d\n", mclkPin);
return;
} else
_routeMclk(mclkPin);
}
@ -220,15 +266,25 @@ class I2SSource : public AudioSource {
.data_in_num = i2ssdPin
};
//DEBUGSR_PRINTF("[AR] I2S: SD=%d, WS=%d, SCK=%d, MCLK=%d\n", i2ssdPin, i2swsPin, i2sckPin, mclkPin);
esp_err_t err = i2s_driver_install(I2S_NUM_0, &_config, 0, nullptr);
if (err != ESP_OK) {
DEBUGSR_PRINTF("Failed to install i2s driver: %d\n", err);
DEBUGSR_PRINTF("AR: Failed to install i2s driver: %d\n", err);
return;
}
DEBUGSR_PRINTF("AR: I2S#0 driver %s aPLL; fixed_mclk=%d.\n", _config.use_apll? "uses":"without", _config.fixed_mclk);
DEBUGSR_PRINTF("AR: %d bits, Sample scaling factor = %6.4f\n", _config.bits_per_sample, _sampleScale);
if (_config.mode & I2S_MODE_PDM) {
DEBUGSR_PRINTLN(F("AR: I2S#0 driver installed in PDM MASTER mode."));
} else {
DEBUGSR_PRINTLN(F("AR: I2S#0 driver installed in MASTER mode."));
}
err = i2s_set_pin(I2S_NUM_0, &_pinConfig);
if (err != ESP_OK) {
DEBUGSR_PRINTF("Failed to set i2s pin config: %d\n", err);
DEBUGSR_PRINTF("AR: Failed to set i2s pin config: %d\n", err);
i2s_driver_uninstall(I2S_NUM_0); // uninstall already-installed driver
return;
}
@ -236,7 +292,7 @@ class I2SSource : public AudioSource {
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 2, 0)
err = i2s_set_clk(I2S_NUM_0, _sampleRate, I2S_SAMPLE_RESOLUTION, I2S_CHANNEL_MONO); // set bit clocks. Also takes care of MCLK routing if needed.
if (err != ESP_OK) {
DEBUGSR_PRINTF("Failed to configure i2s clocks: %d\n", err);
DEBUGSR_PRINTF("AR: Failed to configure i2s clocks: %d\n", err);
i2s_driver_uninstall(I2S_NUM_0); // uninstall already-installed driver
return;
}
@ -288,6 +344,7 @@ class I2SSource : public AudioSource {
currSample = (float) newSamples[i]; // 16bit input -> use as-is
#endif
buffer[i] = currSample;
buffer[i] *= _sampleScale; // scale samples
}
}
}
@ -328,18 +385,25 @@ class ES7243 : public I2SSource {
private:
// I2C initialization functions for ES7243
void _es7243I2cBegin() {
Wire.begin(pin_ES7243_SDA, pin_ES7243_SCL, 100000U);
bool i2c_initialized = Wire.begin(pin_ES7243_SDA, pin_ES7243_SCL, 100000U);
if (i2c_initialized == false) {
DEBUGSR_PRINTLN(F("AR: ES7243 failed to initialize I2C bus driver."));
}
}
void _es7243I2cWrite(uint8_t reg, uint8_t val) {
#ifndef ES7243_ADDR
Wire.beginTransmission(0x13);
#define ES7243_ADDR 0x13 // default address
#else
Wire.beginTransmission(ES7243_ADDR);
#endif
Wire.write((uint8_t)reg);
Wire.write((uint8_t)val);
Wire.endTransmission();
uint8_t i2cErr = Wire.endTransmission(); // i2cErr == 0 means OK
if (i2cErr != 0) {
DEBUGSR_PRINTF("AR: ES7243 I2C write failed with error=%d (addr=0x%X, reg 0x%X, val 0x%X).\n", i2cErr, ES7243_ADDR, reg, val);
}
}
void _es7243InitAdc() {
@ -353,15 +417,28 @@ class ES7243 : public I2SSource {
}
public:
ES7243(SRate_t sampleRate, int blockSize) :
I2SSource(sampleRate, blockSize) {
ES7243(SRate_t sampleRate, int blockSize, float sampleScale = 1.0f) :
I2SSource(sampleRate, blockSize, sampleScale) {
_config.channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT;
};
void initialize(int8_t sdaPin, int8_t sclPin, int8_t i2swsPin, int8_t i2ssdPin, int8_t i2sckPin, int8_t mclkPin) {
// check that pins are valid
if ((sdaPin < 0) || (sclPin < 0)) {
DEBUGSR_PRINTF("\nAR: invalid ES7243 I2C pins: SDA=%d, SCL=%d\n", sdaPin, sclPin);
return;
}
if ((i2sckPin < 0) || (mclkPin < 0)) {
DEBUGSR_PRINTF("\nAR: invalid I2S pin: SCK=%d, MCLK=%d\n", i2sckPin, mclkPin);
return;
}
// Reserve SDA and SCL pins of the I2C interface
if (!pinManager.allocatePin(sdaPin, true, PinOwner::HW_I2C) ||
!pinManager.allocatePin(sclPin, true, PinOwner::HW_I2C)) {
PinManagerPinType es7243Pins[2] = { { sdaPin, true }, { sclPin, true } };
if (!pinManager.allocateMultiplePins(es7243Pins, 2, PinOwner::HW_I2C)) {
pinManager.deallocateMultiplePins(es7243Pins, 2, PinOwner::HW_I2C);
DEBUGSR_PRINTF("\nAR: Failed to allocate ES7243 I2C pins: SDA=%d, SCL=%d\n", sdaPin, sclPin);
return;
}
@ -375,8 +452,8 @@ public:
void deinitialize() {
// Release SDA and SCL pins of the I2C interface
pinManager.deallocatePin(pin_ES7243_SDA, PinOwner::HW_I2C);
pinManager.deallocatePin(pin_ES7243_SCL, PinOwner::HW_I2C);
PinManagerPinType es7243Pins[2] = { { pin_ES7243_SDA, true }, { pin_ES7243_SCL, true } };
pinManager.deallocateMultiplePins(es7243Pins, 2, PinOwner::HW_I2C);
I2SSource::deinitialize();
}
@ -385,6 +462,13 @@ public:
int8_t pin_ES7243_SCL;
};
#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 2, 0)
#if !defined(SOC_I2S_SUPPORTS_ADC) && !defined(SOC_I2S_SUPPORTS_ADC_DAC)
#warning this MCU does not support analog sound input
#endif
#endif
#if !defined(CONFIG_IDF_TARGET_ESP32S2) && !defined(CONFIG_IDF_TARGET_ESP32C3) && !defined(CONFIG_IDF_TARGET_ESP32S3)
// ADC over I2S is only availeable in "classic" ESP32
@ -395,8 +479,8 @@ public:
*/
class I2SAdcSource : public I2SSource {
public:
I2SAdcSource(SRate_t sampleRate, int blockSize) :
I2SSource(sampleRate, blockSize) {
I2SAdcSource(SRate_t sampleRate, int blockSize, float sampleScale = 1.0f) :
I2SSource(sampleRate, blockSize, sampleScale) {
_config = {
.mode = i2s_mode_t(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_ADC_BUILT_IN),
.sample_rate = _sampleRate,
@ -430,7 +514,7 @@ class I2SAdcSource : public I2SSource {
// Determine Analog channel. Only Channels on ADC1 are supported
int8_t channel = digitalPinToAnalogChannel(_audioPin);
if (channel > 9) {
DEBUGSR_PRINTF("Incompatible GPIO used for audio in: %d\n", _audioPin);
DEBUGSR_PRINTF("Incompatible GPIO used for analog audio input: %d\n", _audioPin);
return;
} else {
adc_gpio_init(ADC_UNIT_1, adc_channel_t(channel));
@ -465,11 +549,12 @@ class I2SAdcSource : public I2SSource {
//return;
}
#else
err = i2s_adc_disable(I2S_NUM_0);
//err = i2s_stop(I2S_NUM_0);
if (err != ESP_OK) {
DEBUGSR_PRINTF("Failed to initially disable i2s adc: %d\n", err);
}
// bugfix: do not disable ADC initially - its already disabled after driver install.
//err = i2s_adc_disable(I2S_NUM_0);
// //err = i2s_stop(I2S_NUM_0);
//if (err != ESP_OK) {
// DEBUGSR_PRINTF("Failed to initially disable i2s adc: %d\n", err);
//}
#endif
_initialized = true;
@ -585,8 +670,8 @@ class I2SAdcSource : public I2SSource {
// a user recommended this: Try to set .communication_format to I2S_COMM_FORMAT_STAND_I2S and call i2s_set_clk() after i2s_set_pin().
class SPH0654 : public I2SSource {
public:
SPH0654(SRate_t sampleRate, int blockSize) :
I2SSource(sampleRate, blockSize)
SPH0654(SRate_t sampleRate, int blockSize, float sampleScale = 1.0f) :
I2SSource(sampleRate, blockSize, sampleScale)
{}
void initialize(uint8_t i2swsPin, uint8_t i2ssdPin, uint8_t i2sckPin, int8_t = I2S_PIN_NO_CHANGE, int8_t = I2S_PIN_NO_CHANGE, int8_t = I2S_PIN_NO_CHANGE) {

61
usermods/audioreactive/readme.md
View File

@ -1,36 +1,73 @@
# Audioreactive usermod
This usermod allows controlling LEDs using audio input. Audio input can be either microphone or analog-in (AUX) using appropriate adapter.
Supported microphones range from analog (MAX4466, MAX9814, ...) to digital (INMP441, ICS-43434, ...).
Supported microphones range from cheap analog (MAX4466, MAX9814, ...) to high quality digital (INMP441, ICS-43434, ...) and dgital Line-In.
The usermod does audio processing and provides data structure that specially written effect can use.
The usermod **does not** provide effects or draws anything to LED strip/matrix.
## Additional Documentation
This usermod is an evolution of [SR-WLED](https://github.com/atuline/WLED), and a lot of documentation and information can be found in the [SR-WLED wiki](https://github.com/atuline/WLED/wiki):
* [getting started with audio](https://github.com/atuline/WLED/wiki/First-Time-Setup#sound)
* [Sound settings](https://github.com/atuline/WLED/wiki/Sound-Settings) - similar to options on the usemod settings page in WLED.
* [Digital Audio](https://github.com/atuline/WLED/wiki/Digital-Microphone-Hookup)
* [Analog Audio](https://github.com/atuline/WLED/wiki/Analog-Audio-Input-Options)
* [UDP Sound sync](https://github.com/atuline/WLED/wiki/UDP-Sound-Sync)
## Supported MCUs
This audioreactive usermod works best on "classic ESP32" (dual core), and on ESP32-S3 which also has dual core and hardware floating point support.
It will compile succesfully for ESP32-S2 and ESP32-C3, however might not work well, as other WLED functions will become slow. Audio processing requires a lot of computing power, which can be problematic on smaller MCUs like -S2 and -C3.
Analog audio is only possible on "classic" ESP32, but not on other MCUs like ESP32-S3.
Currently ESP8266 is not supported, due to low speed and small RAM of this chip.
There are however plans to create a lightweight audioreactive for the 8266, with reduced features.
## Installation
Add `-D USERMOD_AUDIOREACTIVE` to your PlatformIO environment as well as `arduinoFFT` to your `lib_deps`.
### using customised _arduinoFFT_ library for use with this usermod
Add `-D USERMOD_AUDIOREACTIVE` to your PlatformIO environment `build_flags`, as well as `https://github.com/blazoncek/arduinoFFT.git` to your `lib_deps`.
If you are not using PlatformIO (which you should) try adding `#define USERMOD_AUDIOREACTIVE` to *my_config.h* and make sure you have _arduinoFFT_ library downloaded and installed.
Customised _arduinoFFT_ library for use with this usermod can be found at https://github.com/blazoncek/arduinoFFT.git
### using latest (develop) _arduinoFFT_ library
Alternatively, you can use the latest arduinoFFT development version.
ArduinoFFT `develop` library is slightly more accurate, and slighly faster than our customised library, however also needs additional 2kB RAM.
* `build_flags` = `-D USERMOD_AUDIOREACTIVE` `-D UM_AUDIOREACTIVE_USE_NEW_FFT`
* `lib_deps`= `https://github.com/kosme/arduinoFFT#develop @ 1.9.2`
## Configuration
All parameters are runtime configurable though some may require hard boot after change (I2S microphone or selected GPIOs).
If you want to define default GPIOs during compile time use the following (default values in parentheses):
If you want to define default GPIOs during compile time use the following addtional build_flags (default values in parentheses):
- `DMTYPE=x` : defines digital microphone type: 0=analog, 1=generic I2S, 2=ES7243 I2S, 3=SPH0645 I2S, 4=generic I2S with master clock, 5=PDM I2S
- `AUDIOPIN=x` : GPIO for analog microphone/AUX-in (36)
- `I2S_SDPIN=x` : GPIO for SD pin on digital mcrophone (32)
- `I2S_WSPIN=x` : GPIO for WS pin on digital mcrophone (15)
- `I2S_CKPIN=x` : GPIO for SCK pin on digital mcrophone (14)
- `ES7243_SDAPIN` : GPIO for I2C SDA pin on ES7243 microphone (-1)
- `ES7243_SCLPIN` : GPIO for I2C SCL pin on ES7243 microphone (-1)
- `MCLK_PIN=x` : GPIO for master clock pin on digital mcrophone (-1)
- `-D SR_DMTYPE=x` : defines digital microphone type: 0=analog, 1=generic I2S (default), 2=ES7243 I2S, 3=SPH0645 I2S, 4=generic I2S with master clock, 5=PDM I2S
- `-D AUDIOPIN=x` : GPIO for analog microphone/AUX-in (36)
- `-D I2S_SDPIN=x` : GPIO for SD pin on digital microphone (32)
- `-D I2S_WSPIN=x` : GPIO for WS pin on digital microphone (15)
- `-D I2S_CKPIN=x` : GPIO for SCK pin on digital microphone (14)
- `-D MCLK_PIN=x` : GPIO for master clock pin on digital Line-In boards (-1)
- `-D ES7243_SDAPIN` : GPIO for I2C SDA pin on ES7243 microphone (-1)
- `-D ES7243_SCLPIN` : GPIO for I2C SCL pin on ES7243 microphone (-1)
**NOTE** Due to the fact that usermod uses I2S peripherial for analog audio sampling, use of analog *buttons* (i.e. potentiometers) is disabled while running this usermod with analog microphone.
### Advanced Compile-Time Options
You can use the following additional flags in your `build_flags`
* `-D SR_SQUELCH=x` : Default "squelch" setting (10)
* `-D SR_GAIN=x` : Default "gain" setting (60)
* `-D I2S_USE_RIGHT_CHANNEL`: Use RIGHT instead of LEFT channel (not recommended unless you strictly need this).
* `-D I2S_USE_16BIT_SAMPLES`: Use 16bit instead of 32bit for internal sample buffers. Reduces sampling quality, but frees some RAM ressources (not recommended unless you absolutely need this).
* `-D I2S_GRAB_ADC1_COMPLETELY`: Experimental: continously sample analog ADC microphone. Only effective on ESP32. WARNING this _will_ cause conflicts(lock-up) with any analogRead() call.
* `-D MIC_LOGGER` : (debugging) Logs samples from the microphone to serial USB. Use with serial plotter (Arduino IDE)
* `-D SR_DEBUG` : (debugging) Additional error diagnostics and debug info on serial USB.
## Release notes
2022-06 Ported from [soundreactive](https://github.com/atuline/WLED) by @blazoncek (AKA Blaz Kristan)
* 2022-06 Ported from [soundreactive WLED](https://github.com/atuline/WLED) - by @blazoncek (AKA Blaz Kristan) and the [SR-WLED team](https://github.com/atuline/WLED/wiki#sound-reactive-wled-fork-team).
* 2022-11 Updated to align with "[MoonModules/WLED](https://amg.wled.me)" audioreactive usermod - by @softhack007 (AKA Frank M&ouml;hle).