Weird UC3 Reset behavior after user page NVRAM usage - c++

I recently need to use in build NVRAM/EEPROM of AT32UC3L0256 to store some configuration data. I finally managed to use the user page NVRAM of the MCU (after days of trial and error and cursing on GCC ignoring noinit directives and fixing and workarounding bugs in ASF as usual) to something like this:
typedef struct
{
int writes; // write cycles counter
int irc_pot; // IRC_POT_IN position state
} _cfg;
volatile static int *nvram_adr=(int*)(void*)0x80800000; // user page NVRAM
volatile static _cfg ram_cfg; // RAM copy of cfg
void cfg_load() // nvram_cfg -> ram_cfg
{
ram_cfg.writes =nvram_adr[8];
ram_cfg.irc_pot=nvram_adr[9];
}
void cfg_save() // nvram_cfg <- ram_cfg
{
int i;
U32 buf[128];
// blank
for (i=0;i<128;i++) buf[i]=0xFFFFFFFF;
// cfg
buf[8]=ram_cfg.writes;
buf[9]=ram_cfg.irc_pot;
// Bootloader default cfg
buf[126]=0x929E0B79;
buf[127]=0xE11EFFD7;
flashcdw_memcpy(nvram_adr ,buf ,256,true); // write data -> nvram_cfg with erase
flashcdw_memcpy(nvram_adr+64,buf+64,256,false); // write data -> nvram_cfg without erase (fucking ASF cant write more than 256Bytes at once but erases whole page !!!)
}
I had to update flashcdw.c,flashcdw.h from ASF 3.48.0.98 in order to be able to write the full 512 Bytes as old ASF did program just up to 256 BYTES but erases whole page making a mess. I also had to store the full page (instead of just 8 bytes due to erase) and as usual due ASF bugs I needed to do it as is instead of just calling flashcdw_memcpy just once...
Its working now but I found out that some addresses are causing weird behavior. When 0xFF is not on some address the device will no longer RESET normally (but still until reset runs OK). On non Bootloader RESET it start the firmware code but after few [ms] it resets again and this goes on forever. To be clear the RESET occurs in this part of code (in my case):
for (U8 i=0;i<4;i++)
{
gpio_tgl_gpio_pin(_LED);
wait_ms(200);
}
its simple blink of LED after the system is configured (PLL CPU clock, configured timers and ISRs but interrupts still disabled). The LED blinks as should few times (PLL works on correct speed) but before loop is finished reset occurs. The wait is simple:
//------------------------------------------------------------------------------------------------
#define clk_cpu 50000000
#define RDTSC_mask 0x0FFFFFFF
void wait_ms(U32 dt)
{
U32 t0,t1;
t0=Get_system_register(AVR32_COUNT);
static const U32 ms=(clk_cpu+999)/1000;
t0&=RDTSC_mask;
for (;dt>0;)
{
t1=Get_system_register(AVR32_COUNT);
t1&=RDTSC_mask;
if (t0>t1) t1+=RDTSC_mask+1;
if ((t1-t0)>=ms)
{
dt--;
t0+=ms;
t0&=RDTSC_mask;
continue;
}
}
}
//------------------------------------------------------------------------------------------------
Even weirder is that if I boot into Bootloader and then reset normally again device is resetting properly and firmware works again (without any erasing/programing) however if I reset normally again the resetting loop occurs again ...
If I reprogram the .userpage NVRAM back to original state using BatchISP (flip) the chip works again normally.
So finally the questions:
What addresses in userpage of NVRAM are causing this or should be reserved/avoided to change?
I know the last 8 Bytes are Bootloader configuration. I suspect the problematic addresses are first 16 bytes. The .userpage should be for user data and does not contain fuses.
What is happening?
its some kind of watchdog or something? I thought those are inside fuses which are stored elsewhere. I do not see anything in datasheet.
Here hex of original .userpage:
:020000048080FA
:10000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00
:10001000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0
:10002000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE0
:10003000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFD0
:10004000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC0
:10005000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFB0
:10006000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA0
:10007000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF90
:10008000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF80
:10009000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF70
:1000A000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF60
:1000B000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF50
:1000C000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF40
:1000D000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF30
:1000E000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF20
:1000F000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF10
:10010000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
:10011000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEF
:10012000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFDF
:10013000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFCF
:10014000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFBF
:10015000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFAF
:10016000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF9F
:10017000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF8F
:10018000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7F
:10019000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF6F
:1001A000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5F
:1001B000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4F
:1001C000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF3F
:1001D000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2F
:1001E000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF1F
:1001F000FFFFFFFFFFFFFFFF929E0B79E11EFFD77E
:00000001FF
I use these (Flip commands) to obtain and restore it:
Batchisp -device AT32UC3L0256 -hardware RS232 -port COM1 -baudrate 115200 -operation memory user read savebuffer cfg_userpage.hex hex386 start reset 0
Batchisp -device AT32UC3L0256 -hardware RS232 -port COM1 -baudrate 115200 -operation onfail abort memory user loadbuffer cfg_userpage.hex program start reset 0
The Bootloader in question is USART version: 1.0.2
And the firmware with this behavior uses PLL,TC,GPIO,PWMA,ADC modules however reset occurs before any ISR and or ADC,PWMA,TC usage.
[Edit1] Watchdog
according to this the first word in .userpage of NVRAM is a fuse for the watchdog which explains the reset after few ms once repairing the data to original values and disabling the WDT the reseting stops. However now instead of booting program the Bootloader is started instead so there is still something fishy. The Bootloader pin selection is in last 8 Bytes
Also I looked into USART Bootloader ver: 1.0.2 source and found out they are using FLASHC instead of FLASHCDW and forcing boot with watchdog (which might reset its state and enable my program to run again somehow).
[Edit2] bug isolated
I finally found out that the problem is caused by writing to last 32bit word of the 512 Byte .userpage:
U32 btldr[2]={0x929E0B79,0xE11EFFD7};
flashcdw_memcpy(&nvram_adr[127],(U32*)&btldr[1] ,4,false);
which is a huge problem as in order to store data correctly I must use erase which erases whole page no matter what and in order to still be able to boot correctly to Bootloader or My firmware I have to restore the Bootloader configuration data:
U32 btldr[2]={0x929E0B79,0xE11EFFD7};
flashcdw_memcpy(&nvram_adr[126],(U32*)&btldr[0] ,4,false);
flashcdw_memcpy(&nvram_adr[127],(U32*)&btldr[1] ,4,false);
I need to find a workaround how to restore the chip to functional state. Maybe duplicate the watchdog reset from Bootloader (but that would be very problematic and even risky in my application) as it restores the chip even without any flashing...
so map for now:
:020000048080FA
:10000000---WDT--FFFFFFFFFFFFFFFFFFFFFFFF00
:10001000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF0
:10002000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFE0
:10003000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFD0
:10004000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFC0
:10005000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFB0
:10006000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFA0
:10007000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF90
:10008000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF80
:10009000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF70
:1000A000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF60
:1000B000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF50
:1000C000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF40
:1000D000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF30
:1000E000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF20
:1000F000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF10
:10010000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
:10011000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEF
:10012000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFDF
:10013000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFCF
:10014000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFBF
:10015000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFAF
:10016000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF9F
:10017000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF8F
:10018000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF7F
:10019000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF6F
:1001A000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF5F
:1001B000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF4F
:1001C000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF3F
:1001D000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2F
:1001E000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF1F
:1001F000FFFFFFFFFFFFFFFF------BTLDR-----7E
:00000001FF
[Edit3] workaround
I managed to successfully write/read program memory FLASH (checked with BatchISP and MCU itself) it also boots OK.
Here code I test this with:
// **** test ****
volatile static U32 *flash_adr=(U32*)(void*)0x00000000;
const U32 buf[4]=
{
0xDEADBEEF,0x00112233,
0xDEADBEEF,0x44556677,
};
volatile static U32 *adr=(U32*)(void*)0x80030000;
flashcdw_memcpy(&adr[0],(U32*)buf,4*4,true ); // erase
flash_adr=(U32*)(void*)0x80000000;
for (U32 i=0;i<0x08000000;i++,flash_adr++)
{
if (flash_adr!=buf)
if (flash_adr[0]==buf[0])
if (flash_adr[1]==buf[1])
if (flash_adr[2]==buf[2])
if (flash_adr[3]==buf[3])
{ break; }
if ((i&0xFFF)==0) gpio_tgl_gpio_pin(_LED);
}
where flash_adr is the found address which contents matches the buf[] signature... (I print it on LCD so I see if it matches what I expect) and it finally does :). So I will use this instead of .userpage.
However the .userpage booting problem fix is still open question

disable wdt early in main function
wdt_disable();
Also I think you not need to write the full page each time.
flashc_memcpy takes bytes length to write while preserving other data unchanged.
unsigned char buf[AVR32_FLASHCDW_PAGE_SIZE];
void* flash_addr = AVR32_FLASHCDW_USER_PAGE;
memcpy(buf, flash_addr, 512);
// modify data in buf
flashcdw_memcpy(flash_addr, buf,AVR32_FLASHCDW_PAGE_SIZE,TRUE);
or just use
int mydata = 123;
int *nvmydata=AVR32_FLASHCDW_USER_PAGE + 16 // offset
flashcdw_memcyp(nvmydata,&mydata,sizeof(mydata),TRUE);
flashcdw_memcpy
volatile void* flashcdw_memcpy(volatile void* dst, const void* src, size_t nbytes, Bool erase)
{
// Use aggregated pointers to have several alignments available for a same address.
UnionCVPtr flash_array_end;
UnionVPtr dest;
UnionCPtr source;
StructCVPtr dest_end;
UnionCVPtr flash_page_source_end;
Bool incomplete_flash_page_end;
Union64 flash_dword;
Bool flash_dword_pending = FALSE;
UnionVPtr tmp;
unsigned int error_status = 0;
unsigned int i, j;
// Reformat arguments.
flash_array_end.u8ptr = AVR32_FLASH + flashcdw_get_flash_size();
dest.u8ptr = dst;
source.u8ptr = src;
dest_end.u8ptr = dest.u8ptr + nbytes;
// If destination is outside flash, go to next flash page if any.
if (dest.u8ptr < AVR32_FLASH)
{
source.u8ptr += AVR32_FLASH - dest.u8ptr;
dest.u8ptr = AVR32_FLASH;
}
else if (flash_array_end.u8ptr <= dest.u8ptr && dest.u8ptr < AVR32_FLASHCDW_USER_PAGE)
{
source.u8ptr += AVR32_FLASHCDW_USER_PAGE - dest.u8ptr;
dest.u8ptr = AVR32_FLASHCDW_USER_PAGE;
}
// If end of destination is outside flash, move it to the end of the previous flash page if any.
if (dest_end.u8ptr > AVR32_FLASHCDW_USER_PAGE + AVR32_FLASHCDW_USER_PAGE_SIZE)
{
dest_end.u8ptr = AVR32_FLASHCDW_USER_PAGE + AVR32_FLASHCDW_USER_PAGE_SIZE;
}
else if (AVR32_FLASHCDW_USER_PAGE >= dest_end.u8ptr && dest_end.u8ptr > flash_array_end.u8ptr)
{
dest_end.u8ptr = flash_array_end.u8ptr;
}
// Align each end of destination pointer with its natural boundary.
dest_end.u16ptr = (U16*)Align_down((U32)dest_end.u8ptr, sizeof(U16));
dest_end.u32ptr = (U32*)Align_down((U32)dest_end.u16ptr, sizeof(U32));
dest_end.u64ptr = (U64*)Align_down((U32)dest_end.u32ptr, sizeof(U64));
// While end of destination is not reached...
while (dest.u8ptr < dest_end.u8ptr)
{
// Clear the page buffer in order to prepare data for a flash page write.
flashcdw_clear_page_buffer();
error_status |= flashcdw_error_status;
// Determine where the source data will end in the current flash page.
flash_page_source_end.u64ptr =
(U64*)min((U32)dest_end.u64ptr,
Align_down((U32)dest.u8ptr, AVR32_FLASHCDW_PAGE_SIZE) + AVR32_FLASHCDW_PAGE_SIZE);
// Determine if the current destination page has an incomplete end.
incomplete_flash_page_end = (Align_down((U32)dest.u8ptr, AVR32_FLASHCDW_PAGE_SIZE) >=
Align_down((U32)dest_end.u8ptr, AVR32_FLASHCDW_PAGE_SIZE));
// If destination does not point to the beginning of the current flash page...
if (!Test_align((U32)dest.u8ptr, AVR32_FLASHCDW_PAGE_SIZE))
{
// Fill the beginning of the page buffer with the current flash page data.
// This is required by the hardware, even if page erase is not requested,
// in order to be able to write successfully to erased parts of flash
// pages that have already been written to.
for (tmp.u8ptr = (U8*)Align_down((U32)dest.u8ptr, AVR32_FLASHCDW_PAGE_SIZE);
tmp.u64ptr < (U64*)Align_down((U32)dest.u8ptr, sizeof(U64));
tmp.u64ptr++)
{
* tmp.u32ptr = *tmp.u32ptr;
* (tmp.u32ptr + 1) = *(tmp.u32ptr + 1);
}
// If destination is not 64-bit aligned...
if (!Test_align((U32)dest.u8ptr, sizeof(U64)))
{
// Fill the beginning of the flash double-word buffer with the current
// flash page data.
// This is required by the hardware, even if page erase is not
// requested, in order to be able to write successfully to erased parts
// of flash pages that have already been written to.
for (i = 0; i < Get_align((U32)dest.u8ptr, sizeof(U64)); i++)
flash_dword.u8[i] = *tmp.u8ptr++;
// Fill the end of the flash double-word buffer with the source data.
for (; i < sizeof(U64); i++)
flash_dword.u8[i] = *source.u8ptr++;
// Align the destination pointer with its 64-bit boundary.
dest.u64ptr = (U64*)Align_down((U32)dest.u8ptr, sizeof(U64));
// If the current destination double-word is not the last one...
if (dest.u64ptr < dest_end.u64ptr)
{
// Write the flash double-word buffer to the page buffer.
*dest.u32ptr++ = flash_dword.u32[0];
*dest.u32ptr++ = flash_dword.u32[1];
}
// If the current destination double-word is the last one, the flash
// double-word buffer must be kept for later.
else flash_dword_pending = TRUE;
}
}
// Read the source data with the maximal possible alignment and write it to
// the page buffer with 64-bit alignment.
switch (Get_align((U32)source.u8ptr, sizeof(U32)))
{
case 0:
for (i = flash_page_source_end.u64ptr - dest.u64ptr; i; i--)
{
*dest.u32ptr++ = *source.u32ptr++;
*dest.u32ptr++ = *source.u32ptr++;
}
break;
case sizeof(U16) :
for (i = flash_page_source_end.u64ptr - dest.u64ptr; i; i--)
{
for (j = 0; j < sizeof(U64) / sizeof(U16); j++) flash_dword.u16[j] = *source.u16ptr++;
* dest.u32ptr++ = flash_dword.u32[0];
* dest.u32ptr++ = flash_dword.u32[1];
}
break;
default:
for (i = flash_page_source_end.u64ptr - dest.u64ptr; i; i--)
{
for (j = 0; j < sizeof(U64); j++) flash_dword.u8[j] = *source.u8ptr++;
dest.u32ptr++ = flash_dword.u32[0];
dest.u32ptr++ = flash_dword.u32[1];
}
}
// If the current destination page has an incomplete end...
if (incomplete_flash_page_end)
{
// If the flash double-word buffer is in use, do not initialize it.
if (flash_dword_pending) i = Get_align((U32)dest_end.u8ptr, sizeof(U64));
// If the flash double-word buffer is free...
else
{
// Fill the beginning of the flash double-word buffer with the source data.
for (i = 0; i < Get_align((U32)dest_end.u8ptr, sizeof(U64)); i++)
flash_dword.u8[i] = *source.u8ptr++;
}
// This is required by the hardware, even if page erase is not requested,
// in order to be able to write successfully to erased parts of flash
// pages that have already been written to.
{
tmp.u8ptr = (volatile U8*)dest_end.u8ptr;
// If end of destination is not 64-bit aligned...
if (!Test_align((U32)dest_end.u8ptr, sizeof(U64)))
{
// Fill the end of the flash double-word buffer with the current flash page data.
for (; i < sizeof(U64); i++)
flash_dword.u8[i] = *tmp.u8ptr++;
// Write the flash double-word buffer to the page buffer.
* dest.u32ptr++ = flash_dword.u32[0];
* dest.u32ptr++ = flash_dword.u32[1];
}
// Fill the end of the page buffer with the current flash page data.
for (; !Test_align((U32)tmp.u64ptr, AVR32_FLASHCDW_PAGE_SIZE); tmp.u64ptr++)
{
tmp.u32ptr = *tmp.u32ptr;
* (tmp.u32ptr + 1) = *(tmp.u32ptr + 1);
}
}
}
// If the current flash page is in the flash array...
if (dest.u8ptr <= AVR32_FLASHCDW_USER_PAGE)
{
// Erase the current page if requested and write it from the page buffer.
if (erase)
{
flashcdw_erase_page(-1, FALSE);
error_status |= flashcdw_error_status;
}
flashcdw_write_page(-1);
error_status |= flashcdw_error_status;
// If the end of the flash array is reached, go to the User page.
if (dest.u8ptr >= flash_array_end.u8ptr)
{
source.u8ptr += AVR32_FLASHCDW_USER_PAGE - dest.u8ptr;
dest.u8ptr = AVR32_FLASHCDW_USER_PAGE;
}
}
// If the current flash page is the User page...
else
{
// Erase the User page if requested and write it from the page buffer.
if (erase)
{
flashcdw_erase_user_page(FALSE);
error_status |= flashcdw_error_status;
}
flashcdw_write_user_page();
error_status |= flashcdw_error_status;
}
}
// Update the FLASHC error status.
flashcdw_error_status = error_status;
// Return the initial destination pointer as the standard memcpy function does.
return dst;
}

OK here is official resolution from Atmel/Microchip:
Proposed Resolution:
Looks like the DFU word being restored doesn't
have the boot flag + CRC change. For the customer's application where
userpage data is not constant, it's preferable to flash instead for
non-volatile storage.
So my understanding its a HW bug (on the newer chips like AT32UC3L0256) and can not be work-around-ed ... other than using different non volatile memory like Flash for program (just like I ended up doing in the first place).
[edit1] Atmel/Microchip just confirmed its definately HW bug

Related

Getting External Headphones (8): EXC_BAD_ACCESS (code=1, address=0x0) error when I am trying to use maximilian

I am testing out using the maximilian library with JUCE. I am trying to use the maxiSample feature and I have implemented it exactly how the example code says to. Whenever I run the standalone app, I get the error "External Headphones (8): EXC_BAD_ACCESS (code=1, address=0x0)" and it gives me a breakpoint at line 747 of maximilian.cpp. It's not my headphones as it does the same thing with any playback device. Truly at a loss.
I've attached my MainComponent.cpp below. Any advice would be great, thank you!
#include "MainComponent.h"
#include "maximilian.h"
//==============================================================================
MainComponent::MainComponent()
{
// Make sure you set the size of the component after
// you add any child components.
setSize (800, 600);
// Some platforms require permissions to open input channels so request that here
if (juce::RuntimePermissions::isRequired (juce::RuntimePermissions::recordAudio)
&& ! juce::RuntimePermissions::isGranted (juce::RuntimePermissions::recordAudio))
{
juce::RuntimePermissions::request (juce::RuntimePermissions::recordAudio,
[&] (bool granted) { setAudioChannels (granted ? 2 : 0, 2); });
}
else
{
// Specify the number of input and output channels that we want to open
setAudioChannels (2, 2);
}
}
MainComponent::~MainComponent()
{
// This shuts down the audio device and clears the audio source.
shutdownAudio();
sample1.load("/Users/(username)/JuceTestPlugins/maxiSample/Source/kick.wav");
}
//==============================================================================
void MainComponent::prepareToPlay (int samplesPerBlockExpected, double sampleRate)
{
// This function will be called when the audio device is started, or when
// its settings (i.e. sample rate, block size, etc) are changed.
// You can use this function to initialise any resources you might need,
// but be careful - it will be called on the audio thread, not the GUI thread.
// For more details, see the help for AudioProcessor::prepareToPlay()
}
void MainComponent::getNextAudioBlock (const juce::AudioSourceChannelInfo& bufferToFill)
{
// Your audio-processing code goes here!
// For more details, see the help for AudioProcessor::getNextAudioBlock()
// Right now we are not producing any data, in which case we need to clear the buffer
// (to prevent the output of random noise)
//bufferToFill.clearActiveBufferRegion();
for(int sample = 0; sample < bufferToFill.buffer->getNumSamples(); ++sample){
//float sample2 = sample1.
//float wave = tesOsc.sinewave(200);
//double sample2 = sample1.play();
// leftSpeaker[sample] = (0.25 * wave);
// rightSpeaker[sample] = leftSpeaker[sample];
double *output;
output[0] = sample1.play();
output[1] = output[0];
}
}
void MainComponent::releaseResources()
{
// This will be called when the audio device stops, or when it is being
// restarted due to a setting change.
// For more details, see the help for AudioProcessor::releaseResources()
}
//==============================================================================
void MainComponent::paint (juce::Graphics& g)
{
// (Our component is opaque, so we must completely fill the background with a solid colour)
g.fillAll (getLookAndFeel().findColour (juce::ResizableWindow::backgroundColourId));
// You can add your drawing code here!
}
void MainComponent::resized()
{
// This is called when the MainContentComponent is resized.
// If you add any child components, this is where you should
// update their positions.
}
Can't say for sure, but couple of things catch my attention.
In getNextAudioBlock() you are dereferencing invalid pointers:
double *output;
output[0] = sample1.play();
output[1] = output[0];
The pointer variable output is uninitialised and will probably be filled with garbage or zeros, which will make the program read from invalid memory. This problem is most likely to cause the EXC_BAD_ACCESS. This method is called from the realtime audio thread, so you probably get a crash on a non-main thread (in this case the thread of External Headphones (8)).
It also is no clear to me what exactly it is you're trying to do here, so it's hard for me to say how it should be. What I can say is that assigning the result of sample1.play() to a double value looks suspicious.
Normally, when dealing with juce::AudioSourceChannelInfo you would get pointers to the audio buffers like so:
auto** bufferPointer = bufferToFill.buffer->getArrayOfWritePointers()
Further, you are loading a file inside the destructor of MainComponent. This at least is suspicious, why would you load a file during destruction?
MainComponent::~MainComponent()
{
// This shuts down the audio device and clears the audio source.
shutdownAudio();
sample1.load("/Users/(username)/JuceTestPlugins/maxiSample/Source/kick.wav");
}

Why does setting SO_SNDBUF and SO_RCVBUF destroy performance?

Running in Docker on a MacOS, I have a simple server and client setup to measure how fast I can allocate data on the client and send it to the server. The tests are done using loopback (in the same docker container). The message size for my tests was 1000000 bytes.
When I set SO_RCVBUF and SO_SNDBUF to their respective defaults, the performance halves.
SO_RCVBUF defaults to 65536 and SO_SNDBUF defaults to 1313280 (retrieved by calling getsockopt and dividing by 2).
Tests:
When I test setting neither buffer size, I get about 7 Gb/s throughput.
When I set one buffer or the other to the default (or higher) I get 3.5 Gb/s.
When I set both buffer sizes to the default I get 2.5 Gb/s.
Server code: (cs is an accepted stream socket)
void tcp_rr(int cs, uint64_t& processed) {
/* I remove this entire thing and performance improves */
if (setsockopt(cs, SOL_SOCKET, SO_RCVBUF, &ENV.recv_buf, sizeof(ENV.recv_buf)) == -1) {
perror("RCVBUF failure");
return;
}
char *buf = (char *)malloc(ENV.msg_size);
while (true) {
int recved = 0;
while (recved < ENV.msg_size) {
int recvret = recv(cs, buf + recved, ENV.msg_size - recved, 0);
if (recvret <= 0) {
if (recvret < 0) {
perror("Recv error");
}
return;
}
processed += recvret;
recved += recvret;
}
}
free(buf);
}
Client code: (s is a connected stream socket)
void tcp_rr(int s, uint64_t& processed, BenchStats& stats) {
/* I remove this entire thing and performance improves */
if (setsockopt(s, SOL_SOCKET, SO_SNDBUF, &ENV.send_buf, sizeof(ENV.send_buf)) == -1) {
perror("SNDBUF failure");
return;
}
char *buf = (char *)malloc(ENV.msg_size);
while (stats.elapsed_millis() < TEST_TIME_MILLIS) {
int sent = 0;
while (sent < ENV.msg_size) {
int sendret = send(s, buf + sent, ENV.msg_size - sent, 0);
if (sendret <= 0) {
if (sendret < 0) {
perror("Send error");
}
return;
}
processed += sendret;
sent += sendret;
}
}
free(buf);
}
Zeroing in on SO_SNDBUF:
The default appears to be: net.ipv4.tcp_wmem = 4096 16384 4194304
If I setsockopt to 4194304 and getsockopt (to see what's currently set) it returns 425984 (10x less than I requested).
Additionally, it appears a setsockopt sets a lock on buffer expansion (for send, the lock's name is SOCK_SNDBUF_LOCK which prohibits adaptive expansion of the buffer). The question then is - why can't I request the full size buffer?
Clues for what is going on come from the kernel source handle for SO_SNDBUF (and SO_RCVBUF but we'll focus on SO_SNDBUF below).
net/core/sock.c contains implementations for the generic socket operations, including the SOL_SOCKET getsockopt and setsockopt handles.
Examining what happens when we call setsockopt(s, SOL_SOCKET, SO_SNDBUF, ...):
case SO_SNDBUF:
/* Don't error on this BSD doesn't and if you think
* about it this is right. Otherwise apps have to
* play 'guess the biggest size' games. RCVBUF/SNDBUF
* are treated in BSD as hints
*/
val = min_t(u32, val, sysctl_wmem_max);
set_sndbuf:
sk->sk_userlocks |= SOCK_SNDBUF_LOCK;
sk->sk_sndbuf = max_t(int, val * 2, SOCK_MIN_SNDBUF);
/* Wake up sending tasks if we upped the value. */
sk->sk_write_space(sk);
break;
case SO_SNDBUFFORCE:
if (!capable(CAP_NET_ADMIN)) {
ret = -EPERM;
break;
}
goto set_sndbuf;
Some interesting things pop out.
First of all, we see that the max possible value is sysctl_wmem_max, a setting which is difficult to pin down within a docker container. We know from the context above that this is likely 212992 (half your max value you retrieved after trying to set 4194304).
Secondly, we see SOCK_SNDBUF_LOCK being set. This setting is in my opinion not well documented in the man pages, but it appears to lock dynamic adjustment of the buffer size.
For example, in the function tcp_should_expand_sndbuf we get:
static bool tcp_should_expand_sndbuf(const struct sock *sk)
{
const struct tcp_sock *tp = tcp_sk(sk);
/* If the user specified a specific send buffer setting, do
* not modify it.
*/
if (sk->sk_userlocks & SOCK_SNDBUF_LOCK)
return false;
...
So what is happening in your code? You attempt to set the max value as you understand it, but it's being truncated to something 10x smaller by the sysctl sysctl_wmem_max. This is then made far worse by the fact that setting this option now locks the buffer to that smaller size. The strange part is that apparently dynamically resizing the buffer doesn't have this same restriction, but can go all the way to the max.
If you look at the first code snip above, you see the SO_SNDBUFFORCE option. This will disregard the sysctl_wmem_max and allow you to set essentially any buffer size provided you have the right permissions.
It turns out processes launched in generic docker containers don't have CAP_NET_ADMIN, so in order to use this socket option, you must run in --privileged mode. However, if you do, and if you force the max size, you will see your benchmark return the same throughput as not setting the option at all and allowing it to grow dynamically to the same size.

Problem with esp8266 sending large JSON Document via MQTT

I developed a little application that read data from a sensor, store them in SPIFFS memory of my wemos D1 mini (esp8266) and then create a JSON Document and send it via MQTT to my topic. The problem is that as long as I send a JSON Doc with 10 object everything works great, but when I increase the size of the doc over 10 object nothing works. Eventually I need to send a JSON doc with 100 object inside.
What have I already done?
I'm using PubSubClient and I already set the MAX_PACKET_SIZE to the correct value
Using arduinojson assistant I found out the size of my JSON Document (8192 bytes)
I tried to use mqtt.fx to test if the problem was the esp8266 or the mqtt broker. Using mqtt.fx I'm able to send a JSON doc with 100 objects
As soon as I increase the size of the JSON doc I get a wdt error from the serial monitor of my arduino IDE.
I search the internet for wdt error but I don't get what they are and how to solve my problem
Last things I already tried to show on the serial monitor the file.txt in the SPIFFS where I store the data and I can store and then read the 100 object
So in the end I think it's an esp8266 problem and not PubSubClient or MQTT. Am I right?
Does anyone of you here ever encountered this problem before or have some other test I can run?
I search the internet for wdt error but I don't get what they are and how to solve my problem
WDT stands for a Watch Dog Timer. https://os.mbed.com/cookbook/WatchDog-Timer#:~:text=A%20watchdog%20timer%20(WDT)%20is,a%20software%20or%20hardware%20fault.
A watchdog timer (WDT) is a hardware timer that automatically generates a system reset if the main program neglects to periodically service it. It is often used to automatically reset an embedded device that hangs because of a software or hardware fault. Some systems may also refer to it as a computer operating properly (COP) timer. Many microcontrollers including the mbed processor have watchdog timer hardware.
Let's paint a better picture with an example. Let's say that you setup a WDT with a time of 10 seconds. Then the WDT starts counting down from 10 seconds. If it reaches 0 the processor will reset. "Feeding" the WDT will reset the countdown to the original value in this case 10 seconds. So if the WDT has counted down to 4 seconds remaining and you feed it, it resets the countdown back to 10 and starts counting down again.
Does anyone of you here ever encountered this problem before or have some other test I can run?
It looks to me like sending a larger JSON object takes a longer period of time than what the WDT is set for. One possibility would be to break up the JSON object into multiple pieces and send it in smaller chunks instead of one large one. This way the time between WDT "feedings" is reduced. I have no idea if this would be possible for you to change. But this should at least give you a better idea of what's happening.
OK in the end the problem was that sending a large JsonDocument triggered the WDT and the only way I found to overcome this problem was, as suggested by adamvz, to create a main file with all the 100 object, then call a function to split that file in 10 smaller one and send each of them over the internet through an HTTP request or Mosquitto.
Supposing you already created the main file in the spiffs memory, then:
This to split the main file:
void WritePacks() {
sourceFile = LittleFS.open("/file.txt", "r");
if (!sourceFile) {
Serial.println(F("Error: file.txt open failed"));
} else {
Serial.println("File open w/ success");
for (byte idx = 0; idx < outputCount; idx++) {
String aLine;
aLine.reserve(capacity);
if (sourceFile.available() == 0) break;
destinationFile = LittleFS.open(outputFileNames[idx], "w");
if (!destinationFile) {
Serial.print(F("can't open destination "));
Serial.println(outputFileNames[idx]);
break;
} else {
int lineCount = 0;
while (sourceFile.available() && (lineCount <= 10)) {
aLine = sourceFile.readStringUntil('\n');
destinationFile.println(aLine); // double check if the '\n' is in the String or not (--> print or println accordingly)
lineCount++;
}
outputIndex = idx;
Serial.println(outputIndex);
destinationFile.close();
}
} // end for
sourceFile.close();
}
}//end WritePacks
This to publish:
//------ HTTP Publish ------
void httpPublish(){
const char * outputFileNames[] = {"/out1.txt", "/out2.txt", "/out3.txt", "/out4.txt", "/out5.txt", "/out6.txt", "/out7.txt", "/out8.txt", "/out9.txt", "/out10.txt"};
const byte outputCount = sizeof outputFileNames / sizeof outputFileNames[0];
byte outputIndex = 0;
File sourceFile;
File destinationFile;
//Serial.println(capacity);
for (byte idx = 0; idx < outputCount; idx++) {
DynamicJsonDocument doc(capacity);
DynamicJsonDocument globalDoc(capacity);
StaticJsonDocument <1024> localDoc;
String aLine;
aLine.reserve(capacity);
destinationFile = LittleFS.open(outputFileNames[idx], "r");
if (!destinationFile) {
Serial.print(F("can't open destination "));
Serial.println(outputFileNames[idx]);
break;
} else {
Serial.print("Reading: ");
Serial.println(outputFileNames[idx]);
//int lineCount = 0;
while (destinationFile.available()) {
aLine = destinationFile.readStringUntil('\n');
DeserializationError error = deserializeJson(localDoc, aLine);
if (!error) globalDoc.add(localDoc);
else{ Serial.println("Error Writing All files");}
}//while
JsonObject Info = doc.createNestedObject("Info");
Info["Battery"] = battery;
Info["ID"] = id;
Info["Latitudine"] = latitudine;
Info["Longitudine"] = longitudine;
JsonArray Data = doc.createNestedArray("Data");
Data.add(globalDoc);
HTTPClient http;
//Send request
http.begin("yourURL");
char buffer[capacity];
size_t n = serializeJson(doc, buffer);
http.POST(buffer);
Serial.println(buffer);
http.end();
destinationFile.close();
}
}// end for
}//end httpPublish

Finding the rendezvous structure of tracee (program being debugged)

I need debugger I am writing to give me the name of shared lib that program being debugged is linking with, or loading dynamically. I get the rendezvous structure as described in link.h, and answers to other questions, using DT_DEBUG, in the loop over _DYNAMIC[].
First, debugger never hits the break point set at r_brk.
Then I put a break in the program being debugged, and use link_map to print all loaded libraries. It only prints libraries loaded by the debugger, not the program being debugged.
It seems that, the rendezvous structure I am getting belongs to the debugger itself. If so, could you please tell me how to get the rendezvous structure of the program I am debugging? If what I am doing must work, your confirmation will be helpful, perhaps with some hint as to what else might be needed.
Thank you.
// You need to include <link.h>. All structures are explained
// in elf(5) manual pages.
// Caller has opened "prog_name", the debugee, and fd is the
// file descriptor. You can send the name instead, and do open()
// here.
// Debugger is tracing the debugee, so we are using ptrace().
void getRandezvousStructure(int fd, pid_t pd, r_debug& rendezvous) {
Elf64_Ehdr elfHeader;
char* elfHdrPtr = (char*) &elfHeader;
read(fd, elfHdrPtr, sizeof(elfHeader));
Elf64_Addr debugeeEntry = elfHeader.e_entry; // entry point of debugee
// Here, set a break at debugeeEntry, and after "PTRACE_CONT",
// and waitpid(), remove the break, and set rip back to debugeeEntry.
// After that, here it goes.
lseek(fd, elfHeader.e_shoff, SEEK_SET); // offset of section header
Elf64_Shdr secHeader;
elfHdrPtr = (char*) &secHeader;
Elf64_Dyn* dynPtr;
// Keep reading until we get: secHeader.sh_addr.
// That is the address of _DYNAMIC.
for (int i = 0; i < elfHeader.e_shnum; i++) {
read(fd, elfHdrPtr, elfHeader.e_shentsize);
if (secHeader.sh_type == SHT_DYNAMIC) {
dynPtr = (Elf64_Dyn*) secHeader.sh_addr; // address of _DYNAMIC
break;
}
}
// Here, we get "dynPtr->d_un.d_ptr" which points to rendezvous
// structure, r_debug
uint64_t data;
for (;; dynPtr++) {
data = ptrace(PTRACE_PEEKDATA, pd, dynPtr, 0);
if (data == DT_NULL) break;
if (data == DT_DEBUG) {
data = ptrace(PTRACE_PEEKDATA, pd, (uint64_t) dynPtr + 8 , 0);
break;
}
}
// Using ptrace() we read sufficient chunk of memory of debugee
// to copy to rendezvous.
int ren_size = sizeof(rendezvous);
char* buffer = new char[2 * ren_size];
char* p = buffer;
int total = 0;
uint64_t value;
for (;;) {
value = ptrace(PTRACE_PEEKDATA, pd, data, 0);
memcpy(p, &value, sizeof(value));
total += sizeof(value);
if (total > ren_size + sizeof(value)) break;
data += sizeof(data);
p += sizeof(data);
}
// Finally, copy the memory to rendezvous, which was
// passed by reference.
memcpy(&rendezvous, buffer, ren_size);
delete [] buffer;
}

AVR XMEGA USART Simulation

Using the test code below, I'm trying to send data over the USART of an xmega128a3u using the simulator in Atmel Studio. Watching the I/O view the Data register is never set even though I'm setting it. Is there something wrong with my code or the simulator or what?
#include <avr/io.h>
#include <avr/interrupt.h>
#define bscale 0
#define bsel 0x0003 //250kbps
#define packetFormat (USART_SBMODE_bm | USART_CHSIZE_8BIT_gc | USART_PMODE_DISABLED_gc)
uint8_t n;
int main(void)
{
//ALLOW PORTB AND PORTF TO BE WRITTEN TO! TURNS OFF JTAG
CCP = 0xD8; //Allow Protected IO changing
MCU_MCUCR = 0x1;
//CRYSTAL SETUP
OSC_XOSCCTRL = OSC_FRQRANGE_12TO16_gc | OSC_XOSCSEL_XTAL_16KCLK_gc; // 16Mhz Crystal
OSC_CTRL |= OSC_XOSCEN_bm;
while(!(OSC_STATUS & OSC_XOSCRDY_bm)); //Wait for crystal to stabilize.
CCP = CCP_IOREG_gc;
CLK_CTRL = CLK_SCLKSEL_XOSC_gc;
//END CRYSTAL SETUP
cli();
//Enable Interrupts
USARTF0.CTRLA = USART_TXCINTLVL_LO_gc | USART_DREINTLVL_LO_gc;
//Enable transmitter
USARTF0.CTRLB = USART_TXEN_bm;
PMIC.CTRL |= PMIC_LOLVLEX_bm;
//Set baud
USARTF0.BAUDCTRLB = bscale;
USARTF0.BAUDCTRLA = bsel;
//Set packet format
USARTF0.CTRLC = packetFormat;
sei();
while (1)
{
if(n < 255) {
USARTF0.DATA = n;
} else {
n = 0;
}
}
}
ISR(USARTF0_TXC_vect) {
n++;
}
ISR(USARTF0_DRE_vect) {
n++;
}
Are other registers being updated, just not DATA? If not, make sure you've enabled the clock to the USART. Many micros also use one DATA register for reads and writes. Thus attempting to read the DATA register after writing (I.e., the debug view is doing a read) won't return anything unless data has been received. Thus you'd likely see the same behavior even if executing on hardware.
That said, in my experience the simulator in Atmel Studio isn't very good at simulating interrupts or peripheral operation.
Edit to include informatino from comments below:
Since you can't read back something you just wrote to the DATA register, checking the DREIF flag in the STATUS register right after you write the data will confirm whether or not the data is being transmitted.