Un-defined behavior when using hex values without Casting - c++

This is the strangest problem I have ever encountered. I am programming a chess engine and I have a function that returns a rank mask. For example, Consider this 64-bit bitboard.
00000000
00000000
00000000
00000000
00000100
00000000
00000000
00000000
The 1 in the middle is a piece.
Now my function should return
00000000
00000000
00000000
00000000
11111111
00000000
00000000
00000000
Very simple, just return a mask of the rank. Here is my code
uint64_t get_rank_mask(int sq)
{
return (0xFF<<(sq-(sq%8)));
}
Here is what the function returns when I pass 36, or the square e5
00000000
00000000
00000000
00000000
00000000
00000000
00000000
11111111
Hmm, strange it is. Let me make a small change in the code.
uint64_t get_rank_mask(int sq)
{
uint64_t mask = 0xFF;
mask<<=(sq-(sq%8));
return mask;
}
Here is the return value now
00000000
00000000
00000000
11111111
00000000
00000000
00000000
00000000
And magically, that is the intended result. I don't even know the difference between the two code samples. Can anyone explain this behaviour?

0xFF is an int. Assuming an int has 32 bits, shifting this value left by 32 will invoke undefined behavior.
In the second version, you are storing 0xFF in an unsigned integral type. This can be shifted any number of bits, and that operation is well defined. In your case, you are storing it in an unsigned int with 64 bits, and so shifting left by 32 gives the result you want.

Related

How to redefine symbol of object file from the arm-none-eabi compile with -flot flag?

I want redefine malloc symbol to own_malloc, so that I can know memory alloc details and print alloc-size.But I found it don't work when I excuting order:
arm-none-eabi-objcopy --redefine-sym malloc=own_malloc test.o
the test source:
#include <stdlib.h>
extern "C" void test();
void test(){
void* ptr = ::malloc(16);
free(ptr);
}
I exec objdump to look the symbol table
arm-none-eabi-objdump -t test.o
I fount there was not malloc/free symbol in symbols table.
SYMBOL TABLE:
00000000 l df *ABS* 00000000 test.cxx
00000000 l d .text 00000000 .text
00000000 l d .data 00000000 .data
00000000 l d .bss 00000000 .bss
00000000 l d .gnu.debuglto_.debug_info 00000000 .gnu.debuglto_.debug_info
00000000 l d .gnu.debuglto_.debug_abbrev 00000000 .gnu.debuglto_.debug_abbrev
00000000 l d .gnu.debuglto_.debug_line 00000000 .gnu.debuglto_.debug_line
00000000 l d .gnu.debuglto_.debug_str 00000000 .gnu.debuglto_.debug_str
00000000 l d .gnu.lto_.profile.7c13183f5b427e41 00000000 .gnu.lto_.profile.7c13183f5b427e41
00000000 l d .gnu.lto_.icf.7c13183f5b427e41 00000000 .gnu.lto_.icf.7c13183f5b427e41
00000000 l d .gnu.lto_.ipa_sra.7c13183f5b427e41 00000000 .gnu.lto_.ipa_sra.7c13183f5b427e41
00000000 l d .gnu.lto_.inline.7c13183f5b427e41 00000000 .gnu.lto_.inline.7c13183f5b427e41
00000000 l d .gnu.lto_.jmpfuncs.7c13183f5b427e41 00000000 .gnu.lto_.jmpfuncs.7c13183f5b427e41
00000000 l d .gnu.lto_.pureconst.7c13183f5b427e41 00000000 .gnu.lto_.pureconst.7c13183f5b427e41
00000000 l d .gnu.lto_.lto.7c13183f5b427e41 00000000 .gnu.lto_.lto.7c13183f5b427e41
00000000 l d .gnu.lto_test.0.7c13183f5b427e41 00000000 .gnu.lto_test.0.7c13183f5b427e41
00000000 l d .gnu.lto_.symbol_nodes.7c13183f5b427e41 00000000 .gnu.lto_.symbol_nodes.7c13183f5b427e41
00000000 l d .gnu.lto_.refs.7c13183f5b427e41 00000000 .gnu.lto_.refs.7c13183f5b427e41
00000000 l d .gnu.lto_.decls.7c13183f5b427e41 00000000 .gnu.lto_.decls.7c13183f5b427e41
00000000 l d .gnu.lto_.symtab.7c13183f5b427e41 00000000 .gnu.lto_.symtab.7c13183f5b427e41
00000000 l d .gnu.lto_.ext_symtab.7c13183f5b427e41 00000000 .gnu.lto_.ext_symtab.7c13183f5b427e41
00000000 l d .gnu.lto_.opts 00000000 .gnu.lto_.opts
00000000 l d .comment 00000000 .comment
00000000 l d .ARM.attributes 00000000 .ARM.attributes
00000000 w .gnu.debuglto_.debug_info 00000000 .hidden test.cxx.cee7ff5d
00000001 O *COM* 00000001 __gnu_lto_slim
I used the following compilation parameters:
arm-none-eabi-g++ -c -fno-common -nostdinc++ -Wall -Wshadow -Wundef -fno-exceptions -fcheck-new -fno-rtti -std=c++17 -pipe -Os -fno-strict-aliasing -fomit-frame-pointer -flto -fuse-linker-plugin -fno-builtin -ffunction-sections -fdata-sections -g -march=armv8-m.main+dsp -mtune=cortex-m33 -mthumb -mfpu=fpv5-sp-d16 -mfloat-abi=hard _ -Wno-cpp --std=c++17 -Werror test.cxx -o test.o
So how can I redefine malloc and free to own_malloc and own_free with lto flag ?

In chess engines where bitboards are used, how are the edges detected?

For example, all white pawns' attacks are either generated by shifting 7 or 9 bits to the left (or right, I could be mistaken, but I think it's easy to get the gist).
So the white pawn bitboard that looks like this
00000000000000000000000000000000000000001111111100000000
Would be shifted into this
00000000000000000000000000000001111111100000000000000000
However, if you attempt to portray this example in a 8x8 array, one of the pawns goes through the chessboard's edge.
So when the bits are shifted to generate attacks, how do the engines prevent themselves from generating ones that go through the chessboard's edges?
We'll add some whitespace. Suppose you start with a pawn position:
00000000
00000000
00000000
00000000
00000000
10000000
01100101
00011010
00000000
00000000
The newlines are just there for ease of reading, everything is packed bits. Where can the pawns move?
00000000
00000000
00000000
00000000
10000000
01100101
00011010
00000000
00000000
00000000
here, the only risk of going past an edge is at the top, which is easy to deal with.
Next, where can the pawns take? The naive solution is you take the above "move" mask, and shift it left and right 1. As you noticed, this causes wraparound.
00000000
00000000
00000000
00000001
00000000
11001010
00110100
00000000
00000000
00000000
|
00000000
00000000
00000000
00000000
01000000
00110010
10001101
00000000
00000000
00000000
(apologies if you dislike my choice of endianness).
We can fix this with some masking. The bits that will wrap around are at known locations. So we mask them out before we do the operation.
This mask:
11111110
11111110
11111110
11111110
11111110
11111110
11111110
11111110
before we shift right, and
01111111
01111111
01111111
01111111
01111111
01111111
01111111
01111111
before we shift left.
So to see the spots threatened by pawns, where "forward" is a right shift, we do:
auto pawn_moves = pawn_mask >> 8;
auto pawn_left_take = (pawn_moves & not_left_column) << 1;
auto pawn_right_take = (pawn_moves & not_right_column) << 1;
auto pawn_take = pawn_left_take | pawn_right_take;
and there we have it.
Similar masking can be done for other cases. For example, if we are calculating the knight take mask we need 8 masks and 8 shifts, then or them all together.
For a rook moving right 3, you mask out the right 3 columns, then do a right 3 shift on the rook mask.
At some point, however, you are likely to want to use intrinsics and SIMD processing; so the exact bit operations you have that are fast are going to depend on your handware. Maybe unpacking 8 8 bit values into 8 24 bit values, doing your mask-free operation there, then extracting the middle 8 bits is the fastest way on your particular hardware.

Why there is no .bss segment in COFF object file?

A simple hello world program:
#include <stdio.h>
int main(void)
{
printf("Hello, world!\n");
return 0;
}
After dumpbin it with /HEADERS flag, i get those segments:
8 .bss
A0 .debug$S
62 .drectve
F .rdata
8B .text$mn
If compile the program with /TC, so that it's a C program, i get those segments after the same use of a dumpbin:
2000 .data
1000 .gfids
7000 .rdata
1000 .reloc
10000 .text
point is:
How do i get this similar kind of output:
# objdump -hrt hello.o
hello.o: file format elf32-i386
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00000011 00000000 00000000 00000034 2**2
CONTENTS, ALLOC, LOAD, RELOC, READONLY, CODE
1 .data 00000000 00000000 00000000 00000048 2**2
CONTENTS, ALLOC, LOAD, DATA
2 .bss 00000000 00000000 00000000 00000048 2**2
ALLOC
3 .rodata.str1.1 0000000d 00000000 00000000 00000048 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .comment 00000033 00000000 00000000 00000055 2**0
CONTENTS, READONLY
SYMBOL TABLE:
00000000 l df *ABS* 00000000 hello.c
00000000 l d .text 00000000
00000000 l d .data 00000000
00000000 l d .bss 00000000
00000000 l d .rodata.str1.1 00000000
00000000 l d .comment 00000000
00000000 g F .text 00000011 main
00000000 *UND* 00000000 puts

How can the stack be broken at the very start of application

The stack frame pointer (EBP) should always point to where the previous stack frame was right but why is that not the case in my application! That would indicate something is really wrong..hard to believe!
I created multiple simple demo applications where the stack pointer always points to the previous stack frame but I can't make sense of why this is not the case in this application and it is happening when my application is just starting up!
The following is my call stack
0:000> k
# ChildEBP RetAddr
00 0018fee4 6381d1cd acn!CAcnApp::InitInstance+0x41 [c:\acn-project\acn\acn.cpp # 527]
01 0018fef4 00428575 MFC80U!AfxWinMain+0x48 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp # 37]
02 0018ff88 765d336a acn!__tmainCRTStartup+0x150 [f:\dd\vctools\crt_bld\self_x86\crt\src\crtexe.c # 589]
03 0018ff94 76f59902 kernel32!BaseThreadInitThunk+0xe
04 0018ffd4 76f598d5 ntdll!__RtlUserThreadStart+0x70
05 0018ffec 00000000 ntdll!_RtlUserThreadStart+0x1b
0:000> dc 0018fee4
0018fee4 ffffffff 6381d1cd 00489498 00000001 .......c..H.....
0018fef4 00000000 00428575 00400000 00000000 ....u.B...#.....
0018ff04 01e53fd2 0000000a 87b8aee0 00000000 .?..............
0018ff14 00000000 7efde000 00000044 01e54012 .......~D....#..
0018ff24 01e53ff2 01e53fd4 00000000 00000000 .?...?..........
0018ff34 00000000 00000000 00000000 00000000 ................
0018ff44 00000000 00000000 00000000 00000000 ................
0018ff54 00000000 00000000 0018ff84 00428e5d ............].B.
0:000> dc 0018fef4
0018fef4 00000000 00428575 00400000 00000000 ....u.B...#.....
0018ff04 01e53fd2 0000000a 87b8aee0 00000000 .?..............
0018ff14 00000000 7efde000 00000044 01e54012 .......~D....#..
0018ff24 01e53ff2 01e53fd4 00000000 00000000 .?...?..........
0018ff34 00000000 00000000 00000000 00000000 ................
0018ff44 00000000 00000000 00000000 00000000 ................
0018ff54 00000000 00000000 0018ff84 00428e5d ............].B.
0018ff64 01e53fd2 00000000 00000000 0018ff0c .?..............
0:000> dc 0018ff88
0018ff88 0018ff94 765d336a 7efde000 0018ffd4 ....j3]v...~....
0018ff98 76f59902 7efde000 7d7a657d 00000000 ...v...~}ez}....
0018ffa8 00000000 7efde000 00000000 00000000 .......~........
0018ffb8 00000000 0018ffa0 00000000 ffffffff ................
0018ffc8 76f958c5 0b965c89 00000000 0018ffec .X.v.\..........
0018ffd8 76f598d5 0042873d 7efde000 00000000 ...v=.B....~....
0018ffe8 00000000 00000000 00000000 0042873d ............=.B.
0018fff8 7efde000 00000000 78746341 00000020 ...~....Actx ...
The control is in the first line of InitInstance() so it's like my application is drawing its first breath and the stack seems to be already corrupted? Well the app class constructor comes before this but I checked the call stack is in similar state there as well.
Notice the stack frame pointer (EFP) check for both 1st and 2nd frame fails but the stack is good beyond that.
My first question is that is there any explanation the call stack can be like this and this being Ok? In other words, can we say the call stack is definitely broken? The application does load and calls various dlls if that may play any part (don't know why it would though).
What could be the suspect in this case since the application has only barely started!?
Update (code)
Here is constructor which is very simple. And as for InitInstance(), that function is very long but my breakpoint is on first line so its code is never executed when call stack is like that.
CAcnApp::CAcnApp()
{
m_bServMode = FALSE;
m_bFactory = FALSE;
m_bDownload = FALSE;
m_pEngine = NULL;
m_hWiztomMod = NULL;
m_pServer = new CAcnServer;
}
Second update
I posted a follow up question to share extra information after further investigation and that does qualify as separate question due to differences.
BTW, global or file global objects are created before the main function is called.
If an object's constructor has problems, you could see any number of defects occurring before main is called.
Review your object's constructors. You should be able to set a breakpoint in the object's constructor.
Edit 1: Memory allocation in constructor
Having a global object that has dynamic memory allocation may cause problems. The object requires that the dynamic memory allocation is initialized before the object is constructed. Try commenting out the dynamic memory allocation and see if the issue goes away.
A workaround is to create an "initialize" method that can be called after the main entry point is reached. The initialize method would perform dynamic memory allocation.

windbg coredump analysis address with "bad"

I have a corruption memory heap problem with an application.
by using windbg and a dump file of the crash as an input I have the following output with dd esp command
0:002> dd esp
00000000`03e3e490 14badf55 00000000 03e3e8c0 00000000
00000000`03e3e4a0 00000000 00000000 03e3e8c0 00000000
00000000`03e3e4b0 03e3e8c0 00000000 6b0064f2 00000000
00000000`03e3e4c0 03e3f030 00000000 6b002510 00000000
00000000`03e3e4d0 00000000 00000000 03dfede8 00000000
00000000`03e3e4e0 c0000005 00000000 00000000 7d6210e8
00000000`03e3e4f0 00000002 00000000 00000000 00000000
00000000`03e3e500 00000000 00001000 78b83980 036b0000
There is this adress : 14badf55
I really don't know how to interpret this "bad"..
Is anyone have an idea of the meaning of this bad ?
EDIT:
when I try to use this command :
u 14badf55
the following output comes :
00000000`14badf55 ?? ???
^ Memory access error in 'u 14badf55'
The .ecxr command give me :
rax=0000000003e3e488 rbx=0000000003e3e8c0 rcx=0000000003e3dfb0
rdx=0000000000000000 rsi=000000006b005a17 rdi=0000000000000000
rip=000000006b006369 rsp=0000000003e3e490 rbp=0000000003dfede8
r8=000000006b00254a r9=0000000003e3e4d8 r10=0000000000000007
r11=0000000000000000 r12=000000006b01fe90 r13=0000000000000000
r14=0000000003e3f110 r15=0000000078b83980
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=0000 es=0000 fs=0000 gs=0000 efl=00000204
wow64!Wow64NotifyDebugger+0x9:
00000000`6b006369 b001 mov al,1
You can see the c0000005 output in the file. This is the sign of a access violation.
Run the following:
- .cxr 00000000`03e3e4e0 (To set the exception context)
- kL (to get a stack trace)