Hanging of XShmPutImage event notification - c++
I am using XShm extension to draw and manipulate images in Linux.
In order to not have screen flickering, I am passing send_event = TRUE to XShmPutImage and then waiting for the event with XIfEvent, immediately after the call to XScmPutImage.
This way, I am making the image drawing blocking in order to not changing the image until it is displayed on the window surface.
Usually everything works fine. But sometimes, when I have intensive image drawing, it seems that the event never comes and the drawing procedure hangs.
Where to see for the problem? Is using XIfEvent appropriate for this task? How can the event dissapear from the message queue?
Is it possible XShmPutImage to not send event (if send_event = TRUE) or to send event different than ShmCompletion on some circumstances? (for example on some internal error or something?)
EDIT:
After some more research, I found that such hangs happens only when the window manager intensively generate events to the window. For example when I resize the window by dragging its corners.
EDIT2:
I tried several ways to solve this problem, but without success. At the end I was forced to use some timeout and to cancel waiting after some time. But of course this is dirty hack and I want to fix it anyway.
So, what can be the reason XShmPutImage to not send event if send_event=TRUE or is it possible this event to disappear from the message queue?
EDIT3:
Here is the questionable code (FASM):
cinvoke XShmPutImage, ......, TRUE
.loop:
lea eax, [.event]
cinvoke XCheckTypedEvent, [Display], [ShmCompletionEvent], eax
test eax, eax
jz .loop ; there is no message
NB: XShmPutImage always return TRUE, regardless whether the event check hangs or not, so I didn't put error check after it.
EDIT4:
Because of request I am posting the whole code of the drawing function. The code uses some macro libraries of FASM, but at least the ideas are clear (I hope)
Notice that this code contains workaround code that limits the event waiting for only 20ms. Without this timeout the waiting loop simply hangs forever. The number of the XShm event is acquired by calling XShmGetEventBase as recommended in the Xshm documentation.
; Draws the image on a OS provided window surface.
proc DrawImageRect, .where, .pImage, .xDst, .yDst, .xSrc, .ySrc, .width, .height
.event XEvent
rb 256
begin
pushad
mov esi, [.pImage]
test esi, esi
jz .exit
mov ebx, [esi+TImage.ximage]
cinvoke XCreateGC, [hApplicationDisplay], [.where], 0, 0
mov edi, eax
cinvoke XShmPutImage, [hApplicationDisplay], [.where], edi, [esi+TImage.ximage], [.xSrc], [.ySrc], [.xDst], [.yDst], [.width], [.height], TRUE
stdcall GetTimestamp
lea esi, [eax+20] ; 20ms timeout
.loop:
lea eax, [.event]
cinvoke XCheckTypedEvent, [hApplicationDisplay], [ShmCompletionEvent], eax
test eax, eax
jnz .finish
stdcall GetTimestamp
cmp eax, esi
jb .loop
.finish:
cinvoke XFreeGC, [hApplicationDisplay], edi
.exit:
popad
return
endp
And here is the code of the main event loop of the application.
The procedure __ProcessOneSystemEvent simply dispatches the events to the GUI objects and ignores all events it does not use. It does not process ShmCompletionEvent at all.
All the windows created in the application have events mask of:
ExposureMask+FocusChangeMask+KeyPressMask+KeyReleaseMask+ButtonPressMask+ButtonReleaseMask+EnterWindowMask+LeaveWindowMask+PointerMotionMask+StructureNotifyMask
proc ProcessSystemEvents
.event XEvent
rb 256
begin
push ebx ecx edx
.event_loop:
; check for quit
get eax, [pApplication], TApplication:MainWindow
test eax, eax
jz .continue
cmp dword [eax], 0
jne .continue
cinvoke XFlush, [hApplicationDisplay]
xor eax, eax
mov [fGlobalTerminate], 1
stc
pop edx ecx ebx
return
.continue:
cinvoke XPending, [hApplicationDisplay]
test eax, eax
jz .noevents
push edi ecx
lea edi, [.event]
mov ecx, sizeof.XEvent/4
xor eax, eax
rep stosd
pop ecx edi
lea ebx, [.event]
cinvoke XNextEvent, [hApplicationDisplay], ebx
stdcall __ProcessOneSystemEvent, ebx
jmp .event_loop
.noevents:
clc
pop edx ecx ebx
return
endp
The full source code is available in the repository but it is a very big project not easy for navigation. The discussed source is in check-in 8453c99b1283def8.
The files: "freshlib/graphics/images.asm" and "freshlib/graphics/Linux/images.asm" are about the image drawing.
The files: "freshlib/gui/Main.asm" and "freshlib/gui/Linux/Main.asm" are about the general events handling in the application.
What is the X server doing?
The X server can and will suppress a ShmCompletionEvent if the parameters passed to XShmPutImage exceed the geometry of the shared memory area attached to the XImage in the call. The server checks X/Y and width/height against the previously stored limits for the given shared area and if the call parameters are out-of-bounds, the server will return BadValue, suppress the drawing operation, and suppress the completion event.
The above is exactly what is happening in your library. Here's how:
The main event dispatcher routine is ProcessSystemEvents. It performs an XEventNext, and based on event type, using a jump table .jump_table dispatches to an event specific handler function.
The event specific function for an Expose event is .expose
The .expose function will, in turn, call DrawImageRect using X/Y and width/height values from the XExposeEvent struct. This is wrong and is the true source of the bug, as we shall see momentarily.
DrawImageRect will pass these values along in a call to XShmPutImage
The handler for XShmPutImage in the X server will examine these parameters and reject if they're out of bounds.
The parameters are being rejected because they come from an exposure event and are related to the geometry of the window and not the geometry of the shared memory attached to the XImage used in the XShmPutImage call.
Specifically, if the window has just be resized (e.g. by the window manager) and has been enlarged, and there has been a prior ConfigureNotify event for resize. Now, with a new Expose event, it will have larger width/height that will exceed the width/height of the shared memory area that the server knows about.
It is the responsibility of the client to field the window resize events [etc.] and teardown/recreate the shared memory area with the enlarged size. This is not being done and is the source of the bug.
NOTE: Just to be completely clear on this, the server can only report on the error and not do anything about it for a few reasons:
The server knows about the window [and its resize].
It knows about the XImage, its shared memory area and size
But they're only associated during the XShmPutImage call [AFAIK]
Even if the server could associate them, it couldn't adjust the shmarea
That's because it has no way to relink the shmarea to the client side
Only the client can do that via XShmDetach/XShmAttach
Below are redacted versions of the relevant source files from the c5c765bc7e commit. They have been cleaned up a bit, so only the most germane parts remain. Some lines have been truncated or wrapped to eliminate horizontal scrolling.
The files have been annotated with NOTE and NOTE/BUG which I did while analyzing them.
gui/Main.asm The top level generic main loop. Nothing to see much to see here.
; FILE: gui/Main.asm
; _____________________________________________________________________________
;| |
;| ..::FreshLib::.. Free, open source. Licensed under "BSD 2-clause" license." |
;|_____________________________________________________________________________|
;
; Description: Main procedure of GUI application library.
;
; Target OS: Any
;
; Dependencies:
;
; Notes: Organize the main message/event loop needed by every GUI engine.
; This file contains only OS independent part and includes OS dependent
; files.
;______________________________________________________________________________
module "Main library"
proc Run
begin
.mainloop:
stdcall ProcessSystemEvents
jc .terminate
mov eax, [pApplication]
test eax, eax
jz .eventok
get ecx, eax, TApplication:OnIdle
jecxz .eventok
stdcall ecx, eax
.eventok:
stdcall WaitForSystemEvent
jmp .mainloop
.terminate:
DebugMsg "Terminate GUI application!"
return
endp
include '%TargetOS%/Main.asm'
endmodule
gui/Linux/Main.asm The event handlers
; FILE: gui/Linux/Main.asm
; _____________________________________________________________________________
;| |
;| ..::FreshLib::.. Free, open source. Licensed under "BSD 2-clause" license." |
;|_____________________________________________________________________________|
;
; Description: Main procedure of GUI application library.
;
; Target OS: Linux
;
; Dependencies:
;
; Notes: Organize the main message/event loop needed by every GUI engine.
;______________________________________________________________________________
body ProcessSystemEvents
; NOTE: this is the storage for the dequeued event -- all dispatch routines
; should use it and process it
.event XEvent
rb 256
begin
push ebx ecx edx
.event_loop:
; check for quit
get eax, [pApplication], TApplication:MainWindow
test eax, eax
jz .continue ; ???????????
cmp dword [eax], 0
jne .continue
cinvoke XFlush, [hApplicationDisplay]
xor eax, eax
mov [fGlobalTerminate], 1
stc
pop edx ecx ebx
return
; NOTE: it is wasteful for the main loop to call WaitForSystemEvent, then call
; us and we do XPending on the first loop -- we already know we have at least
; one event waiting in the queue
.continue:
cinvoke XPending, [hApplicationDisplay]
test eax, eax
jz .noevents
push edi ecx
lea edi, [.event]
mov ecx, sizeof.XEvent/4
xor eax, eax
rep stosd
pop ecx edi
lea ebx, [.event]
cinvoke XNextEvent, [hApplicationDisplay], ebx
stdcall __ProcessOneSystemEvent, ebx
jmp .event_loop
.noevents:
clc
pop edx ecx ebx
return
endp
body WaitForSystemEvent
.event XEvent
begin
push eax ecx edx
lea eax, [.event]
cinvoke XPeekEvent, [hApplicationDisplay], eax
pop edx ecx eax
return
endp
proc __ProcessOneSystemEvent, .linux_event
begin
pushad
mov ebx, [.linux_event]
; mov eax, [ebx+XEvent.type]
; cmp eax, [ShmCompletionEvent]
; je .shm_completion
stdcall _GetWindowStruct, [ebx+XEvent.window]
jc .notprocessed
test eax, eax
jz .notprocessed
mov esi, eax
mov ecx, [ebx+XEvent.type]
cmp ecx, LASTEvent
jae .notprocessed
mov ecx, [.jump_table+4*ecx]
jecxz .notprocessed
jmp ecx
.notprocessed:
popad
stc
return
.finish:
popad
clc
return
;.shm_completion:
; DebugMsg "Put back completion event!"
;
; int3
; cinvoke XPutBackEvent, [hApplicationDisplay], ebx
; jmp .finish
;.........................................................................
; seMove and seResize events.
;-------------------------------------------------------------------------
.moveresize:
; NOTE/BUG!!!!: we must not only process a resize/move request, but we must also
; adjust the size of the shmarea attached to the XImage -- that is _not_ being
; done. (e.g.) if the window is enlarged, the shmarea must be enlarged
cinvoke XCheckTypedWindowEvent, [hApplicationDisplay],
[ebx+XConfigureEvent.window], ConfigureNotify, ebx
test eax, eax
jnz .moveresize
; resize event...
mov eax, [esi+TWindow._width]
mov edx, [esi+TWindow._height]
cmp eax, [ebx+XConfigureEvent.width]
jne .resize
cmp edx, [ebx+XConfigureEvent.height]
je .is_move
.resize:
exec esi, TWindow:EventResize, [ebx+XConfigureEvent.width],
[ebx+XConfigureEvent.height]
; move event...
.is_move:
mov eax, [esi+TWindow._x]
mov edx, [esi+TWindow._y]
cmp eax, [ebx+XConfigureEvent.x]
jne .move
cmp eax, [ebx+XConfigureEvent.y]
je .finish
.move:
exec esi, TWindow:EventMove,
[ebx+XConfigureEvent.x], [ebx+XConfigureEvent.y]
jmp .finish
;.........................................................................
; DestroyNotify handler it invalidates the handle in TWindow structure and
; then destroys TWindow.
.destroy:
test esi, esi
jz .finish
mov [esi+TWindow.handle], 0
destroy esi
jmp .finish
;.........................................................................
; Window paint event
.expose:
get edi, esi, TWindow:ImgScreen
; NOTE:BUG!!!!!
;
; if the window has been resized (e.g. enlarged), these values are wrong!
; they relate to the _window_ but _not_ the shmarea that is attached to the
; XImage
;
; however, DrawImageRect will call XShmPutImage with these values, they
; will exceed the geometry of what the X server knows about the shmarea and
; it will return BadValue and _suppress_ the completion event for XShmPutImage
stdcall DrawImageRect, [esi+TWindow.handle], edi,
[ebx+XExposeEvent.x],[ebx+XExposeEvent.y],
[ebx+XExposeEvent.x], [ebx+XExposeEvent.y],
[ebx+XExposeEvent.width], [ebx+XExposeEvent.height]
jmp .finish
;.........................................................................
; Mouse event handlers
.mousemove:
cinvoke XCheckTypedWindowEvent, [hApplicationDisplay],
[ebx+XConfigureEvent.window], MotionNotify, ebx
test eax, eax
jnz .mousemove
stdcall ServeMenuMouseMove, [ebx+XMotionEvent.window],
[ebx+XMotionEvent.x], [ebx+XMotionEvent.y],
[ebx+XMotionEvent.state]
jc .finish
cinvoke XCheckTypedWindowEvent, [hApplicationDisplay],
[ebx+XMotionEvent.window], MotionNotify, ebx
test eax, eax
jnz .mousemove
mov edi, [__MouseTarget]
test edi, edi
jz .search_target_move
stdcall __GetRelativeXY, edi, [ebx+XMotionEvent.x], [ebx+XMotionEvent.y]
jmp .target_move
.search_target_move:
exec esi, TWindow:ChildByXY, [ebx+XMotionEvent.x],
[ebx+XMotionEvent.y], TRUE
mov edi, eax
.target_move:
cmp edi, [__LastPointedWindow]
je .move_event
cmp [__LastPointedWindow], 0
je .leave_ok
exec [__LastPointedWindow], TWindow:EventMouseLeave
.leave_ok:
mov [__LastPointedWindow], edi
exec edi, TWindow:EventMouseEnter
.move_event:
exec edi, TWindow:EventMouseMove, ecx, edx, [ebx+XMotionEvent.state]
jmp .finish
;.........................................................................
; event jump table
.jump_table dd 0 ; event 0
dd 0 ; event 1
dd .key_press ; KeyPress = 2
dd .key_release ; KeyRelease = 3
dd .mouse_btn_press ; ButtonPress = 4
dd .mouse_btn_release ; ButtonRelease = 5
dd .mousemove ; MotionNotify = 6
dd 0 ; EnterNotify = 7
dd 0 ; LeaveNotify = 8
dd .focusin ; FocusIn = 9
dd .focusout ; FocusOut = 10
dd 0 ; KeymapNotify = 11
dd .expose ; Expose = 12
dd 0 ; GraphicsExpose = 13
dd 0 ; NoExpose = 14
dd 0 ; VisibilityNotify = 15
dd 0 ; CreateNotify = 16
dd .destroy ; DestroyNotify = 17
dd 0 ; UnmapNotify = 18
dd 0 ; MapNotify = 19
dd 0 ; MapRequest = 20
dd 0 ; ReparentNotify = 21
dd .moveresize ; ConfigureNotify = 22
dd 0 ; ConfigureRequest = 23
dd 0 ; GravityNotify = 24
dd 0 ; ResizeRequest = 25
dd 0 ; CirculateNotify = 26
dd 0 ; CirculateRequest = 27
dd 0 ; PropertyNotify = 28
dd 0 ; SelectionClear = 29
dd 0 ; SelectionRequest = 30
dd 0 ; SelectionNotify = 31
dd 0 ; ColormapNotify = 32
dd .clientmessage ; ClientMessage = 33
dd .mapping_notify ; MappingNotify = 34
graphics/Linux/images.asm The image drawing code [including the DrawImageRect function] and the shared memory create/destroy code.
; FILE: graphics/Linux/images.asm
; _____________________________________________________________________________
;| |
;| ..::FreshLib::.. Free, open source. Licensed under "BSD 2-clause" license." |
;|_____________________________________________________________________________|
;
; Description: Memory based images manipulation library.
;
; Target OS: Linux
;
; Dependencies: memory.asm
;
; Notes:
;______________________________________________________________________________
uses libX11, xshm
struct TImage
.width dd ? ; width in pixels.
.height dd ? ; height in pixels.
.pPixels dd ? ; pointer to the pixel memory.
; os dependent data
.ximage dd ?
.shminfo XShmSegmentInfo
ends
body CreateImage
begin
pushad
stdcall GetMem, sizeof.TImage
jc .finish
mov esi, eax
xor eax, eax
inc eax
mov ecx, [.width]
mov edx, [.height]
cmp ecx, 0
cmovle ecx, eax
cmp edx, 0
cmovle edx, eax
mov [esi+TImage.width], ecx
mov [esi+TImage.height], edx
lea eax, [4*ecx]
imul eax, edx
cinvoke shmget, IPC_PRIVATE, eax, IPC_CREAT or 777o
test eax, eax
js .error
mov [esi+TImage.shminfo.ShmID], eax
cinvoke shmat, eax, 0, 0
cmp eax, -1
je .error_free
mov [esi+TImage.shminfo.Addr], eax
mov [esi+TImage.pPixels], eax
mov [esi+TImage.shminfo.fReadOnly], 1
lea ebx, [esi+TImage.shminfo]
cinvoke XShmCreateImage, [hApplicationDisplay], 0, $20, ZPixmap, eax,
ebx, [esi+TImage.width], [esi+TImage.height]
mov [esi+TImage.ximage], eax
cinvoke XShmAttach, [hApplicationDisplay], ebx
clc
mov [esp+4*regEAX], esi
.finish:
popad
return
.error_free:
cinvoke shmctl, [ebx+XShmSegmentInfo.ShmID], IPC_RMID, 0
.error:
stdcall FreeMem, esi
stc
jmp .finish
endp
body DestroyImage
begin
pushad
mov esi, [.ptrImage]
test esi, esi
jz .finish
lea eax, [esi+TImage.shminfo]
cinvoke XShmDetach, [hApplicationDisplay], eax
cinvoke XDestroyImage, [esi+TImage.ximage]
cinvoke shmdt, [esi+TImage.shminfo.Addr]
cinvoke shmctl, [esi+TImage.shminfo.ShmID], IPC_RMID, 0
stdcall FreeMem, esi
.finish:
popad
return
endp
;if used ___CheckCompletionEvent
;___CheckCompletionEvent:
;
;virtual at esp+4
; .display dd ?
; .pEvent dd ?
; .user dd ?
;end virtual
;
;; timeout
; stdcall GetTimestamp
; cmp eax, [.user]
; jbe #f
;
; DebugMsg "Timeout!"
;
; mov eax, 1
; retn
;
;##:
; mov eax, [.pEvent] ;.pEvent
; mov eax, [eax+XEvent.type]
;
; cmp eax, [ShmCompletionEvent]
; sete al
; movzx eax, al
; retn
;end if
body DrawImageRect
.event XEvent
rb 256
begin
pushad
mov esi, [.pImage]
test esi, esi
jz .exit
mov ebx, [esi+TImage.ximage]
; NOTE: is this necessary? it seems wasteful to create and destroy a GC
; repeatedly. Dunno, does this _have_ to be done here, _every_ time?
cinvoke XCreateGC, [hApplicationDisplay], [.where], 0, 0
mov edi, eax
; NOTE/BUG: The return ShmCompletionEvent will be suppressed due to a BadValue
; if the X/Y and width/height parameters given to us by caller exceed the
; geometry/range of the shmarea attached to .ximage
;
; the routine that calls us is .expose and it _is_ giving us bad values. it is
; passing us X/Y width/height related to an exposure event of the .where
; _window_ which we put in the call. The X server will compare these against
; the size of the shmarea of TImage.xmage and complain if we exceed the bounds
cinvoke XShmPutImage, [hApplicationDisplay], [.where], edi,
[esi+TImage.ximage], [.xSrc], [.ySrc], [.xDst], [.yDst],
[.width], [.height], TRUE
; NOTE/BUG: this code should _not_ be looping on XCheckTypedEvent because it
; disrupts the normal event processing. if we want to be "synchronous" on this
; we should loop on the main event dispatcher (ProcessSystemEvents) and let it
; dispatch to a callback we create. we can set a "pending" flag that our [not
; yet existent] dispatch routine can clear
; THIS CODE SOMETIMES CAUSES HANGS!
stdcall GetTimestamp
lea esi, [eax+20]
.loop:
lea eax, [.event]
cinvoke XCheckTypedEvent, [hApplicationDisplay], [ShmCompletionEvent],
eax
test eax, eax
jnz .finish
stdcall GetTimestamp
cmp eax, esi
jb .loop
.finish:
cinvoke XFreeGC, [hApplicationDisplay], edi
.exit:
popad
return
endp
Xext/shm.c The X server code that checks and processes the XShmPutImage call.
// FILE: Xext/shm.c
static int
ProcShmPutImage(ClientPtr client)
{
GCPtr pGC;
DrawablePtr pDraw;
long length;
ShmDescPtr shmdesc;
REQUEST(xShmPutImageReq);
REQUEST_SIZE_MATCH(xShmPutImageReq);
VALIDATE_DRAWABLE_AND_GC(stuff->drawable, pDraw, DixWriteAccess);
VERIFY_SHMPTR(stuff->shmseg, stuff->offset, FALSE, shmdesc, client);
// NOTE: value must be _exactly_ 0/1
if ((stuff->sendEvent != xTrue) && (stuff->sendEvent != xFalse))
return BadValue;
if (stuff->format == XYBitmap) {
if (stuff->depth != 1)
return BadMatch;
length = PixmapBytePad(stuff->totalWidth, 1);
}
else if (stuff->format == XYPixmap) {
if (pDraw->depth != stuff->depth)
return BadMatch;
length = PixmapBytePad(stuff->totalWidth, 1);
length *= stuff->depth;
}
else if (stuff->format == ZPixmap) {
if (pDraw->depth != stuff->depth)
return BadMatch;
length = PixmapBytePad(stuff->totalWidth, stuff->depth);
}
else {
client->errorValue = stuff->format;
return BadValue;
}
// NOTE/BUG: The following block is the "check parameters" code. If the
// given drawing parameters of the request (e.g. X, Y, width, height) [or
// combinations thereof] exceed the geometry/size of the shmarea, the
// BadValue error is being returned here and the code to send a return
// event will _not_ be executed. The bug isn't really here, it's on the
// client side, but it's the client side bug that causes the event to be
// suppressed
/*
* There's a potential integer overflow in this check:
* VERIFY_SHMSIZE(shmdesc, stuff->offset, length * stuff->totalHeight,
* client);
* the version below ought to avoid it
*/
if (stuff->totalHeight != 0 &&
length > (shmdesc->size - stuff->offset) / stuff->totalHeight) {
client->errorValue = stuff->totalWidth;
return BadValue;
}
if (stuff->srcX > stuff->totalWidth) {
client->errorValue = stuff->srcX;
return BadValue;
}
if (stuff->srcY > stuff->totalHeight) {
client->errorValue = stuff->srcY;
return BadValue;
}
if ((stuff->srcX + stuff->srcWidth) > stuff->totalWidth) {
client->errorValue = stuff->srcWidth;
return BadValue;
}
if ((stuff->srcY + stuff->srcHeight) > stuff->totalHeight) {
client->errorValue = stuff->srcHeight;
return BadValue;
}
// NOTE: this is where the drawing takes place
if ((((stuff->format == ZPixmap) && (stuff->srcX == 0)) ||
((stuff->format != ZPixmap) &&
(stuff->srcX < screenInfo.bitmapScanlinePad) &&
((stuff->format == XYBitmap) ||
((stuff->srcY == 0) &&
(stuff->srcHeight == stuff->totalHeight))))) &&
((stuff->srcX + stuff->srcWidth) == stuff->totalWidth))
(*pGC->ops->PutImage) (pDraw, pGC, stuff->depth,
stuff->dstX, stuff->dstY,
stuff->totalWidth, stuff->srcHeight,
stuff->srcX, stuff->format,
shmdesc->addr + stuff->offset +
(stuff->srcY * length));
else
doShmPutImage(pDraw, pGC, stuff->depth, stuff->format,
stuff->totalWidth, stuff->totalHeight,
stuff->srcX, stuff->srcY,
stuff->srcWidth, stuff->srcHeight,
stuff->dstX, stuff->dstY, shmdesc->addr + stuff->offset);
// NOTE: this is where the return event gets sent
if (stuff->sendEvent) {
xShmCompletionEvent ev = {
.type = ShmCompletionCode,
.drawable = stuff->drawable,
.minorEvent = X_ShmPutImage,
.majorEvent = ShmReqCode,
.shmseg = stuff->shmseg,
.offset = stuff->offset
};
WriteEventsToClient(client, 1, (xEvent *) &ev);
}
return Success;
}
Your source code will be the ultimate piece which we can analyse but since I understand Assembly very less, I will give you an answer on macro level. Exact answer is still unknown to me.
I case of too many events only it is creating this issue but not with normal occurrence of events which means you framework is running out of virtual memory or another frame of event is triggered before the previous one releases its memory. In this kind of situations you can do few things
Try to check if there is any memory leak or not.. After one frame of event is over, try to clean up the memory or end that frame object properly before triggering the new one.
You can also develop a mechanism to make the second frame wait for the first frame to get over. In C/C++ we do it using so many synchronization methods like Mutex or select system calls. If you design follows that kind of pattern, then you can do these
If anywhere you have the authority to change the allocated memory given to your window, try to increase it. Because one thing for sure (according to your explanation ) that this is some memory issue.
Replying on Edit 3 It looks like you are calling some method cinvoke. How it is internally handling the even is unknown to me. Why don't you implement it directly in C. I am sure for whatever target that you are working, you will get some Cross-compiler.
Related
Get device encryption support information
I want to detect the device encryption support in my program. This info is available in the System Information program. Please check out the screenshot below: What kind of Win API functions are used/available to detect the device encryption support? What System Information program uses to detect it? I just need some information.
TL;DR: it uses undocumented functions from fveapi.dll (Windows Bitlocker Drive Encryption API). It seems to rely only on the TPM capabilities. Note that I only spent like 15 mins on it, but I doubt I missed something crucial, althoug this might be possible. A bit of Reverse engineering Typed system information in search bar, saw it spawned msinfo32.exe. Put the binary in a disassembler. It uses a MUI file so I'll have to search for the strings in the MUI file and not the executable. Searching Device Encryption Support leads to string ID 951 (0x3b7) STRINGTABLE LANGUAGE LANG_ENGLISH, SUBLANG_ENGLISH_US { 951, "Device Encryption Support|%s" Searching for the contant in the disassembler leads to a function named: DeviceEncryptionDataPoints(struct CWMIHelper *, struct CPtrList *) The load ing of the aforementioned string is almost right at the start: .text:00000001400141E9 mov edx, 3B7h .text:00000001400141EE lea rcx, [rsp+2C8h+var_280] .text:00000001400141F3 .text:00000001400141F3 loc_1400141F3: .text:00000001400141F3 ; try { .text:00000001400141F3 call cs:__imp_?LoadStringW#CString##QEAAHI#Z ; CString::LoadStringW(uint) So we are definitely in the right function. It loads the fveapi.dll module: .text:0000000140014269 xor edx, edx ; hFile .text:000000014001426B mov r8d, 800h ; dwFlags .text:0000000140014271 lea rcx, LibFileName ; "fveapi.dll" .text:0000000140014278 call cs:__imp_LoadLibraryExW Gets a pointer on FveQueryDeviceEncryptionSupport: .text:00000001400142AB lea rdx, aFvequerydevice ; "FveQueryDeviceEncryptionSupport" .text:00000001400142B2 mov rcx, rdi ; hModule .text:00000001400142B5 call cs:__imp_GetProcAddress And immediately calls the function (this is a protected CFG call, but it's here): .text:00000001400142CA mov [rsp+2C8h+var_254], rbx .text:00000001400142CF mov [rsp+2C8h+var_260], 14h .text:00000001400142D7 mov [rsp+2C8h+var_25C], 1 .text:00000001400142DF mov [rsp+2C8h+var_258], 1 .text:00000001400142E7 lea rcx, [rsp+2C8h+var_260] .text:00000001400142EC call cs:__guard_dispatch_icall_fptr Return value If the function fails: .text:00000001400142EC call cs:__guard_dispatch_icall_fptr .text:00000001400142F2 mov esi, eax .text:00000001400142F4 test eax, eax .text:00000001400142F6 js loc_1400143F0 ; check failure We land here: .text:00000001400143F7 mov edx, 2FFh .text:00000001400143FC lea rcx, [rsp+2C8h+var_288] .text:0000000140014401 call cs:__imp_?LoadStringW#CString##QEAAHI#Z ; CString::LoadStringW(uint) The string 0x2FF (767) is: 767, "Elevation Required to View" If the call succeed, the code checks one of the parameter which is definitly an out parameter: .text:00000001400142EC call cs:__guard_dispatch_icall_fptr .text:00000001400142F2 mov esi, eax .text:00000001400142F4 test eax, eax .text:00000001400142F6 js loc_1400143F0 .text:00000001400142FC cmp dword ptr [rsp+2C8h+var_254], ebx ; rbx = 0 .text:0000000140014300 jnz short loc_14001431D .text:0000000140014302 mov edx, 3B8h .text:0000000140014307 lea rcx, [rsp+2C8h+var_288] .text:000000014001430C call cs:__imp_?LoadStringW#CString##QEAAHI#Z ; CString::LoadStringW(uint) If it's 0, the string 0x3b8 (952) is used: 952, "Meets prerequisites" Otherwise various failure functions are called. Failure In case of a failure, the UpdateDeviceEncryptionStateFailureString function is called: .text:0000000140014325 lea r9, [rsp+2C8h+var_294] ; int * .text:000000014001432A lea r8, [rsp+2C8h+var_290] ; int * .text:000000014001432F mov edx, 3C1h ; unsigned int .text:0000000140014334 lea rcx, [rsp+2C8h+var_288] ; struct CString * .text:0000000140014339 call ?UpdateDeviceEncryptionStateFailureString##YAXPEAVCString##IPEAH1#Z ; UpdateDeviceEncryptionStateFailureString(CString *,uint,int *,int *) Its main goal is to fetch some string from the resource file. One that stands out is 0x3b9: .text:0000000140013A37 mov edx, 3B9h .text:0000000140013A3C mov rcx, rbx .text:0000000140013A3F call cs:__imp_?LoadStringW#CString##QEAAHI#Z ; CString::LoadStringW(uint) 953, "Reasons for failed automatic device encryption" Which is the case for me since I don't have a TPM. Other Functions All of the other functions called from the DeviceEncryptionDataPoints (at least to get the needed results) are all from the fveapi.dll. There are a lot in a function called PerformIndividualHardwareTests(HINSTANCE hModule, struct CString *, int *, int *): .text:0000000140013AEF lea rdx, aNgscbcheckisao ; "NgscbCheckIsAOACDevice" .text:0000000140013AF6 mov [rbp+var_1F], 0 .text:0000000140013AFA mov rdi, r9 .text:0000000140013AFD mov [rbp+var_20], 0 .text:0000000140013B01 mov rsi, r8 .text:0000000140013B04 mov [rbp+var_1E], 0 .text:0000000140013B08 mov rbx, rcx .text:0000000140013B0B call cs:__imp_GetProcAddress .text:0000000140013B12 nop dword ptr [rax+rax+00h] .text:0000000140013B17 mov r12, rax .text:0000000140013B1A test rax, rax .text:0000000140013B1D jz loc_140013CB9 .text:0000000140013B23 lea rdx, aNgscbcheckishs ; "NgscbCheckIsHSTIVerified" .text:0000000140013B2A mov rcx, rbx ; hModule .text:0000000140013B2D call cs:__imp_GetProcAddress .text:0000000140013B34 nop dword ptr [rax+rax+00h] .text:0000000140013B39 mov r15, rax .text:0000000140013B3C test rax, rax .text:0000000140013B3F jz loc_140013CB9 .text:0000000140013B45 lea rdx, aNgscbcheckhsti ; "NgscbCheckHSTIPrerequisitesVerified" .text:0000000140013B4C mov rcx, rbx ; hModule .text:0000000140013B4F call cs:__imp_GetProcAddress .text:0000000140013B56 nop dword ptr [rax+rax+00h] .text:0000000140013B5B mov r13, rax .text:0000000140013B5E test rax, rax .text:0000000140013B61 jz loc_140013CB9 .text:0000000140013B67 lea rdx, aNgscbcheckdmas ; "NgscbCheckDmaSecurity" .text:0000000140013B6E mov rcx, rbx ; hModule .text:0000000140013B71 call cs:__imp_GetProcAddress There's also a registry key checked SYSTEM\CurrentControlSet\Control\BitLocker\AutoDE\HSTI: .text:0000000140013C10 lea r8, ?NGSCB_AUTODE_HSTI_REQUIRED##3QBGB ; "HSTIVerificationRequired" .text:0000000140013C17 mov [rsp+60h+pcbData], rax ; pcbData .text:0000000140013C1C lea rdx, ?NGSCB_AUTODE_HSTI_PREREQS##3QBGB ; "SYSTEM\\CurrentControlSet\\Control\\Bit"... .text:0000000140013C23 lea rax, [rbp+var_1C] .text:0000000140013C27 mov r9d, 10h ; dwFlags .text:0000000140013C2D mov [rsp+60h+pvData], rax ; pvData .text:0000000140013C32 mov rcx, 0FFFFFFFF80000002h ; hkey .text:0000000140013C39 and [rsp+60h+var_40], 0 .text:0000000140013C3F call cs:__imp_RegGetValueW and some other functions (NgscbCheckPreventDeviceEncryption, NgscbGetWinReConfiguration, FveCheckTpmCapability, ...) , once again, all from the fveapi.dll module. So basically the checks are all based on functions from this DLL. It seems that none of them are documented (as far as I can see with a quick search). I didn't find anything around in the DeviceEncryptionDataPoints caller (which is basically the main() function), since the next calls are dealing with checking the hypervisor capabilities.
How do I call a different subroutine using parameters passed into the current one (for use in recursion)?
I have two functions that take integers x and y read from input. product returns x * y power returns x ^ y, however it uses recursion and product to compute this. so x would be "base" and y is "exponent". They called from C++: int a, b, x, y; a = product(x, y); b = power(x, y); and here is the asm. I got the product to work, however am having trouble with power because I am not sure of the syntax/method/convention to call product from it (and call itself for the recursion). EDIT: Recursion must be used. global product global power section .text product: push ebp mov ebp, esp sub esp, 4 push edi push esi xor eax, eax mov edi, [ebp+8] mov esi, [ebp+12] mov [ebp-4], edi product_loop: add [ebp-4], edi mov eax, [ebp-4] sub esi, 1 cmp esi, 1 jne product_loop product_done: pop esi pop edi mov esp, ebp pop ebp ret power: push ebp mov ebp, esp sub esp, 4 push edi push esi push ebx xor eax, eax mov edi, [ebp+8] mov esi, [ebp+12] ;;; check: cmp esi, 1 ; if exp < 1 jl power_stop recursion: ; else (PLEASE HELP!!!!!!!!) ; eax = call product (base, (power(base, exp-1)) power_stop: mov eax, 1 ; return 1 power_done: push ebx pop esi pop edi mov esp, ebp pop ebp ret EDIT: My solution! power: ; Standard prologue push ebp ; Save the old base pointer mov ebp, esp ; Set new value of the base pointer sub esp, 4 ; make room for 1 local variable result push ebx ; this is exp-1 xor eax, eax ; Place zero in EAX. We will keep a running sum mov eax, [ebp+12] ; exp mov ebx, [ebp+8] ; base cmp eax, 1 ; n >= 1 jge L1 ; if not, go do a recursive call mov eax, 1 ; otherwise return 1 jmp L2 L1: dec eax ; exp-1 push eax ; push argument 2: exp-1 push ebx ; push argument 1: base call power ; do the call, result goes in eax: power(base, exp-1) add esp, 8 ; get rid of arguments push eax ; push argument 2: power(base, exponent-1) push ebx ; push argument 1: base call product ; product(base, power(base, exponent-1)) L2: ; Standard epilogue pop ebx ; restore register mov esp, ebp ; deallocate local variables pop ebp ; Restore the callers base pointer. ret ; Return to the caller.
You are using CDECL calling convention, so you have to first push the arguments in the stack in backward direction, then call the function and then clean the stack after the return. push arg_last push arg_first call MyFunction add esp, 8 ; the argument_count*argument_size But here are some notes on your code: Your function product does not return any value. Use mov eax, [ebp-4] immediately after product_done label. Multiplication is much easy to be made by the instruction mul or imul. Using addition is the slowest possible way. Computing the power by recursion is not the best idea. Use the following algorithm: Y = 1; if N=0 exit. if N is odd -> Y = Y*x; N=N-1 if N is even -> Y = Y*Y; N=N/2 goto 2 Use SHR instruction in order to divide N by 2. Use test instrction in order to check odd/even number. This way, you simply don't need to call product from power function.
If you're not sure how to write the assembly, you can generally write it in C++ and assemble it for clues - something like: int power(int n, int exp) { return exp == 0 ? 1 : exp == 1 ? n : product(n, power(n, exp - 1)); } Then you should just be able to use gcc -S or whatever your compiler's equivalent switch for assembly output is, or disassemble the machine code if you prefer. For example, the function above, thrown in with int product(int x, int y) { return x * y; } and int main() { return product(3, 4); }, compiled with Microsoft's compiler ala cl /Fa power.cc: ; Listing generated by Microsoft (R) Optimizing Compiler Version 15.00.30729.01 TITLE C:\home\anthony\user\dev\power.cc .686P .XMM include listing.inc .model flat INCLUDELIB LIBCMT INCLUDELIB OLDNAMES PUBLIC ?product##YAHHH#Z ; product ; Function compile flags: /Odtp _TEXT SEGMENT _x$ = 8 ; size = 4 _y$ = 12 ; size = 4 ?product##YAHHH#Z PROC ; product ; File c:\home\anthony\user\dev\power.cc ; Line 1 push ebp mov ebp, esp mov eax, DWORD PTR _x$[ebp] imul eax, DWORD PTR _y$[ebp] pop ebp ret 0 ?product##YAHHH#Z ENDP ; product _TEXT ENDS PUBLIC ?power##YAHHH#Z ; power ; Function compile flags: /Odtp _TEXT SEGMENT tv73 = -8 ; size = 4 tv74 = -4 ; size = 4 _n$ = 8 ; size = 4 _exp$ = 12 ; size = 4 ?power##YAHHH#Z PROC ; power ; Line 4 push ebp mov ebp, esp sub esp, 8 ; Line 7 cmp DWORD PTR _exp$[ebp], 0 jne SHORT $LN5#power mov DWORD PTR tv74[ebp], 1 jmp SHORT $LN6#power $LN5#power: cmp DWORD PTR _exp$[ebp], 1 jne SHORT $LN3#power mov eax, DWORD PTR _n$[ebp] mov DWORD PTR tv73[ebp], eax jmp SHORT $LN4#power $LN3#power: mov ecx, DWORD PTR _exp$[ebp] sub ecx, 1 push ecx mov edx, DWORD PTR _n$[ebp] push edx call ?power##YAHHH#Z ; power add esp, 8 push eax mov eax, DWORD PTR _n$[ebp] push eax call ?product##YAHHH#Z ; product add esp, 8 mov DWORD PTR tv73[ebp], eax $LN4#power: mov ecx, DWORD PTR tv73[ebp] mov DWORD PTR tv74[ebp], ecx $LN6#power: mov eax, DWORD PTR tv74[ebp] ; Line 8 mov esp, ebp pop ebp ret 0 ?power##YAHHH#Z ENDP ; power _TEXT ENDS PUBLIC _main ; Function compile flags: /Odtp _TEXT SEGMENT _main PROC ; Line 11 push ebp mov ebp, esp ; Line 12 push 4 push 3 call ?power##YAHHH#Z ; power add esp, 8 ; Line 13 pop ebp ret 0 _main ENDP _TEXT ENDS END To walk you through this: ?power##YAHHH#Z PROC ; power ; Line 4 push ebp mov ebp, esp sub esp, 8 The above is the entry code for the power function - just adjusting the stack pointer to jump over the function arguments, which it will access below as _exp$[ebp] (that's exp) and _n$[ebp] (i.e. n). ; Line 7 cmp DWORD PTR _exp$[ebp], 0 jne SHORT $LN5#power mov DWORD PTR tv74[ebp], 1 jmp SHORT $LN6#power Basically, if exp is not equal to 0 we'll continue at label $LN5#power below, but if it is 0 then load 1 into the return value location on the stack at tv74[ebp] and jump to the function return instructions at $LN6#power. $LN5#power: cmp DWORD PTR _exp$[ebp], 1 jne SHORT $LN3#power mov eax, DWORD PTR _n$[ebp] mov DWORD PTR tv73[ebp], eax jmp SHORT $LN4#power Similar to the above - if exp is 1 then put n into eax and therefrom into the return value stack memory, then jump to the return instructions. Now it starts to get interesting... $LN3#power: mov ecx, DWORD PTR _exp$[ebp] sub ecx, 1 push ecx Subtract 1 from exp and push in onto the stack... mov edx, DWORD PTR _n$[ebp] push edx Also push n onto the stack... call ?power##YAHHH#Z ; power Recursively call the power function, which will use the two values pushes above. add esp, 8 A stack adjustment after the function above returns. push eax Put the result of the recursive call - which the power return instructions leave in the eax register - onto the stack... mov eax, DWORD PTR _n$[ebp] push eax Also push n onto the stack... call ?product##YAHHH#Z ; product Call the product function to multiple the value returned by the call to power above by n. add esp, 8 mov DWORD PTR tv73[ebp], eax Copy the result of product into a temporary address on the stack.... $LN4#power: mov ecx, DWORD PTR tv73[ebp] mov DWORD PTR tv74[ebp], ecx Pick up the value from that tv73 temporary location and copy it into tv74... $LN6#power: mov eax, DWORD PTR tv74[ebp] Finally, move the the product() result from tv74 into the eax register for convenient and fast access after the product call returns. ; Line 8 mov esp, ebp pop ebp ret 0 Clean up the stack and return.
In NASM, I want to compare an input keystroke based on its ASCII value
A simple program I am working on (for Homework) requires that I take a keystroke as input and return the categories it falls under (is it a printable charater, decimal, etc..) I'm using cmp to compare the keystroke against the values of the maximum and/or minimum values in it's category (for example if the ASCII code of the keystroke is above 0x7F then it is a printable character) However, there is obviously something not working in my comparison since no matter what, i.e. when I use the escape button as input, it is not printing "Control Key". Could it be that keys need some more processing before they can be compared based on ASCII value? Here is my code segment .data controlKey: db "Control Key", 10 controlLen: equ $-controlKey printableKey: db "Printable", 10 printableLen: equ $-printableKey decimalKey: db "Decimal", 10 decimalLen: equ $-decimalKey segment .bss key resb 2 segment .text global main main: mov eax, 3 ; system call 3 to get input mov ebx, 0 ; standart input device mov ecx, key ; pointer to id mov edx, 2 ; take in this many bytes int 0x80 control: ; check if it's a control key mov ebx, 31 ; highest control key mov edx, key cmp edx, ebx jg printable mov eax, 4 mov ebx, 1 mov ecx, controlKey mov edx, controlLen int 0x80 ; jmp exit ; It's obviously not any of the other categories printable: ; Tell that it's a printable symbol mov eax, 4 mov ebx, 1 mov ecx, printableKey mov edx, printableLen int 0x80 decimal: mov ebx, 30h ; smallest decimal ASCII mov edx, key cmp edx, ebx jl uppercase mov ebx, 39h ; test against 9 cmp edx, ebx jg uppercase mov eax, 4 mov ebx, 1 mov ecx, decimalKey mov edx, decimalLen int 0x80 uppercase: lowercase: mov eax, 4 ; system call 4 for output mov ebx, 1 ; standard output device mov ecx, key ; move the content into ecx mov edx, 1 ; tell edx how many bytes int 0x80 ; exit: mov eax, 1 xor ebx, ebx int 0x80
The Escape key won't be read by your application, since it is - most probably - caught by the terminal that your application runs in. I can see that you're using the read syscall in your code, which is, of course, fine, but you should remember that this function only provides reading from a file descriptor, which doesn't necessarily have to contain all the control signals sent from the keyboard. The file descriptor (stdin) doesn't even have to come from the keyboard, since a file might be redirected to your process as standard input. I don't know if there's a good way of achieving (capturing keystrokes, not the data that they represent - and this is what you're doing now) what you're trying to do just with system calls in Linux. You could try using some terminal controlling library, for example ncurses or termios, but I guess that isn't a part of your assignment.
I have done this a while back, here is a sample to show how to turn character echo on/off, and canonical mode on/off. When run, when you press a key, the keycode will be displayed on the screen, the program will exit once shift+q is pressed: terminos.asm ICANON equ 1<<1 ECHO equ 1<<3 sys_exit equ 1 sys_read equ 3 sys_write equ 4 stdin equ 0 stdout equ 1 global _start SECTION .bss lpBufIn resb 2 lpBufOut resb 2 termios resb 36 section .text _start: call echo_off call canonical_off .GetCode: call GetKeyCode movzx esi, byte[lpBufIn] push esi call PrintNum pop esi cmp esi, 81 jne .GetCode call echo_on call canonical_on mov eax, sys_exit xor ebx, ebx int 80H ;~ ######################################### GetKeyCode: mov eax, sys_read mov ebx, stdin mov ecx, lpBufIn mov edx, 1 int 80h ret ;~ ######################################### canonical_off: call read_stdin_termios ; clear canonical bit in local mode flags mov eax, ICANON not eax and [termios+12], eax call write_stdin_termios ret ;~ ######################################### echo_off: call read_stdin_termios ; clear echo bit in local mode flags mov eax, ECHO not eax and [termios+12], eax call write_stdin_termios ret ;~ ######################################### canonical_on: call read_stdin_termios ; set canonical bit in local mode flags or dword [termios+12], ICANON call write_stdin_termios ret ;~ ######################################### echo_on: call read_stdin_termios ; set echo bit in local mode flags or dword [termios+12], ECHO call write_stdin_termios ret ;~ ######################################### read_stdin_termios: mov eax, 36h mov ebx, stdin mov ecx, 5401h mov edx, termios int 80h ret ;~ ######################################### write_stdin_termios: mov eax, 36h mov ebx, stdin mov ecx, 5402h mov edx, termios int 80h ret PrintNum: push lpBufOut push esi call dwtoa mov edi, lpBufOut call GetStrlen inc edx mov ecx, lpBufOut mov eax, sys_write mov ebx, stdout int 80H ret ;~ ######################################### GetStrlen: push ebx xor ecx, ecx not ecx xor eax, eax cld repne scasb mov byte [edi - 1], 10 not ecx pop ebx lea edx, [ecx - 1] ret ;~ ######################################### dwtoa: ;~ number to convert = [ebp+8] ;~ pointer to buffer that receives number = [ebp+12] push ebp mov ebp, esp push ebx push esi push edi mov eax, [ebp + 8] mov edi, [ebp + 12] test eax, eax jnz .sign .zero: mov word [edi], 30H jmp .done .sign: jns .pos mov byte [edi], "-" neg eax add edi, 1 .pos: mov ecx, 3435973837 mov esi, edi .doit: mov ebx, eax mul ecx shr edx, 3 mov eax, edx lea edx, [edx * 4 + edx] add edx, edx sub ebx, edx add bl, "0" mov [edi], bl add edi, 1 cmp eax, 0 jg .doit mov byte [edi], 0 .fixit: sub edi, 1 mov al, [esi] mov ah, [edi] mov [edi], al mov [esi], ah add esi, 1 cmp esi, edi jl .fixit .done: pop edi pop esi pop ebx mov esp, ebp pop ebp ret 4 * 2 makefile APP = terminos $(APP): $(APP).o ld -o $(APP) $(APP).o $(APP).o: $(APP).asm nasm -f elf $(APP).asm
Call not returning properly [X86_ASM]
This is C++ using x86 inline assembly [Intel syntax] Function: DWORD *Call ( size_t lArgs, ... ){ DWORD *_ret = new DWORD[lArgs]; __asm { xor edx, edx xor esi, esi xor edi, edi inc edx start: cmp edx, lArgs je end push eax push edx push esi mov esi, 0x04 imul esi, edx mov ecx, esi add ecx, _ret push ecx call dword ptr[ebp+esi] //Doesn't return to the next instruction, returns to the caller of the parent function. pop ecx mov [ecx], eax pop eax pop edx pop esi inc edx jmp start end: mov eax, _ret ret } } The purpose of this function is to call multiple functions/addresses without calling them individually. Why I'm having you debug it? I have to start school for the day, and I need to have it done by evening. Thanks alot, iDomo
Thank you for a complete compile-able example, it makes solving problems much easier. According to your Call function signature, when the stack frame is set up, the lArgs is at ebp+8 , and the pointers start at ebp+C. And you have a few other issues. Here's a corrected version with some push/pop optimizations and cleanup, tested on MSVC 2010 (16.00.40219.01) : DWORD *Call ( size_t lArgs, ... ) { DWORD *_ret = new DWORD[lArgs]; __asm { xor edx, edx xor esi, esi xor edi, edi inc edx push esi start: cmp edx, lArgs ; since you started counting at 1 instead of 0 ; you need to stop *after* reaching lArgs ja end push edx ; you're trying to call [ebp+0xC+edx*4-4] ; a simpler way of expressing that - 4*edx + 8 ; (4*edx is the same as edx << 2) mov esi, edx shl esi, 2 add esi, 0x8 call dword ptr[ebp+esi] ; and here you want to write the return value ; (which, btw, your printfs don't produce, so you'll get garbage) ; into _ret[edx*4-4] , which equals ret[esi - 0xC] add esi, _ret sub esi, 0xC mov [esi], eax pop edx inc edx jmp start end: pop esi mov eax, _ret ; ret ; let the compiler clean up, because it created a stack frame and allocated space for the _ret pointer } } And don't forget to delete[] the memory returned from this function after you're done.
I notice that, before calling, you push EAX, EDX, ESI, ECX (in order), but don't pop in the reverse order after returning. If the first CALL returns properly, but subsequent ones don't, that could be the issue.
analyzing assembly code of some c++ code [closed]
Closed. This question needs details or clarity. It is not currently accepting answers. Want to improve this question? Add details and clarify the problem by editing this post. Closed 9 years ago. Improve this question I were asked to analyze an assembly code, which was generated from following c++ code in Visual studio IDE: here is c++ code: int plus(int a,int b); int main() { cout<<plus(2,4); getchar(); return 0; } int plus(int a,int b) { static int t=2; return a+b+t; } And here is the assembly code (the reduced form): _main PROC ; COMDAT ; 8 : { push ebp mov ebp, esp sub esp, 192 ; 000000c0H push ebx push esi push edi lea edi, DWORD PTR [ebp-192] mov ecx, 48 ; 00000030H mov eax, -858993460 ; ccccccccH rep stosd ; 9 : cout<<plus(2,4); push 4 push 2 call ?plus##YAHHH#Z ; plus add esp, 8 mov esi, esp push eax mov ecx, DWORD PTR __imp_?cout#std##3V?$basic_ostream#DU?$char_traits#D#std###1#A call DWORD PTR __imp_??6?$basic_ostream#DU?$char_traits#D#std###std##QAEAAV01#H#Z cmp esi, esp call __RTC_CheckEsp ; 10 : getchar(); mov esi, esp call DWORD PTR __imp__getchar cmp esi, esp call __RTC_CheckEsp ; 11 : return 0; xor eax, eax ; 12 : } pop edi pop esi pop ebx add esp, 192 ; 000000c0H cmp ebp, esp call __RTC_CheckEsp mov esp, ebp pop ebp ret 0 _main ENDP ; Function compile flags: /Odtp /RTCsu /ZI _TEXT ENDS ; COMDAT ?plus##YAHHH#Z _TEXT SEGMENT _a$ = 8 ; size = 4 _b$ = 12 ; size = 4 ?plus##YAHHH#Z PROC ; plus, COMDAT ; 15 : { push ebp mov ebp, esp sub esp, 192 ; 000000c0H push ebx push esi push edi lea edi, DWORD PTR [ebp-192] mov ecx, 48 ; 00000030H mov eax, -858993460 ; ccccccccH rep stosd ; 16 : static int t=2; ; 17 : return a+b+t; mov eax, DWORD PTR _a$[ebp] add eax, DWORD PTR _b$[ebp] add eax, DWORD PTR ?t#?1??plus##YAHHH#Z#4HA ; 18 : } pop edi pop esi pop ebx mov esp, ebp pop ebp ret 0 ?plus##YAHHH#Z ENDP ; plus _TEXT ENDS END I have to find how does the code deal with stack and how variables stored and retrieved? Regards.
push 4 push 2 call ?plus##YAHHH#Z This pushes the values 4 and 2 onto the stack (reverse order to how you'd think of them in C, remember 2 is now on top of 4), then calls plus. mov eax, DWORD PTR _a$[ebp] add eax, DWORD PTR _b$[ebp] add eax, DWORD PTR ?t#?1??plus##YAHHH#Z#4HA pop edi pop esi pop ebx mov esp, ebp pop ebp ret 0 I've ignored some stack fiddling at the top of plus, but this moves a from the stack into eax, adds b to it (which it gets from the stack) then adds t to it (I'm not familiar with MASM at all so I'm not actually sure where it gets t from). You can see the stack offsets of a and b have been stored into _a and _b as 8 and 12 further up the code. This is performed in %eax because this is where you stick the first return value of a function. There's some stack clearing and then the usual function epilogue before returning. The main code then pushes %eax onto the stack and calls the iostream stuff, which will pop it off and output it to screen.
Here is the complete tutorial: http://www.codeproject.com/KB/cpp/reversedisasm.aspx Please ask a specific question if you have ? Your original question is too broad.