I am trying to create a Video Player inside GTKmm, for this I am using mpv. The documentation says, that I can embed the video player using an OpenGL view. However, I am having difficulties implementing the player inside a GTKmm app.
I have a GLWindow, that contains a GLArea which should then contain the video player. The problem is, that when I try to initialize the mpv render context, I get an error, telling me that the OpenGL was not initialized.
The following is my Constructor for the main window that I have:
GLWindow::GLWindow(): GLArea_{}
{
set_title("GL Area");
set_default_size(400, 600);
setlocale(LC_NUMERIC, "C");
VBox_.property_margin() = 12;
VBox_.set_spacing(6);
add(VBox_);
GLArea_.set_hexpand(true);
GLArea_.set_vexpand(true);
GLArea_.set_auto_render(true);
GLArea_.set_required_version(4, 0);
VBox_.add(GLArea_);
mpv = mpv_create();
if (!mpv)
throw std::runtime_error("Unable to create mpv context");
mpv_set_option_string(mpv, "terminal", "yes");
mpv_set_option_string(mpv, "msg-level", "all=v");
if (mpv_initialize(mpv) < 0)
throw std::runtime_error("could not initialize mpv context");
mpv_render_param params[] = {
{MPV_RENDER_PARAM_API_TYPE, const_cast<char*>(MPV_RENDER_API_TYPE_OPENGL)},
{MPV_RENDER_PARAM_OPENGL_INIT_PARAMS, static_cast<void*>(new (mpv_opengl_init_params){
.get_proc_address = get_proc_address,
})},
{MPV_RENDER_PARAM_INVALID}
};
if (mpv_render_context_create(&mpv_gl, mpv, params) < 0)
throw std::runtime_error("Failed to create render context");
mpv_render_context_set_update_callback(mpv_gl, GLWindow::onUpdate, this);
}
As far as I know, this should just initialize the video player view, but the problem arises when I try to create the render context with mpv_render_context_create. I get the following error on that line:
[libmpv_render] glGetString(GL_VERSION) returned NULL.
[libmpv_render] OpenGL not initialized.
Then the app terminates with a SIGSEGV Signal.
The problem may be from my get_proc_address function, currently I have only implemented it for linux, it looks like the following:
static void *get_proc_address(void *ctx, const char *name) {
return (void *)glXGetProcAddress(reinterpret_cast<const GLubyte *>(name));
}
To be honest, I am overwhelmed as to why the OpenGL context is not being created. How do I have to adjust my GTKmm app to allow the mpv video player to initialize correctly?
As the error suggested, the problem was that there was no OpenGL context. The GLArea is not instantly created, there is an event signal_realize on the GLArea when the OpenGL view has been created. I had to listen to that event, and in there initialize the mpv variables after setting GLArea.make_current(), to set the GLArea's context to the one we want to connect to mpv
Related
ok so basically I am trying to inject a DLL into a game for an external menu for debugging and my hook works completely fine, i can render a normal square fine to the screen but when i try to render imgui, some games DirectX just dies and some others nothing renders at all. The issue makes no sense because I've tried everything, i've switched libraries, tried different compile settings and just started doing random shit but still to no avail, the library i am using for hooking is minhook (was using kiero but from trying to figure out the issue switched to manually getting the D3D Device).
My hooks work entirely fine as I said earlier, I can render a square to the screen without issues but I cant render imgui (and yes i checked it is the DX9 version of imgui), code:
long __stdcall EndSceneHook(IDirect3DDevice9* pDevice) // Our hooked endscene
{
D3DRECT BarRect = { 100, 100, 200, 200 };
pDevice->Clear(1, &BarRect, D3DCLEAR_TARGET, D3DCOLOR_ARGB(255, 0, 255, 0), 0.0f, 0);
if (!EndSceneInit) {
ImGui::CreateContext();
ImGuiIO& io = ImGui::GetIO();
ImGui_ImplWin32_Init(TrackmaniaWindow);
ImGui_ImplDX9_Init(pDevice);
EndSceneInit = true;
return OldEndScene(pDevice);
}
ImGui_ImplDX9_NewFrame();
ImGui_ImplWin32_NewFrame();
ImGui::NewFrame();
ImGui::ShowDemoWindow();
ImGui::EndFrame();
ImGui::Render();
ImGui_ImplDX9_RenderDrawData(ImGui::GetDrawData());
return OldEndScene(pDevice); // Call original ensdcene so the game can draw
}
And if you are considering saying i forgot to hook Reset I did but the game pretty much never calls it so I probably did it wrong, code for that:
long __stdcall ResetHook(IDirect3DDevice9* pDevice, D3DPRESENT_PARAMETERS Parameters) {
/* Delete imgui to avoid errors */
ImGui_ImplDX9_Shutdown();
ImGui_ImplWin32_Shutdown();
ImGui::DestroyContext();
/* Check if its actually being called */
if (!ResetInit) {
std::cout << "Reset called correctly" << std::endl;
ResetInit = true;
}
/* Return old function */
return OldReset(pDevice, Parameters);
}
Just incase I did mess up the hooking process for 1 of the functions i will also include the code i used to actually hook them
if (MH_CreateHook(vTable[42], EndSceneHook, (void**)&OldEndScene) != MH_OK)
ThrowError(MinHook_Hook_Creation_Failed);
if (MH_CreateHook(vTable[16],OldReset,(void**)&OldReset)!=MH_OK)
ThrowError(MinHook_Hook_Creation_Failed);
MH_EnableHook(MH_ALL_HOOKS);
Ok so I solved the issue already, but just incase anyone else needs help I have found a few fixes as to why it would crash/not render.
First one being EnumWindow(), if you are using EnumWindows() to get your target processes HWND then that is likely one or your entire issue,
For internal cheats, Use GetForegroundWindow() when the game is loaded, or you can use FindWindow(0,"Window Name") (works for both external and internal [game needs to be loaded])
void MainThread(){
HWND ProcessWindow = 0;
WaitForProcessToLoad(GameHandle); // This is just an example of waiting for the game to load
ProcessWindow = GetForegroundWindow(); // We got the HWND
// or
ProcessWindow = FindWindow(0,"Thing.exe");
}
To start off with the 2nd possible issue, make sure your replacement functions for the functions your hooking are actually passing the right arguments (this is if your hook instantly crashes), and make sure you are returning the original function.
Make sure your WndProc function is working correctly (if you dont know how then google DX9 Hooking Tutorials and copy + paste there code for that).
Last fix is how you are rendering imgui to the screen, if imgui isnt rendering after the first fix then it is likely because you arent calling a function that is required, This is an example of correctly made imgui rendering
long __stdcall EndSceneHook(IDirect3DDevice9* pDevice) // Our hooked endscene
{
if (!EndSceneInit) {
ImGui::CreateContext();
ImGuiIO& io = ImGui::GetIO();
ImGui::StyleColorsDark();
ImGui_ImplWin32_Init(Window);
ImGui_ImplDX9_Init(pDevice);
EndSceneInit = true;
return OldEndScene(pDevice);
}
ImGui_ImplDX9_NewFrame();
ImGui_ImplWin32_NewFrame();
ImGui::NewFrame();
ImGui::ShowDemoWindow();
ImGui::EndFrame();
ImGui::Render();
ImGui_ImplDX9_RenderDrawData(ImGui::GetDrawData());
return OldEndScene(pDevice); // Call original ensdcene so the game can draw
}
If none of these fixes worked then google the error or youtube DX9 hooking tutorials
I need to request and retrieve an image of a window from the X server or the WM (I believe the WM does the actual compositing). I need to be able to get an image of the window even if it is obscured by another window or located on another workspace (but still mapped). I need to use C or C++ and would like to use XCB or Xlib. I think I also have Gtk+ development tools installed but it's been a while since I used them.
I have tried using xcb_composite_name_window_pixmap() with no success (XCB errors).
I tried using the render extension to render the window to a picture which produced no errors, but I still needed to get the image loaded into my program's memory.
I tried to use xcb_get_image to get the picture which, IIRC, failed (bad resource I believe).
I tried to copy the picture to a pixmap and use xcb_get_image on that. I was able to download what appeared to be an image, but it looked more like random regions of the screen than the actual window.
In short, I'm just making guesses at this point.
I'm having a difficult time finding organized and complete documentation. The most recent publications I can find on X11, XCB, or Xlib is from 1994 (The O'Reilly book on Xlib and/or X11R6) which I doubt has much, if anything, valid on these extensions. Most of the man pages and online documentation has a lot of "TODO: explain this" and/or function descriptions like "deliver a request to the X server". Any help that anyone can provide will be of use to me.
I'm currently running compiz for my WM and emerald for window decorations and nothing else standard in terms of a "desktop environment". I'm working on some utility applications for a custom desktop that I plan to release when they are ready. I would like whatever I make to work with some other WMs but, if it requires special codepaths for each, I can add the others down the line.
EDIT: I originally had added a non-working example here, and then a stripped down working example, which has all been moved to an answer as suggested in the comments.
I now have a working example that I will post here:
#include <xcb/xcb.h>
#include <xcb/xproto.h>
#include <xcb/composite.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char **argv) {
if (argc < 2) {
fprintf(stderr, "usage: %s windowId\n", argv[0]);
return EXIT_FAILURE;
}
xcb_window_t req_win_id = strtoul(argv[1], NULL, 0);
xcb_connection_t *connection = xcb_connect(NULL, NULL);
xcb_generic_error_t *err = NULL, *err2 = NULL;
xcb_composite_query_version_cookie_t comp_ver_cookie = xcb_composite_query_version(connection, 0, 2);
xcb_composite_query_version_reply_t *comp_ver_reply = xcb_composite_query_version_reply(connection, comp_ver_cookie, &err);
if (comp_ver_reply)
{
if (comp_ver_reply->minor_version < 2) {
fprintf(stderr, "query composite failure: server returned v%d.%d\n", comp_ver_reply->major_version, comp_ver_reply->minor_version);
free(comp_ver_reply);
return EXIT_FAILURE;
}
free(comp_ver_reply);
}
else if (err)
{
fprintf(stderr, "xcb error: %d\n", err->error_code);
free(err);
return EXIT_FAILURE;
}
const xcb_setup_t *setup = xcb_get_setup(connection);
xcb_screen_iterator_t screen_iter = xcb_setup_roots_iterator(setup);
xcb_screen_t *screen = screen_iter.data;
// request redirection of window
xcb_composite_redirect_window(connection, req_win_id, XCB_COMPOSITE_REDIRECT_AUTOMATIC);
int win_h, win_w, win_d;
xcb_get_geometry_cookie_t gg_cookie = xcb_get_geometry(connection, req_win_id);
xcb_get_geometry_reply_t *gg_reply = xcb_get_geometry_reply(connection, gg_cookie, &err);
if (gg_reply) {
win_w = gg_reply->width;
win_h = gg_reply->height;
win_d = gg_reply->depth;
free(gg_reply);
} else {
if (err) {
fprintf(stderr, "get geometry: XCB error %d\n", err->error_code);
free(err);
}
return EXIT_FAILURE;
}
// create a pixmap
xcb_pixmap_t win_pixmap = xcb_generate_id(connection);
xcb_composite_name_window_pixmap(connection, req_win_id, win_pixmap);
// get the image
xcb_get_image_cookie_t gi_cookie = xcb_get_image(connection, XCB_IMAGE_FORMAT_Z_PIXMAP, win_pixmap, 0, 0, win_w, win_h, (uint32_t)(~0UL));
xcb_get_image_reply_t *gi_reply = xcb_get_image_reply(connection, gi_cookie, &err);
if (gi_reply) {
int data_len = xcb_get_image_data_length(gi_reply);
fprintf(stderr, "data_len = %d\n", data_len);
fprintf(stderr, "visual = %u\n", gi_reply->visual);
fprintf(stderr, "depth = %u\n", gi_reply->depth);
fprintf(stderr, "size = %dx%d\n", win_w, win_h);
uint8_t *data = xcb_get_image_data(gi_reply);
fwrite(data, data_len, 1, stdout);
free(gi_reply);
}
return EXIT_SUCCESS;
}
It was a case of overdoing it. I got the idea that I should use Xrender from somewhere and started pulling tons of data from that extension that I didn't really need. But the real problem I had was that the pixmap given to xcb_composite_name_window_pixmap should not be created first, but rather I just needed the new id for it. It turns out that it is really as simple as I expected it to be. All you need is the Window ID and the size, do some checks to make sure composite is there and a good version (>=0.2 in this case), and call xcb_composite_redirect_window, xcb_composite_name_window_pixmap (with a generated pixmap id), and xcb_get_image (+reply,data_len,data) in that order, and you have the image.
I have developed an OpenGL ES 2.0 win32 application, that works fine in a single thread. But I also understand that UI thread and a rendering thread should be separate.
Currently my game loop looks something like that:
done = 0;
while(!done)
{
msg = GetMessage(..); // getting messages from OS
if(msg == QUIT) // the window has been closed
{
done = 1;
}
DispatchMessage(msg,..); //Calling KeyDown, KeyUp events to handle user input;
DrawCall(...); //Render a frame
Update(..); // Update
}
Please view it as a pseudo code, cause i don't want to bother you with details at this point.
So my next step was to turn done into an std::atomic_int and create a function
RenderThreadMain()
{
while(!done.load())
{
Draw(...);
}
}
and create a std::unique_ptr<std::thread> m_renderThread variable. As you can guess nothing has worked for me so far, so i made my code as stupid and simple as possible in order to make sure i don't break anything with the order i call methods in. So right now my game loop works like this.
done.store(0);
bool created = false;
while(!done)
{
msg = GetMessage(..); // getting messages from OS
if(msg == QUIT) // the window has been closed
{
done.store(1);
}
DispatchMessage(msg,..); //Calling KeyDown, KeyUp events to handle user input;
// to make sure, that my problem is not related to the fact, that i'm rendering too early.
if(!created)
{
m_renderThread = std::make_unique<std::thread>(RenderThreadMain, ...);
created = true;
}
Update(..); // Update
}
But this doesn't work. On every draw call, when i try to somehow access or use my buffers \ textures anything else, i get the GL_INVALID_OPERATION error code.
So my guess would be, that the problem is in me calling glGenBuffers(mk_bufferNumber, m_bufferIds); in the main thread during initialization and then calling glBindBuffer(GL_ARRAY_BUFFER, m_bufferIds[0]); in a render thread during the draw call. (the same applies to every openGL object i have)
But I don't now if i'm right or wrong.
I'm working on olimex a13 board with just eglfs i.e, no windowing system. Because of this Qt Multimedia stuff video and camera aren't working as Qt uses Gstreamer which in turn needs X. So I'm using QtGstreamer library which is here.
I've followed the examples and created a media player which is working as expected. Now, I want to do a camera and using camerabin2 which is from bad plugins.
This is my code:
//init QGst
QGst::init(&argc, &argv);
//create video surface
QGst::Quick::VideoSurface* videoSurface = new QGst::Quick::VideoSurface(&engine);
CameraPlayer player;
player.setVideoSink(videoSurface->videoSink());
//cameraplayer.cpp
void open()
{
if (!m_pipeline) {
m_pipeline = QGst::ElementFactory::make("camerabin").dynamicCast<QGst::Pipeline>();
if (m_pipeline) {
m_pipeline->setProperty("video-sink", m_videoSink);
//watch the bus for messages
QGst::BusPtr bus = m_pipeline->bus();
bus->addSignalWatch();
QGlib::connect(bus, "message", this, &CameraPlayer::onBusMessage);
//QGlib::connect(bus, "image-done", this, &CameraPlayer::onImageDone);
} else {
qCritical() << "Failed to create the pipeline";
}
}
}
//-----------------------------------
void CameraPlayer::setVideoSink(const QGst::ElementPtr & sink)
{
m_videoSink = sink;
}
//-----------------------------------
void CameraPlayer::start()
{
m_pipeline->setState(QGst::StateReady);
m_pipeline->setState(QGst::StatePlaying);
}
I then call cameraPlayer.start() which isn't working i.e, no video. Am I missing something here? Has anyone used QtGstreamer to stream webcam? Thanks in advance.
I realised some plugins (multifilesink) were missing. Started my Qt application with --gst-debug-level=4 argument and gstreamer then reported about missing plugins.
I try to write down code from this tutorial. I have the code of InitializeOGL():
bool Ogl::InitializeOGL(bool vSync)
{
cout<<"Init OpenGL"<<endl;
int pixelFormat;
PIXELFORMATDESCRIPTOR pixelFormatDescriptor;
int result;
char *vendorChar, *rendererChar;
hDC = GetDC(hWnd);
if(!hDC)
return false;
pixelFormat = ChoosePixelFormat(hDC,&pixelFormatDescriptor);
if(pixelFormat==0)
return false;
result = SetPixelFormat(hDC,pixelFormat,&pixelFormatDescriptor);
if(result!=1)
return false;
HGLRC tempDeviceContext = wglCreateContext(hDC);
wglMakeCurrent(hDC,tempDeviceContext);
// glewExperimental = GL_TRUE;
if(glewInit()!=GLEW_OK)
return false;
int attribList[5] =
{
WGL_CONTEXT_MAJOR_VERSION_ARB, 3,
WGL_CONTEXT_MINOR_VERSION_ARB, 1, 0
};
hGLRC = wglCreateContextAttribsARB(hDC,0,attribList);
if(hGLRC!=NULL)
{
wglMakeCurrent(NULL,NULL);
wglDeleteContext(tempDeviceContext);
result = wglMakeCurrent(hDC,hGLRC);
if(result!=1)
return false;
}
vendorChar = (char*)glGetString(GL_VENDOR);
rendererChar = (char*)glGetString(GL_RENDERER);
strcpy_s(videoCardInfo,vendorChar);
strcat_s(videoCardInfo,"-");
strcat_s(videoCardInfo,rendererChar);
if(vSync)
result = wglSwapIntervalEXT(1);
else
result = wglSwapIntervalEXT(0);
if(result!=1)
return false;
int glVersion[2] = {-1,-1};
glGetIntegerv(GL_MAJOR_VERSION,&glVersion[0]);
glGetIntegerv(GL_MINOR_VERSION,&glVersion[1]);
cout<<"Initializing OpenGL"<<endl;
cout<<"OpenGL version"<<glVersion[0]<<"."<<glVersion[1]<<endl;
cout<<"GPU"<<videoCardInfo<<endl;
return 0;
}
When I try to change context version to OpenGL 3.1, here crashes wglCreateContextAttribsARB()
function (Pointer for this is getting in LoadExtensions() correctly). When I try to create OpenGL 4.0 here crashes function wglSwapIntervalEXT(). My graphic card handles only OpenGL 3.1.
My question is how to succefully init OpenGL context here? What I have to do to create OpenGL context in version 3.1.
There are a couple of things that need to be mentioned here:
1. Driver Version
If your graphics card / driver only support OpenGL 3.1, then WGL_CONTEXT_MAJOR_VERSION_ARB and friends are generally going to be undefined. Before OpenGL 3.2 introduced core / compatibility, context versions were not particularly meaningful.
This requires support for either WGL_ARB_create_context or WGL_ARB_create_conext_profile.
2. Incorrect usage of ChoosePixelFormat and SetPixelFormat
PIXELFORMATDESCRIPTOR pixelFormatDescriptor; // <--- Uninitialized
At minimum, the Win32 API needs you to initialize the size field of this structure. In years past, the size of structures was used to determine the version of Windows that a particular piece of code was written for. These days structures like PIXELFORMATDESCRIPTOR are generally static in size because they are used by a part of Windows that is deprecated (GDI), but if you do not set the size you can still thoroughly confuse Windows. Furthermore, you need to flag your pixel format to support OpenGL to guarantee that you can use it to create an OpenGL render context.
Also note that once you set the pixel format for a device context on Windows, it cannot be changed. Generally, this means if you want to create a dummy render context to initialize your extensions you should also create a dummy window with a dummy pixel format. After you initialize your extensions, you can use wglChoosePixelFormatARB (...) and its associated functions to select the pixel format for your main window's device context.
This is (was) particularly important back in the days before FBOs when you wanted to implement multi-sampling. You cannot get a multi-sample pixel format using ChoosePixelFormat (...), but you need to call ChoosePixelFormat (...) to setup the extension necessary to get a multi-sample pixel format. Kind of a catch 22.