I understand that you usually create complex 3D models in Blender or some other 3D modelling software and export it afterwords as .obj. This .obj file gets parsed into your program and openGL will render it. This as far as I understand real-time rendering.
Now I was wondering if there is something like pre-rendered objects. I'm a little bit confused because there are so many articles/videos about real-time rendering but I haven't found any information about none real-time rendering. Does something like this exists or not? The only thing which would come into my mind as none real-time rendering would be a video.
I guess this is pretty much a yes or no question :) but if it exists maybe someone could point me to some websites with explanations.
"Real-time rendering" means that the frames are being generated as fast as they can be displayed. "Non-real-time rendering", or "offline rendering" means generating frames one at a time, taking as much time as necessary to achieve the desired image quality, and then later assembling them into a movie. Video at the quality of video games can be rendered in real time; something as elaborate as a Pixar movie, though, has to be done in offline mode. Individual frames can still take hours of rendering time!
It's not entirely clear what you mean by "prerendered objects", however there are things called VBOs and Vertex Arrays that store the object's geometry in VRAM so as to not have to load it into the rendering pipeline using glVertex3f() or similar every frame. This is called Immediate Mode.
VBOs and Vertex arrays are used instead of immediate mode because they're far faster than calling the graphics driver to load data into VRAM for every vertex because they are kept in VRAM, which is faster than normal RAM, ready to be booted into the render pipeline.
The page here may help, too.
There's nothing stopping you from rendering to an off-screen frame-buffer (i.e., an FBO) and then saving that to disk rather than displaying it to the screen. For instance, that's how GPGPU techniques used to work before the advent of CUDA, OpenCL, etc. ... You would load your data as an unfiltered floating point texture, perform your calculation using pixel-shaders on the FBO, and then save the results back to disk.
In the link I posted above, it states in the overview:
This extension defines a simple interface for drawing to rendering
destinations other than the buffers provided to the GL by the
window-system.
It then goes on to state,
By allowing the use of a framebuffer-attachable image as a rendering
destination, this extension enables a form of "offscreen" rendering.
So, you would get your "non-real-time" rendering by rendering off-screen some scene that renders slower than 30fps, and then saving those results to some movie file or file-sequence format that can be played back at a later date.
Related
I've read a lot of posts describing how people use AVAssetReader or AVPlayerItemVideoOutput to get video frames as raw pixel data from a video file, which they then use to upload to an OpenGL texture. However, this seems to create the needless step of decoding the video frames with the CPU (as opposed to the graphics card), as well as creating unnecessary copies of the pixel data.
Is there a way to let AVFoundation own all aspects of the video playback process, but somehow also provide access to an OpenGL texture ID it created, which can just be drawn into an OpenGL context as necessary? Has anyone come across anything like this?
In other words, something like this pseudo code:
initialization:
open movie file, providing an opengl context;
get opengl texture id;
every opengl loop:
draw texture id;
If you were to use the Video Decode Acceleration Framework on OS X, it will give you a CVImageBufferRef when you "display" decoded frames, which you can call CVOpenGLTextureGetName (...) on to use as a native texture handle in OpenGL software.
This of course is lower level than your question, but it is definitely possible for certain video formats. This is the only technique that I have personal experience with. However, I believe QTMovie also has similar functionality at a much higher level, and would likely provide the full range of features you are looking for.
I wish I could comment on AVFoundation, but I have not done any development work on OS X since 10.6. I imagine the process ought to be similar though, it should be layered on top of CoreVideo.
I am very new to 3D programming, namely with DirectX. I have been trying to follow tutorials on how to do basic things, and I have been looking at the samples provided by Microsoft. One of the big questions I have had is how to tell what calculations should be done in the actual game code and what calculations should be done in HLSL. I have not been able to understand what should be done where, because it looks like, to me, you could have almost all code pertaining to calculations in your shader file, or you could have it all in the executable code and only send the bear minimum to the pixel and vertex shaders. How can one tell what code should go where? If you need an example, I'll try to find one.
"Code" - CPU code
"HLSL" - GPU code
Basically, you want everything that is pure graphics to happen on the GPU. That is, when the information about what you want to render has been sent to the GPU, it should take over and use that information to generate the final image.
You want to the CPU to say to the GPU "this is what I want to render, and here is everything you need to make it happen" and then make sure to tell the GPU "this is how you render it".
Some examples (not a complete or final list in anyway):
CPU:
Anything dealing with window opening/closing/resizing
User input from mouse, keyboard
Reading and setting configuration
Generating and updating view matrices
Application logic
Setting up and initializing rendering (textures, buffers etc)
Generating vertex data (position, texture coordinates etc)
Creating graphic entities (triangles, textures, colors etc)
Handling animation (timestepping, swapping buffers)
Sending updated data to the GPU for each frame
GPU:
Use the view matrices to put things on the right place on the screen
Interpolate from vertex data to fragment data
Shading (usually, this is the most complicated part)
Calculate and write final pixel color
I realize this is probably a ridiculous question, but before trying to figure out what libraries to use for which projects, I think it makes sense to really understand the purpose of such libraries first.
A lot of video games use libraries like OpenGL. All the tutorials I've seen of such libraries demonstrate how to write code that tells the computer to draw something. Thing is, in games these days everything is modeled using software such as Zbrush, Maya, or 3ds Max. The models are textured and are good to go. It seems like all you'd need to do is write an animation loop that draws the models and updates repeatedly rather than actually program the code to draw every little thing. That would be both extremely time consuming and would make the models useless. So where does OpenGL or Direct 3D come in in relation to video games and 3d art? What is so crucial about them when all the graphics are already created and just need to be loaded and drawn? Are they used mainly for shaders and effects?
This question may just prove how new I am to this, but it's one I've never heard asked. I'm just starting to learn programming and I'm understanding the code and logic fairly well, but I don't understand graphics libraries or certain frameworks at all and tutorials are not helping.
It seems like all you'd need to do is write an animation loop that draws the models and updates repeatedly rather than actually program the code to draw every little thing.
Everything that happens in a computer does so because a program of some form tells it exactly what to do. The letters that this message is composed of only appear because your web-browser of choice downloaded this file via TCP/IP over an HTTP protocol, decoded its UTF-8-encoded text, interpreted that text as defined by the XML, HTML, JavaScript, and so forth standards, and then displayed the visible portion as defined by the Unicode standard for text layout and in accord with HTML et al, using the displaying and windowing abilities of your OS or window manager or whatever.
Every single part of that operation, from the downloading of the file to its display, is governed by a piece of code. Every pixel you are looking at on the screen is where it is because some code put it there.
HTML alone doesn't mean anything. You cannot just take an HTML file and blast it to the screen. Some code must interpret it. You can interpret HTML as a text file, but if you do, it loses all formatting, and you get to see all of the tags. A web browsers can interpret it as proper HTML, in which case you get to see the formatting. But in every case, the meaning of the HTML file is determined by how it is used.
The "draws the model" part of your proposed algorithm must be done by someone. If you don't write that code, then you must be using a library or some other system that will cause the model to appear. And what does that library do? How does it cause the model to appear?
A model, like an HTML web page, is meaningless by itself. Or to put it another way, your algorithm can be boiled down to this:
Animate the model.
????
Profit!
You're missing a key component: how to actually interpret the model and cause it to appear on the screen. OpenGL/D3D/a software rasterizer/etc is vital for that task.
A lot of video games use libraries like OpenGL.
First and foremost: OpenGL is not a library per-se, but an API (specification). The OpenGL API may be implemented in form as a software library, but these days is much more common to implement OpenGL in form of a driver that turns OpenGL function calls into control commands to a graphics processor sitting on a graphics card (GPU).
All the tutorials I've seen of such libraries demonstrate how to write code that tells the computer to draw something.
Yes. This is because things need to be drawn to make any use of them.
Thing is, in games these days everything is modeled using software such as Zbrush, Maya, or 3ds Max.
At this point the models just consist of a large list of numbers, and further numbers that tell, how the other numbers form some sort of geometry. Those numbers are not some sort of ready to use image.
The models are textured and are good to go.
They are a bunch of numbers, and what they have is some additional numbers controlling texturing. The textures themself are in turn just numbers.
It seems like all you'd need to do is write an animation loop that draws the models
And how do you think this drawing is going to happen? There's no magic "here you have a model, display it" function. Because for one the way in which the numbers making up a model may have any kind of meaning. So some program must give meaning to those numbers. And that is a renderer.
and updates repeatedly rather than actually program the code to draw every little thing.
Again, there is no magic "draw it" function. Drawing a model involves going through each of its numbers, it consists of, and turning those into drawing commands to the GPU.
That would be both extremely time consuming and would make the models useless.
How are the models useless, when they are what is controlling the issuing of commands to OpenGL. Or do you think OpenGL is used to actually "create" models?
So where does OpenGL or Direct 3D come in in relation to video games and 3d art?
It is used to turn the numbers a 3D model, as it is saved away from a modeller, into something pleasant to look at.
What is so crucial about them when all the graphics are already created
The graphics is not yet created, when the model is done. What's created is a model, and some auxilliary data in form of textures and shaders, which are then turned into graphics in realtime, at the execution time of the program.
and just need to be loaded and drawn?
Again, after being loaded, a model is just a bunch of numbers. And drawing means, turning those numbers into something to look at, which requires sending drawing commands to the graphics processor (GPU), which happens using a API like OpenGL or Direct3D
Are they used mainly for shaders and effects?
They are used to turn the numbers generated by a 3D modelling program (Blender, Maya, ZBrush) into an actual picture.
You have data. Like a model, with vertices, normals, and textures. As #datenwolf stated above, those are all just numbers sitting on the hard drive or in RAM, not colors on the screen.
Your CPU (which is where the program you write runs) can't talk to the screen directly. Instead, you send the data you want to draw to the GPU. The GPU then draws the data. Graphics APIs like OpenGL and Direct3D allow programs running on the CPU to send data to the GPU and customize how the GPU draws it. This is a gross simplification, but it sounds like you just need an overview.
Ultimately, every graphics program must go through a graphics API. When you draw an image, for example, you send the GPU the image, and the GPU draws it on the screen. Draw some text? Send the data to the GPU. The GPU draws it. Remember, your code can't talk to the screen. It CAN talk to the GPU through OpenGL or Direct3D, and the GPU then draws the data.
Before OpenGL and DirectX, the games had to use special instructions depending on what graphics card you had. When you bought a new game, you had to check carefully if your card was supported, or you couldn't use the game.
OpenGL and DirectX is a standardized API to the grapics cards. A library is delivered by the manufacturer of the card. If they follow the specification, you are guaranteed that games will work (if they also follow the same specification).
Open Graphics Library (OpenGL) is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. The API is typically used to interact with a graphics processing unit (GPU), to achieve hardware-accelerated rendering.
I need to render certain scenes and read the whole image back in main memory. I've search for this and it seems that most video cards will accelerate the rendering but the read-back will be very slow. After a bit of research i only found this card mentioning "Hardware-Accelerated Pixel Read-Back"
The other approach would do software rendering and the read-back problem doesn't exist, but then the rendering performance will be bad.
Likely, i will have to implement both in order to be able to find the optimal trade-off, but my question is about what other alternative can i have hardware-wise; i understand Quadro is for modelling and designer market segment, which is precisely the client target of this application, Does this means that i'm not likely to find better pixel read-back performance in other video card lines? i.e: Tesla or Fermi, which don't even have video outputs btw
I don't know if the performance would be any different, but you could at least try rendering to an off-screen buffer, then setting that as a texture of a full-screen quad (or outputting that to video in some other way)
Is it possible to allocate some memory on the GPU without cuda?
i'm adding some more details...
i need to get the video frame decoded from VLC and have some compositing functions on the video; I'm doing so using the new SDL rendering capabilities.
All works fine until i have to send the decoded data to the sdl texture... that part of code is handled by standard malloc which is slow for video operations.
Right now i'm not even sure that using gpu video will actually help me
Let's be clear: are you are trying to accomplish real time video processing? Since your latest update changed the problem considerably, I'm adding another answer.
The "slowness" you are experiencing could be due to several reasons. In order get the "real-time" effect (in the perceptual sense), you must be able to process the frame and display it withing 33ms (approximately, for a 30fps video). This means you must decode the frame, run the compositing functions (as you call) on it, and display it on the screen within this time frame.
If the compositing functions are too CPU intensive, then you might consider writing a GPU program to speed up this task. But the first thing you should do is determine where the bottleneck of your application is exactly. You could strip your application momentarily to let it decode the frames and display them on the screen (do not execute the compositing functions), just to see how it goes. If its slow, then the decoding process could be using too much CPU/RAM resources (maybe a bug on your side?).
I have used FFMPEG and SDL for a similar project once and I was very happy with the result. This tutorial shows to do a basic video player using both libraries. Basically, it opens a video file, decodes the frames and renders them on a surface for displaying.
You can do this via Direct3D 11 Compute Shaders or OpenCL. These are similar in spirit to CUDA.
Yes, it is. You can allocate memory in the GPU through OpenGL textures.
Only indirectly through a graphics framework.
You can use OpenGL which is supported by virtually every computer.
You could use a vertex buffer to store your data. Vertex buffers are usually used to store points for rendering, but you can easily use it to store an array of any kind. Unlike textures, their capacity is only limited by the amount of graphics memory available.
http://www.songho.ca/opengl/gl_vbo.html has a good tutorial on how to read and write data to vertex buffers, you can ignore everything about drawing the vertex buffer.