Chromium/WebKit render to OpenGL texture - c++

Last few days I have been looking around the chromium and WebKit source codes, reading wikis, and watching Google videos. What I want to do is take what WebKit renders and place it into a GL texture. But I need to have different DOM nodes in different textures. I have a few questions and Im not sure if I should go about using Chromium or implementing my own simple browser. Chromium obviously has many nice features, but it is very large and extensive. I also figure that it's algorithms for splitting render layers are unpredictable (I want pretty much full control).
Where should in WebKit or Chromium's source to find where it outputs raster data? It would be convenient if I could get access to Chromium's render layer raster data before it is composted. But as I said the render layers would probably me mixed in a way I didn't want them to be.
Is WebKit GPU accelerated, in that case I should be able to access the data directly. I know Chromium+Blink is but I can't find out if WebKit on its own is.
How much work is it too put together a simple browser?
P.S. I can't use Awesomium because I need to render different DOM nodes/subtrees into different textures. Chromium Embedded Framework doesn't appear to have support for DOM manipulation either and I believe it just renders the entire page and gives you the raster data.

Related

Pango layout flow around container (image)

I'm using Pango for text layouting without the cairo backend (currently testing with the win32 backend). And I like to know if pango is capable of a flow layout around an image, or any given container. Or maybe inside a custom container.
Something like this: Flow around image
I have checked many examples and the Pango API and didn't found such a feature. Maybe I'm missing something or Pango does not have this feature.
As I said in this answer, you can't. I went through the source code Pango graphics handling is primitive to the point of uselessness. Unless there's been some major reworking in the past year, which the release notes don't indicate, it's probably the same now.
The image you provide as an example is only available as PDF at the moment which requires every line, word and glyph be hard-positioned on the page. While theoretically possible to check the alpha channel of the image to wrap the text around the actual image instead of the block it contains, this has not (to the best of my knowledge) ever been implemented in a dynamic output system.
Pango, specifically, cannot even open "holes" in the text for graphics to be added later and, at the code level, doesn't even have the concept of a multi-line cell - hence a line being the size of its largest component.
Your best bet is to look at WebKit for more complex displays. I, for one, have pretty much given up on Pango and it seems to be getting less popular.

Capture UIView to texture continously

I am looking for a way to create an updating texture from iOS to an OpenGL Context. I have seen Render contents of UIView as an OpenGL texture but this is quite slow, as it requires the whole view to be rerendered every time it changes. This means webviews in particular are hit very hard as the whole page needs blitting around. This breaks animations in web views, and makes everything very static. I was wondering if there is a technique using some other APIs in iOS that would enable a link to be created between view to texture (much like video textures do).
This seems to be a fundamental requirement of OS display composition, but it feels like it always happens under the covers and is not exposed to developers. Perhaps I am wrong!
Bonus points for anyone that can tell me if any other OSes support this feature.
Take a look at RosyWriter sample project form Apple.
It uses CVOpenGLESTextureCache to improve performance of rendering camera frames as opengl textures.

What is the purpose of a graphics library such as OpenGL?

I realize this is probably a ridiculous question, but before trying to figure out what libraries to use for which projects, I think it makes sense to really understand the purpose of such libraries first.
A lot of video games use libraries like OpenGL. All the tutorials I've seen of such libraries demonstrate how to write code that tells the computer to draw something. Thing is, in games these days everything is modeled using software such as Zbrush, Maya, or 3ds Max. The models are textured and are good to go. It seems like all you'd need to do is write an animation loop that draws the models and updates repeatedly rather than actually program the code to draw every little thing. That would be both extremely time consuming and would make the models useless. So where does OpenGL or Direct 3D come in in relation to video games and 3d art? What is so crucial about them when all the graphics are already created and just need to be loaded and drawn? Are they used mainly for shaders and effects?
This question may just prove how new I am to this, but it's one I've never heard asked. I'm just starting to learn programming and I'm understanding the code and logic fairly well, but I don't understand graphics libraries or certain frameworks at all and tutorials are not helping.
It seems like all you'd need to do is write an animation loop that draws the models and updates repeatedly rather than actually program the code to draw every little thing.
Everything that happens in a computer does so because a program of some form tells it exactly what to do. The letters that this message is composed of only appear because your web-browser of choice downloaded this file via TCP/IP over an HTTP protocol, decoded its UTF-8-encoded text, interpreted that text as defined by the XML, HTML, JavaScript, and so forth standards, and then displayed the visible portion as defined by the Unicode standard for text layout and in accord with HTML et al, using the displaying and windowing abilities of your OS or window manager or whatever.
Every single part of that operation, from the downloading of the file to its display, is governed by a piece of code. Every pixel you are looking at on the screen is where it is because some code put it there.
HTML alone doesn't mean anything. You cannot just take an HTML file and blast it to the screen. Some code must interpret it. You can interpret HTML as a text file, but if you do, it loses all formatting, and you get to see all of the tags. A web browsers can interpret it as proper HTML, in which case you get to see the formatting. But in every case, the meaning of the HTML file is determined by how it is used.
The "draws the model" part of your proposed algorithm must be done by someone. If you don't write that code, then you must be using a library or some other system that will cause the model to appear. And what does that library do? How does it cause the model to appear?
A model, like an HTML web page, is meaningless by itself. Or to put it another way, your algorithm can be boiled down to this:
Animate the model.
????
Profit!
You're missing a key component: how to actually interpret the model and cause it to appear on the screen. OpenGL/D3D/a software rasterizer/etc is vital for that task.
A lot of video games use libraries like OpenGL.
First and foremost: OpenGL is not a library per-se, but an API (specification). The OpenGL API may be implemented in form as a software library, but these days is much more common to implement OpenGL in form of a driver that turns OpenGL function calls into control commands to a graphics processor sitting on a graphics card (GPU).
All the tutorials I've seen of such libraries demonstrate how to write code that tells the computer to draw something.
Yes. This is because things need to be drawn to make any use of them.
Thing is, in games these days everything is modeled using software such as Zbrush, Maya, or 3ds Max.
At this point the models just consist of a large list of numbers, and further numbers that tell, how the other numbers form some sort of geometry. Those numbers are not some sort of ready to use image.
The models are textured and are good to go.
They are a bunch of numbers, and what they have is some additional numbers controlling texturing. The textures themself are in turn just numbers.
It seems like all you'd need to do is write an animation loop that draws the models
And how do you think this drawing is going to happen? There's no magic "here you have a model, display it" function. Because for one the way in which the numbers making up a model may have any kind of meaning. So some program must give meaning to those numbers. And that is a renderer.
and updates repeatedly rather than actually program the code to draw every little thing.
Again, there is no magic "draw it" function. Drawing a model involves going through each of its numbers, it consists of, and turning those into drawing commands to the GPU.
That would be both extremely time consuming and would make the models useless.
How are the models useless, when they are what is controlling the issuing of commands to OpenGL. Or do you think OpenGL is used to actually "create" models?
So where does OpenGL or Direct 3D come in in relation to video games and 3d art?
It is used to turn the numbers a 3D model, as it is saved away from a modeller, into something pleasant to look at.
What is so crucial about them when all the graphics are already created
The graphics is not yet created, when the model is done. What's created is a model, and some auxilliary data in form of textures and shaders, which are then turned into graphics in realtime, at the execution time of the program.
and just need to be loaded and drawn?
Again, after being loaded, a model is just a bunch of numbers. And drawing means, turning those numbers into something to look at, which requires sending drawing commands to the graphics processor (GPU), which happens using a API like OpenGL or Direct3D
Are they used mainly for shaders and effects?
They are used to turn the numbers generated by a 3D modelling program (Blender, Maya, ZBrush) into an actual picture.
You have data. Like a model, with vertices, normals, and textures. As #datenwolf stated above, those are all just numbers sitting on the hard drive or in RAM, not colors on the screen.
Your CPU (which is where the program you write runs) can't talk to the screen directly. Instead, you send the data you want to draw to the GPU. The GPU then draws the data. Graphics APIs like OpenGL and Direct3D allow programs running on the CPU to send data to the GPU and customize how the GPU draws it. This is a gross simplification, but it sounds like you just need an overview.
Ultimately, every graphics program must go through a graphics API. When you draw an image, for example, you send the GPU the image, and the GPU draws it on the screen. Draw some text? Send the data to the GPU. The GPU draws it. Remember, your code can't talk to the screen. It CAN talk to the GPU through OpenGL or Direct3D, and the GPU then draws the data.
Before OpenGL and DirectX, the games had to use special instructions depending on what graphics card you had. When you bought a new game, you had to check carefully if your card was supported, or you couldn't use the game.
OpenGL and DirectX is a standardized API to the grapics cards. A library is delivered by the manufacturer of the card. If they follow the specification, you are guaranteed that games will work (if they also follow the same specification).
Open Graphics Library (OpenGL) is a cross-language, cross-platform application programming interface (API) for rendering 2D and 3D vector graphics. The API is typically used to interact with a graphics processing unit (GPU), to achieve hardware-accelerated rendering.

Cinder: How to get a pointer to data\frame generated but never shown on screen?

There is that grate lib I want to use called libCinder, I looked thru its docs but do not get if it is possible and how to render something with out showing it first?
Say we want to create a simple random color 640x480 canvas with 3 red white blue circles on it, and get RGB\HSL\any char * to raw image data out of it with out ever showing any windows to user. (say we have console application project type). I want to use such feaure for server side live video stream generation and for video streaming I would prefer to use ffmpeg so that is why I want a pointer to some RGB\HSV or what ever buffer with actuall image data. How to do such thing with libCInder?
You will have to use off-screen rendering. libcinder seems to be just a wrapper for OpenGL, as far as graphics go, so you can use OpenGL code to achieve this.
Since OpenGL does not have a native mechanism for off-screen rendering, you'll have to use an extension. A tutorial for using such an extension, called Framebuffer Rendering, can be found here. You will have to modify renderer.cpp to use this extension's commands.
An alternative to using such an extension is to use Mesa 3D, which is an open-source implementation of OpenGL. Mesa has a software rendering engine which allows it to render into memory without using a video card. This means you don't need a video card, but on the other hand the rendering might be slow. Mesa has an example of rendering to a memory buffer at src/osdemos/ in the Demos zip file. This solution will probably require you to write a complete Renderer class, similar to Renderer2d and RendererGl which will use Mesa's intrusctions instead of Windows's or Mac's.

CoreImage for Win32

For those not familiar with Core Image, here's a good description of it:
http://developer.apple.com/macosx/coreimage.html
Is there something equivalent to Apple's CoreImage/CoreVideo for Windows? I looked around and found the DirectX/Direct3D stuff, which has all the underlying pieces, but there doesn't appear to be any high level API to work with, unless you're willing to use .NET AND use WPF, neither of which really interest me.
The basic idea would be create/load an image, attach any number of filters that can be chained together, forming a graph, and then render the image to an HDC, using the GPU to do most of the hard work. DirectX/Direct3D has these pieces, but you have to jump through a lot of hoops (or so it appears) to use it.
There are a variety of tools for working with shaders (such as RenderMonkey and FX-Composer), but no direct equivalent to CoreImage.
But stacking up fragment shaders on top of each other is not very hard, so if you don't mind learning OpenGL it would be quite doable to build a framework that applies shaders to an input image and draws the result to an HDC.
Adobe's new Pixel Blender is the closest technology out there. It is cross-platform -- it's part of the Flash 10 runtime, as well as the key pixel-oriented CS4 apps, namely After Effects and (soon) Photoshop. It's unclear, however, how much is currently exposed for embedding in other applications at this point. In the most extreme case it should be possible to embed by embedding a Flash view, but that is more overhead than would obviously be idea.
There is also at least one smaller-scale 3rd party offering: Conduit Pixel Engine. It is commercial, with no licensing price clearly listed, however.
I've now got a solution to this. I've implemented an ImageContext class, a special Image class, and a Filter class that allows similar functionality to Apple's CoreImage. All three use OpenGL (I gave up trying to get this to work on DirectX due to image quality issues, if someone knows DirectX well contact me, because I'd love to have a Dx version) to render an image(s) to a context and use the filters to apply their effects (as HLGL frag shaders). There's a brief write up here:
ImageKit
with a screen shot of an example filter and some sample source code.