How to place text in the certain position when using SceneKit? - swift3

I have tried to follow the same simple logic as for cylinder, box … , so just by defining the position for textNode, but it is not working.
func makeText(text3D: String, position: SCNVector3, depthOfText: CGFloat, color: NSColor, transparency: CGFloat) -> SCNNode
{
let textTodraw = SCNText(string: text3D, extrusionDepth: depthOfText)
textTodraw.firstMaterial?.transparency = transparency
textTodraw.firstMaterial?.diffuse.contents = color
let textNode = SCNNode(geometry: textTodraw)
textNode.position = position
return textNode
}

The default font for SCNText is 36 point Helvetica, and a "point" in font size is the same as a unit of scene space. (Well, of local space for the node containing the SCNText geometry. But unless you've set a scale factor on your node, local space units are the same as scene space units.) That means even a short label can be tens of units tall and hundreds of units wide.
It's typical to build SceneKit scenes with smaller scope — for example, simple test scenes like you might throw together in a Swift playground using the default sizes for SCNBox, SCNSphere, etc might be only 3-4 units wide. (And if you're using SceneKit with ARKit, scene units are meters, so some text in 36 "point" font is the size of a few office blocks downtown.)
Also, the anchor point for a text geometry relative to its containing node is at the lower left corner of the text. Put all this together and it's entirely possible that there are giant letters looming over the rest of your scene, hiding just out of camera view.
Note that if you try to fix this by setting a much smaller font on your SCNText, the text might get jagged and chunky. That's because the flatness property is measured relative to the point size of the text (more precisely, it's measured in a coordinate system where one unit == one point of text size). So if you choose a font size that'd be tiny by screen/print standards, you'll need to scale down the flatness accordingly to still get smooth curves in your letters.
Alternatively, you can leave font sizes and flatness alone — instead, set a scale factor on the node containing the text geometry, or set that node's pivot to a transform matrix that scales down its content. For example, if you set a scale factor of 1/72, one unit of scene space is the same as one "inch" (72 points) of text height — depending on the other sizes in your scene, that might make it a bit easier to think of font sizes the way you do in 2D.

The fact is you generally just use "small numbers" for font sizes in SceneKit.
In 3D you always use real meters. A humanoid robot must be about "2" units tall, a car is about "3" units long and so on.
Very typical sizes for the font is about "0.1"
Note that the flatness value is SMALL, usually about one hundredth the size of the font. (Which is obvious, it's how long the line segments are.)
Typical:
t.font = UIFont(name: "Blah", size: 0.10)
t.flatness = 0.001
Set the flatness to about 1/4 the size of the font (hence, 0.025 in the example) to understand what "flatness" is.
I would never change the scale of the type node, that's a bad idea for many reasons. There's absolutely no reason to do so, and it makes it very difficult to genuinely set the flatness appropriately.
But note ...
That being said, on the different platforms and different versions, SCNText() often does a basically bad job drawing text, and it can go to hell at small numbers. So yeah, you may indeed have to scale in practice, if the text construction is crap at (very) small values :/

Related

Incorrect metrics and sizes of font created by CreateFont()

I trying to render a font into bitmap using WinAPI, but I can't reach needed sizes of font.
Here's how the font is initialized:
HDC dc = ::CreateCompatibleDC(NULL);
::SetMapMode(dc, MM_TEXT);
::SetTextAlign(dc, TA_LEFT | TA_TOP | TA_UPDATECP);
int size_in_pixels = 18;
HFONT font = ::CreateFontA(-size_in_pixels, ..., "Arial");
::SelectObject(dc, font);
::TEXTMETRICW tm = { 0 };
GetTextMetricsW(dc, &tm);
But after it I getting incorrect values both in GetGlyphOutlineW and GetTextMetricsW, it's not size I passed as parameter
I know that it's expecting value in logical units, but in MM_TEXT 1 unit should be 1 pixel, don't it?
I expecting that CreateFontA accepting point size when I passing a negative value (like here https://i.stack.imgur.com/tEt8J.png), but in fact it's wrong.
I tried bruteforcing values, and find out proper parameter for a few sizes:
18px = -19; 36px = -39; 73px = -78;
Also I tried formula that provided by Microsoft:
nHeight = -MulDiv(PointSize, GetDeviceCaps(hDC, LOGPIXELSY), 72);
But it's also giving me a wrong result, rendered text (using GetGlyphOutlineW) is larger if measure it (for example height of 'j' should have exact size that I passed)
Also metrics from GetTextMetricsW are wrong, for example tmAscent. I know that on Windows it's including internal leading, but even if subtract tmInternalLeading from tmAscent it's still incorrect.
By the way, values from GetCharABCWidthsW are correct, so a+b+c is width of glyph in pixels (while documentation says it should be in logical units).
Also I should say about DPI, usually I using 125% on Windows 10 scale in settings, but I tried even with 100%, interesting that ::GetDeviceCaps(dc, LOGPIXELSY) not changing with scale I using, it's always 96
Here's example of CreateFontA(-128, ...) with final atlas and metrics:
rendered atlas
Question #1: What should I do to pass wanted point size in pixels and receive glyphs in proper size with correct metrics in pixels?
Question #2: What the strange units all these functions are using?
When you use ::SetMapMode(dc, MM_TEXT); the font size is specified in device pixels. Negative value excludes internal leading, so for the same absolute value the negative ones produce visually bigger fonts. If you want to get same height from GetTextExtentPoint32 for different fonts, use positive values.
In your example with -128 height, you are requesting font for which, after internal leading exclusion, height is 128 pixels. Font mapper selects 143 which is correct for internal leading of 15 pixels (128+15=143). tmAscent + tmDescent are also correct (115+28=143). You get what you specified.
You should take into account that values in text metric don't state hard bounds. Designer can design fonts so its glyphs sometimes go beyond guiding lines or don't reach them.
for example height of 'j' should have exact size that I passed
Dot over j can go beyond or not reach top line if designer finds it visually plausible to design it that way.
interesting that ::GetDeviceCaps(dc, LOGPIXELSY) not changing with scale I using, it's always 96
Unless you log off and log in, system DPI doesn't change. For per monitor DPI aware application you have to get DPI from monitor parameters or cache value given by WM_DPICHANGED.
Question #1: What should I do to pass wanted point size in pixels and receive glyphs in proper size with correct metrics in pixels?
I think you want to get specific distance between top and bottom lines and this is exactly how you create font HFONT font = ::CreateFontA(-size_in_pixels, ..., "Arial");. The problem lies in your assumption that font design lines are hard boundaries for each glyph, but font's designer don't have to strictly align glyphs to these lines. If you want glyphs strictly aligned, probably there is no way to get it. Maybe check different font.
Question #2: What the strange units all these functions are using?
When mode is set to WM_TEXT, raw device pixels are used. Positive height specifies height including tmInternalLeading, negative excludes it.
For positive value:
tmAscent + tmDescent = requestedHeight
For negative value:
tmAscent + tmDescent - tmInternalLeading = requestedHeight
Bellow I have pasted screen shots with different fonts showing that depending on selected font glyphs could be designed so they don't reach top line or go beyond it and bottom line in most cases also isn't reached.
Seems that for your requirements Arial Unicode MS would be better fit (but j still doesn't reach where you want it).
Arial:
Arial Unicode MS
Input Mono
Trebuched MS

How to interpret anchor boxes in Yolo or R-CNN?

For algorithms like yolo or R-CNN, they use the concept of anchor boxes for predicting objects. https://pjreddie.com/darknet/yolo/
The anchor boxes are trained on specific dataset, one for COCO dataset is:
anchors = 0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828
However, i don't understand how to interpret these anchor boxes? What does a pair of values (0.57273, 0.677385) means?
In the original YOLO or YOLOv1, the prediction was done without any assumption on the shape of the target objects. Let's say that the network tries to detect humans. We know that, generally, humans fit in a vertical rectangle box, rather than a square one. However, the original YOLO tried to detect humans with rectangle and square box with equal probability.
But this is not efficient and might decrease the prediction speed.
So in YOLOv2, we put some assumption on the shapes of the objects. These are Anchor-Boxes. Usually we feed the anchor boxes to the network as a list of some numbers, which is a series of pairs of width and height:
anchors = [0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828]
In the above example, (0.57273, 0.677385) represents a single anchor box, in which the two elements are width and height respectively. That is, this list defines 5 different anchor boxes. Note that these values are relative to the output size. For example, YOLOv2 outputs 13x13 feature mat and you can get the absolute values by multiplying 13 to the values of anchors.
Using anchor boxes made the prediction a little bit faster. But the accuracy might decrease. The paper of YOLOv2 says:
Using anchor boxes we get a small decrease in accuracy. YOLO only
predicts 98 boxes per image but with anchor boxes our model predicts
more than a thousand. Without anchor boxes our intermediate model gets
69.5 mAP with a recall of 81%. With anchor boxes our model gets 69.2 mAP with a recall of 88%. Even though the mAP decreases, the increase
in recall means that our model has more room to improve
That's what I understood: YOLO divides an 416x416 image into 13x13 grids. Each grid as 32 pixels. The anchor boxes size are relative to the size of the grid.
So an anchor box of width and height 0.57273, 0.677385 pixels has actually a size of
w = 0.57273 * 32 = 18.3 pixels
h = 0.677385 * 32 = 21.67 pixels
If you convert all those values, you can then plot them on a 416x416 image to visualise them.

How to center a sf::Text in a sf::RectangleShape

Is there a easy way to center an sf::Text object in an sf::RectangleShape object?
The text has variable lengths, but isn't changing after creation.
I'm using SFML 2.4.
Centering one object over the other is usually a trivial task: Just determine the size difference between the objects and use it as an offset by dividing it by 2.
However, once you use (True Type) font rendering, things become a tiny bit more tricky, because the origin of these font glyphs isn't necessarily on the top left of the actual glyph. It might be anywhere, depending on the font and the character/glyph to be rendered (serifs and other decoration elements are a typical example for this).
Getting back to the basic formula:
offset = (shape.size() - text.size()) / 2
Now let's add the text offset:
offset = (shape.size() - text.size()) / 2 - text.offset()
There are multiple ways to do this using SFML. Personally I'd probably create my custom sf::Drawable derived class drawing both the box and the text.
For standalone drawing (like you do), I actually prefer setting the text's origin to account for the offset. This way you're able to set both shape and text's position identically and they'll be perfectly aligned without touching the origin/offset again (unless you change the text).
const sf::FloatRect bounds(text.getLocalBounds());
const sf::Vector2f box(shape.getSize());
text.setOrigin((bounds.width - box.x) / 2 + bounds.left, (bounds.height - box.y) / 2 + bounds.top);
Note that I've basically swapped operands since I'm setting the origin rather than offset. For example, to move the text 5 units to the right, I have to set the origin to -5 rather than 5.
The constant is really just there to make the whole line a bit more readable. You could also do everything inline, but I prefer the cleaner way.
Here's the whole thing in action using Arial and Lucida Handwriting:
Note how "Hello World" is aligned differently depending on whether the font glyphs take more space downwards. If you don't want that behavior, you'd have to use some fixed line height rather than relying on the height of the sf::Text object.

DirectWrite text position different by font size

I'm using DirectWrite to render some text to a window. Everything seems to work except positioning when using different font sizes: I'd expect 2 texts with font size v1 and v2 and both with (x, y) = (0, 0) to be at the top left but as you can see:
neither "Test" nor "X" are really at the top left.
Is there a way to make this work?
Welcome to the world of fonts. Fonts are probably the most difficult thing to use, because there is surprises in font themselves ( there is so many new standards that supposed to solves everything and just confuse more because almost no font support it at 100%, even some 'classic' font have partial/bad information in them) the GDI, GDI+, DirectDraw don't draw font at the same position in pixels because of math, coordinate rounding, anti-aliasing... ( you can have one more bonus if you do the math with freetype ).
When you try to print the font there are other pb. So the only way around this is for me. Don't even try to draw font at certain pixel coordinates. Do your job at drawing font, picture, lines on the screen that render well, do your best to convert them to printing coordinate for exports but never expect to control pixel in fonts, everything is round approximates.
PS : Don't trust internal fields in fonts. On Arial they are good on all other fonts some are missing or initialised to zero, but the "fun" part it's not always the same field which are not present it depends of the fonts. You could only use the fields if you try them before by font. Yes fonts are fantastic!
The term #evilruff is referring to is called 'internal leading'. You can use IDWriteFontFace::GetMetrics or possibly IDWriteFontFace::GetDesignGlyphMetrics to get this value (for GetMetrics, the value you're looking for is most likely metrics.ascent - metrics.capHeight).
The values here are in Font Design Units, not pixels (of any sort). You can convert these values to em height by dividing by metrics.designUnitsPerEm; typically, font sizes in DirectWrite are specified by the pixel size of the (lowercase) m; so if you multiply the values in ems by the font size, you should get the values in pixels.
I'm assuming you are using an IDWriteTextLayout in conjunction with DrawTextLayout (rather than creating your own DWRITE_GLYPH_RUN). IDWriteTextLayout aligns glyphs to their layout cell (including the full ascent and line gap), not the glyph ink, and this is true of pretty much all text layouts, be they web browsers or word processors or simple edit controls. If they did not (instead aligning to the top of the letter), then diacritics in words like Ťhis would be clipped.
If you always want to align to the ink, create an IDWriteTextLayout, call IDWriteTextLayout::GetOverhangMetrics, and then call DrawTextLayout with an origin equal to negative DWRITE_OVERHANG_METRICS::left&top. If you want to align to the cap-height always (that way "hello" and "Hello" would both draw at the same vertical coordinate), then Eric's approach will work.

Directwrite: Getting a font's height

My objective:
I want to get the height of an IDWriteTextFormat's font so I can calculate how many lines of text can fit in an IDWriteTextLayout of a certain height.
My problem:
Right now I'm using this code to calculate the visible number of lines:
inline int kmTextCtrl::GetVisLines() const
{
/* pTextFormat is an IDWriteTextFormat pointer, dpi_y is the desktop's vertical dpi,
and GetHeight() returns the height (in pixels) of the render target. */
float size = (pTextFormat->GetFontSize()/72.0f)*dpi_y;
return (int)(GetHeight()/size);
}
The calculation seems to be accurate for some fonts, but not for any of the TrueType fonts (e.g.: Courier New, Arial, Times New Roman). For these fonts, the text shown is clipped well short of the lower vertical boundary of the render target.
Some context:
I am making a text scroll back buffer control which uses an IDWriteTextLayout to put text to the control's render target. I use the result of GetVisLines() to determine how many lines of text from a circular buffer (which stores text in std::strings by the line) to pull into the layout, and recreate it every time the window is scrolled or resized.
This is being done using "native" Win32 API C++.
The simplest and most robust approach is to just ask the layout itself for text metrics, as that's one of the two things it was designed for, drawing and measurement. You would create an IDWriteTextLayout using the text format and call GetMetrics to get the DWRITE_TEXT_METRICS::height. I'm guessing you're using ID2D1RenderTarget::DrawText and passing a text format, so you may not have created a layout directly, but calling DrawText is just like calling CreateTextLayout yourself followed by DrawTextLayout.
Beware that going through the lower layers to get this answer (IDWriteFontFace and the like) makes certain assumptions that a generic world ready text control should not assume, such as assuming the base font will be used and that all the lines being the same height. So long as all characters are present in the given base font, this happens to work out (chances are you're mostly displaying English which is why all appears well), but throw in some CJK or RTL languages or emoji (which a base font like Times New Roman certainly doesn't support), and the line height will grow or shrink accordingly to the substituted fonts. GDI rescales substituted fonts such that they fit into the base font's height, but this leads to poorly scrunched letters in languages like Thai and Tibetan which need more breathing room for ascenders and descenders. IDWriteTextLayout and other layouts like those in WPF/Word keep all the font glyphs at the same em size, which means they line up more nicely when adjacent to each other; but it does mean the line height is variable.
If you do just draw each line of text as if they were all the same height, you can see overlap between glyphs and non-uniform baselines between lines, or clipping at the top and bottom of the control. So the ideal thing to do is to use the actual height of each line; but if you need them to all be the same height (or if it complicates the control too much), then at least set an explicit line spacing using SetLineSpacing with DWRITE_LINE_SPACING_UNIFORM to that of the base font - that way the baselines are uniformly spaced.
Though for the curious, IDWriteTextLayout computes the line height as the maximum of all run heights on that line, and the height of a single run (same font and em size) just uses the design metrics: ascent + descent, plus any lineGap that happens to be present (most fonts set this to zero, but Gabriola is a good example of large line gap). Note all em sizes are in DIP's (which at typical 96DPI means 1:1, DIP's exactly == pixels), not points (1/72 inch).
(ascent + descent + lineGap) * emSize / designUnitsPerEm
I have found an answer.
To find the spacing of a line (font height plus gap) in Directwrite, you must do something akin to the following:
inline int kmTextCtrl::GetVisLines() const
{
IDWriteFontCollection* collection;
TCHAR name[64]; UINT32 findex; BOOL exists;
pTextFormat->GetFontFamilyName(name, 64);
pTextFormat->GetFontCollection(&collection);
collection->FindFamilyName(name, &findex, &exists);
IDWriteFontFamily *ffamily;
collection->GetFontFamily(findex, &ffamily);
IDWriteFont* font;
ffamily->GetFirstMatchingFont(pTextFormat->GetFontWeight(), pTextFormat->GetFontStretch(), pTextFormat->GetFontStyle(), &font);
DWRITE_FONT_METRICS metrics;
font->GetMetrics(&metrics);
float ratio = pTextFormat->GetFontSize() / (float)metrics.designUnitsPerEm;
float size = (metrics.ascent + metrics.descent + metrics.lineGap) * ratio;
float height = GetHeight();
int retval = static_cast<int>(height/size);
ffamily->Release();
collection->Release();
font->Release();
return retval;
}
Of course, you probably don't want to do all that every time you have to call a frequently-used inline function.