Text recognition app - computer-vision

now i`m trying to make camera app which can recognize text. For this I use the information from guides by google. This site describes how to make a full-screen reader. But i need to set Mobile Vision text scanner active in small rectangle(like in picture).Screenshot of barcode reader app(i need the same solution for text). Please help me).
required result

I have implemented this feature by integrating "filter" in front of the detector which crops the image (in my case a line of text from the center of image). Look at the code below:
public class LineDetector extends Detector {
Detector mDelegate;
public LineDetector(Detector delegate) {
mDelegate = delegate;
}
#Override
public SparseArray detect(Frame frame) {
int width = frame.getMetadata().getWidth();
int height = frame.getMetadata().getHeight();
int mBoxHeight = height;
int mBoxWidth = Math.toIntExact(Math.round(mBoxHeight * ConstantsPool.CROP_BOX_ASPECT_RATIO));
int right = (width / 2) + (mBoxWidth / 2);
int left = (width / 2) - (mBoxWidth / 2);
int bottom = height;
int top = 0;
YuvImage yuvImage = new YuvImage(frame.getGrayscaleImageData().array(), ImageFormat.NV21, width, height, null);
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(left, top, right, bottom), 100, byteArrayOutputStream);
byte[] jpegArray = byteArrayOutputStream.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegArray, 0, jpegArray.length);
Frame croppedFrame = new Frame.Builder()
.setBitmap(bitmap)
.setRotation(frame.getMetadata().getRotation())
.build();
return mDelegate.detect(croppedFrame);
}
#Override
public boolean isOperational() {
return mDelegate.isOperational();
}
#Override
public boolean setFocus(int id) {
return mDelegate.setFocus(id);
}
}
And then you can use it in a following way:
TextRecognizer textRecognizer = new TextRecognizer.Builder(this).build();
LineDetector lineDetector = new LineDetector(textRecognizer);
lineDetector.setProcessor(...);
...
Camera2Source camera2Source = new Camera2Source.Builder(getContext(), lineDetector).build();
If you have any questions, just ask
Tom

Related

How can I replace a image in a word document using Novacode Docx

I want to replace a image in a word document using Novacode Docx but i just know insert by this code
var gDoc = DocX.Load(#"MauPhieuNhapCBCC.docx");
using (MemoryStream ms = new MemoryStream())
{
System.Drawing.Image myImg = System.Drawing.Image.FromFile(#"img.jpg");
myImg.Save(ms, myImg.RawFormat); // Save your picture in a memory stream.
ms.Seek(0, SeekOrigin.Begin);
Image img = gDoc.AddImage(ms); // Create image.
Picture pic1 = img.CreatePicture();
// Create picture.
var abc = gDoc.Paragraphs.FirstOrDefault(g => g.Pictures.Count > 0);
abc.Alignment = Alignment.right;
abc.InsertPicture(pic1,29).Position(0.3); // Insert picture into paragraph.
}
gDoc.SaveAs("Exported.docx");
How can I cant replace or insert in a special position?
you can do it like this, it might not be the ideal way but it works properly:
var gDoc = DocX.Load(#"MauPhieuNhapCBCC.docx");
for (int f = 0; f < gDoc.Images.Count; f++) //Loop through the images of the document
{
Novacode.Image imo = gDoc.Images[f]; //Get image of the docuemnt
if (imo.Id == "imageIdInDocument") //Check the id
{
using (SD.Image newImg = SD.Image.FromFile("newImagePath")) //Load the new image for replacement
{
SD.Bitmap bMa = new SD.Bitmap(imo.GetStream(FileMode.Open, FileAccess.ReadWrite));
SD.Graphics graf = SD.Graphics.FromImage(bMa);
graf.Clear(SD.Color.White); //Set the image of the document all white
//Below we insert our new image over the one in the document
//We might need to change the image size to fit the area of our existing image
//If that were the case we give a function to resize images
SD.Image newResizeImg = meths.ResizeImage(newImg, newWidth, newHeight);
//Below we set the point(0, 0) to overlap the new image from the top left corner
graf.DrawImage(newResizeImg, new SD.Rectangle(new SD.Point(0, 0), newResizeImg.Size),
new SD.Rectangle(new SD.Point(), newResizeImg.Size), SD.GraphicsUnit.Pixel);
bMa.Save(imo.GetStream(FileMode.Create, FileAccess.Write), SD.Imaging.ImageFormat.Jpeg);
}
}
}
Here the function to resize images:
public SD.Image ResizeImage(SD.Image img, int newWidth, int newHeight)
{
if (img.Width < newWidth && img.Height < newHeight) return img;
using (img)
{
SD.Bitmap cpy = new SD.Bitmap(newWidth, newHeight, SD.Imaging.PixelFormat.Format32bppArgb);
using (SD.Graphics gr = SD.Graphics.FromImage(cpy))
{
gr.Clear(SD.Color.Transparent);
gr.InterpolationMode = SD.Drawing2D.InterpolationMode.HighQualityBicubic;
gr.DrawImage(img, new SD.Rectangle(0, 0, newWidth, newHeight), new SD.Rectangle(0, 0, img.Width, img.Height), SD.GraphicsUnit.Pixel);
}
return cpy;
}
}
What we basically did was to get the image of the document, paint it white and set the new image inside the existent one with the same size or smaller.
Hope this helps someone.

Android Studio - Support Multiple Screens

I'm trying to support multiple screen sizes in my app... I added this to my Manifest.xml:
`<supports-screens
android:normalScreens="true"
android:smallScreens="true"
android:largeScreens="true"
android:xlargeScreens="true"
android:anyDensity="true"/>`
I also added different layout sizes and densities:
layout-small
layout-large
layout-normal
layout-xlarge
drawable-ldpi
drawable-hdpi
drawable-mdpi
drawable-xhpi
My layout-normal is currently set on Nexus 4: Image here
Now it's set on Nexus One: Image here
How can I set the elements to scale automatically?
Is there any other way to scale the elements automatically without setting their height and width to wrap_content or match_parent?
of course you can do it programmatically
example:
I defined Simple Object ImageSize like this:
public class ImageSize {
private float width;
private float height;
public float getWidth() {
return width;
}
public void setWidth(float width) {
this.width = width;
}
public float getHeight() {
return height;
}
public void setHeight(float height) {
this.height = height;
}
}
then I created a layout which contains Image(another layout) in it
final LinearLayout sellCarImage = (LinearLayout) findViewById(R.id.sellCarImage);
final RelativeLayout sellCarLayout = (RelativeLayout) findViewById(R.id.sellCarLayout);
then I've calculated width and height to scale view sizes based on image real size:
ImageSize img = new ImageSize();
img = getImageSize(R.drawable.indeximage5, MainActivity.this);
int sellCarWidth = sellCarLayout.getWidth();
int sellCarHeight = (int) ((sellCarWidth * img.getHeight()) / img.getWidth());
and programmatically set width and height:
RelativeLayout.LayoutParams sellCarImageIconParams = new RelativeLayout.LayoutParams(sellCarWidth, sellCarHeight);
//sellCarImageIconParams.setMargins(marg, 0, marg, 0);
sellCarImage.setLayoutParams(sellCarImageIconParams);
sellCarImage.setBackgroundResource(R.drawable.indeximage5);
here is getImageSize
public ImageSize getImageSize(int resource, Context ctx) {
ImageSize imageSize = new ImageSize();
BitmapFactory.Options dimensions = new BitmapFactory.Options();
dimensions.inJustDecodeBounds = true;
Bitmap mBitmap = BitmapFactory.decodeResource(ctx.getResources(), resource, dimensions);
imageSize.setHeight(dimensions.outHeight);
imageSize.setWidth(dimensions.outWidth);
return imageSize;
}
it's better to put them in:
sellCarLayout.post(new Runnable() {
#Override
public void run() {
//here
});
}
I hope it helps

Line break when use Replacement in Android

I used the replacementSpan to create background for each word phrases.
It's working right with short phrases, but wrong with long phrases
Like this :
a aa aaa aaa aaaa
123123
213123123123123123123123123
This is my sourceCode
public class FrameSpan extends ReplacementSpan {
private final Paint mPaint;
private int mWidth;
public FrameSpan(int color) {
mPaint = new Paint();
mPaint.setStyle(Paint.Style.FILL_AND_STROKE);
mPaint.setColor(color);
mPaint.setAntiAlias(true);
}
#Override
public int getSize(Paint paint, CharSequence text, int start, int end, Paint.FontMetricsInt fm) {
mWidth = Math.round(paint.measureText(text, start, end));
return mWidth;
}
#Override
public void draw(Canvas canvas, CharSequence text, int start, int end, float x, int top, int y, int bottom, Paint paint) {
int line_space = bottom - top;
canvas.drawRect(x, top, x + mWidth, bottom - line_space / 3, mPaint);
canvas.drawText(text, start, end, x, y, paint);
}
}
List<String> tagList = /* the list of phrases */
int[] tagEndingIndex = /* the array contains the ending index of each phrases*/
StringBuilder stringBuilder = /* the complete text created from tagList */
SpannableString spannableString = new SpannableString(stringBuilder.toString());
for (int i = 0; i < tagEndingIndex.length; i++) {
int tagEndPos = tagEndingIndex[i];
String currentTag = tagList.get(i);
spannableString.setSpan(new FrameSpan(color), tagEndPos - currentTag.length() - 4, tagEndPos, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE);
}
TextView.setText(spannableString, BufferType.SPANNABLE);
With the long phrase, I want ellipsize them.
Please help me. Thank you very much
Have you tried android:ellipsize property? I think many text related view components have this property.

Stop particle effect libgdx

I have a game that involves lots of collisions and explosions.
Basically, enemy-projectiles are launched from the bottom-right corner of the screen towards the left side. The player can intercept them using his weapon.
I want whenever a "bullet" and a projectile collide, the code will draw pre-made particles at the impact position.
My problem is that in real-time, in collision, the particles are drawn nicely but they never stop. The particles continue to be drawn even after the collision.
Now, I use callback mechanism to inform the renderer a collision took place somewhere, and I pass him the coordinates of the collision(x,y)
Here is my renderer code. I cut some parts from the code, leaving only the relevant ones:
package com.david.gameworld;
import com.badlogic.gdx.Gdx;
import com.badlogic.gdx.graphics.GL20;
import com.badlogic.gdx.graphics.OrthographicCamera;
import com.badlogic.gdx.graphics.g2d.Sprite;
import com.badlogic.gdx.graphics.g2d.SpriteBatch;
import com.badlogic.gdx.math.Vector2;
import com.badlogic.gdx.physics.box2d.Body;
import com.badlogic.gdx.physics.box2d.Box2DDebugRenderer;
import com.badlogic.gdx.utils.Array;
import com.david.helpers.AssetsLoader;
import com.david.helpers.CollisionAndTouchCallbacks;
public class GameRenderer implements CollisionAndTouchCallbacks.DrawThings{
/* The GameRenderer class. Responsible for all the rendering work and graphics */
public static final float FLOOR_HEIGHT = 100.0f; // Public variable - The floor height
private static final float Pixels_To_Meters = 32.0f; // PTM Ration
public static float screenWidth; // Screen width
public static float screenHeight; // Screen height
public OrthographicCamera camera; // The orthographic camera
private GameWorld world; // The game world object, for rendering
private boolean drawTrajectory;
private boolean drawParticle;
private boolean isDrawnParticle;
private Box2DDebugRenderer debugRenderer; // Debug renderer, to see the positions and sizes of the bodies/fixtures
private SpriteBatch batch; // A sprite batch
private Array<Body> tempBodies; // An array that contains all the bodies of Box2D
private Array<Vector2> collisionPositions;
private float delta;
public GameRenderer(float screenWidth, float screenHeight, GameWorld world) {
this.world = world;
GameRenderer.screenHeight = screenHeight;
GameRenderer.screenWidth = screenWidth;
debugRenderer = new Box2DDebugRenderer();
batch = new SpriteBatch();
camera = new OrthographicCamera();
tempBodies = new Array<Body>();
collisionPositions = new Array<Vector2>();
drawTrajectory = false;
drawParticle = false;
isDrawnParticle = false;
}
public void render(float delta) {
Gdx.gl.glClearColor(1, 1, 1, 1);
Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);
this.delta = delta;
// Update the world
world.updateWorld();
batch.setProjectionMatrix(camera.combined);
// Get box2d World bodies
world.getBox2DWorld().getBodies(tempBodies);
// Draw the entire scene
batch.begin();
drawBodiesBox2D();
if(drawTrajectory)
drawTrajectory();
if(drawParticle) {
if(!isDrawnParticle) {
drawParticle();
isDrawnParticle = true;
} else {
if(AssetsLoader.explosion.isComplete()) {
drawParticle = false;
isDrawnParticle = false;
}
}
}
batch.end();
// Debug Renderer for box2d world
}
// If user is aiming, draw the trajectory of the acorn
public void drawTrajectory() {
Gdx.app.log("System Out Println!!!!!!!!!", "Draw projectile");
float t = 0.1f, x = world.getArrayAcorns().peek().getX(), y = world
.getArrayAcorns().peek().getY();
float width = AssetsLoader.orbit.getWidth() / 8 / Pixels_To_Meters;
float height = AssetsLoader.orbit.getHeight() / 8 / Pixels_To_Meters;
float timeSeparation = 0.08f;
for (int i = 0; i < 40; i++) {
x = world.getAcornEquation().getX(t);
y = world.getAcornEquation().getY(t);
batch.draw(AssetsLoader.orbit, x, y, width, height);
t += timeSeparation;
}
}
private void drawBodiesBox2D() {
for (Body body : tempBodies) {
if (body.getUserData() != null
&& body.getUserData() instanceof Sprite) {
Sprite sprite = (Sprite) body.getUserData();
if (!sprite.getTexture().equals(AssetsLoader.ground)) {
sprite.setPosition(body.getPosition().x - sprite.getWidth()
/ 2, body.getPosition().y - sprite.getHeight() / 2);
} else {
sprite.setSize(screenWidth, FLOOR_HEIGHT / Pixels_To_Meters);
}
sprite.setRotation((float) Math.toDegrees(body.getAngle()));
batch.draw(AssetsLoader.catapult_base1, 450 / Pixels_To_Meters,
(FLOOR_HEIGHT - 10) / Pixels_To_Meters,
AssetsLoader.catapult_base1.getWidth()
/ Pixels_To_Meters,
AssetsLoader.catapult_base1.getHeight()
/ Pixels_To_Meters);
sprite.draw(batch);
}
}
}
public void dispose() {
AssetsLoader.dispose();
batch.dispose();
world.dispose();
debugRenderer.dispose();
}
// Update method for the orthographic camera
public void updateCamera(int width, int height) {
camera.setToOrtho(false, width, height);
screenWidth = (float) width;
screenHeight = (float) height;
camera.update();
// Already Meters
}
public OrthographicCamera getCamera() {
return this.camera;
}
private void drawParticle() {
// TODO Auto-generated method stub
AssetsLoader.explosion.setPosition(collisionPositions.first().x,collisionPositions.first().y);
AssetsLoader.explosion.start();
AssetsLoader.explosion.draw(batch,delta);
AssetsLoader.explosion.allowCompletion();
}
#Override
public void signalToTrajectory(boolean flag) {
// TODO Auto-generated method stub
drawTrajectory = flag;
}
#Override
public void signalToParticle(boolean flag, Array<Vector2> positions) {
// TODO Auto-generated method stub
if(flag)
collisionPositions = positions;
drawParticle = flag;
}
}
How can I draw the particle effect once and that's it?
You can use/optimise my particle class, look my post here
To draw the particle effect once and that's it : ( it's in myparticle free method ) :
ParticleEffect effect = new ParticleEffect();
Emitter emitter = effect.getControllers().first().emitter;
if (emitter instanceof RegularEmitter){
RegularEmitter reg = (RegularEmitter) emitter;
reg.setEmissionMode(RegularEmitter.EmissionMode.EnabledUntilCycleEnd);
//reg.durationValue.setLow(10f);
reg.dispose();
}
emitter.dispose();

Programmatically grab screenshots in OSX

I am going to port some screenshot grabbing code (C++) for linux to osx. The current solution run graphical applications in xvfb and then use xlib to grab screenshots from the display. (That will also support if we are running without xvfb).
So as I understood osx is moving away from X11 so my question is what to use besides xlib to implement it now ? I have found Quartz Display Services. Is that what makes sense to use now ? Will that work with xvfb ?
Yes, you will be able to call functions like CGDisplayCreateImage (documentation linked for you) by linking the Application Services framework to your C++ tool.
I have written an example for capturing the pc display screen and convert to opencv Mat.
#include <iostream>
#include <opencv2/opencv.hpp>
#include <unistd.h>
#include <stdio.h>
#include <ApplicationServices/ApplicationServices.h>
using namespace std;
using namespace cv;
int main (int argc, char * const argv[])
{
size_t width = CGDisplayPixelsWide(CGMainDisplayID());
size_t height = CGDisplayPixelsHigh(CGMainDisplayID());
Mat im(cv::Size(width,height), CV_8UC4);
Mat bgrim(cv::Size(width,height), CV_8UC3);
Mat resizedim(cv::Size(width,height), CV_8UC3);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef contextRef = CGBitmapContextCreate(
im.data, im.cols, im.rows,
8, im.step[0],
colorSpace, kCGImageAlphaPremultipliedLast|kCGBitmapByteOrderDefault);
while (true)
{
CGImageRef imageRef = CGDisplayCreateImage(CGMainDisplayID());
CGContextDrawImage(contextRef,
CGRectMake(0, 0, width, height),
imageRef);
cvtColor(im, bgrim, CV_RGBA2BGR);
resize(bgrim, resizedim,cv::Size(),0.5,0.5);
imshow("test", resizedim);
cvWaitKey(10);
CGImageRelease(imageRef);
}
// CGContextRelease(contextRef);
// CGColorSpaceRelease(colorSpace);
return 0;
}
and then, the result is here.
I had expected my current display would be captured, but only the back wallpaper was captured actually.
What the CGMainDisplayID() refers would be a hint to this problem.
Anyway, I hope this may approach your goal a bit.
void captureScreen(){
CGImageRef image_ref = CGDisplayCreateImage(CGMainDisplayID());
CGDataProviderRef provider = CGImageGetDataProvider(image_ref);
CFDataRef dataref = CGDataProviderCopyData(provider);
size_t width, height; width = CGImageGetWidth(image_ref);
height = CGImageGetHeight(image_ref);
size_t bpp = CGImageGetBitsPerPixel(image_ref) / 8;
uint8 *pixels = malloc(width * height * bpp);
memcpy(pixels, CFDataGetBytePtr(dataref), width * height * bpp);
CFRelease(dataref);
CGImageRelease(image_ref);
FILE *stream = fopen("/Users/username/Desktop/screencap.raw", "w+");
fwrite(pixels, bpp, width * height, stream);
fclose(stream);
free(pixels);
}
or in C#:
// https://stackoverflow.com/questions/1537587/capture-screen-image-in-c-on-osx
// https://github.com/Acollie/C-Screenshot-OSX/blob/master/C%2B%2B-screenshot/C%2B%2B-screenshot/main.cpp
// https://github.com/ScreenshotMonitor/ScreenshotCapture/blob/master/src/Pranas.ScreenshotCapture/ScreenshotCapture.cs
// https://screenshotmonitor.com/blog/capturing-screenshots-in-net-and-mono/
namespace rtaStreamingServer
{
// https://github.com/xamarin/xamarin-macios
// https://qiita.com/shimshimkaz/items/18bcf4767143ea5897c7
public static class OSxScreenshot
{
private const string LIBCOREGRAPHICS = "/System/Library/Frameworks/CoreGraphics.framework/CoreGraphics";
[System.Runtime.InteropServices.DllImport(LIBCOREGRAPHICS)]
private static extern System.IntPtr CGDisplayCreateImage(System.UInt32 displayId);
[System.Runtime.InteropServices.DllImport(LIBCOREGRAPHICS)]
private static extern void CFRelease(System.IntPtr handle);
public static void TestCapture()
{
Foundation.NSNumber mainScreen = (Foundation.NSNumber)AppKit.NSScreen.MainScreen.DeviceDescription["NSScreenNumber"];
using (CoreGraphics.CGImage cgImage = CreateImage(mainScreen.UInt32Value))
{
// https://stackoverflow.com/questions/17334786/get-pixel-from-the-screen-screenshot-in-max-osx/17343305#17343305
// Get byte-array from CGImage
// https://gist.github.com/zhangao0086/5fafb1e1c0b5d629eb76
AppKit.NSBitmapImageRep bitmapRep = new AppKit.NSBitmapImageRep(cgImage);
// var imageData = bitmapRep.representationUsingType(NSBitmapImageFileType.NSPNGFileType, properties: [:])
Foundation.NSData imageData = bitmapRep.RepresentationUsingTypeProperties(AppKit.NSBitmapImageFileType.Png);
long len = imageData.Length;
byte[] bytes = new byte[len];
System.Runtime.InteropServices.GCHandle pinnedArray = System.Runtime.InteropServices.GCHandle.Alloc(bytes, System.Runtime.InteropServices.GCHandleType.Pinned);
System.IntPtr pointer = pinnedArray.AddrOfPinnedObject();
// Do your stuff...
imageData.GetBytes(pointer, new System.IntPtr(len));
pinnedArray.Free();
using (AppKit.NSImage nsImage = new AppKit.NSImage(cgImage, new System.Drawing.SizeF(cgImage.Width, cgImage.Height)))
{
// ImageView.Image = nsImage;
// And now ? How to get the image bytes ?
// https://theconfuzedsourcecode.wordpress.com/2016/02/24/convert-android-bitmap-image-and-ios-uiimage-to-byte-array-in-xamarin/
// https://stackoverflow.com/questions/5645157/nsimage-from-byte-array
// https://stackoverflow.com/questions/53060723/nsimage-source-from-byte-array-cocoa-app-xamarin-c-sharp
// https://gist.github.com/zhangao0086/5fafb1e1c0b5d629eb76
// https://www.quora.com/What-is-a-way-to-convert-UIImage-to-a-byte-array-in-Swift?share=1
// https://stackoverflow.com/questions/17112314/converting-uiimage-to-byte-array
} // End Using nsImage
} // End Using cgImage
} // End Sub TestCapture
public static CoreGraphics.CGImage CreateImage(System.UInt32 displayId)
{
System.IntPtr handle = System.IntPtr.Zero;
try
{
handle = CGDisplayCreateImage(displayId);
return new CoreGraphics.CGImage(handle);
}
finally
{
if (handle != System.IntPtr.Zero)
{
CFRelease(handle);
}
}
} // End Sub CreateImage
} // End Class OSxScreenshot
} // End Namespace rtaStreamingServer