I'm looking to extract form data utilizing textract. I've tested with a PDF in the demo and results are great. Results using the SDK however are far from optimal, actually, completely inaccurate. If I use StartDocumentAnalysisRequest/StartDocumentAnalysisResult (asynchronous), I only get 1 block returned of type PAGE, never KEY_VALUE_SET. If I convert my PDF to an image and use the synchronous methods, I do get KEY_VALUE_SET back but results are completely inaccurate.
Does anyone know how I can utilize the asynchronous analysis functionality to retrieve form values as the documentation indicates?
Sample Code below:
StartDocumentAnalysisResult startDocumentAnalysisResult = amazonTextract.startDocumentAnalysis(req);
String startJobId = startDocumentAnalysisResult.getJobId();
GetDocumentAnalysisResult documentAnalysisResult = null;
String jobStatus = "IN_PROGRESS";
while (jobStatus.equals("IN_PROGRESS")) {
try {
TimeUnit.SECONDS.sleep(10);
GetDocumentAnalysisRequest documentAnalysisRequest = new GetDocumentAnalysisRequest()
.withJobId(startJobId)
.withMaxResults(1);
documentAnalysisResult = amazonTextract.getDocumentAnalysis(documentAnalysisRequest);
jobStatus = documentAnalysisResult.getJobStatus();
} catch (Exception e) {
logger.error(e);
}
}
if (!jobStatus.equals("IN_PROGRESS")) {
List<Block> blocks = documentAnalysisResult.getBlocks();
logger.error("block list size " + blocks.size());
Map<String, Map<String, Block>> keyValueBlockMap = new HashMap<>();
Map<String, Block> keyMap = new HashMap<>();
Map<String, Block> valueMap = new HashMap<>();
Map<String, Block> blockMap = new HashMap<>();
for (Block block : blocks) {
logger.error("Block Type:" + block.getBlockType());
String blockId = block.getId();
blockMap.put(blockId, block);
if (block.getBlockType().equals("KEY_VALUE_SET")) {
if (block.getEntityTypes().contains("KEY")) {
keyMap.put(blockId, block);
} else {
valueMap.put(blockId, block);
}
}
}
keyValueBlockMap.put("keyMap", keyMap);
keyValueBlockMap.put("valueMap", valueMap);
keyValueBlockMap.put("blockMap", blockMap);
Map<String, String> keyValueRelationShip = getKeyValueRelationShip(keyValueBlockMap);
for (String key : keyValueRelationShip.keySet()) {
logger.error("Key: " + key);
logger.error("Value: " + keyValueRelationShip.get(key));
}
}
Synchronous path which results in completely horrible results:
AnalyzeDocumentRequest request = new AnalyzeDocumentRequest() .withFeatureTypes(FeatureType.FORMS) .withDocument(new Document(). withS3Object(new com.amazonaws.services.textract.model.S3Object() .withName(objectName) .withBucket(awsHelper.getS3BucketName())));
AnalyzeDocumentResult result = amazonTextract.analyzeDocument(request);
You are not using the recommended version for the AWS SDK for Java. You are using a old version and not the recommended one.
I have tested the AWS SDK for Java V2 and I am able to get lines and text that lines up with the AWS Management Console.
You can find textTract V2 examples in the repo linked above.
I am able to get to lines and the corresponding text by using software.amazon.awssdk.services.textract.TextractClient.
For example when i debug through the code using the same PNG as I used in the console, i get the proper result.
Related
The aspose.barcode reader is unable to read the barcode of type DecodeType.Code128
Workflow Steps
1>Using Aspose.Barcode we have created a barcode using DecodeType.Code128 and put on PDF page ( our clients use this page as separator sheet)
2>Our client then insert this barcode page between several physical documents and scanned them all, which creates big single PDF
3>Our splitting process then, loop through all pages and check if any page is barcode page, and splits the big PDF into individual small PDF
Issue is some times the scanned quality of the barcode is not that great, and in such case ASPOSE.Barcode unable to read the barcode.
I have attached couple of barcode PDF with low scanned quality, and aspose is not able to read these barcodes. I have tried different combinations of RecognitionMode and ManualHints options without any luck
Below is my code to identity barcode page
using (var fs = new FileStream(file, FileMode.Open))
{
var pdfDocument = new Document(fs);
foreach (Page page in pdfDocument.Pages)
{
var isSeparator = splitter.IsSeparator(page);
Assert.IsTrue(isSeparator);
}
}
public bool IsSeparator(Page page)
{
if (page.Resources.Images != null && page.Resources.Images.Count >= 1)
{
var img = page.Resources.Images[1];
using (MemoryStream barcodeImage = new MemoryStream())
{
img.Save(barcodeImage, ImageFormat.Jpeg);
barcodeImage.Seek(0L, SeekOrigin.Begin);
using (BarCodeReader barcodeReader = new BarCodeReader(barcodeImage, _barcodeDecodeType))
{
barcodeReader.RecognitionMode = RecognitionMode.MaxQuality;
while (barcodeReader.Read())
{
var barcodeText = barcodeReader.GetCodeText();
if (barcodeText.ToLower() == "eof")
{
return true;
}
}
}
}
}
return false;
}
Unable to reproduce the issue at my end. I used the following sample code snippet to recognize the barcode along with latest version of the API. It is always recommended to use the latest version of the API as it contains new features and improvements.
CODE:
Aspose.Pdf.License licensePdf = new Aspose.Pdf.License();
licensePdf.SetLicense(#"Aspose.Total.lic");
// bind the pdf document
Aspose.Pdf.Facades.PdfExtractor pdfExtractor = new Aspose.Pdf.Facades.PdfExtractor();
pdfExtractor.BindPdf(#"173483_2.pdf");
// extract the images
pdfExtractor.ExtractImage();
// save images to stream in a loop
while (pdfExtractor.HasNextImage())
{
// save image to stream
System.IO.MemoryStream imageStream = new System.IO.MemoryStream();
pdfExtractor.GetNextImage(imageStream);
imageStream.Position = 0;
Aspose.BarCode.BarCodeRecognition.BarCodeReader barcodeReader =
new Aspose.BarCode.BarCodeRecognition.BarCodeReader(imageStream);
while (barcodeReader.Read())
{
Console.WriteLine("Codetext found: " + barcodeReader.GetCodeText() + ", Symbology: " + barcodeReader.GetCodeType().ToString());
}
// close the reader
barcodeReader.Close();
}
Further to update you that the same query has been post on Aspose.BarCode support forum. You may please visit the link for details.
I work as developer evangelist at Aspose.
I need to make a python code available as WCF for another application to access it. The python code was build by the data science team and have no ability to change it. I tried running the program as a process shell but it gives 'System.InvalidOperationException' exception.
I created the same program as C# console application and it works fine. The question is
a. Is this the right way to go about making python code available to another application (REST API is not an option).
b. What is the issue with my code.
public string ClassifyText(string value)
{
string textoutput = "";
string exeFileName = HttpContext.Current.Server.MapPath("~/python.exe");
string argName = HttpContext.Current.Server.MapPath("~/predictionscript.py");
ProcessStartInfo start = new ProcessStartInfo();
start.FileName = exeFileName;
start.Arguments = argName;
start.UseShellExecute = false;
start.RedirectStandardOutput = true;
using (Process process = Process.Start(start))
{
using (StreamReader reader = process.StandardOutput)
{
string result = reader.ReadToEnd();
textoutput = result;
}
}
return textoutput;
}
Writing a WP8 Silverlight app. Is there a standard .NET technique available in this environment I can use to serialize an object like this
private static List<MemoryStream> MemoryStreamList = new List<MemoryStream>();
to save it to a file and restore it later?
I tried to use DataContractJsonSerializer for this which is good to serialize a List of simple custom objects, but it fails for List (I get System.Reflection.TargetInvocationException).
I would suggest converting your list to a list of byte arrays before persisting and then you should be able to serialize. Of course this comes with some overhead at deserialization as well.
Serialization part:
byte[] bytes = null;
var newList = MemoryStreamList.Select(x => x.ToArray()).ToList();
XmlSerializer ser = new XmlSerializer(newList.GetType());
using (var ms = new MemoryStream())
{
ser.Serialize(ms, newList);
//if you want your result as a string, then uncomment to lines below
//ms.Seek(0, SeekOrigin.Begin);
//using (var sr = new StreamReader(ms))
//{
//string serializedStuff = sr.ReadToEnd();
//}
//else you can call ms.ToArray() here and persist the byte[]
bytes = ms.ToArray();
}
Deserialization part:
using (var ms = new MemoryStream(bytes))
{
var result = ser.Deserialize(ms) as List<byte[]>;
}
I want to use "GATE" through web. Then I decide to create a SOAP web service in java with help of GATE Embedded.
But for the same document and saved Pipeline, I have a different run-time duration, when GATE Embedded runs as a java web service.
The same code has a constant run-time when it runs as a Java Application project.
In the web service, the run-time will be increasing after each execution until I get a Timeout error.
Does any one have this kind of experience?
This is my Code:
#WebService(serviceName = "GateWS")
public class GateWS {
#WebMethod(operationName = "gateengineapi")
public String gateengineapi(#WebParam(name = "PipelineNumber") String PipelineNumber, #WebParam(name = "Documents") String Docs) throws Exception {
try {
System.setProperty("gate.home", "C:\\GATE\\");
System.setProperty("shell.path", "C:\\cygwin2\\bin\\sh.exe");
Gate.init();
File GateHome = Gate.getGateHome();
File FrenchGapp = new File(GateHome, PipelineNumber);
CorpusController FrenchController;
FrenchController = (CorpusController) PersistenceManager.loadObjectFromFile(FrenchGapp);
Corpus corpus = Factory.newCorpus("BatchProcessApp Corpus");
FrenchController.setCorpus(corpus);
File docFile = new File(GateHome, Docs);
Document doc = Factory.newDocument(docFile.toURL(), "utf-8");
corpus.add(doc);
FrenchController.execute();
String docXMLString = null;
docXMLString = doc.toXml();
String outputFileName = doc.getName() + ".out.xml";
File outputFile = new File(docFile.getParentFile(), outputFileName);
FileOutputStream fos = new FileOutputStream(outputFile);
BufferedOutputStream bos = new BufferedOutputStream(fos);
OutputStreamWriter out;
out = new OutputStreamWriter(bos, "utf-8");
out.write(docXMLString);
out.close();
gate.Factory.deleteResource(doc);
return outputFileName;
} catch (Exception ex) {
return "ERROR: -> " + ex.getMessage();
}
}
}
I really appreciate any help you can provide.
The problem is that you're loading a new instance of the pipeline for every request, but then not freeing it again at the end of the request. GATE maintains a list internally of every PR/LR/controller that is loaded, so anything you load with Factory.createResource or PersistenceManager.loadObjectFrom... must be freed using Factory.deleteResource once it is no longer needed, typically using a try-finally:
FrenchController = (CorpusController) PersistenceManager.loadObjectFromFile(FrenchGapp);
try {
// ...
} finally {
Factory.deleteResource(FrenchController);
}
But...
Rather than loading a new instance of the pipeline every time, I would strongly recommend you explore a more efficient approach to load a smaller number of instances of the pipeline but keep them in memory to serve multiple requests. There is a fully worked-through example of this technique in the training materials on the GATE wiki, in particular module number 8 (track 2 Thursday).
Ok I have written some basic generic webservices before but I have never tried to consume a 3rd party one.
The one I am trying to consume is
http://opendap.co-ops.nos.noaa.gov/axis/webservices/predictions/wsdl/Predictions.wsdl
I am not getting any results back from this what so ever and cannot figure out why.
More odd is it is not even reaching PredictionsClient_getPredictionsAndMetadataCompleted when I put a break point in the code it doesn't even reach it.
Any suggestions would be greatly appreciated
public void Bouy(double meters)
{
PredictionService.Parameters PredictionParams = new PredictionService.Parameters();
PredictionService.PredictionsPortTypeClient PredictionsClient = new PredictionService.PredictionsPortTypeClient();
GeoCoordinateWatcher gc = new GeoCoordinateWatcher(GeoPositionAccuracy.Default);
//gc.Position.Location.Latitude, gc.Position.Location.Longitude
GeoCoordinate myLocation = new GeoCoordinate(27.931631,-82.802582);
foreach (var bl in BouyLocation.GetAll())
{
GeoCoordinate otherLocation = new GeoCoordinate(bl.Lat, bl.Lon);
PredictionParams.beginDate = DateTime.Now.ToString("yyyyMMdd");
PredictionParams.endDate = DateTime.Now.AddDays(1.0).ToString("yyyyMMdd");
PredictionParams.stationId = bl.LocationID;
PredictionParams.timeZone = 0;
PredictionParams.unit = 1;
PredictionParams.dataInterval = 6;
PredictionsClient.getPredictionsAndMetadataCompleted += new EventHandler<PredictionService.getPredictionsAndMetadataCompletedEventArgs>(PredictionsClient_getPredictionsAndMetadataCompleted);
PredictionsClient.getPredictionsAndMetadataAsync(PredictionParams);
double mymeters = myLocation.GetDistanceTo(otherLocation);
if (mymeters < meters)
{
TextBlock DynTextBlock = new TextBlock
{
Name = "Appearance" + bl.LocationID,
Text = bl.LocationName + PredictionResult,
TextWrapping = System.Windows.TextWrapping.Wrap,
Margin = new Thickness(12, -6, 12, 0),
Style = (Style)Resources["PhoneTextSubtleStyle"]
};
DynamicAppearance.Children.Add(DynTextBlock);
this.nearByLocations.Add(new BouyLocationModel() { LocationName = bl.LocationName, LocationID = bl.LocationID, Lat = bl.Lat, Lon = bl.Lon });
}
}
var test = nearByLocations;
}
void PredictionsClient_getPredictionsAndMetadataCompleted(object sender, PredictionService.getPredictionsAndMetadataCompletedEventArgs e)
{
string err = e.Error.ToString();
PredictionResult = e.Result.ToString();
}
Loooking at the code you have here I think that you have used the importing of a ServiceReference to auto build the classes for you?
Unfortunately I have found that this is rather temperamental on WP7 and the only way I actually got it to work was when I connected it to a Microsoft WCF service. Connecting to anything else just doesn't work.
If you do google searches there are various pages talking about the fact it doesn't work and ways around it (which I couldn't get to work).
However, there are ways around it but it isn't as simple as the auto-generated stuff. Basically you do things manually.
Although there are other ways to manually create the web service what I did was follow the information in the following which worked well: http://zetitle.wordpress.com/2010/10/14/using-reactive-extensions-with-webrequest/
You will need to parse the response yourself but XML to LINQ works really well for this.
Hope that helps, or maybe someone will have the solution as it is something I am interested in knowing how to get working too