google speech recognition weak date transcription - google-cloud-platform

I am using currently google asr/tts with german speech model (de-DE) and experience wrong results in certain usage scenarios of date extractions.
I am really keen on to know whether others might have similar results.
Let me give you some examples:
I am saying:
"der 1.10.1905" -> "1. 10195 11095"
"9.5.78" -> 90587
"22.11.98" -> 22.11 89
BUT:
"22. November 98" -> "22 November 98"
When I fully qualify the month word it works all fine.
I also checked the proposed way to optimize by using hints, without any improvement
"speechContexts": [
{
"phrases": [
"$FULLDATE"
]
}
]
Is this something one has to accept or is there anything that I can try on top?
Cheers Andre
UPDATE:
According to https://issuetracker.google.com/issues/186559805 the problem should be fixed.
But
I could not verify it, maybe somebody else can ??!
I tried it with the following configuration, but it did not improve at all. Maybe overlooked something ?!
Here is my configuration. I performed the request with the beta client:
const {SpeechClient} = require('#google-cloud/speech').v1p1beta1;
const googleRequest = {
config: {
encoding: "MULAW",
sampleRateHertz: 8000,
model: "command_and_search",
languageCode: "de-DE",
speechContexts: [
{
"phrases": [
"$OOV_CLASS_FULLDATE"
]
}
]
},
singleUtterance: true,
interimResults: true
};
I also raised a new ticket to google developers.
https://issuetracker.google.com/issues/243294056

I have the same results as you have when using your example. But it does transcribe correctly if the speaker says ein tausend neun hundert acht und neunzig instead.
It seems that, for some reason, in German, the API doesn't recognize when the speaker uses hundreds to express thousands (I.E. nineteen hundred ninety eight instead of one thousand nine hundred ninety eight).
I don't know German and can't confirm if this way of speaking numbers/dates is official but if you are sure that in German, it is officially accepted, then I suggest that you create an issue on issue-tracker with some references to confirm it.

Related

Google Document AI training fails due to an error that is already addressed

I am training a model using Google's Document AI. The training fails with the following error (I have included only a part of the JSON file for simplicity but the error is identical for all documents in my dataset):
"trainingDatasetValidation": {
"documentErrors": [
{
"code": 3,
"message": "Invalid document.",
"details": [
{
"#type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "INVALID_DOCUMENT",
"domain": "documentai.googleapis.com",
"metadata": {
"num_fields": "0",
"num_fields_needed": "1",
"document": "5e88c5e4cc05ddb8.json",
"annotation_name": "INCOME_ADJUSTMENTS",
"field_name": "entities.text_anchor.text_segments"
}
}
]
}
What I understand from this error is that the model expects the field INCOME_ADJUSTMENTS to appear (at least) once in the document but instead, it finds zero instances of it.
That would have been understandable except I have already defined the field INCOME_ADJUSTMENTS in my schema as "Optional Once", i.e., this field can appear either zero or one time.
Am I missing something? Why does this error persist despite the fact that it is addressed in the schema?
p.s. I have also tried "Optional multiple" (and "Required once" and "Required multiple") and the error persists.
EDIT: As requested, here's what one of the JSON files looks like. Note that there is no PII here as the details (name, SSN, etc.) are synthetic data.
I have/had the same issue as you in the past and also having it right now.
What I managed to do was to get the document string from the error message and then searching for the images in the Storage bucket that has the dataset.
Then I opened the image and searched for that image in my 1000+ images dataset.
Then I deleted the bounding box for the label with the issue and then relabeled it. This seemed to solve 90%of the issues I had.
It`s a ton of manual work and I wish google thought of more when they released the Web app for Doc AI because the ML part is great but the app is really lackluster.
I would also be very happy for any other fixes
EDIT: another quicker workaround I have found is deleting the latest revision of the labeled documents from the Dataset in cloud storage. Like, take faulty document name from the operation json dump, then search for it in documents/ and then just delete latest revision.
Will probably mess up labeling and make you lose work, but it`s a quick fix to at least make some progress if you want.
Removed a few empty boxes and a lot of intersecting boxes fixed it for me.
i had the same problem.
so i deleted all my dataset and imported and re-labeled again.
then the training worked fine.

GCP data Loss Prevention - not detecting local types

I am working with GCP's DLP APi, and I have issues detecting country-specific types. On the other hand, I have no issues with global types (here you can find the list of types). Does anyone have suggestions on how to fix this? In case it might help, I'm working from outside the US.
This is a copy of my config file:
info_types_rep_names = {"PHONE_NUMBER": "[PHONE]",
"EMAIL_ADDRESS": "[EMAIL]",
"US_PASSPORT": "[PASSPORT]",}
info_types = [{"name": key} for key, value in info_types_rep_names.items()]
deidentify_config = {
"info_type_transformations": {
"transformations": [
{
"info_types" : [{"name": key}],
"primitive_transformation": {
"replace_config": {
"new_value": {"string_value": value}
}
}
} for key, value in info_types_rep_names.items()
]
}
}
Locations might be affected in some scenarios. Refer to this document for Country specific values.
There is a piece of code and I could see that there might be a chance that the provided value may be wrong.
I have tested from our end with sample code using Python refer document and for using info_types_rep_names, by changing the project id “input_str = 'Please call me. My phone number is. My email, just in case, is adfsfasfs#gmail.com. Take a note of US passport number: C03005988. Or maybe C03004786'” and works for US_PASSPORT, but the sample number should absolutely work for sure to be valid. Also there may be a possibility that only a country specific one may fail. But when tested with some other country specific value and it worked. Also I could see all the results are aligned with the demo detection.
Also making sure the right InfoTypes are in use on the right section.follow the link , and in the options tab you can view and adjust the InfoTypes.
So my first question is , is it getting aligned with correct Infotypes or not?
If it is aligned, then it is either our detection fails with specific samples or might be the sample is not valid.
If it is not valid then it is either the matter of having the wrong sample or a bug in the code.
Please check and revert if there are any issues.

How do I find out where the response from dialogflow comes from?

I'm not a developer, so this is a little above my head.
My team has implemented a project in dialogflow, one for an old app and one from a new app. I have basic access to the old dialogflow account and I can see that it has an intent called glossaries, same intent name as in the new one. In glossaries, there is a training phrase called "What is a red talk?". This phrase only works in one of my apps and I need to know why.
There is no default response or anything under context. If I copy that curl link into a terminal, the payload doesn't return with any information.
I found the API for the new app and red talks is definitely not in the payload when I do a GET/all. There may be an old API somewhere, but no one knows where.
Where can I find this information? I'm very confused and all the basic training for dialogflow points to default response, which we're not using. I have read through the docs. I have searched the three company github repos that have the application in the name but I have not found anything. I am looking for an app.intent phrase with glossaries in it or just the word glossaries.
I have found only this json and a glossaryTest.php that doesn't seem helpful:
"meta": {
"total": 2,
"page": 1,
"limit": 10,
"sort": "createdAt",
"direction": "desc",
"load-more": false
},
"results": [
{
"term": "This is a term",
"definition": "This is a definition",
"links": [
{
"id": "1",
"url": "http:\/\/example.com\/1",
"title": "KWU Course: Lead Generation 36:12:3",
"ordering": "1"
},
{
"id": "2",
"url": "http:\/\/example.com\/2",
"title": "",
"ordering": "2"
}
]
}
]
}
There is also a json with a lot data for API calls but no glossaries there either.
If we're using fulfillment to handle these intents, I don't see a fullfillment header like google docs say there should be. I may not have full access so perhaps I would be viewing more information in the screen if I had that, I have no idea. The devs who created this are long gone. The devs who also created the new app are also long gone.
Am I missing an API in my environment documentation? Is the intent hard coded? I suspect it was. How do I prove that or move forward?
Yes, your intent are somehow hard-coded [0], or defined through the UI.
Each intent has a setting to enable fulfillment. If an intent requires
some action by your system or a dynamic response, you should enable
fulfillment for the intent. If an intent without fulfillment enabled
is matched, Dialogflow uses the static response you defined for the
intent. [2]
Perhaps you are using a custom integration [1]. So, unless you are using static response (those you see in the UI), the frontend code may be managed by your project API (not Dialogflow API), and perhaps the content modified before performing any further or eventually returning the response.
As I understand you should contact your colleagues for understanding about the integration solution they have created. Or otherwise if the Intent has been created through the API, look for its relative files where there may be They may have created the integration through the SDK, while picking up training data from a source out of the codebase. So perhaps you cannot see it directly in the code. Nonetheless, you should be able to access it through the UI once it has been created.
In case my answer was not of your help, please do not hesitate to further clarify your needs, perhaps providing some further information.
[0] https://cloud.google.com/dialogflow/docs/manage-intents#create_intent
[1] https://cloud.google.com/dialogflow/docs/integrations
[2] https://cloud.google.com/dialogflow/docs/fulfillment-overview

Azure Cosmos DB - Gremlin latitude longitude format conversion issues

I am trying to convert airport GeoCoordinate data i.e. [IATA Code, latitude, longitude] to Gremlin Vertex in an Azure Cosmos DB Graph API project.
Vertex conversion is mainly done through an Asp.Net Core 2.0 console application using CSVReader to stream and convert data from a airport.dat (csv) file.
This process involves converting over 6,000 lines...
So for example, in original airport.dat source file, the Montreal Pierre Elliott Trudeau International Airport would be listed using a similar model as below:
1,"Montreal / Pierre Elliott Trudeau International Airport","Montreal","Canada","YUL","CYUL",45.4706001282,-73.7407989502,118,-5,"A","America/Toronto","airport","OurAirports"
Then if I define a Gremlin Vertex creation query in my cod as followed:
var gremlinQuery = $"g.addV('airport').property('id', \"{code}\").property('latitude', {lat}).property('longitude', {lng})";
then when the console application is launched, the Vertex conversion process would be generated successfully in exact similar fashion:
1 g.addV('airport').property('id', "YUL").property('latitude', 45.4706001282).property('longitude', -73.7407989502)
Note that in the case of Montreal Airport (which is located in N.A not in the Far East...), the longitude is properly formatted with minus (-) prefix, though this seems to be lost underway when doing a query on Azure Portal.
{
"id": "YUL",
"label": "airport",
"type": "vertex",
"properties": {
"latitude": [
{
"id": "13a30a4f-42cc-4413-b201-11efe7fa4dbb",
"value": 45.4706001282
}
],
"longitude": [
{
"id": "74554911-07e5-4766-935a-571eedc21ca3",
"value": 73.7407989502 <---- //Should be displayed as -73.7407989502
}
]
}
This is a bit awkward. If anyone has encountered a similar issue and was able to fix it, then I'm fully open to suggestion.
Thanks
According to your description, I just executed Gremlin query on my side and I could retrieve the inserted Vertex as follows:
Then, I just queried on Azure Portal and retrieved the record as follows:
Per my understanding, you need to check the execution of your code and verify the response of your query to narrow down this issue.
Thank you for your suggestion, though problem has now been solved in my case.
What was previously suggested as a working answer scenario [and voted 1...] has long been settled in case of .Net 4.5.2 [& .Net 4.6.1] version used in combination with Microsoft.Azure.Graph 0.2.4 -preview. The issue of my question didn't really concern that and may have been a bit more subtle... Perhaps I should have put a bit more emphasis on the fact that the issue was mainly related to Microsoft.Azure.Graph 0.3.1 -preview used in Core 2.0 + dotnet CLI scenario.
According to following Graph - Multiple issues with parsing of numeric constants in the graph gremlin query #438 comments on Github,
https://github.com/Azure/azure-documentdb-dotnet/issues/438
there are indeed some fair reasons to believe that the issue was a bug with Microsoft.Azure.Graph 0.3.1 -preview. I chose to use Gremlin.Net approach instead and managed to get the proper result I expected.

GeoCoding Issues with OpenStreetMap/Nominatim

I have a website which needs to obtain the Latitude and Longitude for the address entered by the customer.
Google/Bing/Yahoo are too expensive for us so we went with OpenStreetMap/Nominatim.
Unfortunately while it worked OK during testing, its failing to find about 50% of the addresses entered which is a big issue.
There are 3 things I am interested in knowing:
What is the best way to deal with the situation where the customer really does enter an incorrect address - send them an email and ask them to correct it? Use segments of the address until something is found?
What is the best way to handle the situation where the address is fine but I can't find it with OpenStreetMap? Or am I doing something wrong with my query to Nominatim?
Does anyone know of a free/cheap alternative if OpenStreetMap isn't up to the task? I know its an open source collaboration and therefore not complete, but I thought it did have pretty good coverage, and that it would return a nearby location if it didn't have the exact location - maybe it does and maybe I'm using it wrong.
Here is an example:
182 livington ave,albany,New York,12210,US
Google maps finds that easily.
Nominatim finds nothing: http://nominatim.openstreetmap.org/search?format=xml&addressdetails=0&q=182%20livington%20ave,albany,New%20York,12210,US
I think what you're looking for is address verification. Google, Nominatim, and others, only perform address approximation which is good for finding addresses when you aren't sure what they are, but the results are only a best guess.
I helped develop an API which verifies and geocodes addresses according to stringent CASS™ requirements called LiveAddress. I ran your sample address through Google, Nominatim, and LiveAddress API and these are the results:
Google found the address despite the typo in "Livingston" but could not guarantee its validity, saying, "Address is approximate." -- then again, it says that for just about every address you try.
Nominatim does not find it because of the typo. Perhaps a drawback to using Nominatim is that it doesn't try to compensate for typos, verify the accuracy or completeness of addresses, etc. Fixing the typo returned some information but it was anyone's guess what had to be fixed, and why the query failed anyway.
LiveAddress doesn't recognize the address as entered because of the typo. Missing the "s" in "Livingston" is dramatic because there are streets named "Livington," leaving the query ambiguous, and the results were too much of a mis-match to return according to CASS™ specs. Changing the name with a different typo, "Livingstn," however, produced a valid result, which typo Nominatim did't accept either:
... for some reason I have to break out of my bullet points for code to render properly:
[
{
"input_index": 0,
"candidate_index": 0,
"delivery_line_1": "182 Livingston Ave",
"last_line": "Albany NY 12210-2512",
"delivery_point_barcode": "122102512824",
"components": {
"primary_number": "182",
"street_name": "Livingston",
"street_suffix": "Ave",
"city_name": "Albany",
"state_abbreviation": "NY",
"zipcode": "12210",
"plus4_code": "2512",
"delivery_point": "82",
"delivery_point_check_digit": "4"
},
"metadata": {
"record_type": "S",
"county_fips": "36001",
"county_name": "Albany",
"carrier_route": "C011",
"congressional_district": "21",
"rdi": "Residential",
"latitude": 42.66033,
"longitude": -73.75285,
"precision": "Zip9"
},
"analysis": {
"dpv_match_code": "Y",
"dpv_footnotes": "AABB",
"dpv_cmra": "N",
"dpv_vacant": "N",
"active": "Y",
"ews_match": false,
"footnotes": "M#"
}
}
]
The analysis footnote "M#" indicates a match was achieved by fixing the spelling of the street name. The resulting DPV footnotes "AABB" indicate that the entire address matched a street + city/state on the national ZIP+4 file. Also note that Zip9 precision which is the most precise level of geocoding (currently) — accurate to block (or closer) level.
So, in answer to your questions:
That depends. Are your customers entering an address on a website form? Tell them right away before they continue, that the address isn't valid. We're working on a jQuery plugin to make this cut-and-paste easy for everybody, but until then, you can see our concept in our checkout form which implements a pretty slick system: SmartyStreets has a jQuery Plugin which verifies addresses on website forms (just copy-and-paste). When an address is typed, it is automatically verified. If it is wrong, they slide up a notification asking the user if they'd like to fix it. Sometimes their address is ambiguous, where it returns a few valid results. (Try: "100, new york, ny") — They show a few suggestions and you can pick one. You fix it and the form does not submit until the user gets a valid address or says "Use mine anyway; I guarantee it's right." Or, if the address is correct, they put the standardized results in the address fields and display a green notice: "Address verified!"
I think I discussed this above. Your query is fine; it seems to be a shortcoming in Nominatim.
As suggested, you could try LiveAddress. Try it with a large set of your addresses to get a better idea (comparing from one address alone is, I'll admit, a weak indication) — but so far it seems like, for your needs, LiveAddress is somewhere between Google Maps and Nominatim.
Answer to question in comments
I ran out of room in the comments.
Q:
here is another address causing us issues "7580 E Big Cannon Drive,Anaheim Hills,Anaheim Hills,California,92808,US" even "7580 E Big Cannon Drive,California,92808,US" didn't seem to work with your site.
A:
I did some research on the USPS site and some other service providers as well. None returned any valid results or suggestions. But I found out what's the issue with the address as you submitted it:
Mispelled street name. No biggie; LiveAddress corrected this to Big Canyon.
Bad primary number. There's not much hope here if the primary number is incorrect. There's generally no way for a computer or human to infer what you really meant. In these cases, the address will fail verification and the user must supply something valid to go on. I found a valid primary number at 7584.
Master-planned community, not city/county. "Anaheim Hills" is the name of a master-planned community. Google found it in its business listings, but that has nothing to do with the address.
"Anaheim Hills" twice. It's confusing the parser. Unfortunately, with extra unnecessary information (esp. in a single-line address), it's nearly impossible to tell what part of it is dubious. That second "Anaheim Hills" has to go, but the first one can stay and it will be fine.
Country information. Most of the services I tried your address on got confused with the country in front and put it in the "Company/Firm Name" field. We deal with US addresses, so you can omit the country. It'll reduce the size of your request too.
LiveAddress was actually able to verify the address in these forms, both as a single-line address and split into components:
7584 E Big Cannon Drive anaheim hills ca 92808
7584 bg cannon 92808
7584 big cannon ave aneheim hills ca
The most significant help was finding a valid primary number. In the case that no valid addresses come back, you should alert the user and suggest fixing the primary number and making sure the city/state (if given) align with the zip code ('cause if those two are fighting, it's also impossible to tell what you meant).