I am building an OCR based solution to extract information from certain financial documents.
As per the regulation in my country (India), this data cannot leave India.
Is it possible to find the region where Google Cloud Vision servers are located?
Alternately, is it possible to restrict the serving region from the GCP console?
This is what I have tried:
I went through GCP Data Usage FAQ: https://cloud.google.com/vision/docs/data-usage
GCP Terms of Service:
https://cloud.google.com/terms/
(Look at point 1.4 Data Location on this page)
Talking to the GCP Sales rep. He did not know the answer.
I know that I can talk to Google support but that requires $100 to activate, which is a lot for for me.
Any help would be appreciated. I went through the documentation for Rekognition as well but it seems to send some data outside for training so not considering it at the moment.
PS - Edited to make things I have tried clearer.
For anyone looking at this topic recently, Google Vision has introduced multi-region support in December 2019, as can be see in their release notes.
Currently Google Vision supports 2 different processing regions: eu and us, and they say that using a specific endpoint guarantees that processing will only take place in the chosen territory.
The documentation for regionalization mentions that you can simply replace the default API endpoint vision.googleapis.com with either of the regional ones:
eu-vision.googleapis.com
us-vision.googleapis.com
The vision client libraries offer options for selecting the endpoint as well, and the documentation gives code samples.
For example, here is how you would do it in Python:
from google.cloud import vision
client_options = {'api_endpoint': 'eu-vision.googleapis.com'}
client = vision.ImageAnnotatorClient(client_options=client_options)
As pointed out by #Tedinoz in a comment above, the answer can be found here: https://groups.google.com/forum/#!topic/cloud-vision-discuss/at43gnChLNY
To summarise:
1. Google stores images uploaded to Cloud Vision only in memory
2. Data is not restricted to a particular region (as of Dec 6, 2018)
3. They might add data residency features in Q1, 2019.
Related
I have users uploading videos in all kinds of formats, some not supported by chrome.
I just want to transcode all videos so they play in the browser, e.g. h.264.
I have looked at AWS Elastic Media Convert that does not have any documentation that is actually explaining how to go from zero to hero, let alone having meaningful templates for jobs.
Is it at all possible to do that with media convert and if not, what would be an appropriate service?
In general, yes. Existing AWS Blog articles describe workflows using S3 "watch folders" to convert new files as they arrive, using MediaConvert to covert files to a ubiquitous output format and deliver the converted output to a specified S3 location which can support web access both directly and as an origin for CDN distributions.
There are several considerations (for example: very long input files, exotic file formats, nature of content, etc) for which you may want to account. The design and testing of a multi-step workflow with error handling can be complex. If you wish to use outside expertise, you have at least three options:
[a] AWS Paid Professional Services -
There is a large global AWS ProServices team able to help via paid service engagements. The fastest way to start this dialog is by submitting the AWS Sales team 'contact me' form found at this link and specifying 'Sales Support' : https://aws.amazon.com/contact-us/
[b] AWS Certified Consulting Partners -- The partner search tool & listings are here: https://iq.aws.amazon.com/services/aws/medialive
[c] AWS Solutions Architects -- for AWS Enterprise Support customers. The TAM or the Sales contact linked in item [a] is the best way to engage them. Purchasing AWS Enterprise Support will entitle the customer to a dedicated TAM / SA combination.
Here is the documentation for AWS Elemental MediaConvert and following the getting started steps will get you a basic understanding of the service.
If there is any specific issue with the usage of the service, please create a new question.
This is more of a conceptual question as there is no relevant documentation available. We have an on prem IBM-MQ from which we need to transfer data on our cloud storage bucket (GCP/AWS), what could be possible solutions in this case? Any help or direction would be appreciated. Thank you!
I'm assuming you can reach your goal once the MQ-data has been changed/converted to supported format by the Big Query.
You can refer on this google documentation for full guide on Loading data from local files. You can upload file via GCP Console or using selected programming language that will match on your on-prem. There's also variety of uploads that you can choose from according to data file. This also includes the right permission to use the BigQuery.
If you require authentication you check on this Big Query Authentication Guide
I was using google cloud platform to host a ckan based website. The website had a file library with about 5 gigs of documents. Our project got put on hold and I removed billing on the website about 8 months ago. Now we are trying to migrate the data, but when I look at the project in Google Cloud Platform, there are no compute instances, buckets or files under the various storage modules. I cannot find the 5gigs of files we uploaded and filed by various categories on the ckan website. What could have happened to them? I'm not very experience with this platform and a bit confused. Is there any way for me to recover my data?
As the GCP public documentation indicates, if your billing account remains invalid for a protracted period, some resources might be removed from the projects associated with your account. For example, if you use Google Cloud, your Compute Engine resources might be removed.
Removed resources are not recoverable, the best option you had was to review your case with tech support and have a more specific answer, I noticed that there isn't any way to recover resources 30 days later it had been removed, this is also described in Data deletion term, including existing copies.
I need to transcribe a large number of Handwritten documents. I tried to use cloud services from either Google, Amazon, and Microsoft. Namely:
https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/
https://cloud.google.com/vision/docs/handwriting
https://aws.amazon.com/textract/
Unfortunately, none of them achieved good enough results. I suspect it is because my documents have a weird handwriting style, and as a result, the networks struggle a lot.
I searched whether I could fine-tune (with manually transcribed data), but I have not found anything online, so as a last resort, I ask here.
If it is possible to fine-tune one of these models, could you please point me some resources?
You are correct, with Azure Cognitive Services with Computer Vision you cannot upload your own data to train the API to recognise the handwriting in your documents I'm afraid. I can't comment on the other offerings from AWS and Google I'm afraid, but certainly not for Azure.
I'd like to be able to estimate the cost of a download operation from S3 programmatically but I hesitate to hard-code the prices they list (per GB) on their pricing page in the event they change. Does the SDK provide any kind of access to current pricing data. If it does, I can't seem to find it.
CLARIFICATION: I'm asking if the official Amazon SDK has hooks for pricing data, not if it's possible to get pricing data at all. Obviously, it is possible to get pricing data through non-documented means.
Since you're asking for SDK support and SDKs are language-specific, I have to stress I can only speak for the Ruby SDK.
The Ruby SDK in the latest major version (2.x) is mostly auto-generated by an API description in JSON format for each documented API. There is no official pricing API – only a static file that states:
This file is intended for use only on aws.amazon.com. We do not guarantee its availability or accuracy.
This means there is no way for the Ruby SDK to give you pricing information. Your mileage may vary in other languages (but I doubt it).