I hope you guys are doing well.
We are evaluating some solutions (Apache Camel K and the likes) to allow teams to:
Low Code protocol transformation (Kafka, FTP, S3, MQ, SOAP, SFTP, gRPC, GraphQL, etc.) One team in particular has to integrate their product with 100s of external partners (each one uses a different integration technology), and writing each integration "by hand" would be a waste of time/motivation.
Enrich integrations' payloads (by calling both internal and external services)
Pay per execution/transformation/step (SERVERLESS)
Orchestrate processes that span multiples domain/services (On either our GCP account or Partners external Datacenters)
Strong retry and monitoring capabilities
Be part of our CI/CD pipeline (and not be limited to a Graphical interface)
The items in bold seem to be part of what Cloud Workflows does natively, but are the other requirements something that can be added to (or achieved with) GCW to keep it "serverless"? Please.
Any help would be appreciated.
Thanks
Cloud Workflow can perform basic transformation (on string or date), but I can't recommend that. It's better to have a Cloud Functions or a Cloud Run that perform a transformation with code. You will be able to write unit test on it and to ensure the quality and the evolution of your system without regressions.
For orchestration, it's the purpose of Cloud Workflows. Now, there is also some limits, or some corner case less easy to achieve with it. It depends on the complexity of your process and your expectations (observability, portability, replayability,...)
I am trying to create a carePlan as output by doing $apply operation on PlanDefinition and ActivityDefintion.
For example:
https://healthcare.googleapis.com/v1beta1/projects/<project_id>/locations/<location_name>/datasets/<dataset_name>/fhirStores/<fhirstore_name>/fhir/PlanDefinition/<plandefintion_id>/$apply?subject=Patient/
This endpoint is throwing Error 404 (Not Found)!!1.
Tried many ways of making this endpoint work as per FHIR R4 documentation. But seems like GCP is not yet supporting $apply operation in PlanDefintion and ActivityDefinition in both v1 and v1beta1 version of Healthcare APIs.
Need help on this.
On their documentation page they indicate that you can view your server's capabilities by retrieving the CapabilityStatement. If this does not list the $apply operation, it is not supported.
And actually further down on the page, the documentation mentions: "Most extended operations aren't implemented". If you need that functionality, maybe you can reach out to them to see if it can be integrated, or look into using a different FHIR server implementation.
We are planning to use extensively Google APIs to transcribe audio in text from a single computer (about 10.000 requests approximately at the same time).
Except quota issues, are you aware of any other limitations on APIs calls coming from the same address ?
Thanks a lot !
If you look at the following article, you can see that there are some restrictions and limits when it comes to the speech-to-text api.
Other than the ones mentioned in the article, there are no restrictions or limits. When making a call to an API, it depends on the amount of requests, not where it comes from. The API does not care if the requests come from one or multiples addresses. As long as the requests do not go over the Quota that the API can handle, it will be okay.
Is there any way to programmatically get data similar to APIs overview of Google CLoud dashboard. Specifically, I'm interested in the list of APIs enabled for the project and their usage/error stats for some predefined timeframe. I belive there's an API for that but I struggle to find it.
There's currently no API that gives you a report similar to the one you can see through the Google Cloud Console.
The Compute API can retrieve some quotas with the get method but it's somewhat limited (only Compute Engine quotas) and, for what I understood from your question, not quite what you're looking for.
However, I've found in Google's Issue Tracker a feature request that's close to what you're asking for.
If you would need something more specific or want to do the feature request yourself, check the "Report feature requests" documentation and create your own. The GCP team will take a look at it to evaluate and consider implementation.
I'm planning to use Google Vision for document recognition.
For example, I will upload driver license and I should get all text data and verify that it is driver license and not the cover of a magazine.
The question is: does Google Vision has API for deletion of uploaded images?
Does Google Vision fit my case if I have some security requirements?
If you use Google's mobile vision API, text and face detection is done on device rather than being uploaded:
https://developers.google.com/vision/
For those who wondering the same problem, You can check their data policy here.
https://cloud.google.com/vision/docs/data-usage
My reading of Google APIs Terms of Service indicates that you will not be able to delete the images.
5b. Submission of Content
Some of our APIs allow the submission of content. Google does not acquire any ownership of any intellectual property rights in the content that you submit to our APIs through your API Client, except as expressly provided in the Terms. For the sole purpose of enabling Google to provide, secure, and improve the APIs (and the related service(s)) and only in accordance with the applicable Google privacy policies, you give Google a perpetual, irrevocable, worldwide, sublicensable, royalty-free, and non-exclusive license to Use content submitted, posted, or displayed to or from the APIs through your API Client. "Use" means use, host, store, modify, communicate, and publish. Before you submit content to our APIs through your API Client, you will ensure that you have the necessary rights (including the necessary rights from your end users) to grant us the license.
Being able to "publish" your driver's licenses is probably not something you want.
The above terms are also completely at odds with the GDPR where the user has the right to delete and modify their data.
7a. Google Privacy Policies
By using our APIs, Google may use submitted information in accordance with our privacy policies.
Note that those privacy policies are the ones that govern normal users, not cloud specifically. In plain text, and IANAL, it means that Google assumes that for whatever content you give them, the user has agreed to anything that Google does for a user that directly use, say Google Docs.
That's another indication that it's impossible to use their APIs and be GDPR compliant.
This should solve your issue
tl;dr "The stored image is typically deleted in a few hours."
Will the image I send to the Cloud Vision API, the results or other
information about the request itself, be stored on Google servers? If
so, how long and where is the information kept, and do I have access
to it? When you send an image to Cloud Vision API, we must store that
image for a short period of time in order to perform the analysis and
return the results to you. The stored image is typically deleted in a
few hours. Google also temporarily logs some metadata about your
Vision API requests (such as the time the request was received and the
size of the request) to improve our service and combat abuse.
Some of the other answers a bit outdated so adding my own answer. The data usage FAQ states
When you send an image to Vision API, we must store that image for a short period of time in order to perform the analysis and return the results to you. For asynchronous offline batch operations, the stored image is typically deleted right after the processing is done, with a failsafe Time to live (TTL) of a few hours. For online (immediate response) operations, the image data is processed in memory and not persisted to disk.
If you use the synchronous Vision API methods, the image is never persisted in Vision API and so there is nothing to delete. If you use the asynchronous Vision API methods, the image is only persisted during the operation and is deleted immediately after the operation completes with a fail-safe of a few hours. Again there is nothing for the user to delete, Vision API takes care of deleting the data for you.
A related question that sometimes comes up is about enforcing usage to take palce in a particular region. You can see the answer here: Google Vision: How to enforce processing in EU
Depends on your security requirements, and the exact privacy law one needs to abide by. In my case, it was HIPAA, one needs to jump through a lot of hoops, but according to https://cloud.google.com/security/compliance/hipaa, Google Cloud Vision API is a HIPAA covered product.