How can I orchestrate and call cloud services from different providers: Google Cloud, AWS and Azure
We did check workflows, but it seems that workflows are limited in each provider's platform, so if I build a workflow in Azure, I can only include services from Azure, but not from Google Cloud or AWS.
Cloud Workflow is an orchestrator that call APIs. There is a better integration for Google Cloud services (and with Google Security) but you can also use other API, such as other Cloud Provider services API.
You can also use Cloud Composer, based on Apache Airflow, which has a large library of operator that help you to call different services on different platform. Because it's initially (and still) an open source project, it's portable and there is no Cloud Preference on that solution.
If you want an orchestration tool for multi Cloud usage, Airflow should be the best approach because it proposes many operators for each Cloud Provider.
It's open source, well documented, there are many supports and example in each Cloud Provider.
It's also the best tool if you want to be Cloud Agnostic and migrate it easily to any Cloud provider if needed.
If you host Airflow in Google Cloud, you can use a managed solution with Cloud Composer 2 (GKE autopilot).
Cloud Workflow is a good solution if you stay in the GCP world, it's serverless and based on API calls but it is less documented than Airflow for an usage outside of GCP. It's also not Cloud agnostic and strongly coupled to GCP.
I recommend you using Airflow for your need.
Related
Is it possible for the Google Cloud Workflow to reach the Private VPC (perhaps via a serverless VPC connector)? I can't find anything about it in the documentation. We want to use Workflow to trigger certain things via API on the internal network (no outside access).
Worst case we'll have to proxy it through a Cloud Function
Regards,
Niklas
VPC connector is one of the most demanding feature of Cloud Workflow but for now it's not implemented. There is no ETA for this feature.
For now, a proxy is required with Cloud Run/Functions and a VPC connector.
As stated in the first sentences in Workflows doc, Cloud Workflows is meant to:
link series of serverless tasks together
and it
Combine the power of Google Cloud's APIs, serverless products like Cloud Functions and Cloud Run, and calls to external APIs
So, as you proposed, the workaround is to wrap / proxy your call to your internal API, through a call to Cloud Function or Cloud Run with proper authentication / authorisation.
Google Cloud Workflow has an unknown IP, which is difficult to route.
So you're probably looking for Cloud NAT? This would be the console.
I hear Google's Anthos is supporting hybrid cloud, and has features for building applications that are cloud-agnostic.
Does Amazon-AWS support integration with Anthos AMP?
From enterprise standpoint, I prefer not to be locked with AWS, so would be great if AWS integrates with Anthos AMP.
Given that Google Anthos was just announced this month at Google Next '19, details will change over time.
I do not expect that AWS will directly support Google Anthos.
Google has stated that Google will provide a single managed service that supports Google Anthos. I translate the statements by Urs Hölzle (Google Sr VP) as: Google Anthos support will be provided as part of your Google Support agreement no matter where Google Anthos is running - GCP, Azure, AWS, or On-premises.
I need to use Elixir/Phoenix and Google Serverless DBs for a realtime Forex App.
So how to connect from Phoenix/Elixir to Google Cloud Spanner or Google Cloud Bigtable? Any link for Doc or tutorial would be appreciated.
This question was first asked on ElixirForum. and the answer suggested was Elixir on GCP, however, I have gone through all the materials on that link and found nothing.
Disclosure: I am a product manager for Google Cloud Bigtable.
There's an Elixir client for Cloud Spanner you can take a look at.
There isn't a Google-written and maintained Elixir client for Cloud Bigtable yet, but there's Elixir support for a number of other products in the same repo as well.
However, there's a third-party Elixir client for Cloud Bigtable you can explore; please direct any questions to their GitHub repo.
That said, if you're looking for a true serverless database backend for your app, consider using Google Cloud Firestore, which has also has Elixir support. Cloud Spanner and Cloud Bigtable require provisioning dedicated resources so there will be a minimum charge regardless of usage, while Cloud Firestore provides a pay-as-you-go model that will make it easy to scale from a small app to a large application.
Take a look at the pricing guides for each of these databases for more information:
Cloud Spanner pricing
Cloud Bigtable pricing
Cloud Firestore pricing
I am new to gcloud, I want to create my first gcloud vision and video intelligence project but do not know how to start and what are the pre-requisite for this. I have Windows 10 laptop and also created my goggle cloud account. Can you please guide my on setup work and how to start development work.
Thanks in Advance
Prerequisites: a Google Cloud Platform project with the Video Intelligence API and Vision API both enabled. The APIs are the interfaces between you / your applications and the Cloud services. There are multiple options to access the APIs:
One option is to install the Cloud SDK on your machine and initializing it - linking it with your project.
Afterwards, the Cloud SDK, which
is a set of tools for Cloud Platform. It contains gcloud, gsutil, and bq, which you can use to access Google Compute Engine, Google Cloud Storage, Google BigQuery, and other products and services from the command-line. You can run these tools interactively or in your automated scripts.
has the proper (command-line) tools to access the Google Cloud Vision and the Google Cloud Video Intelligence products.
Another option is to make direct calls to the APIs, i.e. via the REST endpoints. You can read about Cloud Vision APIs here, Video Intelligence APIs here and test them via Google's API Explorer.
I have a MEAN stack application which needs to be cloud hosted. The management needs it to be portable and that brought me to checkout cloud foundry. However, even for cloud foundry there are many provider options like CF on Azure, PCF , IBM Blue Mix and so on. However, I am not able to understand the differences between them. Can you please point me to something that helps me understand the differences between these various providers and make a decision? Also whats the difference between Azure PaaS and Azure Cloud Foundry?
Cloud Foundry is an open source PaaS and because it's open source, you have the freedom to either:
host it yourself on a variety of IaaS
use a public, multi-tenant Cloud Foundry service
have a provider host a private CF for you
This is very similar to hosting options for Kubernetes for example (even though it's worth mentioning that Cloud Foundry predates Kubernetes by a couple of years).
"Pivotal Cloud Foundry" is a commercial distribution of Cloud Foundry targeted at large enterprises. It has a couple of features not found in the open source version, mostly related to deployment automation and integration of application services like MySQL etc. Pivotal is also a main sponsor of development work on the open source version of Cloud Foundry. PCF on Azure is kind of a "template service" that allows you to quickly deploy a private PCF installation on Azure, so it's to some degree a combination of hosting options 1) and 3).
You specifically asked about the difference between various public Cloud Foundry service providers. Here's the most important points:
data center location and related privacy concerns (PWS runs on AWS US locations for example)
choice of managed application services and plans (e.g. MySQL, PostgreSQL etc.)
pricing for apps and application services
performance (available CPU per Diego Cell on which application containers execute, networking)
Cloud Foundry version and supported features like container-to-container networking or deployment of docker containers
quality and availability of support options, onboarding assistance
availability of legal assurances/contracts you may need, e.g. to comply with EU GDPR rules
Also worth reading: Cloud Foundry explained
Cloud Foundry is an OpenSource PaaS that can run on top of any different IaaS. So you can got to https://github.com/cloudfoundry/cf-deployment and use it to install your own instance of Cloud Foundry on Azure, AWS, GCP, vSphere, OpenStack, SoftLayer ... etc.
PCF is a commercial product from Pivotal based on the OpenSource Cloud Foundry. You buy it and then you install and run it on an IaaS of your choice.
BlueMix is a commercial product from IBM which is also based on OpenSource Cloud Foundry. It is also a set of services based on various IBM products so with BlueMix IBM runs and manages the cloud for you.
Azure PaaS is a set of service from Microsoft for deploying applications which only runs an Azure, while Cloud Foundry can be installed on Azure or other IaaS providers.