I want to subscribe to a Google Cloud pub-sub topic (which will push notifications about user's inbox changes). Since Google requires a HTTPS endpoint I created an AWS API Gateway endpoint. But I'm unable to add this endpoint as a subscriber.
I get an error saying - "Sorry, there’s a problem. If you entered information, check it and try again. Otherwise, the problem might clear up on its own, so check back later.".
Any ideas why this is happening and how I can fix this?
Azeem
Related
I have a template that creates a few resources in GCP, and I want it to either call an HTTP endpoint or publish a message to a topic whenever the deployment completes. I've been checking different services all day, and couldn't find anything about it.
In AWS, it is quite easy to deploy an SNS message that is published to an SNS topic, which is subscribed to SQS Queue, and that triggers a lambda function. In Azure ARM templates, we can use az CLI to invoke a web request and call an endpoint directly.
I couldn't find any similar in GCP. Is there any way of either calling an HTTP endpoint, Cloud Function or perhaps publishing a message to a topic whenever a deployment is finished?
I really appreciate any help.
The best approach in GCP is to Create a Logging Sink using a filter and the Logging query language to only send the Deployment Manager logs to a PubSub topic.
Here is an example of a sink sending deployment manager logs to a PubSub topic previously created:
gcloud logging sinks create $SINK_NAME pubsub.googleapis.com/projects/$YOUR_PROJECT/topics/$TOPIC \
--log-filter='resource.type="deployment"' --description="my sink"
Be careful to Set the destination permissions or will not see the logs in the PubSub topic.
Once you are getting the logs in the PubSub topic, you can configure Cloud Pub/Sub Triggers to fire up an HTTP call based on content of the log.
I am setting up alerting for GCP VMs. It works fine for email, but I'm trying to use the webhook option. It shows the incident was caught and the webhook triggered. But I don't see an alert on the receiving end. I don't know how to debug since GCP webhooks seems like a blackbox. Does anybody know where I can see the log for the actual webhook call? I'm not sure it is receiving an alert ID from the webhook call.
I'm using this document:
https://cloud.google.com/monitoring/alerts/using-channels-api#api-create-channels
Thanks!
Gary
You configure a webhook notification channel and expect to be notified when incidents occur, but you might not receive any notifications because of following reasons:
1.Private endpoint
You can't use webhooks for notifications unless the endpoint is public.
To resolve this situation, use Pub/Sub notifications combined with a pull subscription to that notification topic.
When you configure a Pub/Sub notification channel, incident notifications are sent to a Pub/Sub queue that has Identity and Access Management controls. Any service that can query for, or listen to, a Pub/Sub topic can consume these notifications. For example, applications running on App Engine, Cloud Run, or Compute Engine virtual machines can consume these notifications.
2.Public endpoint
To identify why the delivery failed, examine your Cloud Logging log entries for failure information.
For example, you can search for log entries for the notification channel resource by using the Logs Explorer, with a filter like the following:
resource.type="stackdriver_notification_channel"
NOTE : Also check whether you have been mentioned in the recipient list, if not mentioned you will not be able to see the alert on the receiving end.
Refer Troubleshooting Alerting policies for information.
Can I publish an event into Cloud Pub/Sub outside from GCP?
Let me clarify my query a bit. In AWS as we are able to publish events into SNS topics directly by invoking REST API via API Gateway from the non-cloud client (https://github.com/cdk-patterns/serverless/blob/master/the-big-fan/README.md), is there any such method in GCP to publish an event into Pub/Sub?
I can see there is a similar question in SO (Acces Google Pub/Sub from outside of GCP), but it is not fully answered my question I believe. Yes authentication is required and it is a cross-cutting functionality, but what is the basic technic to publish an event in Pub/Sub outside from GCP
Yes, you can publish a message into a topic, and then pull it, or even have PubSub deliver it to you through a Push subscription.
When publishing a message, or pulling it from a subscriber, you can access PubSub through the REST or RPC API. In addition, you can use one of the client libraries.
Here you can find an example of how to publish a message using the gcloud CLI tool, an example with the REST API, python and java among other programming languages.
As mentioned in the question referenced, you will need to authenticate in order to either publish a message, or pull it. You can use the quickstart as a reference on how to do so. Notice that you can follow the quickstart from any computer or VM outside GCP.
Finally, if you're using Push subscriptions to receive your messages, your endpoint will need to be a publicly accessible HTTPS address and have a valid SSL certificate signed by a certificate authority. Again, this endpoint can live outside GCP.
I am trying to route logs from gcp to pub/sub, but the endpoint I need to send the logs to requires a license key in the header of the JSON body in order to accest the POST request. How can I be able to add a header to the JSON body getting routed with logs? Or how can I add this key to the push endpoint screen?
This answer was provided by #John Hanley in the comment section:
Subscriptions do not add data. Subscription direct how messages are
delivered to subscribers. You will need to add your custom headers in
the data being sent to Pub/Sub messages. For details on what you can
do, read this link.
At this point in time it is not possible to include custom (Authorization) headers in a pub/sub push subscription. (1) You either use an endpoint within GCP to send your pub/sub messages to, which makes use of GCP as an authentication provider. (2) The other option (not recommended at all) is to use an API key in the url and send it to an endpoint of your choice.
(1) You can make use of other GCP services like cloud functions or cloud run to forward pub/sub messages. Create a new request with the appropriate headers and pass on the message to another endpoint. The downside of this approach is that compute costs may spin out of control, since you fire up a cloud function or cloud run instance every time a single message passes through.
You could consider to write a cloud function in Golang to increase the speed. On the other hand, cloud build can handle up to 80 concurrent requests per instance. Someone
(2) Although I still see it very often, using API keys is bad practice. API keys have no expiration, so when they are stolen, someone can access your API indefinitely. That's why Bearer tokens exists with a limited lifetime.
Neither options are perfect, but I would write a small footprint cloud function in Golang to pass on messages to the endpoint of your choice.
I try to get real-time notifications on my slack channel from google calendar and gmail apis.
I already created a Pub-Sub account and I want to add a push subscription on my topic with a slack incoming webhook url but when I try to add it, I got :
The supplied HTTP URL is not registered in the subscription's parent project
I've tried the same thing and it seems that first, you need to verify ownership of your webhook domain.
You would do it here :
https://console.developers.google.com/apis/credentials/domainverification
if you would own the slack domain, which I don't think you do...and I don't either...unfortunately I haven't found a solution to circumvent this.
"You need to verify domain ownership to allow webhook notifications to be sent to your external domains. Google verifies that the user owns each of the listed domains via Search Console."
iv the same problem.
Try with this https://github.com/Spencer-Easton/Apps-Script-Gmail-Push-Notifications
or see : https://cloud.google.com/pubsub/docs/advanced#push_endpoints
The documentation explains, in the article "Cloud Pub/Sub > Documentation > Advanced Configuration", in the section "Register Endpoints", how to register endpoints not on App Engine or not on the same cloud project. You'll need to very that you own the domain and register it.