Custom labels coming in text payload - google-cloud-platform

I am trying to use google-cloud/logging-winston Cloud Function.
I want to use prefix & labels but when I configure them based on the google-cloud/logging-winston documentation nothing works.
As it can be seen in the pic. Label is added to the textpayload and prefix is not used at all.
Any idea whats wrong and how to fix this...
/**
* Responds to any HTTP request.
*
* #param {!express:Request} req HTTP request context.
* #param {!express:Response} res HTTP response context.
*/
exports.helloWorld = (req, res) => {
const winston = require('winston');
// Imports the Google Cloud client library for Winston
const {LoggingWinston} = require('#google-cloud/logging-winston');
const loggingWinston = new LoggingWinston({
serviceContext: {
service: 'winston-test',
version: '1'
},
prefix: 'DataInflow'
});
// Create a Winston logger that streams to Stackdriver Logging
// Logs will be written to: "projects/YOUR_PROJECT_ID/logs/winston_log"
const logger = winston.createLogger({
level: 'info',
transports: [
new winston.transports.Console(),
// Add Stackdriver Logging
loggingWinston,
],
});
// Writes some log entries
//logger.debug('Testing labels', { labels: { module: 'Test Winston Logging' }});
logger.error('Testing labels', { custom_metadata: 'yes', labels: { module: 'Test Winston Logging' }});
let message = req.query.message || req.body.message || 'Hello World!';
res.status(200).send(message);
};
Edit#1: If I specify the keyFilename=path-to-sa-json-file then it works.
const loggingWinston = new LoggingWinston({
serviceContext: {
service: 'winston-test',
version: '1'
},
prefix: 'DataInflow',
keyFilename=path-to-sa-json-file.json
});
But this is strange. Cloud Function & logging-winston library should use Application Default Credentials (ADC).
So if the service account I use with my CF has stackdriver write access then it should be fine.
Is my understanding wrong? Anyone having other insights...
Edit#2: Seems to be an issue with Cloud Functions on Nodejs 10. Its working quite fine with Nodejs 8 without specifying the keyFilename... quite strange...

Finally I managed to sort out the issue. All winston logs are stored in stackdriver under custom label so this was the issue in my case.
With the below code I was able to log the error to stackdriver and also to error reporting.
In error reporting I could find my errors under service -> winston-test and in stackdriver under All logs -> winston_log
/**
* Responds to any HTTP request.
*
* #param {!express:Request} req HTTP request context.
* #param {!express:Response} res HTTP response context.
*/
exports.helloWorld = (req, res) => {
const winston = require('winston');
// Imports the Google Cloud client library for Winston
const {LoggingWinston} = require('#google-cloud/logging-winston');
const metadata = {
resource: {
type: 'global',
serviceContext: {
service: 'winston-test',
version: '1'
},
prefix: 'DataInflow'
}
};
const resource = {
// This example targets the "global" resource for simplicity
type: 'global',
};
const loggingWinston = new LoggingWinston({ resource: resource });
// Create a Winston logger that streams to Stackdriver Logging
// Logs will be written to: "projects/YOUR_PROJECT_ID/logs/winston_log"
const logger = winston.createLogger({
level: 'info',
transports: [
new winston.transports.Console(),
// Add Stackdriver Logging
loggingWinston,
],
});
// Writes some log entries
//logger.debug('Testing labels', { labels: { module: 'Test Winston Logging' }});
logger.error(Error('Testing labels'));
let message = req.query.message || req.body.message || 'Hello World!';
res.status(200).send(message);
};

Related

Include Stripe in angular project

I am working on a website which uses angular for client and django for backend and I want to include stripe for payments. To configure stripe on the backend I have a microservice which runs on docker and accepts the following requests:
one for creating a stripe customer
another one for creating a payment
I configured stripe on the client using the provided form as follows:
export class PaymentComponent implements OnInit {
paymentHandler:any = null;
constructor(private checkoutService: CheckoutService) { }
ngOnInit() {
this.invokeStripe();
}
initializePayment(amount: number) {
const paymentHandler = (<any>window).StripeCheckout.configure({
key: 'pk_test_51MKU1wDo0NxQ0glB5HRAxUsR9MsY24POw3YHwIXnoMyFRyJ3cAV6FaErUeuEiWkGuWgAOoB3ILWXTgHA1CE9LTFr00WOT5U5vJ',
locale: 'auto',
token: function (stripeToken: any) {
console.log(stripeToken);
alert('Stripe token generated!');
paymentStripe(stripeToken);
}
});
const paymentStripe = (stripeTocken: any) => {
this.checkoutService.makePayment(stripeTocken).subscribe((data:any) => {
console.log(data)
})
}
paymentHandler.open({
name: 'Card Details',
description: 'Introduce the information from your card',
amount: amount * 100
});
}
invokeStripe() {
if(!window.document.getElementById('stripe-script')) {
const script = window.document.createElement("script");
script.id = "stripe-script";
script.type = "text/javascript";
script.src = "https://checkout.stripe.com/checkout.js";
script.onload = () => {
this.paymentHandler = (<any>window).StripeCheckout.configure({
key: 'pk_test_51MKU1wDo0NxQ0glB5HRAxUsR9MsY24POw3YHwIXnoMyFRyJ3cAV6FaErUeuEiWkGuWgAOoB3ILWXTgHA1CE9LTFr00WOT5U5vJ',
locale: 'auto',
token: function (stripeToken: any) {
console.log(stripeToken)
alert('Payment has been successfull!');
}
});
}
window.document.body.appendChild(script);
}
}
}
The makePayment method makes a request to the django server sending the stripe tocken.
From the server I need the above requests to the microservice.
My question is why do I need the configurations from the server as long as all I have to do to perform a payment is to make a request to the microservice? And where should I use the tocken.
I have also read about webhooks, but I don't really understand the concept and how to use this in my situation.
Also, do I need to test everything with angular cli? And why?
Thank you in advance!

Serve Swagger docs via Lambda

I'm trying to publish the documentation for an API I've developed by means of swagger-ui. Since the API is hosted on AWS API Gateway, I have developed the below Lambda to handle the /swagger-simpler API endpoint. I've confirmed that I'm successfully retrieving the docs, but when I go to the /swagger-simpler endpoint, I get the error: Uncaught SyntaxError: expected expression, got '<' in swagger-ui-bundle.js. When I pull up swagger-ui-bundle.js, it's the exact same HTML as I get when I pull up the /swagger-simpler endpoint.
What am I doing wrong?
Swagger Lambda:
/** #format */
import 'source-map-support/register'
import express from 'express'
import serverless from 'serverless-http'
import swaggerUi from 'swagger-ui-express'
import { Handler } from 'aws-lambda'
import { APIGatewayClient, GetExportCommand } from '#aws-sdk/client-api-gateway'
const app = express()
const apiGateway = new APIGatewayClient({})
export const handler: Handler = async (event, context) => {
const apiId = event.requestContext.apiId
const stage = event.requestContext.stage
console.debug('From request context', { apiId, stage })
let swaggerJson: swaggerUi.JsonObject
try {
swaggerJson = await getSwaggerJson(apiId, stage)
} catch (e) {
console.error('Failed to retreive Swagger JSON', e)
throw new Error('Failed to retreive Swagger JSON')
}
console.debug('Got Swagger doc object', { swaggerJson })
app.use('/swagger-simpler', swaggerUi.serve, swaggerUi.setup(swaggerJson))
console.debug('here')
const handler = serverless(app)
console.debug('got handler', { handler })
const ret = await handler(event, context)
console.debug('handler returned', { ret })
return ret
}
const getSwaggerJson = async (
restApiId: string,
stageName: string
): Promise<swaggerUi.JsonObject> => {
const params = {
exportType: 'oas30',
restApiId,
stageName,
accepts: 'application/json',
}
const res = await apiGateway.send(new GetExportCommand(params))
console.debug('GetExportCommand successful', { res })
let swaggerJson: string
if (res.body) {
swaggerJson = Buffer.from(res.body).toString()
} else {
throw new Error('Empty response body from GetExportCommand')
}
console.debug('Got Swagger JSON', { swaggerJson })
return JSON.parse(swaggerJson)
}
Use S3 instead of Lambda to host the auto generated Swagger docs from API Gateway. Within S3 you can host a static website with a clicks.
Just make sure this meets your security needs.
It turned out to be a CDK issue. See the article I wrote about it for details, but basically I needed to install swagger-ui-express and express via node_modules rather than having those modules bundled into the generated JS.

Call AES SES sendEmail from a Alexa skill

I have been trying for weeks now to figure out how to invoke a call to sendEmail from an Alexa-hosted node.js skill. It is a very simple skill where the user makes a selection. when the selection is made I want to send a email to myself with contents of the selection. I have been trying to call sendEmail from my index.js, which contains the logic for my skill. I created a IAM role with the proper .json file as indicated on AWS, and was able to run a basic node file from the aws command line interface that sends me a email. What kind of steps will I have to take to get my Alexa skill to send the email? Can I just invoke the lambda function that is already working from my Alexa skill?
I have been trying to put the code below, and code similar to it without nodemailer doing the basic ses send email. I started with the aws ses webpage. I cannot find a single tutorial that actually walks you through step by step of calling this ses send email function in a Alexa skill and I would be so grateful to be pointed in the right direction.
'''
const Alexa = require('ask-sdk-core');
const AWS = require("aws-sdk");
let nodemailer = require("nodemailer");
let aws = require("#aws-sdk/client-ses");
// configure AWS SDK
process.env.AWS_ACCESS_KEY_ID = "xxxx";
process.env.AWS_SECRET_ACCESS_KEY = "xxxx";
const ses = new aws.SES({
apiVersion: "2010-12-01",
region: "us-east-1",
});
// create Nodemailer SES transporter
let transporter = nodemailer.createTransport({
SES: { ses, aws },
});
// send some mail
transporter.sendMail(
{
from: "xxxx#gmail.com",
to: "xxxx#gmail.com",
subject: "Message",
text: "I hope this message gets sent!",
ses: {
// optional extra arguments for SendRawEmail
Tags: [
{
Name: "tag_name",
Value: "tag_value",
},
],
},
},
(err, info) => {
console.log(info.envelope);
console.log(info.messageId);
}
);
'''
edit: thanks for the responses already guys! The only notable error I am getting is there is a problem with the requested skills response. So my Alexa skill is amazon hosted. Do I have to change this to use SES,sendEmail()? In my lambda page on amazon I have a file in a folder called sendEmail() but that gives me a error about the line AWS = require(etc.. in the debugger with the output "errorMessage": "2021-03-14T00:33:00.315Z e19e74c6-4c00-47dc-9872-77c0a602541a Task timed out after 3.00 seconds" the code I have in the lambda function titled sendEmail is actually the below code. the line sendEmail() also gives there is a problem with the requested skills response from the Alexa. I do not see my Alexa skill in my lambda functions. Do I have to add it in? Sorry, I really am a noob for AWS programming. Thank you!
`AWS = require('aws-sdk');
// Set the region
AWS.config.update({region: 'us-east-1'});
// Create sendEmail params
var params = {
Destination: { /* required */
ToAddresses: [
'x#gmail.com',
/* more items */
]
},
Message: { /* required */
Body: { /* required */
Html: {
Charset: "UTF-8",
Data: "HTML_FORMAT_BODY"
},
Text: {
Charset: "UTF-8",
Data: "TEXT_FORMAT_BODY"
}
},
Subject: {
Charset: 'UTF-8',
Data: 'Test email'
}
},
Source: 'x#gmail.com', /* required */
ReplyToAddresses: [
'pamphl3t#gmail.com',
/* more items */
],
};
// Create the promise and SES service object
var sendPromise = new AWS.SES({apiVersion: '2010-12-01'}).sendEmail(params).promise();
// Handle promise's fulfilled/rejected states
sendPromise.then(
function(data) {
console.log(data.MessageId);
}).catch(
function(err) {
console.error(err, err.stack);
});`
I believe this part of the documentation is what you're looking for. after you put your custom skill on Lambda function you need to send the content of the body to your email:
const querystring = require('querystring');
let post_data = null;
exports.handler = function (event, context) {
post_data = querystring.stringify(event.body);
}
// your emailing code here
Check this repo too
Where are you defining process.env? If you're running in Alexa Hosted, you'll get environment variables for the Alexa-owned IAM role which will have no access to SES.
Have you tried hardcoding the key and secret from your personal AWS account? If that solves it, look at adding the dotenv package in your package.json, and follow its documentation to set the values in a .env file you can add to your .gitignore.
const AWS = require( 'aws-sdk' );
var SES = new AWS.SES( { apiVersion: '2010-12-01', region: 'us-east-1' } );
//when we are sending our email
//we have to return a promise
//this is because the sendEmail funx
//from SES
//is an ASYNCHRONOUS CALL to the simple email server
//we have to wait for this call to complete before RET
//this is the vital line that resolves my issue
if (typeof Promise === 'undefined' ) {
AWS.config.setPromisesDependency( require( 'bluebird' ) );
}
sendEmail("SOME TEXT");
function sendEmail ( text,event, context, callback ){
var link = "";
var params = {
Destination: {
ToAddresses: [ "address#gmail.com" ]
},
Message: {
Body: {
Text: { Data: link
}
},
Subject: { Data: " text + " has arrived!" }
},
Source: "verifiedaddress#gmail.com"
};
return SES.sendEmail( params ).promise();
}

AWS Cognito services APIs? Amplify => Javascript SDK Angular app

I used Amplify for authentication and it seems to work fine. Now I want to setup an admin app for CRUD with the user pool. It seems that I have to leave Amplify and use the JavaScript SDK to use the appropriate api's.
How does this work? I've failed at figuring out how to get the tokens I receive in Amplify into AWS.config or wherever they are supposed to go.
What a struggle this had been. It seems that the code in the docs is dated and what little advice is online is worse. I suspect that is because an Amplify object contains the config options and I have to bring those to the AWS.config object. I've tried below and failed. Any idea what I need to do? I'm sure answers here will be useful for many AWS newbies.
I have this code in my Angular app but thinking about Lambda. I have an EC2 server with Node.js as another option.
This is for dev on my MBP but I'm integrating it with AWS.
With the code below I get an error message that contains in part:
Error in getCognitoUsers: Error: Missing credentials in config
at credError (config.js:345)
at getStaticCredentials (config.js:366)
at Config.getCredentials (config.js:375)
The JWT I inserted into the object below is the AccessKeyID that is in my browser storage and I used for authentication.
In console.log cognitoidentityserviceprovider I have this object, in part:
config: Config
apiVersion: "2016-04-18"
credentialProvider: null
credentials: "eyJraWQiOiJwaUdRSnc4TWtVSlR...
endpoint: "cognito-idp.us-west-2.amazonaws.com"
region: "us-west-2"
endpoint: Endpoint
host: "cognito-idp.us-west-2.amazonaws.com"
hostname: "cognito-idp.us-west-2.amazonaws.com"
href: "https://cognito-idp.us-west-2.amazonaws.com/"
These functions flow down as a sequence. I left some vars in the bodies in case someone wants to know how to get this data from the user object. I used them in various attempts build objects but most aren't needed here, maybe. The all produce the correct results from the Amplify user object.
import { AmplifyService } from 'aws-amplify-angular';
import Amplify, { Auth } from 'aws-amplify';
import { CognitoIdentityServiceProvider } from 'aws-sdk';
import * as AWS from 'aws-sdk';
#Injectable()
export class CognitoApisService {
private cognitoConfig = Amplify.Auth.configure(); // Data from main.ts
private cognitoIdPoolID = this.cognitoConfig.identityPoolId;
private cognitoUserPoolClient = this.cognitoConfig.userPoolWebClientId;
private cognitoIdPoolRegion = this.cognitoConfig.region;
private cognitoUserPoolID = this.cognitoConfig.userPoolId;
...
constructor(
private amplifyService: AmplifyService,
) { }
public getAccessToken() {
return this.amplifyService
.auth() // Calls class that includes currentAuthenticaedUser.
.currentAuthenticatedUser() // Sets up a promise and gets user session info.
.then(user => {
console.log('user: ', user);
this.accessKeyId = user.signInUserSession.accessToken.jwtToken;
this.buildAWSConfig();
return true;
})
.catch(err => {
console.log('getAccessToken err: ', err);
});
}
public buildAWSConfig() {
// Constructor for the global config.
this.AWSconfig = new AWS.Config({
apiVersion: '2016-04-18',
credentials: this.accessKeyId,
region: this.cognitoIdPoolRegion
});
this.cognitoidentityserviceprovider = new AWS.CognitoIdentityServiceProvider(this.AWSconfig);
/* This doesn't get creds, probably because of Amplify.
this.cognitoidentityserviceprovider.config.getCredentials(function(err) {
if (err) console.log('No creds: ', err); // Error: Missing credentials
else console.log("Access Key:", AWS.config.credentials.accessKeyId);
});
*/
console.log('cognitoidentityserviceprovider: ', this.cognitoidentityserviceprovider);
this.getCognitoUsers();
}
public getCognitoUsers() {
// Used for listUsers() below.
const params = {
UserPoolId: this.cognitoUserPoolID,
AttributesToGet: [
'username',
'given_name',
'family_name',
],
Filter: '',
Limit: 10,
PaginationToken: '',
};
this.cognitoidentityserviceprovider.listUsers(params, function (err, data) {
if
(err) console.log('Error in getCognitoUsers: ', err); // an error occurred
else
console.log('all users in service: ', data);
});
}
The one problem was that the credentials needs the whole user object from Amplify, not just the access token as I show above. By the way, I have the Cognito settings in main.ts. They can also go in environment.ts. A better security option is to migrate this to the server side. Not sure how to do that yet.
// Constructor for the global config.
this.AWSconfig = new AWS.Config({
apiVersion: '2016-04-18',
credentials: this.accessKeyId, // Won't work.
region: this.cognitoIdPoolRegion
});
My complete code is simpler and now an observable. Notice another major issue I had to figure out. Import the AWS object from Amplify, not the SDK. See below.
Yeah, this goes against current docs and tutorials. If you want more background on how this has recently changed, even while I was working on it, see the bottom of this Github issue. Amplify is mostly for authentication and the JavaScript SDK is for the service APIs.
import { AmplifyService } from 'aws-amplify-angular';
// Import the config object from main.ts but must match Cognito config in AWS console.
import Amplify, { Auth } from 'aws-amplify';
import { AWS } from '#aws-amplify/core';
import { CognitoIdentityServiceProvider } from 'aws-sdk';
// import * as AWS from 'aws-sdk'; // Don't do this.
#Injectable()
export class CognitoApisService {
private cognitoConfig = Amplify.Auth.configure(); // Data from main.ts
private cognitoIdPoolRegion = this.cognitoConfig.region;
private cognitoUserPoolID = this.cognitoConfig.userPoolId;
private cognitoGroup;
private AWSconfig;
// Used in listUsers() below.
private params = {
AttributesToGet: [
'given_name',
'family_name',
'locale',
'email',
'phone_number'
],
// Filter: '',
UserPoolId: this.cognitoUserPoolID
};
constructor(
private amplifyService: AmplifyService,
) { }
public getCognitoUsers() {
const getUsers$ = new Observable(observer => {
Auth
.currentCredentials()
.then(user => {
// Constructor for the global config.
this.AWSconfig = new AWS.Config({
apiVersion: '2016-04-18',
credentials: user, // The whole user object goes in the config.credentials field! Key issue.
region: this.cognitoIdPoolRegion
});
const cognitoidentityserviceprovider = new CognitoIdentityServiceProvider(this.AWSconfig);
cognitoidentityserviceprovider.listUsers(this.params, function (err, userData) {
if (err) {
console.log('Error in getCognitoUsers: ', err);
} else {
observer.next(userData);
}
});
});
});
return getUsers$;
}
Let's call this service from a component. I'm putting the JS object parsing in the component but for now, I left the console.log here for you to get started and see if the code works for your app.
// Called from button on html component.
public getAllCognitoUsers() {
this.cognitoApisService.getCognitoUsers()
.subscribe(userData => {
console.log('data in cognito component: ', userData);
})
}

Setting Up Apollo Server with subscriptions-transport-ws?

It seems like I have my server set up according to the Apollo docs at http://dev.apollodata.com/tools/apollo-server/setup.html. In my server/main.js file:
//SET UP APOLLO INCLUDING APOLLO PUBSUB
const executableSchema = makeExecutableSchema({
typeDefs: Schema,
resolvers: Resolvers,
connectors: Connectors,
logger: console,
});
const GRAPHQL_PORT = 8080;
const graphQLServer = express();
// `context` must be an object and can't be undefined when using connectors
graphQLServer.use('/graphql', bodyParser.json(), apolloExpress({
schema: executableSchema,
context: {}, //at least(!) an empty object
}));
graphQLServer.use('/graphiql', graphiqlExpress({
endpointURL: '/graphql',
}));
graphQLServer.listen(GRAPHQL_PORT, () => console.log(
`GraphQL Server is now running on http://localhost:${GRAPHQL_PORT}/graphql`
));
//SET UP APOLLO INCLUDING APOLLO PUBSUB
It prints out "GraphQL Server is now running on http://localhost:8080/graphql" to the terminal log indicating that the server was successfully initialized.
But at the top of my main_layout component, when I run this code:
import { Client } from 'subscriptions-transport-ws';
const wsClient = new Client('ws://localhost:8080');
...I get this console message:
WebSocket connection to 'ws://localhost:8080/' failed: Connection closed before receiving a handshake response
What am I missing?
You need to create a dedicated websocket server. It will run on a different port and the code to set it up is provided on the subscriptions-transport-ws package.
Take a look on the following code from GitHunt-API example:
https://github.com/apollostack/GitHunt-API/blob/master/api/index.js#L101-L134
Also you would see that this code is dependent on a class called SubscriptionManager. It is a class from a package called graphql-subscriptions also by the apollo team, and you can find an example of how to use it here:
https://github.com/apollostack/GitHunt-API/blob/master/api/subscriptions.js
TL;DR: You can use graphql-up to quickly get a GraphQL server with subscriptions support up and ready. Here's a more detailed tutorial on using this in combination with Apollo and the websocket client subscriptions-transport-ws.
Obtain a GraphQL Server with one click
Let's say you want to build a Twitter clone based on this GraphQL Schema in IDL syntax:
type Tweet {
id: ID!
title: String!
author: User! #relation(name: "Tweets")
}
type User {
id: ID!
name: String!
tweets: [Tweet!]! #relation(name: "Tweets")
}
Click this button to receive your own GraphQL API and then open the Playground, where you can add some tweets, query all tweets and also test out subscriptions.
Simple to use API
First, let's create a user that will be the author for all coming tweets. Run this mutation in the Playground:
mutation createUser {
createUser(name: "Tweety") {
id # copy this id for future mutations!
}
}
Here's how you query all tweets and their authors stored at your GraphQL server:
query allTweets {
allTweets {
id
title
createdAt
author {
id
name
}
}
}
Subscription support using websockets
Let's now subscribe to new tweets from "Tweety". This is the syntax:
subscription createdTweets {
Message(filter: {
mutation_in: [CREATED]
node: {
author: {
name: "Tweety"
}
}
}) {
node {
id
text
createdAt
sentBy {
id
name
}
}
}
}
Now create a new tab in the Playground and create a new Tweet:
mutation createTweet {
createTweet(
title: "#GraphQL Subscriptions are awesome!"
authorId: "<id-from-above>"
) {
id
}
}
You should see a new event popping up in your other tab where you subscribed before.
Here is a demo about using Apollo GraphQL, React & Hapi: https://github.com/evolastech/todo-react. It's less overwhelmed than GitHunt-React & GitHunt-API
Seems like you aren't actually making the websocket server. use SubscriptionServer. Keep in mind that it is absolutely NOT true that you have to have a dedicated websocket port (I thought this once too) as davidyaha says. I have both my normal queries and subs on the same port.
import { createServer } from 'http';
import { SubscriptionServer } from 'subscriptions-transport-ws';
import { execute, subscribe } from 'graphql';
import { schema } from './my-schema';
// All your graphQLServer.use() etc setup goes here, MINUS the graphQLServer.listen(),
// you'll do that with websocketServer:
// Create WebSocket listener server
const websocketServer = createServer(graphQLServer);
// Bind it to port and start listening
websocketServer.listen(3000, () => console.log(
`Server is now running on http://localhost:3000`
));
const subscriptionServer = SubscriptionServer.create(
{
schema,
execute,
subscribe,
},
{
server: websocketServer,
path: '/subscriptions',
},
);