I am trying to make the primary key of my dynamodb table something like user_uuid. The user is being created in AWS Cognito and I can't seem to find a uuid like field as part of the CognitoUser class. I am trying to avoid using the username as the pk.
Can someone guide me to the right solution? I can't seem to find anything on the internet regarding a user_uuid field and for some reason I can't even find the documentation of CognitoUser class that is imported from "amazon-cognito-identity-js";
Depends if you plan to use email or phone as a 'username'. In that case, I would use the sub because it never changes. But, the sub is not k-sortable so that requires the use of an extra DB item and index/join to make users sortable by date added. If you plan to generate your GUID/KSUID, and only use email/phone as an alias, then I would use the 'username' as a common id between your DB and userpool.
Good luck with your project!
FWIW - the KSUID generators found in wild are massively overbuilt. 3000+ lines of code and 80+ dependencies. I made my own k-sortable and prefixed pseudo-random ID gen for Cognito users. Here's the code.
export function idGen(prefix: any) {
const validPrefix = [
'prefix1',
'prefix2'
];
//check if prefix argument is supplied
if (!prefix) {
return 'error! must supply prefix';
}
//check if value is a valid type
else if (validPrefix.indexOf(prefix) == -1) {
return 'error! prefix value supplied must be: ' + validPrefix;
} else {
// generate epoch time in seconds
const epoch = Math.round(Date.now() / 1000);
// convert epoch time to 6 character base36 string
const time = epoch.toString(36);
// generate 20 character base36 pseudo random string
const random =
Math.random().toString(36).substring(2, 12) +
Math.random().toString(36).substring(2, 12);
// combine prefix, strings, insert : divider and return id
return prefix + ':' + time + random;
}
}
Cognito user unique identifiers can be saved to a database using a combination of the "sub" value and the username, please refer to this question for a more lengthy discussion.
In the description of amazon-cognito-identity-js (found here, use case 5), they show how to get the userAttributes of a CognitoUser. One of the attributes is the sub value, which you can get at for example like this:
user.getUserAttributes(function(err, attributes) {
if (err) {
// Handle error
} else {
// Do something with attributes
const sub = attributes.find(obj => obj.Name === 'sub').Value;
}
});
I couldn't find any documentation on the available user attributes either, I recommend using the debugger to look at the user attributes returned from the function.
Related
I'm trying to use a pre-request script to build out a request object based on data pulled from a CSV file. The problem is that the request seems to be set in stone prior to the pre-request script being run. That would seem to make this a mid-request script almost rather than a pre-request.
My code is as follows:
if(ipList === undefined) ipList = "1.2.3.4,2.3.4.5,123.234.345.465";
let ips = ipList.split(',');
let queryArray = [];
for( i=0; i<ips.length; i++){
queryArray.push({ "key": "ip", "value": ips[i] });
}
console.log(queryArray);
pm.request.url.query = queryArray;
console.log(pm.request);
When I hardcode a url query variable in the request to equal 4.3.2.1, the pm.response.url object like this:
pm.request.url.query[0] = {key:"ip", value:"4.3.2.1"}
Note that the url.query[0] part of the object matches the parameter in the actual get request.
When I change the value of pm.request.url.query to equal the new query array, however as you can see here, the query array is set correctly, but the parameters are not appended to the request URL.
So unless I'm doing something wrong, it appears that the request is immutable even to the pre-request scripts.
So my question is this:
Is there a way to modify the url params of a request prior to making the request?
BTW: I know that is might seem odd to have multiple params with the same key in a query, but that's the way this API works and hard coding multiple ip addresses in the query works just fine.
You could just assign a new value to pm.request.url.
Here I had some query params already in the URL, which I had to edit:
const urlSplit = request.url.split('?');
const paramsString = urlSplit[1]; // the second part actually represents the query string we need to modify
const eachParamArray = paramsString.split('&');
let params = {};
eachParamArray.forEach((param) => {
const key = param.split('=')[0];
const value = param.split('=')[1];
Object.assign(params, {[key]: value});
});
params.bla = params.bla + 'foobar';
newQueryString = Object.keys(params).map(key => key + '=' + params[key]).join('&');
pm.request.url = urlSplit[0] + '?' + newQueryString;
In the end, I just constructed a new URL, using the first part of the previous one & the query string with the edited bla parameter.
This seemed to work for me--it didn't change what the UI shows the query string is, but it changed what the actual request was (looking at the console log)
pm.request.url.addQueryParams(["a=1", "b=2"])
pm.request.url.query.remove("b")
I have some parameters called "script_loginAs" etc... named such that people on my team know the parameter is evaluated and not sent.
I am working on a react app with react-apollo
calling data through graphql when I check in browser network tab response it shows all elements of the array different
but what I get or console.log() in my app then all elements of array same as the first element.
I don't know how to fix please help
The reason this happens is because the items in your array get "normalized" to the same values in the Apollo cache. AKA, they look the same to Apollo. This usually happens because they share the same Symbol(id).
If you print out your Apollo response object, you'll notice that each of the objects have Symbol(id) which is used by Apollo cache. Your array items probably have the same Symbol(id) which causes them to repeat. Why does this happen?
By default, Apollo cache runs this function for normalization.
export function defaultDataIdFromObject(result: any): string | null {
if (result.__typename) {
if (result.id !== undefined) {
return `${result.__typename}:${result.id}`;
}
if (result._id !== undefined) {
return `${result.__typename}:${result._id}`;
}
}
return null;
}
Your array item properties cause multiple items to return the same data id. In my case, multiple items had _id = null which caused all of these items to be repeated. When this function returns null the docs say
InMemoryCache will fall back to the path to the object in the query,
such as ROOT_QUERY.allPeople.0 for the first record returned on the
allPeople root query.
This is the behavior we actually want when our array items don't work well with defaultDataIdFromObject.
Therefore the solution is to manually configure these unique identifiers with the dataIdFromObject option passed to the InMemoryCache constructor within your ApolloClient. The following worked for me as all my objects use _id and had __typename.
const client = new ApolloClient({
link: authLink.concat(httpLink),
cache: new InMemoryCache({
dataIdFromObject: o => (o._id ? `${o.__typename}:${o._id}`: null),
})
});
Put this in your App.js
cache: new InMemoryCache({
dataIdFromObject: o => o.id ? `${o.__typename}-${o.id}` : `${o.__typename}-${o.cursor}`,
})
I believe the approach in other two answers should be avoided in favor of following approach:
Actually it is quite simple. To understand how it works simply log obj as follows:
dataIdFromObject: (obj) => {
let id = defaultDataIdFromObject(obj);
console.log('defaultDataIdFromObject OBJ ID', obj, id);
}
You will see that id will be null in your logs if you have this problem.
Pay attention to logged 'obj'. It will be printed for every object returned.
These objects are the ones from which Apollo tries to get unique id ( you have to tell to Apollo which field in your object is unique for each object in your array of 'items' returned from GraphQL - the same way you pass unique value for 'key' in React when you use 'map' or other iterations when rendering DOM elements).
From Apollo dox:
By default, InMemoryCache will attempt to use the commonly found
primary keys of id and _id for the unique identifier if they exist
along with __typename on an object.
So look at logged 'obj' used by 'defaultDataIdFromObject ' - if you don't see 'id' or '_id' then you should provide the field in your object that is unique for each object.
I changed example from Apollo dox to cover three cases when you may have provided incorrect identifiers - it is for cases when you have more than one GraphQL types:
dataIdFromObject: (obj) => {
let id = defaultDataIdFromObject(obj);
console.log('defaultDataIdFromObject OBJ ID', obj, id);
if (!id) {
const { __typename: typename } = obj;
switch (typename) {
case 'Blog': {
// if you are using other than 'id' and '_id' - 'blogId' in this case
const undef = `${typename}:${obj.id}`;
const defined = `${typename}:${obj.blogId}`;
console.log('in Blogs -', undef, defined);
return `${typename}:${obj.blogId}`; // return 'blogId' as it is a unique
//identifier. Using any other identifier will lead to above defined problem.
}
case 'Post': {
// if you are using hash key and sort key then hash key is not unique.
// If you do query in DB it will always be the same.
// If you do scan in DB quite often it will be the same value.
// So use both hash key and sort key instead to avoid the problem.
// Using both ensures ID used by Apollo is always unique.
// If for post you are using hashKey of blogID and sortKey of postId
const notUniq = `${typename}:${obj.blogId}`;
const notUniq2 = `${typename}:${obj.postId}`;
const uniq = `${typename}:${obj.blogId}${obj.postId}`;
console.log('in Post -', notUniq, notUniq2, uniq);
return `${typename}:${obj.blogId}${obj.postId}`;
}
case 'Comment': {
// lets assume 'comment's identifier is 'id'
// but you dont use it in your app and do not fetch from GraphQl, that is
// you omitted 'id' in your GraphQL query definition.
const undefnd = `${typename}:${obj.id}`;
console.log('in Comment -', undefnd);
// log result - null
// to fix it simply add 'id' in your GraphQL definition.
return `${typename}:${obj.id}`;
}
default: {
console.log('one falling to default-not good-define this in separate Case', ${typename});
return id;
}
I hope now you see that the approach in other two answers are risky.
YOU ALWAYS HAVE UNIQUE IDENTIFIER. SIMPLY HELP APOLLO BY LETTING KNOW WHICH FIELD IN OBJECT IT IS. If it is not fetched by adding in query definition add it.
An alternative option to the accepted is to instead of dataIdFromObject, which appears to be for everything in the query, I was able to provide a keyFields function per type that required it.
const client = new ApolloClient({
cache: new InMemoryCache({
typePolicies: {
ItemType: {
keyFields: (obj) =>
obj.id + "-" + obj.language.id,
},
},
}),
});
In the above example ItemType can be whichever Type is specified in your schema. I happened to be joining a non-unique ID with a language to make a unique key but you can do it however you wish.
I am new to dynamodb. I want to auto increment id value when I use putitem with dynamodb.
Is possible to do that?
This is anti-pattern in DynamoDB which is build to scale across many partitions/shards/servers. DynamoDB does not support auto-increment primary keys due to scaling limitations and cannot be guaranteed across multiple servers.
Better option is to assemble primary key from multiple indices. Primary key can be up to 2048 bytes. There are few options:
Use UUID as your key - possibly time based UUID which makes it unique, evenly distributed and carries time value
Use randomly generated number or timestamp + random (possibly bit-shifting) like: ts << 12 + random_number
Use another service or DynamoDB itself to generate incremental unique id (requires extra call)
Following code will auto-increment counter in DynamoDB and then you can use it as primary key.
var documentClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: 'sampletable',
Key: { HashKey : 'counters' },
UpdateExpression: 'ADD #a :x',
ExpressionAttributeNames: {'#a' : "counter_field"},
ExpressionAttributeValues: {':x' : 1},
ReturnValues: "UPDATED_NEW" // ensures you get value back
};
documentClient.update(params, function(err, data) {});
// once you get new value, use it as your primary key
My personal favorite is using timestamp + random inspired by Instagram's Sharding ID generation at http://instagram-engineering.tumblr.com/post/10853187575/sharding-ids-at-instagram
Following function will generate id for a specific shard (provided as parameter). This way you can have unique key, which is assembled from timestamp, shard no. and some randomness (0-512).
var CUSTOMEPOCH = 1300000000000; // artificial epoch
function generateRowId(shardId /* range 0-64 for shard/slot */) {
var ts = new Date().getTime() - CUSTOMEPOCH; // limit to recent
var randid = Math.floor(Math.random() * 512);
ts = (ts * 64); // bit-shift << 6
ts = ts + shardId;
return (ts * 512) + randid;
}
var newPrimaryHashKey = "obj_name:" + generateRowId(4);
// output is: "obj_name:8055517407349240"
DynamoDB doesn't provide this out of the box. You can generate something in your application such as UUIDs that "should" be unique enough for most systems.
I noticed you were using Node.js (I removed your tag). Here is a library that provides UUID functionality: node-uuid
Example from README
var uuid = require('node-uuid');
var uuid1 = uuid.v1();
var uuid2 = uuid.v1({node:[0x01,0x23,0x45,0x67,0x89,0xab]});
var uuid3 = uuid.v1({node:[0, 0, 0, 0, 0, 0]})
var uuid4 = uuid.v4();
var uuid5 = uuid.v4();
You probably can use AtomicCounters.
With AtomicCounters, you can use the UpdateItem operation to implement
an atomic counter—a numeric attribute that is incremented,
unconditionally, without interfering with other write requests. (All
write requests are applied in the order in which they were received.)
With an atomic counter, the updates are not idempotent. In other
words, the numeric value increments each time you call UpdateItem.
You might use an atomic counter to track the number of visitors to a
website. In this case, your application would increment a numeric
value, regardless of its current value. If an UpdateItem operation
fails, the application could simply retry the operation. This would
risk updating the counter twice, but you could probably tolerate a
slight overcounting or undercounting of website visitors.
Came across a similar issue, where I required auto-incrementing primary key in my table. We could use some randomization techniques to generate a random key and store it using that. But it won't be in a incremental fashion.
If you require something in incremental fashion, you can use Unix Time as your primary key. Not assuring, that you can get a accurate incrementation(one-by-one), but yes every record you put, it would be in incremental fashion, with respect to the difference in how much time each record in inserted in.
Not a complete solution, if you don't want to read the entire table and get it's last id and then increment it.
Following is the code for inserting a record in DynamoDB using NodeJS:
.
.
const params = {
TableName: RANDOM_TABLE,
Item: {
ip: this.ip,
id: new Date().getTime()
}
}
dynamoDb.put(params, (error, result) => {
console.log(error, result);
});
.
.
If you are using NoSQL Dynamo DB then using Dynamoose, you can easily set default unique id, here is the simple user create example
// User.modal.js
const dynamoose = require("dynamoose");
const { v4: uuidv4 } = require("uuid");
const userSchema = new dynamoose.Schema(
{
id: {
type: String,
hashKey: true,
},
displayName: String,
firstName: String,
lastName: String,
},
{ timestamps: true },
);
const User = dynamoose.model("User", userSchema);
module.exports = User;
// User.controller.js
exports.create = async (req, res) => {
const user = new User({ id: uuidv4(), ...req.body }); // set unique id
const [err, response] = await to(user.save());
if (err) {
return badRes(res, err);
}
return goodRes(res, reponse);
};
Update for 2022 :
I was looking for the same issue and came across following research.
DynamoDB still doesn't support auto-increment of primary keys.
https://aws.amazon.com/blogs/database/simulating-amazon-dynamodb-unique-constraints-using-transactions/
Also the package node-uuid is now deprecated. They recommend we use uuid package instead that creates RFC4122 compliant UUID's.
npm install uuid
import { v4 as uuidv4 } from 'uuid';
uuidv4(); // ⇨ '9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d'
For Java developers, there is the DynamoDBMapper, which is a simple ORM. This supports the DynamoDBAutoGeneratedKey annotation. It doesn't increment a numeric value like a typical "Long id", but rather generates a UUID like other answers here suggest. If you're mapping classes as you would with Hibernate, GORM, etc., this is more natural with less code.
I see no caveats in the docs about scaling issues. And it eliminates the issues with under or over-counting as you have with the auto-incremented numeric values (which the docs do call out).
I debug my API using Xdebug and PHPStorm's debugging features. For this to work, the client needs a cookie named XDEBUG_SESSION.
When using Postman, I used to use a Chrome extension to add this cookie, and Postman's cookie interception feature to get this to work in Postman (since it's a sandboxed app).
However, I cannot create cookies in Paw. So, as a workaround, I modified the API response cookie to have the key as XDEBUG_SESSION and value as PHPSTORM, and debugging worked fine. However, this is not ideal as I would also like to set the expiry date to something far in the future (which I can't in Paw).
So, my questions are:
Is there a way to add custom cookies in Paw?
If not, is there a way to to edit the expiry date for an existing cookie (considering that name, value, domain and path are editable)?
Are there any other alternatives to achieve my objective?
I just managed to achieve this exact thing to debug my APIs with Paw (2.1.1).
You just have to Add a Header with the name Cookie and a value of Cookies picked from the dropdown that will appear. You then have to insert a Cookie named XDEBUG_SESSION with a value of PHPSTORM inside the Cookies value of the header just created.
To be more clear, you can see it in the screenshot below:
I messed around with it to see if I could create an extension. I wasn't able to, and the below does not work but thought I'd share in case anyone knows the missing pieces.
First off, there is no extension category (generator, dynamic value, importer) that this functionality falls into. I tried to make use of the dynamic value category but was unsuccessful:
CookieInjector = function(key, value) {
this.key = "XDEBUG_SESSION";
this.value = "PHPSTORM";
this.evaluate = function () {
var f = function (x,y) {
document.cookie=this.key+"="+this.value;
return true;
}
return f(this.key, this.value);
}
// Title function: takes no params, should return the string to display as
// the Dynamic Value title
this.title = function() {
return "Cookie"
}
// Text function: takes no params, should return the string to display as
// the Dynamic Value text
this.text = function() {
return this.key+"="+this.value;
}
}
// Extension Identifier (as a reverse domain name)
CookieInjector.identifier = "com.luckymarmot.PawExtensions.CookieInjector";
// Extension Name
CookieInjector.title = "Inject Cookie Into Cookie Jar";
// Dynamic Value Inputs
CookieInjector.inputs = [
DynamicValueInput("key", "Key", "String"),
DynamicValueInput("value", "Value", "String")
]
// Register this new Extension
registerDynamicValueClass(CookieInjector);
The main thing stopping this from working is I'm not sure how the request is built in PAW and not sure how to attach the cookie. I've looked through the documentation here: https://luckymarmot.com/paw/doc/Extensions/Reference/Reference, and can't find what I need.
I've created a number of users in Parse. There're Facebook users and non-Facebook user. For each Facebook user I've saved an extra column of "facebookID".
Now I'd like to get all the ParseUsers that are Facebook Users. So I applied a Task to query the users with "facebookID" in them.
var Task = ParseUser.Query.WhereExists("facebookID").FindAsync().ContinueWith(t=>{
if (t.IsFaulted || t.IsCanceled) {
Debug.Log ("t.Exception=" + t.Exception);
//cannot load friendlist
return;
}
else{
fbFriends = t.Result;
foreach(var result in fbFriends){
string id = (string)result["facebookID"];
//facebookUserIDList is a List<string>
facebookUserIDList.Add(id);
}
return;
}
});
The above code works perfectly. However, I'd like to get more data from each Facebook User, for example, the current_coins that the user has. So I change the foreach loop to:
foreach(var result in fbFriends){
string id = (string)result["facebookID"];
int coins = (int)result["current_coins"];
//facebookUserIDList is a List<string>
facebookUserIDList.Add(id);
}
I've changed the facebookUserIDList into a List<Dictionary<string,object>> instead of a List<string> in order to save the other data as well. But the foreach loop does not run at all. Does it mean I can only get the facebookID from it because I specified WhereExists("facebookID") in FindAsync()? Can anybody explain it to me please?
Thank you very much in advance.
it should contain all Parse primitive data type(such as Boolean, Number, String, Date...) for each Object. but not Pointers nor Relation.
for Pointers, you can explicitly include them in the result using the "Include" method
for any ParseRelation you have to requery them