I have the two following security rules the later checks if the document premiumUitill value in DB is bigger that current time meaning the premium is valid.
the issue here is with the first rule I want to disable array size so it won't pass 50 of length and I am pushing using arrayUninon(data) should I check for the size of resource.datarather than request.resorce.data ? in my testing request.resource.data.arr.size() < 50 works but it doesn't make sense to check the incoming data since the incoming has the payload only is something with the arrayUnion() that makes it work ?
await updateDoc(docRef, {
arr: arrayUnion(payload),
}).catch((error) => {
errorHandeling(error, 'An error has happened', reduxDispatch, SetSnackBarMsg);
});
&& request.resource.data.arr.size() < 50
&& resource.data.premiumUntill > request.time
In Cloud Firestore security rules, resource refers to the existing document in the database, and request.resource refers to the document as it exists in the request (during a write, i.e. a set or update).
From the documentation on data validation:
The resource variable refers to the requested document, and resource.data is a map of all of the fields and values stored in the document. For more information on the resource variable, see the reference documentation.
When writing data, you may want to compare incoming data to existing data. In this case, if your ruleset allows the pending write, the request.resource variable contains the future state of the document. For update operations that only modify a subset of the document fields, the request.resource variable will contain the pending document state after the operation. You can check the field values in request.resource to prevent unwanted or inconsistent data updates:
service cloud.firestore {
match /databases/{database}/documents {
// Make sure all cities have a positive population and
// the name is not changed
match /cities/{city} {
allow update: if request.resource.data.population > 0
&& request.resource.data.name == resource.data.name;
}
}
}
Additionally you may watch this video and also may have a look at this stackoverflow
Related
(Node.js API)
I am trying to do the following:
Generate file path like /uploads/${uuid.v4()}.extension
Write the file.
This is the code:
const path = `/uploads/${uuidv4()}.${extname(fileName)}`;
const file = bucket.file(path);
await new Promise((resolve, reject) =>
data
.pipe(file.createWriteStream({ contentType }))
.once('error', reject)
.once('finish', resolve),
);
It works fine. But bothers me to no end that there is that miniscule probability that same UUID will be generated. It is not a practical concern.
How can I upload data to Cloud Storage but get an error if there's a clash? I can check if the file exists beforehand but there is still a race condition technically...
The chance of a collision is not just miniscule: it's astronomically low for UUIDs of significant size. Putting effort into solving the problem of such a collision is not likely to be worth the effort.
That said, if you still want to, you won't be able to do it with Cloud Storage APIs alone, since there is no transactional, atomic API to interact with. If you want a "hard" guarantee that there is no collision, you will need to interact with an entirely different Cloud service that does allow you to effectively "lock" some unique string (e.g. a file path) as a flag for all other processes to check so that they don't collide. Since you are working in Google Cloud, you might want to consider using a database (like any SQL database, or Firestore) with atomic transactional operations to "reserve" the path so that only one process can use it (assuming they all correctly observe this reservation and cooperate as such).
Isn't this exactly what preconditions are for?
Copied from the docs: https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-nodejs
const options = {
destination: destFileName,
// Optional:
// Set a generation-match precondition to avoid potential race conditions
// and data corruptions. The request to upload is aborted if the object's
// generation number does not match your precondition. For a destination
// object that does not yet exist, set the ifGenerationMatch precondition to 0
// If the destination object already exists in your bucket, set instead a
// generation-match precondition using its generation number.
preconditionOpts: {ifGenerationMatch: generationMatchPrecondition},
};
await storage.bucket(bucketName).upload(filePath, options);
console.log(`${filePath} uploaded to ${bucketName}`);
I am trying to retrieve an entity immediately after it was saved. When debugging, I insert the entity, and check entities in google cloud console, I see it was created.
Key key = datastore.put(fullEntity)
After that, I continue with getting the entity with
datastore.get(key)
, but nothing is returned. How do I retrieve the saved entity within one request?
I've read this question Missing entities after insertion in Google Cloud DataStore
but I am only saving 1 entity, not tens of thousands like in that question
I am using Java 11 and google datastore (com.google.cloud.datastore. package)*
edit: added code how entity was created
public Key create.... {
// creating the entity inside a method
Transaction txn = this.datastore.newTransaction();
this.datastore = DatastoreOptions.getDefaultInstance().getService();
Builder<IncompleteKey> builder = newBuilder(entitykey);
setLongOrNull(builder, "price", purchase.getPrice());
setTimestampOrNull(builder, "validFrom", of(purchase.getValidFrom()));
setStringOrNull(builder, "invoiceNumber", purchase.getInvoiceNumber());
setBooleanOrNull(builder, "paidByCard", purchase.getPaidByCard());
newPurchase = entityToObject(this.datastore.put(builder.build()));
if (newPurchase != null && purchase.getItems() != null && purchase.getItems().size() > 0) {
for (Item item : purchase.getItems()) {
newPurchase.getItems().add(this.itemDao.save(item, newPurchase));
}
}
txn.commit();
return newPurchase.getKey();
}
after that, I am trying to retrieve the created entity
Key key = create(...);
Entity e = datastore.get(key)
I believe that there are a few issues with your code, but since we are unable to see the logic behind many of your methods, here comes my guess.
First of all, as you can see on the documentation, it's possible to save and retrieve an entity on the same code, so this is not a problem.
It seems like you are using a transaction which is right to perform multiple operations in a single action, but it doesn't seem like you are using it properly. This is because you only instantiate it and close it, but you don't put any operation on it. Furthermore, you are using this.datastore to save to the database, which completely neglects the transaction.
So you either save the object when it has all of its items already added or you create a transaction to save all the entities at once.
And I believe you should use the entityKey in order to fetch the added purchase afterwards, but don't mix it.
Also you are creating the Transaction object from this.datastore before instantiating the latter, but I assume this is a copy-paste error.
Since you're creating a transaction for this operation, the entity put should happen inside the transaction:
txn.put(builder.builder());
Also, the operations inside the loop where you add the purchase.getItems() to the newPurchase object should also be done in the context of the same transaction.
Let me know if this resolves the issue.
Cheers!
My intent sends a webhook as part of slot filling if a parameter is missing from the user query. My webhook then uses logic to estimate the value of the parameter. How can I return this parameter as part of the WebhookResponse object? I am using the C# client library in an ASP.NET Core app.
My code is:
string fulfillmentText;
WebhookRequest request = null;
using (var reader = new StreamReader(Request.Body))
{
request = jsonParser.Parse<WebhookRequest>(reader);
}
//If Parameter-1 has no value
if (request.QueryResult.Fields["Parameter-1].StringValue.Length == 0)
{
fulfillmentText = "I have guessed the value of Parameter-1";
//I apply some logic that is unimportant to this question
//For the sake of simplicity, say I estimate the value of Parameter-1 to be "foobar"
//I want to be able to give the parameter this value like this:
parameter["Parameter-1"] = "foobar"
}
UPDATE
So, I have pretty much got it all working using Prisoner's method. I will retry sid8491's at some point too. My intent is trying to obtain a user's address. I have required custom entities to retrieve the street number, street name, suburb and state.
Without creating any contexts myself, the following context is automatically generated by Dialogflow: projects/telebot-pianomoves-v1/agent/sessions/2b42cbc8-2418-4231-e4c0-bd3a175f2ea8/contexts/1320fe35-4329-4176-b136-9221dfaddd4e_id_dialog_context. I receive this context in my webhook, and can then CHANGE the value of a parameter. Let's assume $Suburb_Entity had no value in the webhook request and my code then returned the above context with the a new value for Suburb_Entity. My code successfully changes the Suburb_Entity from "" to aspendale as can be seen by the webhook response json:
Now the odd thing is, although I changed the Suburb_Entity to an actual value in the outputContext of my webhook response, the actual parameter $Suburb_Entity only changes to the new value of Suburb_Entity from the outputContext on the NEXT detect intent request. So, keeping in mind the fact that I returned the new Suburb_Entity in the outputContext of the webhook response, this is the detect intent response I get - noting that $Suburb_Entity is yet to be changed:
On the next detect intent request, the webhook request parameter Suburb_Entity is set to aspendale and $Suburb_Entity also equals aspendale. The important thing about this, is $Suburb_Entity only changed to the outputContext parameter value on the NEXT detect intent request, of which would have triggered another webhook. $Suburb_Entity did not change during the same detect intent request as when I modified the outputContext parameter Suburb_Entity, but in the next. This leads me to believe that somehow, $Suburb_Entity inherits parameter values from this automatically generated context. The issue here, is that when my webhook responds with the outputContext paramter Suburb_Entity equalling aspendale, $Suburb_Entity does not change to this value until the next request. This means that if all the other parameters have values set, but $Suburb_Entity is yet to have changed value, then allRequiredParamsSet == false. If I return the Suburb_Entity in the outputContext, I want it to immediately change the value of $Suburb_Entity without requiring another detect intent request so that allRequiredParamsSet == true in such a circumstance. I tried setting the default value by doing this (it didn't work):
An alternative of course would be a way for me to force allRequiredParamsSet = true. I save the parameter values from this the context parameter, not the actual response. So I don't need $Suburb_Entity, I just want the intent to think that it has a value.
Cheers
When you use a webhook for "slot filling", the intention is that you return the prompts you want to ask the user for and continue to use the same Intent to handle the responses. You're not expected to create values yourself.
If you want to "fill in" some answers that are used in the static "response" section of the Dialogflow Intent, or if you just want to record the answers so you can use them later, you can set the parameters of a Context. In the response string, you can refer to this value as #context-name.parameter-name.
Update
I don't know the internal mechanics of slot filling, but it doesn't surprise me that setting a value in the internal context for the input parameters doesn't "register" until the next round of handling the Intent.
The webhook for slot filling isn't intended to create values - it is intended to handle values and create prompts for the user to respond to. Intents are generally about processing user inputs and webhooks about handling them.
My workaround suggested that if you want this for output, you use the context for output.
There are multiple steps for get it done:
First give an event in intent
Check your condition in webhook
If your condition is satisfied, invoke the intent by calling the event from webhook which you have defined in step 1
Pass the paylaod (in json format) along with event calling, give parameters in the payload
In the intent, give default value of parameter as #eventName.parameterName
Hope it helps.
As ledger contains a sequence of transaction, How can I query the current state of ledger and Historic data from ledger.
If you interested in history data in context of the chaincode, you could use ChaincodeStubInterface API, e.g.
// GetHistoryForKey returns a history of key values across time.
// For each historic key update, the historic value and associated
// transaction id and timestamp are returned. The timestamp is the
// timestamp provided by the client in the proposal header.
// GetHistoryForKey requires peer configuration
// core.ledger.history.enableHistoryDatabase to be true.
// The query is NOT re-executed during validation phase, phantom reads are
// not detected. That is, other committed transactions may have updated
// the key concurrently, impacting the result set, and this would not be
// detected at validation/commit time. Applications susceptible to this
// should therefore not use GetHistoryForKey as part of transactions that
// update ledger, and should limit use to read-only chaincode operations.
GetHistoryForKey(key string) (HistoryQueryIteratorInterface, error)
which is capable to retrieve entire history for given key, you receiving back iterator, which will provide you insight on historical changes of the the given key, for example:
historyIer, err := stub.GetHistoryForKey(yourKeyIsHere)
if err != nil {
fmt.Println(errMsg)
return shim.Error(errMsg)
}
if historyIer.HasNext() {
modification, err := historyIer.Next()
if err != nil {
fmt.Println(errMsg)
return shim.Error(errMsg)
}
fmt.Println("Returning information about", string(modification.Value))
}
Also note, that you need to make sure history db is enabled, in core.yaml file part of the ledger section:
ledger:
history:
# enableHistoryDatabase - options are true or false
# Indicates if the history of key updates should be stored.
# All history 'index' will be stored in goleveldb, regardless if using
# CouchDB or alternate database for the state.
enableHistoryDatabase: true
You can use QSCC to query the ledger.
It's a system chaincode that is used to query the peer.
It has all kinds of queries such as get block by number, etc.
In my application I have the concept of a Draw, and that Draw has to always be contained within an Order.
A Draw has a set of attributes: background_color, font_size, ...
Quoting the famous REST thesis:
Any information that can be named can be a resource: a document or
image, a temporal service (e.g. "today's weather in Los Angeles"), a
collection of other resources, a non-virtual object (e.g. a person),
and so on.
So, my collection of other resources here would be an Order. An Order is a set of Draws (usually more than thousands). I want to let the User create an Order with several Draws, and here is my first approach:
{
"order": {
"background_color" : "rgb(255,255,255)", "font_size" : 10,
"draws_attributes": [{
"background_color" : "rgb(0,0,0)", "font_size" : 14
}, {
"other_attribute" : "value",
},
]
}
}
A response to this would look like this:
"order": {
"id" : 30,
"draws": [{
"id" : 4
}, {
"id" : 5
},
]
}
}
So the User would know which resources have been created in the DB. However, when there are many draws in the request, since all those draws are inserted in the DB, the response takes a while. Imagine doing 10.000 inserts if an Order has 10.000 draws.
Since I need to give the User the ID of the draws that were just created (by the way, created but not finished, because when the Order is processed we actually build the Draw with some image manipulation libraries), so they can fetch them later, I fail to see how to deal with this in a RESTful way, avoiding to make the HTTP request take a lot time, but at the same time giving the User some kind of Ids for the draws, so they can fetch them later.
How do you deal with this kind of situations?
Accept the request wholesale, queue the processing, return a status URL that represents the state of the request. When the request is finished processing, present a url that represents the results of the request. Then, poll.
POST /submitOrder
301
Location: http://host.com/orderstatus/1234
GET /orderstatus/1234
200
{ status:"PROCESSING", msg: "Request still processing"}
...
GET /orderstaus/1234
200
{ status:"COMPLETED", msg: "Request completed", rel="http://host.com/orderresults/3456" }
Addenda:
Well, there's a few options.
1) They can wait for the result to process and get the IDs when it's done, just like now. The difference with what I suggested is that the state of the network connection is not tied to the success or failure of the transaction.
2) You can pre-assign the order ids before hitting the database, and return those to the caller. But be aware that those resources do not exist yet (and they won't until the processing is completed).
3) Speed up your system to where the timeout is simply not an issue.
I think your exposed granularity is too fine - does the user need to be able to modify each Draw separately? If not, then present a document that represents an Order, and that contains naturally the Draws.
Will you need to query specific Draws from the database based on specific criteria that are unrelated to the Order? If not, then represent all the Draws as a single blob that is part of a row that represents the Order.