I am building an application where users can upload their files and go to the payment page. After payment completion, I want to store the files uploaded by the user in the database. If the user does not complete the payment, then I do not want to save the files on the server or database.
Is there any way to do that?
Two things to unpack here.
First, it is not advised to store files in the database. It's better to use a storage or directly the server's filesystem.
Second, files are usually uploaded and saved according to your strategy (on the server's filesystem, the database or a third party storage), and then a cleanup would happen if the user hasn't proceeded with the payment. You need to define the conditions for the cleanup to happen, whether it's because the user has uploaded file and has been inactive for a certain period of time, or because they click on a specific button, or a combination of both.
To trigger the cleanup, you have different possibilities:
When uploading the files, schedule a task, for instance using django-q to check that, say, 1 hour after uploading the files, if the payment hasn't been completed, the files are deleted
Write a django command that gets triggered by a cron job every day that deletes the files of the user whose payment has been pending for more than 1 hour
You could also work with django-sessions and regularly scan for sessions that haven't been active for 1 hour, whose payment are pending and assume these users will not proceed with the payment and delete their uploaded files
Related
I want to schedule the data transfer job between Cloud Storage to BigQuery.
I have one application that dumps data continuously to the GCS bucket path (let's say gs://test-bucket/data1/*.avro) that I want to move to BigQuery as soon as the object is created in GCS.
I don't want to migrate all the files available within the folder again and again. I just want to move only the newly added object after the last run in the folder.
BigQuery data transfer service is available that takes Avro files as input but not a folder and it does not provide only newly added objects instead all.
I am new to it so might be missing some functionality, How can I achieve it?
Please note- I want to schedule a job to load data at a certain
frequency (every 10 or 15 min), I don't want any solution from a
trigger perspective since the number of objects that will be generated
will be huge.
You can use Cloud Function and Storage event trigger. Just launch Cloud Function that loads data into BigQuery when new file arrives.
https://cloud.google.com/functions/docs/calling/storage
EDIT: If you have more than 1500 loads per day you can workaround with loading using BQ Storage API.
If you do not need superb performance then you can just create an external table on that folder and query it instead loading every file.
I have a scenario where i want to list blobs and then sort it using the last modified time.
I am trying to do it in python api.
I want to execute this script n number of times, and in each execution i want to list 10 files and perform some operation (e.g copy). I want to save the date of the last file in a config file and want to list the files in another iteration after the last saved date.
Need some suggestion as google api doesn't let us sort the files after listing.
blobs = storage_client.list_blobs(bucket_name,prefix=prefix,max_results=10)
Several solutions I can think of.
Get pubsub notification every time a file created.Read 10 messages each time or save the topic data to bigquery.
After using a file move it to another folder with a metadata file, or update the processed files metadata.
Use storage to trigger a function and save the event data to database.
If you control the files names and path save them in a easy to query path by using the prefix parameter.
I think the database solution in the must flexible one which give you the best control over the data and the ability to create a dashboard for your data.
Knowing more about your flow will help in order to give you a more fine grained solution.
In the Authorize.net API, when getSettledBatchList returns a settlementState of settlementError, is that the final state for the batch? What should I expect to happen to the batched transactions?
Is the same batch processed again the following day, using the same batch id, possibly resulting in a settlementState of settledSuccessfully? Or are the affected transactions automatically included in a new batch with a new batch id?
If the transactions are included in a new batch, would they then be included in multiple batches? If transactions are included in multiple batches, would getTransactionList for each of these batches return the exact same transactionStatus for transactions that were included in multiple batches, regardless of which batch id was used to make the getTransactionList request?
Question was originally asked at https://community.developer.authorize.net/t5/Integration-and-Testing/What-happens-to-a-batch-having-a-settlementState-of/td-p/58993. If the question is answered there, I'll also add the answer here.
Here's the answer posted in the Authorize.Net community for those who did not follow the link in the question:
Batch status of "settlement error" means that the batch failed. There are different reasons a batch could fail depending on the processor the merchant is using and different causes of failure. A failed batch needs to be reset and this means that the merchant will need to contact Authorize.Net to request for a batch reset. It is important to note that batches over 30 days old cannot be reset. When resetting a batch, merchant needs to confirm first with their MSP (Merchant Service Provider) that the batch was not funded, and the error that failed the batch has been fixed, before submitting a ticket for the batch to be reset.
Resetting a batch doesn't really modify the batch, what it does, is it takes the transactions from the batch and puts them back into unsetttled so they settle with the next batch. Those transactions that were in the failed batch will still have the original submit date.
Authorize.net just sends the batch to your msp, you'll have to contact your msp to have them three way call authorize.net to sort it out.
I am setting up Sitecore xDB and am trying to test exactly what info gets through the system for authenticated and non-authenticated users. I would like to be able to make a change and see the results quickly in Sitecore. I found the setting to lower session lifetime to 1 minute rather than 20. I have not found a way to just force Sitecore to sync with Mongo on demand or at least within 1-5 minutes rather than, what also appears to be about 20 minutes at the moment. Does it exist or is "rebuilding" the database explained here the only existing process?
See this blog post by Martina Welander for this and more good info about xDB sessions: https://mhwelander.net/2016/08/24/whats-in-a-session-what-exactly-happens-during-a-session-and-how-does-the-xdb-know-who-you-are/
You just need a utility page that calls System.Web.HttpContext.Current.Session.Abandon(). You may also want to redirect the user to a page that doesn't exist.
Update to address comment
My understanding is that once an xDB session has expired, processing should take place quickly. In the Sitecore.Analytics.Processing.Services.config file, the BackgroundService agent is set to run on an interval of 15 seconds by default.
You may just be seeing cached reporting data. Try clearing the cache using the /sitecore/admin/cache.aspx page. You could also decrease the defaultCacheExpiration setting for the reporting cacheProvider in the Sitecore.Analytics.Reporting.config file. The default is 10 minutes.
In my application, I have to run a periodical job where I need data about all the photos on my client's Flickr account. Currently, I perform several calls to flickr.photos.search to retrieve meta about all the photos each time.
What I ask is: is there a way to be get informed by Flickr when a photo is modified or deleted, so that I don't need to retrieve metas for each photos, but rather store theme once, and only download what has changed since the last time I run the job ?
Thx in advance for your help.
There is no such notification possible from the Flickr API to your code.
What you can do on the other hand is (recommended only if volume for change of photo metadata is high) -
Setup a cron job which would scan through the photos and store if the photo id's are deleted or not - which can be used later.