I'm writing Django app and want to send out tokens using Web3 once Coinpayments sends me callback about successfull payment. The problem is that Coinpayments sends multiple callbacks at once and just in one case tokens are sending, other callbacks get replacement transaction underpriced error. I've already tried to use solutions like add +1 to nonce or remove this parameter, but that doesn't help me because transactions are still building with the same nonce. How can that be fixed or what am I doing wrong?
class CoinpaymentsIPNPaymentView(BaseCoinpaymentsIPNView):
def post(self, request, order_id, *args, **kwargs):
status = int(request.POST.get('status'))
order = Order.objects.get(id=order_id)
order.status = request.POST.get("status_text")
if not status >= 100:
order.save()
return JsonResponse({"status": status})
amount = Decimal(request.POST.get('amount1'))
record = Record.objects.create(
user=order.user,
type='i',
amount=amount,
)
order.record = record
order.save()
gold_record = GoldRecord.objects.get(from_record=record)
contract = w3.eth.contract(address=CONTRACT_ADDRESS, abi=ABI_JSON)
transaction = contract.functions.transfer(order.user.wallet.address, int(gold_record.amount * 10 ** 18)).buildTransaction({
'chainId': 1,
'gas': 70000,
'nonce': w3.eth.getTransactionCount(WALLET_ADDRESS) # address where all tokens are stored
})
signed_tx = w3.eth.account.signTransaction(transaction, WALLET_PRIVATE_KEY) # signing with wallet's above private key
tx_hash = w3.eth.sendRawTransaction(signed_tx.rawTransaction)
print(tx_hash.hex())
tx_receipt = w3.eth.waitForTransactionReceipt(tx_hash)
return JsonResponse({"status": status})
P.S. I've already asked it on Ethereum StackExchange, but nobody answered or commented it: https://ethereum.stackexchange.com/questions/80961/sending-tokens-out-on-coinpayments-success-payment-using-web3py
Ok, let the web know answer and solution that I found out by myself
Each transaction should have unique nonce, so I noticed that if I do a loop for sending transactions and set nonce as w3.eth.getTransactionCount(WALLET_ADDRESS) + index then it sends all transactions without any errors. So I removed instant coins sending (even removed waitForTransactionReceipt to speed up it), and made management command where I process all payouts and if it was sent successfully I assign its tx_hash and run it every 10 minutes with Heroku Scheduler
Related
We used stripe for the payment system.
Imagine a user is doing different things in our system and for each part, he has to pay. We send these payments to Stripe by calling:
stripe.InvoiceItem.create()
Then we create and finalize the invoice for him by calling:
invoice=stripe.Invoice.create()
stripe.Invoice.finalize_invoice(invoice.id)
So if the user has to pay for 3 items:
item1 = 1
item2 = 2
item3 = 3
The finalize_invoice will have an id, total, ...., and:
total = 6
To test if all items are sending the correct amount to Stripe, I'd like to check the total.
In order to test our payment system, I had to mock Stripe, but the Stripe invoice total would always be zero.
I mocked stripe.InvoiceItem.create and stripe.Invoice.finalize_invoice and stripe.Invoice.create like this:
#patch("app_name.models.stripe.InvoiceItem.create")
#patch("app_name.models.stripe.Invoice.finalize_invoice")
#patch("app_name.models.stripe.Invoice.create")
def test_method(
self,
mock_create,
mock_finalize,
mock_invoice_item,
):
response = MagicMock()
# api_key and stripe_account from this link https://stripe.com/docs/api/connected_accounts
response.api_key = "sk_test_MSc53AbTknQXPy"
response.stripe_account = "acct_1032D82eZvKYlo2C" # Stripe account ID
# last version here https://stripe.com/docs/upgrades
response.stripe_version = "2022-08-01"
mock_invoice_item.return_value = response
response = MagicMock()
response.total = 20
response.invoice_pdf = "https://google.com"
response.id = "sk_test_MSc53AbTknQXPy"
mock_create.return_value = response
mock_finalize.return_value = response.id
Stripe might have a mocking feature.
Stripe-mock was not clear to me how to use it in unit tests.
I really don't know how you are mocking the different Stripe functions in order to pinpoint the issue with the the invoice total cost.
If you're thinking of using stripe-mock, I guess the best way to handle unit testing is to do so in an agnostic way (regardless of the stack), by running the stripe-mock Docker as described in the github Readme and create a proxy that will route any API call to the Docker URL instead of the actual API URL (https://api.stripe.com). This will allow you to do unit testing locally on your machine and even with your preferred CI/CD.
With that being said, please bear in mind that there are some known limitations described in the Readme doc.
REST API service has a limit of requests (say a maximum of 100 requests per minute). In Django, I am trying to allow USERs to access such API and retrieve data in real-time to update SQL tables. Therefore there is a problem that if multiple users are trying to access the API, the limit of requests is likely to be exceeded.
Here is a code snippet as an example of how I currently perform requests - each user will add a list of objects he wants to request and run request_engine().start(object_list) to access the API. I use multithreading to speed up requests. I also allow retrying failed API requests via setting a limit on the number of requests for each request object upper_limit.
As I understand there should be some queue for API requests. I anticipate there must be a more elegant solution for this, however, I could not find any similar examples. How can one implement/rewrite this for multiUSER usage with Django?
import requests
from multiprocessing.dummy import Pool as ThreadPool
N=50 # number of threads
upper_limit=1 # limit on the number of requests for a single object
class request_engine():
def __init__(self):
pass
def start(self,objs):
self.objs={obj:{'status':0,'data':None} for obj in objs}
done=False
while not done:
self.parallel_requests()
done=all(_['status']>upper_limit or _['status']==-1 for obj,_ in self.objs.items())
return dict(self.objs)
def single_request(self,request_obj):
URL = f"https://reqres.in/api/users?page={request_obj}"
r = requests.get(url = URL)
if r.ok:
res = r.json()
self.objs[request_obj]['status']=-1
self.objs[request_obj]['data']=res
else:
self.objs[request_obj]['status']+=1
def parallel_requests(self):
objs=[obj for obj,_ in self.objs.items() if _['status']!=-1 and _['status']<=upper_limit]
pool = ThreadPool(N)
pool.map(self.single_request, objs)
pool.close()
pool.join()
objs=[1,2,3,4,5,6,7,7,8,234,124,24,535,6,234,24,4,1,3,4,5,4,3,5,3,1,5,2,3,5,3]
result=request_engine().start(objs)
print([_['status'] for obj,_ in result.items()])
# status corresponds to the number of unsuccessful requests
# status=-1 implies success of the request
Thanks in advance.
I am using gspread and a Service Account Key, Other, json file. to continually update a google spreadsheet with python 2.7. I have this running off a Raspberry Pi running the latest Raspian Jessie. my oauth and gspread should all be the latest versions available for my platform. My script runs for one hour(the max token life span),then stops working with the error message : "Invalid token: Statless token expired error" My code is as follows
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import httplib2
from httplib2 import Http
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name(filename.json,scope)
gc = gspread.authorize(credentials)
wks = gc.open('spreadsheet name')
p1 = wks.worksheet('Printer One')
def functon()
...
p1.append_row(printing)
Any Help would be greatly appreciated, Thank You.
Authorisation expires every 0.5/1 hour (I think it depends on which of the two available methods you use to connect).
I have a google sheet connected 24/7 that updates every 2 seconds. Almost always the reason for a bad read/write is an authorisation error but also Google API can throw a variety of errors at you too that normally resolve after a few seconds. Here's one of my functions to update a cell, but using your details for auth_for_worksheet. Every operation (update single cell, update a range, read a column of values) has some similar construct as a function, which always returns an authorised worksheet. It's probably not the most elegant solution but the sheet has been connected for 3 months fine with no downtime.
def auth_for_worksheet():
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name(filename.json,scope)
gc = gspread.authorize(credentials)
wks = gc.open('spreadsheet name')
p1 = wks.worksheet('Printer One')
return p1
def update_single_cell(worksheet, counter, message):
""" No data to return, update a single cell in column B to reflect this """
single_cell_updated = False
while not single_cell_updated:
try:
cell_location = "B" + str(counter)
worksheet.update_acell(cell_location, message)
single_cell_updated = True
except gspread.exceptions.HTTPError:
logger.critical("Could not update single cell")
time.sleep(10)
worksheet = auth_for_worksheet()
logger.info("Updated single cell")
return worksheet
if __name__ == '__main__':
# your code here, but now to update a single cell
wksheet = update_single_cell(wksheet, x, "NOT FOUND")
I have a non-profitable website that I need to handle newsletter emails to probably thousand people (lets be realistic and give an upper bound of at most 2000 - 2500 registered users).
I have implemented email this way:
#login_required
def SendEmail(request):
receivers = []
users = Users.objects.all()
receivers.append(user.Email for user in users)
emailTypeSelected = request.POST.get('email_type', -1)
email_factory = EmailFactory()
emailManager = email_factory.create_email(emailTypeSelected)
emailManager.prepare("Some Title")
emailManager.send_email_to(receivers)
return render(request, 'new_user_email.html')
And here is the "abstract" class.
class Email(object):
title = ""
plain_message = ""
html_message = ""
def send_email_to(self, receivers):
send_mail(
self.title,
self.plain_message,
SENDER,
receivers,
html_message=self.html_message
)
I have tested this code and it takes a while to send 1 email to 1 user. My concern is that for thousand emails will put a big overhead to the server.
I was thinking to do the following:
Break the users into group of 100 and send email to those users every 30 minutes.
But I am not sure how this can be implemented. Seems that I will need to implement a sort of threads that will be triggered independently and handle the email for me.
Is there any design pattern that you are aware on how to solve this problem?
Now I know that the best way to do this is to use an external service that handle email newsletter and free up my server from doing this but as a non-profitable website I am trying to minimise the expenses as I already have to pay the server expenses. So at the moment I am trying to find a way to implement that in-house unless big problem arises which will force me to go into third-party services.
I am using gspread and a Service Account Key, Other, json file. to continually update a google spreadsheet with python 2.7. I have this running off a Raspberry Pi running the latest Raspian Jessie. my oauth and gspread should all be the latest versions available for my platform. My script runs for one hour(the max token life span),then stops working with the error message : "Invalid token: Statless token expired error" My code is as follows
import gspread
from oauth2client.service_account import ServiceAccountCredentials
import httplib2
from httplib2 import Http
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name(filename.json,scope)
gc = gspread.authorize(credentials)
wks = gc.open('spreadsheet name')
p1 = wks.worksheet('Printer One')
def functon()
...
p1.append_row(printing)
Any Help would be greatly appreciated, Thank You.
Authorisation expires every 0.5/1 hour (I think it depends on which of the two available methods you use to connect).
I have a google sheet connected 24/7 that updates every 2 seconds. Almost always the reason for a bad read/write is an authorisation error but also Google API can throw a variety of errors at you too that normally resolve after a few seconds. Here's one of my functions to update a cell, but using your details for auth_for_worksheet. Every operation (update single cell, update a range, read a column of values) has some similar construct as a function, which always returns an authorised worksheet. It's probably not the most elegant solution but the sheet has been connected for 3 months fine with no downtime.
def auth_for_worksheet():
scope = ['https://spreadsheets.google.com/feeds']
credentials = ServiceAccountCredentials.from_json_keyfile_name(filename.json,scope)
gc = gspread.authorize(credentials)
wks = gc.open('spreadsheet name')
p1 = wks.worksheet('Printer One')
return p1
def update_single_cell(worksheet, counter, message):
""" No data to return, update a single cell in column B to reflect this """
single_cell_updated = False
while not single_cell_updated:
try:
cell_location = "B" + str(counter)
worksheet.update_acell(cell_location, message)
single_cell_updated = True
except gspread.exceptions.HTTPError:
logger.critical("Could not update single cell")
time.sleep(10)
worksheet = auth_for_worksheet()
logger.info("Updated single cell")
return worksheet
if __name__ == '__main__':
# your code here, but now to update a single cell
wksheet = update_single_cell(wksheet, x, "NOT FOUND")