Why does Paypal retry IPN when using django-paypal - django

I have been attempting to get the django-paypal app working within my Django project. I'm using the dcramer fork, with Django 1.4. I am also using a Paypal developer account with business and personal accounts, processing transactions through the Paypal sandbox website.
If I have no receiver function connected to the payment_was_successful signal, things seem to work as expected. After a transaction has occured, a new row is created in the paypal_ipn table of the database which has a value of 'VERIFIED' in the response column. The Paypal IPN log reports that there were no retries for this transaction.
When I do have a receiver function connected to the payment_was_successful signal, the paypal_ipn table includes two new rows with created_at timestamps 10-15 seconds apart. They both have a value of 'VERIFIED' in the response column, but the latter of the two is flagged with flag_info indicating something like:
'Duplicate txn_id. (5M907276M1007902B)'
The Paypal business account reports that the IPN was retried 1 time.
I have found possible solutions that mention the use of dispatch_uid when connecting a receiver function to the signal which I am yet to try. My issue is that I have looked through the relevant django-paypal source code and I can't understand why Paypal would retry the IPN when the postback on the first one was verified.
Has anybody else come up against this and found a solution that they understand?
Update:
I have discovered that there was an error in my receiver function code, which would have been raising an exception. Now that I have fixed this, Paypal is no longer retrying the IPN. I'm glad that the problem has gone away, but I still can't figure out why it was happening.
Below is an excerpt of the most recent duplicate records in the database. Note that the first row was created and updated at least 10 seconds before the subsequent one.
created_at updated_at response flag
2013-02-03 07:53:56.628013+00 2013-02-03 07:53:56.628057+00 VERIFIED FALSE
2013-02-03 07:54:07.393795+00 2013-02-03 07:54:07.403008+00 VERIFIED TRUE

I have figured this one out. The short answer is to make sure your receiver functions are working correctly.
When Paypal sends an IPN to the URL you have specified, they are expecting a response with HTTP status code 200. If the response code is anything else, they will retry. Even if you process a callback and get a VERIFIED message, the IPN from Paypal needs a response of 200 OK.
I had a look through my Apache access logs and discovered that when I had errors in my receiver function for the payment_was_successful signal, the initial IPN from paypal received a HTTP status code 500.
The django-paypal package responds with HttpResponse("OKAY") only after everything else has been processed, including the postback to Paypal which was returning "VERIFIED", the PayPalIPN object being saved to the database, and the sending of signals. When things went wrong in my signal receiver function and an unhandled exception was raised, Django was responding with a HTTP status code 500.
When Paypal retried the IPN, the django-paypal package was detecting a duplicate txn_id and sending the payment_was_flagged signal. My receiver function for this signal was error free, so Paypal received the HTTP status code 200 it was expecting and stopped retrying.
Hopefully this helps someone else out there in the future.

Related

Partially Delivered Email Causes Error in CF10

My company just turned on sender validation for the SMTP relay. So in the old days, I could send an email to nobody#company.com and it would not result in any error. Now, that email results in this:
"Error","scheduler-2","10/31/16","09:04:49",,
"com.sun.mail.smtp.SMTPSendFailedException:
250 2.0.0 xxxxxxxx-1 Message accepted for delivery ;
nested exception is: com.sun.mail.smtp.SMTPAddressFailedException:
550 5.1.1 User Unknown on Mail Relay"
Is there anything I can do via JVM arguments or anything else to ignore these errors and consider the email sent? There's a box where we send out 2-3K emails a day and there's about 1K of "undelivered" emails now on a daily basis. :(
The exception SMTPAddressFailedException is a result of a failed authentication on the mail server, so this is not related to the JVM configuration at all. ColdFusion simply builds mail content and adds it to the mail spooler. Due to the async. nature of a spooler, you cannot catch these errors at runtime.
Your only option is to communicate with the actual mail server before using cfmail (and ask if the sender is legit). However, as far as I know there is no built-in function/tag/tool in CF to do so.

Unknown SSL protocol error in connection to graph.facebook.com:8000 FB UnitySDK

I have been trying to post some achievements in game with following code:
FB.API(FB.UserId +"/"+achievementType,
Facebook.HttpMethod.POST,
AchievementCallback,formData);
but the result is sometimes posted on my activity log and sometimes it fails to post with the message
"Unknown SSL protocol error in connection to graph.facebook.com:8000" while sometimes it is sucessful with post ID. I have tested the achievement files from facebook API explorer and also from POSTMAN plugin and it successfully posted it using both of them with post ID. But when I try to post the achievement from Unity3d it fails sometimes and works sometimes. Does anyone know what i am doing wrong while posting using UnitySDK?
Note: I have hosted these files on Github. Example AchievementFileLink
If anyone has suggestion please let me know.
The workaround I found out for this is whenever the above error happens I again send the request in the callback method to post achievement and it gets published on my game activity successfully. This should not be the intended behaviour and is very bad practice but for time being ill take this as an answer.

Boto error repeatedly appearing on Sentry server

I've configured a Sentry (https://github.com/getsentry/sentry) server to log and manage all the uncaught exceptions on my Django application, that uses Boto to send e-mail via Amazon SES. It is working well and sending e-mail alerts whenever something wrong occurs.
The problem is, almost everytime an error happens and Sentry sends me an alert, it also notifies me of another error (an exception on Boto while sending an email), but it doesn't show me what is the e-mail. I've tried all the bits of my code that send e-mail, and they are all ok. My SES config is also fine, since my application is sending e-mails to a lot of different customers daily
The error reported by Sentry is:
boto in _handle_error
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>MessageRejected</Code>
<Message>Email address is not verified.</Message>
</Error>
<RequestId>4a085303-817a-11e4-b846-79819d3408ac</RequestId>
</ErrorResponse>
After this, i usually receive another error, but all it says is
boto in _handle_error
400 Bad Request
But besides all this MessageRejected warnings, when i go to my AWS Dashboard to check on SES, it shows no Rejects or Complaints, and just a really small number of Bounces (smaller than the amount of warnings i receive)
If i could at least see what email is he trying to send, it would help me debug this, but i can't find any clue about what's happening.
Boto config
The Boto section is used to specify options that control the operation
of boto itself. This section defines the following options:
debug: Controls the level of debug messages that will be printed by
the boto library. The following values are defined:
0 - no debug messages are printed
1 - basic debug messages from boto are printed
2 - all boto debugging messages plus request/response messages from httplib
Make sure you restart your Django server for the log level to take effect
The problem was the default email that Django sends to the emails on the ADMINS setting. Everytime we had an exception, it was trying to email root#localhost (the default value for ADMINS), but since this emails does not exists, it was failing to do so.

django-paypal PDT implementation can't receive signals

I am trying to implement django-paypal (dcramers version from git) using PDT with subscriptions.
It works fine (meaning that the return_url is requested and answered), however signals are not triggered (put in models.py). I am using the following signals, connected to two different functions.
from paypal.standard.pdt.signals import pdt_successful, pdt_failed
What I noticed while browsing through my access logs is that I do get a POST request from paypal which is turned down.
"POST /an-obscure-string/pdt/ HTTP/1.0" 401 401 "-" "-"
I tried to modify /paypal/standard/pdt/views.py to accept POST requests but I still get the 401 error. I think this is the reason signals are not triggered.
I am having a bad time with this. Any help would be greatly appreciated.
PS: I am using the sandbox account
EDIT These are my PAYPAL POSTBACK values from conf.py
POSTBACK_ENDPOINT = "https://www.paypal.com/cgi-bin/webscr"
SANDBOX_POSTBACK_ENDPOINT = "https://www.sandbox.paypal.com/cgi-bin/webscr"
You are right, that is the reason why the signal does not get called since signals are sent at the very end of the view if the model's verify succeed.
Your solution is indeed wrong, Paypal will definitely send a GET request with the transaction ID to your app url if you choosed to use PDN.
https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/howto_html_paymentdatatransfer
I really suggest you to check your settings (on paypal account) since you are probably using IPN which uses POST requests.
Another possible cause of this is that after you received the GET request from paypal your app sends the POST data to your app instead of to paypal postback endpoint. (https://github.com/johnboxall/django-paypal/blob/master/standard/pdt/models.py#L47)
I would look at the access logs to see if the POST request comes after a GET request (for the same url), if yes I would check the value of the SANDBOX_POSTBACK_ENDPOINT and POSTBACK_ENDPOINT settings.

django-paypal does not receive ipn signal

I am trying to use django-paypal. I was following what was mentioned in
Jay on Django
Here is what i did...
##in my view.py file
def ask_payment(request):
# What you want the button to do.
paypal_dict = {
"business": settings.PAYPAL_RECEIVER_EMAIL,
"amount": "0.10",
"item_name": "book",
"invoice": "yong138peng",
"notify_url": "http://127.0.0.1:8000/accounts/my-ipn-location/",
"return_url": "http://127.0.0.1:8000/accounts/my-return-location/",
"cancel_return": "http://127.0.0.1:8000/accounts/my-cancel-location/",
}
# Create the instance.
form = PayPalPaymentsForm(initial=paypal_dict)
context = {"PP_form": form}
return render_to_response("paypal/payment.html",{'PP_form':form},context_instance=RequestContext(request))
#csrf_exempt
def payment_status(request,status):
return render_to_response("paypal/payment_status.html",
{'status':status},context_instance=RequestContext(request))
##then in my urls.py file
(r'^askforpayment/$','coltrane.views.ask_payment'),
(r'^my-ipn-location/', include('paypal.standard.ipn.urls')),
(r'^my-return-location/$','coltrane.views.payment_status',{'status':'success'}),
(r'^my-cancel-location/$','coltrane.views.payment_status',{'status':'cancel'}),
##in my models.py
def show_me_the_money(sender, **kwargs):
ipn_obj = sender
print "payment was successful!"
# Undertake some action depending upon `ipn_obj`.
if ipn_obj.custom == "Upgrade all users!": ## what is this for, this is sent by paypal??
Users.objects.update(paid=True)
payment_was_successful.connect(show_me_the_money)
My question are:
According to jay on django, i have to put a #csrf_exempt before paypay.standard.ipn.views.ipn function to avoid django complaining about the #csrf_token problem. I did it but i still facing the same problem. Then i put the #csrf_exempt before my return url view function, in this case the payment_status(request,status), the the csrf_token problem is gone. So I am not sure why it is the case.
what is this statement in the signal handler are for?
"if ipn_obj.custom == "Upgrade all users!": .... "
Is this coming from paypay? What are the possible value besides the "Upgrade all users?"
I manage to do the purchase and complete the whole payment process at sandbox. But now the problem is that the paypal does not post the IPN back to my notify_url which is a localhost. I read from Paypal Sandbox IPN Problem that i cannot use localhost (http://127.0.0.1:8000) to test IPN. Then what are the steps needed to test? I don't understand the solution provided in that post. Can someone teach me how to test IPN without deploying the real production server?
Regarding your third point, as Daniel says in the answer to the question you linked, you need to allow Paypal to POST to your local machine. That means you need to open the port 80 on your router and forward the request to your development machine on port 8000. Go to http://whatismyip.com, get the IP and try to access it in your browser. Unless you have your router configured correctly you'll get nothing.
Once you've got your router set up you'll need to run the django server with:
python manage.py runserver 0.0.0.0:8000
Then you'll be able to access it externally. You can test this by putting your Internet connection's IP into the browser - you should see your Django site. If you don't then Paypal can't 'see' you either and will never be able to post back.
I was stuck on this problem for a very long time! Turns out I had an error in my signals code but the error was never displayed, just appeared as though the signal was not being called. Eventually tracked it down by modifying the code in paypal-django like this:
in paypal.standard.ipn.views.py - 3 lines from the bottom:
try:
ipn_obj.verify(item_check_callable)
except:
import sys, traceback
traceback.print_exc(file=sys.stdout)
Then check the apache error log for any errors.
For #1 -- The only place I needed to put the #csrf_exempt tag was on the view that is called by the return_url. PayPal actually posts a lot of data about the payment to this url as well, for whatever reason.
For #2 -- You can specify a "custom" field in the paypal_dict, which will then be passed back to the notify_url. If you're only selling from one url or endpoint, it will be obvious what the payment is for, in combination with the invoice you have specified. But you might want to provide an additional field here to work with. "Upgrade all users!" is just a random example that the django-paypal documentation has.
For #3 -- I used ngrok, as mentioned in the django-paypal docs now (http://django-paypal.readthedocs.org/en/stable/standard/ipn.html#testing). I found it pretty easy to set up, not knowing anything about the tool before-hand.
Additionally -- one gotcha on the postbacks that nailed me for awhile was the following: I believed that PayPal was not posting to my notify_url, and I was researching answers like this.
Eventually I realized that PayPal was actually posting to my notify_url, but that my local machine was using an old version of SSL which would not complete a handshake when posting back to the PayPal sandbox (to get the VERIFIED/INVALID result). My version was 0.9.8 and they need something 1.0 or above. To make a long story short, the notify_url is a multi-step process and the problem could be something besides PayPal making the initial post to the notify_url.