Sometimes, when submitting a form (pretty much any form on my site that sends me an email), I get the following error:
File "/usr/lib/python2.5/smtplib.py", line 603, in starttls
(resp, reply) = self.docmd("STARTTLS")
File "/usr/lib/python2.5/smtplib.py", line 378, in docmd
return self.getreply()
File "/usr/lib/python2.5/smtplib.py", line 352, in getreply
line = self.file.readline()
File "/usr/lib/python2.5/socket.py", line 381, in readline
data = self._sock.recv(self._rbufsize)
error: (4, 'Interrupted system call')
My code is sending email via gmail. I am also using django contact-form which does the same thing.
The problem doesn't always happen. It seems very random. At one point today it got so bad that it displayed the error every time I submitted a form.
Restarting apache fixes the problem for one submission and then it does it again.
I have checked the RAM and there is plenty available (about 350MB available).
Can someone lead me in the right direction? What does this error mean? What can I do to prevent this.
Thanks.
I would say it got to do with a bad network connection to the smtp server.
Looks like it gets interrupted while trying to read the reply from the server?
As a workaround, you may want to try increasing the socket timeout.
As for fixing this, you may not have a stable connection to GMail's server and there may not be a way around this.
It looks as if the EINTR signal is being thrown before the recv call gets any data back.
The recv call used by smtplib is being interrupted by a signal before any data was read. Per the read(2) manpage, POSIX allows a read() that is interrupted after reading some data to return -1 (with errno set to EINTR) or to return the number of bytes already read.
In Python, EINTR raises an IOError: "[Errno 4] Interrupted system call" (EINTR == 4).
An example of how EINTR is properly handled is subprocess.communicate(). See an excellent post here:
http://znasibov.info/blog/post/inside-python-subprocess-communication.html
However, in Python 2.5, socket.readline() does not properly handle EINTR. See:
http://bugs.python.org/issue1628205: socket.readline() interface doesn't handle EINTR properly
Related
I'm trying to download some decently large files in python 2.7 (between 300 and 700 MB each), and I'm running into the problem of the connection getting reset in the middle of retrieving the files. Specifically, I was using urllib.urlretrieve(url, file_name), and every so often I get socket.error: [Errno 104] Connection reset by peer.
Now, I'm very unfamiliar with how sockets and web protocol works, so I tried the following, not really knowing if it would help:
response = urllib.urlopen(url)
CHUNK_SIZE = 16 * 1024
with open(file_name, 'wb') as f:
for chunk in iter(lambda: response.read(CHUNK_SIZE), ''):
f.write(chunk)
Edit: Guess I should credit the author of this code: https://stackoverflow.com/a/1517728/3002473
It sounds reasonable that we're only downloading a little bit at a time, so it should be "less susceptible" to this Errno 104, but again I know basically nothing about how all of this works so I don't know if this actually makes a difference.
After testing a bit it seems like it works slightly better? But that might just be coincidence. Generally, I'm able to download one, maybe two files before this error gets thrown.
Why am I getting Errno 104, and how can I go about preventing this? Out of curiosity, should I be using urllib2 instead of urllib?
I'm working on a software-defined networking research project, and what I need is to make a simple UDP server that puts a data tag into the destination options field (IPv6) of the UDP packet. I was expecting to either the sendmsg() recvmsg() commands, or by using setsockopt() and getsockopt(). So, Python 2.7 doesn't have sendmsg() or recvmsg(), and while I can get setsockopt() to correctly load a tag into the packet (I see it in Wireshark), the getsockopt() command just returns a zero, even if the header is there.
#Python 2.7 client
#This code does put the dest opts header onto the packet correctly
#dst_header is a packed binary string (construction details irrelevant--
# it appears correctly formatted and parsed in Wireshark)
addr = ("::", 5000, 0, 0)
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_DSTOPTS, dst_header)
s.sendto('This is my message ', addr)
#Python 2.7 server
addr = ("::", 5000, 0, 0)
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_RECVDSTOPTS, 1)
s.bind(addr)
data, remote_address = s.recvfrom(MAX)
header_data = s.getsockopt(socket.IPPROTO_IPV6, socket.IPPROTO_DSTOPTS, 1024)
I also tried this in Python 3.4, which does have sendmsg() and recvmsg(), but I just get an error message of "OSError: [Errno 22]: Invalid argument", even though I'm passing it (apparently) correct types:
s.sendmsg(["This is my message"], (socket.IPPROTO_IPV6, socket.IPV6_DSTOPTS, dst_header), 0, addr) #dst_header is same string as for 2.7 version
It looks like 99% of the usage of sendmsg() and recvmsg() is for passing UNIX file descriptors, which isn't what I want to do. Anybody got any ideas? I thought this would be just a four or five line nothing-special program, but I'm stumped.
OK, I'm going to partially answer my own question here, on the off chance that a search engine will bring somebody here with the same issues as I had.
I got the Python 3.4 code working. The problem was not the header, it was the message body. Specifically, both the message body and the header options value fields must be bytes (or bytearray) objects, stored in an iterable container (here, a list). By passing it ["This is my message"] I was sending in a string, not a bytes object; Python let it go, but the OS couldn't cope with that.
You might say I was "byted" by the changes in the handling of strings in Python 3.X...
We are building a RTB(real time bidding) platform. Using nginx as http server, the bidder is writen in lua, google protocol buffer for serializing data and Zlog for logs. After test runs, we got three error messages in the nginx error log:
"[libprotobuf Error, google/protobuf/wire_format.cc:1053]
String field contains invalid UTF-8 data when parsing a protocol buffer.
Use the 'bytes' type if you intend to send raw bytes."
So we went back to check the source code of protocol buffer, and found that this check is controlled by a macro(-DNDEBUG: it means NOT debug mode?, according to the comment). And -DNDEBUG disables GOOGLE_PROTOBUF_UTF8_VALIDATION(i think?). So, we enabled this macro(-DNDEBUG) in the configuration. However, after testing, we still got the same error message. And then, we changed all the "String" type to "Bytes" typr in XXX.proto. After testing, the same error message showed.
worker process 53574 exited on signal 11(core dumped),then process died.
lua entry thread aborted: runtime error:/home/bilin/rtb/src/lua/shared/log.lua:34: 'short' is not callable"
Hope somebody can help us solving those problems.
Thank you.
We are using Jetty 8.1 as an embedded HTTP server. Under overload conditions the server sometimes starts flooding the log file with these messages:
warn: java.util.concurrent.RejectedExecutionException
warn: Dispatched Failed! SCEP#76107610{l(...)<->r(...),d=false,open=true,ishut=false,oshut=false,rb=false,wb=false,w=true,i=1r}...
The same message is repeated thousands of times, and the amount of logging appears to slow down the whole system. The messages itself are fine, our request handler ist just to slow to process the requests in time. But the huge number of repeated messages makes things actually worse and makes it more difficult for the system to recover from the overload.
So, my question is: is this a normal behaviour, or are we doing something wrong?
Here is how we set up the server:
Server server = new Server();
SelectChannelConnector connector = new SelectChannelConnector();
connector.setAcceptQueueSize( 10 );
server.setConnectors( new Connector[]{ connector } );
server.setThreadPool( new ExecutorThreadPool( 32, 32, 60, TimeUnit.SECONDS,
new ArrayBlockingQueue<Runnable>( 10 )));
The SelectChannelEndPoint is the origin of this log message.
To not see it, just set your named logger of org.eclipse.jetty.io.nio.SelectChannelEndPoint to LEVEL=OFF.
Now as for why you see it, that is more interesting to the developers of Jetty. Can you detail what specific version of Jetty you are using and also what specific JVM you are using?
I have an application which makes use of winsock.
I/O part is handled on an other thread.
And I am using blocking select method for sockets.
But the point is that after 5-6 hours,my application gives 0xC00000FD exception, at the line of select function.
As far as I know, this exception occurs when there is recursion, or very large local variables. But neither of them is the case for me.
So do you have any idea why am I getting this exception?
Or any ideas to discover what actually causes exception?
many thanks
EDIT 2:
Dear All, I am very sorry but since reproducing the case takes long time, I just realized that this has not solved the problem.
Everything seems ok when stack overflow exception occurs at the line of select function.
I mean it is a server socket with a one client connected. So there is 2 socket in rset and 1 in wset. After selecting, I am checking all ready sockets and making required, read,write,accept. Timeout is 250 ms. Do you think can this be the problem? I don't want this function to be blocking so it is not null. But what will be the exact difference if I use {0,0}
An important hint is:
Same code was working without any problem, when client socket wasn't sending any data.
But when I started sending some data from client to server this problem occured.
I am sure that there is no problem with FD_SETs and FD_CLRs, I mean when client was not sending only 1(server) socket was in rset and 1(client) was in wset.
Anyway I had a look a lot of samples, but it seems that there is not a difference.
Please see local variables screenshot below(I have deleted name of executable, since it is a commercial product)
http://img192.imageshack.us/img192/1948/stackoverflow.jpg
And here is the call stack:
ntdll.dll!7c90df3a()
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]
mswsock.dll!71a53c9c()
ntdll.dll!7c90d26c()
mswsock.dll!71a55f9f()
mswsock.dll!71a55974()
ws2_32.dll!71ab314f()
xyz.exe!vm_socket_select(vm_socket * hds=0x04c1fb84, int nhd=1, int masks=7) Line 230 + 0x1b bytes C
xyz.exe!ND::nd_socket::SocketThreadProc() Line 173 + 0x12 bytes C++
xyz.exe!ND::nd_socket::ThreadRoutineStarter(void * u=0x07d63f90) Line 332 C++
xyz.exe!_callthreadstartex() Line 348 + 0x6 bytes C
xyz.exe!_threadstartex(void * ptd=0x011a3ce8) Line 326 + 0x5 bytes C
kernel32.dll!7c80b713()
I am waiting for any advice.
Many thanks
Have you tried stopping your program in a debugger after some time running? Then take a look at the stack it might give you a hint.
Recursion doesn't mean one of your functions call itself endlessly, it can't be more tricky and involve several layers before it comes back where it started.