Pysnmp openServerMode with IPv6 - python-2.7

I try to start a SNMP Agent with pysnmp.
for IPv4 and IPv6 bind, it works pretty fine with localhost ('127.0.0.1' and '::1')
but when I try to use other IPv6 IP which I fetched from interface, it failed due to
[vagrant#test SOURCES]$ sudo python snmp_agent.py enp0s8
Traceback (most recent call last):
File "snmp_agent.py", line 172, in <module>
master_agent_startup(ifname=sys.argv[1])
File "snmp_agent.py", line 101, in master_agent_startup
(get_ipv6_address(interface_name), SNMP_AGENT_PORT))
File "/usr/lib/python2.7/site-packages/pysnmp/carrier/asyncore/dgram/base.py", line 50, in openServerMode
raise error.CarrierError('bind() for %s failed: %s' % (iface, sys.exc_info()[1],))
pysnmp.carrier.error.CarrierError: bind() for ('fe80::a00:27ff:fe9e:9c16', 8001) failed: [Errno 22] Invalid argument
This is the output from the interface 'enp0s8':
[vagrant#test SOURCES]$ ifconfig enp0s8
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.20.20.26 netmask 255.255.255.0 broadcast 172.20.20.255
inet6 fe80::a00:27ff:fe9e:9c16 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:9e:9c:16 txqueuelen 1000 (Ethernet)
RX packets 874053 bytes 115842841 (110.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 862314 bytes 114652475 (109.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
This is the code piece I used for IPv6 bind:
def get_ipv6_address(ifname):
return netifaces.ifaddresses(ifname)[netifaces.AF_INET6][0]['addr'].split('%')[0]
config.addSocketTransport(snmpEngine, udp.domainName,
udp.UdpTransport().openServerMode(
(get_ipv4_address(interface_name), SNMP_AGENT_PORT))
)
config.addSocketTransport(snmpEngine, udp6.domainName,
udp6.Udp6SocketTransport().openServerMode(
(get_ipv6_address(interface_name), SNMP_AGENT_PORT))
)
From pysnmp sample, it seems the parameter inside "openServerMode()" is just a tuple of IP and port.
And from output error I suppose there is no error for given IP and port.
so why it failed due to Invalid argument?
Could you #Ilya Etingof or some other pysnmp expert help me with it?
Thanks.
UPDATE: I try to bind it with given suggestion, but is still doesn't work.
the bind command was run from a new installed CentOS. but it still failed:
[root#test ~]# ifconfig
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.10.20.4 netmask 255.255.255.0 broadcast 10.10.20.255
inet6 fe80::f816:3eff:fee1:5475 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:e1:54:75 txqueuelen 1000 (Ethernet)
RX packets 12242 bytes 962552 (939.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12196 bytes 957826 (935.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root#test ~]# python
Python 2.7.5 (default, Oct 11 2015, 17:47:16)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM, 0)
>>> addr_and_port = ('fe80::f816:3eff:fedb:ba4f', 8001)
>>> s.bind(addr_and_port)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 22] Invalid argument
>>>
[1]+ Stopped python
[root#test ~]# netstat -anp | grep 8001
[root#test ~]#
One more update:
I suppose the bind is failed due to my environment has some issue with IPv6 configuration. As I'm only able to get one IPv4 address by using "socket.getaddrinfo()" method.
Br,
-Dapeng Jiao

You could get this error if attempted to bind the same socket more than once. But I can't see that is the case in your code.
That .openServerMode() method does no magic -- it just calls .bind() on socket object. For inspiration, does this work in at your Python prompt?
from pysnmp.carrier.asyncore.dgram import udp6
addr_and_port = ('fe80::a00:27ff:fe9e:9c16', 8001)
udp6.Udp6SocketTransport().openServerMode(addr_and_port)
or even:
import socket
s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM, 0)
addr_and_port = ('fe80::a00:27ff:fe9e:9c16', 8001)
s.bind(addr_and_port)
My hope is that tests like these may help you figuring out the problem...

When you use an IPv6 link-local address, you must always use the scope along with it. The link-local address is not valid without its scope, thus you will receive an Invalid argument error.
For example, instead of using fe80::a00:27ff:fe9e:9c16 you must use fe80::a00:27ff:fe9e:9c16%enp0s8.

Related

python 2.7: AddressValueError: ip does not appear to be an IPv4 or IPv6 network

I am just running the example on: https://docs.python.org/3/howto/ipaddress.html#ipaddress-howto
However, in the following example:
import ipaddress
net4 = ipaddress.ip_network("192.0.2.0/24")
for x in net4.hosts():
print(x)
I got the following error:
AddressValueErrorTraceback (most recent call last)
<ipython-input-18-256ed42a96d9> in <module>()
1 import ipaddress
2
----> 3 net4 = ipaddress.ip_network("192.0.2.0/24")
4 for x in net4.hosts():
5 print(x)
/usr/local/lib/python2.7/dist-packages/ipaddress.pyc in ip_network(address, strict)
197 '%r does not appear to be an IPv4 or IPv6 network. '
198 'Did you pass in a bytes (str in Python 2) instead of'
--> 199 ' a unicode object?' % address)
200
201 raise ValueError('%r does not appear to be an IPv4 or IPv6 network' %
AddressValueError: '192.0.2.0/24' does not appear to be an IPv4 or IPv6 network. Did you pass in a bytes (str in Python 2) instead of a unicode object?
Did I miss anything here (I am using python 2.7)? Thanks!
You can change second line to
net4 = ipaddress.ip_network(unicode("192.0.2.0/24"))
The complete code looks like
import ipaddress
net4 = ipaddress.ip_network(unicode("192.0.2.0/24"))
for x in net4.hosts():
print(x)
I resolved it by inputing a unicode string, similar to another question:
ValueError: '10.0.0.0/24' does not appear to be an IPv4 or IPv6 network

Different processes showed as same PID within netstat

I spawn few processes using the Python multiprocessing module.
However when I call netstat -nptl, each ip:port listeners listed under the same PID.
I'm using Python 2.7 on Ubuntu 14.04.
netstat -V
>> net-tools 1.60
>> netstat 1.42 (2001-04-15)
Relevant code:
import unittest
import multiprocessing
import socket
import os
import time
import ex1
class Listener(multiprocessing.Process):
def __init__(self, _ttl):
super(Listener, self).__init__()
self.ttl = _ttl
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind(('localhost', 0))
def get_pid(self):
return self.pid
def get_name(self):
return self.socket.getsockname()
def run(self):
self.socket.listen(1)
time.sleep(self.ttl)
def listen(self):
self.start()
class TestEx1(unittest.TestCase):
def test_is_legal_ip(self):
# Legal IP
assert(ex1.is_legal_ip("1.1.1.1:55555"))
assert(ex1.is_legal_ip("0.1.1.255:55555"))
assert(ex1.is_legal_ip("0.0.0.0:55555"))
assert(ex1.is_legal_ip("255.255.255.255:55555"))
assert(ex1.is_legal_ip("0.1.2.3:55555"))
# Illegal IP
assert(not ex1.is_legal_ip("256.1.1.1:55555"))
assert(not ex1.is_legal_ip("1.256.1:55555"))
assert(not ex1.is_legal_ip("1.1.1.1.1:55555"))
assert(not ex1.is_legal_ip("1.a.1.1:55555"))
assert(not ex1.is_legal_ip("1.1001.1.1:55555"))
def test_address_to_pid(self):
# Create 3 listener processes
listener1 = Listener(22)
listener2 = Listener(22)
listener3 = Listener(22)
# Start listening
listener1.listen()
listener2.listen()
listener3.listen()
print listener1.get_pid()
print listener2.get_pid()
print listener3.get_pid()
# For each listener, get appropriate ip:port
address1 = str(str(listener1.get_name()[0])) + \
":" + str(listener1.get_name()[1])
address2 = str(str(listener2.get_name()[0])) + \
":" + str(listener2.get_name()[1])
address3 = str(str(listener3.get_name()[0])) + \
":" + str(listener3.get_name()[1])
# Check if address_to_pid() works as expected.
#assert(str(ex1.address_to_pid(address1)) == str(listener1.get_pid()))
#assert(str(ex1.address_to_pid(address2)) == str(listener2.get_pid()))
#assert(str(ex1.address_to_pid(address3)) == str(listener3.get_pid()))
# Waits for the listener processes to finish
listener2.join()
listener2.join()
listener3.join()
if __name__ == "__main__":
unittest.main()
Output:
4193
4194
4195
..
----------------------------------------------------------------------
Ran 2 tests in 22.019s
OK
Netstat -nptl output:
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.1.1:53 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:37529 0.0.0.0:* LISTEN 4192/python
tcp 0 0 127.0.0.1:53402 0.0.0.0:* LISTEN 4192/python
tcp 0 0 127.0.0.1:49214 0.0.0.0:* LISTEN 4192/python
tcp 1 0 192.168.46.136:49475 209.20.75.76:80 CLOSE_WAIT 2968/plugin_host
tcp 70 0 192.168.46.136:60432 91.189.92.7:443 CLOSE_WAIT 3553/unity-scope-ho
tcp6 0 0 ::1:631 :::* LISTEN -
Using my Mac OS 10.9.5 (Python 2.7.3), I could reproduce the same behavior. After several try-and-error, it turned out that it's because the socket objects are shared among the processes. (lsof -p <pid> helps to identify listening sockets.)
When I made following change to Listener class, each process started to listen on its own port number by its own PID.
def get_name(self):
# this method cannot refer to socket object any more
# self.sockname should be initialized as "not-listening" at __init__
return self.sockname
def run(self):
# Instantiate socket at run
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind(('localhost', 0))
self.sockname = self.socket.getsockname()
# Listener knows sockname
print self.sockname
self.socket.listen(1)
time.sleep(self.ttl)
This behavior should be the same as Ubuntu's Python and netstat.
self.sockname remains "not-listening" at original process
To listen on port as independent process, sockets need to be created at run method of a Listener object (New process calls this method after creating copy of the object). However variables updated in this copied object in the new process are not reflected to the original objects in original process.

Rough regex for multiline text with optional group and named captures at the end of each match

I want to capture some ifconfig output (example below) and from that, I want to have interface names, flags, IP address and netmask (if they exist), status (which also may not exist in loopback devices)
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 00:30:18:c6:03:f0
inet 192.168.16.67 netmask 0xffffff00 broadcast 192.168.16.255
inet6 fe80::230:18ff:fec6:3f0%re0 prefixlen 64 scopeid 0x1
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
re1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 00:30:18:c6:03:f1
inet6 fe80::230:18ff:fec6:3f1%re1 prefixlen 64 scopeid 0x2
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
media: Ethernet autoselect (none)
status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
and I have managed take it up to this point;
^(?<interface>(?:[^\s:]+))(?=: flags):\s(?:flags=\d+<(?<flags>(?:[\w,]+))>).*?(?:(?:inet (?<ip_address>(?:\d+\.?){4})|$)? (?:netmask 0x(?<netmask>(?:\w++\.?))|$)).*?(?:status: (?<status>(?:[\w ]++)))
but this only captures normal interfaces not loopback device because of status line. And if you make the status line optional by putting a question mark at end of status line match group, it does not match even if status line exist for some reasons that I can not manage to understand.
A problem waiting for geniuses all over the world to solve it :)
You can try this fix:
^(?<interface>(?:[^\s:]+)):\s+(?:flags=\d+<(?<flags>(?:[\w,]+))>)(?:(?!\b(?:flags=|inet\b|status:)).)*(?:(?:inet\s+(?<ip_address>(?:\d+\.?){4})|$)?\s+(?:netmask\s+0x(?<netmask>(?:\w++\.?))|$))(?:(?!status:|\binet\b|^[^\s:]+:\s+flags=\d+).)*(?:status:\s+(?<status>(?:[\w ]+)))?
Main points of what I changed:
Removed unnecessary look-ahead (?=: flags)
Added (?:(?!status:|^[^\s:]+:\s+flags=\d+).)* instead of last .*? (featuring tempered greedy token that allows us to match only up to what we need)
Added the ? quantifier for the last status capture group.
See demo

Send m-search packets on all network interfaces

I am implementing a code through which i have to get devices connected to all network interfaces on my machine.
For this, i am first getting the ip of all network interfaces and then sending m-search command on them.
After 2.5 seconds port is stopped to listen.
But it is giving me some assertion error.
Code:
class Base(DatagramProtocol):
""" Class to send M-SEARCH message to devices in network and receive datagram
packets from them
"""
SSDP_ADDR = "239.255.255.250"
SSDP_PORT = 1900
MS = "M-SEARCH * HTTP/1.1\r\nHOST: {}:{}\r\nMAN: 'ssdp:discover'\r\nMX: 2\r\nST: ssdp:all\r\n\r\n".format(SSDP_ADDR, SSDP_PORT)
def sendMsearch(self):
""" Sending M-SEARCH message
"""
ports = []
for address in self.addresses:
ports.append(reactor.listenUDP(0, self, interface=address))
for port in ports:
for num in range(4):
port.write(Base.MS, (Base.SSDP_ADDR,Base.SSDP_PORT))
reactor.callLater(2.5, self.stopMsearch, port) # MX + a wait margin
def stopMsearch(self, port):
""" Stop listening on port
"""
port.stopListening()
Error:
Traceback (most recent call last):
File "work\find_devices.py", line 56, in sendMsearch
ports.append(reactor.listenUDP(0, self, interface=address))
File "C:\Python27\lib\site-packages\twisted\internet\posixbase.py", line 374, in listenUDP
p.startListening()
File "C:\Python27\lib\site-packages\twisted\internet\udp.py", line 172, in startListening
self._connectToProtocol()
File "C:\Python27\lib\site-packages\twisted\internet\udp.py", line 210, in _connectToProtocol
self.protocol.makeConnection(self)
File "C:\Python27\lib\site-packages\twisted\internet\protocol.py", line 709, in makeConnection
assert self.transport == None
AssertionError
Please tell what's wrong in this code and how to correct this.
Also on linux machines, if no device is found on network then it doesn't go to stopMsearch() why ?
A protocol can only have one transport. The loop:
for address in self.addresses:
ports.append(reactor.listenUDP(0, self, interface=address))
tries to create multiple UDP transports and associate them all with self - a single protocol instance.
This is what the assertion error is telling you. The protocol's transport must be None (ie, it must not have a transport). But on the second iteration through the loop, it already has a transport.
Try using multiple protocol instances instead.

Printing part of a string in Python

Trying to obtain and print and further use just the local ip address on a pi:
import os
getipaddr = "ip addr show eth0 | grep inet"
ip = "%s" % (os.system(getipaddr))
print ip
returns:
inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0
0
which correctly includes the local address, but:
import os
getipaddr = "ip addr show eth0 | grep inet"
ip = "%s" % (os.system(getipaddr))
print ip[9:22]
returns:
inet 192.168.1.200/24 brd 192.168.1.255 scope global eth0
not my expected subset of characters from character 10 to 21. Maybe ip is not really a string variable. Any help on how to fix greatly appreciated. Thanks!
[note: there are actually 4 spaces in front of inet in the print returns, just don't know how to show it in this forum]
If you look at this question, you will see that the return value of os.system is the return value of the call, not it's output. The output you are seeing on the screen is not coming from python but from the ip call. If you want to capture output use the subprocess module:
from subprocess import check_output
getipaddr = "ip addr show eth0 | grep inet"
ip = check_output(getipaddr, shell=True)
Now, the output will be in ip and you should get your desired results.
As a side note, '%s' % something is an anti-pattern. The clearer way to convert a string is to do str(something).