I am trying to understand how nameko works for basic RPC.
I am looking to define microservices in separate files and being able to run them from command shell. With this structure service2 is not being able to invoke service1's RPC method. What is missing to get this working?
I have the following file structure:
-rwxrwxr-x 1 user user 240 Dec 15 01:49 nameko.sh*
-rw-rw-r-- 1 user user 251 Dec 15 01:46 service1.py
-rw-rw-r-- 1 user user 305 Dec 15 01:47 service2.py
Content of files are:
$ cat nameko.sh
#!/bin/bash
/usr/local/bin/nameko run service1:Microservice1 &
nameko_id=$!
echo 'Microservice 1 PID: ' $nameko_id
/usr/local/bin/nameko run service2:Microservice2 &
nameko_id=$!
echo 'Microservice 2 PID: ' $nameko_id
wait 2> /dev/null
$ cat service1.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice1(object):
name = "microservice1"
#rpc
def hello(self):
print 'Microservice1 hello method invoked'
return True
$ cat service2.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice2(object):
name = "microservice2"
microservice1 = RpcProxy('microservice1')
microservice1.hello()
#rpc
def hello(self):
print 'Microservice2 hello method invoked'
return True
And I am not being able to understand how to invoke hello method in Microservice 1 from Microservice 2:
$ ./nameko.sh
Microservice 1 PID: 14782
Microservice 2 PID: 14783
Traceback (most recent call last):
File "/usr/local/bin/nameko", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/nameko/cli/main.py", line 66, in main
args.main(args)
File "/usr/local/lib/python2.7/dist-packages/nameko/cli/commands.py", line 85, in main
starting services: microservice1
main(args)
File "/usr/local/lib/python2.7/dist-packages/nameko/cli/run.py", line 179, in main
import_service(path)
File "/usr/local/lib/python2.7/dist-packages/nameko/cli/run.py", line 46, in import_service
__import__(module_name)
File "./service2.py", line 5, in <module>
class Microservice2(object):
File "./service2.py", line 11, in Microservice2
microservice1.hello()
AttributeError: 'RpcProxy' object has no attribute 'hello'
Connected to amqp://guest:**#127.0.0.1:5672//
But invoking nameko shell and running rpc method from microservice1 works:
Broker: pyamqp://guest:guest#localhost
>>> n.rpc.microservice1.hello()
True
Added information
Following Matt's answer I have edited service2.py as follows:
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice2(object):
name = "microservice2"
#rpc
def toctoc(self):
print 'Microservice2 called hello method from Microservice1'
m1 = RpcProxy('microservice1')
m1.hello()
return True
Now both services run, But still does not work, when I run a namejo shell and invoke toctoc method from Microservice2:
$ nameko shell
Nameko Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] shell on linux2
Broker: pyamqp://guest:guest#localhost
>>> n.rpc.microservice2.toctoc()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/nameko/rpc.py", line 374, in
__call__
return reply.result()
File "/usr/local/lib/python2.7/dist-packages/nameko/rpc.py", line 332, in
result
raise deserialize(error)
RemoteError: AttributeError 'RpcProxy' object has no attribute 'hello'
>>>
Last working code
With the help provided in the answers, a working version of this would be (in case it clarifies/helps others):
$ cat nameko.sh
#!/bin/bash
/usr/local/bin/nameko run service1:Microservice1 &
nameko_id=$!
echo 'Microservice 1 PID: ' $nameko_id
/usr/local/bin/nameko run service2:Microservice2 &
nameko_id=$!
echo 'Microservice 2 PID: ' $nameko_id
wait 2> /dev/null
$ cat service1.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice1(object):
name = "microservice1"
#rpc
def hello(self):
print 'Microservice1 hello method invoked'
return True
$ cat service2.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice2(object):
name = "microservice2"
microservice1 = RpcProxy('microservice1')
#rpc
def remote_hello(self):
print 'Microservice2 invokes hello method from Microservice1'
self.microservice1.hello()
return True
there's an example of service-to-service rpc in the docs
class ServiceX:
name = "service_x"
# this _declares_ the dependency (and triggers nameko to set things up)
y = RpcProxy("service_y")
# note that the proxy isn't usable until a service worker is
# running, i.e. until you're inside an executing method
# y.foo() <- this doesn't work.
#rpc
def remote_method(self, value):
res = u"{}-x".format(value)
# this _invokes_ the dependency
return self.y.append_identifier(res)
The problem is that service2.py isn't valid. You can't invoke the RPC proxy outside of a service method.
If you want to call a running service from an external script, use the standalone proxy.
Related
How can I C-c C-c or python-execute-file a django custom command? Currently I get this error (using the emacs settings at the bottom):
-*- mode: compilation; default-directory: "~/src/django/" -*-
Comint started at Fri Feb 17 16:45:33
/home/user/.local/share/virtualenvs/django-OfUvXXat/bin/python csv_import.py
/home/user/.local/share/virtualenvs/django-OfUvXXat/bin/python: can't open file 'csv_import.py': [Errno 2] No such file or directory
Inferior Python exited abnormally with code 2 at Fri Feb 17 16:45:33
This works as expected: python manage.py csv_import
~/django/crm/management/commands/csv_import.py:
from django.core.management.base import BaseCommand, CommandError
from crm.models import Organization,Person
class Command(BaseCommand):
def handle(self, *args, **options):
print(Organization.objects.all())
I tried to no avail many settings e.g.
(setq python-shell-interpreter "python"
python-shell-interpreter-args "-i /home/user/src/django/manage.py shell")
(defun django-shell ()
(interactive)
(let ((python-shell-interpreter (read-file-name "Locate manage.py "))
(python-shell-interpreter-args "shell"))
(run-python (python-shell-calculate-command) nil t)))
Environment: lsp,pyright,pipenv
As per last question about invoking remote RPCs, an additional example to the one included in the documentation would be this one:
$ cat nameko.sh
#!/bin/bash
/usr/local/bin/nameko run service1:Microservice1 &
nameko_id=$!
echo 'Microservice 1 PID: ' $nameko_id
/usr/local/bin/nameko run service2:Microservice2 &
nameko_id=$!
echo 'Microservice 2 PID: ' $nameko_id
wait 2> /dev/null
$ cat service1.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice1(object):
name = "microservice1"
#rpc
def hello(self):
print 'Microservice1 hello method invoked'
return True
$ cat service2.py
# -*- coding: utf-8 -*-
from nameko.rpc import rpc, RpcProxy
class Microservice2(object):
name = "microservice2"
microservice1 = RpcProxy('microservice1')
#rpc
def remote_hello(self):
print 'Microservice2 invokes hello method from Microservice1'
self.microservice1.hello()
return True
I am trying to build an architecture where microservices register themselves against a central microservice on startup (basically this central microservice is in charge of displaying a REST API, where each microservice deal with their part of the REST API -this is done to avoid the usage of a reverse proxy and deal with the port numbers-).
In Nameko, how could you launch a remote procedure just when the microservice is registered? As mentioned in the above post's answers, remote RPC invoke can not be done outside a #rpc method.
Actually stand alone proxy as answered here is the way to achive this:
Nameko - invoking RPC method from another service
use the once entrypoint
from nameko.testing.service import once
class Microservice2(object):
name = "microservice2"
microservice1 = RpcProxy('microservice1')
#rpc
def remote_hello(self):
print 'Microservice2 invokes hello method from Microservice1'
self.microservice1.hello()
return True
I want to use proxy command in python using paramiko.
proxy command:
sftp -o "ProxyCommand /usr/bin/nc --proxy proxy.somegateway.com:8080 %h %p"
but when I use this directly, I am able to connect sftp server.
but if I want to use this proxy command in a Python script I get the issue below:
My script:
>>> import paramiko
>>> cmd = '/usr/bin/ssh proxy.somegateway.com:8080'
>>> proxy = paramiko.ProxyCommand(cmd)
>>> proxy
<paramiko.proxy.ProxyCommand object at 0x23b3bd0>
>>> client = paramiko.SSHClient()
>>> client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>>> client.connect(hostname='some.host.com', username='myuser', password='secretpassword', sock=proxy)
No handlers could be found for logger "paramiko.transport"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/paramiko/client.py", line 366, in connect
t.start_client(timeout=timeout)
File "/usr/lib/python2.7/site-packages/paramiko/transport.py", line 510, in start_client
raise e
paramiko.ssh_exception.ProxyCommandFailure: ('/usr/bin/ssh proxy.somegateway.com:8080', 'Broken pipe')
I am getting ProxyCommandFailure issue.
It should be like this:
target_host = 'sftp.foo.com'
target_port = 22
proxy = paramiko.proxy.ProxyCommand(
'/usr/bin/nc --proxy proxy.bar.com:8080 %s %d' \
% (target_host, target_port) )
client.connect(hostname=target_host, port=target_port, password='...', sock=proxy)
# ^^^^^^^^^^
See paramiko's doc for details.
Just do not forget to add hostname and port to the proxy command
target_host = 'foo'
target_port = 22
proxy = paramiko.proxy.ProxyCommand('/usr/bin/nc --proxy proxy.bar.com:8080 %s %d' % (target_hostname, target_port) )
more info here
Trying to read an .ini db file from python to test a connection to a PostGres database
Python Version 2.7.11
In Fedora, I installed with
sudo dnf install python-configparser
Install 1 Package
Total download size: 41 k
Installed size: 144 k
Is this ok [y/N]: y
Downloading Packages:
python-configparser-3.5.0b2-0.2.fc23.noarch.rpm 40 kB/s | 41 kB 00:01
Total 31 kB/s | 41 kB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Installing : python-configparser-3.5.0b2-0.2.fc23.noarch 1/1
Verifying : python-configparser-3.5.0b2-0.2.fc23.noarch 1/1
Installed:
python-configparser.noarch 3.5.0b2-0.2.fc23
in my config.py I do a
import configparser
when I run my script I get 'module' object is not callable
db.ini
[test1]
host=IP address
database=db
port=5432
user=username
password=pswd
[test2]
host=localhost
database=postgres
port=7999
user=abc
password=abcd
config.py
#!/usr/bin/env python
#from configparser import ConfigParser
import configparser
def config(filename='database.ini', section='gr'):
# create a parser
parser = configparser()
# read config file
parser.read(filename)
# get section, default to gr
db = {}
if parser.has_section(section):
params = parser.items(section)
for param in params:
db[param[0]] = param[1]
else:
raise Exception('Section {0} not found in the {1} file'.format(section, filename))
return db
testing.py
#!/usr/bin/env python
import psycopg2
from config import config
def connect():
""" Connect to the PostgreSQL database server """
conn = None
try:
# read connection parameters
params = config()
# connect to the PostgreSQL server
print('Connecting to the PostgreSQL database...')
conn = psycopg2.connect(**params)
# create a cursor
cur = conn.cursor()
# execute a statement
print('PostgreSQL database version:')
cur.execute('SELECT version()')
# display the PostgreSQL database server version
db_version = cur.fetchone()
print(db_version)
# close the communication with the PostgreSQL
cur.close()
except (Exception, psycopg2.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
print('Database connection closed.')
if __name__ == '__main__':
connect()
[root#svr mytest]# ./testing.py
'module' object is not callable
any ideas ?
thank you.
You have an error in your parser creation, see the ConfigParser examples:
An example of reading the configuration file again:
import ConfigParser
config = ConfigParser.RawConfigParser()
config.read('example.cfg')
In Python, a module is basically just a file, so if you want to use anything from this module, you have to specify what from the module you want. In your case, change the lines to
# create a parser
parser = configparser.ConfigParser()
This way, you are using the class ConfigParser from the module.
You can also use the follow to import the parser:
from configparser import ConfigParser
instead of
import configparser
I had the same issue as you and this worked for me.
Learning Python 2.7 and trying to run it on Vagrant.
Steps taken:
Vagrant up
Vagrant ssh
Run command python webserver.py
Issue when I run this command, it gives out an ImportError: No module named BaseHTTPServer. Is this problem related to pg_config.sh? Thank you in advance for helping me understand.
I have checked my python 2.7 directory. BaseHTTPServer.py seems to be there.
from BaseHTTPServer import BaseHTTPRequestHandler, BaseHTTPServer
class WebServerHandler(BaseHTTPRequestHandler):
def do_Get(self):
if self.path.endswith("/hello"):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_header()
message = ""
message += "<html><body>Hello!</body></html>"
self.wfile.write(message)
print message
return
else:
self.send_error(404,'File Not Found: %s' % self.path)
def main():
try:
port = 8080
server = HTTPServer(('', port), WebServerHandler)
print "Web Server running on port %s" % port
server.serveforever()
except KeyboardInterrupt:
print " ^C entered, stopping web server...."
server.socket.close()
if _name_ == '__main__':
main()
You may consider switching to python 3.x. This is the updated version of your code in Python 3.7.2. Notice in the code below that the BaseHTTPServer module is changed to http.server in python3 link
from http.server import BaseHTTPRequestHandler, HTTPServer
class WebServerHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.endswith("/hello"):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
message = ""
message += "<html><body>Hello!</body></html>"
self.wfile.write(message)
print (message)
return
else:
self.send_error(404, 'File Not Found: %s' % self.path)
def main():
try:
port = 8080
server = HTTPServer(('', port), WebServerHandler)
print ("Web Server running on port %s" % port)
server.serve_forever()
except KeyboardInterrupt:
print (" ^C entered, stopping web server....")
server.socket.close()
if __name__ == '__main__':
main()
You can follow the link I provided above to read more from the documentation.
Are you sure that the python you are running/testing from the command line is the same python that your script is running?
i.e. BaseHTTPServer might be present in one install, but not the other.
For example, on my machine:
$ which python2.7
/usr/bin/python2.7
Is your "python" (in the command line you specified) the same as "python2.7"?
$ python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import BaseHTTPServer
>>> BaseHTTPServer
<module 'BaseHTTPServer' from '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.pyc'>
Try a module that does exist to ensure the path is what you are expecting.
In python earlier than v3 you need to run http server as
python -m SimpleHTTPServer 8069
if you are using pycharm, change the interpreter to Python 2.7 from the latest version.
If the project Interpreter is not present you can also add one by specifying the python 2.7 install path.