Akka Remote: get autogenerated port - akka

I have a Java client, which obtains an autogenerated port. After starting the actor system, I want to access the port.
Config clientConfig = ConfigFactory.parseString("akka.remote.netty.tcp.port = 0")
.withFallback(ConfigFactory.parseString("akka.remote.netty.tcp.hostname = " + serverHostName))
.withFallback(ConfigFactory.load("common"));
actorSystem = ActorSystem.create("clientActorSystem", clientConfig);
// how to access the generated port here..!?
The port must already be set since the log output after ActorSystem.create(...) is like that:
[INFO] [03/31/2016 14:11:32.042] [main] [akka.remote.Remoting] Starting remoting
[INFO] [03/31/2016 14:11:32.233] [main] [akka.remote.Remoting] Remoting started; listening on addresses :[akka.tcp://actorSystem#localhost:58735]
[INFO] [03/31/2016 14:11:32.234] [main] [akka.remote.Remoting] Remoting now listens on addresses: [akka.tcp://actorSystem#localhost:58735]
If I try to get it via the configuration with actorSystem.settings().config().getValue("akka.remote.netty.tcp.port"), I still get 0 as defined before.
Has anyone an idea how this port (58735 in the example) can be accessed?

Using scala you can get Option of port on which Actor system is currently running:
val port = system.provider.getDefaultAddress.port
Hope you will be able to get the same code in Java.

The accepted answer probably worked for older versions of Akka but as of now (version 2.5.x) you will be getting something like:
Error:(22, 18) method provider in trait ActorRefFactory cannot be accessed in akka.actor.ActorSystem
The solution would be to use akka extensions. Here is how I use it:
Example. scala
package example
import akka.actor._
class AddressExtension(system: ExtendedActorSystem) extends Extension {
val address: Address = system.provider.getDefaultAddress
}
object AddressExtension extends ExtensionId[AddressExtension] {
def createExtension(system: ExtendedActorSystem): AddressExtension = new AddressExtension(system)
def hostOf(system: ActorSystem): String = AddressExtension(system).address.host.getOrElse("")
def portOf(system: ActorSystem): Int = AddressExtension(system).address.port.getOrElse(0)
}
object Main extends App {
val system = ActorSystem("Main")
println(AddressExtension.portOf(system))
}

Related

Flask-sqlalchemy / uwsgi: DB connection problem when more than on process is used

I have a Flask app running on Heroku with uwsgi server in which each user connects to his own database. I have implemented the solution reported here for a very similar situation. In particular, I have implemented the connection registry as follows:
class DBSessionRegistry():
_registry = {}
def get(self, URI, **kwargs):
if URI not in self._registry:
current_app.logger.info(f'INFO - CREATING A NEW CONNECTION')
try:
engine = create_engine(URI,
echo=False,
pool_size=5,
max_overflow=5)
session_factory = sessionmaker(bind=engine)
Session = scoped_session(session_factory)
a_session = Session()
self._registry[URI] = a_session
except ArgumentError:
raise Exception('Error')
current_app.logger.info(f'SESSION ID: {id(self._registry[URI])}')
current_app.logger.info(f'REGISTRY ID: {id(self._registry)}')
current_app.logger.info(f'REGISTRY SIZE: {len(self._registry.keys())}')
current_app.logger.info(f'APP ID: {id(current_app)}')
return self._registry[URI]
In my create_app() I assign a registry to the app:
app.DBregistry = DBSessionRegistry()
and whenever I need to talk to the DB I call:
current_app.DBregistry.get(URI)
where the URI is dependent on the user. This works nicely if I use uwsgi with one single process. With more processes,
[uwsgi]
processes = 4
threads = 1
sometimes it gets stuck on some requests, returning a 503 error code. I have found that the problem appears when the requests are handled by different processes in uwsgi. This is an excerpt of the log, which I commented to illustrate the issue:
# ... EVERYTHING OK UP TO HERE.
# ALL PREVIOUS REQUESTS HANDLED BY PROCESS pid = 12
INFO in utils: SESSION ID: 139860361716304
INFO in utils: REGISTRY ID: 139860484608480
INFO in utils: REGISTRY SIZE: 1
INFO in utils: APP ID: 139860526857584
# NOTE THE pid IN THE NEXT LINE...
[pid: 12|app: 0|req: 1/1] POST /manager/_save_task =>
generated 154 bytes in 3457 msecs (HTTP/1.1 200) 4 headers in 601
bytes (1 switches on core 0)
# PREVIOUS REQUEST WAS MANAGED BY PROCESS pid = 12
# THE NEXT REQUEST IS FROM THE SAME USER AND TO THE SAME URL.
# SO THERE IS NO NEED FOR CREATING A NEW CONNECTION, BUT INSTEAD...
INFO - CREATING A NEW CONNECTION
# TO THIS POINT, I DON'T UNDERSTAND WHY IT CREATED A NEW CONNECTION.
# THE SESSION ID CHANGES, AS IT IS A NEW SESSION
INFO in utils: SESSION ID: 139860363793168 # <<--- CHANGED
INFO in utils: REGISTRY ID: 139860484608480
INFO in utils: REGISTRY SIZE: 1
# THE APP AND THE REGISTRY ARE UNIQUE
INFO in utils: APP ID: 139860526857584
# uwsgi GIVES UP...
*** HARAKIRI ON WORKER 4 (pid: 11, try: 1) ***
# THE FAILED REQUEST WAS MANAGED BY PROCESS pid = 11
# I ASSUME THIS IS WHY IT CREATED A NEW CONNECTION
HARAKIRI: -- syscall> 7 0x7fff4290c6d8 0x1 0xffffffff 0x4000 0x0 0x0
0x7fff4290c6b8 0x7f33d6e3cbc4
HARAKIRI: -- wchan> poll_schedule_timeout
HARAKIRI !!! worker 4 status !!!
HARAKIRI [core 0] - POST /manager/_save_task since 1587660997
HARAKIRI !!! end of worker 4 status !!!
heroku[router]: at=error code=H13 desc="Connection closed without
response" method=POST path="/manager/_save_task"
DAMN ! worker 4 (pid: 11) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 4 (new pid: 14)
# FROM HERE ON, NOTHINGS WORKS ANYMORE
This behavior is consistent over several attempts: when the pid changes, the request fails. Even with a pool_size = 1 in the create_engine function the issue persists. No issue instead is uwsgi is used with one process.
I am pretty sure it is my fault, there is something I don't know or I don't understand about how uwsgi and/or sqlalchemy work. Could you please help me?
Thanks
What is hapeening is that you are trying to share memory between processes.
There are some exaplanations in these posts.
(is it possible to share memory between uwsgi processes running flask app?).
(https://stackoverflow.com/a/45383617/11542053)
You can use an extra layer to store your sessions outsite of the app.
For that, you can use uWsgi's SharedArea(https://uwsgi-docs.readthedocs.io/en/latest/SharedArea.html) which is very low level or you can user other approaches like uWsgi's caching(https://uwsgi-docs.readthedocs.io/en/latest/Caching.html)
hope it helps.

Access Kafka Cluster Outside GCP

I'm currently trying to access the kafka cluster(bitnami) from my local machine, however the problem is that even after exposing the required host and ports in server.properties and adding firewall rules to allow 9092 port it just doesn't connect.
I'm running 2 broker and 1 zookeeper configuration.
Expected Output: Producer.bootstrap_connected() should return True.
Actual Output: False
server.properties
listeners=SASL_PLAINTEXT://:9092
advertised.listeners=SASL_PLAINTEXT://gcp-cluster-name:9092
sasl.mechanism.inter.broker.protocol=PLAIN`
sasl.enabled.mechanisms=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT
Consumer.py
from kafka import KafkaConsumer
import json
sasl_mechanism = 'PLAIN'
security_protocol = 'SASL_PLAINTEXT'
# Create a new context using system defaults, disable all but TLS1.2
context = ssl.create_default_context()
context.options &= ssl.OP_NO_TLSv1
context.options &= ssl.OP_NO_TLSv1_1
consumer = KafkaConsumer('organic-sense',
bootstrap_servers='<server-ip>:9092',
value_deserializer=lambda x: json.loads(x.decode('utf-8')),
ssl_context=context,
sasl_plain_username='user',
sasl_plain_password='<password>',
sasl_mechanism=sasl_mechanism,
security_protocol = security_protocol,
)
print(consumer.bootstrap_connected())
for data in consumer:
print(data)

reactivemongo play 2.3 0.11.7 to 0.11.11 upgrade raises ExceptionInInitializerError

[I am re-editing this question to reflect on my last tests]
I am trying to upgrade my akka / play 2.3 application from
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.7.play23"
to
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.11-play23"
Compilation goes fine but at run-time I get the following error:
[ERROR] -- NettyTransport(akka://reactivemongo)
failed to bind to /127.0.0.1:2552, shutting down Netty transport
...
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /127.0.0.1:2552
The Akka part of application.conf reads as follows:
akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "DEBUG"
actor {
provider = "akka.remote.RemoteActorRefProvider"
mailbox {
requirements {
"akka.dispatch.BoundedMessageQueueSemantics" = bounded-mailbox
}
}
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 2552
}
}
}
The exception is raised when trying to instantiate the reactivemongo driver
val driver = new reactivemongo.api.MongoDriver()
This suggests that the mongodriver is using Akka under the hood and is binding to the same address that my main application. And indeed, if I edit my application.conf and change the akka.remote.netty.tcp.port from 2552 to 2553, I get the following exception:
[ERROR] -- NettyTransport(akka://reactivemongo)
failed to bind to /127.0.0.1:2553, shutting down Netty transport
In the previous versions of reactivemongo, by default, instantiating the driver was starting a new actor system so maybe version 0.11.11 tries to reuse the existing system?
I have tried to modify the akka port used by the driver as follows:
val customConf = ConfigFactory.parseString("""
akka {
remote {
netty.tcp.port = 4711
}
}
""")
val typesafeConfig: com.typesafe.config.Config = ConfigFactory.load(customConf)
val driver = new reactivemongo.api.MongoDriver(Some(typesafeConfig))
But this does not work, the new port is not taken into account and I keep getting the same error:
[ERROR] -- NettyTransport(akka://reactivemongo)
failed to bind to /127.0.0.1:2552, shutting down Netty transport
Actually, ReactiveMongo loads its configuration from the key 'mongo-async-driver'.
So, Adding the following permits to configure ReactiveMongo underlying akka system:
val customConf = ConfigFactory.parseString("""
mongo-async-driver {
akka {
loglevel = WARNING
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 4711
}
}
}
}
""")

How to get the remotely created actor?

How to get the remotely created actor?
PS:
I'm study the akka-samples/akka-sample-remote.
Following is the case: I have two host: host1, host2
the app deployed on host2 will create an actor remotely on host1 through the config:
application.conf
remote_app {
include "common"
akka {
actor {
deployment {
/advanced_calculator {
remote = "akka://CalculatorApplication#host1:2552"
}
}
}
remote.netty.port = 2554
}
}
Scala code:
val remoteActor = system.actorOf(Props[AdvancedCalculatorApplication], "advanced_calculator")
The program runs successfully, but I still have a question:
How can I get the remotely created actor(*advanced_calculator*) on host2?
I've tried the following piece of code , but my problem was not sloved
system.actorFor("advanced_calculator"), system.actorFor("akka://CalculatorApplication#host1:2552/user/advanced_calculator")
Have you've followed the instructions over at: http://doc.akka.io/docs/akka/2.0/scala/remoting.html
Are you running: "val remoteActor = system.actorOf(Props[AdvancedCalculatorApplication], "advanced_calculator")" on host2?

Interaction between C++ and Rails applications

I have two applications: c++ service and a RoR web server (they are both running at same VPS)
I need to "send" some variables (and then do something with them) from each other. For exaple, i'm looking for something like this:
// my C++ sample
void SendMessage(string message) {
SendTo("127.0.0.1", message);
}
void GetMessage(string message) {
if (message == "stop")
SendMessage("success");
}
# Ruby sample
# application_controller.rb
def stop
#m = Messager.new
#m.send("stop")
end
I have never used it before, and i even don't know which technology should i search and learn.
Ok, i have found the solution. Its TCP sockets:
Ruby TCP server, to send messages:
require 'socket'
server = TCPServer.open(2000)
loop {
Thread.start(server.accept) do |client|
client.puts(Time.now.ctime)
client.puts "Closing the connection. Bye!"
client.close
end
}
Ruby client, to accept messages:
require 'socket'
host = 'localhost'
port = 2001 # it should be running server, or it will be connection error
s = TCPSocket.open(host, port)
while line = s.gets
puts line.chop
end
s.close
Now you should write TCP-server+client in another application. But you have got the idea.