The title is quite self-explanatory, how can i get the blocknumber of the relay chain I'm connected to (Kusama) instead of my local blocknumber ?
All of this in a substrate/cumulus environment
I assume that you mean not Substrate, but Cumulus, to be specific, since you are talking about parachains.
Cumulus parachains' runtimes receive the so called PersistedValidationData from the collators. It has several pieces of information passed by the relay-chain. It does include relay_parent_number. That is the relay chain block number from which the parachain block in question was created.
In a cumulus parachain runtime, PersistedValidationData can be accessed through the validation_data getter.
Be advised, this data is submitted by a collator in an inherent. That means, among other things, this data is not available in the on_initialize hooks.
Related
I'm having trouble getting my head around the purpose of supply {…} blocks/the on-demand supplies that they create.
Live supplies (that is, the types that come from a Supplier and get new values whenever that Supplier emits a value) make sense to me – they're a version of asynchronous streams that I can use to broadcast a message from one or more senders to one or more receivers. It's easy to see use cases for responding to a live stream of messages: I might want to take an action every time I get a UI event from a GUI interface, or every time a chat application broadcasts that it has received a new message.
But on-demand supplies don't make a similar amount of sense. The docs say that
An on-demand broadcast is like Netflix: everyone who starts streaming a movie (taps a supply), always starts it from the beginning (gets all the values), regardless of how many people are watching it right now.
Ok, fair enough. But why/when would I want those semantics?
The examples also leave me scratching my head a bit. The Concurancy page currently provides three examples of a supply block, but two of them just emit the values from a for loop. The third is a bit more detailed:
my $bread-supplier = Supplier.new;
my $vegetable-supplier = Supplier.new;
my $supply = supply {
whenever $bread-supplier.Supply {
emit("We've got bread: " ~ $_);
};
whenever $vegetable-supplier.Supply {
emit("We've got a vegetable: " ~ $_);
};
}
$supply.tap( -> $v { say "$v" });
$vegetable-supplier.emit("Radish"); # OUTPUT: «We've got a vegetable: Radish»
$bread-supplier.emit("Thick sliced"); # OUTPUT: «We've got bread: Thick sliced»
$vegetable-supplier.emit("Lettuce"); # OUTPUT: «We've got a vegetable: Lettuce»
There, the supply block is doing something. Specifically, it's reacting to the input of two different (live) Suppliers and then merging them into a single Supply. That does seem fairly useful.
… except that if I want to transform the output of two Suppliers and merge their output into a single combined stream, I can just use
my $supply = Supply.merge:
$bread-supplier.Supply.map( { "We've got bread: $_" }),
$vegetable-supplier.Supply.map({ "We've got a vegetable: $_" });
And, indeed, if I replace the supply block in that example with the map/merge above, I get exactly the same output. Further, neither the supply block version nor the map/merge version produce any output if the tap is moved below the calls to .emit, which shows that the "on-demand" aspect of supply blocks doesn't really come into play here.
At a more general level, I don't believe the Raku (or Cro) docs provide any examples of a supply block that isn't either in some way transforming the output of a live Supply or emitting values based on a for loop or Supply.interval. None of those seem like especially compelling use cases, other than as a different way to transform Supplys.
Given all of the above, I'm tempted to mostly write off the supply block as a construct that isn't all that useful, other than as a possible alternate syntax for certain Supply combinators. However, I have it on fairly good authority that
while Supplier is often reached for, many times one would be better off writing a supply block that emits the values.
Given that, I'm willing to hazard a pretty confident guess that I'm missing something about supply blocks. I'd appreciate any insight into what that might be.
Given you mentioned Supply.merge, let's start with that. Imagine it wasn't in the Raku standard library, and we had to implement it. What would we have to take care of in order to reach a correct implementation? At least:
Produce a Supply result that, when tapped, will...
Tap (that is, subscribe to) all of the input supplies.
When one of the input supplies emits a value, emit it to our tapper...
...but make sure we follow the serial supply rule, which is that we only emit one message at a time; it's possible that two of our input supplies will emit values at the same time from different threads, so this isn't an automatic property.
When all of our supplies have sent their done event, send the done event also.
If any of the input supplies we tapped sends a quit event, relay it, and also close the taps of all of the other input supplies.
Make very sure we don't have any odd races that will lead to breaking the supply grammar emit* [done|quit].
When a tap on the resulting Supply we produce is closed, be sure to close the tap on all (still active) input supplies we tapped.
Good luck!
So how does the standard library do it? Like this:
method merge(*#s) {
#s.unshift(self) if self.DEFINITE; # add if instance method
# [I elided optimizations for when there are 0 or 1 things to merge]
supply {
for #s {
whenever $_ -> \value { emit(value) }
}
}
}
The point of supply blocks is to greatly ease correctly implementing reusable operations over one or more Supplys. The key risks it aims to remove are:
Not correctly handling concurrently arriving messages in the case that we have tapped more than one Supply, potentially leading us to corrupt state (since many supply combinators we might wish to write will have state too; merge is so simple as not to). A supply block promises us that we'll only be processing one message at a time, removing that danger.
Losing track of subscriptions, and thus leaking resources, which will become a problem in any longer-running program.
The second is easy to overlook, especially when working in a garbage-collected language like Raku. Indeed, if I start iterating some Seq and then stop doing so before reaching the end of it, the iterator becomes unreachable and the GC eats it in a while. If I'm iterating over lines of a file and there's an implicit file handle there, I risk the file not being closed in a very timely way and might run out of handles if I'm unlucky, but at least there's some path to it getting closed and the resources released.
Not so with reactive programming: the references point from producer to consumer, so if a consumer "stops caring" but hasn't closed the tap, then the producer will retain its reference to the consumer (thus causing a memory leak) and keep sending it messages (thus doing throwaway work). This can eventually bring down an application. The Cro chat example that was linked is an example:
my $chat = Supplier.new;
get -> 'chat' {
web-socket -> $incoming {
supply {
whenever $incoming -> $message {
$chat.emit(await $message.body-text);
}
whenever $chat -> $text {
emit $text;
}
}
}
}
What happens when a WebSocket client disconnects? The tap on the Supply we returned using the supply block is closed, causing an implicit close of the taps of the incoming WebSocket messages and also of $chat. Without this, the subscriber list of the $chat Supplier would grow without bound, and in turn keep alive an object graph of some size for each previous connection too.
Thus, even in this case where a live Supply is very directly involved, we'll often have subscriptions to it that come and go over time. On-demand supplies are primarily about resource acquisition and release; sometimes, that resource will be a subscription to a live Supply.
A fair question is if we could have written this example without a supply block. And yes, we can; this probably works:
my $chat = Supplier.new;
get -> 'chat' {
web-socket -> $incoming {
my $emit-and-discard = $incoming.map(-> $message {
$chat.emit(await $message.body-text);
Supply.from-list()
}).flat;
Supply.merge($chat, $emit-and-discard)
}
}
Noting it's some effort in Supply-space to map into nothing. I personally find that less readable - and this didn't even avoid a supply block, it's just hidden inside the implementation of merge. Trickier still are cases where the number of supplies that are tapped changes over time, such as in recursive file watching where new directories to watch may appear. I don't really know how'd I'd express that in terms of combinators that appear in the standard library.
I spent some time teaching reactive programming (not with Raku, but with .Net). Things were easy with one asynchronous stream, but got more difficult when we started getting to cases with multiple of them. Some things fit naturally into combinators like "merge" or "zip" or "combine latest". Others can be bashed into those kinds of shapes with enough creativity - but it often felt contorted to me rather than expressive. And what happens when the problem can't be expressed in the combinators? In Raku terms, one creates output Suppliers, taps input supplies, writes logic that emits things from the inputs into the outputs, and so forth. Subscription management, error propagation, completion propagation, and concurrency control have to be taken care of each time - and it's oh so easy to mess it up.
Of course, the existence of supply blocks doesn't stop being taking the fragile path in Raku too. This is what I meant when I said:
while Supplier is often reached for, many times one would be better off writing a supply block that emits the values
I wasn't thinking here about the publish/subscribe case, where we really do want to broadcast values and are at the entrypoint to a reactive chain. I was thinking about the cases where we tap one or more Supply, take the values, do something, and then emit things into another Supplier. Here is an example where I migrated such code towards a supply block; here is another example that came a little later on in the same codebase. Hopefully these examples clear up what I had in mind.
I'm trying to do a project described also here: PacketQueue is 0
I've modified the UdpBasicApp.cc to suit my needs, and in the handleMessage() function I added a piece of code to retrieve the length of the ethernet queue of a router. However the value returned is always 0.
My .ned file regarding the queue of routers is this:
**.router*.eth[*].mac.queue.typename = "DropTailQueue"
**.router*.eth[*].mac.queue.packetCapacity = 51
The code added in the UdpBasicApp.cc file is this:
cModule *mod = getModuleByPath("router3.eth[*].mac.queue.");
queueing::PacketQueue *queue = check_and_cast<queueing::PacketQueue*>(mod);
int c = queue->getNumPackets();
So my question is this: is this the right way to create a queue in a router linked to other nodes with an ethernet link?
My doubt is that maybe packets don't pass through the specified interface, i.e. I've set the ini parameters for the wrong queue.
You are not creating that queue. It was already instantiated by the OMNeT++ kernel. You are just getting a reference to an already existing module with the getModuleByPath() call.
The router3.eth[*].mac.queue. module path is rather suspicious in that call. It is hard-coded in all of your application to get the queue length from router3 even if the app is installed in router1. i.e. you are trying to look at the queue length in a completely different node. Then, the eth[*] is wrong. As a router obviously contains more than one ethernet interface (otherwise it would not be a router), you must explicitly specify which interface you want to sepcify. You must not specify patterns in module path (i.e. eth[0] or something like that, with an exact index must be specified). And at this point, you have to answer the question which ethernet interface I'm interested in, and specify that index. Finally the . end the end is also invalid, so I believe, your code never executes, otherwise the check_and_cast part would have raised an error already.
If you wanted to reach the first enthern interface from an UDP app in the same node, you would use relative path, something like this: ^.eth[0].mac.queue
Finally, if you are unsure whether your model works correctly, why not start the model with Qtenv, and check whether the given module receives any packet? Like,, drill down in the model until the given queue is opened as a simple module (i.e. you see the empty inside of the queue module) and then tap the run/fast run until next event in this module. If the simulation does not stop, then that module indeed did not received any packets and your configuration is wrong.
I'm using Doobie in a ZIO application, and sometimes I get deadlocks (total freeze of the application). That can happen if I run my app on only one core, or if I reach the number of maximum parallel connections to the database.
My code looks like:
def mkTransactor(cfg: DatabaseConfig): RManaged[Blocking, Transactor[Task]] =
ZIO.runtime[Blocking].toManaged_.flatMap { implicit rt =>
val connectEC = rt.platform.executor.asEC
val transactEC = rt.environment.get.blockingExecutor.asEC
HikariTransactor
.fromHikariConfig[Task](
hikari(cfg),
connectEC,
Blocker.liftExecutionContext(transactEC)
)
.toManaged
}
private def hikari(cfg: DatabaseConfig): HikariConfig = {
val config = new com.zaxxer.hikari.HikariConfig
config.setJdbcUrl(cfg.url)
config.setSchema(cfg.schema)
config.setUsername(cfg.user)
config.setPassword(cfg.pass)
config
}
Alternatively, I set the leak detection parameter on Hikari (config.setLeakDetectionThreshold(10000L)), and I get leak errors which are not due to the time taken to process DB queries.
There is a good explanation in the Doobie documentation about the execution contexts and the expectations for each: https://tpolecat.github.io/doobie/docs/14-Managing-Connections.html#about-transactors
According to the docs, the "execution context for awaiting connection to the database" (connectEC in the question) should be bounded.
ZIO, by default, has only two thread pools:
zio-default-async – Bounded,
zio-default-blocking – Unbounded
So it is quite natural to believe that we should use zio-default-async since it is bounded.
Unfortunately, zio-default-async makes an assumption that its operations never, ever block. This is extremely important because it's the execution context used by the ZIO interpreter (its runtime) to run. If you block on it, you can actually block the evaluation progression of the ZIO program. This happens more often when there's only one core available.
The problem is that the execution context for awaiting DB connection is meant to block, waiting for free space in the Hikari connection pool. So we should not be using zio-default-async for this execution context.
The next question is: does it makes sense to create a new thread pool and corresponding execution context just for connectEC? There is nothing forbidding you to do so, but it is likely not necessary, for three reasons:
You want to avoid creating thread pools, especially since you likely have several already created from your web framework, DB connection pool, scheduler, etc. Each thread pool has its cost. Some examples are:
More to manage for the jvm JVM
Consumes more OS resources
Switching between threads, which that part is expensive in terms of performance
Makes your application runtime more complex to understand(complex thread dumps, etc)
ZIO thread pool ergonomics start to be well optimized for their usage
At the end of the day, you will have to manage your timeout somewhere, and the connection is not the part of the system which is the most likely to have enough information to know how long it should wait: different interactions (ie, in the outer parts of your app, nearer to use points) may require different timeout/retry logic.
All that being said, we found a configuration that works very well in an application running in production:
// zio.interop.catz._ provides a `zioContextShift`
val xa = (for {
// our transaction EC: wait for aquire/release connections, must accept blocking operations
te <- ZIO.access[Blocking](_.get.blockingExecutor.asEC)
} yield {
Transactor.fromDataSource[Task](datasource, te, Blocker.liftExecutionContext(te))
}).provide(ZioRuntime.environment).runNow
def transactTask[T](query: Transactor[Task] => Task[T]): Task[T] = {
query(xa)
}
I made a drawing of how Doobie and ZIO execution context map one other to each other: https://docs.google.com/drawings/d/1aJAkH6VFjX3ENu7gYUDK-qqOf9-AQI971EQ4sqhi2IY
UPDATE: I created a repos with 3 examples of that pattern usage (mixed app, pure app, ZLayer app) here: https://github.com/fanf/test-zio-doobie
Any feedback is welcome.
I'm learning every day more about dds, so my question my sound weird. I hope it makes sense.
One of the requirements of some dds wrapper I'm writing, is that it times out after some timeout period if it fails to write. My question: How can I do that?
On Prism Tech's website's tutorial, there's explanation on how to use a WaitSet to block a read operation, but what about write?
Here's some code including the question:
dds::domain::DomainParticipant dp(0);
dds::topic::Topic<MyType> topic(dp, "MyTopic");
dds::pub::Publisher pub(dp);
dds::pub::DataWriter<MyType> dw(pub, topic);
MyType t;
dw.write(t); //how can I make this block for 5 seconds (tops), and then throw an error on failure?
I noticed there exists a function in the API DataWriter::wait_for_acknowledgements(int timeout), but this seems to be bound to the DataWriter object, not to the specific call of writing. Can I bind it with the call above?
This is configured in QoS, cf RELIABILITY, field "max_blocking_time". How you set this value will depend on the vendor's implementation. Generally you get the current QoS, update the field, write the QoS back. Keep in mind that certain QoS policies must be set before something else happens. Reliability is "Before Enable" (at least in the implementation I'm most familiar with), which means you need to create the data-writer disabled, update the QoS, then enable the writer.
If QoS can be set outside the application (via XML for example), then you can set the policy easily. Otherwise, you need to do it in code.
From the spec:
The value of the max_blocking_time indicates the maximum time the operation DataWriter::write is allowed to block if the DataWriter does not have space to store the value written. The default max_blocking_time=100ms.
Given a System that contains two components, A and B, and
The System starts up A and B concurrently. Now A can go through states {A.Starting, A.Ready}, and B can be in states {B.Starting, B.DoingX, B.DoingY}. (Events to transition between A's and B's states are named accordingly: B.doingx => B goes to B.DoingX etc...)
I want to model that
While A is in A.Starting, or B is in B.Starting, the System is "Starting"
The System is in state "DoingX" when A is in A.Ready and B is in B.DoingX
The System is in state "DoingY" when A is in A.Ready and B is in B.DoingY
If I'm not mistaken, the fork/join pseudo-states could be used here.
But do these model elements have the declarative semantics of the composed state mentioned above? Is there another way to model this?
(Note: the diagrams are from http://yuml.me)
Why don't you just pull these apart? Here's another idea on how you could model it (assuming I understood it correctly) :
a state "Starting", that contains the states you refer to as A.Starting and B.Starting in parallel regions (you can use fork/joins here, or just rely on the default behavior of all regions being activated when "Starting" state is entered)
another state "Doing" that contains a region with your "A.Ready" state and another parallel region, that contains the two states "B.DoingX" and "B.DoingY".
If you really need to have an overall "DoingX" state, then you may have to create two states that correspond to A.Ready.
Anyways, on a broader perspective: I believe your point of view is a little bit off here, when you say that the "System is in state ...". Rather, the system modeled by such a state machine is in a set of states. So normally, I would be perfectly happy to say that "the system is currently in A.Ready and B.DoingX".
Maybe all you need is a change of terminology. What about this:
The system is in configuration "DoingX" when A.Ready and B.DoingX states are active ?
In response to the comment: Yes, this is standard, here's the corresponding part from the superstructure specification (version 2.4 beta):
In a hierarchical state machine more than one state can be active at the same time. [...] the current active “state” is actually represented by a set of trees of states
starting with the top-most states of the root regions down to the innermost active substate. We refer to such a state tree as
a state configuration.