I was planning on creating a new Layer for logging to gcp LogExplorer. Because we are working in rust, the api for logging is an async function call.
The tracing Layer trait exposes on_event function which is not async, which is where the problem comes in (link here)
I was thinking of ways to solve this, a simple solution which came to my mind was to send the events (the relevant fields) through a channel and then consume the channel on a tokio::spawn where my gcp client can make the grpc calls.
What I have in mind looks roughly like this (glossing over a lot of the details)
fn get_fields_from_event(event: &tracing::Event<'_>) -> LogEntry {
unimplemented!("some code here to convert to a type I can use with grpc coded for logging in gcp");
}
let (events_sender, events_receiver) = futures::channel::mpsc();
impl<S> Layer<S> for CustomLayer
where
S: tracing::Subscriber,
{
fn on_event(
&self,
event: &tracing::Event<'_>,
_ctx: tracing_subscriber::layer::Context<'_, S>,
) {
events_sender.clone().send(get_fields_from_event(event))
}
}
tokio::spawn(async move {
events_receiver.for_each(|event| async {grpc_client.send(event);});
});
Some obvious flaws I see with this approach is:
if the tokio::spawn silently exists, we might loose logging (I can safeguard against it .. but logging is kinda important for debugging so I would have to restart the process completely or manage the tokio::spawn process)
tokio::spawn itself feels a bit weird to get async based logging client supported. For some reason this feels a bit weird to me.
Are there any other alternatives or insights I am missing, or some crate which might provide support for working with async logging clients and integrating them in tracing Layer.
Thanks!
Related
In Ember I have a component that starts a never-ending poll to keep some data up to date. Like so:
export default Component.extend({
pollTask: task(function * () {
while(true) {
yield timeout(this.get('pollRate'));
this.fetchSomeData();
}
}).on('init')
})
This causes a preexisting acceptance test to get stuck in this task and run forever, even though it should be run asynchronously. The test looks like this:
test('my test', async function(assert) {
mockFindRecord(make('fetched-model'));
await visit('/mypage'); // gets stuck here
await typeIn('input', 'abc123');
assert.ok(somethingAboutThePage);
});
I thought at first that I had mocked the request wrong and that the test was just timing out, but it was in fact correctly polling data. Removing this task makes the acceptance test finish normally.
Testing this manually seems to work fine, and nothing gets stuck. Why is this happening and what is the right way to address this?
Saw Unit testing ember-concurrency tasks and yields but it doesn't really help since it only deals with unit tests.
You're not doing anything wrong and this is a common gotcha with ember-concurrency. Ember-concurrency's timeout() function relies on Ember.run.later() to create the timeout and fortunately or unfortunately, Ember's test suite is aware of all timers created with Ember.run.later() and will wait for all timers to settle before letting the test continue. Since your code is using an infinite loop your timers will never settle so the test hangs. There's a nice little section about testing asynchronous code in the Ember guides here.
There's a section in the ember-concurrency docs about this exact problem here. I recommend you look through it to see their recommendations on how to tackle this although it seems as if there's no real elegant solution at the time.
The quickest and probably easiest way to get this to not hang would be to throw in a check to see if the app is being tested (gross, I know):
pollTask: task(function * () {
while(true) {
yield timeout(this.get('pollRate'));
this.fetchSomeData();
if (Ember.testing) return; // stop polling to prevent tests from hanging
}
}).on('init')
You can also try to throw in a call to Ember.run.cancelTimers() in your tests/helpers/start-app.js file (another suggestion in that section):
window._breakTimerLoopsId = Ember.run.later(() => {
Ember.run.cancelTimers();
}, 500);
But it doesn't seem to appear in the API docs so I personally wouldn't rely on it.
I'm working with emberjs during some time and now I want to refine my understanding of ember's nature.
Question 1.
Am I right that the route's model() hook is the best place to write asynchronous code (to get all needed data)? Should I try to place all my network requests (or at least most of them) inside routes' models?
Question 2.
The component hooks have synchronous nature. Does that mean it's a bad decision to write async code inside hooks?
Say, I have async init() hook where some data is calculated and didRender() hook where I expect to use that data. When ember calls init() it returns a Promise, so it's moved from a stack to a special queue and ember doesn't wait until event loop returns that code back to a stack. So ember runs next hooks, and when didRender() is being executed the init() hook may not be fulfilled and the expected data may not exist. Is that right?
Question 3.
Services hooks should also be synchronous. Because when a service is injected inside a component and is used ember also doesn't wait until the async hook is fulfilled.
Say, I have a shopping cart service with products property. The products ids are stored in localstorage and I want to get those products from a server to set them into products property.
import Service from '#ember/service';
import { A } from '#ember/array';
import { inject as service } from '#ember/service';
export default Service.extend({
store: service(),
init(...args) {
this._super(args);
this.set('products', A());
},
async loadProducts() {
const cartProducts = A();
const savedCartProducts = localStorage.getItem('cartProducts');
if (savedCartProducts) {
const parsedSavedCartProducts = JSON.parse(savedCartProducts);
const ids = Object.keys(parsedSavedCartProducts);
if (ids.length > 0) {
const products = await this.store.query('product', {id: Object.keys(parsedSavedCartProducts)});
products.forEach(p => {
p.set('cartQuantity', Number(parsedSavedCartProducts[p.id].qty));
cartProducts.pushObject(p);
});
}
}
this.products.pushObjects(cartProducts);
},
...
});
If I call loadProducts() from service's init() I can't use this.cart.products in controllers/application.js, for example. Because service is ready, but async init() is still executed. So, should I call it in routes/application.js model() hook?
Question 4.
If there is some general component that doesn't refer to any route, but this component needs some data from a server where should I make async requests? Or are computed properties and observers are the only solutions?
Thanks a lot.
Good questions here, you’re on the right track!
Question 1: yes, the general rule of thumb is that until you are familiar with things, doing async work in the route hooks (beforeModel, model and afterModel) make it easier to think about what is going on. Eventually you may want to start bending the defaults to suit your custom UI needs, but it’s simplest to start with this rule
Questions 2-4: you’re asking several questions here about async code, so will answer here more broadly.
First off, services can do async or not but unless you call those methods from a hook (like a route hook) you are responsible for handling the asynchronous results. For instance, the Ember Data store is a service that does return data asynchronously
But if you hit cases where you do need to do async in a component, the recommended addon to help with that is Ember Concurrency: http://ember-concurrency.com/docs/introduction/
Ember Concurrency helps solve many of the async bugs and edge cases that you haven’t yet hit, but will, if you start doing async code in components. So I’d highly recommend learning more about it.
Good luck!
Fairly new to kotlin.
I'm testing a http client which executes the calls asynchronously and takes a callback as argument from the caller.
Now, let's assume the caller is a unit test. Asserting INSIDE the callback doesn't work obviously because it runs in some kind of background io thread (using Kluent for asserts, I really like the syntax).
client.query(identifier, uiCallback = {
System.out.println("Found a total of ${it.total} entries with identifier!")
signal.countDown()
it.total shouldBe 1
})
I could defer the assert to outside of the callback, but i think that's not very elegant.
In an android context with java i would have done something like "runOnUIThread" (before RX). I also have to use that api in android, so I'll run into that problem anyways.
I must be missing sg. very basic. Is is state of the art to use junit for kotlin anyways or is there a better way to go?
EDIT: That "signal" there is a CountdownLatch to wait for the asynchronous execution.
EDIT 2: Retrofit service definition:
#GET("Entity")
fun query(#Query("identifier") identifier: Array<String?> = arrayOfNulls(0),
#Query("_count") count:Int = 10): Call<ResponseBody>
Should Lambda functions call other Lambda functions or should they be self contained?
My environment is
Serverless Framework
Nodejs
AWS API Gateway
AWS Lamda
AWS DynamoDB
I've build several CRUD for API resources each Dynamo table and now I'm creating some specialized ones that cross tables.
If I have a function createTeamForecast, and I need to get a single row from table Team, should I import the function getTeam or just write the Dynamo query. I lean to importing the function, but I haven't see anything saying that is OK.
getTeam.js
import * as dynamoDbLib from "./libs/dynamodb-lib";
import { apiResponse } from "./libs/response-lib";
export async function main(event, context, callback) {
const params = {
TableName: "teams",
Key: {
id: event.pathParameters.team_id
}
};
try {
const result = await dynamoDbLib.call("get", params);
if (result.Item) {
// Return the retrieved item
callback(null, apiResponse(200,"OK",result.Item));
} else {
callback(null, apiResponse(404, "Team not found."));
}
} catch (e) {
callback(null, apiResponse(500,'Server error',e));
}
}
In my createTeamForecast, can I just import that function and then call it.
import { main as getTeam } from "./getTeam";
My alternative is to just do a Dynamo get and check results within my createTeamForecast.js function. That's more self contained, but not very DRY.
The way that Serverless and Lambda manage the functions, it feels a little disconnected. Anyone have any pros or cons?
It's reasonable to import the code you need from another module rather than rewriting it. This has the side benefit of making it easier to maintain your application because you won't have duplicated logic all over the place.
The trick with serverless applications is finding the balance between code re-use and separation of concerns. The specifics of how to do this are somewhat application dependent. However, if you're putting too much code into each function then it's likely that your application is too tightly coupled and could use decomposition into smaller functions that more tightly model their problem space. If you find large swaths of shared code within your Lambda functions that might be a good indicator that they should be refactored into other functions.
If you're modeling really complex business domains then you may also want to consider calling other Lambda functions from within Lambda functions or investigating AWS Step Functions which provide a state machine on top of Lambda.
I added some razor views and use a request filter to check browser version and switch between desktop and mobile views. But when a exception occurs, especially validation exception, it seems the framework return immediately and never touched any custom code. I tried request/response filter, service exception handler, none got executed. It seems to ignore view/template specified in URL query string as well.
Is there way to set view/template during exception? Thanks
The first question is how are you handling validation exceptions?
the most common procedure to perform this kind of task is by using the fluentValidation, the response can return a message for more than one validation at the time, all the validations are against DTOs and you´ll need to implement an AbstractValidator, the first thing you need to do is to register the validators that belons to your applciation like the following:
Plugins.Add(new ValidationFeature());
container.RegisterValidators(typeof(CredentialsAuthValidator).Assembly);
I´m valdiating in this case that the Auth username and password should not be Empty, take a look to the following example:
public class CredentialsAuthValidator : AbstractValidator<ServiceStack.ServiceInterface.Auth.Auth>
{
public CredentialsAuthValidator()
{
RuleSet(ApplyTo.Post, () =>
{
RuleFor(x => x.UserName).NotNull().WithMessage("Username Required").When(x => x.provider == "Credentials");
RuleFor(x => x.Password).NotNull().WithMessage("Password Required").When(x => x.provider == "Credentials");
}
);
}
}
if some of the validation fails you´ll get a responseStatus from the server with the errorCode and the messages.
You can configure a custom httpHandlers in the case you would like to have a handler for specific scenarios or a global error handler, this can be performed in your serviceHost configuration, something like this:
GlobalHtmlErrorHttpHandler = new RazorHandler("/views/error"),
CustomHttpHandlers =
{
{HttpStatusCode.NotFound, new RazorHandler("/views/notfound")},
{HttpStatusCode.Unauthorized, new RazorHandler("/views/login")},
{HttpStatusCode.Forbidden, new RazorHandler("/views/forbidden")},
}
Thanks for the help from Pedro, and especially mythz from ServiceStack. Now I think I start to understand my problems.
ServiceStack is first and foremost a service framework and Razor is just another view over the same result. But I was a little hesitate with a full on client side solution and keep falling back to familiar territory and looking for some kind of code-behind feature. That seems to be the root of lots of my struggles.
After some more research, this is what I come up so far.
ServiceStack for service, of course.
Razor view to build the basic layout and the main page for each major feature
Build a json script tag from model to hold initial data, like in SS's HTML Report
Jquery and Eldarion ajax for all subsequent in-page processing
Handlebars for javascript templating
Verifyjs for validation
So far look promising. Pages are lot smaller in size, running super smooth and mostly pure json flying over the wire.
Still a work in progress, all suggestions welcome.
ViewSwitch works when I changed to use Request Filter instead. Got the correct layout and all the references, etc. Although they have to share the same error page, but there's not much formatting in there.