I would like to be disable a web service inbound-gateway based on a database flag.
I have achieved this with other inbound adapters (e.g. file) by setting the auto-start attribute based on a SPEL expression which gets the database value.
Any advice on a good way to achieve this? I do not see an auto-start attribute on the ws:inbound-gateway.
Well, looks like it is a bug there around Lifecycle and in case of stopped state we should return to the client something like HttpStatus.SERVICE_UNAVAILABLE.
Please, raise a JIRA ticket on the matter and we will take care about that soon.
I see that <int-http:inbound-gateway> has similar problem, even if auto-startup is exposed there.
As a workaround I see something like ChannelInterceptor on the request-channel for you <int-ws:inbound-gateway>, which checks some variable and throws some exception (NoEndpointFoundException ?) from the preSend() implementation to notify WS client that the service isn't available.
Related
I have a client with a guaranteed execution timeout setting here (which can be configured per request)
https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/AmazonWebServiceRequest.html#setSdkClientExecutionTimeout-int-
But I cannot find an equivalent for the SDKV2, sync or async.
I was wondering if anyone in SO or AWS would know about this. Is this an intentional feature drop? Or am I missing some other setting.
Found the solution : https://github.com/aws/aws-sdk-java-v2/pull/657#pullrequestreview-799397170. This is for async clients and at client level. if you want to do request level (for async clients), use the orComplete functionality of completable future instead. https://docs.oracle.com/javase/9/docs/api/java/util/concurrent/CompletableFuture.html#orTimeout-long-java.util.concurrent.TimeUnit-
Has anyone used the sns_publish_operator?
https://airflow.apache.org/docs/stable/_modules/airflow/contrib/operators/sns_publish_operator.html
I am quite new to airflow and am having some issues around setting up the architecture correctly.
I have set up a simple DAG with a data quality check task. Basically, if the dataset fails the data quality checks, I'd like to send an SNS notification. If it passes the data quality checks, I'd like it to reframe from sending an email.
There does not seem to be as much online help in this realm as I thought. Any resources or general tips would be much appreciated.
This question is a bit older, but maybe this helps someone still.
First addressing the SnsPublishOperator: You will need to setup an Airflow connection to AWS. There is multiple ways to do that. The easiest is probably using the Web UI. Go there to Admin->Connections->[+] (add new record). Then you set it up as 'Amazon Webservices Connection' Conn Type. Login and Password are the AWS key and key secret. Finally, you also have to provide the region, where you SNS topic is seated in the 'Extras' section:
{"region_name": "us-east-1"}
Now, you can use the Operator in you code, by providing it also the new connection's conn_id:
my_sns_task = SnsPublishOperator(
task_id='task_name',
target_arn='your_sns_topic_arn',
message='your_message',
aws_conn_id=conn_id
)
Theoretically, that operator also has a "subject" parameter, but I receive an error from the component, when I try to set it.
As for incorporating that operator into your DAG, a possible way would be to have the task, that evaluates your data, fail, if the checks to do not pass, and have the sns task be triggered on failure:
my_sns_task = SnsPublishOperator(
task_id='task_name',
target_arn='your_sns_topic_arn',
message='your_message',
aws_conn_id=conn_id,
trigger_rule='one_failed'
)
my_sns_task.set_upstream(datacheck_task)
So, this problem is happening randomly (it seems) and between different services.
For example we have a service A which needs to talk to service B, and some times we get this error, but after a while, the error goes away. And this error doesn't happen too often.
When this happens, we see the error log in service A throwing the “upstream connect error” message, but none in service B. So we think it might be related with the sidecars.
One thing we notice is that in service B, we get a lot of this error messages in the istio-proxy container:
[src/istio/mixerclient/report_batch.cc:109] Mixer Report failed with: UNAVAILABLE:upstream connect error or disconnect/reset before headers. reset reason: connection failure
And according to documentation when a request comes in, envoy asks Mixer if everything is good (authorization and other things), and if Mixer doesn’t reply, the request is not success. So that’s why exists an option called policyCheckFailOpen.
We have that in false, I guess is a sane default, we don’t want the request to go through if Mixer cannot be reached, but why can’t?
disablePolicyChecks: true
policyCheckFailOpen: false
controlPlaneSecurityEnabled: false
NOTE: istio-policy is running with the istio-proxy sidecar. Is that correct?
We don’t see that error in some other service which can also fail.
Another log that I can see a lot, and this one happens in all the services not running as root with fsGroup defined in the YAML files is:
watchFileEvents: "/etc/certs": MODIFY|ATTRIB
watchFileEvents: "/etc/certs/..2020_02_10_09_41_46.891624651": MODIFY|ATTRIB
watchFileEvents: notifying
One of the leads I'm chasing is about default circuitBreakers values. Could that be related with this?
Thanks
The error you are seeing is because of a failure to establish a connection to istio-policy
Based on this github issue
Community members add two answers here which could help you with your issue
If mTLS is enabled globally make sure you set controlPlaneSecurityEnabled: true
I was facing the same issue, then I read about protocol selection. I realised the name of the port in the service definition should start with for example http-. This fixed the issue for me. And . if you face the issue still you might need to look at the tls-check for the pods and resolve it using destinationrules and policies.
istio-policy is running with the istio-proxy sidecar. Is that correct?
Yes, I just checked it and it's with sidecar.
Let me know if that help.
We have deployed API-M 2.1 in a distributed way (each component, GW, TM, KM are running in their own Docker image) on top on DC/OS 1.9 ( Mesos ).
We have issues to get the gateway to enforce throttling policies (should it be subscription tiers or app-level policies). Here is what we have managed to define so far:
The Traffic Manager itself does it job : it receives the event streams, analyzes them on the fly and pushes an event onto the JMS topic throttledata
The Gateway reads the message properly.
So basically we have discarded a communication issue.
However we found two potential issues:
In the event which is pushed to the TM component, the value of the appTenant is null (instead of carbon.super)- We have a single tenant defined.
When the gateway receives the throttling message, it decides to let the message go thinking the "stopOnQuotaReach" is set to false, when it is set to true (we checked the value in the database).
Digging into the source code, we related those two issues to a single source: the value for both values above are read from the authContext and apparently incorrectly set. We are stuck and running out of ideas of things to try and would need some pointers to what could be a potential source of the problem and things to check.
Can somebody help please ?
Thanks- Isabelle.
Is there two TM with HA enabled available in the system?
If the TM is HA enabled, how gateways publish data to TM. Is it load balanced data publishing or failover data publishing to the TMs?
Did you follow below articles to configure the environment with respect to your deployment?
http://wso2.com/library/articles/2016/10/article-scalable-traffic-manager-deployment-patterns-for-wso2-api-manager-part-1/
http://wso2.com/library/articles/2016/10/article-scalable-traffic-manager-deployment-patterns-for-wso2-api-manager-part-2/
Is throttling completely not working in your environment?
Have you noticed any JMS connection related logs in gateways nodes?
In these tests, we have disabled HA to avoid possible complications. Neither subscription nor app throttling policies are working, both because parameters that should have values have not the adequate value (appTenant, stopOnQuotaReach).
Our scenario is far more basic. If we go with one instance of each component, it fails as Isabelle described. And the only thing we know is that both parameters come from the Authentication Context.
Thank you!
Is there a way that I can retrieve the timestamp of a web service call? I'm trying to get the time of the server hosting the web service.
Easiest thing to do is to just log them in the server implementation of your service contract, you can use PostSharp to make some attributes to take of this aspect.
For instance, you can write a Trace attribute which simply logs a debug message when a method is invoke. Here's one I wrote a while back which tracks how long a method takes and log a warning message if it takes longer than a set threshold:
http://theburningmonk.com/2010/03/aop-method-execution-time-watcher-with-postsharp/
I came across some 'trace' attribute example before, if you want I can look for it for ya.