Graylog2 Alarms for non admins - alarms

How can unprivileged users subscribe to alarms?
Administrators see the alarm tab on each stream. It lets you subscribe in order to get an email if a message limit per timeframe is hit. For "readers" the tab is invisible.

This is not possible in the currently released versions. However, v0.20.0 is released these days (Preview version are already available) that will allow you to do that.

Related

Reduce alert noise in GCP stackdriver

We have set up alerts in my GCP environments. Basically GCP Stackdriver will raise alerts based on certain parameters which we configured (both at infrastructure level and application level).
The issue is that we are getting too many alerts, if the problem is not resolved quickly enough. For example, if a compute engine is down, we are investigating and still we get alerts. Looking for some help to reduce alert noise so that once we acknowledge an issue, the alert frequency should be reduced till we resolve the issue (maybe once every three hours rather than sending one mail each for every 10 minutes OR after the problem is fixed).
Posting this as an answer for better usability.
When the alert is triggered you will be receiving notifications every 10 minutes or so until you acknowledge the incident.
When you do notifications will stop coming, but the incident will be kept open until you close it.
You can also silence the incident, however it may & will close other incidents that were triggered by the same condition that triggered this one.
You may also have a look at the alerting behavior docs since they may prove useful in such cases.

AWS CloudWatch: Repeat Alert Notification after 24h of Alertstate

i created some AWS CW Alerts which have typically a time periode of 1 hour / 1 Datapoint. By occuring an Alert our Service Team has been notificated. During a "normal" workday, someone cares about it and do the work of resetting some programms etc. But it also happens, no one have time or sense to care and the alert keeps in the alert state.
Now i want to repeat the alert if there wasn't any state-change in the past 24 hours. It is possible? I still does not find the "easy" answer.
Thx!
EDIT:
Added a "daily_occurence_alert" which is controlled by eventrules / time control. An additional alert for each observed Alert combined with an AND serves good.
It is a workaround, not a solution. Hope this feature will be added as a standard in future.

Amazon Connect Stop Call Recording

Is it possible to stop call recordings in Amazon Connect so the customer and agent can discuss sensitive material without being recorded?
I am aware of the set call recording behaviour blocks, but they don't seem to work on a call that has already been started with an agent with call recording enabled. Transferring to another contact flow with the recording type set to none doesn't seem to make a difference and the call carries on being recorded.
I am aware of the sample workflow Sample secure input with agent as outlined in this AWS blog https://aws.amazon.com/premiumsupport/knowledge-center/disable-recording-amazon-connect. This does work, however it relies on the customer entering payment details whilst the agent is on hold - preventing the agent and customer from having a sensitive conversation.
It seems the only way to stop recording once it has been enabled is to put the agent on hold?
Do not know if you have not solved your issue yet, but amazon has update their Amazon Connect API that would allow you to suspend the recording.
Boto3 implementation
response = client.suspend_contact_recording(
InstanceId='string',
ContactId='string',
InitialContactId='string'
)
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/connect.html#Connect.Client.suspend_contact_recording
They have also allow you to Start, Pause, Stop. (
We have just started to review this for a POC, turn recording off be default for a group of queues. Allow to Agents to start and stop and pause recording as needed.
You can also read this in an Amazon Blog post that should be able to help you fully implement the solution.
https://aws.amazon.com/blogs/contact-center/pausing-and-resuming-call-recordings-with-a-new-api-in-amazon-connect/#:~:text=is%20not%20recorded.-,End%20the%20call.,you%20start%20and%20stop%20it.
After speaking with Architects at AWS, the desired and designed for solution is to have the customer automatically enter sensitive information with the agent on hold and call recording turned off to remain PCI compliant.
If that is not an option there are workarounds possible that go against the way Amazon Connect has been designed. In order to turn off call recording once it has been enabled on a call, a new contact ID must be established. To do this you would need to transfer the user to your external phone number again or transfer to a queue and disable call recording in that new flow.
This brings in extra issues around how to get the customer back to the original agent once the sensitive information has been discussed. It also means you would potentially have 3+ contact IDs for the same transaction, with call recording spread across them.

Selecting message queue approach for multiple consumers in AWS

Please help selecting a MQ app/system/approach for the following use-case:
Check for incoming messages for a specific user -> read the message if available -> delete from the queue, ideally, staying within AWS.
Context:
Social networking app, users receiving messages, i.e.
I need to identify incoming messages by recipient ID.
The app is doing long-polls for new messages every 30 seconds.
Message size is <1Kb.
As per current estimates, I'll need 100M+ message checks per months in total (however, much less messages, these are just checks).
While users acknowledge messages choosing OK or Ignore, however not sure if ACK support is required from MQ system for that.
I'm in AWS. Initially thought of SQS, but the more I read the less it looks like a good match - cannot set message recipient ID in a way to filter by recipient, etc, however maybe I'm wrong.
One of the options I also thought about is to just use DynamoDB's "messages" table, partition key being userId and sort key being a messageId, thus I'll be able to easily query by a user, however concerned with costs.
If possible, I would much more prefer to stay within AWS or at least use SAAS like SQS, as being a 1-person startup I really want to avoid headaches supporting self-hosted system.
Thank you!
D
You are right on both these counts:
SQS won't work, because of the limitation you pointed.
DynamoDB would work, but cost a lot.
I can suggest the following:
Create a Redis cluster, possibly on Amazon ElastiCache.
In it, make one List per user.
Whenever a new message comes, append it to concerned User's list.
To deliver the message, just read from the User's list. Also, flush the queue if needed.
What I am suggesting is very similar to how Twitter manages each User's news-feed and home-feed.
It should also be cheap.

Avoiding INSUFFICIENT DATA in Cloudwatch?

I have alarms set up to tell me when my load balancers are throwing 5xxs using the HTTPCode_Backend_5XX metric with the sum statistic. The issue is that sum registers 0 as no data points, so when no 5xxs are thrown, the alarm is treated as insufficient data. This is especially frustrating, because I have SNS setup to notify me whenever we get too many 5xxs (alarm state) and whenever things go back to normal. Annoyingly, 0 5xxs means we're in INSUFFICIENT DATA status, but 1 5xx means we're in OK status, so 1 5xx triggers everyone getting notified that stuff is OK. Is there any way around this? Ideally, I'd like to just have 0 of anything show up as a zero data point instead of no data at all (insufficient data).
As of March 2017, you can treat missing data as acceptable. This will prevent the alarm from being marked as INSUFFICIENT.
You can also set this in CloudFormation using the TreatMissingData property. For example:
TreatMissingData: notBreaching
We had a similar issue for some of our alarms. You can actually avoid this behaviour with some work, if you really want to deal with the overhead.
What we have done is, instead of sending SNS notifications directly to e-mails, we have created a lambda function and triggered it once we have the notification in the SNS topic.
This way, you will have more control over the actions you can take once the alarms are triggered. As the context will provide you old state value as well.
The good news is, there is already a lambda template to get started.
https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/
Just pick the one that is designed to send cloudwatch alarms to slack. You can then modify the code as you wish, either dismiss the slack part and just use emails, or keep it with slack. (which is what we did and it works like a charm)
I asked for this in the AWS forums two years ago :-(
https://forums.aws.amazon.com/thread.jspa?threadID=153753&tstart=0
Unfortunately you cannot create notifications based on specific state changes (in your case you want a notification when state changes from ALARM to OK, but not when state changes from INSUFFICIENT to OK). I can only suggest that you also ask for it and hopefully it will eventually be added.
For metrics that are often in the INSUFFICIENT state I generally just create notifications for ALARMS and I don't have notifications on OK for these metrics - if I want to confirm that things are OK I use the AWS mobile app to check on things and see if they have resolved.