CloudWatch Custom logs are not rendering properly - amazon-web-services

When I try to push my IIS or MSSQL logs into CloudWatch, I can see logs in the server are appearing however they are in the single line in CW where as in the servers they are two different events with different timestamp
I've treid using "multi_line_start_pattern": "yyyy-MM-dd HH:mm:ss" however this doesn't solve my problem
CloudWatch Json file:
{
"FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Id": "IISLogs",
"Parameters": {
"CultureName": "en-US",
"Encoding": "UTF-8",
"Filter": "",
"LineCount": "5",
"LogDirectoryPath": "C:\\logfiles",
"TimeZoneKind": "UTC",
"TimestampFormat": "\\%Y-%m-%d %H:%M:%S\\" (also tried "yyyy-MM-dd HH:mm:ss" format)
}
},
{
"FullName": "AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch",
"Id": "CloudWatchIISLogs",
"Parameters": {
"LogGroup": "/application/iis",
"LogStream": "{instance_id}",
"Region": "eu-west-1",
"multi_line_start_pattern": "yyyy-MM-dd HH:mm:ss"
}
}
under flows:
"(IISLogs),CloudWatchIISLogs",
Logs I see in CW: I see its not finding difference between each end line, however in the IIS server I do have the logs seperated in next line. same is happening for MSSQL.
I would expect the logs to be pushed into the CW same as mentioned in the server/instance unlike below:
Under time: I have the timestamp:
Under Message: this is coming under single message where as it consists of multiple messages (3 events of user1)
2019-05-31 12:19:42 ::1 GET / - 80 user ::1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko - 200 0 0 2032019-05-31 12:19:43 ::1 GET / - 80 user1 ::1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko - 200 0 0 152019-05-31 12:19:43 ::1 GET /libs/jquery-1.7.1.min.js - 80 user1 ::1 Mozilla/5.0+(Windows+NT+10.0;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko http://localhost/ 304 0 0 02019-05-31 12:19:43 ::1 GET /libs/canvg/canvg.js - 80 user1 ::1
status code is merging with the next line which is date/time due to which logs are not showing/split up properly.
Any help would be appreciated.
Thanks

I have got an answer to this, this was due to the agent that we were using - SSM, post migration to CW agent its resolved.

Related

Using graph-notebook to connect to Blazegraph Database

I tried to use https://github.com/aws/graph-notebook to connect to Blazegraph Database. I have verified Blazegraph is running from the following.
serviceURL: http://192.168.1.240:9999
Welcome to the Blazegraph(tm) Database.
Go to http://192.168.1.240:9999/bigdata/ to get started.
I did the following the jupyter notebook
%%graph_notebook_config
{
"host": "localhost",
"port": 9999,
"auth_mode": "DEFAULT",
"iam_credentials_provider_type": "ENV",
"load_from_s3_arn": "",
"aws_region": "us-west-2",
"ssl": false,
"sparql": {
"path": "blazegraph/namespace/foo/sparql"
}
}
then do the following
%status
gives error {'error': JSONDecodeError('Expecting value: line 1 column 1 (char 0)',)}
I tried to replace host with 192.168.1.240 and is still having the same problem.
Looks like you found a bug!
This is now fixed in https://github.com/aws/graph-notebook/pull/137.

Looking for REST API (s) in google cloud to pull health, cpu load etc information of servers deployed in kubernetes

i have a SpringBoot application which is showing helath of all the servers in react charts today. we have some applications(servers) deployed to GCP using Kubernetes. i would like to pull and show health of the servers, number of pods, cpu utilization etc in my spring boot application. i have searched all GKE related REST apis in documentation, how ever i found REST urls at https://container.googleapis.com. but, none of them are seems to help me. please help me find the set of REST api's to fetch the above said heath statistics.
You can follow the documentation
You will find all info you need like cpu utilization and other useful metrics
The "metric type" strings in this table must be prefixed with actions.googleapis.com/
Metric type: instance/cpu/utilization:
Fractional utilization of allocated CPU on this instance. Values are typically numbers between 0.0 and 1.0 (but some machine types allow bursting above 1.0). Charts display the values as a percentage between 0% and 100% (or more). This metric is reported by the hypervisor for the VM and can differ from agent.googleapis.com/cpu/utilization, which is reported from inside the VM. Sampled every 60 seconds. After sampling, data is not visible for up to 240 seconds.
instance_name: The name of the VM instance
Creating the GET request
#Raj: This is not the url for the get request, check this tutorial, you want to format your get request the following way (change parameters depending on your own values):
curl -X GET -H "Authorization: Bearer $TOKEN"\
"https://monitoring.googleapis.com/v3/projects/{{YOUR_PROJECT}}/timeSeries/?filter=metric.type+%3D+%22compute.googleapis.com%2Finstance%2Fcpu%2Futilization%22&\
interval.endTime=2017-01-30T21%3A45%3A00.000000Z\
&interval.startTime=2017-01-30T21%3A43%3A00.000000Z"
{
"timeSeries": [
{
"metric": {
"labels": {
"instance_name": "evan-test"
},
"type": "compute.googleapis.com/instance/cpu/utilization"
},
"resource": {
"type": "gce_instance",
"labels": {
"instance_id": "743374153023006726",
"zone": "us-east1-d",
"project_id": "evan-testing"
}
},
"metricKind": "GAUGE",
"valueType": "DOUBLE",
"points": [
{
"interval": {
"startTime": "2017-01-30T21:44:01.763Z",
"endTime": "2017-01-30T21:44:01.763Z"
},
"value": {
"doubleValue": 0.00097060417263416339
}
},
{
"interval": {
"startTime": "2017-01-30T21:43:01.763Z",
"endTime": "2017-01-30T21:43:01.763Z"
},
"value": {
"doubleValue": 0.00085122420706227329
}
}
]
},
...
]

Where is the GCP Cloud Scheduler HTTP body?

I am trying to work with a cron job on GCP Cloud Scheduler. I am using the HTTP target with the "GET" method.
I am trying to post messages to a discord channel but first need to GET the body my server webhook sends back to me. The CRON job runs successfully but I cannot find the body of what the webhook returned on the GCP Cloud Scheduler. I have checked the logs as well, it does not contain the body. Here is what the log has:
{
"insertId": "a06j1cfzy21xe",
"jsonPayload": {
"targetType": "HTTP",
"jobName": "projects/website-274422/locations/us-central1/jobs/discord_sec_bot",
"url": "https://discordapp.com/api/webhooks/<redacted>/<redacted>",
"#type": "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
},
"httpRequest": {
"status": 200
},
"resource": {
"type": "cloud_scheduler_job",
"labels": {
"project_id": "website-274422",
"job_id": "discord_sec_bot",
"location": "us-central1"
}
},
"timestamp": "2020-08-10T21:42:13.290867117Z",
"severity": "INFO",
"logName": "projects/website-274422/logs/cloudscheduler.googleapis.com%2Fexecutions",
"receiveTimestamp": "2020-08-10T21:42:13.290867117Z"
}
Could anyone tell me where I could find what my GET request received?
Although it's not mentioned directly in the documentation I don't think it's possible to see this. I am not sure what do you want to do, however if you need any information to pass to the logs you can use response status. I have done quick test on my cloud function, which was randomly sending response status from 200 to 204.
For each job I get 2 different log items. In the 2nd one there is following field with random status:
httpRequest: {
status: 201
}
According this is only chance to see anything returned by the endpoint to logs. You can use this status to code some information.

ELB health check failing

an instance was taken out of service in response to a ELB system
health check failure.
I hit the health check endpoint with my browser and it returns fine, but I'm getting the above message.
How can I debug this?
I've looked at instant settings => Get System Logs and nginx logs,
edit
nginx has
- [27/Mar/2020:05:35:42 +0000] "GET /littlehome/heartbeat/ HTTP/1.1" 200 2 2.920 2.920 "-" "ELB-HealthChecker/2.0"
- [27/Mar/2020:05:35:42 +0000] "GET /littlehome/heartbeat/ HTTP/1.1" 200 2 2.858 2.856 "-" "ELB-HealthChecker/2.0"
it returned 200 for sure..
and still aws think it received 502
{
"Target": {
"Id": "i-085e8dffe8781f876",
"Port": 80
},
"HealthCheckPort": "80",
"TargetHealth": {
"State": "unhealthy",
"Reason": "Target.ResponseCodeMismatch",
"Description": "Health checks failed with these codes: [502]"
}
},
Based on the comments, the issue was that grace period in Auto Scaling Group was too short. The solution was to increase it.

Ansible task based on debug output

I'm using Ansible to run few test cases against our web services. Following is my playbook:
- hosts: localhost
connection: local
tasks:
- name: Testing the API...
uri:
url: https://api.example.com/v1/
method: GET
user: username
password: password
status_code: 200
return_content: yes
force_basic_auth: yes
register: results
- debug: var=results
Above playbook works fine and returns the following output:
ok: [localhost] => {
"results": {
"access_control_allow_origin": "*",
"cache_control": "max-age=864000, private",
"changed": false,
"connection": "close",
"content": "\r\n{\"id\":1,\"deleted\":false,\"first-name\":\"xxx\",\"last-name\":\"xxx\",\"name\":\"xxx\",\"title\":\"xxx\",\"background\":\"\",\"company-id\":xx,\"company-name\":\"example\",\"company-type-id\":2,\"company-type-name\":\"Customer\",\"email-address-work\":\"kk#example.info\",\"email-address-personal\":\"\",\"email-address-alternate\":\"\",\"email-address-other\":\"\",\"phone-number-work\":\1234567890\",\"phone-number-home\":\"\",\"phone-number-mobile\":\"252654789\",\"phone-number-alternate\":\"256471235\",\"business-street\":\"526574, usa\",\"business-city\":\"San Antonio\",\"business-state\":\"TX\",\"business-zip\":\"1234607\",\"business-country\":\"United States\",\"home-street\":\"\",\"home-city\":\"\",\"home-state\":\"\",\"home-zip\":\"\",\"home-country\":\"\",\"created-time\":\"2015-11-03T20:56:33.000Z\",\"last-modified-time\":\"2017-11-21T06:27:55.000Z\",\"tags\":[]}",
"content_length": "857",
"content_type": "application/json",
"date": "Tue, 21 Nov 2017 09:59:34 GMT",
"expires": "Fri, 01 Dec 2017 09:59:34 GMT",
Now, I want to run another task if there is any data outside the flower brackets of the 'content' section. In the above output, it starts with 'content": "\r\n{\"id\":1,\"deleted...'. Any idea how I can achieve this?
This condition should suffice:
when: "(results.content | regex_replace('{.*}', '')) != ''"
Just check if the string is empty after removing everything between { and }.