GCP K8s No Scale Up Events Log Based Metrics - google-cloud-platform

I am trying to create a custom alert from a log-based metric in GCP for a K8s node pool no.scale.up event.
I am using the current query:
resource.labels.project_id="my_project"
resource.labels.location="my_zone"
resource.labels.cluster_name="my_cluster"
logName="projects/my_project/logs/container.googleapis.com%2Fcluster-autoscaler-visibility"
jsonPayload.noDecisionStatus.noScaleUp.reason.messageId="no.scale.up.in.backoff" OR
jsonPayload.noDecisionStatus.noScaleUp.napFailureReason.messageId="no.scale.up.nap.disabled" OR
jsonPayload.noDecisionStatus.noScaleUp.napFailureReason.messageId="no.scale.up.nap.no.locations.available" OR
jsonPayload.noDecisionStatus.noScaleUp.skippedMigs[].reason.messageId="no.scale.up.mig.skipped" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].rejectedMigs[].reason.messageId="no.scale.up.mig.failing.predicate" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].napFailureReasons[].reason.messageId="no.scale.up.nap.pod.gpu.no.limit.defined" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].napFailureReasons[].reason.messageId="no.scale.up.nap.pod.gpu.type.not.supported" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].napFailureReasons[].reason.messageId="no.scale.up.nap.pod.gpu.other.error" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].napFailureReasons[].reason.messageId="no.scale.up.nap.pod.zonal.resources.exceeded" OR
jsonPayload.noDecisionStatus.noScaleUp.unhandledPodGroups[].napFailureReasons[].reason.messageId="no.scale.up.nap.pod.zonal.failing.predicates"
Its a pretty exhaustive query that does not look right to me.
This query works and it does extract data from the logs.
Reading the documentation here: https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler-visibility#eventresult-event
it talks about eventResult's but I cannot figure out how to get to these?
They do not appear to be in the cluster-autoscaler-visibility log.
Are the eventResults in this log: logName="projects/my_project/logs/events"?
(If so I could not find them with any query)
I have tried a query:
resource.type="k8s_cluster"
resource.labels.project_id="my_project"
resource.labels.location="my_zone"
resource.labels.cluster_name="my_cluster"
logName="projects/my_project/logs/events"
jsonPayload.resultInfo.results[].errorMsg = "scale.down.error.failed.to.mark.to.be.deleted" OR
jsonPayload.resultInfo.results[].errorMsg = "scale.down.error.failed.to.evict.pods" OR
jsonPayload.resultInfo.results[].errorMsg = "scale.down.error.failed.to.delete.node.min.size.reached" OR
jsonPayload.resultInfo.results[].errorMsg = "scale.up.error.out.of.resources" OR
jsonPayload.resultInfo.results[].errorMsg = "scale.up.error.quota.exceeded" OR
jsonPayload.resultInfo.results[].errorMsg = "scale.up.error.waiting.for.instances.timeout"
This does not produce anything useful even though I have no.scale.down and no.scale.up events searchable in the log explorer.

Related

Offline Maps HERE-SDK - offline search and navigation

I am using HERE-Android-SDK to build a simple navigation solution with offline maps.
While using the offline mode for the search addresses and calculation of a route, I can see that there are results returned from the address-search, which are not included in the installed offline map datasets. Is there anything additional which I need to do, in order to get only search results which are located inside the offline-map data installed on my device?
I am using the following code snippets.
download offline maps for a specific country:
mapsLoader.selectDataGroup(MapPackage.SelectableDataGroup.TruckAttributes)
mapsLoader.installMapPackages(listOf(mapPackageId))
search request for addresses:
val term = "New York"
val center = GeoCoordinate(lastWayPoint.latitude, lastWayPoint.longitude)
val request = SearchRequest(term)
request.connectivity = Request.Connectivity.OFFLINE
request.locale = Locale.GERMAN
request.setSearchCenter(center)
request.collectionSize = 5
request.execute { data, error ->
if (error != ErrorCode.NONE) return
// handle search results here
}
Thanks for all of your help in advance!

CloudWatch Metric Filter for checking JSON key exists

I'm trying to come up with a metric filter expression that filters CloudWatch Logs when a special JSON key attribute is present.
Use case is the following: the application does all kinds of logging(in JSON format) and whenever it has a special JSON key(nested JSON response from third-part service), I would like to filter it.
Example logs:
{"severity":"INFO","msg":"EVENT","event":{"key1":"value1"}}
{"severity":"INFO","msg":"FooService responded","response":{"response_code":800}}
Filter patterns that I've tried that don't work:
{ $.response }
{ $.response = *}
{ $.response = "*"}
{ $.response EXISTS }
{ $.response IS TRUE }
{ $.response NOT NULL }
{ $.response != NULL }
Expected filtering result:
{"severity":"INFO","msg":"FooService responded","response":{"response_code":800}}
{ $.response EXISTS } does the opposite of what I expect(returns the 1st line rather than then 2nd) but I'm not sure how to negate it.
Reference material: Filter and pattern syntax # CloudWatch User Guide
I haven't found a good solution.
But I did find one at least.
If you search for a key being != a specific value, it seems to do a null check on it.
So if you say:
{$.response != "something_no_one_should_have_ever_saved_this_response_as"}
Then you get all entries where response exists in your json, and where it's not your string (hopefully all of the valid entries)
Definitly not a clean solution, but it seems to be pretty functional
I don't have a solution to the task of finding records where a field exists. Indeed, the linked document in the question specifically calls this out as not supported.
but
If we simply reverse our logic this becomes a more tractable problem. Looking at your data, you want All records where there's a response key but that could also be stated as All records where there isn't an events key.
This means you could accomplish the task with {$.event NOT EXISTS}. Of course, this becomes more complicated the more types of log messages you get (I had to chain three different NOT EXISTS queries for my use case) but it does solve the problem.

How to get item attributes via api?

We are using cloud Dynamics 365 Business Central and trying to get items with all attributes via OData.
In Microsoft documentation we found this endpoint:
api.businesscentral.dynamics.com/v1.0[our tenant id]/Sandbox/ODataV4/Company('CRONUS%20DE')/items
But unfortunately, the response does not contain item attributes and values, such as Farbe, Tiefe, etc.
Next, we tried to add new Web Services. But some of this endpoints return empty values and some of them (7506, 7507, 7508, 7510) don't work and return:
No HTTP resource was found that matches the request URI
Objects 7500, 7501, 7503 contain information about attributes. But non of them (7500 - 7510) does not contain a relation between Item, Attributes, and Values.
Maybe there is another way to get items with their attribute values? We also tried to research microsoft graph but not successful.
i am having similar troubles with this. i find the dynamics api to be exceptionally unintuitive and difficult to use. the furthest i have been able to get has been to go into the api settings for dynamics and uncover the tables for a few item attributes (i believe that the table numbers are as those below:
7500 - Item Attribute
7501 - Item Attribute Value
7502 - Item Attribute Translation
7504 - Item Attribute Value Selection
7505 - Item Attribute Value Mapping
i cannot comment on why 7503 is missing.
using 7500 as an example, when you uncover the table, the system provides a resulting endpoint (unfortunately, they always promote OData, and the outdated SOAP resource; i can't figure out why they have such a vendetta against the simple and easy-to-use REST endpoint).
https://api.businesscentral.dynamics.com/v2.0/<TENANT_ID>/<ENVIRONMENT_NAME>/ODataV4/Company('COMPANY_IDENTIFIER')/ItemAttributes
using this endpoint, you can get a listing of the attribute types themselves (e.g. let's say you've defined an attribute called 'BaseColor', you should get a result here for the name of the attribute 'BaseColor', its ID, its type, etc.),
with the ItemAttributeValues endpoint, you should get the actual attribute values that are in existence (e.g. for some item, you happened to set its 'BaseColor' attribute to 'Blue', you should get a response for this attribute value with a attribute type of 'BaseColor', a value, as 'Blue' along with the entity's ID, etc).
yet, when it comes to any instantiated attribute values for items, i can't figure out how to get the association of the attributes with those items. i expect that the "item attribute value mapping" option would be something along the lines of a item_id - attribute_id pair so that for any item in question, one could query the attributes list with something like a filter. but as you said, upon uncovering some of these elements, their respective endpoints return nothing. you get to the point where you say 'OH...AWSOME! there is a value-item mapping. that makes sense, and i can definitely use that'. a few moments later, the API spits in your face with an error, or craps on you by returning something you don't expect like an empty data set.
this api is a constant uphill battle, and totally riddled with landmines. a complete blow-me-up-pain-in-the-arse.
EDIT: 2021-06-09
i've looked into this some more. i was able to set up an export package for the various tables in question, specifically 7500, 7501, and 7505. the magical table was 7505 as it is the relationship between an attribute value and the item with which it is associated. exporting the package to excel results in good data. yet, when trying to expose this information in the OData resource, something strange happens:
in web services, i try to open up page 7505 which populates the object name as ItemAttributeValueMapping and i set the service name to 'ItemAttributeValueMapping'. This is normal.
the system complains when i fail to specify that the object type is a page. so, i go back in the line and set the selection to "Page"
when i tab through to publish the change, the object name automatically changes to 'ItemAttributeValueTranslations'.
EDIT: 2021-06-15
After a lot of fiddling about, i finally reached a point where i decided that the only way to address this was to write an al query which would expose the appropriate value-item mapping information. there is a page which provides some source code for doing this:
github AL-Code-Samples
to get something out of the API, i had to use microsoft visual studio code. there are a few good videos on how to get this up and running to get a test app working for your business central instance (i used eric hougaar's videos: Getting started with Business Central AL Development).
when you have set up your app to connect to your instance by inserting your tenant and entering your credentials, you can modify the source code as below to create a query in your system.
query 50102 "<YOUR_QUERY_NAME>"
{
QueryType = API;
APIPublisher = '<YOUR_NAME>';
APIGroup = '<YOUR_APP_GROUP>';
APIVersion = 'v1.0';
Caption = '<YOUR CAPTION>';
EntityName = '<YOUR_QUERY_NAME>';
EntitySetName = '<YOUR_API_ENDPOINT_NAME>';
elements
{
dataitem(Item; Item)
{
column(No_; "No.")
{
}
column(Description; Description)
{
}
column(Unit_Cost; "Unit Cost")
{
}
dataitem(Item_Attribute_Value_Mapping; "Item Attribute Value Mapping")
{
DataItemLink = "No." = Item."No.";
column(Item_Attribute_ID; "Item Attribute ID")
{
}
column(Item_Attribute_Value_ID; "Item Attribute Value ID")
{
}
dataitem(QueryElement6; "Item Attribute")
{
DataItemLink = ID = Item_Attribute_Value_Mapping."Item Attribute ID";
column(Name; Name)
{
}
dataItem(Queryelement10; "Item Attribute Value")
{
DataItemLink = "Attribute ID" = Item_Attribute_Value_Mapping."Item Attribute ID",
ID = Item_Attribute_Value_Mapping."Item Attribute Value ID";
column(Value; Value)
{
}
column(Numeric_Value; "Numeric Value")
{
}
}
}
}
}
}
}
once this code gets successfully uploaded to your server and returning a page (you have to wait for it), you can then use specified query number to expose the data in the API by going to business central's "web services" and adding a 'Query' to item 50102 (or whatever number you use). the endpoint will automatically be populated and you can use it to send you back the necessary JSON which will show a product, with its attribute values.
hope that helps.
You should try with below endpoint:
/v2.0/tenant_id/enviornment_name/ODataV4/Company(company_id)/Items

how to get all values of a patricular metrics in sonar REST api

I was able to get the list of projects(resources) from sonar REST API using the following java code
Sonar sonar = Sonar.create("wwww.example.com/sonar/api/resources?metrics=nloc");
ResourceQuery query = new ResourceQuery();
List resourceList = sonar.findAll(query);
for(Resource resource : resourceList){
System.out.println(resource.getid()+":"+resource.getMetricIntValue("nloc"));
}
But this resulted in
1001:null
1002:null
1003:null
1004:null
Why is the lines of code value returning null?
The docs show the metrics parameter with a value of ncloc, not nloc. I think you've just got a typo.

Public Insights

With the new ability for anyone to see certain insights from pages without admin permissions, I was curious if this extends to the graph api, specifically in regards to the graph of fan growth. If you hover your mouse over a data point, it'll tell you the new fans / 7 days of that particular point in time -- is there any way to query this automatically with either fql or the regular graph api?
I've tried SELECT metric, value FROM insights WHERE object_id = 224383614973 AND metric = 'page_fan_adds' AND end_time=1342915199 AND period=86400 to query the McDonald's Canada new fans for a one-day period, but it requires a valid access_token.
Sorry for the late reply but it might help someone. You can use the graph explorer to build your FQL query:
https://developers.facebook.com/tools/explorer/?method=GET&path=CocaColaCanada%2Finsights
Also all the metrics for the page start with page_ and are available here:
https://developers.facebook.com/docs/reference/api/insights/#page_stories
Some of them are not public.
In all cases you need valid access_token
UPDATE:
I've run a script for macdonald using the 233 metrics that facebook offers to see which one are public metrics for a page. Here is the result:
McDonalds(233 metrics)
application_active_users: no
application_active_users_locale: no
application_active_users_city: no
application_active_users_country: no
application_active_users_gender: no
application_active_users_age: no
application_active_users_gender_age: no
application_installed_users: no
application_installed_users_locale: no
application_installed_users_city: no
application_installed_users_country: no
application_installed_users_gender: no
application_installed_users_age: no
application_installed_users_gender_age: no
application_installation_adds: no
application_installation_adds_unique: no
application_installation_removes: no
application_installation_removes_unique: no
application_tos_views: no
application_tos_views_unique: no
application_permission_views_top: no
application_permission_views_top_unique: no
application_permission_grants_top: no
application_permission_grants_top_unique: no
application_block_adds: no
application_block_adds_unique: no
application_block_removes: no
application_block_removes_unique: no
application_mobile_app_installs: no
application_like_adds: no
application_like_adds_unique: no
application_like_removes: no
application_like_removes_unique: no
application_comment_adds: no
application_comment_adds_unique: no
application_photos: no
application_photos_unique: no
application_shares: no
application_shares_unique: no
application_status_updates: no
application_status_updates_unique: no
application_stream_stories: no
application_stream_stories_unique: no
application_feed_form_views: no
application_feed_form_views_unique: no
application_feed_form_views_login: no
application_feed_form_views_login_unique: no
application_feed_form_views_logout: no
application_widget_activity_views: no
application_widget_activity_views_unique: no
application_widget_activity_views_login: no
application_widget_activity_views_login_unique: no
application_widget_activity_views_logout: no
application_widget_activity_views_external_referrals: no
application_widget_comments_views: no
application_widget_comments_views_unique: no
application_widget_comments_views_login: no
application_widget_comments_views_login_unique: no
application_widget_comments_views_logout: no
application_widget_fan_views: no
application_widget_fan_views_unique: no
application_widget_fan_views_login: no
application_widget_fan_views_login_unique: no
application_widget_fan_views_logout: no
application_widget_fan_views_external_referrals: no
application_widget_like_views: no
application_widget_like_views_unique: no
application_widget_like_views_login: no
application_widget_like_views_login_unique: no
application_widget_like_views_logout: no
application_widget_like_views_external_referrals: no
application_widget_live_stream_views: no
application_widget_live_stream_views_unique: no
application_widget_live_stream_views_login: no
application_widget_live_stream_views_login_unique: no
application_widget_live_stream_views_logout: no
application_widget_live_stream_views_external_referrals: no
application_widget_recommendation_views: no
application_widget_recommendation_views_unique: no
application_widget_recommendation_views_login: no
application_widget_recommendation_views_login_unique: no
application_widget_recommendation_views_logout: no
application_widget_recommendation_views_external_referrals: no
application_widget_share_views: no
application_widget_share_views_unique: no
application_widget_views: no
application_widget_views_unique: no
application_widget_views_login: no
application_widget_views_login_unique: no
application_widget_views_logout: no
application_opengraph_action_create: no
application_opengraph_action_delete: no
application_opengraph_object_create: no
application_opengraph_object_update: no
application_opengraph_story_impressions: no
application_opengraph_story_click: no
application_opengraph_story_like: no
application_opengraph_story_unlike: no
application_opengraph_story_comment: no
application_opengraph_story_hide: no
application_opengraph_story_hide_all: no
application_opengraph_story_report_spam: no
application_opengraph_link_impression: no
application_opengraph_link_click: no
application_opengraph_timeline_impressions: no
application_opengraph_timeline_spam: no
application_opengraph_timeline_clicked: no
application_canvas_views: no
application_canvas_views_unique: no
application_canvas_views_login: no
application_canvas_views_login_unique: no
application_canvas_views_logout: no
application_canvas_views_internal_referrals: no
application_canvas_views_external_referrals: no
application_tab_views: no
application_tab_views_unique: no
application_api_calls: no
application_api_calls_top: no
application_api_calls_unique: no
application_api_errors: no
application_api_errors_rate: no
application_api_errors_top: no
application_api_time_average: no
application_canvas_errors: no
application_canvas_errors_rate: no
application_canvas_time_average: no
page_fans_locale: no
page_fans_city: no
page_fans_country: ok
page_fans_gender_age: no
page_fan_adds: no
page_fan_adds_unique: no
page_fans_by_like_source: no
page_fans_by_like_source_unique: no
page_fan_removes: no
page_fan_removes_unique: no
page_friends_of_fans: no
page_tab_views_login_top_unique: no
page_tab_views_login_top: no
page_views: no
page_views_unique: no
page_views_login: no
page_views_login_unique: no
page_views_logout: no
page_views_external_referrals: no
page_posts_impressions: no
page_posts_impressions_unique: no
page_posts_impressions_paid: no
page_posts_impressions_paid_unique: no
page_posts_impressions_organic: no
page_posts_impressions_organic_unique: no
page_posts_impressions_viral: no
page_posts_impressions_viral_unique: no
page_posts_impressions_frequency_distribution: no
post_stories: no
post_storytellers: no
post_stories_by_action_type: no
post_storytellers_by_action_type: no
post_impressions: no
post_impressions_unique: no
post_impressions_paid: no
post_impressions_paid_unique: no
post_impressions_fan: no
post_impressions_fan_unique: no
post_impressions_fan_paid: no
post_impressions_fan_paid_unique: no
post_impressions_organic: no
post_impressions_organic_unique: no
post_impressions_viral: no
post_impressions_viral_unique: no
post_impressions_by_story_type: no
post_impressions_by_story_type_unique: no
post_consumptions: no
post_consumptions_unique: no
post_consumptions_by_type: no
post_consumptions_by_type_unique: no
post_engaged_users: no
post_negative_feedback: no
post_negative_feedback_unique: no
post_negative_feedback_by_type: no
post_negative_feedback_by_type_unique: no
domain_feed_clicks: no
domain_feed_views: no
domain_stories: no
domain_widget_comments_adds: no
domain_widget_comments_views: no
domain_widget_comments_feed_views: no
domain_widget_comments_feed_clicks: no
domain_widget_like_views: no
domain_widget_likes: no
domain_widget_like_feed_views: no
domain_widget_like_feed_clicks: no
domain_widget_send_views: no
domain_widget_send_clicks: no
domain_widget_send_inbox_views: no
domain_widget_send_inbox_clicks: no
page_stories: no
page_storytellers: no
page_stories_by_story_type: no
page_storytellers_by_story_type: no
page_storytellers_by_age_gender: no
page_storytellers_by_country: ok
page_storytellers_by_locale: no
page_impressions: no
page_impressions_unique: no
page_impressions_paid: no
page_impressions_paid_unique: no
page_impressions_organic: no
page_impressions_organic_unique: no
page_impressions_viral: no
page_impressions_viral_unique: no
page_impressions_by_story_type: no
page_impressions_by_story_type_unique: no
page_impressions_by_city_unique: no
page_impressions_by_age_gender_unique: no
page_impressions_frequency_distribution: no
page_impressions_viral_frequency_distribution: no
page_engaged_users: no
page_consumptions: no
page_consumptions_unique: no
page_consumptions_by_consumption_type: no
page_consumptions_by_consumption_type_unique: no
page_places_checkin_total: no
page_places_checkin_total_unique: no
page_places_checkin_mobile: no
page_places_checkin_mobile_unique: no
page_places_checkins_by_age_gender: no
page_places_checkins_by_locale: no
page_places_checkins_by_country: no
page_negative_feedback: no
page_negative_feedback_unique: no
page_negative_feedback_by_type: no
page_negative_feedback_by_type_unique: no
Started at: 2013-04-17 02:14:29
Ended at: 2013-04-17 02:17:13
Second elapsed: 164s
enter code here