I have an IstioOperator deployment with logs enabled in JSON format:
spec:
meshConfig:
accessLogFile: /dev/stdout
accessLogEncoding: JSON
No specific accessLogFormat is defined so default one applies.
[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %RESPONSE_CODE_DETAILS% %CONNECTION_TERMINATION_DETAILS%
\"%UPSTREAM_TRANSPORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\"
\"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME%\n
However, what i want is to add another field at the end of log by the name PATH_MAIN which is derived from original path attribute but based on same regex (regex patterns already figured out) it would alter some values, such as redacting GUIDs etc.
My question is, how can I, if possible define a new field in Log Format by giving another field as attribute and defining its value based on regex.
Related
In one of my deployment files, I want to set an environment variable. The variable is KUBE_VERSION and values must be fetched from a ConfigMap.
kube_1_21: 1.21.10_1550
This is part of ConfigMap where I want to set 1.21.10_1550 to KUBE_VERSION, but if the cluster is of IKS 1.20, then the key will be:
kube_1_20: 1.20.21_3456
kube_ is always static. How can I set environment variable using a regex expression?
Something of this sort:
- name: KUBE_VERSION
valueFrom:
configMapKeyRef:
name: cluster-info
key: "kube_1*"
As far as I know it is unfortunately not possible to use the regular expression as you would like. Additionally, you have information about the regular expression that validates the entered data:
regex used for validation is '[-._a-zA-Z0-9]+')
It follows that you have to enter key as an alphanumeric string and additionally you can use the characters -, _ and . So it is not possible to use regex in this place.
To workaround you can write your custom script i.e. in Bash and replace the proper line with sed command.
I'm using Google Data Studio to create a report analyzing specific referral sites. My data source is my site Google Analytics.
I want to replace the Full Referrer (e.g. of the format webaddress.com/page-name-one) with a text only value (i.e Page name one), so that it's clearer to see in the report which page is which in my charts and tables.
I've used the below formulae in the calculated fields, but none of them seem to change Full Referrer to match what I need it to. Data studio recognizes them all as valid formulae too.
I've anonymised my examples, but it has the same principles. I've tried:
REGEXP_REPLACE(Full Referrer,"[webaddress\\.com\\/page\\-name\\-one].*","Page name one")
REGEXP_REPLACE(Full Referrer, 'webaddress.com/page-name-one', 'Page name one')
REGEXP_REPLACE(Full Referrer, 'webaddress\\.com\\/page\\-name\\-one', 'Page name one')
REGEXP_REPLACE(Full Referrer, 'name', 'Page name one')
REGEXP_REPLACE(Full Referrer, 'page-name-one', 'Page name one')
REGEXP_REPLACE(Full Referrer, 'page\\-name\\-one', 'Page name one')
In testing this on one of my own GA data sources, I was able to achieve this using one of your patterns:
REGEXP_REPLACE(Full Referrer,'webaddress.com/page-name-one','Page name one')
It should be noted, however, that the . should be properly escaped (either by \ or wrapping it in a character class like [.]; see re2 syntax for details). Because you have to double-backslash, I also prefer to use something Data Studio borrowed from BigQuery (sort of an undocumented feature), which is the regular expression string type (r"" or r''). When using this, you only have to single-backslash (unless you want a literal backslash):
REGEXP_REPLACE(Full Referrer,r'webaddress\.com/page-name-one','Page name one')
Because you're using REGEXP_REPLACE, anything before or after your match string will still exist after the replacement—meaning that for a Full Referrer of "m.facebook.com/l", REGEXP_REPLACE(Full Referrer,r'facebook\.com','FB') would return "m.FB/l"
So your pattern above will match the value anywhere in the string, which likely isn't what you want. To anchor it to the beginning, use the ^ (start of string) assertion:
REGEXP_REPLACE(Full Referrer,r'^webaddress\.com/page-name-one','Page name one')
If you want to only match that exact value of Full Referrer (i.e. not including any additional path levels), make sure to use the $ (end of string) assertion as well:
REGEXP_REPLACE(Full Referrer,r'^webaddress\.com/page-name-one$','Page name one')
Keep in mind that if you're doing this in the data source as a calculated field, you aren't actually changing the original metric—you're working on a copy of it. So you need to replace Full Referrer with whatever you named your calculated field in the data source.
Often you're wanting to do this for a bunch of sites or pages, so you can use CASE and REGEXP_MATCH to handle all this logic in a single field:
CASE
WHEN REGEXP_MATCH(Full Referrer,r'^webaddress\.com/page-name-one$') THEN 'Page name one'
WHEN REGEXP_MATCH(Full Referrer,r'^site2\.com/example$') THEN 'S2 Example'
ELSE Full Referrer
END
These matches are done in order, so you can even match a specific page or pages, and then still provide a different value for anything on that domain that you didn't match:
CASE
WHEN REGEXP_MATCH(Full Referrer,r'^site\.com/$') THEN 'Site - Home'
WHEN REGEXP_MATCH(Full Referrer,r'^site\.com/about$') THEN 'Site - About'
WHEN REGEXP_MATCH(Full Referrer,r'^site\.com/') THEN 'Site - (other)'
ELSE Full Referrer
END
You can also use the ELSE if you want to bucket all of the unmatched values into an "other" grouping instead of just leaving the original value.
Another thing to remember is that due to shared fields in GA, things like Source (utm_source) also show up in Full Referrer, so you could be seeing values there that you wouldn't normally expect. Often you can get rid of these by also filtering to only the Default Channel Grouping of "Referral".
If your patterns still aren't matching, please update the question with some additional details such as what the output actually is, whether there's an error message, etc.—and also whether you're doing this as a calculated field in the data source or the "Create Field" button on a single chart.
Here I can read that when configuring a prospect I can add a custom field to the data, which later I can use for filtering.
So for example I can write
- type: log
paths:
- /my/path/app1.csv
fields:
app_name: app1
- type: log
paths:
- /my/path/app2.csv
fields:
app_name: app2
This means that anytime I will have a new CSV file to track I have to add it to the filebeat.yml file adding the custom app_name field accordingly.
I was wondering if I could use a regex with a capture group in the prospect definition to "automatically" track any new file and assign the right app_name value. Something like this:
- type: log
paths:
- /my/path/(.*).csv
fields:
app_name: \1
What do you think? I didn't find any documentation regarding this possibility with the fields feature.
As adviced here, I can use the source Filebeat field to filter the data. This field is the path of the harvested files, so no other field is required to filter.
I have a sharepoint 2013 site that uses a managed metadata term set for navigation. Documents can be tagged with the managed metadata so they appear in whatever category or categories is appropriate for the document. I need to allow documents to be saved as favorites. I created a custom list that saves the file name and path but I can't get the managed metadata settings to save correctly. I am using spservices.UpdateListItems via javascript and pass the ids and terms in the valuepairs property of the call like so ;#. Although the method saves the record, it either does not save the term or it saves one completely unrelated. Does anyone have any further advice on how to do this?
$().SPServices({
operation: "UpdateListItems",
async: true,
batchCmd: "New",
listName: "UserFavorites",
valuepairs: [["Title", title], ["DocumentId", itemid],["AssetCategory", assetCategoriesString]],
completefunc: function (xData, Status) {
alert(Status + " -- " + xData.responseText);
}
});
Example of the assetCategoriesString variable contents:
"fc8d083a-fc5e-4525-8fef-04ba982d1633;#Print Publications"
I'd like my REST API endpoints to use all lower case letters for the model.
In the case of the built-in User model, would I simply make a new model named user with User as the base? or is there another method I should use?
There are a few options:
At the moment, the endpoint name is case-insenstive, i.e., /api/users and /api/Users both work.
You can customize the model endpoint name in the model definition json, for example
"http": {"path": "/my-users"}
There are a pending PR: https://github.com/strongloop/loopback/pull/433
As Simon pointed out, you can subclass the User model. Please the default endpoint name is derived from the plural of the model name.
Yes. You just generate a new model using slc loopback:model user, follow the prompts, then change the base property value from PersistedModel to User in common/models/user.json.
I have an example here: https://github.com/strongloop/loopback-example-access-control/blob/master/common/models/user.json#L3
The best way to achieve this is to set normalizeHttpPath setting to true in your /server/config.json file.
...
"remoting": {
"rest": {
...
"normalizeHttpPath": true,
},
},
When normalizeHttpPath is set to true, it converts (in HTTP paths) from:
Uppercase letters to lowercase.
Underscores (_) to dashes (-).
CamelCase to dash-delimited.
For example, "MyClass" or "My_class" becomes "my-class" in HTTP path.
Note: It does not affect placeholders (for example ":id").
For more information, look at the remoting properties in official documentation for config.json.