How do I control Drupal 8 monolog dblog log-levels - drupal-8

I'm using Drupal 8 with the Monolog module (1.3) for external log support and Sentry logging, while preserving core database logging (dblog/watchdog). This all works, except that I can't find a way to control the watchdog logging level (so I'm getting a lot of debug logging in production environments).
Here is my monolog configuration (monolog.services.yml):
parameters:
monolog.channel_handlers:
default: ['drupal.dblog', 'drupal.raven']
custom1: ['rotating_file_custom1']
custom2: ['rotating_file_custom2']
custom3: ['rotating_file_custom3']
services:
monolog.handler.rotating_file_custom1:
class: Monolog\Handler\RotatingFileHandler
arguments: ['file://../logs/custom1.log', 25, 'monolog.level.debug']
monolog.handler.rotating_file_custom2:
class: Monolog\Handler\RotatingFileHandler
arguments: ['file://../logs/custom2.log', 25, 'monolog.level.debug']
monolog.handler.rotating_file_custom3:
class: Monolog\Handler\RotatingFileHandler
arguments: ['file://../logs/custom3.log', 25, 'monolog.level.debug' ]
I tried adding a new services handler for drupal.dblog using the DrupalHandler, but I couldn't figure out what arguments to use (arguments are required and it is expecting a LoggerInterface implementation as the first argument).
I've read through the module documentation, but it almost exclusively focuses on file/external logging.
Any suggestions?
TIA

I also was trying to accomplish this and it was also not clear to me. solution was to add the core dblog service with the '#' notation as first argument. What also should be mentioned is the fact, that log-levels are documented as 'monolog.level.debug' but should be implemented without the prefix 'monolog.level.' because the monolog parameters are not loaded correctly as they should. Here a working example:
parameters:
monolog.channel_handlers:
default: ['rotating_file_handler', 'db_warning_handler']
locale: ['null']
monolog.processors: ['message_placeholder', 'current_user', 'request_uri', 'ip', 'referer', 'memory_usage']
services:
monolog.handler.db_warning_handler:
class: Drupal\monolog\Logger\Handler\DrupalHandler
arguments: ['#logger.dblog', 'warning']
monolog.handler.rotating_file_handler:
class: Monolog\Handler\RotatingFileHandler
arguments: ['private://logs/debug.log', 30, 'debug']

Related

Vertex Pipeline Metric values not being added to metrics artifact?

We are trying to return some metrics from our Vertex Pipeline, such that they are visible in the Run Comparison and Metadata tools in the Vertex UI.
I saw here that we can use this output type Output[Metrics], and the subsequent metrics.log_metric("metric_name", metric_val) method to add the metrics, and it seemed from the available documentation that this would be enough.
We want to use the reusable component method as opposed to python function based components, around which the example is based. So we implemented it within our component code like so:
We added the output in the component.yaml:
outputs:
- name: metrics
type: Metrics
description: evaluation metrics path
then added the output to the command in the implemenation:
command: [
python3, main.py,
--gcs-test-data-path, {inputValue: gcs_test_data_path},
--gcs-model-path, {inputValue: gcs_model_path},
--gcs-output-bucket-id, {inputValue: gcs_output_bucket_id},
--project-id, {inputValue: project_id},
--timestamp, {inputValue: timestamp},
--batch-size, {inputValue: batch_size},
--img-height, {inputValue: img_height},
--img-width, {inputValue: img_width},
--img-depth, {inputValue: img_depth},
--metrics, {outputPath: metrics},
]
Next in the components main python script, we parse this argument with argparse:
PARSER.add_argument('--metrics',
type=Metrics,
required=False,
help='evaluation metrics output')
and pass it to the components main function:
if __name__ == '__main__':
ARGS = PARSER.parse_args()
evaluation(gcs_test_data_path=ARGS.gcs_test_data_path,
gcs_model_path=ARGS.gcs_model_path,
gcs_output_bucket_id=ARGS.gcs_output_bucket_id,
project_id=ARGS.project_id,
timestamp=ARGS.timestamp,
batch_size=ARGS.batch_size,
img_height=ARGS.img_height,
img_width=ARGS.img_width,
img_depth=ARGS.img_depth,
metrics=ARGS.metrics,
)
in the declaration of the component function, we then typed this metrics parameter as Output[Metrics]
from kfp.v2.dsl import Output, Metrics
def evaluation(gcs_test_data_path: str,
gcs_model_path: str,
gcs_output_bucket_id: str,
metrics: Output[Metrics],
project_id: str,
timestamp: str,
batch_size: int,
img_height: int,
img_width: int,
img_depth: int):
finally, we implement the log_metric method within this evaluation function:
metrics.log_metric('accuracy', acc)
metrics.log_metric('precision', prec)
metrics.log_metric('recall', recall)
metrics.log_metric('f1-score', f_1)
When we run this pipeline, we can see this metric artifact materialised in the DAG:
And Metrics Artifacts are listed in the Metadata UI in Vertex:
However, clicking through to view the artifacts JSON, there is no Metadata listed:
In addition, No Metadata is visible when comparing runs in the pipeline UI:
Finally, navigating to the Objects URI in GCS, we are met with 'Requested entity was not found.', which I assume indicates that nothing was written to GCS:
Are we doing something wrong with this implementation of metrics in the reusable components? From what I can tell, this all seems right to me, but it's hard to tell given the docs at this point seem to focus primarily on examples with Python Function based components.
Do we perhaps need to proactively write this Metrics object to an OutputPath?
Any helps is appreciated.
----- UPDATE ----
I have since been able to get artifact metadata and URI To update. In the end we used kfp sdk to generate a yaml file based on a #component decorated python function, we then adapted this format for our reusable components.
Our component.yaml now looks like this:
name: predict
description: Prepare and create predictions request
implementation:
container:
args:
- --executor_input
- executorInput: null
- --function_to_execute
- predict
command:
- python3
- -m
- kfp.v2.components.executor_main
- --component_module_path
- predict.py
image: gcr.io/PROJECT_ID/kfp/components/predict:latest
inputs:
- name: input_1
type: String
- name: intput_2
type: String
outputs:
- name: output_1
type: Dataset
- name: output_2
type: Dataset
with this change to the yaml, we can now successfully update the artifacts metadata dictionary, and uri through artifact.path = '/path/to/file'. These updates are displayed in the Vertex UI.
I am still unsure why the component.yaml format specified in the Kubeflow documentation does not work - I think this may be a bug with Vertex Pipelines.
As I can see in the code you are running, everything should work without a problem; but, as you commented, I would recommend you to write the metrics object into a path so that it can reach somewhere within your project.

How to get Instance name in CommandBox CF 2018?

I recently started using commandBox to run ColdFusion in my local environment. After I played around for a while one issue I run into was related to adminapi. Here is the code that I use in one of my projects:
adminObj = createObject("component","cfide.adminapi.runtime");
instance = adminObj.getInstanceName();
This code is pretty straight forward and work just fine if I install traditional ColdFusion Developer version on my machine. I tried running this on commandBox: "app":{ "cfengine":"adobe#2018.0.7" }
After I run the code above this is the error message I got:
Object Instantiation Exception.
Class not found: com.adobe.coldfusion.entman.ProcessServer
The first debugging step was to check if component exists. I simply checked that like this:
adminObj = createObject("component","cfide.adminapi.runtime");
writeDump(adminObj);
The result I got on the screen was this:
component CFIDE.adminapi.runtime
extends CFIDE.adminapi.base
METHODS
Then I tried this to make sure method exists in the scope:
adminObj = createObject("component","cfide.adminapi.runtime");
writeDump(adminObj.getInstanceName);
The output looks like this, and that confirmed that method getInstanceName exists.
function getInstanceName
Arguments: none
ReturnType: any
Roles:
Access: public
Output: false
DisplayName:
Hint: returns the current instance name
Description:
The error is occurring only if I call the function getInstanceName(). Does anyone know what could be the reason of this error? Is there any solution for this particular problem? Like I already mentioned this method works in traditional ColdFusion 2018 developer environment. Thank you.
This is a bug in Adobe ColdFusion. The CFC you're creating is trying to create an instance of a specific Java class. I recognize the class name com.adobe.coldfusion.entman.ProcessServer as being related to their enterprise manager which controls features only available in certain versions of CF as well as features only available on their "standard" Tomcat installation (as opposed to a J2E deployment like CommandBox).
Please report this to Adobe in the Adobe bug tracker as they appear to be incorrectly detecting the servlet installation. I worked with them a couple years ago to improve their servlet detection on CommandBox, but I guess they still have some issues.
As a workaround, you could try and find out what jar that class is from on a non-CommandBox installation of Adobe ColdFusion and add it to the path, but I can't promise that it will work and that it won't have negative consequences.

TraciMobility::getExternalId error when adding custom module into Veins_Inet example (Omnet++)

I am attempting to add a new custom RSU module (extending AdHocHost) into the Veins_Inet example. Here is my updated scenario (with 1 RSU).
network TestScenario {
submodules:
radioMedium: Ieee80211ScalarRadioMedium;
manager: VeinsInetManager;
node[0]: VeinsInetCar;
// added rsu
rsu: VeinsInetRSU;
connections allowunconnected:}
I also updated the ini file so that the RSU mobility is
*.rsu.mobility.typename = "inet.mobility.static.StationaryMobility"
and the RSU application is barebones with minor implementation:
*.rsu.app[0].typename = "practice.veins_inet.VeinsInetRSUSampleApplication".
However, I get the following error:
TraCIMobility::getExternalId called with no external id set yet.
In the example, the VeinsInetManager is managing the cars with TRACI. Here is the ned file associated with the manager. The source file has 2 functions, pre-initialize module and update module position.
simple VeinsInetManager extends TraCIScenarioManagerLaunchd {
parameters:
#class(veins::VeinsInetManager);}
How can I add a custom module into the scenario without raising any errors?
Your application might be inheriting from VeinsInetApplicationBase, which calls TraCI methods (that fail for nodes that are not a TraCI-managed vehicle). See also its source code.
To be doubly-sure, run your simulation in debug mode, turn on debug-on-errors, and check the stack trace to see where the call is coming from.

How to increase deploy timeout limit at AWS Opsworks?

I would like to increase the deploy time, in a stack layer that hosts many apps (AWS Opsworks).
Currenlty I get the following error:
Eror
[2014-05-05T22:27:51+00:00] ERROR: Running exception handlers
[2014-05-05T22:27:51+00:00] ERROR: Exception handlers complete
[2014-05-05T22:27:51+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache/chef-stacktrace.out
[2014-05-05T22:27:51+00:00] ERROR: deploy[/srv/www/lakers_test] (opsworks_delayed_job::deploy line 65) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 600s:
Thanks in advance.
First of all, as mentioned in this ticket reporting a similar issue, the Opsworks guys recommend trying to speed up the call first (there's always room for optimization).
If that doesn't work, we can go down the rabbit hole: this gets called, which in turn calls Mixlib::ShellOut.new, which happens to have a timeout option that you can pass in the initializer!
Now you can use an Opsworks custom cookbook to overwrite the initial method, and pass the corresponding timeout option. Opsworks merges the contents of its base cookbooks with the contents of your custom cookbook - therefore you only need to add & edit one single file to your custom cookbook: opsworks_commons/libraries/shellout.rb:
module OpsWorks
module ShellOut
extend self
# This would be your new default timeout.
DEFAULT_OPTIONS = { timeout: 900 }
def shellout(command, options = {})
cmd = Mixlib::ShellOut.new(command, DEFAULT_OPTIONS.merge(options))
cmd.run_command
cmd.error!
[cmd.stderr, cmd.stdout].join("\n")
end
end
end
Notice how the only additions are just DEFAULT_OPTIONS and merging these options in the Mixlib::ShellOut.new call.
An improvement to this method would be changing this timeout option via a chef attribute, that you could in turn update via your custom JSON in the Opsworks interface. This means passing the timeout attribute in the initial Opsworks::ShellOut.shellout call - not in the method definition. But this depends on how the shellout method actually gets called...

How to enable DEBUG level logging with Jetty embedded?

I'm trying to set the logging level to DEBUG in an embedded Jetty instance.
The documentation at http://docs.codehaus.org/display/JETTY/Debugging says to -
call SystemProperty.set("DEBUG", "true") before calling new
org.mortbay.jetty.Server().
I'm not sure what the SystemProperty class is, it doesn't seem to be documented anywhere. I tried System.setProperty(), but that didn't do the trick.
My question was answered on the Jetty mailing list by Joakim Erdfelt:
You are looking at the old Jetty 6.x docs at docs.codehaus.org.
DEBUG logging is just a logging level determined by the logging
implementation you choose to use.
If you use slf4j, then use slf4j's docs for configuring logging level. http://slf4j.org/manual.html
If you use java.util.logging, use the JVM docs. http://docs.oracle.com/javase/6/docs/technotes/guides/logging/overview.html
If you use the built-in StdErrLog, then there is a pattern to follow.
-D{classref}.LEVEL={level}
Where {classref} is the class reference you want to set the level on,
and all sub-class refs. and {level} is one of the values ALL, DEBUG,
INFO, WARN
Example:
-Dorg.eclipse.jetty.LEVEL=INFO - this will enable INFO level logging for all jetty packages / classes.
-Dorg.eclipse.jetty.io.LEVEL=DEBUG - this will enable DEBUG level logging for IO classes only
-Dorg.eclipse.jetty.servlet.LEVEL=ALL - this will enable ALL logging (trace events, internally ignored exceptions, etc..) for servlet
packages.
-Dorg.eclipse.jetty.util.thread.QueuedThreadPool.LEVEL=ALL - this will enable level ALL+ on the specific class only.
Add this
-Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.StdErrLog
-Dorg.eclipse.jetty.LEVEL=DEBUG
In case you just want to quickly get log messages to stderr add something like this to java command line:
-Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.StdErrLog -D{classref}.LEVEL=DEBUG
You can use this snippet to enable logging:
import org.eclipse.jetty.util.log.Log;
import org.eclipse.jetty.util.log.StdErrLog;
.
.
.
StdErrLog logger = new StdErrLog();
logger.setDebugEnabled(true);
Log.setLog(logger);