I am aware that go language has an excellent support for collecting metric data. We can visualize the data in any manner we want using handy tools like go-metrics and influxDB.
I have a CPP application (similar to bitcoin) and looking for an easy way to collect some metrics. Can any CPP developer recommend anything you are aware of?
Related
When using google cloud ML to train models:
The official examples https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/census/tensorflowcore/trainer/task.py uses hooks, is_client, MonitoredTrainingSession and some other complexity.
Is this required for cloud ml or is using this example enough: https://github.com/amygdala/tensorflow-workshop/tree/master/workshop_sections/wide_n_deep?
The documentation is a bit limited in terms of best practices and optimisation, will GCP ML handle the client/worker mode or do we need to set devices e.g. replica_device_setter and so on?
CloudML Engine is largely agnostic to how you write your TensorFlow programs. You provide a Python program, and the service executes it for you, providing it with some environment variables you can use to perform distributed training (if necessary), e.g., task index, etc.
census/tensorflowcore demonstrates how to do things with the "core" TensorFlow library -- how to do everything "from scratch", including using replica_device_setters, MonitoredTrainingSessions, etc.. This may be necessary sometimes for ultimate flexibility, but can be tedious.
Alongside the census/tensorflowcore example, you'll also see a sample called census/estimator. This example is based on a higher level library, which unfortunately is in contrib and therefore does not yet have a fully stable API (expect lots of deprecation warnings, etc.). Expect it to stabilize in a future version of TensorFlow.
That particularly library (known as Estimators) is a higher level API that takes care of a lot of the dirty work for you. It will parse TF_CONFIG for you and setup the replica_device_setter as well as handle the MonitoredTrainingSession and necessary Hooks, while remaining fairly customizable.
This is the same library that the wide and deep example you pointed to is based on and they are fully supported on the service.
I would like to measure various code metrics (e.g. McCabe, Halstead) for .eol-scripts (Epsilon Object Language, for querying models).
I've already found and tried metrics for my modeling project, but it did not compute any metrics for .eol-files in my modeling project.
Eclipse Version is Luna (4.4.2).
Can anyone point me to a tool or into a direction, where I could find a tool that measures code metrics for EOL?
Sadly modelling languages are not popular enough to make it into the standard langauges that get support from metric anlaysis tools and it is often the case that you have to develop you own (or an extension if the metrics tool supports extensions e.g. via plugins).
Depending on the complexity of the metrics a simple scripts can be used to measure LOC and number of mappings, for example.
But for more complex metrics you definitley need to first do a static analysis of the EOL script and then compute the metrics.
Regarding the first part, static analysis, the Epsilon framework has recently been enhanced with an EOL static analysis tool! (early versions lacked this support). The tool is available here: Haetae. With it you can get the static analysis information of your script.
Regarding the second part I am no metrics expert but I guess once the static information is available it should no the hard to calculate them. Sadly a quick overview of the Eclipse metrics project you reference does not provide any information on how to add support for additional languages.
My task is to define and implement a homogeneous update system for all tools of a big organization.
The various tools have grown over many years and are implemented in many different technologies and languages.
For the Eclipse/RCP based tools the equinox/p2 mechanism seems to be perfect and easy to implement.
But what to do regarding e.g. the .net/C# tools and the C/C++ tools? p2 seems to be tailored for RCP applications.
I like to have a common/uniform update repository for all tools. Any ideas/thoughts?
Use the tools you already have.
The eclipse repository format should be easy to generate (eg, via some task you could integrate to your CI server.)
For the .net applications, clickonce is a solution.
There are frameworks out there, like googke's ohama http://code.google.com/p/omaha/ for dealing with applications update
You could build your owns tools, it's basically files download & replacement. It can get nasty if you have to deals with the UAC thought.
Having only one repository seems difficult. But you could build a tool to generate the needed formats (equinox/p2, clickone, etc)
Building a server and update tools yourself could be time consuming, avoid it unless you have a really good reason to do so.
A build server, some scripts on the top of git repositories (or something like that) and an open source framework should do the job.
I am looking at using some Dataflow programming techniques in a clojure program but I am having difficulty in finding much information from projects using Java, C#, or other mainstream languages that have used such techniques in the real world. I would be grateful to hear if anyone has any expereinces they could share regarding this.
Here, we are! We've made... (quotation is from one of my older post):
We've designed and implemented a DF
server for our automation project
(dispatcher, component iterface, a
bunch of components, DF language, DF
compiler, UI). It is written in bare
C++, and runs on several Unix-like
systems (Linux x86, MIPS, avr32 etc.,
Mac OSX). It lacks several features,
e.g. sophisticated flow control,
complex thread control (there is only
a not too advanced component for it),
so it is just a prototype, even it
works. We're now working on a
full-featured server. We've learnt lot
during implementing and using the
prototype.
Also, we'll make a visual editor some
day.
There're dataflow systems wich don't even mention dataflow approach:
SynthEdit: http://www.synthedit.com/ - It's an audio related framework and component set for creating VST plugins
TinyOS: http://www.tinyos.net/ - It's an embedded operating system/framework
Digital synthetisers/samplers are dataflow systems, programmed - supposedly - in C or some parts in Assembly, check my answer to another post about some examples.
Quartz Composer, a graphic magic tool for Mac,
Blender has dataflow subsystem for image composing.
Writing a dataflow system is not rocket science. Here's my older post about the basics of dataflow framework.
The term dataflow is wide. There are realtime synchronous dataflow systems, like synthetisers and samplers, there are asynchronous ones, like our home aut. system (the system is in idle unless the user presses a button or a timer runs out), and there're even different architectures, like spreadsheets or make.
Wanna reading more about dataflow programming? Read J. Paul Morrison's site and book.
Pervasive DataRush is a framework for parallel dataflow programming for any JVM language, including Clojure.
Pervasive DataRush uses a dataflow architecture. The architecture implements a program that executes as a graph of computation nodes interconnected by dataflow queues. The nodes use the queues to share data. As the data is streaming, only data required by any active operation needs to be in memory at any given time, allowing very large data sets to be analyzed. Besides offering the potential for scaling to problems larger than available memory, dataflow graphs exploit multiple forms of parallelism.
Customers are using DataRush for big data analytics and data preparation (ETL).
We've made another one: a collaborative spreadsheet with MySQL/PHP backend and AJAX frontend. The software is in beta state, documentation is under construction.
I am developing a distributed processing API in Java, Erlang and Ruby. What basic commands can I include from which I can build mapreduce, pipelining, and all the most used parallell algorithms on top of it.
You may get some hints from this blog post, which I found quite useful.
http://ox.no/posts/minimalistic-mapreduce-in-net-4-0-with-the-new-task-parallel-library-tpl