I am currently working on a project which seeks to analyze the vibrations of a laundry machine throughout its washing cycle.
The program runs on a Raspberry Pi 3, and gets the vibration information from a vibration sensor, which sends out the state using a digital signal (as opposed to an analog signal)
Our current idea is the implement a program which "machine learns" the cycles by recording the first 10 times the laundry machine runs, and stores that information in a text file.
The program does this by running the following loop every 10 milliseconds.
for(;;) {
read state of /sys/class/gpio/export/
write the state into a vector<bool>
analyze the bool vector to figure out when the machine is turned on or off (compare against a template vector read from an external txt file)
if the machine just got turned on, cout a message
if the machine just gor turned off, cout a message
}
A few problems with our current analysis model:
Because we are recording data every 10 milliseconds, there are always going to be small lags which would make analyzing the live boolian against the template very difficult:
for example:
live boolian vector
0 1 0 1 1 1 0 0 1 1 0
template:
0 0 0 1 0 1 1 1 0 0 1 1 0
even a minute shift in the live data would render the analysis useless. Therefore, what kind of method could I use to be able to shift the data so that the live data:
0 1 0 1 1 1 0 0 1 1 0
can be compared against the shifted template
0 0 0 1 0 1 1 1 0 0 1 1 0
My guess would be we would need to program an algorithm which constantly checks multiple shifts in both directions to find the shift which is statistically most likely to be the true position of the vector.
Thank you
Related
I am trying to solve Optimisation problem with pyomo (Pyomo 5.3 (CPython 2.7.13 on Linux 3.10.0-514.26.2.el7.x86_64)) using CBC solver (Version: 2.9.8) and specifying a time limit in solver of 60 sec. The solver is getting a feasible solution (-1415.8392) but apparently not yet optimal (-1415.84) as you can see below.
After time limit ends model seemingly exits with an error code. I want to print or get values of all variables of feasible solution using CBC in specified time limit. Or is there any other way by which I can set, if Model gets 99% value of an Optimal solution, to exit and print the feasible solution.
The error code is posted below.
Cbc0004I Integer solution of -1415.8392 found after 357760 iterations and 29278 nodes (47.87 seconds)
Cbc0010I After 30000 nodes, 6350 on tree, -1415.8392 best solution, best possible -1415.84 (48.87 seconds)
Cbc0010I After 31000 nodes, 6619 on tree, -1415.8392 best solution, best possible -1415.84 (50.73 seconds)
Cbc0010I After 32000 nodes, 6984 on tree, -1415.8392 best solution, best possible -1415.84 (52.49 seconds)
Cbc0010I After 33000 nodes, 7384 on tree, -1415.8392 best solution, best possible -1415.84 (54.31 seconds)
Cbc0010I After 34000 nodes, 7419 on tree, -1415.8392 best solution, best possible -1415.84 (55.73 seconds)
Cbc0010I After 35000 nodes, 7824 on tree, -1415.8392 best solution, best possible -1415.84 (57.37 seconds)
Traceback (most recent call last):
File "model_final.py", line 392, in
solver.solve(model, timelimit = 60*1, tee=True)
File "/home/aditya/0r/lib/python2.7/site-packages/pyomo/opt/base/solvers.py", line 655, in solve
default_variable_value=self._default_variable_value)
File "/home/aditya/0r/lib/python2.7/site-packages/pyomo/core/base/PyomoModel.py", line 242, in load_from
% str(results.solver.status))
ValueError: Cannot load a SolverResults object with bad status: error
When I run the model generated by pyomo manually using the same command-line parameters as pyomo /usr/bin/cbc -sec 60 -printingOptions all -import /tmp/tmpJK1ieR.pyomo.lp -import -stat=1 -solve -solu /tmp/tmpJK1ieR.pyomo.soln it seems to exit normally and also writes the solution as shown below.
Cbc0010I After 35000 nodes, 7824 on tree, -1415.8392 best solution, best possible -1415.84 (57.06 seconds)
Cbc0038I Full problem 205 rows 289 columns, reduced to 30 rows 52 columns
Cbc0010I After 36000 nodes, 8250 on tree, -1415.8392 best solution, best possible -1415.84 (58.73 seconds)
Cbc0020I Exiting on maximum time
Cbc0005I Partial search - best objective -1415.8392 (best possible -1415.84), took 464553 iterations and 36788 nodes (60.11 seconds)
Cbc0032I Strong branching done 15558 times (38451 iterations), fathomed 350 nodes and fixed 2076 variables
Cbc0035I Maximum depth 203, 5019 variables fixed on reduced cost
Cbc0038I Probing was tried 31933 times and created 138506 cuts of which 0 were active after adding rounds of cuts (4.431 seconds)
Cbc0038I Gomory was tried 30898 times and created 99534 cuts of which 0 were active after adding rounds of cuts (4.855 seconds)
Cbc0038I Knapsack was tried 30898 times and created 12926 cuts of which 0 were active after adding rounds of cuts (8.271 seconds)
Cbc0038I Clique was tried 100 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Cbc0038I MixedIntegerRounding2 was tried 30898 times and created 13413 cuts of which 0 were active after adding rounds of cuts (3.652 seconds)
Cbc0038I FlowCover was tried 100 times and created 4 cuts of which 0 were active after adding rounds of cuts (0.019 seconds)
Cbc0038I TwoMirCuts was tried 30898 times and created 15292 cuts of which 0 were active after adding rounds of cuts (2.415 seconds)
Cbc0038I Stored from first was tried 30898 times and created 15734 cuts of which 0 were active after adding rounds of cuts (0.000 seconds)
Cbc0012I Integer solution of -1411.9992 found by Reduced search after 467825 iterations and 36838 nodes (60.12 seconds)
Cbc0020I Exiting on maximum time
Cbc0005I Partial search - best objective -1411.9992 (best possible -1415.4522), took 467825 iterations and 36838 nodes (60.12 seconds)
Cbc0032I Strong branching done 476 times (1776 iterations), fathomed 1 nodes and fixed 18 variables
Cbc0035I Maximum depth 21, 39 variables fixed on reduced cost
Cuts at root node changed objective from -1484.12 to -1415.45
Probing was tried 133 times and created 894 cuts of which 32 were active after adding rounds of cuts (0.060 seconds)
Gomory was tried 133 times and created 1642 cuts of which 0 were active after adding rounds of cuts (0.047 seconds)
Knapsack was tried 133 times and created 224 cuts of which 0 were active after adding rounds of cuts (0.083 seconds)
Clique was tried 100 times and created 0 cuts of which 0 were active after adding rounds of cuts (0.001 seconds)
MixedIntegerRounding2 was tried 133 times and created 163 cuts of which 0 were active after adding rounds of cuts (0.034 seconds)
FlowCover was tried 100 times and created 5 cuts of which 0 were active after adding rounds of cuts (0.026 seconds)
TwoMirCuts was tried 133 times and created 472 cuts of which 0 were active after adding rounds of cuts (0.021 seconds)
ImplicationCuts was tried 25 times and created 41 cuts of which 0 were active after adding rounds of cuts (0.003 seconds)
Result - Stopped on time limit
Objective value: -1411.99922848
Lower bound: -1415.452
Gap: 0.00
Enumerated nodes: 36838
Total iterations: 467825
Time (CPU seconds): 60.13
Time (Wallclock seconds): 60.98
Total time (CPU seconds): 60.13 (Wallclock seconds): 61.01
The top few lines of the CBC solution file are:
Stopped on time - objective value -1411.99922848
0 c_e_x1454_ 0 0
1 c_e_x1455_ 0 0
2 c_e_x1456_ 0 0
3 c_e_x1457_ 0 0
4 c_e_x1458_ 0 0
5 c_e_x1459_ 0 0
6 c_e_x1460_ 0 0
7 c_e_x1461_ 0 0
8 c_e_x1462_ 0 0
Can anyone tell me how can I get these values without generating any error?
Thanks in advance.
You could try to set the bound gap tolerance such that it will accept the other answer. I'm surprised that the solver status is coming back with error if there is a feasible solution found. Could you print out the whole results object?
Created a pull request https://github.com/Pyomo/pyomo/pull/265 to address this issue.
For those who want the patch right away:
In file pyomo/solvers/plugins/solvers/CBCplugin.py
## -264,7 +264,8 ## def _check_and_escape_options(options):
cmd.append('-AMPL')
if self._timelimit is not None and self._timelimit > 0.0:
- cmd.extend(['-sec', str(self._timelimit)])
+ cmd.extend(['-sec', str(self._timelimit - 1 )])
+ cmd.extend(['-timeMode', "elapsed"])
if "debug" in self.options:
cmd.extend(["-log","5"])
for key, val in _check_and_escape_options(self.options):
## -276,7 +277,8 ## def _check_and_escape_options(options):
#"-stat"])
else:
if self._timelimit is not None and self._timelimit > 0.0:
- cmd.extend(['-sec', str(self._timelimit)])
+ cmd.extend(['-sec', str(self._timelimit - 1 )])
+ cmd.extend(['-timeMode', "elapsed"])
if "debug" in self.options:
cmd.extend(["-log","5"])
# these must go after options that take a value
The timelimit is set to 1 sec lower than provided option to ensure CBC process gets time to exit properly before the pyutilib.subprocess.processmngr checks the exit code. In my test runs the process manager checked the exit status at T+0.02 seconds and CBC process typically exited after T+0.1 seconds.
Also changed the CBC code to use the wallclock seconds from the default of CPU seconds as the process manager is also checking the same.
I feel like this must have a simple answer, but I really don't know how to approach this.
For background, the stack of things is like this:
Python script -> C++ binary -(fork)-> actual thing we want to measure.
Essentially, we have a python script that simulates an environment by using tmp directories and running multiple instances of this network software stack we're developing. The script calls a host binary (which is unimportant here), and then, after it loads, a helper binary. The helper binary can be passed a parameter to daemonize, and when it does this, it forks in the usual way.
What we need to do is measure the daemon's CPU utilization, but I don't really know how to. What I have done is read the stat file periodically, but since the process daemonizes, I can't use echo $! to get its PID. Using ps aux | grep 'thing' works fine, but I think this is giving me the parent process, because the stat information looks like this:
1472582561 9455 (nlsr) S 1 9455 9455 0 -1 4218944 394 0 0 0 13 0 0 0 20 0 2 0 909820 184770560 3851 18446744073709551615 4194304 5318592 140734694817376 140734694810512 140084250723843 0 0 16781312 0 0 0 0 17 0 0 0 0 0 0 7416544 7421528 16224256 140734694825496 140734694825524 140734694825524 140734694825962 0
I know that the parent process should not be PID1, and definitely the utime field and similar should be greater than 13 clock ticks. This is what is leading me to conclude that this process is really the parent process, and not the forked child that's doing all the work.
I can modify pretty much any file necessary, but because of code review constraints, design specs., etc., the less I have to change along many files, the better.
Get the PID of the child reliably
fork() returns the PID of the child to the parent
Get the CPU stats from /proc/[PID]/stat
#14 utime - CPU time spent in user code, measured in clock ticks
I recently faced flash overflow problem. After doing some optimization in code, I saved some flash memory and executed software successfully. I want to how much flash memory is saved through my changes. Please let me know how can I check for used flash / available flash memory. Also I want to how much flash is utilized by particular function/file.
Below mentioned are some info about my developing environment.
- Avr microcontroller with 64 k ram and 512 K flash.
- Using freeRtos.
- Using GNU C++ compiler.
- Using AVRATJTAGEICE for programming and Debugging.
Please let me know the solution.
Regards,
Jagadeep.
GCC's size program is what you're looking for.
size can be passed the full compiled .elf file. It will, by default, output something like this:
$ size linked-file.elf
text data bss dec hex filename
11228 112 1488 12828 321c linked-file.elf
This is saying:
There are 11228 bytes in the .text "section" of this file. This is generally for functions.
There are 112 bytes of initialized data: global variables in the program with initial values.
There are 1488 bytes of uninitialized data: global variables without initial values.
dec is simply the sum of the previous 3 values: 11228 + 112 + 1488 = 12828.
hex is simply the hexadecimal representation of the dec value: 0x321c == 12828.
For embedded systems, generally dec needs to be smaller than the flash size of your target device (or the available space on the device).
It is generally sufficient to simply watch the dec or text outputs of GCC's size command to monitor the size of your compiled code over time. A large jump in size often indicates a poorly implemented new feature or constexpr that are not getting compiled away. (Don't forget function-sections and data-sections).
Note: For AVR's, you'll want to use avr-size for checking the linked size of AVR .elf files. avr-size takes an extra argument of the target chip and will automatically calculate the percentage of used flash for your chosen chip.
GCC's size also works directly on intermediate object files.
This is particularly useful if you want to check the compiled size of functions.
You should see something like this excerpt:
$ size -A main.cpp.o
main.cpp.o :
section size addr
.group 8 0
.group 8 0
.text 0 0
.data 0 0
.bss 0 0
.text._Z8sendByteh 8 0
.text._ZN3XMC5IOpin7setModeENS0_4ModeE 64 0
.text._ZN7NamSpac6OptionIN5Clock4TimeEEmmEi 76 0
.text.Default_Handler 24 0
.text.HardFault_Handler 16 0
.text.SVC_Handler 16 0
.text.PendSV_Handler 16 0
.text.SysTick_Handler 28 0
.text._Z5errorPKc 8 0
.text._ZN7NamSpac5Motor2goEi 368 0
.text._ZN7NamSpac5Motor3getEv 12 0
.rodata.cst1 1 0
.text.startup.main 632 0
.text._ZN7NamSpac7Program3runEv 380 0
.text._ZN7NamSpac8Position4tickEv 24 0
.text.startup._GLOBAL__sub_I__ZN7NamSpac7displayE 292 0
.init_array 4 0
.bss._ZN5Debug9formatterE 4 0
.rodata._ZL10dispDigits 8 0
.bss.position 4 0
.bss.motorState 4 0
.bss.count 4 0
.rodata._ZL9diameters 20 0
.bss._ZN7NamSpac8diameterE 16 0
.bss._ZN5Debug3pinE 12 0
.bss._ZN7NamSpac7displayE 24 0
.rodata.str1.4 153 0
.rodata._ZL12dispSegments 32 0
.bss._ZL16diametersDisplay 10 0
.bss.loadAggregate 4 0
.bss.startCount 4 0
.bss._ZL15runtimesDisplay 10 0
.bss._ZN7NamSpac7runtimeE 16 0
.bss.startTime 4 0
.rodata._ZL8runtimes 20 0
.comment 111 0
.ARM.attributes 49 0
Total 2494
Please let me know the solution.
Sorry, there's no the solution! You've gotta getting through what's linked to your final ELF, and decide if it was linked by intend, or unwanted default.
Please let me know how can I check for used flash / available flash memory.
That primarily depends on your actual target hardware platform, so you have to manage to get your .text section fitting in there.
Also I want to how much flash is utilized by particular function/file.
The nm tool of the GCC binutils provides detailed information about any (global) symbol found in an ELF file and the space it occupies in it's associated section. You'll just need to grep the results for particular functions/classes/namespaces (best demangled!) to accumulate section type and symbol filtered outputs for analysis.
That's the approach, I've been using for a little tool called nmalyzr. Sorry to say, as it stands on the GIT repo, its not really working as intended (I've got working versions, that aren't pushed back).
In general, it's a good strategy to chase for code that has #include <iostream> statements (no matter if std::cout or alike are used or not, static instances are provided!), or unwanted newlib/libstdc++ bindings as for e.g. default exception handling.
Use size command from binutils on the generated elf file. As you seem to use an AVR chip, use avr-size.
To get the size of functions, use nm command from binutils (avr-nm on AVR chips).
I have an upper triangular matrix and the result vector b.
My program need to solve the linear system:
Ax = b
using the pipeline method.
And one of the constraints is that the number of process is smaller than the number of
the equations (let's say it can be from 2 to numberOfEquations-1).
I don't have the code right now, I'm thinking about the pseudo code..
My Idea was that one of the processes will create the random upper triangular matrix (A)
the vector b.
lets say this is the random matrix:
1 2 3 4 5 6
0 1 7 8 9 10
0 0 1 12 13 14
0 0 0 1 16 17
0 0 0 0 1 18
0 0 0 0 0 1
and the vector b is [10 5 8 9 10 5]
and I have a smaller amount of processes than the number of equations (lets say 2 processes)
so what I thought is that some process will send to each process line from the matrix and the relevant number from vector b.
so the last line of the matrix and the last number in vector b will be send to
process[numProcs-1] (here i mean to the last process (process 1) )
than he compute the X and sends the result to process 0.
Now process 0 need to compute the 5 line of the matrix and here i'm stuck..
I have the X that was computed by process 1, but how can the process can send to himself
the next line of the matrix and the relevant number from vector b that need to be computed?
Is it possible? I don't think it's right to send to "myself"
Yes, MPI allows a process to send data to itself but one has to be extra careful about possible deadlocks when blocking operations are used. In that case one usually pairs a non-blocking send with blocking receive or vice versa, or one uses calls like MPI_Sendrecv. Sending a message to self usually ends up with the message simply being memory-copied from the source buffer to the destination one with no networking or other heavy machinery involved.
And no, communicating with self is not necessary a bad thing. The most obvious benefit is that it makes the code more symmetric as it removes/reduces the special logic needed to handle self-interaction. Sending to/receiving from self also happens in most collective communication calls. For example, MPI_Scatter also sends part of the data to the root process. To prevent some send-to-self cases that unnecessarily replicate data and decrease performance, MPI allows in-place mode (MPI_IN_PLACE) for most communication-related collectives.
Is it possible? I don't think it's right to send to "myself"
Sure, it is possible to communicate with oneself. There is even a communicator for it: MPI_COMM_SELF. Talking to yourself is not too uncommon.
Your setup sounds like you would rather use MPI collectives. Have a look at MPI_Scatter and MPI_Gather and see if they don't provide you with the functionality, you are looking for.
I am in a situation where I am extremely bandwidth limited and must devote most of the bandwidth to transferring one type of measurement data. Sometimes I will be sending out lots of this measurement data and other times I will just be waiting for events to occur (all of this is over a TCP socket).
I would like to be able to stream out the full data capture file (different than the measurements) in the background at a speed that is inversely proportional to the amount of measurements that I am sending back.
I am looking for a way to monitor how many bytes are being sent out the network interface in much the same was as the system monitor on Ubuntu. The source code for the system monitor relies on gnome libraries and since my program is on an embedded device, I would like to reduce the number of external libraries that I use. Does anybody know of a way to do this in C/C++ without many additional libraries on a standard Linux distribution?
One of the simplest ways is to parse the file: /proc/net/dev
Mine contains:
Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
lo: 44865 1431 0 0 0 0 0 0 44865 1431 0 0 0 0 0 0
eth0:150117850 313734 0 0 0 0 0 0 34347178 271210 0 0 0 0 0 0
pan0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
You could then write a parser that uses nothing other than the C/C++ libraries.
Bytes transmitted and recieved accessable via
/sys/class/net/eth0/statistics/tx_bytes and /sys/class/net/eth0/statistics/rx_bytes files.
$ cat /sys/class/net/net1/statistics/rx_bytes
1055448
Use NetLink sockets of RTNetLink interface socket, they will get you the required in struct net_device_stats format