How much memory consumed by my object - c++

I want to know programmatic way to get the memory consumed by my user defined class.
Following is the declaration of the class
struct TrieNode {
typedef std::map<char, TrieNode *> ChildType;
std::string m_word;
bool m_visited;
}
I have inserted around 264061 words into this Trie. After this when i do sizeof(trieobject) it just show me 32. How do i know how much exact memory is used by such data structures.

I use
valgrind --tool=massif ./myprogram -opt arg1 arg2
ms_print massif.* | less -SR
for that. Sample output from this page
19.63^ ###
| #
| # ::
| # : :::
| :::::::::# : : ::
| : # : : : ::
| : # : : : : :::
| : # : : : : : ::
| ::::::::::: # : : : : : : :::
| : : # : : : : : : : ::
| ::::: : # : : : : : : : : ::
| ###: : : # : : : : : : : : : #
| ::# : : : # : : : : : : : : : #
| :::: # : : : # : : : : : : : : : #
| ::: : # : : : # : : : : : : : : : #
| ::: : : # : : : # : : : : : : : : : #
| :::: : : : # : : : # : : : : : : : : : #
| ::: : : : : # : : : # : : : : : : : : : #
| :::: : : : : : # : : : # : : : : : : : : : #
| ::: : : : : : : # : : : # : : : : : : : : : #
0 +----------------------------------------------------------------------->KB 0 29.48
Number of snapshots: 25
Detailed snapshots: [9, 14 (peak), 24]
The remainder of the log details the highest percentiles of memory allocations, you can specifically see what type of class takes what % of heap memory (and where the allocations originate in terms of call stack), e.g.:
--------------------------------------------------------------------------------
n time(B) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
10 10,080 10,080 10,000 80 0
11 12,088 12,088 12,000 88 0
12 16,096 16,096 16,000 96 0
13 20,104 20,104 20,000 104 0
14 20,104 20,104 20,000 104 0
99.48% (20,000B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->49.74% (10,000B) 0x804841A: main (example.c:20)
|
->39.79% (8,000B) 0x80483C2: g (example.c:5)
| ->19.90% (4,000B) 0x80483E2: f (example.c:11)
| | ->19.90% (4,000B) 0x8048431: main (example.c:23)
| |
| ->19.90% (4,000B) 0x8048436: main (example.c:25)
|
->09.95% (2,000B) 0x80483DA: f (example.c:10)
->09.95% (2,000B) 0x8048431: main (example.c:23)

Well, this is not so easy to do. First of all m_word is a string with variable size right? Internally the std::string holds an array of chars among other things. The same stands for std::map. I guess you could get a rough estimation based on the size of the map * TrieNode but this will be just a rough estimate.
I think some code profiling with an external tool would be of more help. Hell you can even use the task manager if you are out of any tools left :).

Your "object size" is sizeof(std::string) + sizeof(bool) + m_word.capacity() + padding bytes or sizeof(trieobject) + m_word.capacity()

Here's a piece of code for GCC that I came up with that you can use in a test program where you only instantiate one object of your class and do some typical work with it. The code replaces the global operator new() and operator delete(); so it will only track allocations through ::new expressions and the standard allocator, provided that the standard allocator itself uses ::operator new() (this is the case for GCC).
Since we need to track the pointers and their allocations, we need a separate map for that, which of course cannot use the standard allocator itself; GCC's malloc-allocator comes to the rescue.
We use a statically initialized global to make the memory tracker print its data after main returns.
#include <unordered_map>
#include <string>
#include <iostream>
#include <ext/malloc_allocator.h>
struct Memtrack
{
typedef std::unordered_map<void*, std::size_t, std::hash<void*>,
std::equal_to<void*>, __gnu_cxx::malloc_allocator<void*>> AllocMap;
static int memtrack;
static int memmax;
static AllocMap allocs;
Memtrack() { std::cout << "starting tracker: cur = " << memtrack << ", max = " << memmax << ".\n"; }
~Memtrack() { std::cout << "ending tracker: cur = " << memtrack << ", max = " << memmax << ".\n"; }
static void track_new(std::size_t n, void * p)
{
memtrack += n;
if (memmax < memtrack) memmax = memtrack;
allocs[p] = n;
std::cout << "... allocating " << n << " bytes...\n";
}
static void track_delete(void * p)
{
const int n = int(allocs[p]);
memtrack -= n;
std::cout << "... freeing " << n << " bytes...\n";
}
} m;
int Memtrack::memtrack = 0;
int Memtrack::memmax = 0;
Memtrack::AllocMap Memtrack::allocs;
void * operator new(std::size_t n) throw(std::bad_alloc)
{
void * const p = std::malloc(n);
Memtrack::track_new(n, p);
return p;
}
void operator delete(void * p) throw()
{
Memtrack::track_delete(p);
std::free(p);
}
int main()
{
std::cout << "Beginning of main.\n";
std::unordered_map<std::string, int> m; // this piece of code
m["hello"] = 4; // is a typical test for working
m["world"] = 7; // with dynamic allocations
std::cout << "End of main.\n";
}
Some typical output:
starting tracker: cur = 0, max = 0.
Beginning of main.
... allocating 48 bytes...
... allocating 12 bytes...
... allocating 12 bytes...
End of main.
... freeing 12 bytes...
... freeing 12 bytes...
... freeing 48 bytes...
ending tracker: cur = 0, max = 72.

Trivial. If you have some time (which might be the case, if you are only interested in the size for debugging/optimising purposes). This approach might be unsuited for production code!
#include <malloc.h>
template <typename T> int objSize(T const* obj) {
// instead of uordblks, you may be interested in 'arena', you decide!
int oldSize = mallinfo().uordblks;
T* dummy = new T(*obj);
int newSize = mallinfo().uordblks;
delete dummy;
return newSize - oldSize;
}

Related

pyomo: an indexed variable should be linear or integer depending on the variable index

I have a indexed variable New_UnitsBuilt[p] and this variabele should be integer for the index "GasPowerplant"
but linear for the index "batterystorage".
new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(new_units_built_set, domain=(pyo.NonNegativeIntegers if p="GasPowerplant" else NonNegativeReals)
Please help me how to do this in pyomo.
I am new in pyomo
Best Greetings
Gerhard
There are a couple ways you can accomplish this. For the following, I am assuming that your params.Installable_units = ["GasPowerplant", "batterystorage"]:
If the number of elements in new_units_built_set is small, then you can use a dictionary:
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set,
domain={"GasPowerplant": pyo.NonNegativeIntegers, "batterystorage": pyo.NonNegativeReals})
Or if there are a lot – or there is a simple formula to get the return value – you can use a function (rule):
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
def _new_unitsbuilt_domain(m, p):
return pyo.NonNegativeIntegers if p=="GasPowerplant" else pyo.NonNegativeReals
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set, domain=_new_unitsbuilt_domain)
Or you can just set everything to one value and override later (assuming you are using a ConcreteModel):
model.new_units_built_set = pyo.Set(initialize=list(params.Installable_units))
model.New_UnitsBuilt = pyo.Var(model.new_units_built_set, domain=pyo.NonNegativeReals)
model.New_UnitsBuilt["GasPowerplant"].domain = pyo.NonNegativeIntegers
All of these will produce:
>>> model.pprint()
1 Set Declarations
new_units_built_set : Size=1, Index=None, Ordered=Insertion
Key : Dimen : Domain : Size : Members
None : 1 : Any : 2 : {'GasPowerplant', 'batterystorage'}
1 Var Declarations
New_UnitsBuilt : Size=2, Index=new_units_built_set
Key : Lower : Value : Upper : Fixed : Stale : Domain
GasPowerplant : 0 : None : None : False : True : NonNegativeIntegers
batterystorage : 0 : None : None : False : True : NonNegativeReals
2 Declarations: new_units_built_set New_UnitsBuilt

Get rid of nested for loops with std::ranges

Let I have a code:
for (auto& a : x.as)
{
for (auto& b : a.bs)
{
for (auto& c : b.cs)
{
for (auto& d : c.ds)
{
if (d.e == ..)
{
return ...
}
}
}
}
}
as, bs, cs, ds - std::vector of corresponding elements.
Is it possible with std::ranges to convert four ugly loops into a beatifull one line expression?
With join and transform views, you might do:
for (auto& e : x.as | std::views::transform(&A::bs) | std::views::join
| std::views::transform(&B::cs) | std::views::join
| std::views::transform(&C::ds) | std::views::join
| std::views::transform(&D::e))
{
// ...
}
Demo

Aggregate Initialization Branch Coverage

I am building a tool that is essentially an alternative to lcov. I am trying to make the default branch coverage it generates have as little noise as possible. One source of branch noise seems to be initializer lists:
#include <vector>
#include <string>
using namespace std;
struct A {
vector<string> reference_tokens;
};
int main() {
A a;
vector<string> rt = {"a", "b", "c"};
a.reference_tokens = {rt[0]};
return 0;
}
When I generate coverage for this snippet, I get:
9 : : struct A {
10 : : vector<string> reference_tokens;
11 : : };
12 : :
13 : 1 : int main() {
14 : 2 : A a;
15 : :
16 [ + - ][ + - ]: 6 : vector<string> rt = {"a", "b", "c"};
[ + - ][ + - ]
17 : :
18 [ + - ][ + - ]: 2 : a.reference_tokens = {rt[0]};
[ + + ][ - - ]
19 : 1 : return 0;
20 : : }
Now, I realize GCC inserts branches for handling exceptions. However, if I filter out exceptional branches, I'm still left with:
9 : : struct A {
10 : : vector<string> reference_tokens;
11 : : };
12 : :
13 : 1 : int main() {
14 : 2 : A a;
15 : :
16 : 6 : vector<string> rt = {"a", "b", "c"};
17 : :
18 [ + + ][ - - ]: 2 : a.reference_tokens = {rt[0]};
19 : 1 : return 0;
20 : : }
I'm not sure what these 4 branches [ + + ][ - - ] are for. They are not exceptional branches according to gcov, and it seems they always are 1:1. For example, a bigger initializer list will result in [ + + ][ + + ][ + + ][ - - ][ - - ][ - - ] non-exceptional branches.
So my question is... what are these branches? Are the reachable? Are they noise that can be safely removed?

Informatica error : RowType=-4(error but dropped due to abort)

I am getting the below error:
There is an error in the port [NEXTVAL]:
The default value for the port is set to: ERROR(<>
[ERROR]: transformation error ... nl:ERROR(u:'transformation error')).
2015-11-27 08:00:45 : ERROR : (582188 | TRANSF_1_1_1) : (IS | Oracle_BI_DW_Server) :
node01_ : TT_11023 : An error occurred converting data in the port [NEXTVAL]. The row was dropped.
2015-11-27 08:00:45 : ERROR : (582188 | TRANSF_1_1_1) : (IS | Oracle_BI_DW_Server) :
node01_ : TT_11021 :
An error occurred moving data from the transformation RECON_SEQ_GEN: to the transformation FILTRANS.
2015-11-27 08:00:45 : ERROR : (582188 | TRANSF_1_1_1) : (IS | Oracle_BI_DW_Server) :
node01 : CMN_1086 : RECON_SEQ_GEN: Number of errors exceeded threshold [1].
2015-11-27 08:00:45 : INFO : (582188 | TRANSF_1_1_1) : (IS | Oracle_BI_DW_Server) :
node01_ : CMN_1053 : : Rowdata:
( RowType=-4(error but dropped due to abort) Src Rowid=1186281 Targ Rowid=1186281
NEXTVAL (ROW_WID:Int:): "(NULL)"
)

Returning list in ANTLR for type checking, language java

I am working on ANLTR to support type checking. I am in trouble at some point. I will try to explain it with an example grammar, suppose that I have the following:
#members {
private java.util.HashMap<String, String> mapping = new java.util.HashMap<String, String>();
}
var_dec
: type_specifiers d=dec_list? SEMICOLON
{
mapping.put($d.ids.get(0).toString(), $type_specifiers.type_name);
System.out.println("identext = " + $d.ids.get(0).toString() + " - " + $type_specifiers.type_name);
};
type_specifiers returns [String type_name]
: 'int' { $type_name = "int";}
| 'float' {$type_name = "float"; }
;
dec_list returns [List ids]
: ( a += ID brackets*) (COMMA ( a += ID brackets* ) )*
{$ids = $a;}
;
brackets : LBRACKET (ICONST | ID) RBRACKET;
ID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*;
LBRACKET : '[';
RBRACKET : ']';
In rule dec_list, you will see that I am returning List with ids. However, in var_dec when I try to put the first element of the list (I am using only get(0) just to see the return value from dec_list rule, I can iterate it later, that's not my point) into mapping I get a whole string like
[#4,6:6='a',<17>,1:6]
for an input
int a, b;
What I am trying to do is to get text of each ID, in this case a and b in the list of index 0 and 1, respectively.
Does anyone have any idea?
The += operator creates a List of Tokens, not just the text these Tokens match. You'll need to initialize the List in the #init{...} block of the rule and add the inner-text of the tokens yourself.
Also, you don't need to do this:
type_specifiers returns [String type_name]
: 'int' { $type_name = "int";}
| ...
;
simply access type_specifiers's text attribute from the rule you use it in and remove the returns statement, like this:
var_dec
: t=type_specifiers ... {System.out.println($t.text);}
;
type_specifiers
: 'int'
| ...
;
Try something like this:
grammar T;
var_dec
: type dec_list? ';'
{
System.out.println("type = " + $type.text);
System.out.println("ids = " + $dec_list.ids);
}
;
type
: Int
| Float
;
dec_list returns [List ids]
#init{$ids = new ArrayList();}
: a=ID {$ids.add($a.text);} (',' b=ID {$ids.add($b.text);})*
;
Int : 'int';
Float : 'float';
ID : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*;
Space : ' ' {skip();};
which will print the following to the console:
type = int
ids = [a, b, foo]
If you run the following class:
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
TLexer lexer = new TLexer(new ANTLRStringStream("int a, b, foo;"));
TParser parser = new TParser(new CommonTokenStream(lexer));
parser.var_dec();
}
}