I'm using open62541 to connect to an OPC/UA server and I'm trying to call methods that a certain object on that server provides. Those methods have custom types as input arguments; for example, the following method takes a structure of three booleans:
<opc:Method SymbolicName="SetStatusMethodType" ModellingRule="Mandatory">
<opc:InputArguments>
<opc:Argument Name="Status" DataType="VisionStatusDataType" ValueRank="Scalar"/>
</opc:InputArguments>
<opc:OutputArguments />
</opc:Method>
Here, VisionStatusDataType is the following structure:
<opc:DataType SymbolicName="VisionStatusDataType" BaseType="ua:Structure">
<opc:ClassName>VisionStatus</opc:ClassName>
<opc:Fields>
<opc:Field Name="Camera" DataType="ua:Boolean" ValueRank="Scalar"/>
<opc:Field Name="StrobeController" DataType="ua:Boolean" ValueRank="Scalar"/>
<opc:Field Name="Server" DataType="ua:Boolean" ValueRank="Scalar"/>
</opc:Fields>
</opc:DataType>
Now, when calling the method, I'm encoding the data into an UA_ExtensionObject, and wrap that one as an UA_Variant to provide it to UA_Client_call. The encoding looks like this:
void encode(const QVariantList& vecqVar, size_t& nIdx, const DataType& dt, std::back_insert_iterator<std::vector<UAptr<UA_ByteString>>> itOut)
{
if (dt.isSimple())
{
auto&& qVar = vecqVar.at(nIdx++);
auto&& uaVar = convertToUaVar(qVar, dt.uaType());
auto pOutBuf = create<UA_ByteString>();
auto nStatus = UA_encodeBinary(uaVar.data, dt.uaType(), pOutBuf.get());
statusCheck(nStatus);
itOut = std::move(pOutBuf);
}
else
{
for (auto&& dtMember : dt.members())
encode(vecqVar, nIdx, dtMember, itOut);
}
}
UA_Variant ToUAVariant(const QVariant& qVar, const DataType& dt)
{
if (dt.isSimple())
return convertToUaVar(qVar, dt.uaType());
else
{
std::vector<UAptr<UA_ByteString>> vecByteStr;
auto&& qVarList = qVar.toList();
size_t nIdx = 0UL;
encode(qVarList, nIdx, dt, std::back_inserter(vecByteStr));
auto pExtObj = UA_ExtensionObject_new();
pExtObj->encoding = UA_EXTENSIONOBJECT_ENCODED_BYTESTRING;
auto nSizeAll = std::accumulate(vecByteStr.cbegin(), vecByteStr.cend(), 0ULL, [](size_t nSize, const UAptr<UA_ByteString>& pByteStr) {
return nSize + pByteStr->length;
});
auto&& uaEncoded = pExtObj->content.encoded;
uaEncoded.typeId = dt.uaType()->typeId;
uaEncoded.body.length = nSizeAll;
auto pData = uaEncoded.body.data = new UA_Byte[nSizeAll];
nIdx = 0UL;
for (auto&& pByteStr : vecByteStr)
{
memcpy_s(pData + nIdx, nSizeAll - nIdx, pByteStr->data, pByteStr->length);
nIdx += pByteStr->length;
}
UA_Variant uaVar;
UA_Variant_init(&uaVar);
UA_Variant_setScalar(&uaVar, pExtObj, &UA_TYPES[UA_TYPES_EXTENSIONOBJECT]);
return uaVar;
}
}
The DataType class is a wrapper for the UA_DataType structure; the original open62541 type can be accessed via DataType::uaType().
Now, once a have the variant (containing the extension object), the method call looks like this:
auto uavarInput = ToUAVariant(qvarArg, dtInput);
UA_Variant* pvarOut;
size_t nOutSize = 0UL;
auto nStatus = UA_Client_call(m_pClient, objNode.nodeId(), m_uaNodeId, 1UL, &uavarInput, &nOutSize, &pvarOut);
The status is 2158690304, i.e. BadInvalidArgument according to UA_StatusCode_name.
Is there really something wrong with the method argument? Are we supposed to send ExtensionObjects, or what data type should the variant contain?
Is it possible that the server itself (created using the .NET OPC/UA stack) is not configured correctly?
N.B., the types here are custom types; that is, the encoding is done manually (see above) by storing the byte representation of all members next to each other in an UA_ByteString - just the opposite of what I'm doing when reading variables or output arguments, which works just fine.
The problem is the typeId of the encoded object. For the server in order to understand the received data, it needs to know the NodeId of the encoding, not the actual NodeId of the type itself. That encoding can be found by following the HasEncoding reference (named "Default Binary") of the type:
auto pRequest = create<UA_BrowseRequest>();
auto pDescr = pRequest->nodesToBrowse = UA_BrowseDescription_new();
pRequest->nodesToBrowseSize = 1UL;
pDescr->nodeId = m_uaNodeId;
pDescr->resultMask = UA_BROWSERESULTMASK_ALL;
pDescr->browseDirection = UA_BROWSEDIRECTION_BOTH;
pDescr->referenceTypeId = UA_NODEID_NUMERIC(0, UA_NS0ID_HASENCODING);
auto response = UA_Client_Service_browse(m_pClient, *pRequest);
for (auto k = 0UL; k < response.resultsSize; ++k)
{
auto browseRes = response.results[k];
for (auto n = 0UL; n < browseRes.referencesSize; ++n)
{
auto browseRef = browseRes.references[n];
if (ToQString(browseRef.browseName.name).contains("Binary"))
{
m_nodeBinaryEnc = browseRef.nodeId.nodeId;
break;
}
}
}
Once you have that NodeId, you pass it to UA_ExtensionObject::content::encoded::typeId:
auto pExtObj = UA_ExtensionObject_new();
pExtObj->encoding = UA_EXTENSIONOBJECT_ENCODED_BYTESTRING;
auto nSizeAll = std::accumulate(vecByteStr.cbegin(), vecByteStr.cend(), 0ULL, [](size_t nSize, const UAptr<UA_ByteString>& pByteStr) {
return nSize + pByteStr->length;
});
auto&& uaEncoded = pExtObj->content.encoded;
uaEncoded.typeId = dt.encoding();
uaEncoded.body.length = nSizeAll;
auto pData = uaEncoded.body.data = new UA_Byte[nSizeAll];
nIdx = 0UL;
for (auto&& pByteStr : vecByteStr)
{
memcpy_s(pData + nIdx, nSizeAll - nIdx, pByteStr->data, pByteStr->length);
nIdx += pByteStr->length;
}
I don't understand memory management in C++ Arrow API. I use Arrow 1.0.0 and I'm reading CSV file. After a few runs of ReadArrowTableFromCSV, my memory is full of allocated data. Am I missing something? How can I free that memory? I don't see any method in memory pool to clear all allocated memory. Code Listing below.
void LoadCSVData::ReadArrowTableFromCSV( const std::string & filePath )
{
auto tableReader = CreateTableReader( filePath );
ReadArrowTableUsingReader( *tableReader );
}
std::shared_ptr<arrow::csv::TableReader> LoadCSVData::CreateTableReader( const std::string & filePath )
{
arrow::MemoryPool* pool = arrow::default_memory_pool();
auto tableReader = arrow::csv::TableReader::Make( pool, OpenCSVFile( filePath ),
*PrepareReadOptions(), *PrepareParseOptions(), *PrepareConvertOptions() );
if ( !tableReader.ok() )
{
throw BadParametersException( std::string( "CSV file reader error: " ) + tableReader.status().ToString() );
}
return *tableReader;
}
void LoadCSVData::ReadArrowTableUsingReader( arrow::csv::TableReader & reader )
{
auto table = reader.Read();
if ( !table.ok() )
{
throw BadParametersException( std::string( "CSV file reader error: " ) + table.status().ToString() );
}
this->mArrowTable = *table;
}
std::unique_ptr<arrow::csv::ParseOptions> LoadCSVData::PrepareParseOptions()
{
auto parseOptions = std::make_unique<arrow::csv::ParseOptions>( arrow::csv::ParseOptions::Defaults() );
parseOptions->delimiter = mDelimiter;
return parseOptions;
}
std::unique_ptr<arrow::csv::ReadOptions> LoadCSVData::PrepareReadOptions()
{
auto readOptions = std::make_unique<arrow::csv::ReadOptions>( arrow::csv::ReadOptions::Defaults() );
readOptions->skip_rows = mNumberOfHeaderRows;
readOptions->block_size = 1 << 27; // 128 MB
readOptions->column_names.reserve( mTable->GetNumberOfColumns() );
for ( auto & colName : mTable->GetColumnsOrder() )
{
readOptions->column_names.emplace_back( colName );
}
return readOptions;
}
std::unique_ptr<arrow::csv::ConvertOptions> LoadCSVData::PrepareConvertOptions() const
{
auto convertOptions = std::make_unique<arrow::csv::ConvertOptions>( arrow::csv::ConvertOptions::Defaults() );
for ( auto & col : mTable->GetColumsInfo() )
{
convertOptions->column_types[col.second.GetName()] = MyTypeToArrowDataType( col.second.GetType() );
}
convertOptions->strings_can_be_null = true;
return convertOptions;
}
std::shared_ptr<arrow::io::ReadableFile> LoadCSVData::OpenCSVFile( const std::string & filePath )
{
MTR_SCOPE_FUNC();
auto inputFileResult = arrow::io::ReadableFile::Open( filePath );
if ( !inputFileResult.ok() )
{
throw BadParametersException( std::string( "CSV file reader error: " ) + inputFileResult.status().ToString() );
}
return *inputFileResult;
}
Maciej, the method TableReader::Read should return shared_ptr<arrow::Table>. The arrow::Table itself has a number of shared pointers to structures which eventually contain the data. To free up the data you will need to make sure that the arrow::Table and any copies of it are destroyed. This should happen as soon as that shared_ptr goes out of scope. However, it appears you are storing the table in a member variable here (which is to be expected, you probably want to use the data after you read it):
this->mArrowTable = *table;
So now you have a second copy of the arrow::Table instance. You could reassign this->mArrowTable to a new blank table or you could destroy whatever this is. Of course, if you are making any other copies of the table then you will need to ensure those go out of scope as well.
Please help me fulfill my dreams of turning this sequence into a meaningful output. :)
See regex in action, it works!: http://regex101.com/r/iM4yN2/1
Now all I need is to know how to use it. If I could put this into a multidimensional array e.g. configFile[0][0] = [Tuner,] that would work. Or if I could turn this into a comma separated list, I could then parse that again and put it into arrays and finally out to individual variables. Anyway, you don't need to spell out how to actually assign the variables, I'll create another question if I really need help with that. Mainly I need help with the use of regex functions and outputting data into SOME variable where I can access the various text on either side of the = sign per line.
regex:
^[\t ]*(.*?)\s*=[\t ]*(.*?)(#.*)?$
test string:
### MODULES ###
Tuner =
PitchDetector = 0
PhaseLocker = 0
FileOutput = 1
### FILE MANAGER ###
RenameFile_AvgFreq = dfgsdfg dsf gdfs g #gdrgk
RenameFile_NoteName = 0
RenameFile_Prefix = "The String Is Good"
RenameFile_Suffix = ""
OutputFolder = "..\Folder\String\"
### PITCH DETECTOR ###
AnalysisChannel = 1 #int starting from 1
BlockSize = 8 #power of 2
Overlap = 16 #power of 2
NormalizeForDetection = 0
### TUNER ###
Smoothing = 0.68
Envelope = 0.45
### PHASELOCKER ###
FFTSize = 1024 #powert of 2
FFTOverlap = 54687
WindowType = 0
MaxFreq = 5000
my variables:
//Modules
bool Tuner;
bool PitchDetector;
bool PhaseLocker;
bool FileOutput;
//File Manager
bool RenameFile_AvgFreq;
bool RenameFile_NoteName;
std::string RenameFile_Prefix;
std::string RenameFile_Suffix;
std::string OutputFolder;
//Pitch Detector
int AnalysisChannel;
int BlockSize;
int Overlap;
bool NormalizeForDetection;
//Tuner
float Smoothing;
float Envelope;
//Phaselocker
int FFTSize;
int FFTOverlap;
int FFTWindowType;
float FFTMaxFreq;
final notes: i spent a long time looking at c++ regex functions... very confusing stuff. I know how to do this in python without thinking twice.
Include the following:
#include <string>
#include <regex>
Declare a string and regex type:
std::string s;
std::regex e;
In your main function, assign string and regex variables and call regex function (you could assign the variables when you declare them as well):
int main()
{
s="i will only 349 output 853 the numbers 666"
e="(\\d+)"
s = std::regex_replace(s, e, "$1\n", std::regex_constants::format_no_copy);
return 0;
}
Notice how I am putting the results right back into the string (s). Of course, you could use a different string to store the result. The "std::regex_constants::format_no_copy" is a flag that tells the regex function to output only "substrings" aka group matches. Also notice how I am using double slash on the "\d+". Try double slashes if your regex pattern isn't working.
To find key/value pairs with regex, e.g. "BlockSize = 1024", you could create a pattern such as:
BlockSize\s*=\s*((?:[\d.]+)|(?:".*"))
in c++ you could create that regex pattern with:
expr = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";
and return the match with:
config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);
and put it all together in a function with the ability to return a default value:
std::string Config_GetValue(std::string key, std::string config, std::string defval)
{
std::regex expr;
match = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";
config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);
return config == "" ? defval : config;
}
FULL CODE (using std::stoi and std::stof to convert string to number when needed, and using auto type because right-hand side (RHS) makes it clear what the type is):
#include "stdafx.h"
#include <string>
#include <regex>
#include <iostream>
std::string Config_GetValue(std::string key, std::string config, std::string defval)
{
std::regex expr;
match = key+"\\s*=\\s*((?:[\\d.]+)|(?:\".*\"))";
config = std::regex_replace(config, expr, "$1", std::regex_constants::format_no_copy);
return config == "" ? defval : config;
}
int main()
{
//test string
std::string s = " ### MODULES ###\nTuner = \n PitchDetector = 1\n PhaseLocker = 0 \nFileOutput = 1\n\n### FILE MANAGER ###\nRenameFile_AvgFreq = dfgsdfg dsf gdfs g #gdrgk\nRenameFile_NoteName = 0\n RenameFile_Prefix = \"The String Is Good\"\nRenameFile_Suffix = \"\"\nOutputFolder = \"..\\Folder\\String\\\"\n\n### PITCH DETECTOR ###\nAnalysisChannel = 1 #int starting from 1\nBlockSize = 1024 #power of 2\nOverlap = 16 #power of 2\nNormalizeForDetection = 0\n\n### TUNER ###\nSmoothing = 0.68\nEnvelope = 0.45\n\n### PHASELOCKER ###\nFFTSize = 1024 #powert of 2\nFFTOverlap = 54687\nWindowType = 0\nMaxFreq = 5000";
//Modules
auto FileOutput = stoi(Config_GetValue("FileOutput", s, "0"));
auto PitchDetector = stoi(Config_GetValue("PitchDetector", s, "0"));
auto Tuner = stoi(Config_GetValue("Tuner", s, "0"));
auto PhaseLocker = stoi(Config_GetValue("PhaseLocker", s, "0"));
//File Manager
auto RenameFile_AvgFreq = stoi(Config_GetValue("RenameFile_AvgFreq", s, "0"));
auto RenameFile_NoteName = stoi(Config_GetValue("RenameFile_NoteName", s, "0"));
auto RenameFile_Prefix = Config_GetValue("RenameFile_Prefix", s, "");
auto RenameFile_Suffix = Config_GetValue("RenameFile_Suffix", s, "");
auto OutputFolder = Config_GetValue("FileOutput", s, "");
//Pitch Detector
auto AnalysisChannel = stoi(Config_GetValue("AnalysisChannel", s, "1"));
auto BlockSize = stoi(Config_GetValue("BlockSize", s, "4096"));
auto Overlap = stoi(Config_GetValue("Overlap", s, "8"));
auto NormalizeForDetection = stoi(Config_GetValue("NormalizeForDetection", s, "0"));
//Tuner
auto Smoothing = stof(Config_GetValue("Smoothing", s, ".5"));
auto Envelope = stof(Config_GetValue("Envelope", s, ".3"));
auto TransientTime = stof(Config_GetValue("TransientTime", s, "0"));
//Phaselocker
auto FFTSize = stoi(Config_GetValue("FFTSize", s, "1"));
auto FFTOverlap = stoi(Config_GetValue("FFTOverlap", s, "1"));
auto FFTWindowType = stoi(Config_GetValue("FFTWindowType", s, "1"));
auto FFTMaxFreq = stof(Config_GetValue("FFTMaxFreq", s, "0.0"));
std::cout << "complete";
return 0;
}
Another way of doing this is with regex_iterator:
#include <regex>
using std::regex;
using std::sregex_iterator;
void CreateConfig(string config)
{
//group 1,2,3,4,5 = key,float,int,string,bool
regex expr("^[\\t ]*(\\w+)[\\t ]*=[\\t ]*(?:(\\d+\\.+\\d+|\\.\\d+|\\d+\\.)|(\\d+)|(\"[^\\r\\n:]*\")|(TRUE|FALSE))[^\\r\\n]*$", std::regex_constants::icase);
for (sregex_iterator it(config.begin(), config.end(), expr), itEnd; it != itEnd; ++it)
{
if ((*it)[2] != "") cout << "FLOAT -> " << (*it)[1] << " = " <<(*it)[2] << endl;
else if ((*it)[3] != "") cout << "INT -> " << (*it)[1] << " = " <<(*it)[3] << endl;
else if ((*it)[4] != "") cout << "STRING -> " << (*it)[1] << " = " <<(*it)[4] << endl;
else if ((*it)[5] != "") cout << "BOOL -> " << (*it)[1] << " = " << (*it)[5] << endl;
}
}
int main()
{
string s = "what = 1\n: MODULES\nFileOutput = \"on\" :bool\nPitchDetector = TRuE :bool\nTuner = on:bool\nHarmSplitter = off:bool\nPhaseLocker = on\n\nyes\n junk output = \"yes\"\n\n: FILE MANAGER\nRenameFile AvgFreq = 1 \nRenameFile_NoteName = 0 :bool\nRenameFile_Prefix = \"The Strin:g Is Good\" :string\nRenameFile_Suffix = \"\":string\nOutputFolder = \"..\\Folder\\String\\\" :relative path\n\n: PITCH DETECTOR\nAnalysisChannel = 1 :integer starting from 1\nBlockSize = 8 :power of 2\nOverlap = 16 :power of 2\nNormalizeForDetection = 0 :bool\n\n: TUNER\nSmoothing = 0.68 :float\nEnvelope = 0.45 :float\n\n: PHASE LOCKER\nFFTSize = 1024 :power of 2\nFFTOverlap = 54687 :power of 2\nWindowType = 0 :always set to 0\nMaxFreq = 5000 :float";
CreateConfig(s);
return 0;
}
Let's break this down. The regex expression I created uses a ^regexy stuff goes here$ format so that each line of text is considered individually: ^=start of line, $=end of line. The regex looks for: variable_name = decimal OR number OR string OR (true OR false). Because each type is stored in its own group, we know what type every match is going to be.
To explain the for loop, I will write the code a few different ways
//You can declare more than one variable of the same type:
for (sregex_iterator var1(str.begin(), str.end(), regexExpr), var2); var1 != var2; var1++)
//Or you can delcare it outside the for loop:
sregex_iterator var1(str.begin(), str.end(), regexExpr);
sregex_iterator var2;
for (; var1 != var2; var1++)
//Or the more classic way:
sregex_iterator var1(str.begin(), str.end(), regexExpr);
for (sregex_iterator var2; var1 != var2; var1++)
Now for the body of the for loop. It says "If group2 is not blank, print group 2 which is a float. If gorup3 is not blank, print group3 which is an int. If group4 is not blank, print group 4 which is a string. If group5 is not blank, print group5 which is a bool. When inside a loop, the syntax is:
//group0 is some kind of "currently evaluating" string plus group matches.
//group1 is my key group
//group2/3/4/5 are my values groups float/int/string/bool.
theString = (*iteratorVariableName)[groupNumber]
Given a gdcm tag, e.g. gdcm::Tag(0x0010,0x0010) how can it be convert to the corresponding tag name string, in this case "PatientsName" in C++?
Here is what I do in a Qt based application with GDCM:
QString duDicomDictionary::getTagName( const gdcm::Tag & tag )
{
QString retVal;
const gdcm::Global& g = gdcm::Global::GetInstance();
const gdcm::Dicts &dicts = g.GetDicts();
const gdcm::Dict &pubdict = dicts.GetPublicDict();
gdcm::DictEntry ent = pubdict.GetDictEntry(tag);
if (ent.GetVR() != gdcm::VR::INVALID ) {
retVal = QString::fromStdString(ent.GetName());
}
return retVal;
}
This code will only work for public groups.
To get the private groups I use (after I populated the private dictionary):
QString duDicomDictionary::getTagName( const gdcm::PrivateTag & tag )
{
QString retVal;
const gdcm::Global& g = gdcm::Global::GetInstance();
const gdcm::Dicts &dicts = g.GetDicts();
const gdcm::PrivateDict &privdict = dicts.GetPrivateDict();
gdcm::DictEntry ent = privdict.GetDictEntry(tag);
if (ent.GetVR() != gdcm::VR::INVALID ) {
retVal = QString::fromStdString(ent.GetName());
}
else
{
ent = g_privateDict.GetDictEntry(tag);
if (ent.GetVR() != gdcm::VR::INVALID ) {
retVal = QString::fromStdString(ent.GetName());
}
}
return retVal;
}
it seems I am unable to find any source telling me how to use lists with libconfig.
Let's say my config file looks like this:
Layer1 = {
Layer2 = {
SomeOption = "MyValue";
Options = (
{
Option = "Full Screen";
Value = "No";
},
{
Option = "Title";
Value = "Test";
}
);
};
};
How can I read Options with libconfig++ ? I can only find methods for reading single values.
OK, I have found the answer:
using namespace libconfig;
// ...
Config *pConfig = new Config();
// ...
Setting& settings = pConfig->lookup("Layer1.Layer2.Options");
const char* op0 = settings[0]["Option"];
// ...