C++ char arrays - same input giving different outputs - c++

Question

I have a cpp class that implements a Netezza User-defined function (documentation here). It takes a argument that will be a string of some date format, and converts it to YYYYMMDD format. If it isn't a valid date, it will return "99991231". Whenever I run the code on some tables, I get different outputs every time for the same inputs. I assume there is some memory issue that I am not seeing.
Logically, we set the char array retval equal to the output of the date command. If it gave a null output, we set to "99991231". Then we set a temp char array to the first 9 bytes of retval (last one being the null terminator). Then we memcpy into ret->data (a char ptr of the struct we must return).
#include <stdarg.h>
#include <string.h>
#include "udxinc.h"
#include "udxhelpers.h"
using namespace nz::udx_ver2;
class Dateconvert: public Udf
{
public:
Dateconvert(UdxInit *pInit) : Udf(pInit){}
~Dateconvert(){}
static Udf* instantiate(UdxInit *pInit);
virtual ReturnValue evaluate()
{
StringReturn* ret = stringReturnInfo();
StringArg *str;
str = stringArg(0);
int lengths = str->length;
char *datas = str->data;
string tempData = datas;
string shell_arg = tempData;
shell_arg = "'" + shell_arg + "'";
string cmd="date -d " + shell_arg + " +%Y%m%d 2>/dev/null";
FILE *ls = popen(cmd.c_str(), "r");
char retval[100];
retval[0]='n';
fgets(retval, sizeof(retval), ls);
if(!isdigit(retval[0]))
{
strcpy(retval,"99991231");
}
pclose(ls);
char temp1[9];
memcpy(temp1, retval, 8);
temp1[8]='\0';
ret->size = 9;
memcpy(ret->data, temp1, 9);
NZ_UDX_RETURN_STRING(ret);
}
};
Udf* Dateconvert::instantiate(UdxInit *pInit)
{
return new Dateconvert(pInit);
}
When I run the UDF on one distinct value in Netezza, it gives me the expected output. However, when I run it over multiple columns, the output is sometimes correct, sometimes wrong, seemingly randomly. I assume this has to be an internal memory issue. Examples:
input output
1) 8/11/2014 20140811
2) 8/11/2014 20140811
Fri 10/17/14 20141017
3) 8/11/2014 99991231
Fri 10/17/14 20141017
4) 8/11/2014 20140811
Fri 10/17/14 20141017
5) 8/11/2014 20140811
Fri 10/17/14 20141017
9-Nov-12 20121109
6) 8/11/2014 20140811
Fri 10/17/14 20141017
9-Nov-12 01241109 (what?)
7) 8/11/2014 99991231
Fri 10/17/14 20141017
9-Nov-12 20121109
Anytime there is only one call of the function, it returns the correct answer. The problem arises when it is called multiple times, which I don't understand. Why would anything be carrying over? Changing the return value size to 8 from 9 at the end of the evaluate function does not solve the issue.
This is the format by which the function is called:
select a.val1, DATECONVERT(a.val1)
from
(
select '8/11/2014' as val1 from calendar
union
select 'Fri 10/17/14' as val1 from calendar
union
select '9-Nov-12' as val1 from calendar
) a
And compile command for the UDF:
nzudxcompile /export/home/nz/dateconvert.cpp -o dateconvert.o --sig "Dateconvert(VARCHAR(200))" --version 2 --return "VARCHAR(200)" --class Dateconvert --user user1 --pw mypw --db mydb

Answer 1

To cut to the chase, the problem here is in how you assign tempData.
StringReturn* ret = stringReturnInfo();
StringArg *str;
str = stringArg(0);
int lengths = str->length;
char *datas = str->data;
string tempData = datas;
StringArg does not store a NUL-terminated string, but instead provides the length and expects you to manage that yourself.
select a.val1, ADMIN.DATECONVERT(a.val1)
from
(
select '09-Nov-12'::varchar(20) as val1
union all
select '9-Nov-12'::varchar(20) as val1
) a;
VAL1 | DATECONVERT
-----------+-------------
09-Nov-12 | 20121109
9-Nov-12 | 01221109
(2 rows)
In this example, what is happening is that the longer first string still has a character hanging around in memory when the second, shorter string is assigned to tempData. That hanging '2' at the end gets added on like so:
09-Nov-12
9-Nov-122
Each of these are both valid inputs to date, which nicely explains the output you are seeing.
$ date -d 09-Nov-12 +%Y%m%d
20121109
$ date -d 09-Nov-122 +%Y%m%d
01221109
Change the assignment to use that length, and you'll avoid the problem.
//string tempData = datas;
string tempData(datas, datas+lengths);
And then you get the expected output:
select a.val1, ADMIN.DATECONVERT(a.val1)
from
(
select '09-Nov-12'::varchar(20) as val1
union all
select '9-Nov-12'::varchar(20) as val1
) a;
VAL1 | DATECONVERT
-----------+-------------
09-Nov-12 | 20121109
9-Nov-12 | 20121109
(2 rows)
All that being said, I don't know that the overall approach you are taking in this UDF will work. As I'm running it above, the rows are generated on the host, because they are hard-coded in the SQL, and date is certainly available on the host. However, you can't expect the code that runs on the MPP backend (which we often refer to as SPUs) to have the same availability of linux utilities that you will find on the host, or if they exist that they have the same capabilities.
If I move the date into an actual table, the UDF will operate on it on the SPU, and it will give me bad output because the date command on the SPU image is significantly different than that of the host, and doesn't understand this input format at all.
select a.col1, admin.DATECONVERT(a.col1) from calendar a;
COL1 | DATECONVERT
-----------+-------------
09-Nov-12 | 99991231
9-Nov-12 | 99991231
(2 rows)