CTE with HierarchyID suddenly causes parse error - common-table-expression

So I have this self-referencing table in my database named Nodes, used for storing the tree structure of an organization:
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](max) NULL,
[ParentId] [int] NULL,
(+ other metadata columns)
And from it I'm using HIERARCHYID to manage queries based on access levels and such. I wrote a table-valued function for this, tvf_OrgNodes, a long time ago, tested and working on SQL Server 2008 through 2014 and it's remained unchanged since then since it's been doing great. Now, however, something has changed because the parsing of HIERARCHYIDs from path nvarchars ("/2/10/8/") results in the following error, matching only 4 hits (!) on Google:
Msg 6522, Level 16, State 2, Line 26
A .NET Framework error occurred during execution of user-defined routine or aggregate "hierarchyid":
Microsoft.SqlServer.Types.HierarchyIdException: 24000: SqlHierarchyId operation failed because HierarchyId object was constructed from an invalid binary string.
When altering the function to only return NVARCHAR instead of actual HIERARCHYID's, the paths all look fine, beginning with / for the root, followed by /2/ etc etc. Simply selecting HIERARCHYID::Parse('path') also works fine. I actually got the function working by leaving the paths as strings all the way until the INSERT into the function result, parsing the paths there. But alas, I get the same error when I then try and insert the reusulting data into a table of same schema.
So the question is, Is this a bug, or does anybody know of any (new?) pitfalls in working with HIERARCHYIDs<->Path strings that could cause this? I don't get where the whole binary string idea comes from.
This is the code of the TVF:
CREATE FUNCTION [dbo].[tvf_OrgNodes] ()
RETURNS #OrgNodes TABLE (
OrgNode HIERARCHYID,
NodeId INT,
OrgLevel INT,
ParentNodeId INT
) AS
BEGIN
WITH orgTree(OrgNode, NodeId, OrgLevel, ParentNodeId) AS (
-- Anchor expression = root node
SELECT
CAST(HIERARCHYID::GetRoot() AS varchar(180))
, n.Id
, 0
, NULL
FROM Nodes n
WHERE ParentId IS NULL -- Top level
UNION ALL
-- Recursive expression = organization tree
SELECT
CAST(orgTree.OrgNode + CAST(n.Id AS VARCHAR(180)) + N'/' AS VARCHAR(180))
, n.Id
, orgTree.OrgLevel + 1
, n.ParentId
FROM Nodes AS n
JOIN orgTree
ON n.ParentId = orgTree.NodeId
)
INSERT INTO #OrgNodes
SELECT
HIERARCHYID::Parse(OrgNode),
NodeId,
OrgLevel,
ParentNodeId
FROM orgTree;
RETURN;
END
I might have recently installed .NET 4.53 aka 4.6 for the lolz. Can't find much proof of it anywhere except in the reg, though: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\v4.0.30319\SKUs.NETFramework,Version=v4.5.3

Related

display the content of a file split by a delimiter character

I am trying to display the content of a file, split by a delimiter character.
More exactly, starting from this topic, I am trying to display the result as:
bbb
aaa
qqq
ccc
but the data source to be taken from a file.
Until now, I tried:
DECLARE
l_bfile bfile;
BEGIN
l_bfile := bfilename(my_dir, my_file);
dbms_lob.fileopen(l_bfile);
FOR i IN
(SELECT TRIM(regexp_substr(TO_CHAR(l_bfile),'[^;]+',1,level) ) AS q
FROM dual
CONNECT BY regexp_substr(TO_CHAR(l_bfile),'[^;]+',1,level) IS NOT NULL
ORDER BY level
)
LOOP
dbms_output.put_line(i.q);
END LOOP;
EXCEPTION
WHEN No_Data_Found THEN
NULL;
END;
As result, I got
PL/SQL: ORA-00932: inconsistent datatypes: expected NUMBER got FILE
Can anyone give me a hint, please?
Have to write this as a new answer since this is too big for a comment to #SmartDumb:
Be advised the regex of the form '[^;]+' (commonly used for parsing delimited lists) fails when NULL elements are found in the list. Please see this post for more information: https://stackoverflow.com/a/31464699/2543416
Instead please use this form of the call to regexp_substr (note I removed the second element):
SELECT TRIM(regexp_substr('bbb;;qqq;ccc','(.*?)(;|$)',1,level, null, 1) ) AS q
FROM dual
CONNECT BY regexp_substr('bbb;;qqq;ccc','(.*?)(;|$)',1,level) IS NOT NULL
ORDER BY level
It may or may not be important in this example, it depends on if the order of the element in the string has importance to you or if you need to preserve the NULL. i.e. if you need to know the second element is NULL then this will work.
P.S. Do a search for external tables and see if that is a solution you could use. That would let you query a file as if it were a table.
You could possible try this if your file contains single line (hence the question about file structure):
DECLARE
utlFileHandle UTL_FILE.FILE_TYPE;
vLine varchar2(100);
BEGIN
utlFileHande := UTL_FILE.FOPEN(my_dir, my_file, 'r');
utl_file.get_line(utlFileHande, vLine);
FOR i IN
(SELECT TRIM(regexp_substr(vLine,'[^;]+',1,level) ) AS q
FROM dual
CONNECT BY regexp_substr(vLine,'[^;]+',1,level) IS NOT NULL
ORDER BY level
)
LOOP
dbms_output.put_line(i.q);
END LOOP;
utl_file.fclose(utlFileHande);
EXCEPTION
WHEN No_Data_Found THEN
utl_file.fclose(utlFileHande);
null;
END;

Lua functions use "self" in source but no metamethod allows to use them

I've been digging into Lua's source code, both the C source from their website and the lua files from Lua on Windows. I found something odd that I can't find any information about, as to why they chose to do this.
There are some methods in the string library that allows OOP calling, by attaching the method to the string like this:
string.format(s, e1, e2, ...)
s:format(e1, e2, ...)
So I dug into the source code for the module table, and found that functions like table.remove(), also allows for the same thing.
Here's the source code from UnorderedArray.lua:
function add(self, value)
self[#self + 1] = value
end
function remove(self, index)
local size = #self
if index == size then
self[size] = nil
elseif (index > 0) and (index < size) then
self[index], self[size] = self[size], nil
end
end
Which indicate that the functions should support the colon method. Lo' and behold when I copy table into my new list, the methods carry over. Here's an example using table.insert as a method:
function copy(obj, seen) -- Recursive function to copy a table with tables
if type(obj) ~= 'table' then return obj end
if seen and seen[obj] then return seen[obj] end
local s = seen or {}
local res = setmetatable({}, getmetatable(obj))
s[obj] = res
for k, v in pairs(obj) do res[copy(k, s)] = copy(v, s) end
return res
end
function count(list) -- Count a list because #table doesn't work on keyindexed tables
local sum = 0; for i,v in pairs(list) do sum = sum + 1 end; print("Length: " .. sum)
end
function pts(s) print(tostring(s)) end -- Macro function
local list = {1, 2, 3}
pts(list.insert) --> nil
pts(table["insert"]) --> function: 0xA682A8
pts(list["insert"]) --> nil
list = copy(_G.table)
pts(table["insert"]) --> function: 0xA682A8
pts(list["insert"]) --> function: 0xA682A8
count(list) --> Length: 9
list:insert(-1, "test")
count(list) --> Length: 10
Was Lua 5.1 and newer supposed to support table methods like the string library but they decided to not implement the meta method?
EDIT:
I'll explain it a little further so people understand.
Strings have metamethods attached that you can use on the strings OOP style.
s = "test"
s:sub(1,1)
But tables doesn't. Even though the methods in the table's source code allow for it using "self" functions. So the following code doesn't work:
t = {1,2,3}
t:remove(#t)
The function has a self member defined in the argument (UnorderedArray.lua:25: function remove(self,index)).
You can find the metamethods of strings by using:
for i,v in pairs(getmetatable('').__index) do
print(i, tostring(v))
end
which prints the list of all methods available for strings:
sub function: 0xB4ABC8
upper function: 0xB4AB08
len function: 0xB4A110
gfind function: 0xB4A410
rep function: 0xB4AD88
find function: 0xB4A370
match function: 0xB4AE08
char function: 0xB4A430
dump function: 0xB4A310
gmatch function: 0xB4A410
reverse function: 0xB4AE48
byte function: 0xB4A170
format function: 0xB4A0F0
gsub function: 0xB4A130
lower function: 0xB4AC28
If you attach the module/library table to a table like Oka showed in the example, you can use the methods that table has just the same way the string metamethods work.
The question is: Why would Lua developers allow metamethods of strings by default but tables doesn't even though table's library and it's methods allow it in the source code?
The question was answered: It would allow a developer of a module or program to alter the metatables of all tables in the program, leading to the result where a table would behave differently from vanilla Lua when used in a program. It's different if you implement a class of a data type (say: vectors) and change the metamethods of that specific class and table, instead of changing all of Lua's standard table metamethods. This also slightly overlaps with operator overloading.
If I'm understanding your question correctly, you're asking why it is not possible to do the following:
local tab = {}
tab:insert('value')
Having tables spawn with a default metatable and __index breaks some assumptions that one would have about tables.
Mainly, empty tables should be empty. If tables were to spawn with an __index metamethod lookup for the insert, sort, etc., methods, it would break the assumption that an empty table should not respond to any members.
This becomes an issue if you're using a table as a cache or memo, and you need to check if the 'insert', or 'sort' strings exist or not (think arbitrary user input). You'd need to use rawget to solve a problem that didn't need to be there in the first place.
Empty tables should also be orphans. Meaning that they should have no relations without the programmer explicitly giving them relations. Tables are the only complex data structure available in Lua, and are the foundation for a lot of programs. They need to be free and flexible. Pairing them with the the table table as a default metatable creates some inconsistencies. For example, not all tables can make use of the generic sort function - a weird cruft for dictionary-like tables.
Additionally, consider that you're utilizing a library, and that library's author has told you that a certain function returns a densely packed table (i.e., an array), so you figure that you can call :sort(...) on the returned table. What if the library author has changed the metatable of that return table? Now your code no longer works, and any generic functions built on top of a _:sort(...) paradigm can't accept these tables.
Basically put, strings and tables are two very different beasts. Strings are immutable, static, and their contents are predictable. Tables are mutable, transient, and very unpredictable.
It's much, much easier to add this in when you need it, instead of baking it into the language. A very simple function:
local meta = { __index = table }
_G.T = function (tab)
if tab ~= nil then
local tab_t = type(tab)
if tab_t ~= 'table' then
error(("`table' expected, got: `%s'"):format(tab_t), 0)
end
end
return setmetatable(tab or {}, meta)
end
Now any time you want a table that responds to functions found in the table table, just prefix it with a T.
local foo = T {}
foo:insert('bar')
print(#foo) --> 1

TypeError during executemany() INSERT statement using a list of strings

I am trying to just do a basic INSERT operation to a PostgreSQL database through Python via the Psycopg2 module. I have read a great many of the questions already posted regarding this subject as well as the documentation but I seem to have done something uniquely wrong and none of the fixes seem to work for my code.
#API CALL + JSON decoding here
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers
)
The loop creates a list of strings that looks like this when printed:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
I am just trying to have the code INSERT these strings as 1 row each into the table.
The error specified when running is:
TypeError: not all arguments converted during string formatting
I tried changing the INSERT to:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", (activeUsers,) )
But that seems like it's merely treating the entire list as a single string as it yields:
psycopg2.DataError: value too long for type character varying(30)
What am I missing?
First in the code you pasted:
x = 0
for item in ulist:
idValue = list['members'][x]['name']
activeUsers.append(str(idValue))
x += 1
Is not the right way to accomplish what you are trying to do.
first list is a reserved word in python and you shouldn't use it as a variable name. I am assuming you meant ulist.
if you really need access to the index of an item in python you can use enumerate:
for x, item in enumerate(ulist):
but, the best way to do what you are trying to do is something like
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append(str(item['name']))
Your first try was:
['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo']
Your second attempt was:
(['b2ong', 'dune', 'drble', 'drars', 'feman', 'got', 'urbo'], )
What I think you want is:
[['b2ong'], ['dune'], ['drble'], ['drars'], ['feman'], ['got'], ['urbo']]
You could get this many ways:
dbShell.executemany("INSERT INTO slackusers (username) VALUES (%s)", [ [a] for a in activeUsers] )
or event better:
for item in ulist: # or list['members'] Your example is kinda broken here
activeUsers.append([str(item['name'])])
dbShell.executemany("""INSERT INTO slickusers (username) VALUES (%s)""", activeUsers)

ODBC error "String data, right truncation State code: 22001" with SQL Server database

I have a test table using a Microsoft SQL Server that is defined like this:
CREATE TABLE [dbo].[Table] (
[FirstName] NVARCHAR (255) NULL,
[LastName] NVARCHAR (255) NULL
);
There's just one row in the table with the values "person" and "man", respectively.
I'm trying to add a function that will update the values of that row but I keep running into this "[Microsoft][ODBC SQL Server Driver]String data, right truncation State code: 22001" error and I cannot figure out what the problem is. I've looked around and people say that it is caused by the data being too long to fit in the column but that's impossible because the string I'm trying to update with is only two characters, and as you can see in the table definition there is plenty of space for it.
I'm using a prepared statement for optimization purposes and the code creating it looks something like this:
const tString query("UPDATE \"" + tableName + "\" SET " + setClause + " WHERE " + whereClause + ";");
SQLHSTMT statement;
SQLAllocHandle(SQL_HANDLE_STMT, fSQLConnection, &statement);
SQLPrepareW(statement, (SQLWCHAR *) query.mWideStr(), SQL_NTS);`
The query string looks like this:
UPDATE "Table" SET "FirstName" = ?, "LastName" = ? WHERE "FirstName" = ? AND "LastName" = ?;
And then I am binding the parameters like this:
// We have our own string class that we use, which is where the mWideStr() and mGetStrSize()
// come from. mWideStr() returns a pointer to a UCS-2 buffer and mGetStrSize() returns the
// size in bytes.
SQLLEN pcbValue(SQL_NTS);
SQLUSMALLINT paramIndex(1);
// Call this for each parameter in the correct order they need to be bound, incrementing the
// index each time.
SQLBindParameter(statement, paramIndex++, SQL_PARAM_INPUT, SQL_C_WCHAR, SQL_VARCHAR, 255, 0, (SQLPOINTER) paramValue.mWideStr(), paramValue.mGetStrSize(), &pcbValue);
The first and second bound parameters are the new values which are both just "55", then third would be "person" and fourth would be "man".
Then to execute the statements it's just a call to SQLExecute():
SQLExecute(statement);
The call to SQLExecute() fails and then the error is generated and there is some more code that outputs the error message. As far as I can tell this should all be working perfectly fine. I have another database using Oracle that uses the exact same setup and code and it works without any issues, it's just SQL Server that's barfing for some reason. Is there something obviously wrong here that I'm missing? Does SQL Server have some weird rules that I need to add somewhere?
The SQLLEN pcbValue(SQL_NTS); variable being passed to SQLBindParameter() was going out of scope between binding the parameters and executing the statement, which means that some garbage data was being pointed to in the parameter binding. I also realized that you don't need to specify the last parameter. You can just pass NULL and it will act as if it is a nul-terminated string.
So the fix was to remove the SQLLEN pcbValue(SQL_NTS); variable and to just pass NULL to SQLBindParameter() for the last parameter instead.
Stupid mistake, but worth noting I suppose.

SQL Server function to eliminate replicated characters

I wonder if there is an easy and efficient way in SQL Server 2005 to eliminate replicated characters in a string. Like converting
'ABBBCDEEFFFFG' to 'ABCDEFG'
It really sucks that SQL Server has such a poor string library and no ready-to-use regexp feature...
You can use the CLR functionality built into SQL Server 2005/2008 to get this done by .NET code.
MSDN magazine wrote about it in their February 2007 issue.
If this is not an acceptable solution, here is a UDF that will do the same, mind you this is about two orders of magnitude slower than the CLR solution.
YMMV. This appears to work for your string above. But not ABBBCDEEBBBBG
DECLARE #Numbers TABLE (Num smallint NOT NULL PRIMARY KEY)
INSERT #Numbers (Num)
SELECT TOP 8000
ROW_NUMBER() OVER (ORDER BY c1.NAME)
FROM
sys.columns c1
DECLARE #STuff TABLE (Seq varchar(100) NOT NULL PRIMARY KEY)
INSERT #STuff (Seq) VALUES ('ABBBCDEEFFFFG') --works
SELECT
Single
FROM
(
SELECT DISTINCT
CAST(Single AS varchar(100))
FROM
#Numbers N
CROSS APPLY
(SELECT Seq, SUBSTRING(Seq, Num, 1) AS Single FROM #Stuff) S
WHERE
Num <= LEN(Seq)
FOR XML PATH ('')
) foo(Single)
I know about the CLR solution, but as I said, I am neither responsible nor authorized to implement it in the DB of question.
For this particular problem, I decided to write a very simple and kinda silly loop. I am afraid it won't be fast enough for millions of records, but anyways... I wish I could do this stuff in the application layer but I am bound to T-SQL here..
DECLARE #i int ; -- counter
DECLARE #input varchar(200) ;
SET #input = 'AAABCDEEFFBBBXYZSSSWWWNT'
IF LEN(#input) > 1
BEGIN
DECLARE #unduplicated varchar(200) ;
SET #unduplicated = SUBSTRING(#input,1,1) ;
SET #i = 2 ;
WHILE #i <= LEN(#input)
BEGIN
-- If current char is different from the last char, concatenate, else not
IF SUBSTRING(#unduplicated, LEN(#unduplicated), 1) <> SUBSTRING(#input, #i, 1)
SET #unduplicated = #unduplicated + SUBSTRING(#input, #i, 1) ;
SET #i = #i + 1;
END
END
SELECT #unduplicated AS unduplicated;
Result:
unduplicated
ABCDEFBXYZSWNT