Django Q object is adding extra - IS NOT NULL - django

for the below raw query, I want to build Django ORM query
Select * from employee
WHERE (address_id is not null OR address_id<>'*')
AND (xyz_id is null OR xyz_id='*')
AND (abc_id is null OR abc_id='*')
So I have build ORM query as below
data = Employee.objects.filter(~Q(address_id="23") | Q(address_id__isnull=False),
Q(xyz_id__isnull=False) | Q(xyz_id='*'),
Q(abc_id__isnull=True) | Q(abc_id=‘*’))
and when i print the query, it is showing as below
SELECT "employee"."id", "employee"."xyz_id", "employee"."abc_id" FROM
"employee" WHERE ((NOT ("employee"."address_id" = 23 AND
"employee"."address_id" IS NOT NULL) OR "employee"."address_id" IS
NOT NULL) AND ("employee"."xyz_id" IS NOT NULL OR "employee"."xyz_id"
= *) AND ("employee"."abc_id" IS NULL OR "employee"."abc_id" = *))
my question is why Q object in Django is adding bolded line in above generated query ? which is not solving for above raw sql criteria.
i am using Django 2.2 version
Kindly advise

If you write Q(foo=bar), then an implicit constraint is that foo is not NULL, if you negate that however, it will write NOT (foo = bar) but if foo is NULL, then NOT (foo = bar) is NOT (NULL = bar) which is NULL, so this will be filtered out of the queryset. which would thus result in FALSE whereas for NULL, one would expect that this is TRUE.
To thus ensure that ~Q(…) is the full opposite of Q(…), it thus should add foo IS NOT NULL to the condition.
In this specific case however the query optimizer could probably indeed figure out that it is not necessary due to the address_id IS NOT NULL later in the query, but here apparently the optimizer does not eliminate that clause.

Related

Power Query M - We cannot convert the value null to type Logical

In Power BI I have an M Query that tests for the value in a column equaling or not equaling to null.
When I add the statement for [Sale.Revenue] <> null I get an error however it works fine for the [UserRole.Name] = null it works fine. Tested just by removing the statement and adding it back.
We cannot convert the value null to type Logical.
This seems like it should work but just can't figure it out.
add_user_role_group = Table.AddColumn(
join_expand_sale,
"UserRole.Group1",
each (
if [UserRole.Name] = null and
[Sale.Revenue] <> null then
"Group1"
else if Text.Contains([UserRole.Name], "Manager") then
"Group2"
else
"Undefined"
)
)
I am sure it is something glaringly obvious :/ Thanks for your thoughts on this.
One of your rows has a null value for both UserRole.Name and Sale.Revenue. You need to check for that explicitly, and then add it to the "Undefined" group.
What happened is that the first condition fails because Sale.Revenue is null. The second condition calls Text.Contains, which returns null when [UserRole.Name] is null (Text.Contains returns a nullable logical value). null is not true or false, so you get the error.
After a such journey, finaly I found Text.Length !!
You can solve your problem like this:
if Text.Length([UserRole.Name]) = 0 and
Text.Length([Sale.Revenue]) > 0 then
I hope I have helped you.
Reference: Power Query M - Text.Length
Your issue is in the Text.Contains formula. You create an if statement that expects an expression that returns either true or false.
When the Text.Contains formula contains a null value, it returns 'null' as answer, and not true or false. You can adjust your code:
Text.Contains([UserRole.Name], "Manager")
To
Text.Contains([UserRole.Name]??"", "Manager")
The ?? is the COALESCE operator. In case it finds a null value, it now treats it as "". Instead of returning null it now returns true or false.
More on text functions in this article: https://gorilla.bi/power-query/text-functions/
Enjoy Power Query,
Rick

How to get associative arrays notation to assign simple values

I created sample code using the Microsoft SQL Server Northwind demo database. If you don't have access to this demo database here is a simple (MS-SQL) script to create the table and a row of data for this question.
CREATE TABLE [dbo].[Products](
[ProductID] [int] IDENTITY(1,1) NOT NULL,
[ProductName] [nvarchar](40) NOT NULL,
[SupplierID] [int] NULL,
[CategoryID] [int] NULL,
[QuantityPerUnit] [nvarchar](20) NULL,
[UnitPrice] [money] NULL CONSTRAINT [DF_Products_UnitPrice] DEFAULT (0),
[UnitsInStock] [smallint] NULL CONSTRAINT [DF_Products_UnitsInStock] DEFAULT (0),
[UnitsOnOrder] [smallint] NULL CONSTRAINT [DF_Products_UnitsOnOrder] DEFAULT (0),
[ReorderLevel] [smallint] NULL CONSTRAINT [DF_Products_ReorderLevel] DEFAULT (0),
[Discontinued] [bit] NOT NULL CONSTRAINT [DF_Products_Discontinued] DEFAULT (0),
CONSTRAINT [PK_Products] PRIMARY KEY CLUSTERED
(
[ProductID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 90) ON [PRIMARY]
) ON [PRIMARY]
GO
SET IDENTITY_INSERT [dbo].[Products] ON
GO
INSERT [dbo].[Products] ([ProductID], [ProductName], [SupplierID], [CategoryID], [QuantityPerUnit], [UnitPrice], [UnitsInStock], [UnitsOnOrder], [ReorderLevel], [Discontinued]) VALUES (1, N'Chai', 1, 1, N'10 boxes x 20 bags', 18.0000, 39, 0, 10, 0)
GO
SET IDENTITY_INSERT [dbo].[Products] OFF
GO
Here is the ColdFusion code:
<cfset variables.useTempVar = false>
<cfquery datasource="Northwind2014" name="qryNWProducts">
SELECT TOP 1 * from Products;
</cfquery>
<cfdump var="#qryNWProducts#" label="qryNWProducts">
<cfset variables['stProduct'] = {}>
<cfloop index="vcColName" list="#qryNWProducts.columnlist#">
<cfif variables.useTempVar>
<cfset variables['temp'] = qryNWProducts[vcColName]>
<cfset variables['stProduct'][vcColName] = variables.temp>
<cfelse>
<cfset variables['stProduct'][vcColName] = qryNWProducts[vcColName]>
</cfif>
</cfloop>
<cfdump var="#variables['stProduct']#" label="variables['stProduct']">
<cfloop collection="#variables['stProduct']#" item="key"><cfoutput>
variables['stProduct']['#key#'] JVM datatype = #getMetadata(variables['stProduct'][key]).getName()#<br>
</cfoutput></cfloop>
<br>
This always works:<br>
<cfset variables['aPhrase'] = "I ordered " & variables.stProduct.ProductName & " for " & DollarFormat(variables.stProduct.UnitPrice) & ".">
<cfoutput>#variables['aPhrase']#<br></cfoutput>
<br>
With "variables.useTempVar = false", the next line will throw a "Complex object types cannot be converted to simple values. " error.<br>
<cfset variables['aPhrase'] = "I ordered " & variables['stProduct']['ProductName'] & " for " & DollarFormat(variables['stProduct']['UnitPrice']) & ".">
<cfoutput>#variables['aPhrase']#<br></cfoutput>
The code above has a boolean variable named "variables.useTempVar" at the top that can be flipped to see the error that I'm getting.
It looks like the direct assignment (when variables.useTempVar = false) from the query to the structure causes the structure values to be of JVM type "coldfusion.sql.QueryColumn".
Another note: if this line of code:
<cfset variables['stProduct'][vcColName] = variables.temp>
is changed to:
<cfset variables['stProduct'][vcColName] = variables['temp']>
The JVM datatype will be "coldfusion.sql.QueryColumn".
When the dot notation temp variable is used to assign the query field (when variables.useTempVar = true); the JVM datatypes are simple types that matches up pretty well with the database column types (java.lang.Integer, java.math.BigDecimal, java.lang.String, etc.).
I've also experiemented with statements like this and that provided some odd results:
<cfset variables['stProduct'][vcColName] = qryNWProducts[vcColName].toString()>
Here's the question. Is this the best way to transfer the simple values from a query to a structure? It seems odd to be forced to use a temp variable and dot notation to make this work.
Comment: I've always thought that dot notation and associative array notation were equivalent. This code example appears to contradict that opinion.
#Leigh is correct in that you need to supply the row number when using associative array notation with a query object. So you'd reference row 1 like: qryNWProducts[vcColName][1]
As for your question
Is this the best way to transfer the simple values from a query to a structure?
Are you sure you need a struct? Your question doesn't really specify the use case, so it is entirely possible that you would be better off using the query object as-is.
If you do need it to be a struct (and since you are using ColdFusion 11) might I suggest you take a look at serializeJSON/deSerializeJSON to convert this to a struct. The serializeJSON has a new attribute that will properly serialize a query object into an "AJAX friendly" JSON array of structs. You can then deSerialize the JSON into a CF array, like so:
NWProducts = deSerializeJSON( serializeJSON( qryNWProducts, 'struct' ) )[1];
Which would return a struct representation of the first row in that query object.
Although it's not obvious from the Adobe docs for serializeJSON, the second parameter can be one of: true|false|struct|row|column which will change how the resulting data is formatted.
Here's a runnable example of using the above technique showcasing each serializeQueryAs option.
It's also a better practice to start moving that kind of code into cfscript. queryExecute is quite easy to use and makes script based queries very easy to develop. See the How To Create a Query in cfscript tutorial at trycf.com for more on how to develop script based queries.
Final note, and this is a bit off topic but it is a generally accepted best practice to not use Hungarian Notation when naming variables.
#Abram's covered the mains answer, but just to pick up one tangential point you raise.
Dot notation and associative array notation are generally equivalent in CFML. However in the case of queries, there is a slight variation. Dot notation: query.columnName is treated as shorthand for query.columnName[currentRow] (where currentRow defaults to 1).
Associative array notation with queries does not have this "syntactic sugar", so query["columnName"] refers to the entire column, as the syntax actually indicates.
There are no functions I am aware of in CFML that take a query column as an argument, however the CFML engine will convert the column to an array if it's used in an array function. This is quite handy sometimes.

CTE with HierarchyID suddenly causes parse error

So I have this self-referencing table in my database named Nodes, used for storing the tree structure of an organization:
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](max) NULL,
[ParentId] [int] NULL,
(+ other metadata columns)
And from it I'm using HIERARCHYID to manage queries based on access levels and such. I wrote a table-valued function for this, tvf_OrgNodes, a long time ago, tested and working on SQL Server 2008 through 2014 and it's remained unchanged since then since it's been doing great. Now, however, something has changed because the parsing of HIERARCHYIDs from path nvarchars ("/2/10/8/") results in the following error, matching only 4 hits (!) on Google:
Msg 6522, Level 16, State 2, Line 26
A .NET Framework error occurred during execution of user-defined routine or aggregate "hierarchyid":
Microsoft.SqlServer.Types.HierarchyIdException: 24000: SqlHierarchyId operation failed because HierarchyId object was constructed from an invalid binary string.
When altering the function to only return NVARCHAR instead of actual HIERARCHYID's, the paths all look fine, beginning with / for the root, followed by /2/ etc etc. Simply selecting HIERARCHYID::Parse('path') also works fine. I actually got the function working by leaving the paths as strings all the way until the INSERT into the function result, parsing the paths there. But alas, I get the same error when I then try and insert the reusulting data into a table of same schema.
So the question is, Is this a bug, or does anybody know of any (new?) pitfalls in working with HIERARCHYIDs<->Path strings that could cause this? I don't get where the whole binary string idea comes from.
This is the code of the TVF:
CREATE FUNCTION [dbo].[tvf_OrgNodes] ()
RETURNS #OrgNodes TABLE (
OrgNode HIERARCHYID,
NodeId INT,
OrgLevel INT,
ParentNodeId INT
) AS
BEGIN
WITH orgTree(OrgNode, NodeId, OrgLevel, ParentNodeId) AS (
-- Anchor expression = root node
SELECT
CAST(HIERARCHYID::GetRoot() AS varchar(180))
, n.Id
, 0
, NULL
FROM Nodes n
WHERE ParentId IS NULL -- Top level
UNION ALL
-- Recursive expression = organization tree
SELECT
CAST(orgTree.OrgNode + CAST(n.Id AS VARCHAR(180)) + N'/' AS VARCHAR(180))
, n.Id
, orgTree.OrgLevel + 1
, n.ParentId
FROM Nodes AS n
JOIN orgTree
ON n.ParentId = orgTree.NodeId
)
INSERT INTO #OrgNodes
SELECT
HIERARCHYID::Parse(OrgNode),
NodeId,
OrgLevel,
ParentNodeId
FROM orgTree;
RETURN;
END
I might have recently installed .NET 4.53 aka 4.6 for the lolz. Can't find much proof of it anywhere except in the reg, though: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\v4.0.30319\SKUs.NETFramework,Version=v4.5.3

Getting error: Prelude.(!!): index too large

I'm getting this 'program: Prelude.(!!): index too large' error for the following code:
select :: Field -> Field -> Table -> Table
select column_name column_value (header:t) = header:filterT t
where filterT = filter testR
field_idx = (elemIndices column_name header)!!0
testR r | r!!field_idx == column_value = True
testR r | otherwise = False
I suppose the error is regarding the following part of the code:
field_idx = (elemIndices column_name header)!!0
testR r | r!!field_idx == column_value = True
Does anyone know why it's giving me this error or how I could fix it?
I'm not sure what you're doing, but I hope you're aware, that !! is not a safe operation. an element with the index doesn't necessarily exist.
So you could get this error, if, for example, the header doesn't contain column_name.
Again, not sure what you exactly want to do, but if there's a chance there's no result, perhaps you want to wrap the result with Maybe?

How to make =NULL work in SQLite?

Given the following table:
Table: Comedians
=================
Id First Middle Last
--- ------- -------- -------
1 Bob NULL Sagat
2 Jerry Kal Seinfeld
I want to make the following prepared query:
SELECT * FROM Comedians WHERE Middle=?
work for all cases. It currently does not work for the case where I pass NULL via sqlite3_bind_null. I realize that the query to actually search for NULL values uses IS NULL, but that would mean that I cannot use the prepared query for all cases. I would actually have to change the query depending on the input, which largely defeats the purpose of the prepared query. How do I do this? Thanks!
You can use the IS operator instead of =.
SELECT * FROM Comedians WHERE Middle IS ?
Nothing matches = NULL. The only way to check that is with IS NULL.
You can do a variety of things, but the straight forward one is...
WHERE
middle = ?
OR (middle IS NULL and ? IS NULL)
If there is a value you know NEVER appears, you can change that to...
WHERE
COALESCE(middle, '-') = COALESCE(?, '-')
But you need a value that literally NEVER appears. Also, it obfuscates the use of indexes, but the OR version can often suck as well (I don't know how well SQLite treats it).
All things equal, I recommend the first version.
NULL is not a value, but an attribute of a field. Instead use
SELECT * FROM Comedians WHERE Middle IS NULL
If you want match everything on NULL
SELECT * FROM Comedians WHERE Middle=IfNull(?, Middle)
if want match none on NULL
SELECT * FROM Comedians WHERE Middle=IfNull(?, 'DUMMY'+Middle)
See this answer: https://stackoverflow.com/a/799406/30225