I have a lookup table with 2 columns, and both are the key columns. The logic in the mapping transformation is to see if a table has both keys exist in this lookup table.
So, is there a lookup function that doesn't need to return anything. Similar to the EXIST in PL/SQL, or the lookup_match function in abinitio.
Use a joiner - you can use both columns as keys, with nothing extra. Or does it have to be lookup for some reason?
Just build the SQL for the lookup so that it returns a dummy value like 'A' and in a subsequent Expression transformation, have an output port to check if the value is 'A' or null in order to determine if you found a match.
Your Lookup SQL Override might look like:
SELECT
COLUMN_A as COLUMN_A,
COLUMN_B as COLUMN_B,
'A' as MATCH_FLAG
FROM TABLE_NAME
With COLUMN_A and COLUMN_B being your lookup conditions and MATCH_FLAG being the returned column from the transformation.
Then your Expression output port might look like:
IIF(ISNULL(MATCH_FLAG),FALSE,TRUE)
Related
I am trying to run some dynamoDB operations with AWS.DynamoDB.DocumentClient from aws-sdk module but I am unable to find an easy solution to select items where an attribute is not equals to an array of values.
e.g
attribute <> ["value1", "value2]
This is equivalent to a simple typical SQL operation in the form of:
select * from sometable where attribute not in("value1", "value2"...);
After trying out different ScanFilter and QueryFilter following the documentation here, it seems that the AttributeValueList for NE and NOT_CONTAINS does not accept multiple values.
I wish to arrive at the results as shown below without having to define multiple 'AND' queries
I have since arrived at this solution but it seems clumsy and I would have to write logic to create the filter condition string and ExpressionAttributeValues as the filter condition is dynamic.
FilterExpression: 'answer <> :answer1 AND answer <> :answer2',
ExpressionAttributeValues : {
':answer1' : "test1",
':answer2' : "test2"
}
I have therefore, 2 questions:
Is there a better way of doing this?
Is there a length limit to the string of KeyConditionExpression? I
am very sure there is but I cannot seem to find information with
regards to this.
There is no other way to achieve what you need. If all these values have something in common, and you know it in the write time, you can insert them with some kind of prefix and create GSI where they will be a sort key. In this case you'll be able to query them by prefix in the key condition expression. Otherwise, what you've suggested is your only option.
4KB for all of the expressions combined. As described in the Expression Parameters:
Expression parameters include ProjectionExpression, ConditionExpression, UpdateExpression, and FilterExpression.
The maximum length of any expression string is 4 KB. For example, the size of the ConditionExpression a=b is 3 bytes.
I'm using the histogram() function https://prestodb.github.io/docs/current/functions/aggregate.html
It "Returns a map containing the count of the number of times each input value occurs."
The result may look something like this:
{ORANGES=1, APPLES=165, BANANAS=1}
Is there a function that will return APPLES given the above input?
XY Problem?
The astute reader may notice the end-result of histogram() combined with what I'm trying to do, would be equivalent to the mythical Mode Function, which exists in textbooks but not in real-world database engines.
Here's my complete query at this point. I'm looking for the most frequently occurring value of upper(cmplx) for each upper(address),zip tuple:
select * from (select upper(address) as address, zip,
(SELECT max_by(key, value)
FROM unnest(histogram(upper(cmplx))) as t(key, value)),
count(*) as N
from apartments
group by upper(address), zip) t1
where N > 3
order by N desc;
And the error...
SYNTAX_ERROR: line 2:55: Constant expression cannot contain column
references
Here's what I use to get the key that corresponds to the max value from an arbitrary map:
MAP_KEYS(mapname)[
ARRAY_POSITION(
MAP_VALUES(mapname),
ARRAY_MAX(MAP_VALUES(mapname))
)
]
substitute your histogram map for 'mapname'.
Not sure how this solution compares computationally to the other answer, but I do find it easier to read.
You can convert the map you got from histogram to an array with map_entries. Then you can UNNEST that array to a relation and you can call max_by. Please see the below example:
SELECT max_by(key, value) FROM (
SELECT map_entries(histogram(clerk)) as entries from tpch.tiny.orders
)
CROSS JOIN UNNEST (entries) t(key, value);
EDIT:
As noted by #Alex R, you can also pass histogram results dirrectly to UNNEST:
SELECT max_by(key, value) FROM (
SELECT histogram(clerk) as histogram from tpch.tiny.orders )
CROSS JOIN UNNEST (histogram) t(key, value);
In your question the query part (SELECT max_by(key, value) FROM unnest(histogram(upper(cmplx)) is a correlated subquery which is not yet supported. However the error you are seeing is misleading. IIRC Athena is using Presto 0.172, and this error reporting was fixed in 0.183 (see https://docs.starburstdata.com/latest/release/release-0.183.html - that was in July 2017, btw map_entries was also added in 0.183)
i am new in informatica software. now i have two tables, say AAA and BBB table.
AAA: last_post_date
BBB: Trx_No, Field1, Field2, trx_date
I want to move BBB table to the target table which the trx_date must be greater than last_post_date. I cannot use joiner transformation as it doesnt have >, < , >= and <= operators. If I want to use lookup transformation, how to use it for this case or any other way can help me do this. I searched many websites about the lookup transformation , still don't know how to use it.
Please help.
Thanks!
I am assuming that AAA has only 1 row which contains last_post_date. If both the tables are in same database, you can use Source Qualifier override
select Trx_No, Field1, Field2, trx_date from BBB where trx_date > last_post_date
But if both tables are in different database and / or you are not able to create DB link between them, then use below solution.
After Source Qualifier for both source, use Expression transformation.
Add an output port in both Expression Transformations, say o_Dummy and hardcode the value as 1 (for both transformations)
Use Joiner and use normal join. Join condition would be o_Dummy = o_Dummy1.
After it use a filer to filter records where trx_date > last_post_date.
This would be your flow.
SQ_AAA -> Expression -> Joiner -> Filter -> Target
SQ_BBB -> Expression -^
Use the Source Qualifier to read data from BBB, followed by lookup to AAA and a filter witha a condition trx_date>last_post_date.
Ideally you'd use an unconnected lookup reffered from Expression variable port e.g. v_LastPostDate = IIF(ISNULL(v_LastPostDate), LKP.LoopkupToAAA, v_LastPostDate) - this would ensure you perform the lookup only once. Not that it would matter a lot with a single value, but I thought I'll share some good practice :)
Is it possible to do a bulk insert with Sitecore Rocks? Something along the lines of SQL's
INSERT INTO TABLE1 SELECT COL1, COL2 FROM TABLE2
If so, what is the syntax? I'd like to add an item under any other item of a given template type.
I've tried using this syntax:
insert into (
##itemname,
##templateitem,
##path,
[etc.]
)
select
'Bulk-Add-Item',
//*[##id='{B2477E15-F54E-4DA1-B09D-825FF4D13F1D}'],
Path + '/Item',
[etc.]
To this, Query Analyzer responds:
"values" expected at position 440.
Please note that I have not found a working concatenation operator. For example,
Select ##item + '/value' from //sitecore/content/home/*
just returns '/value'. I've also tried ||, &&, and CONCATENATE without success.
There is apparently a way of doing bulk updates with CSV, but doing bulk updates directly from Sitecore Query Analyzer would be very useful
Currently you cannot do bulk inserts, but it is a really nice idea. I'll see what I can do.
Regarding the concatenation operator, this following works in the Query Analyzer:
select #Text + "/Value" from /sitecore/content/Home
This returns "Welcome to Sitecore/Value".
The ##item just returns empty, because it is not a valid system attribute.
I use HSQLDB2.0 and JPA2.0 for my current project and have few date columns in DB.
I would like to run wildcard queries on the date columns. How could I do that?
Ex : If my DB contains two rows with date values as : 10-01-2011 and 15-02-2011
and my search criteria will be "%10-01%", then result should be 10-01-2011.
Else if search criteria is "%2011%" then both rows need to be fetched with the select query.
Thanks in advance,
Satya
You can define an autogenerated column of type VARCHAR containing a copy of the date. You can then perform queries with both LIKE predicates and REGEXP_MATCHES() function. An example of the column definition is below:
DATEGEN VARCHAR(10) GENERATED ALWAYS AS (CAST(DATECOL AS VARCHAR(10))
Note the string representation of DATE is in the form '2011-02-26' and your query strings should follow this pattern.
This can be achieved in this format :
select date_birth from member where to_char(date_birth,'MM-yyyy') like '%02-2011%'
select date_birth from member where to_char(date_birth,'MM-dd') like '%02-15%'
select date_birth from member where to_char(date_birth,'dd-yyyy') like '%30-2011%'
Regards,
Satya