I've a table "City" with more than 100k records.
The field "name" contains strings like "Roma", "La Valletta".
I receive a file with the city name, all in upper case as in "ROMA".
I need to get the id of the record that contains "Roma" when I search for "ROMA".
In SQL, I must do something like:
select id from city where upper(name) = upper(%name%)
How can I do this in kettle?
Note: if the city is not found, I use an Insert/update field to create it, so I must avoid duplicates generated by case-sensitive names.
You can make use of the String Operations steps in Pentaho Kettle. Set the Lower/Upper option to Y
Pass the city (name) from the City table to the String operations steps which will do the Upper case of your data stream i.e. city name. Join/lookup with the received file and get the required id.
More on String Operations step in pentaho wiki.
You can use a 'Database join' step. Here you can write the sql:
select id from city where upper(name) = upper(?)
and specify the city field name from the text file as parameter. With 'Number of rows to return' and 'Outer join?' you can control the join behaviour.
This solution doesn't work well with a large number of rows, as it will execute one query per row. In those cases Rishu's solution is better.
This is how I did:
First "Modified JavaScript value" step for create a query:
var queryDest="select coalesce( (select id as idcity from city where upper(name) = upper('"+replace(mycity,"'","\'\'")+"') and upper(cap) = upper('"+mycap+"') ), 0) as idcitydest";
Then I use this string as a query in a Dynamic SQL row.
After that,
IF idcitydest == 0 then
insert new city;
else
use the found record
This system make a query for file's row but it use few memory cache
Related
I want to Assign 'Y' to the Duplicate Records and 'N' to the Unique Records, And Display those 'Y' and 'N'
Flags in 'Duplicate' Column.
Like Below
Source Table:
Name,Location
Vivek,India
Vivek,UK
Vivek,India
Vivek,USA
Vivek,Japan
Target Table:
=============
Name,Location,Duplicate
Vivek,India,Y
Vivek,India,Y
Vivek,Japan,N
Vivek,UK,N
Vivek,USA,N
How to Create a Mapping in Informatica Powercenter?
Which Logic I Should use?
[See the Image for More Clarification][1]
[1]: https://i.stack.imgur.com/2F20A.png
You need to calculate count grouping by key columns using aggregator. And then join back to original flow based on key columns.
use Sorter sort the data based on key columns like name and country in your example.
use Aggregator to calculate count() group by key columns.
out_count= count(*)
in_out - key_column
use Joiner to join aggregator data and sorter data based on key columns. Drag out_count and key columns from aggregator to joiner. Drag all columns from sorter. Do a inner join on key columns.
use Expression and create an out expression. Use out_count column to calculate your duplicate flag.
out_Duplicate = iif( out_count>1, 'Y','N')
Whole map should look like this
SRC -->SRT ---->AGG-->\
|------------->JNR-->EXP-->TGT
There's one more way to solve it without the Joiner, which is costly. I'm going to use the Name, Location sample columns from your example.
Use the Sorter on the Name and Location
Add an Expression with variable port for each key column called e.g. v_prev_Name and v_prev_Location.
Assign the expressions accordingly:
v_prev_Name = Name
v_prev_Location = Location
Next create another variable v_is_duplicate with following expression:
IIF(v_prev_Name = Name and v_prev_Location = Location, 1, 0)
Move v_is_duplicate up the list of ports so that it is before v_prev_Name and v_prev_Location - THIS IS IMPORTANT. The order needs to be:
v_is_duplicate
v_prev_Name
v_prev_Location
Add output port is_duplicate with expression simply matching v_is_duplicate.
I am trying to get today's current date and format it to yymmdd because my table name change daily. e.g. MICRINFO210616 and tomorrow it will be MICRINFO210617
When I run thecode below I get the following error:
Expression.Error: The key didn't match any rows in the table.
Key=
Schema=dbo
Item=MICRINFO210617
Table=[Table]
code:
let
Source = Sql.Database("TEST", "TEST"),
formattedDate = Date.ToText(DateTime.Date(DateTime.LocalNow()), "yyMMdd"),
combine = "MICRINFO" & formattedDate,
dbo_MICRINFO210616 = Source{[Schema="dbo", Item=combine]}[Data]
in
dbo_MICRINFO210616
Make sure the account you're using has at least read permissions (to the new table).
Check if the structure of both tables is the same (same number of columns, same datatype).
IR based on PL/SQL Function Body returning SQL Query.
How do i can create Interactive reports based on multiple table and Deferent column name.
Exp :-
Select list item return three value
1 or 2 or 3
And the function return query basen on select list value
when Value equal 1
Select name, satate, country_id from cities
when value equal 2 Return
Select country, id from country
when value equal 3 Return
Select ocean,oc_id,from oceans
The three query return different column name and value.
Ok firstly, your question is poorly written. But from what I gather, you want an SQL query that returns different things based on an input.
I dont think you even need a plsql function body for this.
Simply do something like this:
SELECT * FROM
(SELECT name as name,
state as state,
country_id as id,
1 as value
FROM cities
UNION ALL
SELECT country as name,
NULL as state,
id as id,
2 as value
FROM country
UNION ALL
SELECT ocean as name,
NULL as state,
oc_id as id,
3 as value
FROM oceans)
WHERE value = :input_parameter_value;
Because if you are trying to display a variable number of columns and constantly changing their names and such. You are gonna have a bad time, it can be done, as can everything. But afaik its not exactly simple
No objections to what #TineO has said in their answer, I'd probably do it that way.
Though, yet another option: if your Apex version allows it, you can create three Interactive Report regions on the same page, each selecting values from its own table, keeping its own column labels.
Then create a server condition for each region; its type would be Function that returns a Boolean and look like
return :P1_LIST_ITEM = 1;
for the 1st region; = 2 for the 2nd and = 3 for the 3rd.
When you run the page, nothing would be displayed as P1_LIST_ITEM has no value. Once you set it, one of conditions would be met and appropriate region would be displayed.
Without saving SHA1 digest string in table directly. Is it possible to format the column in select statement ?
For example (Hope you know what i mean):
#item = Item.where(Digest::SHA1.hexdigest id.to_s:'356a192b7913b04c54574d18c28d46e6395428ab')
No, not the way you want it. The hexdigest method you're using won't be available at the database level. You could use database-specific functions though.
For example:
Item.where("LOWER(name) = ?", entered_name.downcase)
The LOWER() function will be available to the database so it can pass the name column to it.
For your case, I can suggest two solutions:
Obviously, store the encrypted field in the table. And then match.
key = '356a192b7913b04c54574d18c28d46e6395428ab'
Item.where(encrypted_id: key)
Iterate over all column values (ID, in your case) and find the one that matches:
all_item_ids = Item.pluck("CAST(id AS TEXT)")
item_id = all_item_ids.find{ |val| Digest::SHA1.hexdigest(val) == key }
Then you could use Item.find(item_id) to get the item or Item.where(id: item_id) to get an ActiveRecord::Relation object.
I use HSQLDB2.0 and JPA2.0 for my current project and have few date columns in DB.
I would like to run wildcard queries on the date columns. How could I do that?
Ex : If my DB contains two rows with date values as : 10-01-2011 and 15-02-2011
and my search criteria will be "%10-01%", then result should be 10-01-2011.
Else if search criteria is "%2011%" then both rows need to be fetched with the select query.
Thanks in advance,
Satya
You can define an autogenerated column of type VARCHAR containing a copy of the date. You can then perform queries with both LIKE predicates and REGEXP_MATCHES() function. An example of the column definition is below:
DATEGEN VARCHAR(10) GENERATED ALWAYS AS (CAST(DATECOL AS VARCHAR(10))
Note the string representation of DATE is in the form '2011-02-26' and your query strings should follow this pattern.
This can be achieved in this format :
select date_birth from member where to_char(date_birth,'MM-yyyy') like '%02-2011%'
select date_birth from member where to_char(date_birth,'MM-dd') like '%02-15%'
select date_birth from member where to_char(date_birth,'dd-yyyy') like '%30-2011%'
Regards,
Satya