I have never used Informatica PowerCenter before and just don't know where to begin. To summarize my goal, I need to run a simple count query against a Teradata database using Informatica PowerCenter. This query needs to be ran on a specific day, but doesn't require me to store or manipulate the data returned. Looking at Informatica PowerCenter Designer is a bit daunting to me as I'm not sure what to be looking for.
Any help is greatly appreciated in understanding how to setup (if needed):
Sources
Targets
Transformations
Mappings
Is a transformation the only way to query data using PowerCenter? I've looked at a lot of tutorials, but most seem to be oriented to familiar users.
You can run a query against a database using informatica, only if you create a mapping, session and workflow to run that. But you cannot see the result unless you store it somewhere, either in a flatfile or a table.
Here are the steps to create it anyway.
Import your source table in source analyzer from Teradata.
Create a flat file target or import a relational target in target analyzer
Create a mapping m_xyz, drag and drop your source into the mapping.
You will see your source and source qualifier in the mapping. Write your custom query in source qualifier, say select count(*) as cnt from table
Remove all the ports from SQ except one numeric port from source to SQ and name it as cnt, count from your select will be assigned to this port.
Now drag and drop this port to an expression transformation.
Drag and drop your target into the mapping
Propagate the column from expression to this flat file/relational target.
Create a workflow and a session for this mapping.
In workflow you can schedule it to run on specific date.
When you execute this, count will be loaded into the column of that flat file or table.
Related
I have a task to create backup\copies of certain select tables from various datasets from one project into another (or) within the same project. A model query is listed below. There are about 300 odd tables in total.
CREATE OR REPLACE TABLE $target_project.$target_dataset.$table_name_$suffix AS
SELECT * FROM $source_project.$source_dataset.$table_name
In order to accomplish this, I have created a config table with the dataset name and table name. I have two approaches in mind -
Option 1 -
Create SQL script which loops through all the records from the config table, generates dynamic sql and executes them. Am unable to find a proper way to loop through table records, getting values in variables. The struct command only takes one query at a time.
Option 2 -
Create a sql file containing all the CREATE TABLE SCRIPT statements with placeholders for the project name and suffix names.
Use a DAG, pass the variables into the sql file and execute the script using Cloud Composer.
Option 2 is working and feasible. It is just not scalable owing the fact that any changes to be made would require one to modify the script again, re-upload etc.
Can someone help me with Option 1 (or) advise if there is any better way to accomplish this task?
We work with the gcp suite of products and am happy to engage with various tools to accomplish this.
I am new in informatica cloud. I have list of queries ready in my table. Like below.
Now I want to take one by one query from this table which work as a source query and whatever results return which I need to load into target. All tables were already created in source and target.
I just need to copy the data based on dynamic queries which kept in my one of sql tables.
If anyone have any idea then please share your toughs with me. It great helps to me.
The source connection will be the connector to your source database and the Source Type will be query. From there it depends how you are managing your variables. See thread on Informatica Network for links to multiple examples.
Read the table like normally you would do in the cloud. Then pass each of the record into the sql transformation for execution. configure where the sql transformation has to execute and it will run the queries in the database you want.
you can use a SQL task to run dynamic SQL queries.
link to using SQL task approach: https://www.datastackpros.com/2019/12/informatica-cloud-incremental-load_14.html
I am facing some challenge to create multiple PDF files in informatica 10.2.0, Please find the details below:
Requirement : - We need to spilt single xml file into multiple PDF files based on condition.
Tools used : - Informatica Powercenter, Informatica Developer.
Challenge : - I have created data processor in informatica developer and used this as service in informatica powercenter and created single PDF file
But not able to create multiple PDF files with this service. I have used sorter to sort the file based on my condition to split the file and then used transaction control to commit records based on the condition used and passing these records to the UDT transformation (calling service in this transformation) and then passing the output Buffer port to target.
Here in UDT transformation I have given Input type and Output type as 'file' in UDT settings tab in mapping.
Could anyone please provide suggestion to achieve a solution for this technical challenge.
create a variable to identify when a change or new xml is processed and use this to issue commit/continue in the Transaction control.
I think the transaction control is probably the wrong route to use here - try this https://kb.informatica.com/howto/6/Pages/1/154788.aspx
I want to create an audit, which can be re-used across multiple mappings to capture Source record count and target record count in when source database is oracle and target database is sql server
We are using it from source to staging mappings
It's all there, in Metadata tables. There's no need to add anything that will make your loads longer and more complex.
You can review this Framework for some ideas.
When there is any change in DDL of any table, we have to again import source and target definition and change mapping. Is there a way to dynamically fetch the DDL of the table and do the data copy using Informatica mapping.
The ETL uses an abstractive layer, separated from any physical database. It uses Source and Target definition that indicate what should be expected to find in DB to which the job will be connecting. Keep in mind that the same data mapping can be applied to many different source and / or target systems. It's not bound to any of them, it just defines what data to fetch and what to do with them.
In Informatica this is reflected by separating Mappings, that define data flow, and Sessions, which indicate where the logic should be applied.
Imagine you're transferring data from multiple servers. A change applied on one of them should not break the whole data integration. If the changes would be dynamically reflected, then a column added on one server would make it impossible to read data from the others.
Of course if perfectly fine to have such requirement as you've mentioned. It's just not something Informatica supports with their approach.
The only way workaround is to create your own application that would fetch table definitions, generate the Workflows and import them into Informatica prior to execution.