everyone,
I've a problem in Informatica PowerCenter.
In my mapping I have 5 objects:
1x Source Table
1x Source Qualifier
1x Expression Transformation
1x Update Strategy
1x Target Table
The source and target table have no primary key, how come Informatica PowerCenter expects a key?
I have tried changing the "Treat source rows as" property of my workflow session from "Insert" to "Data driven" and it is working.
You have an update strategy in your mapping. Which expects you must have some key defined on target. Infa fires query like
UPDATE tgt SET col =? WHERE KEY = ?
Question mark 1 is updated column and question mark 2 is the key.
You can set unique keys as primary key.
If you don't have a primary or unique keys in target, pls define all columns as keys except the updatable column.
Or, you can use target overwrite to write sql to update target, but here too, you have to set similar query like above.
Data driven should be set.
In Informatica, the ports marked as keys in Target Transformation indicate what should be used to build the Update statement in DB. It has nothing physically to do with real Primary Key defined in the database itself. Usually you use same columns as keys in Informatica and in DB - but this is not necessary. DB is unaware of what is set in Informatica and vice versa.
It's even perfectly valid to have same database table defined multiple times in Informatica and have different mappings that will update the data using different columns as keys.
Note however that if you use Update Strategy you have to define which columns to use as keys.
Related
I'm trying to set up a Postgresql migration using the DMS to s3 as target. But after running I noticided that some tables were missing some columns.
After checking the logs I noticed this message:
Column 'column_name' was removed from table definition 'schema.table': the column data type is LOB and the table has no primary key or unique index
In the settings of the task migration I tried to increase the lob limit in the option
Maximum LOB size to 2000000
But still getting the same result.
Does anyone know a workaround for this problem?
I guess, the problem is you do not have the primary key in your table.
From AWS documentation:
Currently, a table must have a primary key for AWS DMS to capture LOB
changes. If a table that contains LOBs doesn't have a primary key,
there are several actions you can take to capture LOB changes:
Add a primary key to the table. This can be as simple as adding an ID
column and populating it with a sequence using a trigger.
Create a materialized view of the table that includes a
system-generated ID as the primary key and migrate the materialized
view rather than the table.
Create a logical standby, add a primary key to the table, and migrate
from the logical standby.
Learn more
It is also important to have the primary key of a simple type, not LOB:
In FULL LOB or LIMITED LOB mode, AWS DMS doesn't support replication of primary keys that are LOB data types.
Learn more
What will happen if one column mentioned as primary key in DB and other column as primary key in mapping structure in informatica Powercenter or IICS.
It would not create an issue, as long as it is for reading the data.
But, if we are going to write the data in the target(using different keys) then Informatica would perform the record operation(I/U/D) on the basis of the key defined in Informatica.
I have a table in my model which is extracting data from an adls location in Azure Datalake.
I am getting the following error while deploying the model:
'Column 'xy_id' in Table 'ABC' contains a duplicate value '' and this is not allowed for columns on the one side of a many-to-one relationship or for columns that are used as the primary key of a table.
I have checked the adls file for duplicates. There are no duplicate values. Also checked the count and distinct count in AAS which is same.
All the tables are getting processed successfully. The error comes on the "Deploy Metadata" step and the deployment fails.
There are only 3 tables in the model. I have created a One to Many relationship from table ABC to other 2 tables.
Can anyone suggest any fixes? I am not able to figure out why I am facing this error.
Thanks in advance.
You are interacting with different systems which stores data in different ways. By default Azure Analysis Services is case INsensitive.
This means that AAA and aAa are the exact same thing. Therefore, it will cause issues if:
the field is defined as Unique in AAS
the field is defined as Primary Key in AAS
the field is on the one-side of a one-to-many relationship
You can either solve this issue in the data source or change the database settings in SSMS.
However, as a general best-practice please define relationship only on integer fields. For example, you could create a Surrogate Key in the data source.
Creating a table in Azure SQL Data Warehouse, I would like to make a hash distribution on an identity column, but get an error that
Cannot insert explicit value for identity column in table 'Table_ff4d8c5d544f4e26a31dbe71b44851cb_11' when IDENTITY_INSERT is set to OFF.
Is this not possible? And if not, why? And is there a work-around? (And where does this odd table name come from?)
Thanks!
You cannot use an IDENTITY column as the hash distributed column in your table.
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-tables-identity#limitations
In SQLDW the name you give to your table is its logical name not its physical name. Logical metadata such as table names is maintained centrally on the control node so that operations such as table renames are quick and painless. However, SQLDW is still bound by the rules of table creation - we need to make sure the table name is unique both now and in the future. Therefore the physical names contain guids to deliver that uniqueness.
Saying that, the error you have here is not ideal. It would be helpful if you can post a repro so that we can improve the experience for you.
You are also welcome to post a feature request on our uservoice channel for hash distribution on the IDENTITY column. https://feedback.azure.com/forums/307516-sql-data-warehouse
Background: I am new to Informatica. Informatica powercenter express Version: 9.6.1 HotFix 2
In my etl project I have several mappings to load different dimension and fact tables in a data mart. The ETL will run daily, one requirement is to add a audit key as a column to each of these tables. The audit key is an integer and is generated from a audit table (next value from the audit key column (primary key)). So everyday the audit key is increased by 1 etc. So after each etl load, all the new or updated rows in all tables (dimension/fact) will have this audit key in a column. The purpose is the ability to trace when or how each row is inserted/updated etc.
Now the question is how to generate such key and pass on to all the mappings? The key should be from the next value from auditkey column of audit table.
You could build a mapplet that generates/maintains the key you want and use it in all your workflows
If you have a RDBMS source, I would suggest creating a oracle sequencer in the DB and create oracle function to get the next value...
Call the the newly created oracle function in SQL Override and use the next value sequence number in all the mapping