I have two tables in different databases. In a table A is the data, in the other table B are information for incremental load of the data from the first table. I want to load from table B and store the date of the last successful load from table A in a mapping variable $$LOAD_DATE. To achieve this, I read a date from table B and use the SETVARIABLE() function in a expression to set the $$LOAD_DATE variable. The port in which I do this is marked as output and writes into a dummy flat file. I only read on row of this source!
Then I use this $$LOAD_DATE variable in the Source Filter of the Source Qualifier of table A to only load new records which are younger than the date stored in the $$LOAD_DATE variable.
My problem is that I am not able to set the $$LOAD_DATE variable correctly. It is always the date 1753-1-1-00.00.00, which is the default value for mapping variables of the type date/time.
How do I solve this? How can I store a date in that variable and use it later in a Source Qualifiers source filter? Is it even possible?
EDIT: Table A has too much records to read them all and filter them later. This would be to expensive, so they have to be filtered at source filter level.
Yes, it's possible.
In the first map you have to initialize the variable, like this:
In first session configuration you have to define the Post-session on success variable assignment:
The second map (with your table A) will get the variable after this configuration of the session in Pre-session variable assignment:
It will work.
It is not possible to set a mapping variable and use it's value somewhere else in the same run, because, the variable is actually set when the session completes.
If you really want to implement it using mapping variables you have to create two mappings, one for setting the mapping variable and another for actual incremental load. You can pass a mapping variable value from one session to another in a workflow using a workflow variable. https://stackoverflow.com/a/26849639/2626813
Other solutions could be to use a lookup on B and a filter after that.
You can also write some scripts to query table B and modify the parameter file with the latest $LOAD_DATE value prior to executing the mapping.
Since we're having two different DBs, use two sessions. Get values in the first one and pass the parameters to the second one.
Related
I have a source schema in which a particular record is optional and in the source message instance the record does not exist. I need to map this record to destination record, scenario goes like if the source record doesn't exist, need to map a default value 0 to destination nodes. and If it does exists , need to pass the source node values as it is (followed by few arithmetic operations).
I have tried using various combinations of functoids like logical existence followed by value mapping,record count ,string existence,etc. Also tried using c# within scripting functoid and also xslt , nothing works.its very tough to deal with mapping non existing records. I have several records on top of this record which are mapped just fine and they do exists. having trouble only with this one.No matter how many combination of c# and xslt code i write , it feels like scripting functoid will never accept a non existence record or node link. Mind you that this record if exists ,can repeat multiple times.
Using BizTalk2013r2.
If the record doesn't exist (record is not coming, not even as < record/>) you can use this simple combination of Functoids.
Link the record to Logical Existence, if exist it will be sent by the top Value Mapping. If doesn't exit the second condition will be true and the zero will be sent from the value mapping in the bottom.
I am new to "Rapid miner" tool. There are two data set in my process. What I want to do is, generate a process which does the following:
To create this process should use Generate Attribute, Append and type conversion operators in RapidMiner
The first data set has a car name attribute, whereas the second data set has a name attribute. name should be renamed to car name.
The second data set has an additional other attribute which is not present in the first data set. Update the first data set to add an additional other attribute, with a default value of 1. This attribute should also have a type of Integer.
Append the modified second data set to the modified first data set
Export the new data to a new excel spreadsheet
I found the solution. Hope it will help for others
Please use below process flow
http://i.stack.imgur.com/omfDe.png
I am trying to realize some mappings as supposed in this answer. I created a mapping, which reads from a table in which a date for incremental load is stored. After that I set a mapping variable to pass this date value to the next mapping. In the Post-session on success variable assignment the mapping variable is stored in a workflow variable and passed to the next mapping.
Here is the mapping to read the date value and store it in the mapping variable. The value is stored in the variable in the expression, the port is an output, which is linked to a dummy target. This target simply writes it in a flat file.
The port expression is SETVARIABLE($$LOAD_FROM_DATE,LOAD_DATE).
My problem is that the value is read correctly, but it is not persisted in the mapping variable. It is always falling back on the date default value. Where is my mistake?
So you basically need to calculate a value from one mapping and use it in the second? If so, I have implemented something similar and it works fine for me. I have the port where I set variable (RESULT_LOAD in your case) value marked as a variable port. In the workflow you will be defining a variable to capture the variable value from mapping, mark it as persistent so that the value is stored in the repository after each run.
I'm trying to use a 2 workflows to archive any files when created or updated. The first simply moves a copy to a separate doc library. no issues
The second should rename the file once it arrives to append a date (and possible timestamp) to the end of the file so that it is a unique record.
I am trying to set a variable called Archive_Name and then setting the field value to the Archive_Name before commiting the change.
I am using this fomula to set the variable
Name-fn-FormatDate(Current Date,yyyy-MM-dd)
Both Name and Current Date are recognised variable.
When I run this the Name stays the same and does not append a date. If I run it as
fn-FormatDate(Current Date,yyyy-MM-dd)
the Name changes to my desired date proving that the formula is working, the text is being assigned to the Archive_Date variable and the variable is being applied to the field value.
What am I doing wrong?
I believe you need to concatenate the two variables. The & operator can be used in place of the CONCATENATE function. Thus; Name&fn-FormatDate(Current Date,yyyy-MM-dd)
Hope this helps
I have a workflow which writes data from a table into a flatfile. It works just fine, but I want to insert a blank line inbetween each records. How can this be achieved ? Any pointer ?
Here, you can create 2 target instances. One with the proper data and in other instance pass blank line. Set Merge Type as "Concurrent Merge" in session properties.
Multiple possibilities -
You can prepare appropriate dataset into a relational table, and afterwards, dump data from that into a flat file. For preparation of that data set, you can insert blank rows into that relational target.
Send a blank line to a separate target file (based on some business condition using a router or something similar), after that you can use merge files option (in session config) to get that data into a single file.