Search Informatica for text in SQL override - informatica

Is there a way to search all the mappings, sessions, etc. in Informatica for a text string contained within a SQL override?
For example, suppose I know a certain stored procedure (SP_FOO) is being called somewhere in an INFA process, but I don't know where exactly. Somewhere I think there is a Post SQL on a source or target calling it. Could I search all the sessions for Post SQL containing SP_FOO ? (Similar to what I could do with grep with source code.)

You can use Repository queries for querying REPO tables(if you have enough access) to get data related with all the mappings,transformations,sessions etc.
Please use the below link to get almost all kind of repo queries.Ur answers can be find in the below link.
https://uisapp2.iu.edu/confluence-prd/display/EDW/Querying+PowerCenter+data
select *--distinct sbj.SUBJECT_AREA,m.PARENT_MAPPING_NAME
from REP_SUBJECT sbj,REP_ALL_MAPPINGS m,REP_WIDGET_INST w,REP_WIDGET_ATTR wa
where sbj.SUBJECT_ID = m.SUBJECT_ID AND
m.MAPPING_ID = w.MAPPING_ID AND
w.WIDGET_ID = wa.WIDGET_ID
and sbj.SUBJECT_AREA in ('TLR','PPM_PNLST_WEB','PPM_CURRENCY','OLA','ODS','MMS','IT_METRIC','E_CONSENT','EDW','EDD','EDC','ABS')
and (UPPER(ATTR_VALUE) like '%PSA_CONTACT_EVENT%'
-- or UPPER(ATTR_VALUE) like '%PSA_MEMBER_CHARACTERISTIC%'
-- or UPPER(ATTR_VALUE) like '%PSA_REPORTING_HH_CHRSTC%'
-- or UPPER(ATTR_VALUE) like '%PSA_REPORTING_MEMBER_CHRSTC%'
)
--and m.PARENT_MAPPING_NAME like '%ARM%'
order by 1
Please let me know if you have any issues.

Another less scientific way to do this is to export the workflow(s) as XML and use a text editor to search through them for the stored procedure name.

If you have read access to the schema where the informatica repository resides, try this.
SELECT DISTINCT f.subj_name folder, e.mapping_name, object_type_name,
b.instance_name, a.attr_value
FROM opb_widget_attr a,
opb_widget_inst b,
opb_object_type c,
opb_attr d,
opb_mapping e,
opb_subject f
WHERE a.widget_id = b.widget_id
AND b.widget_type = c.object_type_id
AND ( object_type_name = 'Source Qualifier'
OR object_type_name LIKE '%Lookup%'
)
AND a.widget_id = b.widget_id
AND a.attr_id = d.attr_id
AND c.object_type_id = d.object_type_id
AND attr_name IN ('Sql Query')--, 'Lookup Sql Override')
AND b.mapping_id = e.mapping_id
AND e.subject_id = f.subj_id
AND a.attr_value is not null
--AND UPPER (a.attr_value) LIKE UPPER ('%currency%')

Yes. There is a small java based tool called Informatica Meta Query.
Using that tool, you can search for any information that is present in the Informatica meta data tables.
If you cannot find that tool, you can write queries directly in the Informatica Meta data tables to get the required information.

Adding few more lines to solution provided by Data Origin and Sandeep.
It is highly advised not to query repository tables directly. Rather, you can create synonyms or views and then query those objects to avoid any damage to rep tables.
In our dev/ prod environment application programmers are not granted any direct access to repo. tables.

As querying the Informatica database isn't the best idea, I would suggest you to export all the workflows in your folder into xml using Repository Manager. From Rep Mgr you can select all of them once and export them at once. Then write a java program to search the pattern from the xml's you have.
I have written a sample prog here, please modify it as per your requirement:
make a spec file with workflow names(specFileName).
main()
{
try {
File inFile = new File(specFileName);
BufferedReader reader = new BufferedReader(newFileReader(infile));
String tectToSearch = '<YourString>';
String currentLine;
while((currentLine = reader.readLine()) != null)
{
//trim newline when comparing with String
String trimmedLine = currentLine.trim();
if(currentline has the string pattern)
{
SOP(specFileName); //specfile name
}
}
reader.close();
}
catch(IOException ex)
{
System.out.println("Error reading to file '" + specFileName +"'");
}
}

Related

Cannot create index on non-empty table

I'm currently using AWS Lambda (NodeJS) with AWS QLDB.
The scenario is like this.
I have the first table and its indexes when I deployed the service. So the table and indexes will be created. My problem is that, once I need to add new table and its indexes; it can't create the index because there's existing table.
My workaround to be able to create new table even if there's an existing table in my Ledger is that I'm querying the list of tables I have.
const getTables = async (transactionExecutor: TransactionExecutor) => {
const statement = `SELECT name FROM information_schema.user_tables`;
return await transactionExecutor.execute(statement);
};
Then I have this condition to check if the table is already existing
const tables = JSON.stringify(result.getResultList());
if (
!JSON.parse(tables).some((object): boolean => object.name === process.env.TABLE_NAME)
) {
console.log('TABLE A NOT EXISTING');
await createTable(transactionExecutor, process.env.TABLE_NAME);
}
if (
!JSON.parse(tables).some(
(object): boolean => object.name === process.env.TABLE_NAME_1,
)
) {
console.log('TABLE B NOT EXISTING');
await createTable(transactionExecutor, process.env.TABLE_NAME_1);
}
I don't know how to do it with indexes, I tried using SQL commands in QLDB but it's not working.
I hope you can help me.
Thank you
I'm not quite sure what your question is (the post title and body hint at different things), but I'm going to do my best to answer.
First, QLDB stores data in Ion, not JSON. So, please use the Ion APIs to parse data and not the JSON ones. The reason your code works at all is because Ion is a superset of JSON and the result set doesn't include types that are unknown to JSON. So, for example, if the result set was changed to include an Ion Timestamp, then your code would break.
Next, actually getting a list of tables has first class support in the driver. Simply use driver.getTableNames.
Third, I think you have a question "can I add an index to a non-empty table?". The answer is "no". This is planned functionality and I will update this answer when it is available. UPDATE: Now you can! https://aws.amazon.com/about-aws/whats-new/2020/09/amazon-qldb-launches-index-improvements/
Finally, I think you're also asking if there is a way to list indexes on a table in the same way as you can list tables in a ledger. The answer to that is 'yes'. The documents returned in information_schema.user_tables look like this:
{
tableId:"...",
name:"THE_TABLE_NAME",
indexes:[
{
expr:"[THE_FIELD_BEING_INDEXED]"
}
],
status:"ACTIVE"
}

Can we change location from US to other region while reading data from Bigquery using Bigquery java library?

I am trying to read data from Bigquery using Bigquery java library.
My dataset is not in US location, so when i am giving my dataset name to library , it is throwing an error that dataset not found in US location because it searches by default in US location.
I have also tried giving the location using setLocation("asia-southeast1") but still it is finding in US location.
This is my code snippet:
val bigquery: BigQuery =BigQueryOptions.newBuilder().setLocation("asia-southeast1").build().getService
val query = "SELECT TO_JSON_STRING(t, true) AS json_row FROM "+dbName+"."+tableName+" AS t"
logger.info("Query is " + query)
val queryResult: QueryJobConfiguration = QueryJobConfiguration.newBuilder(query).build
val result: TableResult = bigquery.query(queryResult)
I am writing code in SCALA. As it uses same libraries as JAVA and JAVA is more popular, thats why I am asking this for JAVA.
Please help me to know that how I can change location from US to southeast.
Can I change something inside QueryJobConfiguration as i have searched a-lot but i am unable to find anything.
My only requirement is that I want final result as TableResult.
This is the exception being thrown
com.google.cloud.bigquery.BigQueryException: Not found: Dataset XXXXXXXX was not found in location US
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:106)
at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.getQueryResults(HttpBigQueryRpc.java:584)
at com.google.cloud.bigquery.BigQueryImpl$34.call(BigQueryImpl.java:1203)
at com.google.cloud.bigquery.BigQueryImpl$34.call(BigQueryImpl.java:1198)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1197)
at com.google.cloud.bigquery.BigQueryImpl.getQueryResults(BigQueryImpl.java:1181)
at com.google.cloud.bigquery.Job$1.call(Job.java:329)
at com.google.cloud.bigquery.Job$1.call(Job.java:326)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.poll(RetryHelper.java:64)
at com.google.cloud.bigquery.Job.waitForQueryResults(Job.java:325)
at com.google.cloud.bigquery.Job.getQueryResults(Job.java:291)
at com.google.cloud.bigquery.BigQueryImpl.query(BigQueryImpl.java:1168)
...
Thanks in advance.
You shouldn't actually need to specify the location because BigQuery will infer it from the dataset being referenced in your query. See here.
When loading data, querying data, or exporting data, BigQuery
determines the location to run the job based on the datasets
referenced in the request. For example, if a query references a table
in a dataset stored in the asia-northeast1 region, the query job will
run in that region.
I just tested using the Java SDK on a dataset/table I created in asia-southeast1, and it worked without needing to explicitly specify the location.
If it's still not working for you by default (check the table you're referncing actually exists), then you can specify the location by setting it in the JobId and passing that to the overloaded method:
String query = "SELECT * FROM `grey-sort-challenge.asia_southeast1.a_table`;";
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query)
.setUseLegacySql(Boolean.FALSE)
.build();
JobId id = JobId.newBuilder().setLocation("asia-southeast1")
.setRandomJob()
.build();
try {
for (FieldValueList row : BIGQUERY.query(queryConfig, id).iterateAll()) {
for (FieldValue val : row) {
System.out.printf("%s,", val.toString());
}
System.out.printf("\n");
}
} catch (InterruptedException e) {
e.printStackTrace();
}

Annotating a document with JAPE

I have been searching for a solution to this for weeks, I have some documents(about 95) that I am trying to classify using GATE. I have put them in one corpus I called training_corpus, however, after ANNIE has annotated the corpus, I have to go back into each file, select all token in the document, and create an annotation called Mention, with feature type and value the class for the document. for example:
type Start End id Features
Mention 0 70000 2588 {type=neg}
Is there anyway to automatically do this with JAPE? Basically, I want to select all tokens and create a new annotation with feature(type=class). Also, the class is appended to the document. Since there are many documents, can JAPE extract the class from the document name and set it to the value of Mentions feature. Example document name is neg_data1.txt, so the annotation will be Mention.type = neg?
Any help will be greatly appreciated. Thanks
I think you answered to your question by yourself.If the class assignment based on just a token present in text - why not simply process text outside of GATE?
For example to create an xml file like:
text and then use it in training process.
Also you can create a simple JAPE rule which will:
a) will take a text within document boundaries (see gate.Utils.length methods AFAIR)
b) based on presence of your token will create a new Annotation instance with features necessary.
an abstract example:
Phase: Instance
Input: Token
Options: control = once
Rule:Instance
(
{Token}
):instance
-->
{
AnnotationSet instances = outputAS.get("INSTANCE_ANNOTATION");
FeatureMap featureMap = Factory.newFeatureMap();
if (instances!=null&&!instances.isEmpty()){
featureMap.put("features when annotation presented in doc");
}else{
featureMap.put("features when annotation not in doc");
}
outputAS.add(new Long(0), new Long(documentLength), "Mention", featureMap);
}

Sitecore Multisite Manager and 'source' field in template builder

Is there any way to parametise the Datasource for the 'source' field in the Template Builder?
We have a multisite setup. As part of this it would save a lot of time and irritation if we could point our Droptrees and Treelists point at the appropriate locations rather than common parents.
For instance:
Content
--Site1
--Data
--Site2
--Data
Instead of having to point our site at the root Content folder I want to point it at the individual data folders, so I want to do something like:
DataSource=/sitecore/content/$sitename/Data
I can't find any articles on this. Is it something that's possible?
Not by default, but you can use this technique to code your datasources:
http://newguid.net/sitecore/2013/coded-field-datasources-in-sitecore/
You could possibly use relative paths if it fits with the rest of your site structure. It could be as simple as:
./Data
But if the fields are on random items all over the tree, that might not be helpul.
Otherwise try looking at:
How to use sitecore query in datasource location? (dynamic datasouce)
You might want to look at using a Querable Datasource Location and plugging into the getRenderingDatasource pipeline.
It's really going to depend on your use cases. The thing I like about this solution is there is no need to create a whole bunch of controls which effectively do he same thing as the default Sitecore ones, and you don't have to individually code up each datasource you require - just set the query you need to get the data. You can also just set the datasource query in the __standard values for the templates.
This is very similar to Holger's suggestion, I just think this code is neater :)
Since Sitecore 7 requires VS 2012 and our company isn't going to upgrade any time soon I was forced to find a Sitecore 6 solution to this.
Drawing on this article and this one I came up with this solution.
public class SCWTreeList : TreeList
{
protected override void OnLoad(EventArgs e)
{
if (!String.IsNullOrEmpty(Source))
this.Source = SourceQuery.Resolve(SContext.ContentDatabase.Items[ItemID], Source);
base.OnLoad(e);
}
}
This creates a custom TreeList control and passes it's Source field through to a class to handle it. All that class needs to do is resolve anything you have in the Source field into a sitecore query path which can then be reassigned to the source field. This will then go on to be handled by Sitecore's own query engine.
So for our multi-site solution it enabled paths such as this:
{A588F1CE-3BB7-46FA-AFF1-3918E8925E09}/$sitename
To resolve to paths such as this:
/sitecore/medialibrary/Product Images/Site2
Our controls will then only show items for the correct site.
This is the method that handles resolving the GUIDs and tokens:
public static string Resolve(Item item, string query)
{
// Resolve tokens
if (query.Contains("$"))
{
MatchCollection matches = Regex.Matches(query, "\\$[a-z]+");
foreach (Match match in matches)
query = query.Replace(match.Value, ResolveToken(item, match.Value));
}
// Resolve GUIDs.
MatchCollection guidMatches = Regex.Matches(query, "^{[a-zA-Z0-9-]+}");
foreach (Match match in guidMatches)
{
Guid guid = Guid.Parse(match.Value);
Item queryItem = SContext.ContentDatabase.GetItem(new ID(guid));
if (item != null)
query = query.Replace(match.Value, queryItem.Paths.FullPath);
}
return query;
}
Token handling below, as you can see it requires that any item using the $siteref token is inside an Site Folder item that we created. That allows us to use a field which contains the name that all of our multi-site content folders must follow - Site Reference. As long at that naming convention is obeyed it allows us to reference folders within the media library or any other shared content within Sitecore.
static string ResolveToken(Item root, string token)
{
switch (token)
{
case "$siteref":
string sRef = string.Empty;
Item siteFolder = root.Axes.GetAncestors().First(x => x.TemplateID.Guid == TemplateKeys.CMS.SiteFolder);
if (siteFolder != null)
sRef = siteFolder.Fields["Site Reference"].Value;
return sRef;
}
throw new Exception("Token '" + token + "' is not recognised. Please disable wishful thinking and try again.");
}
So far this works for TreeLists, DropTrees and DropLists. It would be nice to get it working with DropLinks but this method does not seem to work.
This feels like scratching the surface, I'm sure there's a lot more you could do with this approach.

output variable stored in database

I'm storing the page content in a database table. The page content also includes some CF variables (for example "...this vendor provides services to #VARIABLES.vendorLocale#").
VARIABLES.vendorLocal is set on the page based on a URL string.
Next a CFC is accessed to get the corresponding page text from the database.
And this is then output on the page: #qryPageContent.c_content#
But #VARIABLES.vendorLocale# is showing up as is, not as the actual variable. Is there anyway to get a "variable within a variable" to be output correctly?
This is on a CF9 server.
If you have a string i.e.
variables.vendorLocal = 'foo';
variables.saveMe = 'This is a string for supplier "#variables.vendorLocal#'"' ;
WriteOutput(variables.saveMe); // This is a string for locale "foo"
then coldfusion will attempt to parse that to insert whatever variable variables.vendorLocale is. To get around this, you can use a placeholder string that is not likely to be used elsewhere. Commonly you'll see [[NAME]] used for this purpose, so in this example
variables.saveMe = 'This is a string for supplier "[[VENDORLOCALE]]'"' ;
WriteOutput(variables.saveMe); // This is a string for supplier "[[VENDORLOCALE]]"
Now you've got that you can then later on replace it for your value
variables.vendorLocal = 'bar';
variables.loadedString = Replace(variables.saveMe,'[[VENDORLOCALE]]',variables.vendorLocal);
WriteOutput(variables.loadedString); // This is a string for locale "bar"
I hope this is of help
There are lots of reasons storing code itself in the database is a bad idea, but that's not your question, so I won't go into that. One way to accomplish what you want is to take the code you have stored as as string, write a temporary file, include that file in the page, then delete that temporary file. For instance, here's a little UDF that implements that concept:
<cfscript>
function dynamicInclude(cfmlcode){
var pathToInclude = createUUID() & ".cfm";
var pathToWrite = expandPath(pathToInclude);
fileWrite(pathToWrite,arguments.cfmlcode);
include pathToInclude;
fileDelete(pathToWrite);
}
language = "CFML";
somecfml = "This has some <b>#language#</b> in it";
writeOutput(dynamicInclude(somecfml));
</cfscript>