I get the following error:Service Invocation Exeption i am working with Version 8.7 IBM InfoSphere DataStage and QualityStage Designer and using a server job and there, i have 1 sequential file, web service, sequential file.
Any idea what could be the reason of this error ?
Make sure you have chosen proper DataStage job type and your stage that operates on web service is configured properly.
You should also check the DataStage logs to get more information about root cause of the error.
Related
I am using Sitecore 8.2 Update 6. Earlier my CM, Processing and Reporting roles are on single CM server. Now I just need to use separate Processing server and my Reporting and CM will be on one server.
I have configured my processing server as mentioned in the following url:
https://doc.sitecore.net/sitecore_experience_platform/82/setting_up_and_maintaining/xdb/configuring_servers/configure_a_processing_server
and configured my connection strings as per the following url:
https://doc.sitecore.net/sitecore_experience_platform/81/setting_up_and_maintaining/xdb/configuring_servers/database_connection_strings_for_configuring_servers
Now I have couple of questions:
1) IS there any change required in my CM or CD to know about my separate processing server
2) How can I test whether my processing server is doing the required tasks.
Thanks,
Nicks
Your CM and CD do not need to know about the processing server, but you need to make sure that processing functions are not enabled on the CM or CD.
You will know if processing is working by looking at the logs and seeing if the pipelines are executing and not throwing errors.
You will also see analytics data being processed and showing up in the reporting database. If you are not seeing analytics data, this is an indication you might have errors in processing.
Note that there are several possible reasons reporting data might not be working, but if it is succeeding at getting your new analytics data than processing is running.
I have created a Talend job and deployed it as a Axis Web Service.
I am calling this Web Service from my Web Application.
My application and web service are deployed over Tomcat server.
For Valid Inputs:
Runs Perfectly.
For Invalid Inputs:
It throws Error/exceptions in Tomcat Server.
I want this Errors into my application. I didn't get any error related information in the Response sent by web service.
Error message on Tomcat Server
XML response of Web service
If I understand your question you are not so much asking how to fix this error, but how to get feedback about your error. There are several ways but here is a simple method.
Add a tLogCatcher to your job, and point its output to a tSendMail component. You can run the output thru a tMap do things like adding HTML formatting. You can also create a joblet with these two components and use the joblet in every job so you do not have to recreate it every time.
The diagram below shows a real simple case where I add tLogCatcher and tSendMail directly to the job. When there is an error it will send me an email with the details. You can use any of the columns in row11, I am only using job and message in my example.
This error : For input string "fabrik"
is generated by the tFileInputExcel component when encountering a non valid field (expected : integer ; given : string). You can't directly catch this error with the tFileInputExcel component (it's just printed out on the console).
However, you can use a tSchemaComplianceCheck component after your tFileInputExcelComponent : it will throw an error if a data is not valid for your schema. This component has a reject link to catch the error.
tFileInputExcel->tSchemaComplianceCheck->tMap->tMSOutput
If you use tLogCatcher component to catch the error then, at the end of the flow, you can add a tBufferOutput component. Keep the fields with the information you want for.
component configuration
The result will be like this (note that they are shown in the same order):
result
I registered my task app in Spring Cloud Data Flow, created a definition for it and the status shows 'unknown'. I created the stream and trying to launch the task through task-sink and I get an error:
java.lang.IllegalStateException: failed to resolve MavenResource:
How to launch a task from the task-sink? Am I missing something? Any help is appreciated. Another question I have is how do I access the payload sent via TaskLaunchRequest in my task?
S1 http | step1: transformer-rabbit | log
S2 :S1.step1 > filter --expression=payload.contains('CUSTADDRMODRQ_V15') | task-processor | task-sink
task-sink is launching the task provided by the uri in the TaskLaunchRequest. It is looking for the resource as shown in the log
OUT Using manager EnhancedLocalRepositoryManager with priority 10.0 for /home/vcap/.m2/repository
OUT Using transporter HttpTransporter with priority 5.0 for https://repo.spring.io/libs-snapshot and finally failing.
The task is deployed in our repository and as mentioned I registered and created the definition for it as well.
This one is in cf environment and I am using SCDF server 1.0.0.M4.
In the application.properties for the task-sink i am providing maven.remote.repositories.snapshots.url=**
task create fis-ifx-event-task --definition "fis-event-task"
My goal is launching the task from the stream.
Thanks for the information. I am in fact using the BUILD-SNAPSHOT as I am unable to enable taks in 1.0.0M4 version. Here is the one I am using spring-cloud-dataflow-server-cloudfoundry-1.0.0.BUILD-20160808.144306-116. I am able to register and create task definitions. The status of the task definition is showing as 'unknown' even when I am using the sample task module provided by your team. But when I initiate the flow of the stream and when task-sink tries to launch the task, it is unable to find the maven resource. When I create the task definition, does the task module gets deployed? I don't see any app in Pivotal Apps Manager. As mentioned earlier, I provided maven.remote.repositories.snapshot.url in the application.properties file for the task-sink application. Another thing I observed is when I launch the task manually from dataflow shell it gives an error CF-UnprocessableEntity(10008): The request is semantically invalid: Unknown field(s): 'staging_disk_in_mb', 'staging_memory_in_mb' and also a message saying 'Source is empty'. Presently the task is supposed to print the timestamp and is not dependent on any input.
TaskProcessor code:
#EnableBinding(Processor.class)
#EnableConfigurationProperties(TaskProcessorProperties.class)
public class TaskProcessor {
#Autowired
private TaskProcessorProperties processorProperties;
public TaskProcessor() {
}
#Transformer(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
#ELI(level = "info", eventType = ELIEventType.INBOUND)
public Object setupRequest(String message) {
Map<String, String> properties = new HashMap<String, String>();
properties.put("payload", message);
TaskLaunchRequest request = new TaskLaunchRequest(processorProperties.getUri(), null, properties, null);
return new GenericMessage<>(request);
}
}
TaskSink code:
#SpringBootApplication
#EnableTaskLauncher
#EnableBinding(Sink.class)
#EnableConfigurationProperties(TaskSinkProperties.class)
public class FisIfxEventTaskSinkApplication {
public static void main(String[] args) {
SpringApplication.run(FisIfxEventTaskSinkApplication.class, args);
}
}
I provided the stream I am using earlier in the post. Sink is receiving the TaskLaunchRequest with uri and payload as you can see here and unable to launch the task.
OUT registering [40, java.io.File] with serializer org.springframework.integration.codec.kryo.FileSerializer
2016-08-10T16:08:55.02-0600 [APP/0]
OUT Launching Task for the following resource TaskLaunchRequest{uri='maven://com.xxx:fis.ifx.event-task:jar:1.0-SNAPSHOT', commandlineArguments=[], environmentProperties={payload={"statusCode":0,"fisT
opic":"CustomerDataUpdated","payloadId":"CUSTADDRMODR``Q_V15","customerIds":[1597304]}}, deploymentProperties={}}
Before I begin, you have a number of questions here. In the future, it's better to break them up into multiple questions so that they are easier to find by other users and easier to answer. That being said:
A little context on the current state of things
In order to understand how things will work, it's important to understand the current state of things. The current releases of the software involved are:
Pivotal Cloud Foundry (PCF) - 1.7.12. This version is required for any task support.
Spring Cloud Task (SCT) - 1.0.2.RELEASE
Spring Cloud Data Flow CF (SCDF) - 1.0.0.BUILD-SNAPSHOT (current as of the date of this post).
Currently PCF 1.7.12+ has all the capabilities to run tasks. You can create v3 applications (the type of application used to launch a task), run it as a task, etc. However, the tooling around that functionality is not currently complete. There is no support for v3 applications in Apps Manager or the CLI. There is a plugin for the CLI that is more of a dev tool that can be used to help with some functions (it will show you logs, etc), but it is not fully functional and requires a specific version of the CLI to work [1]. This is one of the reasons that the task functionality within PCF is still considered experimental.
Spring Cloud Task is currently GA and supports all the functionality needed to effectively run tasks on CF. However, it's important to note that SCT doesn't handle orchestration so the actual launching of tasks on CF is the responsibility of either the user, or Spring Cloud Data Flow (the easier route).
Spring Cloud Data Flow's Cloud Foundry server implementation currently has functionality to launch tasks on PCF in the latest snapshots. We have validated this against 1.7.12 as well as the development branch of 1.8.
The task workflow within SCDF
Tasks are fundamentally different from stream applications within the context of SCDF. When you create a stream definition, you are given the option to deploy it. What this does is it actually downloads the Spring Boot über jars and deploys them to PCF as long running processes. If they go down, PCF, will relaunch them as expected, etc.
Tasks on the other hand, are not deployed. They are launched. The difference is that while you create a task definition, there is nothing deployed until you click launch. And when the task completes, the software is shut down and cleaned up. So while a stream definition may have states, it's really a one to one relationship between the definition and the deployed software. Where with a task, you can launch a task definition as many times as you want.
Your issues
Reading through your post, I see a few things that you are struggling with. Let me see if I can help:
Task Definitions within SCDF and launching them via a stream - When launching a task from a stream, the task registry within SCDF is not used. The sink expects the URL for the resource to be within the TaskLauchRequest.
Apps Manager and tasks - As mentioned above, there is no support for v3 applications in Apps Manager yet so you won't be able to see your tasks there.
Viewing the logs - In order to debug what's going wrong with launching your task on CF, you're going to want to view the logs. To do so, use the v3 CLI plugin mentioned above to view them. It's important to note that you can only tail live logs with the plugin, not view logs that have previously been rendered. Because of that, when testing, you'll want to tail the logs as soon as the app is created, before it's launched.
Error in SCDF Shell - The error you received from the SCDF shell (CF-UnprocessableEntity(10008):...) leads me to wonder if you have both the correct version of PCF (1.7.12+) and the correct version of the following other libraries:
spring-cloud-deployer-cloudfoundry - The latest snapshots
cf-java-client - 2.0.0.M10+
reactor-core - 3.0.0.RC1+
I hope this helps!
[1] https://github.com/cloudfoundry/v3-cli-plugin
Task support is not available in 1.0.0.M4 release of SCDF's CF-server. In this release, the task commands/REST-APIs should be disabled - see here. And for that reason, you wouldn't see any docs related to Tasks in the 1.0.0.M4 reference guide.
That said, the Task support is available/enabled in the BUILD-SNAPSHOT release. If you're locally building the CF-server and upon pushing it to CF, you could take advantage the task commands in the shell to create and launch task definitions.
While trying to follow the instructions from the wso2am (1.10.0) manual, regarding working with statistics with the wso2das (3.0.1) server i have encountered a problem.
If i choose to let the wso2am server define the stream while making the first call of the api, the wso2das server refuses to post statistics to the WSO2_STATS_DB.
If on the other hand i choose to import the analytics.car file in wso2das (as stated here ) i get an exception (AsyncDataPublisher Stream definition already exist) because the org.wso2.apimgt.statistics.request defined in the latest Analytics.car is different to the one being send from wso2am.
I pinpointed the problem in the definition of the Eventstream_request_1.0 in files
org.wso2.apimgt.statistics.request_1.0.0.json ,
throttledOutORG_WSO2_APIMGT_STATISTICS_REQUEST.xml
where the definition of the throttledOut option is missing
Is there a way to solve this issue?
Thank you.
I think your DAS is in some kind of a corrupted state. Can you first delete the car application (/repository/deployment/server/carbonapps) and then log in to DAS and go to Manage > Event > Streams and delete any existing streams. Then try again to deploy the car app in the /repository/deployment/server/carbonapps location.
If everything goes well you would see two scripts in Manage > Batch Analytics > Scripts section. Try to execute each script and see if there is any error. If not then you can point the API manager to DAS
I get the error below when I try to access the Amazon SQS WSDL:
http://queue.amazonaws.com/doc/2012-11-05/QueueService.wsdl
Cannot access the WSDL or the WSDL file is invalid.
I believe I have .NET 4 SDK installed and I have tried downloading the WSDL file to a local drive and pointing the proxy wizard to it. Still the same error.
Can someone try to use it and let me know your outcomes?
Try running the .Net WSDL utility directly on the WSDL. That utility reports back error information. It's also what PowerBuilder is calling under the covers, but is not sharing the error information back to you.
When I do that, I get this result:
Error: Unable to import binding 'SimpleQueueServicePostBinding' from namespace '
http://queue.amazonaws.com/doc/2012-11-05/'.
- The operation 'GetQueueUrl' on portType 'SimpleQueueServicePortType' from na
mespace 'http://queue.amazonaws.com/doc/2012-11-05/' had the following syntax error:
The operation has no matching binding. Check if the operation, input and
output names in the Binding section match with the corresponding names in the PortType section.
It looks like it might be a problem with the format of the WSDL. Not the first time that's happened, I've had to edit one of their other WSDL files by hand to correct an error in it.
If you choose to do that, you can download the file to your local machine, make the edits, and then run the PB proxy tool against the local file.