I'm trying to implement custom pig storer that based on org.apache.pig.builtin.PigStorage . In my storer I want to calculate the real hdfs location from the string that sent at 'INTO' word and other job properties.
-- pig
STORE data INTO 'MyObject' using ...
I extend PigStorage and override setStoreLocation function, like this:
public void setStoreLocation(String location, Job job) throws IOException {
// location is 'MyObject' here
// my manipulations on location
String newLocation = basePath + '/' + someVar + '/' + location;
super.setStoreLocation(newLocation, job);
}
It works and write the file into the newLocation (/basePath/someVar/MyObject) , but I'm getting following message in the log, how I can to avoid it?
java.io.FileNotFoundException: File hdfs://myMachine:8020/user/hdfs/MyObject does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:654)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:102)
at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:712)
at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:708)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:708)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader.getOutputSize(FileBasedOutputSizeReader.java:65)
at org.apache.pig.tools.pigstats.JobStats.getOutputSize(JobStats.java:543)
at org.apache.pig.tools.pigstats.JobStats.addOneOutputStats(JobStats.java:567)
at org.apache.pig.tools.pigstats.JobStats.addOutputStatistics(JobStats.java:516)
at org.apache.pig.tools.pigstats.PigStatsUtil.addSuccessJobStats(PigStatsUtil.java:360)
at org.apache.pig.tools.pigstats.PigStatsUtil.accumulateStats(PigStatsUtil.java:257)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:341)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1322)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1307)
at org.apache.pig.PigServer.execute(PigServer.java:1297)
at org.apache.pig.PigServer.executeBatch(PigServer.java:375)
at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:478)
at org.apache.pig.PigRunner.run(PigRunner.java:49)
at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:286)
at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:226)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38)
at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Looks like overriding of org.apache.pig.StoreFuncInterface#relToAbsPathForStoreLocation solves the problem:
#Override
public String relToAbsPathForStoreLocation(String location, Path curDir)
throws IOException {
return location;
}
Related
I have a table defined 2 filed:integer id,varchar name
I can execute common query like this right:
ResultSet resultSet = statement.executeQuery("select \"id\",\"name\" from test.test01 where \"id\" in (1,2)");
Now，I want use "if" condition in select statement，execute this query：
ResultSet resultSet = statement.executeQuery("select \"id\",IF(\"id\">=1, 100, 200) as myid,\"name\" from test.test01 where \"id\" in (1,2)");
Caught Exception：
java.sql.SQLException: Error while executing SQL "select "id",IF("id">=1, 100, 200) as myid,"name" from test.test01 where "id" in (1,2)": From line 1, column 13 to line 1, column 33: No match found for function signature IF(<BOOLEAN>, <NUMERIC>, <NUMERIC>)
at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:163)
at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at org.example.Client.main(Client.java:40)
Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, column 13 to line 1, column 33: No match found for function signature IF(<BOOLEAN>, <NUMERIC>, <NUMERIC>)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:932)
at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:917)
at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:5266)
at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1948)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:326)
at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:231)
at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6277)
at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6264)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1862)
at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1847)
at org.apache.calcite.sql.SqlAsOperator.deriveType(SqlAsOperator.java:133)
at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6277)
at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:6264)
at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:161)
at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1862)
at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1847)
at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:463)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4409)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3652)
at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:64)
at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:89)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1100)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:1071)
at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:247)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:1046)
at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:752)
at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:586)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:257)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:648)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:514)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:484)
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:234)
at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:623)
at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:674)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
... 2 more
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match found for function signature IF(<BOOLEAN>, <NUMERIC>, <NUMERIC>)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:505)
at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:599)
... 39 more
How To Troubleshot this problem, is there any thing wrong?
————————
Solution1：add connection config like below
URL url = Client.class.getResource("/model.json");
String str = URLDecoder.decode(url.toString(), "UTF-8");
Properties info = new Properties();
info.put("model", str.replace("file:", ""));
info.put("fun","hive");
Connection connection = DriverManager.getConnection("jdbc:calcite:", info);
Solution2：use case-when instead：
ResultSet resultSet = statement.executeQuery("select \"id\",\"id\"*10 as myid1,case when \"id\">1 then 100 else 200 end as myid2,\"name\" from test.test01 where \"id\" in (1,2,3)");
I was not able to find the IF function in the calcite docs. I think it is MySQL specific.
Instead you have CASE
I think the correct way to use it here would be
CASE WHEN
"id">=1 THEN 100
ELSE 200
END
Check out the docs here: https://calcite.apache.org/docs/reference.html
EDIT: good spot by OP, IF exists, but can only be used with the right databases and connection strings.
When i work with s3 on Amazon, I keep getting the ugly ConnectionPoolTimeoutException.
The problem is that due to the front end of the application, i can not close the opened s3 objects before front end is done with them so i have implemented this solution :
#Autowired
private AmazonS3 s3client; // credentials are set properly.
private static List<S3Object> openedObjects = new ArrayList<S3Object>();
// initialize bucket :
private String bucketName = "myShinyNewBucket";
private synchronized boolean initBucket(){
try{
Boolean exists = null;
try{
exists = s3client.doesBucketExist(bucketName);
}catch(Exception e1){
System.out.println("\n\n\tToo many opened objects ; closing...\n\n");
deleteOpenedS3Objects();
exists = s3client.doesBucketExist(bucketName);
}
if(exists!=null){
if(!exists){
s3client.createBucket(new CreateBucketRequest(bucketName));
}
return true;
}
}
catch(Exception e){
System.out.println("\n\n\tFailed to initialize bucket.\n");
e.printStackTrace();
}
return false;
}
private synchronized void deleteOpenedS3Objects(){
System.out.println("\n\tClosing opened objects...");
try{
for(int i=0 ; i<openedObjects.size() ; i++){
openedObjects.get(i).close();
openedObjects.remove(i);
}
}catch(Exception e1){
System.out.println("\tCould not close all opened s3 objects, only the first "+i);
}
System.out.println("\tTrying again :\n\n");
}
// GET :
public final String getFromAWS(final String amazonName){
S3Object s3object = null;
if(initBucket()){
try{
try{
s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
}catch(AmazonClientException e){
deleteOpenedS3Objects();
s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
}
openedObjects.add(s3object);
return s3object.getObjectContent().getHttpRequest().getURI().toString();
}catch(Exception e1){
if (((AmazonS3Exception)e1).getStatusCode() == HttpStatus.SC_NOT_FOUND){
System.out.println("\n\nNo such object in bucket.\n");
}
else{
System.out.println("\n\n\tCould not read bject from bucket.\n\n");
e1.printStackTrace();
}
}
}
return null;
}
Yet, the exception is still happening.
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
...
Only when i do ctrl+c in console, does it get to the part where it is closing the opened s3 connections :
... Caused by: java.lang.InterruptedException: Operation interrupted
at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:142) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107) ~[httpcore-4.4.4.jar!/:4.4.4]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) ~[httpclient-4.5.1.jar!/:4.5.1]
... 148 common frames omitted
Too many opened objects. // <-- This is where it catches it.
Closing opened objects...
Could not close all opened s3 objects.
Trying again :
Failed to initialize bucket.
Again, I am unfortunately not in a position where i can close opened s3objects before i leave the functions in my s3-client class. The only hope i have had, was to wait until TimeoutException happens, catch it, then closing all opened objects and trying again.
However, i can't seem to catch it in the right place.
Please, help.
Thank you.
We must use try with resources whenever we are dealing with any type of file read or write. This will auto close the resources used, even if we face any sort of exception.
In My case : I was getting 404 while reading from S3 for bulk operation, which i ignored while writing script by logging the error and forget to close the connection in finally block.
Example snippet:
static String readFirstLineFromFile(String path) throws IOException {
try (BufferedReader br =
new BufferedReader(new FileReader(path))) {
return br.readLine();
}
}
Note:
Try with resources have support from JDK 8 and above.
All the resources declared in try() must implement the AutoCloseable interface.
I think the sollution is to catch and process the ugly TimeoutException in my #ControllerAdvice class. I have done so and so far i have not had it happen in the app. I'll post a confirmation once i'm sure.
I have been using org.elasticsearch.test.ElasticsearchIntegrationTesting class for testing elasticsearch functionality.
I Use below code to import data into Elasticsearch and to search that data :**
#Test
public void searchData() throws Exception{
logger.info("searchData : searching data from Elasticsearch");
try {
csvImporter.client = client();
csvImporter.importData(); // In the importData method after data is imported we close the client
//In the below line getting Exception and it is inconsistent. Some times works fine
SearchResponse searchResponse = csvImporter.client.prepareSearch().setQuery(matchAllQuery()).execute().actionGet();
for (SearchHit searchHit : searchResponse.getHits()){
logger.info("Source Data >>>>>>>>>>>>>>>>>>>>> " + searchHit.sourceAsString());
}
} finally {
csvImporter.client.close();
}
}
Exception:
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: []
at __randomizedtesting.SeedInfo.seed([86EB1B4AAE1739A5:99315B8D3D924DA1]:0)
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:305)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:200)
at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:86)
at org.elasticsearch.client.support.AbstractIndicesAdminClient.deleteTemplate(AbstractIndicesAdminClient.java:687)
at org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequestBuilder.doExecute(DeleteIndexTemplateRequestBuilder.java:40)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65)
at org.elasticsearch.test.TestCluster.wipeTemplates(TestCluster.java:177)
at org.elasticsearch.test.TestCluster.wipe(TestCluster.java:76)
at org.elasticsearch.test.ElasticsearchIntegrationTest.beforeInternal(ElasticsearchIntegrationTest.java:292)
at org.elasticsearch.test.ElasticsearchIntegrationTest.before(ElasticsearchIntegrationTest.java:1946)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:898)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:59)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:59)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Since it is a Unit Testing I have not used configuration Settings to create client.
is there a way to recreate the client Object with same configuration settings??
Please help me in this, #BeforeClass runs successfully, but going into the #Test method it throws the below exception. The code in the `#Test' doesn't run properly.
I was able to run the code successfully before few months but now it is not working. I'm using selenium-server-standalone-2.39.0.jar.
The exception:
java.lang.NullPointerException
at scripts.IRS_TP_3_Contribution.test990TP2(IRS_TP_3_Contribution.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:335)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
Here is the Code:
#BeforeClass
public void launchBrowser() throws IOException, InterruptedException
{
System.setProperty("webdriver.chrome.driver","E://FELIX//Automation Testing//Jar files//chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("Link Hided");
driver.findElement(By.id("LEmailAddress")).clear();
driver.findElement(By.id("LEmailAddress")).sendKeys("felix#gmail.com");
driver.findElement(By.id("Password")).clear();
driver.findElement(By.id("Password")).sendKeys("123456");
driver.findElement(By.id("btnSubmit")).click();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.findElement(By.xpath("//body/div[2]/div[3]/div[3]/div/div[1]/table/tbody/tr/td[1]/form/div/div[3]/table/tbody/tr[9]/td[3]/a")).click();
//driver.findElement(By.xpath("(//a[contains(text(),'Edit')])[5]")).click();
driver.findElement(By.id("btnContributionRevenueStart")).click();
driver.findElement(By.id("btnContributionGiftsStart")).click();
driver.findElement(By.linkText("Add Contributions and Grants")).click();
//driver.findElement(By.id("ContributorName")).clear();
// Need Iteration for 992
//driver.findElement(By.id("Line1_IsGrantsEquivalentYes")).click();
//driver.findElement(By.xpath("//input[#value='Go to Express990']")).click();
/*for (int second = 0;; second++) {
if (second >= 20) fail("timeout");
try { if (isElementPresent(By.xpath("//input[#value='Go to Express990']"))) break; } catch (Exception e) {}
Thread.sleep(1000);
}
driver.findElement(By.xpath("//input[#value='Go to Express990']")).click();*/
}
#Test (dataProvider = "DP1")
public void test990TP2(String contributorName, String contributorType, String add1, String city, String state, String zip, String amount) throws Exception {
driver.manage().timeouts().implicitlyWait(05, TimeUnit.SECONDS);
//driver.findElement(By.id("btnAddAnother")).click();
// Orgnaization Info
// ERROR: Caught exception [ERROR: Unsupported command [typeKeys | id=EIN | 00-9000004]]
/*driver.findElement(By.linkText("Add Contributions and Grants")).click();
//driver.findElement(By.id("ContributorName")).clear();
driver.findElement(By.id("ContributorName")).sendKeys(contributorName);*/
driver.findElement(By.id("ContributorName")).sendKeys(contributorName);
new Select(driver.findElement(By.id("ContributorType"))).selectByVisibleText(contributorType);
driver.findElement(By.id("Address1")).clear();
driver.findElement(By.id("Address1")).sendKeys(add1);
driver.findElement(By.id("City")).clear();
driver.findElement(By.id("City")).sendKeys(city);
new Select(driver.findElement(By.id("USStates"))).selectByVisibleText(state);
driver.findElement(By.id("ZipCode")).clear();
driver.findElement(By.id("ZipCode")).sendKeys(zip);
driver.findElement(By.xpath("//div[#id='codeRefer']/table/tbody/tr[16]/td[2]/p/label/span")).click();
driver.findElement(By.id("IsContributionNonCashYes")).click();
driver.findElement(By.id("GrantTotalContribution")).clear();
driver.findElement(By.id("GrantTotalContribution")).sendKeys(amount);
driver.findElement(By.id("btnAddAnother")).click();
// Ends
}
You get NullPointerException because your pointer - driver is created in method launchBrowser().
If you want to use pointer driver in both methods, you have to set it as instance variable.
Example:
public class Example {
Webdriver driver; //instance variable - available for all methods in this class
#BeforeTest
public void launchBrowser() {
driver = new Chromedriver();
}
}
Hope it helps!
I am customizing an appfuse 3.0.0 Web Service Only artifact. I have faced this stacktrace when trying to add a checked exception to my Web Service:
[ERROR] Failed to execute goal org.codehaus.enunciate:maven-enunciate-cxf-plugin:1.28:assemble (default) on project ibnInq: Problem assembling the enunciate app. local part cannot be "null" when creating a QName -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.enunciate:maven-enunciate-cxf-plugin:1.28:assemble (default) on project ibnInq: Problem assembling the enunciate app.
Problem assembling the enunciate app.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.executeForkedExecutions(MojoExecutor.java:364)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:198)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Problem assembling the enunciate app.
at org.codehaus.enunciate.AssembleMojo.execute(AssembleMojo.java:75)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 23 more
Caused by: java.lang.IllegalArgumentException: local part cannot be "null" when creating a QName
at javax.xml.namespace.QName.<init>(QName.java:244)
at javax.xml.namespace.QName.<init>(QName.java:188)
at org.codehaus.enunciate.contract.jaxws.WebFault.getParticleQName(WebFault.java:280)
at org.codehaus.enunciate.contract.jaxws.WebMethod.getReferencedNamespaces(WebMethod.java:232)
at org.codehaus.enunciate.contract.jaxws.EndpointInterface.getReferencedNamespaces(EndpointInterface.java:229)
at org.codehaus.enunciate.config.WsdlInfo.getImportedNamespaces(WsdlInfo.java:132)
at org.codehaus.enunciate.modules.xml.WsdlInfoModel.get(WsdlInfoModel.java:50)
at freemarker.core.Dot._getAsTemplateModel(Dot.java:76)
at freemarker.core.Expression.getAsTemplateModel(Expression.java:89)
at freemarker.core.IteratorBlock.accept(IteratorBlock.java:94)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Macro$Context.runMacro(Macro.java:172)
at freemarker.core.Environment.visit(Environment.java:614)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:106)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.visit(Environment.java:310)
at freemarker.core.CompressedBlock.accept(CompressedBlock.java:73)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.visit(Environment.java:310)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:130)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:79)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.visit(Environment.java:310)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:130)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.MixedContent.accept(MixedContent.java:92)
at freemarker.core.Environment.visit(Environment.java:221)
at freemarker.core.Environment.process(Environment.java:199)
at freemarker.template.Template.process(Template.java:259)
at org.codehaus.enunciate.modules.FreemarkerDeploymentModule.processTemplate(FreemarkerDeploymentModule.java:106)
at org.codehaus.enunciate.modules.FreemarkerDeploymentModule.processTemplate(FreemarkerDeploymentModule.java:85)
at org.codehaus.enunciate.modules.FreemarkerDeploymentModule.processTemplate(FreemarkerDeploymentModule.java:70)
at org.codehaus.enunciate.modules.xml.XMLDeploymentModule.doFreemarkerGenerate(XMLDeploymentModule.java:340)
at org.codehaus.enunciate.modules.FreemarkerDeploymentModule.doGenerate(FreemarkerDeploymentModule.java:51)
at org.codehaus.enunciate.modules.BasicDeploymentModule.step(BasicDeploymentModule.java:107)
at org.codehaus.enunciate.apt.EnunciateAnnotationProcessor.process(EnunciateAnnotationProcessor.java:128)
at com.sun.mirror.apt.AnnotationProcessors$CompositeAnnotationProcessor.process(AnnotationProcessors.java:84)
at com.sun.tools.apt.comp.Apt.main(Apt.java:480)
at com.sun.tools.apt.main.AptJavaCompiler.compile(AptJavaCompiler.java:270)
at com.sun.tools.apt.main.Main.compile(Main.java:1127)
at com.sun.tools.apt.main.Main.compile(Main.java:989)
at com.sun.tools.apt.Main.processing(Main.java:113)
at com.sun.tools.apt.Main.process(Main.java:103)
at com.sun.tools.apt.Main.process(Main.java:85)
at org.codehaus.enunciate.main.Enunciate.invokeApt(Enunciate.java:817)
at org.codehaus.enunciate.main.Enunciate.doGenerate(Enunciate.java:401)
at org.codehaus.enunciate.ConfigMojo$MavenSpecificEnunciate.doGenerate(ConfigMojo.java:670)
at org.codehaus.enunciate.main.Enunciate$Stepper.step(Enunciate.java:1799)
at org.codehaus.enunciate.main.Enunciate$Stepper.stepTo(Enunciate.java:1831)
at org.codehaus.enunciate.AssembleMojo.execute(AssembleMojo.java:71)
... 25 more
My enunciate version is 1.28, and I have enabled cxf module in enunciate.xml:
<cxf disabled="false"/>
I have used JAX-WS Spec for exception throwing.
Any help is appreciated.
Finally I have done following workaround to fix my problem: adding #XmlRootElement to my FaultBean.
#XmlRootElement
#XmlAccessorType(XmlAccessType.FIELD)
#XmlType(name = "FaultBean",propOrder = {"errorDescription",errorCode"})
public class FaultBean implements Serializable {
#XmlElement(required = true, nillable = true)
protected String errorDescription;
#XmlElement(required = true, nillable = true)
protected String errorCode;
public FaultBean() {
}
public String getErrorDescription() {
return this.errorDescription;
}
public void setErrorDescription(String var1) {
this.errorDescription = var1;
}
public String getErrorCode() {
return this.errorCode;
}
public void setErrorCode(String var1) {
this.errorCode = var1;
}
}