Errors when upgrade WSO2 IS as KM and APIs are not migrated - wso2

I tried to upgrade WSO2 IS as KM from 5.6.0 to 5.7.0 and API Manager from 2.5.0 to 2.6.0 in corresponding with instructions:
https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release+when+WSO2+IS+is+the+Key+Manager
https://docs.wso2.com/display/IS570/Upgrading+from+the+Previous+Release
https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release
In instructions there is no mention that i need to import SQl script again to DB apimgt from /dbscripts/apimgt/mysql.sql, cause 5.7.0 IS as KM has more tables in that DB than 5.6.0 version.
During upgrade of IS i have errors in logs:
ERROR {org.wso2.carbon.is.migration.service.SchemaMigrator} - Error occurred while executing SQL script for migrating database
java.lang.Exception: Error occurred while executing : CREATE INDEX IDX_RID ON IDN_UMA_RESOURCE (RESOURCE_ID)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate key name 'IDX_RID'
ERROR {org.wso2.carbon.is.migration.service.SchemaMigrator} - Error occurred while executing SQL script for migrating database
java.lang.Exception: Error occurred while executing : CREATE INDEX IDX_SP_TEMPLATE ON SP_TEMPLATE (TENANT_ID, NAME)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate key name 'IDX_SP_TEMPLATE'
ERROR {org.wso2.carbon.is.migration.service.SchemaMigrator} - Error occurred while executing SQL script for migrating database
java.lang.Exception: Error occurred while executing : ALTER TABLE CM_PURPOSE ADD COLUMN PURPOSE_GROUP VARCHAR(255) NOT NULL, ADD COLUMN GROUP_TYPE VARCHAR(255) NOT NULL, DROP KEY NAME, ADD UNIQUE KEY (NAME, TENANT_ID, PURPOSE_GROUP, GROUP_TYPE)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate column name 'PURPOSE_GROUP'
INFO {org.wso2.carbon.is.migration.service.v570.migrator.OAuthDataMigrator} - WSO2 Product Migration Service Task : Migration starting on OAuth2 access token table.
java.lang.OutOfMemoryError: GC overhead limit exceeded
Dumping heap to /usr/lib64/wso2/wso2is-km-5.7.0/repository/logs/heap-dump.hprof ...
Unable to create /usr/lib64/wso2/wso2is-km-5.7.0/repository/logs/heap-dump.hprof: File exists
ERROR {org.wso2.carbon.is.migration.MigrationClientImpl} - Migration process was stopped.
java.lang.OutOfMemoryError: GC overhead limit exceeded
Despite on errors IS is launched, after i launched API-M without errors, but my APIs are not migrated.
I use MariaDB 5.5.
What is the problem with IS and why APIs dont migrated?

"So i shouldn't do Steps from 2.1 to 2.4, yes? But what with errors that i have during upgrade of IS?"
That means, didn't you do the steps 2.1 to 2.4 in https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release#c130adc044364015ae336f584909e3ac?
If you have already followed the steps mentioned in https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release+when+WSO2+IS+is+the+Key+Manager#250, then you need to skip only the 2.4 step in https://docs.wso2.com/display/AM260/Upgrading+from+the+Previous+Release#c130adc044364015ae336f584909e3ac.
Also please make sure that you have given the versions correctly in the migration-config.yaml file as below.
migrationEnable: "true"
currentVersion: "5.6.0"
migrateVersion: "5.7.0"
Note: You do not need to create tables manually.

Related

wso2is-km upgrade from 5.7 to 5.10

I am upgrading wso2is-km from 5.7 to 5.10. I am following document.
https://is.docs.wso2.com/en/latest/setup/migrating-to-5100/
But I am getting below error. Can you please provide link for db script which is required for migration?
Caused by: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO IDN_OIDC_SCOPE_CLAIM_MAPPING (SCOPE_ID, EXTERNAL_CLAIM_ID) SELECT 6,IDN_CLAIM.ID FROM IDN_CL
AIM LEFT JOIN IDN_CLAIM_DIALECT ON IDN_CLAIM_DIALECT.ID = IDN_CLAIM.DIALECT_ID WHERE CLAIM_URI='sub' AND IDN_CLAIM_DIALECT.DIALECT_URI='http://wso2.org/oidc/cla
im' AND IDN_CLAIM_DIALECT.TENANT_ID=-1234 was aborted: ERROR: insert or update on table "idn_oidc_scope_claim_mapping" violates foreign key constraint "idn_oidc
_scope_claim_mapping_scope_id_fkey"
Detail: Key (scope_id)=(6) is not present in table "idn_oidc_scope". Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:148)
Caused by: org.postgresql.util.PSQLException: ERROR: column "uuid" does not exist
Hint: Perhaps you meant to reference the column "sp_app.id".
Position: 310
Caused by: org.postgresql.util.PSQLException: ERROR: column "image_url" does not exist
Position: 302
Detailed error:
Caused by: java.lang.Exception: Error occurred while executing : DROP FUNCTION IF EXISTS OIDC_SCOPE_DATA_MIGRATE_FUNCTION
at org.wso2.carbon.is.migration.service.SchemaMigrator.executeSQL(SchemaMigrator.java:303)
at org.wso2.carbon.is.migration.service.SchemaMigrator.executeSQLScript(SchemaMigrator.java:234)
... 31 more
Caused by: org.postgresql.util.PSQLException: ERROR: syntax error at end of input
Position: 59
When you run the provided migration client in the doc, it executes the relevant DB scripts and you don't have to execute any DB scripts manually again.
Can you please recheck whether you followed the steps correctly in the doc?
Please note the following in the doc for PostgreSQL.
"During the migration, "uuid-ossp" extension is created in the database. In order to create this extension, the database user should have the 'Superuser' permission. If the user is not already a superuser, assign the permission before starting the migration.
ALTER USER <user> WITH SUPERUSER;"

How to fix `user must specify LSN` when using AWS DMS for Postgres RDS

I'm trying to migrate and synchronize a PostgreSQL database using AWS DMS and I'm getting the following error.
Last Error Task error notification received from subtask 0, thread 0
[reptask/replicationtask.c:2673] [1020101] When working with Configured Slotname, user must
specify LSN; Error executing source loop; Stream component failed at subtask 0, component
st_0_D27UO7SI6SIKOSZ4V6RH4PPTZQ ; Stream component 'st_0_D27UO7SI6SIKOSZ4V6RH4PPTZQ'
terminated [reptask/replicationtask.c:2680] [1020101] Stop Reason FATAL_ERROR Error Level FATAL
I already created a replication slot and configured its name in the source endpoint.
DMS Engine version: 3.1.4
Does anyone knows anything that could help me?
Luan -
I experienced the same issue - I was trying to replicate data from Postgres to an S3 bucket.I would check two things - your version of Postgres and the DMS version being used.
I downgraded my RDS postgres version to 9.6 and my DMS version to 2.4.5 to get replication working.
You can find more details here -
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html
I wanted to try the newer versions of DMS (3.1.4 and 3.3.0[beta]) as it has parquet support but I have gotten the same errors you mentioned above.
Hope this helps.
It appears AWS expects you to use the pglogical extension rather than test_decoding. You have to:
add pglogical to shared_preload_libraries in parameter options
reboot
CREATE EXTENSION pglogical;
On dms 3.4.2 and postgres 12.3 without the slotName= setting DMS created the slot for itself. Also make sure you exclude the pglogical schema from the migration task as it has unsupported data types.
P.S. When DMS hits resource limits it silently fails. After resolving the LSN errors, I continued to get failures of the type Last Error Task 'psql2es' was suspended due to 6 successive unexpected failures Stop Reason FATAL_ERROR Error Level FATAL without any errors in the logs. I resolved this issue using the Advanced task settings > Full load tuning settings and tuning the parameters downward.

WSO2CEP 4.2.0 error: A mandatory attribute null does not exist

We are using WSO2CEP version 4.2.0. We are connecting to a MySQL database (version 5.6.34-1 community edition from Oracle) on the back-end with mysql-connector-java-5.1.40.jar. We have set up several connections in the master-datasources.xml, and receive "Connection is healthy" for all connections when testing them in Datasources. When we attempt to use an event publisher that accesses the referenced databases an error appears:
[2017-01-24 17:11:22,178] ERROR {org.wso2.carbon.event.publisher.admin.EventPublisherAdminService} - org.wso2.carbon.event.output.adapter.core.exception.OutputEventAdapterRuntimeException: A mandatory attribute null does not exist
org.wso2.carbon.event.publisher.core.exception.EventPublisherConfigurationException: org.wso2.carbon.event.output.adapter.core.exception.OutputEventAdapterRuntimeException: A mandatory attribute null does not exist
at org.wso2.carbon.event.publisher.core.EventPublisherDeployer.processDeployment(EventPublisherDeployer.java:227)
at org.wso2.carbon.event.publisher.core.EventPublisherDeployer.executeManualDeployment(EventPublisherDeployer.java:249)
.........several lines after this ...............
Our team are kind of at a loss, we have tried things like giving blanket permissions including DDL to the database user, trying an old database that "used to work", and changing out versions of the mysql-connector-java jar.
We found that we had a configuration problem - invalid XML in output-event-adapters.xml that was causing the error. Bad XML fixed, error gone.
WSO2, please consider addressing error verbosity in your products. The error that was being logged provided no indication that invalid XML might be the cause, and we wasted several hours troubleshooting the problem as a result. We have experienced similar error-verbosity-related issues in other WSO2 products. A simple "could not parse XML" with the file name would have literally saved us several hours this time.

wso2 is and esb UTF8

Error installing IS
I am installing wso2 is MySql.
MySQL is set to UTF-8. These are the errors:
1 :
CREATE INDEX REG_PATH_IND_BY_PATH_VALUE USING HASH ON REG_PATH(REG_PATH_VALUE, REG_TENANT_ID);
Error
MySQL Database Error: Table '#sql-197b_412' uses an extension that doesn't exist in this MySQL version
2 :
CREATE TABLE IDN_OAUTH2_AUTHORIZATION_CODE (
AUTHORIZATION_CODE VARCHAR(255),
CONSUMER_KEY VARCHAR(255),
CALLBACK_URL VARCHAR(1024),
SCOPE VARCHAR(2048),
AUTHZ_USER VARCHAR(512),
TIME_CREATED TIMESTAMP,
VALIDITY_PERIOD BIGINT,
PRIMARY KEY (AUTHORIZATION_CODE),
FOREIGN KEY (CONSUMER_KEY) REFERENCES IDN_OAUTH_CONSUMER_APPS(CONSUMER_KEY) ON DELETE CASCADE
)TABLESPACE tb_regdb engine ndb storage disk;
Error:
MySQL Database Error: Got error 851 'Maximum 8052 bytes of FIXED columns supported, use varchar or COLUMN_FORMAT DYNAMIC instead' from NDBCLUSTER
wso2 can be installed on a database in UTF8?
regards
I assume that you are using a MySQL cluster with NDB engine. Therefore you need to run the MySQL script which can be found inside the <IS_HOME>/dbscripts/mysql_cluster.sql
In addition Identity Server also has identity scripts under <IS_HOME>/dbscripts/identity/ and IS_HOME>/dbscripts/identity/application-mgt/. It have not shipped the identity script for mysql_cluster with NDB engine.
But you can modify the existing mysql.sql script under the <IS_HOME>/dbscripts/identity/ and IS_HOME>/dbscripts/identity/application-mgt/ and make them work for MySQL cluster. In dbscripts/identity/mysql.sql and dbscripts/identity/application-mgt/mysql.sql ENGINE INNODB clauses can be replaced with ENGINE NDB. The value of configuration option MaxNoOfTriggers had to be increased to avoid 4239 NDB errors.

INFORMATICA LOADING ERROR

I am newbie to informatica.
I am using INFORMATICA 9.1.0 and oracle 11g as source and target database.
I tried to create one table in target database and tried to load data from source to target.
Table is getting created in target database. and I created mapping and workflow which is valid and i start work flow but it gave me following error.
Message Code RR_4036
Message
Error connecting to database [ Arun
ORA-00900: invalid SQL statement
Database driver error...
Function Name : executeDirect
SQL Stmt : Arun
Oracle Fatal Error
Database driver error...
Function Name : ExecuteDirect
Oracle Fatal Error
].
please help me with good solutions.
I got solution for this.
Previously while creating remote connection in Relational Connection Editor for a session, In code page option i chose "UTF-8 encoding of unicode". Now i changed to "ms windows latin 1 (ansi) superset of latin1" and I restarted the workflow which is succeeded.
The following video link shows how to create relational connection for a session.
http://www.youtube.com/watch?v=oM2d-IHfRUw