We have a "mavenized" project with several containers (wso2esb, wso2dss, tomcat) and many components to deploy to them.
We are trying to find a way to deploy the datasource configuration for all our DSS services but I notice it is stored in its own DB (H2).
Do you know if there is any way to declare something like a XML file in order to create the datasource in the DSS in an automated way?
I tried to see the documentation but did not find anything useful for automatic deployment (meaning without using the admin pages).
Yeah, you can use the Carbon data source configuration file datasources.properties, to provide this information. This file should be located at $SERVER_ROOT/repository/conf.
A sample for this configuration file can be found in BPS sources.
After the data sources are defined using this, you can use them using the data source type "carbon data source" from data services.
You can easily deploy artifacts with the hot deployment functionality in WSO2 Servers by simply copying them to a specific directory in the server.
For Data Services Server you can copy the dbs files (in your case with the help of Maven) to $WSO2DSS_HOME/repository/deployment/server/dataservices dirctory. Similarly for BPELs its $WSO2BPS_HOME/repository/deployment/server/bpel
For CAR files created with carbon studio, its $WSO2CARBON_HOME/repository/deployment/server/carbonapps.
For ESB configs, its $WSO2ESB_HOME/repository/deployment/server/synapse-configs.
Related
In some of my application I have to manage environment specific attributes / variables like:
- folder path
- rest api urls
- credentials
- ...
At the moment I'm manually setting variables in the configuration registry of each server. This is quite heavy when you've to deploy a new server because you've to recreate everything manually (I haven't find a way to initialize the repository from an xml file for instance)
I've seen different approaches like
- writing different version of the endpoints, sequences,... and create different car for distribution on each environment
- Using local registry with different entries
- Using governance registry (I've no experience with this)
What is according to you the best approach for this?
Thanks for helping
You can find the best practices guide for WSO2 Enterprise Integrator (ESB, DSS, BPS and MB) at [1]. It also explains how to manage environment specific variables.
[1] https://docs.wso2.com/display/EI611/WSO2+Enterprise+Integrator+Best+Practices
Finally what I've done (and what is working since some weeks now) is for each of my project :
Create a "master" maven project that will contain:
An ESB project
One registry project per environment that contains all environment dependant variables (like hosts, passwords, paths, ...)
One Composite Application project per environment that will package the ESB project with the correct registry values (Note that if you deploy everything in the ESB event the registry project must be considered as a "EnterpriseServiceBus" Role)
Next step will be to integrate everything in jenkins and automatize the building of car with maven.
I'm trying to automate the process of deploying code using github and jenkins job to deploy my Springboot Application on AWS .
I want to know where should I place the application.properties file in case I m deploying a war file on Tomcat and don't want this file to be pushed onto github as it may contain some database credentials , not to be exposed.
Should I put separate application-prod.properties file in Tomcat (AWS) so that my war file will be independent of these properties ?
See my answer here.
In a nutshell, you externalise the properties and then pass one or more profiles that will activate one or more Spring Configuration classes. Each Configuration class will load one or more property file. In your case, if you only have one environment, you can just create a configuration file for one profile.
Then, on your AWS instance, you will deploy the configuration file separately. At runtime, you will need to pass to your Spring Boot application the active profile(s). You can do this by passing the VM argument: -Dspring.profiles.active=[your-profile]
I'm completing the final lectures on an online course that shows how to create from scratch a Spring Boot website with Thymeleaf, Spring Security, Email and Data JPA, how to process credit card payments with Stripe and how to deploy to AWS. You can register your interest here.
how about using spring-cloud-starter-config instead of local properties ?
If using spring-cloud-start-config, all configurations should be loaded from your config-center instead of reading them locally.
Even if you have multiple different environments, spring-cloud-starter-config could handle it with different profiles.
What's more, spring-cloud-starter-config could use local environment variables too.
By the way, the only local resource could be bootstrap.yml if you are using spring-cloud-starter-config.
Wish I can help you!
I am using WSO2 EMM 1.1.0. The documents talk about using a MySQL instead of H2 https://docs.wso2.com/display/EMM110/Setting+up+MySQL. It talks about editing the master-datasource.xml file and updating the WSO2_CARBON_DB, WSO2_EMM_DB and WSO2AM_DB databases. It then gives steps on priming those db's. But the master-datasource.xml file also contains the WSO2_IDENTITY_DB, SOCIAL_CACHE, SOCIAL_CASSANDRA_DB and JAGH2. I expect all of those can be moved to MySQL as well but I don't see the database scripts to set them up. What is the proper procedures to set up a system that uses MySQL instead of H2? Not to mention that the emm database had the database name hard coded into the setup script "USE WSO2EMM_DB" thus nullifying the master-datasource.xml file.
Thanks,
Brian
It is mentioned in this documentation[1] under the topic 'How to migrate from H2 to MySQL'
[1] - https://docs.wso2.com/display/EMM110/Upgrading+from+a+Previous+Release
You need to configure WSO2EMM_DB, WSO2AM_DB and WSO2CARBON_DB and WSO2IDENTITY_DB if you are going ahead with a larger deployment. H2 is setup just for make the out of the box experience better. You can create those DBs, Configure master_datasources.xml properly for all above DBs. And then run the server with the flag -Dsetup. It will get the configurations done automatically.
If it fails, you can also go to SERVER_HOME/dbscripts folder and find all the scripts for all above databases. Run them separately and run the server in the usual way which mentioned in our documentation.
What is the best / flexible WSO2 upgrade strategy?
Because now we are upgrading WSO2 DSS 3.0.1 to DSS 3.1.1, therefore there is some difficult changes in dbs file one by one
wso2dss-3.0.1
<data name="BASE_PERSON_DataService" serviceNamespace=
"http://company.mn/base/BASE_PERSON">
wso2dss-3.1.1
<data description="multiple services per each table" enableBatchRequests="false"
enableBoxcarring="false" name="BASE_PERSON_DataService"
serviceNamespace="http://company.mn/base/BASE_PERSON" serviceStatus="active">
What is the easy way, we have many data services (dbs files)?
Regards,
Eba
As far as I know, there is usually no standard migration tool or procedure available. Check that the newer version uses a compliant schema for the wso2 registry database and so on; maybe it's the same or you just need to create new additional tables. Sometimes you find things like migration scripts in the dbscripts folder. You should also check for differences in newer xml configuration files, and adjust your older custom configuration to the new format (usually few or no changes could be required). As far as the artifacts are concerned, I never heard of any way to convert them. If there are many of them, I would probably try some script and regex to batch modify and adjust them to the new format.
These are the steps you should follow if you are upgrading
Step 1 - Deploy artifacts {dbs/datasource/drivers}
Cappy the deployed data services from current installation to new installation by copying repository/deployment/server folder.(all dbs files are backword compatible so what ever worked in WSO2 DSS 3.0.1, should work on DSS 3.1.1) Also note you need to copy data source configuration properties if you have created carbon data sources therefore copy master-datasources.xml from repository/conf/datasources to the new installation.
Also Copy all the content of repository/component/lib to the new installation to ensure the the jdbc drivers are properly installed.
Step 2- Change the configuration files
Apply the same changes you have done to configurations files inside OLD_DSS/repository/conf to NEW_DSS/repository/conf (if you have done any such to any configuration files)
Note - If you have done registry mounting make sure you apply to the new installation as done before by changing relevant configuration files such as
carbon.xml,axis2.xml,user-mgt.xml,mgt-transports.xml
I am using a config file to store passwords / keys for DB and connection to other services like AWS.
I am using Travis CI for build, and running my tests, and then deploying it to live server.
I would like to encrypt the variables in my config file and only the application should be able to read it. This is the criteria -
1) The application should be able to decrypt it in multiple environments, from the build server to multiple deployment servers.
2) The password used to decrypt the config file is not avalaible to the developers.
The project is in clojure, are there any leiningen plugins / features that will aid in this ?
Thanks,
Murtaza
I think the best solution to what you are trying to do is simply to remove the config files from the application source tree. Save the config files to the build and deployment severs and don't give the developers direct access to those.
The chief problem with what you are trying to do is that you are trying to make it impossible for your developers to have access to something but at the same make it possible for something that the developers have complete control over, the application code, to have access to the same thing. That is, for the application to be able to decrypt the configs, the developers have to have access to the decyption keys.
You could try making it only possible to decrypt your configs through tools available only on the build and deployment servers, (ie a Leiningen plugin,) but it would be simpler to just store the config files separately.