I have setup the cluster for WSO2-IS (2 instances on different machines) based on the information provided here - https://docs.wso2.com/display/CLUSTER44x/WSO2+Clustering+and+Deployment+Guide
Setup DB with a user store, shared registry, 2 local registries
Copied the DB driver jar to component lib
Updated the master-datasource.xml
Updated the registry.xml (made sure the master is read-only false and worker is read-only true)
Updated the AXIS2.xml and used WKA for membership scheme
Performed other changes as suggested in the link
Started the master with -Dsetup option and the worker without -Dsetup option.
Verified that the governance folder is shown as a symlink
I can see the interaction between both the nodes, there are Hazelcast messages related to node joining when the worker is started.
User created in 1 is able to login to the other instance, service provider are also automatically available when viewed through UI.
The problem is that when I create a secondary user store (JDBC) in the first node and goto the list in the second node - the secondary user store is not present and I cannot view the users in the user list too.
Am I missing something or is it the way the cluster is supposed to perform i.e. secondary user stores have to be shared in some other way?
Thanks,
Vikas
Secondary user store configurations are not synced between two nodes by default. Once you create a secondary user store from UI, it will create a file in following location.
[WSO2_IS]/repository/deployment/server/userstores/
These configuration file need to copy by manually or have to use some synchronization mechanism to copy file to other node. since this is not a frequent task better to copy this file.
Fore more information
https://docs.wso2.com/display/IS500/Configuring+Secondary+User+Stores
Related
There are 2 properties within configuration files I am confused with:
The property yarn.nodemanager.remote-app-log-dir in yarn-site.xml:
a.) This property controls, where the logs of map/reduce tasks will be logged?
b.) This is the responsibility of Node Manager (NM)?
The property mapreduce.jobhistory.done-dir from mapred-site.xml:
a.) Job related files like configurations etc. are stored in this location?
b.) This is the responsibility of Application Master (AM)?
Does the History Server (HS) combines both of these information and shows a consolidated information in UI?
Assuming you have enabled log-aggregation,
1.a. This is the log-aggregation dir, usually HDFS where NMs aggregate container-logs to.
1.b. Yes.
2.a. Yes.
2.b. No. MR JobHistory Server will do that, by deleting JobSummary file and mv other files to ${mapreduce.jobhistory.done-dir} from ${mapreduce.jobhistory.intermediate-done-dir}.
3. Yes. MR JobHistory Server Web, includes job info(from ${mapreduce.jobhistory.done-dir}) and container logs(from ${yarn.nodemanager.remote-app-log-dir}).
In Node-Red, I'm using some Amazon Web Services nodes (from module node-red-node-aws), and I would like to read some configuration settings from a file (e.g. the access key ID & the secret key for the S3 nodes), but I can't find a way to set everything up dynamically, as this configuration has to be made in a config node, which can't be used in a flow.
Is there a way to do this in Node-Red?
Thanks!
Unless a node implementation specifically allows for dynamic configuration, this is not something that Node-RED does generically.
One approach I have seen is to have a flow update itself using the admin REST API into the runtime - see https://nodered.org/docs/api/admin/methods/post/flows/
That requires you to first GET the current flow configuration, modify the flow definition with the desired values and then post it back.
That approach is not suitable in all cases; the config node still only has a single active configuration.
Another approach, if the configuration is statically held in a file, is to insert them into your flow configuration before starting Node-RED - ie, have a place-holding config node configuration in the flow that you insert the credentials into.
Finally, you can use environment variables: if you set the configuration node's property to be something like $(MY_AWS_CREDS), then the runtime will substitute that environment variable on start-up.
You can update your package.json start script to start Node-RED with your desired credentials as environment variables:
"scripts": {
"start": "AWS_SECRET_ACCESS_KEY=<SECRET_KEY> AWS_ACCESS_KEY_ID=<KEY_ID> ./node_modules/.bin/node-red -s ./settings.js"
}
This worked perfect for me when using the node-red-contrib-aws-dynamodbnode. Just leave the credentials in the node blank and they get picked up from your environment variables.
I have shared a bunch of AMIs from an AWS account to another.
I used this EC2conn1.modify_image_attribute(AMI_id, operation='add', attribute='launchPermission', user_ids=[second_aws_account_id]) to do it.
But, by only adding launch permission for the 2nd account, I can launch an instance but I cannot copy the shared AMI to another region [in the 2nd account].
When I tick the checkbox to "create volume" from the UI of the 1st account, I can copy the shared AMI from the 2nd:
I can modify the launch permissions using the modify_image_attribute function from boto.
In the documentation says, attribute (string) – The attribute you wish to change but I understand that it can only change the launch permissions and add an account.
Yet, the get_image_attribute has 3 options Valid choices are: * launchPermission * productCodes * blockDeviceMapping.
So, is there a way to programmatically change it from the API along with the launch permissions or, it has not been implemented yet??
The console uses the API so there's almost nothing you can do in the console that you can't to using the API.
Remember that an AMI is just a configuration entity -- basic launch configuration, linked to (not containing) one or more backing snapshots, which are technically separate entities.
The console is almost certainly making an additional API request the ModifySnapshotAttribute API when it offers to optionally "add Create Volume permissions to the following associated snapshot."
See also http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-modifying-snapshot-permissions.html
Presumably, copying a snapshot to another region relies on the same "Create Volume" permission (indeed, you'll see that a copied snapshot has a fake source volume ID, presumably an artifact of the copying process).
Based on the accepted answer, this is the code I wrote for anyone interested.
# Add copy permission to the image's snapshot
# Find the snapshot of the specific AMI
image_object = EC2conn.get_image(AMI_id)
# Grab the block device mapping dynamically
ami_devices = []
for key in image_object.block_device_mapping.iterkeys():
# print key #debug
ami_devices.append(key)
# print ami_devices #debug
for ami_device in ami_devices:
snap_id = image_object.block_device_mapping[ami_device].snapshot_id
# Add permission
EC2conn.modify_snapshot_attribute(snap_id, attribute='createVolumePermission', operation='add', user_ids=second_aws_account_id)
print "{0} [{1}] Permission added to snapshot".format(AMI_name,snap_id)
all. I have some confusion about the registry.
1.the remote registry mount is done like this
in [1]
but it is done like
in [2] with the port NO. and /registry. are they the same??
2.I gonna install apim and IS and GREG, apim and IS should share their infomation, so that when a new tenant is registered in apim, IS should be able to use this new tenant too. My question is whether both the config and governance of both server should be configurated to GREG? because I don't know which (config or governance) folder contains the user resources?
[1] http://docs.wso2.org/display/CLUSTER420/Clustering+API+Manager
[2] http://docs.wso2.org/display/Governance453/Governance+Partition+in+a+Remote+Registry
To answer your second question, If you wanted to share same tenant information then you need to share user store and the realm database. One way of doing that is, pointing to same ldap from all nodes and refering central database for realm.
By default, IS has embedded ldap and if you do not have external/central ldap or user database that embedded ldap of IS can be use as the central user store. To do so, copy element in <IS_HOME>/repository/conf/user-mgt.xml and replace the element of user-mgt.xml in other nodes. You need to change ConnectionURL appropriately with hostname and port. If you have started IS without any port offset and all nodes run on same server, you can use
<Property name="ConnectionURL">ldap://localhost:10389</Property>
or, if the IS run on some other machine having the ip 192.168.33.66 and start with portoffset 1, connection url would be like follows,
<Property name="ConnectionURL">ldap://192.168.33.66:10380</Property>
And you need to share registry database also. To do so create a central database and create a datasource with a name like WSO2_REALM_DB and refer to jndi name of that datasource in user-mgt xml by changing the property <Property name="dataSource">jdbc/WSO2_REALM_DB</Property>. From this post you can find steps to create database and configuring datasource for that db.
Its not clear on your first question but regarding registry mounting in general,
Local partition is used to store resources specific to a node. And its
usually do not shared among nodes.(ie. APIM node 1 may have local registry 1 and APIM node 2 have local registry 2 while IS node 1 have local registry 3)
Config partition is used to store resources for specific product and its usually shared among nodes in same cluster. (ie. APIM node 1 may have config registry 1 and APIM node 2 also points to config registry 1 while IS node 1 have config registry 2)
Governance partition is used to share resources among different products.(ie. APIM node 1, APIM node 2 and IS node 1 all points to same gov registry 1)
You can more detailed explanation in this article.
I'm rushing (never a good thing) to get Sync Framework up and running for a "offline support" deadline on my project. We have a SQL Express 2008 instance on our server and then will deploy SQLCE to the clients. Clients will only sync with server, no peer-to-peer.
So far I have the following working:
Server schema setup
Scope created and tested
Server provisioned
Client provisioned w/ table creation
I've been very impressed with the relative simplicity of all of this. Then I realized the following:
Schema created through client provisioning to SQLCE does not setup default values for uniqueidentifier types.
FK constraints are not created on client
Here is the code that is being used to create the client schema (pulled from an example I found somewhere online)
static void Provision()
{
SqlConnection serverConn = new SqlConnection(
"Data Source=xxxxx, xxxx; Database=xxxxxx; " +
"Integrated Security=False; Password=xxxxxx; User ID=xxxxx;");
// create a connection to the SyncCompactDB database
SqlCeConnection clientConn = new SqlCeConnection(
#"Data Source='C:\SyncSQLServerAndSQLCompact\xxxxx.sdf'");
// get the description of the scope from the SyncDB server database
DbSyncScopeDescription scopeDesc = SqlSyncDescriptionBuilder.GetDescriptionForScope(
ScopeNames.Main, serverConn);
// create CE provisioning object based on the scope
SqlCeSyncScopeProvisioning clientProvision = new SqlCeSyncScopeProvisioning(clientConn, scopeDesc);
clientProvision.SetCreateTableDefault(DbSyncCreationOption.CreateOrUseExisting);
// starts the provisioning process
clientProvision.Apply();
}
When Sync Framework creates the schema on the client I need to make the additional changes listed earlier (default values, constraints, etc.).
This is where I'm getting confused (and frustrated):
I came across a code example that shows a SqlCeClientSyncProvider that has a CreatingSchema event. This code example actually shows setting the RowGuid property on a column which is EXACTLY what I need to do. However, what is a SqlCeClientSyncProvider?! This whole time (4 days now) I've been working with SqlCeSyncProvider in my sync code. So there is a SqlCeSyncProvider and a SqlCeClientSyncProvider?
The documentation on MSDN is not very good in explaining what either of these.
I've further confused whether I should make schema changes at provision time or at sync time?
How would you all suggest that I make schema changes to the client CE schema during provisioning?
SqlCeSyncProvider and SqlCeClientSyncProvider are different.
The latter is what is commonly referred to as the offline provider and this is the provider used by the Local Database Cache project item in Visual Studio. This provider works with the DbServerSyncProvider and SyncAgent and is used in hub-spoke topologies.
The one you're using is referred to as a collaboration provider or peer-to-peer provider (which also works in a hub-spoke scenario). SqlCeSyncProvider works with SqlSyncProvider and SyncOrchestrator and has no corresponding Visual Studio tooling support.
both providers requires provisioning the participating databases.
The two types of providers provisions the sync objects required to track and apply changes differently. The SchemaCreated event applies to the offline provider only. This get's fired the first time a sync is initiated and when the framework detects that the client database has not been provisioned (create user tables and the corresponding sync framework objects).
the scope provisioning used by the other provider dont apply constraints other than the PK. so you will have to do a post-provisioning step to apply the defaults and constraints yourself outside of the framework.
While researching solutions without using SyncAgent I found that the following would also work (in addition to my commented solution above):
Provision the client and let the framework create the client [user] schema. Now you have your tables.
Deprovision - this removes the restrictions on editing the tables/columns
Make your changes (in my case setting up Is RowGuid on PK columns and adding FK constraints) - this actually required me to drop and add a column as you can't change the "Is RowGuid" property an existing columns
Provision again using DbSyncCreationOption.CreateOrUseExisting