Using testng factory and Data provider annotations we have set of test cases that needs to be executed in parallel using selenium grid. As of now we have say for example three AWS instances with the IPs required. For now, we are able to run set of cases in parallel in single AWS instance. i.e able to run set of 30 cases in parallel in single instance.
<?xml version="1.0"?>
<suite name="reg_tests" parallel="tests" thread-count="90">
<test name="sanity_01" parallel="instances" thread-count="30">
<classes>
<class name="com.X.Y"/>
</classes>
</test>
<test name="sanity_02" parallel="instances" thread-count="30">
<classes>
<class name="com.X.Y1"/>
</classes>
</test>
<test name="sanity_03" parallel="instances" thread-count="30">
<classes>
<class name="com.X.Y2"/>
</classes>
</test>
</suite>
Have properties file where we get the IP of the machine where we want to run which is obviously pointing to single AWS machine.
WebDriver driver = new RemoteWebDriver(new URL(url),
desiredCapabilities);
url - IP of the AWS machine.
So, the above code directs to run in single machine. So, now is there a way to ask the selenium grid to run in all three Grid machines which are already set up for executing test cases. Since the thread maintenance are managed internally, can this be done?
Yes of course. But it is depend on the hub.
The node must able to register to the hub successfully.
Note that your selenium code target to hub only not to node and hub then decide to whom they need to redirect as per the capability set by you.
For an example if your sanity_01 having capabilities of chrome and when you target to the hub, the hub understood the capablity and redirect your code to node machine/ec2 which is register for chrome
baseURL = "http://demo.xyz.com/test/";
hubURL = "http://192.168.43.223:4444/wd/hub";
DesiredCapabilities capability = DesiredCapabilities.chrome();
capability.setBrowserName("chrome");
capability.setPlatform(Platform.WIN10);
driver = new RemoteWebDriver(new URL(hubURL), capability);
In above code the hub is hubURL = "http://192.168.43.223:4444/wd/hub"; and as capability is set to chrome, it will send it to chrome node.
if 2 chrome node added to hub then it will redirect to anyone as per the node availability
Related
I have 3 OrientDB (2.2.7) nodes setup on AWS. They are running in distributed mode.
Whenever I connect to the server on port 2424, the connection locks up in pyorient.
I'm aware of some issues in regards to running OrientDB in distributed mode as per this question:
Creating a database in Orientdb in distributed mode
In order to avoid any issues, I'm running permanent instances as suggested by the documentation.
I also configued the EC2 instances to be "c3.4xlarge" instances as suggested by the hazelcast EC2 whitepaper. (Amazon_EC2_Deployment_Guide_v0.3_web.pdf)
I had my hazelcast.xml configured to use tcp-ip and aws discovery strategies and both delivered the same results. The servers can be seen connecting to one another via hazelcast to the discovery is working fine.
I have the following policies attached to my user.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stm7747196888759",
"Action": [
"ec2:DescribeInstances"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Each have hazelcast.xml configured like so:
<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.7.xsd"
xmlns="http://www.hazelcast.com/schema/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<group>
<name>orientdb</name>
<password>xxxxxxxxx</password>
</group>
<properties>
<property name="hazelcast.local.localAddress">{{LOCAL_IP}}</property>
<property name="hazelcast.icmp.enabled">true</property>
</properties>
<network>
<public-address>{{PUBLIC_IP}}</public-address>
<port auto-increment="true">2434</port>
<join>
<multicast enabled="false">
<multicast-group>235.1.1.1</multicast-group>
<multicast-port>2434</multicast-port>
</multicast>
<tcp-ip enabled="true">
<member>57.xx.xx.165</member>
<member>57.xx.xx.236</member>
<member>57.xx.xx.133</member>
</tcp-ip>
<aws enabled="false">
<access-key>xxxx</access-key>
<secret-key>xxxx</secret-key>
<host-header>ec2.amazonaws.com</host-header>
<region>eu-west-1</region>
</aws>
</join>
<interfaces enabled="false">
<interface>{{LOCAL_IP}}</interface>
</interfaces>
</network>
<executor-service>
<pool-size>16</pool-size>
</executor-service>
</hazelcast>
As can be seen from my hazelcast.xml, I also tried upgrading hazelcast to version 3.7. It doesn't matter which version of hazelcast I use, the results are the same.
As soon as I connect to the server, the connection locks up. The server still works fine over port 2480. You can still use the front-end in the browser but can't open a connection via pyorient.
We have a large DB and collect around 2.5 million vertices data each month with about 5 million edges. It's vital for us to run in distributed mode because a single server won't be able to scale beyond that capacity. As things are at the moment, it seem like OrientDB has the capability to run as a distributed database but that functionality doesn't seem to work.
We were running the dockers but switched to the binaries in order to upgrade to hazelcast 3.7.
Has anyone been able to get OrientDB working in production as distributed and what are we missing?
This does not seem to be an issue with Hazelcast or AWS.
There was 2 issues with my setup.
The first issue has to do with OrientDB not refreshing of replacing my distributed-config.json with settings from
default-distributed-db-config.json. The result was that every node, that have ever connected to my DB, was appended to that file and none of my default-distributed-db-config.json settings were reflecting in that config.
I added a start-up, script to delete that distributed-config.json every time my server starts up in order to refresh the list of nodes and update my settings.
The second issue has to do with Pyorient. Pyorient has a bug in that it can't parse the messages returned from OrientDB when in distributed mode. This causes the connection to go into an infinite loop.
There is currently a development branch on pyorient that implements the missing binary serialiser (OrientSerialization.Binary). I have another branch that has some fixes merged into it.
Install it with:
pip install https://github.com/anber500/pyorient/tarball/17f5e42e83859a661c6483f7fa812226194694dd#egg=pyorient
Set your serialiser as follows:
client = pyorient.OrientDB("localhost", 2424, serialization_type=pyorient.OrientSerialization.Binary)
You will also need an updated version of pyorient_native. The first release had a memory leak so use the version from the master branch:
pip install https://github.com/nikulukani/pyorient_native/tarball/master#egg=pyorient_native
This works perfectly on AWS in distributed mode and is much faster than the CSV serializer.
Hope it helps.
You are using a ec2 public ip address and not the ec2 private ip address. Public ip addresss often start with 57 or 54. Private ip addresses often with 10.
Is the Azure Service Fabric Reverse Proxy available in an on-premises cluster? If so, how can I enable it for an existing cluster?
The Service Fabric Reverse Proxy is described here. It allows clients external to the cluster to access application services by name with a special URL, without needing to know the exact host:port on which an instance of the service is running (which may change as services are automatically moved around).
By default the Service Fabric Reverse Proxy does not appear to be enabled for my on-prem cluster with two instances of a stateless service. I tried using the documented port 19008 but could not reach the service using the recommended URI syntax.
To wit, this works:
http://fqdn:20001/api/odata/v1/$metadata
but this does not:
http://fqdn:19008/MyApp/MyService/api/odata/v1/$metadata
In the NodeTypes section of the ClusterConfig JSON used to set up my on-prem cluster, there is a property "httpGatewayEndpointPort": "19080", but that port does not appear to work as a reverse proxy (it is the Service Fabric Explorer web-app endpoint). I am guessing that the needed configuration is specified somehow in the cluster config JSON. There are instructions in the referenced article that explain how to configure the reverse proxy in the cloud, but not on-premises.
What I am looking for are instructions on how to set up the Service Fabric reverse proxy in an on-premises multi-machine cluster or dev cluster.
Yes, the reverse proxy is available on-premises.
To get it working for an existing cluster, it must be configured and enabled in the cluster config XML and then the new config must be deployed, as described below.
For a new cluster, set it up in the cluster config JSON before creating the cluster, as described by #Scott Weldon.
#Senj provided the clue (thanks!) that led me to the answer. I had recently updated my Service Fabric bits on my dev box to 5.1.163.9590. When I looked in C:\SfDevCluster\Data\FabricHostSettings.xml, I noticed the following:
<Section Name="FabricNode">
...
<Parameter Name="NodeVersion" Value="5.1.163.9590:1.0:0" />
...
<Parameter Name="HttpApplicationGatewayListenAddress" Value="19081" />
<Parameter Name="HttpApplicationGatewayProtocol" Value="http" />
...
</Section>
Interesting! With the dev cluster fired up, I browsed to:
http://localhost:19081/MyApp/MyService/api/odata/v1/$metadata
and voila! My API returned the expected data. So #Senj was correct that it has to do with the HttpApplicationGateway settings. I am guessing that in the latest SDK version it is pre-configured and enabled by default. (What threw me off is all the docs refer to port 19008, but the actual configured port was 19081!)
In order to get the reverse proxy to work on the 'real' multi-machine (VM) cluster, I did the following (Note: I don't think upgrading the cluster codepackage was necessary, but since I had nothing in my image store for the cluster upgrade, and the cluster upgrade process requires a code package, I used the latest version):
Copy the existing cluster manifest (from the Manifest tab in Service Fabric Explorer), paste into a new XML file, bump the version number and modify as follows:
To the NodeType Endpoints section, add:
<NodeTypes>
<NodeType Name="NodeType0">
<Endpoints>
<HttpApplicationGatewayEndpoint Port="19081" Protocol="http" />
...
</Endpoints>
</NodeType>
</NodeTypes>
and under <FabricSettings>, add the following section:
<Section Name="ApplicationGateway/Http">
<Parameter Name="IsEnabled" Value="true" />
</Section>
Using Service Fabric PowerShell commands:
Copy the new cluster config (the previously copied manifest.xml) to the fabric image store
Register the new cluster config
Copy the Service Fabric Runtime cluster codepackage (available here - see the release notes for the link to the MSI) to the image store
Register the cluster codepackage
Start and complete cluster upgrade (I used unmonitored manual mode, which does one VM at a time and requires a manual Resume command after each node is complete)
After the cluster upgrade was complete, I was able to query my service API using the reverse proxy endpoint and appname/servicename URL syntax:
http://fqdn:19081/MyApp/MyService/api/odata/v1/$metadata
I enabled this in the standalone installer version (5.1.156) by adding the following line to the JSON configuration file under the nodeTypes element (I used ClusterConfig.Unsecure.MultiMachine.json but I assume any of the JSON files would work):
"httpApplicationGatewayEndpointPort": "19081"
So the final nodeTypes looked like this:
"nodeTypes": [
{
"name": "NodeType0",
"clientConnectionEndpointPort": "19000",
"clusterConnectionEndpoint": "19001",
"httpGatewayEndpointPort": "19080",
"httpApplicationGatewayEndpointPort": "19081",
"applicationPorts": {
"startPort": "20001",
"endPort": "20031"
},
"ephemeralPorts": {
"startPort": "20032",
"endPort": "20062"
},
"isPrimary": true
}
]
I think it has something to do with the HttpApplicationGatewayEndpoint property, see also my question on https://github.com/Azure/service-fabric-issues/issues/5
But it doesn't work for me..
Also notice that
<Section Name="ApplicationGateway/Http">
<Parameter Name="IsEnabled" Value="true" />
</Section>
is true for me.
Edit:
I noticed that on my Windows-Only installation, HttpApplicationGatewayListenAddress has value 0 in the FabricHostSettings.xml
<Parameter Name="HttpGatewayListenAddress" Value="19080" />
<Parameter Name="HttpGatewayProtocol" Value="http" />
<Parameter Name="HttpApplicationGatewayListenAddress" Value="0" />
<Parameter Name="HttpApplicationGatewayProtocol" Value="" />
I am writing a generic application deployment tool. It takes an application from the user and deploys it to Elastic Beanstalk. That part is working. The issue is that the users want to compose the use of the deployment tool with other operations, and right now my tool reports success when it has told the Beanstalk APIs to start the application.
Unfortunately, it is thus returning before the application itself has started. So the user is forced to write polling logic themselves to await the starting of their application.
Looking at the AWS Elastic Beanstalk API and I cannot see any methods that return any indication of such a state reporting. The closest I can find is DescribeEvents... which looks hopeful, however it seems from the examples that the granularity of the application / application container starting within the environment is not part of that API:
<DescribeEventsResponse xmlns="https://elasticbeanstalk.amazonaws.com/docs/2010-12-01/">
<DescribeEventsResult>
<Events>
<member>
<Message>Successfully completed createEnvironment activity.</Message>
<EventDate>2010-11-17T20:25:35.191Z</EventDate>
<VersionLabel>New Version</VersionLabel>
<RequestId>bb01fa74-f287-11df-8a78-9f77047e0d0c</RequestId>
<ApplicationName>SampleApp</ApplicationName>
<EnvironmentName>SampleAppVersion</EnvironmentName>
<Severity>INFO</Severity>
</member>
Note: the INFO level event is that the environment was created, nothing at the lower level of the application container starting within the environment appears to be reported...
I could mandate that the applications deployed with this tool expose a status REST endpoint, but that puts restrictions on the application.
Is there some API that I am missing that will report when the application container (e.g. Tomcat, Node, etc) is started... or better yet when the application deployed within the container is started... but I can live with the application container
Every application is supposed to expose a health URL (Beanstalk/ELB will have problems any case otherwise - it will think the instances are not responding, and might replace). This is typically a HEAD request expecting a 200 OK.
Since this is anyway expected to be there in all apps, you can probably hit this URL and check the deployment is OK. I guess Beanstalk console itself is using this method.
You can also poll using DescribeEnvironments API call which will give you the Environment CNAME (the URL to check), Health of the environment (RED, GREEN), Status (Launching | Updating | Ready | Terminating | Terminated). This API takes Environment Name as an argument. So you can just get the description of one environment.
API Documentation: http://docs.aws.amazon.com/elasticbeanstalk/latest/APIReference/API_DescribeEnvironments.html
Explanation of Environment Description in response: http://docs.aws.amazon.com/elasticbeanstalk/latest/APIReference/API_EnvironmentDescription.html
Sample Response Below:
<DescribeEnvironmentsResponse xmlns="https://elasticbeanstalk.amazonaws.com/docs/2010-12-01/">
<DescribeEnvironmentsResult>
<Environments>
<member>
<VersionLabel>Version1</VersionLabel>
<Status>Available</Status>
<ApplicationName>SampleApp</ApplicationName>
<EndpointURL>elasticbeanstalk-SampleApp-1394386994.us-east-1.elb.amazonaws.com</EndpointURL>
<CNAME>SampleApp-jxb293wg7n.elasticbeanstalk.amazonaws.com</CNAME>
<Health>Green</Health>
<EnvironmentId>e-icsgecu3wf</EnvironmentId>
<DateUpdated>2010-11-17T04:01:40.668Z</DateUpdated>
<SolutionStackName>32bit Amazon Linux running Tomcat 7</SolutionStackName>
<Description>EnvDescrip</Description>
<EnvironmentName>SampleApp</EnvironmentName>
<DateCreated>2010-11-17T03:59:33.520Z</DateCreated>
</member>
</Environments>
</DescribeEnvironmentsResult>
<ResponseMetadata>
<RequestId>44790c68-f260-11df-8a78-9f77047e0d0c</RequestId>
In your case you may want to read following documentation:
Monitoring Application Health
You can also configure Application Health Check URL for your environment. By default AWS Elastic Beanstalk uses TCP:80 check on your instances. But using the Application Health Check URL you can override this health check to use HTTP:80 by using the Application Health Check URL option as described here.
Using Status/Health from DescribeEnvironments you can check if the application has been deployed.
Just installed Windows Server AppFabric 1.1 on my Windows 7 box and I'm trying to write some code in a console application against the cache client API as part of an evaluation of AppFabric. My app.config looks like the following:
<configSections>
<section name="dataCacheClient" type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" allowLocation="true" allowDefinition="Everywhere" />
</configSections>
<dataCacheClient>
<hosts>
<host name="pa-chouse2" cachePort="22233" />
</hosts>
</dataCacheClient>
I created a new cache and added my domain user account as an allowed client account using the Powershell cmdlet Grant-CacheAllowedClientAccount. I'm creating a new DataCache instance like so:
using (DataCacheFactory cacheFactory = new DataCacheFactory())
{
this.cache = cacheFactory.GetDefaultCache();
}
When I call DataCache.Get, I end up with the following exception:
ErrorCode<ERRCA0017>:SubStatus<ES0006>:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.)
I'd be very grateful if anyone could point out what I'm missing to get this working.
Finally figured out what my problem was after nearly ripping out what little hair I have left. Initially, I was creating my DataCache like so:
using (DataCacheFactory cacheFactory = new DataCacheFactory())
{
this.cache = cacheFactory.GetDefaultCache();
}
It turns out that DataCache didn't like having the DataCacheFactory created it disposed of. Once I refactored the code so that my DataCacheFactory stayed in scope as long as I needed my DataCache, it worked as expected.
This is pretty simple question. I am posting this because I couldn't get any satisfying answer. First the background: I have Jenkins job that builds and deploys a web application on to a server. The server takes some time (in the order of 5 to 10 minutes). I would like to setup a job (or modify the existing as required) to rig up the unit test case execution which will test the application. I am thinking of the following approaches. I would like you to validate or suggest any alternatives:
Have an Ant target that waits for a fixed time
Have a custom Ant target which pings the URL and checks for app availability
Thanks in advance for your help.
-Vadiraj.
Waiting for a fixed time has the problem that the time you choose is either to short (build fails) or to long (waste of build time). So I think it would be better to check if the app is available.
I have done something similar for my Selenium tests. I had to wait until the Selenium Remote Server has started. I used the waitfor element. For a detailled documentation see here.
Here is a stripped down version of my ant-Target:
<parallel>
<sequential>
... Start web application server ...
</sequential>
<sequential>
<waitfor maxwait="10" maxwaitunit="minute">
<socket server="localhost" port="8080" />
</waitfor>
<junit>
...
</junit>
</sequential>
</parallel>
If your server is available before the web app is deployed you can try to use the http condition instead of socket to check for a HTTP error code. The conditions are documented here.