I am re-engineering a site from an existing site, and I am writing acceptance tests to prove out the Use cases and ensure that I don't lose any functionality. The site has several Quartz jobs that are triggered throughout the day, and perform processing for DB records.
For my test strategy, I have access to the existing site and the new site, my plan is to perform each of several use cases against the existing site, and when the tests pass I will be able to use those tests to verify the functionality of the new site.
Here is the catch...
I have the source code, but I will be running my test against the working site. What I would like to do is trigger the Quartz job that needs to be run, and then query the DB to ensure that all of the business rules and Use Case flows have been satisfied. Can I trigger Quartz jobs programmatically on a remote server in my acceptance tests, or is this an impossibility?
I am using JEE5, Quartz, Weblogic, Oracle, Selenium, and jUnit.
Sure, quartz has an rmi interface, and a jmx interface through which you can trigger the jobs.
Related
I am using cloud firestore + cloud functions + firestore auth to support my game.
I developed the main part of the app with unit tests in the app plus typescript tests for cloud functions. Now I want to add security rules to secure the data.
When I do so, requiring the calls to be authenticated, all my unit tests in unity (naturally) fails, as I do not authenticate a user but mocks them as data representation of the user in the db.
I want to keep using my unit tests in unity but still requiring the real db to demand authentication.
I have tried to look around for mock auth, or auth test environment, but found nothing except the library rules-unit-testing.
I see the content of it with specialized logic for mocking user, making me think that I am understanding this the wrong way by trying to do this in unity. My question is, How to continue to do game tests in unity, which requires interacting with the firestore server, while keeping security rules?
I am answering my own question after more time.
My analysis was that I ran into issues because I had coupled my code too tightly: server logic was on the client side and broke when introducing security rules. I decided to move logic to cloud functions and have only simple client side calls (sets, gets, updates, http functions).
So if someone runs into similar problems (architecture hampers use of best practices) I suggest to re-think the architecture. Feels obvious when writing...
Have fun coding all ^_^
I've hosted my website on azure and now I want to schedule payments on a monthly basis. I am using Authorize.net for payments but I cannot use their recurring billing feature as it gives very little control. I've to perform checks in the database, make payments and update records. What should I use Azure Scheduler, Azure WebJob or Azure Functions a Worker Role?
Definitely not a Worker Role. They are very heavyweight and generally not worth the effort for a single, simple job like this.
Web jobs might be a good solution. It can run in the context of your web app, so you can use this with no additional cost. But you'll need to do some development with this - you have to create an app that calls Authorize.net.
If you only need to fire a single HTTP request, then using Azure Scheduler to schedule this HTTP action might be a good choice. You can configure the request itself (headers, payload) and it has error handling as well. But you might have to store sensitive information in the Azure portal, in the configuration of the scheduled job.
So I'd say forget about the Worker Role, then weigh simplicity against flexibility and development effort. That being sad, I would probably try it with the scheduler, and then move on to the WebJob, if I encounter something that is not feasible with the scheduler.
Edit:
Azure Functions can also be a good option - I'd say it's sort of a middle ground between the webjob and the simple scheduled option. It is part of the app services featureset, so it can run in the same appservice plan as the web app, so no costs. But here you have to code the http request to Authorize.net yourself as well. But Azure Functions is a lot more lightweight compared to webjobs - you do not have to create an exe (or ps script or whatever), you can just code the http request in a script editor inside the Azure portal. But you still have to do it yourself. This is a bit more flexible than the simple scheduled option though, which is something to consider when it comes to error handling.
So this is a good middleground, but I think it's still a lot of work given the complexity of the task (which is to fire a single HTTP request).
To get it working quickly, Logic Apps is a good choice. With Logic Apps, you can trigger it with a timer based on schedule you defined, use the out-of-box SQL/DocDB (depending on your exact scenario) to connect to your database. Although there's currently no Authorize.net connector available, you should be able to use the generic HTTP action to talk to its RESTful APIs. Most likely, you should be able to get this working very quickly. I'd also recommend submit a suggestion on aka.ms/logicapps-wish so we can track the request for Authorize.net connector, when available, is going to make this ever easier.
I am currently writing integration/functional tests for my system. Part of the functionality is to hit a web service via http (another system I run).
How should I set up a test instance of the web service to allow for good functional testing? I want to have my system run against this service with live production data.
Should the web service be an independent instance that always has the live production data that I reload manually (maybe reset every time I start an instance of it)?
Should the web service be setup and teared down with every test?
What are some common practices to deal with situations like this?
As first thing please make sure you know difference between Functional Testing and Integration Testing. You can do a pretty good functional tetsing wthout major efforts which are required by integration testing (instantiating web services, accessing a data base). Basically Mocking technique works pretty well even to simulate the Data Layer responses and web service behaviour (I believe such details like HTTP as transport could be ignored for most test cases)
For such integration testing I would suggest having a separate SIT environment which includes a separate Web service and a Data Base as well.
Should the web service be an independent instance that always has the live production data
that I reload manually (maybe reset every time I start an instance of it)?
Yep, it should be completely separate but data could be manually generated/prepared. For instance, you can prepare some set of data which allows testing some predefined test cases, this could be test data sets which are deployed to the SIT DB instance before the actual test run and then cleaned up in test TearDown.
Should the web service be setup and teared down with every test?
Yep, tests should be isolated one from each other so should not affect each one in any way.
My test team currently uses QTP to test through the GUI, but like any automated test suite that relies on the interface, it is more fragile than automating tests that directly interact with the code. I am attempting to learn more about Siebel and Siebel Tools to better understand how we might be able to test below the GUI, but would like to hear from someone with more expertise to find out if this is feasible.
it really depends on what you want to test, I guess.
I'm using the Siebel Java Data Bean (JDB) a lot to access Siebel. You basically connect to the Siebel Server and execute code very similar to eScript. That means you could create records, invoke workflows and so on; basically everything you could do in eScript. That might be helpful. This will apply all the usual validations, runtime events and events.
As soon as some of your scripts in BusComps or in Business Services or elsewhere access data that needs a UI context (TheApplication().ActiveBusObject() or TheApplication().ActiveApplet() for instance) this approach will fail, though, because the Siebel Data Bean doesn't have a UI context.
Another drawback is that you have to connect to a Siebel Server. That means you have to deploy your SRF to the development server and only then you can run your tests. It sure would be much better if the JDB could connect to your local instance, but as far as I know this is not possible. Have a look at the Object Interfaces guide in the Bookshelf, though. There are different ways to connect to Siebel, not just Java.
Let me know if you have any questions about this. I could maybe post some sample code of how to connect to the Siebel Server etc.
Right now QTP is the best way to go - it's still a PITA but there really is nothing else out there to test the full Siebel Web Client. This is because the Siebel UI is delivered through Internet Explorer with proprietary Active X and Java controls and so you really need a bespoke pack to test it.
Because the UI is a re-interpretation, not just an abstraction, of the Business Object layer (that one accesses with Data Beans / COM etc.) it is not useful to test at that layer except in a small number of unit test cases (such as when you have complex scripting in Siebel).
If you change the end of the URL for the client (login to Siebel first of course) to something like "SWEcmd=GotoPageTab&SWEScreen=Accounts+Screen&SWESetMarkup=XML" then you'll see lots of XML mark-up which is then consumed by the proprietary controls - you might think this would be a cool way to build an automation tool, but it is not (I've tried).
If you want to really use a proper UI testing tool, like Selenium, you'll have to test the HTML Siebel Web Client - this is a 'skinny' 'Standard Interactivity' UI that doesn't use Active X or Java ... it has a lot less cool UI controls but it works essentially the same at the full Siebel Web Client (aka High Interactivity Siebel Web Client, or HI for short), and it works in Firefox!
Have you looked at Oracle Application testing Suite. It comes pre-built accelerators for testing Siebel which makes it all the more easier to test Siebel.
Since Siebel version 7.7 QTP uses Siebel Test Automation (STA) which needs to be purchased separately from Oracle, a quick search found this explanation on how to set up testing with STA (this is written from a QTP prespective but is true for all STA usage).
If you really want to avoid using GUI testing then you can hunt down the API documentation and try to use STA directly but I would not recommend it, QTP has already done all the heavy lifting for you, why would you want to reproduce the effort (especially since your company already owns QTP licenses).
Should we unit test web service or really be looking to unit test the code that the web service is invoking for us and leave the web service alone or at least until integration testing, etc.... ?
EDIT: Further clarification / thought
My thought is that testing the web service is really integration testing not unit testing?? I ask because our web service at this point (under development) is coded in such a way there is no way to unit test the code it is invoking. So I am wondering if it is worth while / smart to refactor it now in order to be able to unit test the code free of the web service? I would like to know the general consensus on if it's that important to separate the two or if it's really OK to unit test the web service and call it good/wise.
If I separate them I would look to test both but I am just not sure if separation is worth it. My hunch is that I should.
Unit testing the code that the web service is invoking is definitely a good idea since it ensures the "inside" of your code is stable (and well designed). However, it's also a good idea to test the web service calls, especially if a few of them are called in succession to accomplish a certain task. This will ensure that the web services that you've provided are usable, as well as, work properly when called along with other web service calls.
(Not sure if you're writing these tests before or after writing your code, but you should really consider writing your web service tests before implementing the actual calls so that you ensure that they are usable in advance of writing the code behind them.)
Why not do both? You can unit test the web service code, as well as unit test it from the point of view of a client of the web service.
Under my concept, the WS is just a mere encapsulation of a Method of a Central Business Layer Object, in other words, the Web Method is just a "gate" to access methods deeper in the model.
With the former said, i do both operations:
Inside the Server, I create a Winform App who do Load Testing on the Business Layer Method.
Outside the Server (namely a Machine out of the LAN where the Web App "lives"), I create a Tester (Winform or Web) that consumes the WS, doing that way the Load Testing.
That Way I can evaluate the performance of my solution considering and discarding the "Web Effect" (i.e. The time for the data to travel and reach the WS, the WS object creation, etc).
All the above said is of course IMHO. At least that worked a lot for me!
Haj.-
We do both.
We unit test the various code elements.
Plus, we use the unit test framework to execute tests against the web service as a whole. This is rather complex because we have to create (and load) a database, start a server, and then execute requests against that server.
Testing the web service API is easy (it's got an API) and valuable. It's not a unit test, though - it's an "integration", "sub-system", or "system" test (depends on who you ask).
There's no need to delay the testing until some magical period called "integration testing" though, just get some simple tests now and reap the benefit early.
I like the idea of writing unit tests which call your web service through one of its public interfaces. For instance, a given WCF web service may expose HTTP, TCP, and "web" bindings. Such a unit tests proves that the web service can be called through a binding.
Integration testing would involve testing all of the bindings of the service, testing with particular client scenarios, and with particular client tools. For instance, it would be important to show that a Java client can be created with IBM's Rational Web Developer that can access the service when using WS-Security.
If you can, try consuming your web service using some of the development tools your customers will use (Delphi, C#, VB.Net, ColdFusion, hand crafted XML, etc...). Within reason, of course.
1) Different tools may have problems consuming your web service. Better for you to run in to this before your customers do.
2) If a customer runs in to a problem, you can easily prove that your web service is working as expected. In the past year or so, this has stopped finger pointing in its tracks at least a dozen times.
The worst was the developer in a different time zone who was hand crafting the XML for SOAP calls and parsing the responses. Every time he ran in to a problem, he would insist that it was on our end and demand (seriously) that we prove otherwise. I made a dead simple Delphi app to consume the web service, demonstrate that the methods worked as expected and even displayed the XML for each request and response.
re your updated question.
The integration testing and unit testing are only superficially similar, so yes, they should be done and thought of separately.
Refactoring existing code to make it testable can be risky. It's up to you to decide if the benefits outweigh the time and effort it will take. In my case, I'd certainly try, even if you do it a little bit at a time.
On the bright side, a web service has a defined interface, so you don't really have to change anything to add integration testing. Go crazy. If you can, try to do this before you roll the web service out to customers. There's a good chance that using the web service lead to changes in the interface, and you don't want this to mess up customers too much.