After reading http://blog.plataformatec.com.br/2015/10/mocks-and-explicit-contracts/ article I have in my code:
defp rest_adapter, do: Application.get_env(:app_name, :rest_adapter)
I'm using it to "mock" rest adapter during tests and return different results and error codes.
However, during these tests, there's a race condition because I set different rest_adapter for different test cases.
Sometimes they work as intended but sometimes they don't "catch" different rest_adapter set particularly for this test.
How can avoid that problem there?
The issue comes from the fact that application environment is global - you can't concurrently change it in multiple places to have different values.
The simplest solution is to disable concurrent tests by removing async: true this however, makes your tests slower, since they can't run concurrently.
Fortunately there are other possible solutions. The simplest one (and arguably the most elegant one) is to pass the adapter as an option to the function that uses it, and when none is provided use the one from application environment. Another solution, when the calls happen in the same process (which is often the case) is to use process dictionary to pass the adapter, instead of relying on the application environment.
Furthermore, it's possible to adapt hybrid solutions for example have two adapters: a real one and one returning the response from the process dictionary. Then use the process dictionary one always in the tests and just put the expected response before calling the function.
Finally there are things like Bypass that work on a slightly different layer giving you a mocked HTTP endpoint, instead of replacing the code for calling the endpoint (which is the usual approach to "replace" http calls in tests).
Related
I am splitting an application in two, and therefore replacing some C++ function calls by gRPC calls. The new modules can run on different machines or on the same one (even in the same process). Do I need a mechanism (e.g. mutex/locking) to avoid multithreading issues, which where not present before?
As I understand, the gRPC server uses thread-pools and may handle each request on a different thread. That basically opens up potential for multithreading issues. However, my client code is single-threaded, so the second (gRPC) call is executed only after the first (gRPC) call has returned. I am unsure whether this guarantees that ALL side-effects of the first call are visible to the second call, and everything still works as if it was run on a single thread. So far, the application seems to work fine without any locks, but that might just be "luck", as it often is the case with threading issues...
The only related question I found is Order of function calls -gRPC, but that question does not specify how the client code is invoking the calls. And it also seems the poster can actually reproduce an issue, while my application seems to work fine so far
I have an existing C/C++ application that communicates with other applications/systems through several interfaces (TCP, DB, shared memory). I would like to run the application once with the real environment and "record" all function calls and their return values (or changes to the buffers passed as parameters). I would record only the calls related with external interfaces (TCP, DB) with a "spy". Then I could run the application again but using "fake" functions that should return the previous recorded values. This way I could "replay" an execution, and check if the results match the original execution.
One important feature is to mock the time functions as well (sleep, time, GetLocalTime), because (for example) the calls to the DB may have the current date or time in the selects. It would be even better to be able to "replay" all the calls faster than the original execution (one day of execution could be replayed in a few minutes). For example a call to sleep(1000) should return without waiting, but successive calls to GetLocalTime should return 1 second more. This should take into account that other threads should have consistent values for the time (for example the library should allow 1 call to sleep(1000) for one thread and 10 calls to sleep(100) in another thread).
Ideally it shouldn't require a lot of changes or refactoring to the application, just redefining the calls to time functions as well as the calls to the libraries of the external interfaces (DB, TCP).
I would like to know if there exists some library or framework that implements these features or what could be a good starting point.
I have implemented several times solutions for similar problems, but for very simple modules, for example mocking a TCP connection to test a protocol implementation, but I feel like reinventing the wheel each time, and these simple solutions wouldn't scale well with more threads or interactions with more interfaces.
Since you mention that your application is C/C++, I will assume you have the ability to use C frameworks. I will talk about CMock and Unity in this answer. If object oriented principles are more prevalent in your task, then you can take a look at googlemock which is similar to CMock, but developed for C++ and accounts for classes.
--
CMock in combination with Unity may provide a framework for your testing needs.
CMock provides you the ability to easily mock public interfaces. For example, if you had the following trivial code to test:
void sendPacket(char * packet, int packetLen, int tcpSocket)
{
if (packet)
{
send(tcpSocket, packet, packetLen, 0);
}
}
You could use CMock to create a mock of the send call. This would allow you to verify the parameters passed to send, and thus verify content in buffers. You would also be able to verify the size returned by send. CMock's ExpectAndReturn variant of the mocked send function would allow you to perform these two verifications.
CMock also allows you to provide callbacks for mocked functions through the StubWithCallback variant. StubWithCallback will allow you to manually verify each call, in addition to doing special checks. You would be able to utilize StubWithCallback to record number of times a function is called, or almost anything else you can imagine. You would also be able to use the StubWithCallback for your needs with time, sleep, etc. You can mock individual calls with different stub callbacks-- some threads will start with a stub sleep callback that immediately returns, while threads you specify for testing can use a stub sleep callback that performs the full sleep.
You can find more information on CMock here: http://throwtheswitch.org/white-papers/cmock-intro.html
I have a Web Service (in java) on a Oracle WebLogic 10.3 that does all kinds of database queries. Recently I started stress tests. It passed the repetition tests (invoke the WS several 1000 times serially) but problems become to arise when concurrency testing began. Making as much as 2 concurrent calls results in errors. When doing proper tests the results looked like the WS wasn't able to handle concurrent calls at all, which obviously should not be the case. Error included null pointer exceptions, closed connections or prepared statements, etc. I am bit stumped at this specially since I was unable to find any kind of configuration options that could effect this but then again my knowledge of the WLS is quite limited.
Thanks for any suggestions in advance.
The answer you marked as correct is totally wrong.
The webservice methods should not be made in order to be thread safe.
Webservice implenmtation of weblogic are multithreaded.
It's like for the servlets
"Servlets are multithreaded. Servlet-based applications have to recognize and handle this appropriately. If large sections of code are synchronized, an application effectively becomes single threaded, and throughput decreases dramatically."
http://www.ibm.com/developerworks/websphere/library/bestpractices/avoiding_or_minimizing_synchronization_in_servlets.html
The code inside the WS you might want to synchronize depending what you do.
Does it make sense to synchronize web-service method?
Just so there is a clear answer.
When there are several concurrent calls to a given Web Service (in this case SOAP/JAX-WS was used) on WLS, the same object is used (no pooling or queues are used), therefore the implementation must be thread safe.
EDIT:
To clarify:
Assume there is a class attribute in the WebService implementation class generated by JDeveloper. If you modify this attribute in your web method (and then use it) it will cause synchronization problems when the method is called (ie WS is called) concurrently. When I first started creating web services I though the whole WebService object was created anew for each WS call but this does not seem to be the case.
What are some methods for testing concurrent data structures to make sure the data structs behave correctly when accessed from multiple threads ?
All of the other answers have focused on actually testing the code by putting it through its paces and actually running it in one form or another or politely saying "don't do it yourself, use an existing library".
This is great and all, but IMO, the most important (practical tests are important too) test is to look at the code line by line and for every line of code ask "what happens if I get interrupted by another thread here?" Imagine another thread, running just about any of the other lines/functions during this interruption. Do things still stay consistent? When competing for resources, does the other thread[s] block or spin?
This is what we did in school when learning about concurrency and it is a surprisingly effective approach. Bottom line, I feel that taking the time to prove to yourself that things are consistent and work as expected in all states is the first technique you should use when dealing with this stuff.
Concurrent systems are probabilistic and errors are often difficult to replicate. Therefore you need to run various input/output cases, each tested over time (hours, days, etc) in order to detect possible errors.
Tests for concurrent data structure involves examining the container's state before and after expected events such as insert and delete.
Use a pre-existing, pre-tested library that meets your needs if possible.
Make sure that the code has appropriate self-consistency checks (preferably fast sanity checks), and run your code on as many different types of hardware as possible to help narrow down interesting timing problems.
Have multiple people peer review the code, preferably without a pre-explanation of how it's supposed to work. That way they have to grok the code which should help catch more bugs.
Set up a bunch of threads that do nothing but random operations on the data structures and check for consistency at some rate.
Start with the assumption that your calls to access/modify data are not thread safe and use locks to ensure only a single thread can access/modify any part of the data at a time. Only after you can prove to yourself that a specific type of access is safe outside of the lock by multiple threads at once should you move that code outside of the lock.
Assume worst case scenarios, e.g. that your code will stop right in the middle of some pointer manipulation or another critical point, and that another thread will encounter that data in mid-transition. If that would have a bad result, leave it within the lock.
I normally test these kinds of things by interjecting sleep() calls at appropriate places in the distributed threads/processes.
For instance, to test a lock, put sleep(2) in all your threads at the point of contention, and spawn two threads roughly 1 second apart. The first one should obtain the lock, and the second should have to wait for it.
Most race conditions can be tested by extending this method, but if your system has too many components it may be difficult or impossible to know every possible condition that needs to be tested.
Run your concurrent threads for one or a few days and look what happens. (Sounds strange, but finding out race conditions is such a complex topic that simply trying it is the best approach).
I have a Django app, which spawns a thread to communicate with another server, using Pyro.
Unfortunately, it seems like under fastcgi, multiple versions of this thread are fired off, and a dictionary that should be globally constant within my program, isn't. (Sometimes it has the values I expect, sometimes not)
What's the best way to ensure that there's one and only one copy of a dictionary in a django / fastcgi app?
I strongly recommend against relying on global anything in django. The problem is that, just as you seem to be encountering, the type of deployment will determine how (or whether or not) this global state is shared. To be a style nazi, that's a completely different level of abstraction from the code, which is relying on some guarantee of consistent global state.
I'm not experienced with fastcgi, but my understanding is that it, like many other frameworks, has a pre-forked and a threaded mode. In pre-forked mode, you have separate processes, not threads, running your python code. This spells nightmare for shared global state.
Barring some fragile workaround, which ought to be possible and which someone may or may not suggest, the only persistence you can really rely on is in the database, and, to a lesser extent, whatever caching mechanism you choose. You could use the low-level api to cache and retrieve keys and values.