We are looking at db40 for a high volume e-commerce website using Java on the server side. Concurrency and transactional support is very important to us. When a customer purchases an item we need to lock the product and customer objects to update inventory and customer order history, respectively, in a single transaction. Is this possible with db4o? I want to make sure it support multi-object transactions.
There are already similar questions here, like this one. And my answer is more or less the same.
About the high volume e-commerce website: db4o was never build as a high volume, big database but rather for embedded use cases like desktop and mobile apps. Well it depends what a 'high volume' means. I assume that it means hundreds or concurrent transactions. Thats certainly out of scope of db4o.
Concurrency and transactional support: The db4o core is still inherently single threaded and therefore can only serve a small amount of concurrent operations. db4o supports transactions with the read committed isolation. That means that a transaction can only see the committed state of other transactions. In practice thats a very weak guarantee.
To your example: you can update the purchase with the product and consumer in one transaction. However another transaction could update any of these objects and commit. Then a running transaction which already has read some objects might does calculations with the old value and stores it. So the weak isolation 'taints' your state.
You could use locks to prevent that, but db4o hasn't any nice object-locking mechanism. And it would decrease the performance further.
All in all I think you probably need a 'larger' database, which has better support for Concurrency and transaction-handling.
It sounds like you need to use db4o semaphores.
Related
I am developing a web application server on AWS that need to support high throughput on reading and write. My boss gave me a high-level design like this.
I am stuck on the "Write Queue". The team told me that we need it to increase the performance of writing because we can have only 1 master replica to which we can write. I have some basic knowledge about message queues such as SQS and RabbitMQ but don't know anything about using it as a database write queue.
At the current stage, I have 3 questions:
Using this architecture, is it really able to increase the performance of writing to the database (as opposed to writing directly to the master replica).
How to handle transactions, especially how to rollback, when errors occur during the writing. Normally, we would control the transaction in application code such that when an error occurs, the whole transaction is rollback and the App Server response to the client with some error code.
I mentioned that I have researched about using message queue as a write queue, but I am not sure if I am looking in the right direction. Maybe, there is some other technology already that is suitable to be a write queue to the database?
In addition to the questions, I believe this should be a big topic and would like to know the resources where I can research in detail on this topic.
In similar cases, queues are used as a mean for de-coupling two systems. There are several advantages and disadvantages when implementing such architectural patterns. I will try to list what I believe are the main ones.
Advantages
Improved response time
As queues do not require complex transactions they are usually a fast and, if correctly configured, safe storage. This means the perceived response latency from the client side will decrease giving the feeling that the service is "faster".
Separation of concerns
Correctly de-coupling services increases their resilience to errors. For example, if the DB cannot accept more write requests, the clients will be unaffected and their requests will still not be lost as they will be in the queue. This gives Operators more time to react to problems while the service value is only partially affected.
Improved scalability
When operations become complex, it's usually a good idea to separate them into microcomponents. It is way easier to scale up microcomponents than monolithic services. Job queues enable such design patterns.
Disadvantages
Recovering from errors becomes more complex
As said above, if the DB stops accepting requests, jobs will pile up in the queue. Now you have 2 problems to deal with: a full DB and a full job queue. System problems start propagating across your architecture like ripples causing several side effects and making hard to understand what is the root cause.
Identifying bottlenecks requires more time
If the DB writes are slow, putting a queue in front of it won't make things faster. Jobs will still pile up in the queue and your next task will be figuring out why this happens. When dealing with complex ETL pipelines, improving performance becomes a quite tedious whack-a-mole operation where your bottlenecks just shift from system to system.
Cost per operation increases
The more stages a job needs to traverse for its completion, the more time and money that job will require.
De-coupling components is usually seen as a silver bullet for dealing with performance issues. The correct separation of concerns and responsibilities is a very beneficial practice but requires a lot of experience and care. Nowadays monolithic services are seeing as the root of all evils. Personally I prefer to deal with a monolithic bunch of spaghetti rather than a distributed one.
At present, we are using Redis as an in-memory, fast cache. It is working well. The problem is, once Redis is restarted, we need to re-populate it by fetching data from our persistent store. This overloads our persistent
store beyond its capacity and hence the recovery takes a long time.
We looked at Redis persistence options. The best option (without compromising performance) is to use AOF with 'appendfsync everysec'. But with this option, we can loose last second data. That is not acceptable. Using AOF with 'appednfsync always' has a considerable performance penalty.
So we are evaluating single node Aerospike. Does it guarantee no data loss in case of power failures? i.e. In response to a write operation, once Aerospike sends success to the client, the data should never be lost, even if I pull the power cable of the server machine. As I mentioned above, I believe Redis can give this guarantee with the 'appednfsync always' option. But we are not considering it as it has the considerable performance penalty.
If Aerospike can do it, I would want to understand in detail how persistence works in Aerospike. Please share some resources explaining the same.
We are not looking for a distributed system as strong consistency is a must for us. The data should not be lost in node failures or split brain scenarios.
If not aerospike, can you point me to another tool that can help achieve this?
This is not a database problem, it's a hardware and risk problem.
All databases (that have persistence) work the same way, some write the data directly to the physical disk while others tell the operating system to write it. The only way to ensure that every write is safe is to wait until the disk confirms the data is written.
There is no way around this and, as you've seen, it greatly decreases throughput. This is why databases use a memory buffer and write batches of data from the buffer to disk in short intervals. However, this means that there's a small risk that a machine issue (power, disk failure, etc) happening after the data is written to the buffer but before it's written to the disk will cause data loss.
On a single server, you can buy protection through multiple power supplies, battery backup, and other safeguards, but this gets tricky and expensive very quickly. This is why distributed architectures are so common today for both availability and redundancy. Distributed systems do not mean you lose consistency, rather they can help to ensure it by protecting your data.
The easiest way to solve your problem is to use a database that allows for replication so that every write goes to at least 2 different machines. This way, one machine losing power won't affect the write going to the other machine and your data is still safe.
You will still need to protect against a power outage at a higher level that can affect all the servers (like your entire data center losing power) but you can solve this by distributing across more boundaries. It all depends on what amount of risk is acceptable to you.
Between tweaking the disk-write intervals in your database and using a proper distributed architecture, you can get the consistency and performance requirements you need.
I work for Aerospike. You can choose to have your namespace stored in memory, on disk or in memory with disk persistence. In all of these scenarios we perform favourably in comparison to Redis in real world benchmarks.
Considering storage on disk when a write happens it hits a buffer before being flushed to disk. The ack does not go back to the client until that buffer has been successfully written to. It is plausible that if you yank the power cable before the buffer flushes, in a single node cluster the write might have been acked to the client and subsequently lost.
The answer is to have more than one node in the cluster and a replication-factor >= 2. The write then goes to the buffer on the client and the replica and has to succeed on both before being acked to the client as successful. If the power is pulled from one node, a copy would still exist on the other node and no data would be lost.
So, yes, it is possible to make Aerospike as resilient as it is reasonably possible to be at low cost with minimal latencies. The best thing to do is to download the community edition and see what you think. I suspect you will like it.
I believe aerospike would serves your purpose, you can configure it for hybrid storage at namespace(i.e. DB) level in aerospike.conf
which is present at /etc/aerospike/aerospike.conf
For details please refer official documentation here: http://www.aerospike.com/docs/operations/configure/namespace/storage/
I believe you're going to be at the mercy of the latency of whatever the storage medium is, or the latency of the network fabric in the case of cluster, regardless of what DBMS technology you use, if you must have a guarantee that the data won't be lost. (N.B. Ben Bates' solution won't work if there is a possibility that the whole physical plant loses power, i.e. both nodes lose power. But, I would think an inexpensive UPS would substantially, if not completely, mitigate that concern.) And those latencies are going to cause a dramatic insert/update/delete performance drop compared to a standalone in-memory database instance.
Another option to consider is to use NVDIMM storage for either the in-memory database or for the write-ahead transaction log used to recover from. It will have the absolute lowest latency (comparable to conventional DRAM). And, if your in-memory database will fit in the available NVDIMM memory, you'll have the fastest recovery possible (no need to replay from a transaction log) and comparable performance to the original IMDB performance because you're back to a single write versus 2+ writes for adding a write-ahead log and/or replicating to another node in a cluster. But, your in-memory database system has to be able to support direct recovery of an in-memory database (not just from a transaction log). But, again, two requirements for this to be an option:
1. The entire database must fit in the NVDIMM memory
2. The database system has to be able to support recovery of the database directly after system restart, without a transaction log.
More in this white paper http://www.odbms.org/wp-content/uploads/2014/06/IMDS-NVDIMM-paper.pdf
I have an app that spins up multiple processes to read large amounts of data from several PostgreSQL tables to do number crunching, and then stores the results in separate tables.
When I tested this with just a single process, it was blazing fast and was using almost 100% CPU, but when I tried using 8 processes on an 8 core machine, all processes registered about 1% CPU and the whole task seemed to take even longer.
When I check pg_stat_activity, I saw several connections listed as "<IDLE> in transaction". Following some advice here, I looked at pg_locks, and I'm seeing hundreds of "AccessShareLock" locks on the dozens of read-only tables. Based on the docs, I believe this is the default, but I think this is causing the processes to step on each others feet, negating any benefit to multi-processing.
Is there a more efficient isolation level to use, or better way to tune PostgreSQL to allow faster read-only access to several processes, so each doesn't need to lock the table? Specifically, I'm using Django as my ORM.
Not sure what throttles your multiple cores, but it has nothing to do with the isolation level. Even if you have concurrent write operations. Per documentation:
The main advantage of using the MVCC model of concurrency control
rather than locking is that in MVCC locks acquired for querying
(reading) data do not conflict with locks acquired for writing data,
and so reading never blocks writing and writing never blocks reading.
PostgreSQL maintains this guarantee even when providing the strictest
level of transaction isolation through the use of an innovative
Serializable Snapshot Isolation (SSI) level.
Bold emphasis mine.
Of course, reading also never blocks reading.
Maybe you need to reconfigure resource allocation on your server? Default configuration is regularly to conservative. On the other hand, some parameters should not be set too high in a multi-user environment. work_mem comes to mind. Check the list for Performance Optimization in the Postgres Wiki.
And finally:
Django as my ORM.
ORMs often try to stay platform-independent and fail to get the full potential out of a particular RDBMS. They are primitive crutches and don't play well with performance optimization.
hi we try to implement a process like when a user does something, his company's credit will be deducted accordingly.
But there is a concurrency issue when multiple users in one company participant in the process because the credit got deducted wrong.
Can anyone point a right direction for such issue?
thanks very much.
This is a classic problem that is entirely independent of the implementation language(s).
You have a shared resource that is maintaining a persistent data store. (This is typically a database, likely an RDBMS).
You also have a (business) process that uses and/or modifies the information maintained in the shared data store.
When this process can be performed concurrently by multiple actors, the issue of informational integrity arises.
The most common way to address this is to serialize access to the shared resources, so that the operation against the shared resources occur in sequence.
This serialization can happen at the actor level, or, at the shared resource itself, and can take many forms, such as queuing actions, or using messaging, or using transactions at the shared resource. Its here that considerations such as system type, application, and the platforms and systems that are used become important and determine the design of the overall system.
Take a look at this wikipedia article on db transactions, and then google your way to more technical content on this topic. You may also wish to take a look at messaging systems, and if you are feeling adventurous, also read up on software transactional memory.
What's the optimal level of concurrency that the C++ implementation of BerkeleyDB can reasonably support?
How many threads can I have hammering away at the DB before throughput starts to suffer because of resource contention?
I've read the manual and know how to set the number of locks, lockers, database page size, etc. but I'd just like some advice from someone who has real-world experience with BDB concurrency.
My application is pretty simple, I'll be doing gets and puts of records that are about 1KB each. No cursors, no deleting.
It depends on what kind of application you are building. Create a representative test scenario, and start hammering away. Then you will know the definitive answer.
Besides your use case, it also depends on CPU, memory, front-side bus, operating system, cache settings, etcetera.
Seriously, just test your own scenario.
If you need some numbers (that actually may mean nothing in your scenario):
Oracle Berkeley DB:
Performance Metrics and
Benchmarks
Performance Metrics
& Benchmarks:
Berkeley DB
I strongly agree with Daan's point: create a test program, and make sure the way in which it accesses data mimics as closely as possible the patterns you expect your application to have. This is extremely important with BDB because different access patterns yield very different throughput.
Other than that, these are general factors I found to be of major impact on throughput:
Access method (which in your case i guess is BTREE).
Level of persistency with which you configured DBD (for example, in my case the 'DB_TXN_WRITE_NOSYNC' environment flag improved write performance by an order of magnitude, but it compromises persistency)
Does the working set fit in cache?
Number of Reads Vs. Writes.
How spread out your access is (remember that BTREE has a page level locking - so accessing different pages with different threads is a big advantage).
Access pattern - meanig how likely are threads to lock one another, or even deadlock, and what is your deadlock resolution policy (this one may be a killer).
Hardware (disk & memory for cache).
This amounts to the following point:
Scaling a solution based on DBD so that it offers greater concurrency has two key ways of going about it; either minimize the number of locks in your design or add more hardware.
Doesn't this depend on the hardware as well as number of threads and stuff?
I would make a simple test and run it with increasing amounts of threads hammering and see what seems best.
What I did when working against a database of unknown performance was to measure turnaround time on my queries. I kept upping the thread count until turn-around time dropped, and dropping the thread count until turn-around time improved (well, it was processes in my environment, but whatever).
There were moving averages and all sorts of metrics involved, but the take-away lesson was: just adapt to how things are working at the moment. You never know when the DBAs will improve performance or hardware will be upgraded, or perhaps another process will come along to load down the system while you're running. So adapt.
Oh, and another thing: avoid process switches if you can - batch things up.
Oh, I should make this clear: this all happened at run time, not during development.
The way I understand things, Samba created tdb to allow "multiple concurrent writers" for any particular database file. So if your workload has multiple writers your performance may be bad (as in, the Samba project chose to write its own system, apparently because it wasn't happy with Berkeley DB's performance in this case).
On the other hand, if your workload has lots of readers, then the question is how well your operating system handles multiple readers.