I have a question about a specific functionality in Siebel, regarding service requests.
Is there a way to track time when certain service request is in a given status/substatus, for example "Waiting on Customer"? When the service request is changed again to another status that isn't "Wait for somebody" anymore, I have to stop counting the time.
I don't know of any out of the box solution to your needs, however there are many ways to achieve it with a bit of customisation. For example:
Create two new fields, Waiting Time (with predefault value: 0) and Waiting Date.
Create the following BC user properties:
On Field Update Set x = "Status", "Waiting Time", "IIF([Waiting Date] IS NULL, [Waiting Time], [Waiting Time] + (Timestamp() - [Waiting Date]))
On Field Update Set y = "Status", "Waiting Date", "IIF([Status]='Waiting on Customer',Timestamp(),NULL)"
Your Waiting Date field will store the last time the service request changed to "Waiting on Customer", or NULL if it's on another status. Then, Waiting Time will accumulate the total time the request has been in that status.
I have not tested the solution, it might need some more work, for example, it's possible that Siebel doesn't allow you to use the expression [Waiting Time] + (Timestamp() - [Waiting Date]) directly and you'll have to decompose it using auxiliary calculated fields.
Note also that the On Field Update Set user property has changed its syntax from Siebel 7.7-7.8 to Siebel 8.x.
If you're familiar with server scripting, you could implement something similar quite easily, on the BusComp_PreSetFieldValue event. If the field being changed is Status, check if you're entering or exiting (or not) the "Waiting on Customer" status, and update the two fields accordingly.
Related
Hi everyone,
I'm a little bit lost with a problem thinking in ddd way.
Imagine you have an application to sell concert ticket. So you have an entity which is called Concert with the quantity number and a method to buy a ticket.
class Concert {
constructor(
public id: string,
public name: string,
public ticketQuantity: number,
) {}
buyTicket() {
this.ticketQuantity = this.ticketQuantity - 1;
}
}
The command looks like this:
async execute(command: BookConcertCommand): Promise<void> {
const concert = await this.concertRepository.findById(command.concertId);
concert.buyTicket();
await this.concertRepository.save(concert);
}
Imagine, your application has to carry a lot of users and 1000 users try to buy a ticket at the same when the ticketQuantity is 500.
How can you ensure the invariant of the quantity can't be lower than 0 ?
How can you deal with concurrency here because even if two users try to buy a ticket at the same time the data can be false ?
What are the patterns we can use to ensure consistency and concurrency ?
Optimistic or pessismistic concurrency can't be a solution because it will frustrate a lot of users and we try to put all our logic domain into our domain so we can't put any logic inside sql/db or use a transactional script approach.
How can you ensure the invariant of the quantity can't be lower than 0
You include logic in your domain model that only assigns a ticket if at least one unassigned ticket is available.
You include locking (either optimistic or pessimistic) to ensure "first writer wins" -- the loser(s) in a data race should abort or retry.
If your book of record was just data in memory, then you would ensure that all attempts to buy tickets for concert 12345 must first acquire the same lock. In effect, you serialize the requests so that the business logic is running one at a time only.
If your book of record was a relational database, then within the context of each transaction you might perform a "select for update" to get a copy of the current data, and perform the update in the same transaction. The database will raise it's flavor of concurrent modification exception to the connections that lose the race.
Alternatively, you use something like the semantics of a conditional-write / compare and swap: you get an unlocked copy of the concert from the book of record, make your changes, then send a "update this record if it still looks like the unlocked copy" message, if you get the response announcing you've won the race, congratulations - you're done. If not, you retry or fail.
Optimistic or pessismistic concurrency can't be a solution because it will frustrate a lot of users
Of course it can
If the concert is overbooked, they are going to be frustrated anyway
The business logic doesn't have to run synchronously with the request - it might be acceptable to write down that they want a ticket, and then contact them asynchronously to let them know that a ticket has been assigned to them
It may be helpful to review some of Udi Dahan's writing on collaborative and competitive domains; for instance, this piece from 2011.
In a collaborative domain, an inherent property of the domain is that multiple actors operate in parallel on the same set of data. A reservation system for concerts would be a good example of a collaborative domain – everyone wants the “good seats” (although it might be better call that competitive rather than collaborative, it is effectively the same principle).
You might be following these steps:
1- ReserveRequested -> ReserveRequestAccepted -> TicketReserved
2- ReserveRequested -> ReserveRequestRejected
When somebody clicks on the buy ticket button, you should create a reserve request entity, and then you can process the reservation in the background and by a queue system.
On the user side, you can return a unique reserve request-id to check the result of the process. So the frontend developer should fetch the result of process periodically until it succeeds or fails.
I need to update documents in a CosmosDb.
{
firstname: "...",
lastname: "...",...
lastmodified: "2022-01-13T12:06:18.712Z"
}
Due to "concurrency" ( i.e. multiple concurrent "updaters" ), the update ( or insertion, hence Upsert? ) should only take place if the data I update is "newer" ( data.lastmodified ) than the one already persisted.
What is the proposed way to achieve this?
In plain SQL I'd opt for:
UPDATE address SET ... WHERE address.lastmodified < newdata.lastmodified
or INSERT ON DUPLICATE KEY UPDATE
If Upsert had the possibility for specifying contraints when to effectively upsert ( i.e. address.lastmodified < newdata.lastmodified ), I'd use these. But I guess ItemRequestOptions is not meant for that?
"concurrency"-context: updates are being posted into a service bus queue and an event-triggered AzureFunction handles the Events. Chances are, that multiple Events for the same data end up "concurrently" in the queue and hence are being executed "concurrently"
Thx for your advices
Clemens ( being new to ComosDb et al )
This can be achieved using partial document update with conditional update. You can use Add or Set patch operations and specify your WHERE clause of WHERE address.lastmodified < newdata.lastmodified to specify whether it executes or not.
For more information see, Partial Document Updates in Azure Cosmos DB
I have a DynamoDB "My_Table" with index on "lock_status":
{
my_pk: string,
lock_status: string,
useful_data: string,
}
Is it possible for two different thread to execute the update code below on the same record?
Essentially, I want exactly one thread to have access to any given record's "useful_data". To do this, I'm "locking" the record via lockStatus while the thread is working with this item. What I am afraid is that two thread executes this code at the same time. They both find the same record based on the "ConditionExpression" and locks the same record.
const client = new AWS.DynamoDB.DocumentClient();
return await client.update({
TableName: 'My_Table',
Limit: 1,
UpdateExpression: 'set lockStatus = :status_locked',
ConditionExpression: 'lockStatus <> :status_available',
ExpressionAttributeValues: {
':status_locked': 'LOCKED',
':status_available': 'AVAILABLE',
},
ReturnValues: 'ALL_NEW',
}).promise();
This seems possible to avoid this problem if I was using TransactWriteItem, but can I get away with using simple update for my simple scenario?
Edit
Allow me to add a little context so that things make more sense. I'm building a "library" of reusable user accounts for testing. A test would "check out" and "check in" the user account. While the account is checked out, I want to prevent all other tests from using the same account.
One piece of information I neglected to mention in my original post was that I'm first getting the My_Table data by getting the next not locked item. Something like this:
const client = new AWS.DynamoDB.DocumentClient();
return await client.query({
TableName: 'My_Table',
IndexName: 'LOCK_STATUS_INDEX',
Limit: 1,
KeyConditionExpression: 'lockStatus = :status_available',
ExpressionAttributeValues: { ':status_available': 'AVAILABLE' }
}).promise();
Then in my subsequent update call, I'm locking the row as mentioned in my original post.
As #maurice suggested, I was looking into the optimistic locking. As a matter of fact, this article perfectly describes a scenario that I'm facing.
However, there is a problem that I will likely run into under high load. The problem goes something like this:
10 threads come and asks for the next not locked record. All 10 threads get the same record. (This is a very possible since all I'm doing is Limit 1 and the dynamoDb will likely return the first record it runs across, which would be the same for all threads).
10 threads try to update the same record with a give version number. One thread succeeds in the update and the rest fail.
9 threads retry and goes back to step 1. (Worst case, more threads are added)
I'm starting to think that my design is flawed. Or perhaps dynamoDb is not the right technology. Any help with this problem would be useful.
You could use optimistic locking for this - the idea is fairly simple.
You create a version attribute for your item that's an integer which will be incremented.
{
pk: 123
sk: 123
version: 0
randomValue: abc
}
When you read the item to update it, you note the current version number. After you update the item, you also increment the version number. So if you wanted to update the random value, the item you'll write to DynamoDB would look like this:
{
pk: 123
sk: 123
version: 1
randomValue: newValue
}
You now add a condition expression to your update or putitem call, to ensure this only succeeds, when the current version of that item is still 0.
That way the call will fail, if somebody else updated the item while you were processing it and you can read it again, update it and write again.
If the call succeeds, you know there has been nobody else that messed with the item.
I also wrote a more detailed blog post about this if you're curious: link
I'm encountering an issue where I have a function that is intended to require serialized access dependent on some circumstances. This seemed like a good case for using advisory locks. However, under fairly heavy load, I'm finding that the serialized access isn't occurring and I'm seeing concurrent access to the function.
The intention of this function is to provide "inventory control" for a event. Meaning, it is intended to limit concurrent ticket purchases for a given event such that the event is not oversold. These are the only advisory locks used within the application/database.
I'm finding that occasionally there are more tickets in an event than the eventTicketMax value. This doesn't seem like it should be possible because of the advisory locks. When testing with low volume (or manually introduced delays such as pg_sleep after acquiring the lock), things work as expected.
CREATE OR REPLACE FUNCTION createTicket(
userId int,
eventId int,
eventTicketMax int
) RETURNS integer AS $$
DECLARE insertedId int;
DECLARE numTickets int;
BEGIN
-- first get the event lock
PERFORM pg_advisory_lock(eventId);
-- make sure we aren't over ticket max
numTickets := (SELECT count(*) FROM api_ticket
WHERE event_id = eventId and status <> 'x');
IF numTickets >= eventTicketMax THEN
-- raise an exception if this puts us over the max
-- and bail
PERFORM pg_advisory_unlock(eventId);
RAISE EXCEPTION 'Maximum entries number for this event has been reached.';
END IF;
-- create the ticket
INSERT INTO api_ticket (
user_id,
event_id,
created_ts
)
VALUES (
userId,
eventId,
now()
)
RETURNING id INTO insertedId;
-- update the ticket count
UPDATE api_event SET ticket_count = numTickets + 1 WHERE id = eventId;
-- release the event lock
PERFORM pg_advisory_unlock(eventId);
RETURN insertedId;
END;
$$ LANGUAGE plpgsql;
Here's my environment setup:
Django 1.8.1 (django.db.backends.postgresql_psycopg2 w/ CONN_MAX_AGE 300)
PGBouncer 1.7.2 (session mode)
Postgres 9.3.10 on Amazon RDS
Additional variables which I tried tuning:
setting CONN_MAX_AGE to 0
Removing pgbouncer and connecting directly to DB
In my testing, I have noticed that, in cases where an event was oversold, the tickets were purchased from different webservers so I don't think there is any funny business about a shared session but I can't say for sure.
As soon as PERFORM pg_advisory_unlock(eventId)is executed, another session can grab that lock, but as the INSERT of session #1 is not yet commited, it will not be counted in the COUNT(*)of session #2, resulting in the over-booking.
If keeping the advisory lock strategy, you must use transaction-level advisory locks (pg_advisory_xact_lock), as opposed to session-level. Those locks are automatically released at COMMIT time.
I have requirements:
I have few heavy-resource-consume task - exporting different reports that require big complex queries, sub queries
There are lot users.
I have built project in django, and queue task using celery
I want to restrict user so that they can request 10 report per minute. The idea is they can put hundreds of request 10 minute, but I want celery to execute 10 task for a user. So that every user gets their turn.
Is there any way so that celery can do this?
Thanks
Celery has a setting to control the RATE_LIMIT (http://celery.readthedocs.org/en/latest/userguide/tasks.html#Task.rate_limit), it means, the number of task that could be running in a time frame.
You could set this to '100/m' (hundred per second) maning your system allows 100 tasks per seconds, its important to notice, that setting is not per user neither task, its per time frame.
Have you thought about this approach instead of limiting per user?
In order to have a 'rate_limit' per task and user pair you will have to do it. I think (not sure) you could use a TaskRouter or a signal based on your needs.
TaskRouters (http://celery.readthedocs.org/en/latest/userguide/routing.html#routers) allow to route tasks to a specify queue aplying some logic.
Signals (http://celery.readthedocs.org/en/latest/userguide/signals.html) allow to execute code in few well-defined points of the task's scheduling cycle.
An example of Router's logic could be:
if task == 'A':
user_id = args[0] # in this task the user_id is the first arg
qty = get_task_qty('A', user_id)
if qty > LIMIT_FOR_A:
return
elif task == 'B':
user_id = args[2] # in this task the user_id is the seconds arg
qty = get_task_qty('B', user_id)
if qty > LIMIT_FOR_B:
return
return {'queue': 'default'}
With the approach above, every time a task starts you should increment by one in some place (for example Redis) the pair user_id/task_type and
every time a task finishes you should decrement that value in the same place.
Its seems kind of complex, hard to maintain and with few failure points for me.
Other approach, which i think could fit, is to implement some kind of 'Distributed Semaphore' (similar to distributed lock) per user and task, so in each task which needs to limit the number of task running you could use it.
The idea is, every time a task which should have 'concurrency control' starts it have to check if there is some resource available if not just return.
You could imagine this idea as below:
#shared_task
def my_task_A(user_id, arg1, arg2):
resource_key = 'my_task_A_{}'.format(user_id)
available = SemaphoreManager.is_available_resource(resource_key)
if not available:
# no resources then abort
return
try:
# the resourse could be acquired just before us for other
if SemaphoreManager.acquire(resource_key):
#execute your code
finally:
SemaphoreManager.release(resource_key)
Its hard to say which approach you SHOULD take because that depends on your application.
Hope it helps you!
Good luck!