I have a client who is using the ECM and just dispatched an email blast to approximately 18,000 users. The dispatch is taking quite a while (about 2 hours to process not even half of the users).
Has anyone encountered this issue?
Can the ECM not handle such large lists?
As mentioned elsewhere, ECM can handle that load just fine. In general, throughput on ECM is limited by:
Fragmented indices on the "analytics" database (or just limited capacity on same)
Bandwidth limitations. If each mail is 500KB (lots of images), sending 10 mails per second requires 5Mbit bandwidth
CPU on the server in question
From what you've shared so far, I cannot guess as to which of the above is limiting the throughput on your installation. My personal guess would be capacity and speed of the database.
More information here: http://sdn.sitecore.net/upload/sdn5/products/ecm/200/ecm_tuning_guide_20-a4.pdf
I had a similar issue with a client where it was taking hours to send emails. Check the NumberThreads setting in the Sitecore.EmailCampaign.config file. The default is quite low at "1" and most servers should be able to handle more threads.
Definitely follow the tuning guide that Mark posted. The Performance Measurement Tool can help you get the ideal number of sending threads so that you're not over or under utilizing your server.
Related
I created the website for sending sms, but in some point, the report (success and fail) of sms is very slow.
After that, I tried to check out what is problem in query, but I could not find.
Here's the workflow that I experienced.
I sent 200,000 sms, and the report (pending to success or fail) was properly worked. 300 sms per a second report was updated from pending to success or fail
After sending about 200,000 sms, the report is messed up, but sending sms worked fine.
The report speed was about 1 sms report updated per a second
So, I checked the statistics of aws how it is working
Currently, I am using 4 CPU and 16 memory
At some point, the cpu using rate is close to the 100%, and network traffic was very high.
Is it problem with the query that I wrote? or should I increase the cpu, ram, and ssd?
I would love to know increasing cpu is the problem of code (query) or not.
If it is because high traffic of sending sms, CPU, and ram need to be increased or not.
Thank you
This is a non-trivial quetion.
Looking at the graphs of syetem performance, you can see that there are a few CPU spikes where CPU usage gets quite high, but they are breif spikes. The overall CPU usage isn't bad, outside of the brief spikes.
So, first, I'd look at the code. How does it work? Are there obvious places where the CPU may spike?
Without seeing the code or even knowing what language it's in, there's not much can do to help,
Suitable technologies I am aware of:
Websockets
Zeromq
Please suggest others if they are a better fit for my problem.
For this use case I have just two machines, the sender and the receiver, and it's important to note they are fixed "nearby" each other, as they will be in the same availability zone on AWS. Answers which potentially relate to message passing over large spans of the internet aren't necessarily applicable. Note also the receiver server isn't queuing these up as tasks, it will just be forwarding select message feeds to website visitors over a websocket. The sending server does a lot of pre-processing and collating to the messages.
The solution needs to:
Be very high throughput. At present the sending server is processing about 10,000 messages per second (written in Rust) without breaking a sweat. Bursty traffic may increase this up to 20,000 or a bit more. I know zeromq can handle this.
Robust. The communication pipe will be open 24/7 365 days per year. My budget is extremely limited in terms of setting up clusters of machines as failovers so I have to do the best I can with two machines.
Message durability isn't required or a concern, the receiving server isn't required to store anything, it just needs all the data. The sender server asynchronously writes a durable 5 second summary of the data to a database and to a cache.
Messages must retain the order in which they are sent.
Low latency. This is very important as the data needs to be as realtime as possible.
A websocket seems to get this job done for 1 to 4. What I don't know is how robust a websocket is for communication that's 24 hours a day 7 days a week. I've observed websocket connections getting dropped online in general (Of course I will write re-connect code, heartbeat mointoring if required but still this concerns me). I also wonder if the high throughput is too much for the websocket.
I have zero experience in this kind of problem but I have a very good websocket library that I'm comfortable using. I ruled out Apache Kafka as it seems expensive to get high throughput, tricky to manage with dev ops (zookeeper) and seems overkill as I don't need durability and it's only communication between 2 machines. So I'm hoping for a simple solution.
It sounds like you are explaining precicely what EC2 cluster placement groups provide: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-placementgroup.html
Edit: You should be able to create the placement group with 2 machines to cover your limited budget. Using larger instances, according to your budget, will also support higher network throughput.
Point 4 looks like it would be supported by SQS FIFO though, despite the fact that SQS FIFO queues only support up to 3,000 messages per second with batching.
A managed streaming solution like Kinesis Data Streams would definitely cover your use case, at scale, much better than a raw web socket. Using Kinesis Client Libraries, you can write your consumer to read from the stream.
AWS also has a Managed Kafka service to rule out the overhead and management of necessary components like Apache ZK: https://aws.amazon.com/msk/
I was recently reading about blockchains and am very intrigued by this technology. I had a few questions regarding blockchains:
Do Blockchains use web-sockets to transmit information between users? If yes then is the information(blocks) sent always a JSON object?
Do all users have the entire copy of the blockchain, do they each just see a partial copy of the blockchain? if yes then how big can the file get?
Also, what determines transactions/second? I read bitcoin does about 7transactions/seconds and what is needed to make them more scalable, is it coding factors such as writing a more efficient algorithm (big-O) or is it some kind of hardware limitation?
Sorry if these questions seem trivial but I am a newbie trying to learn the technology. Any help would be appreciated.
No, usually they use low-level protocol build on top of TCP.
Users should have an entire copy of blockchain in order to verify transactions. Database size of Bitcoin: 200 GB, Ethereum 660 GB. You can use lightweight clients, which don't have a full copy, but in this case, you are not part of the network.
In Bitcoin, there is a limit on block size, 1 MB. Average transaction size is about 400 bytes, so the average block contains 2000 transactions. There is no problem with increasing block size limit, this was done in Bitcoin Cash network (32 MB). But we cannot increase it to infinity since internet connection speed and transaction verification disk/CPU resources are not infinite.
If I have a server running on my machine, and several clients running on other networks, what are some concepts of testing for synchronicity between them? How would I know when a client goes out-of-sync?
I'm particularly interested in how network programmers in the field of game design do this (or just any continuous network exchange application), where realtime synchronicity would be a commonly vital aspect of success.
I can see how this may be easily achieved on LAN via side-by-side comparisons on separate machines... but once you branch out the scenario to include clients from foreign networks, I'm just not sure how it can be done without clogging up your messaging system with debug information, and therefore effectively changing the way that synchronicity would result without that debug info being passed over the network.
So what are some ways that people get around this issue?
For example, do they simply induce/simulate latency on the local network before launching to foreign networks, and then hope for the best? I'm hoping there are some more concrete solutions, but this is what I'm doing in the meantime...
When you say synchronized, I believe you are talking about network latency. Meaning, that a client on a local network may get its gaming information sooner than a client on the other side of the country. Correct?
If so, then I'm sure you can look for books or papers that cover this kind of topic, but I can give you at least one way to detect this latency and provide a way to manage it.
To detect latency, your server can use a type of trace route program to determine how long it takes for data to reach each client. A common Linux program example can be found here http://linux.about.com/library/cmd/blcmdl8_traceroute.htm. While the server is handling client data, it can also continuously collect the latency statistics and provide the data to the clients. For example, the server can update each client on its own network latency and what the longest latency is for the group of clients that are playing each other in a game.
The clients can then use the latency differences to determine when they should process the data they receive from the server. For example, a client is told by the server that its network latency is 50 milliseconds and the maximum latency for its group it 300 milliseconds. The client then knows to wait 250 milliseconds before processing game data from the server. That way, each client processes game data from the server at approximately the same time.
There are many other (and probably better) ways to handle this situation, but that should get you started in the right direction.
I am currently working on a newsletter application which needs to sent around 20K mails in a single shot. I feel, ColdFusion mail sending capacity will not be suitable for this. I don’t have any metrics with me to prove this.
I searched around in web about this, I found there are lot of complains about this kind of bulk mail sending process using built in CFMAIL. Few of them are,
Java heap size error
IOException while sending message
Mail Spool Lock
Mail Spool Timeout
We are using Adobe ColdFusion 9 standard edition.
Is anyone worked like this requirement? Did you faced any issues? How did you fixed that? Is any other alternative ways are there instead of using regular cfmail tag?
We have a system sending out nearly 500k messages about 2 to 3 times per day. CF does not have a problem with it if you have adequate resources allocated - meaning decent drives and space, adequate heap space, and (especially) a well performing SMTP relay. In our case we set up a dedicated relay server that we could fine tune carefully.
CF Mail has not had significant issues since ColdFusion 7. CF 8 and CF 9 both perform well. There is a level at which I would suggest a different system to be sure - and other requirements might mean an SaaS would serve you better - but from a per performance standpoint 20k messages is fairly small and should not be a problem for an enterprise class server.
I think you may face issue if you are using Standard version, I am not sure but there is some thread limit (I guess it it is 10) for spool for standard version which will delay email. Let's say there are 10 threads for spool and every thread wait for 15 seconds for next run (can be adjust from CFIDE) still it will take around 8.333 hours to send out all emails.
We have been using ColdFusion as a webmail delivery system since version 6. We have mail groups with 20k recipients.
We experienced one issue with it back then, the spooler hung and started retransmitting the same mail over and over again, making recipients receive the mail multiple times.
I think this issue was resolved with version 7.
If I recall well there's a difference between coldfusion standard and enterprise. Enterprise versions has enhanced CFMAIL throughput. At least it used to be in previous versions.