SRS for the system I'm currently working on includes the following non-functional requirement: "the SuD shall be scalable to 200 concurrent users". How can I convert this statement to a more measurable characteristic: "hits per second"?
Assuming you're talking about a web application (based on your desire to estimate "hits" per second), you have to work on a number of assumptions.
- How long will a user spend between interactions? For typical content pages, that might be 10 seconds; for interactive web apps, perhaps only 5 seconds.
- Divide the number of users by the "think time" to get hits per second - 200 concurrent users with a think time of 10 seconds gives you 20 concurrent users on average.
- Then multiply by a "peak multiplier" - most web sites are relatively silent during the night, but really busy around 7PM. So your average number needs to take account of that - typically, I recommend a peak of between 4 and 10 times.
This gives you a peak page requests per second - this is usually the limiting factor for web applications (though by no means always - streaming video is often constrained by bandwidth, for instance).
If you really want to know "hits", you then need to work through the following:
- How many assets on your page? Images, stylesheets, javascript files etc. - "hit" typically refers to any kind of request, not just the HTML page (or ASPX or PHP or whatever). Most modern web apps include dozens of assets.
- How cacheable are your pages and/or assets? Most images, CSS, JS files etc. should be set to cacheable by the browser.
Multiply the page requests by the number of non-cacheable assets. Add to this the number of visitors multiplied by the number of assets if you want to be super precise.
All of this usually means you have to make lots and lots of assumptions - so the final number is an indicator at best. For scalability measurements, I usually spend more time trying to understand the bottlenecks in the system and observing the system under load.
Well that's impossible to answer without knowing anything about your app or what it does. You need to figure out how many hits per second one user is likely to make when using the app, and multiply by 200.
Incidently, hits/second is not the only metric you need to be concerned with. With 200 concurrent users how much memory overhead will that be? How much disk access or open file handles? How many db reads/writes? How much bandwidth (does the app involve streaming media)? Can it all be handled by one machine? etc etc
Related
When using the Django debug toolbar, it says a page might load in say 4000 ms. But when we reload the page (with ctrl+F5 to clear the cache) it says it loads in say 4400 ms -- or 3600 ms. Is there a more accurate way to benchmark the load time? The reason is that we want to optimize page load times and want to make sure that we can see cause and effect clearly.
There will always be some variation in the amount of time it takes a program to do anything--on a typical computer there are tens to hundreds of processes simultaneously competing for resources, so the exact load time will vary depending on how much else is going on at that exact moment.
The best way to benchmark is not to look at the time take by a single page load, but rather the average time over a bunch of loads. There are many tools to help you do that--Apache jMeter is one.
You may also want to look into profiling your app rather than just measuring the overall load time--that will help you identify which bits of your code are called most frequently and contribute the most to the total time taken. Guess-and-check optimizations are likely to be much more time consuming. See the Django docs or Google "profiling django" many more resources.
I'm aware that a web application should render it's pages as fast as possible, with few database requests only in milliseconds. What are the guidelines about this response time (like Microsoft guidelines for UI or something like that)?
What is the absolute maximum time a webpage should respond under?
Are there any "limits" or general guidelines for this?
When should I put jobs into task queues (like Python celery for example)?
My concrete problem is that I have to parse a bunch of text files, which users submits. The average time these can be parsed are 2-3 seconds (response times are 3-4sec with database inserts) but if the file is very big, it takes 8sec to parse (10sec to respond).
Is it okay to let the user without feedback for this time? If not, what is the best way for handling these kinds of situations?
Is it even okay to put these in the request-response cycle ?
Is there any difference if I provide a REST API vs a website form ? Are these "allowed" slower to respond?
I think this is really hard to answer. Different guidelines exist.
When I was at university during interface / interaction design courses I learned that no user should be left with response times over 50 ms.
If that is exceeded, something like a loading icon etc should be displayed.
Also users are educated enough to expect right loading times form websites... So the user will accept 2 seconds loading time for a ticket booking page but not accept more than 300 ms from a search engine.
The limits I hear about during this days are 0.1 sec, 1 sec and 10 sec.
0.1 feels instantly to the user on websites
1 sec is slow but no interruption
10 sec is the maximum for the user to endure before loosing attention (for example light a smoke, check facebook feed in the meantime, etc...)
There is a nice article along with a lot of useful comments which I lately read which I would like to point you to:
http://www.nngroup.com/articles/response-times-3-important-limits/
I think it answers your questions well.
Please understand that this is all purely subjective but I think this is a very subjective topic...
I often heard this question coming from different sources, but never got a good idea of the technologies to achieve this. Can anyone shed some lights? The question is: you have a website which has high volume of users access per day. Your website is deployed in a distributed manner, have multiple webservers and load balancers responding incoming requests from lots of locations. How do you get the 1000000th user access, and show him a special page saying "congrats, you are our 1000000th visitor!". Assuming you had a distributed backend.
You could do it with jQuery, for example:
$("#linkOfInterest").click(function() { //code for updating a variable/record that contains the current number of clicks });
CSS:
a#linkOfInterest {
//style goes here
}
somewhere in the html :
<a id="linkOfInterest" href="somepage.htm"></a>
You are going to have to trade off performance or accuracy. The simplest way to do this would be have a memcached instance keep track of your visitor counts, or some other datastore with an atomic increment operation. Since there is only a single source of truth, only 1 visitor will get the message. This will delay the loading of your page by the roundtrip to the store at minimum.
If you can't afford the delay, then you will have to trade off accuracy. A distributed data store will not be able to atomically increment the field any faster than a single instance. Every web server can read and write to a local node, but another node at another datacenter may also reach 1 million users counts before the transactions are reconciled. In that case 2 or more people may get the 1 millionth user message.
It is possible to do so after the fact. Eventually, the data store will reconcile the increments, and your application can decide on a strict ordering. However, if you have already decided that a single atomic request takes too long, then this logic will take place too late to render your page.
I have some results obtained through WS requests from a couple of different providers, then i gather and order the results and i show them at the user.
The number of the results is somewhere between 0 and 60-70, with an average of 10-20.
My problem is: how to handle pagination?
I'm trying to figure out which is the best solution for my situation, because i have find out several ways to do that... and I am sure I am missing other good (probably better) solutions... The solutions i thought until now:
1)Making for each page (15 results) a new aggregated search through the WebServices. This is stupid, but since the average number of results is 10-20, the pagination wont be used often.
2)Saving in the database all the results as a temporary cache and then showing 15 results at time
3)Loading all the results in a single page but showing only 15 a page using a Jquery pagination plugin (client side?)
It depends how big is 1 result, but I'd prefer no pagination if you have max 60-70 results, especially if it's not often. Better user experience.
Are you really sure that someday the web services aren't going to start returning a lot more results? What if someday there is a bug in one of them where it accidentally returns 50,000 copies of the same result to you? In each of your solutions:
A larger than expected number of results would cause you to spam the web services with repeated requests for the same results, as users page through them.
A larger than expected number of results will end up temporarily taking up space in your database. Also, in a web app, how will you know when to clear the cache?
A larger than expected number of results will end up as a huge page in the user's browser, possibly not rendering correctly until the whole thing is downloaded.
I really like option 3. The caching is done at the place where the data is wanted, there are no redundant hits to the web services, and paging will be super fast for the users.
If you're really certain no more than 60-70 results will ever be returned, and/or that your users will never want a really large number of results, you could combine option 3 with a cap on the number of results you will return.
Even in the worst case where the web services return erroneous/unexpected results, you could trim it to the first so many, send them down to the browser, and paginate them there with JavaScript.
I currently have a django site, and it's kind of slow, so I want to understand what's going on. How can I profile it so to differentiate between:
effect of the network
effect of the hosting I'm using
effect of the javascript
effect of the server side execution (python code) and sql access.
any other effect I am not considering due to the massive headache I happen to have tonight.
Of course, for some of them I can use firebug, but some effects are correlated (e.g. javascript could appear slow because it's doing slow network access)
Thanks
client side:
check with firebug if/which page components take long to load, and how long the browser needs to render the page after loading is completed. If everything is fast but rendering takes its time, then probably your html/css/js is the problem, otherwise it's server side.
server side (i assume you sit on some unix-alike server):
check the web server with a small static content (a small gif or a little html page), using apache bench (ab, part of the apache webserver package) or httperf, the server should be able to answerat least 100 requests per second (of course this depends heavily on the size of your test content, webserver type, hardware and other stuff, so dont take that 100 to seriously). if that looks good,
test django with ab or httperf on a "static view" (one that doesnt use a database object), if thats slow it's a hint that you need more cpu power. check cpu utilization on the server with top. if thats ok, the problem might be in the way the web server executes the python code
if serving semi-static content is ok, your problem might be the database or IO-bound. Database problems are a wide field, here is some general advice:
check i/o throughput with iostat. if you see lot's of writes then you have get a better disc subsystem, faster raid, SSD hard drives .. or optimize your application to write less.
if its lots of reads, the host might not have enough ram dedicated as file system buffer, or your database queries might not be optimized
if i/o looks ok, then the database might be not be suited for your workload or not correctly configured. logging slow queries and monitoring database activity, locks etc might give you some idea
if you let us know what hardware/software you use i might be able to give more detailed advice
edit/PS: forgot one thing: of course your app might have a bad design and does lots of unnecessary/inefficient things ...
Take a look at the Django debug toolbar - that'll help you with the server side code (e.g. what database queries ran and how long they took); and is generally a great resource for Django development.
The other non-Django specific bits you could profile with yslow.
There are various tools, but problems like this are not hard to find because they are big.
You have a problem, and when you remove it, you will experience a speedup. Suppose that speedup is some factor, like 2x. That means the program is spending 50% of its time waiting for the slow part. What I do is just stop it a few times and see what it's waiting for. In this case, I would see the problem 50% of the times I stop it.
First I would do this on the client side. If I see that the 50% is spent waiting for the server, then I would try stopping it on the server side. Then if I see it is waiting for SQL queries, I could look at those.
What I'm almost certain to find out is that more work is being requested than is actually needed. It is not usually something esoteric like a "hotspot" or an "algorithm". It is usually something dumb, like doing multiple queries when one would have been sufficient, so as to avoid having to write the code to save the result from the first query.
Here's an example.
First things first; make sure you know which pages are slow. You might be surprised. I recommend django_dumpslow.