Though it is not a direct programming related query, I was looking out for the approach one should have if web services fail in Service Now application. How to handle the data at source during the web service outage and how to trigger data transfer after the resolution of web service incident?
Posted the same question in Service Now community and the response is
"Service Now operates 16 data centers and availability is 99.995%. You can try store and process functionality at source during Service Now down time."
The business continuity is planned in such a way that the web-service will failover to another DC causing no disruption to the users/applications.
So the risk is very minimal (infact near to 0) and the scenario can be ignored.
Related
Want to keep this question generic and expected answer in terms of best practice/approach/guidelines,
We need to know the best way to performance test and load test AWS cloud based applications.
What we have tried:
We used Gatling and Jmeter to execute our performance tests. These frameworks are pretty useful to test our functionality and to benchmark our applications latency and request rate.
Problem:
Performance benchmarks and limits of AWS managed services like Lambda and DDB are already specified by AWS e.g. Lambda concurrency behavior and DDB autoscaling under load etc. AWS also provides high availability and guaranteed performance of managed services.
Is it really worth executing expensive performance test and load test jobs for AWS managed services?
How to ensure that we are testing our application and not actually testing AWS limits which are already known.
What is the best practice and approach to performance test cloud based applications.
Any suggestions will help tremendously.
Thanks,
It depends on how you can use the AWS infrastructure
If you don't use Auto Scaling you can treat the AWS cloud-based application as a "normal" application just deployed not on your company premises, but somewhere at Amazon
If you use Auto Scaling you might want to come up with a some form of scalability testing. AWS instances can scale up in order to adapt to the increased load but your application must be able to scale as well. So you can test:
how fast are scale-up and scale-down processes, i.e. if you rapidly increase the load do AWS/your application react fast enough or there will be performance drop
scalability factor. For example with 100 virtual users you have 50 requests per second. Ideally with 200 virtual users you should have 100 requests per second, with 300 virtual users - 150 requests per second. Response time should remain the same. But normally the situation differs from the "ideal" so it would be good to know this scalability factor.
Also if your application is behind the AWS ELB you will need to add DNS Cache Manager to your JMeter test plan otherwise you will be hitting only one backend instance while others will be idle.
I am building infrastructure primitives to support workers and http services.
workers are standalone
http services have a web server and a load balancer
The way I understand it, a worker generally pull from an external resource to consume tasks while a service handles inbound requests and talks to upstream services.
Celery is an obvious worker and a web app is an obvious service. The lines can get blurry though and I'm not sure what the best approach is:
Is the worker/service primitive a good idea?
What if there's a service that consumes tasks like a worker but also handles some http requests to add tasks? Is this a worker or a service?
What about services that don't go through nginx, does that mean a third "network" primitive with an NLB is the way to go?
What about instances of a stateful service that a master service connects to? The master has to know the individual agent instances so we cannot
hide them behind a LB. How would you go about representing that?
Is the worker/service primitive a good idea?
IMO, the main difference between service and worker can be that workers should be only designated to one task but service can perform multiple tasks. A service can
utilize a worker or a chain of workers to process the user request.
What if there's a service that consumes tasks like a worker but also handles some http requests to add tasks?
Services can be of different forms like web service, FTP service, SNMP
service or processing service. Writing the processing logic in service may not be a good idea unless it is taking the form of a worker.
What about services that don't go through nginx, does that mean a third "network" primitive with an NLB is the way to go?
I believe you are assuming service to be only HTTP based but as I mentioned in the previous answer, services can be of different types. Yes you may write a TCP service for a particular protocol implementation that can be attached behind an NLB
What about instances of a stateful service that a hub service connects to? The hub has to know the individual instances so we cannot hide them behind a LB.
Not sure what you mean by hub here? But good practice for a scalable architecture is to use stateless servers/services behind. The session state should not be stored in the service memory but should be serialized to a data store like DynamoDB.
One way to see the difference is to look at their names - workers do what their name says - they perform (typically) heavy tasks (work) - something you do not want your service to be bothered with, especially if it is a microservice. - For this particular reason, you will rarely see 32-core machines with hundred GBs of RAM running services, but you will very often see them running workers. Finally, they complement each other well. Services off-load heavy tasks to the workers. This aligns with the well-known UNIX philosophy "do one thing, and do it well".
Just curious if this is possible or how you would accomplish this.
Regardless if I use duration based stickiness or application based, when the instance a user is connected to fails their session gets reset because they have to connect to a new server.
Is there a way to not have this happen? To be able to have that session persist even if the instance they are connected to dies? Im also using SSL with a cert if that changes things.
The only way to accomplish that is persisting your session state in some Storage service, could be a database table, s3, Caching service, NoSQL table, Etc.
These are some approaches
Session state Inside Your Database
Saving session state inside the database is common in lightweight web frameworks like Django. That way you can add as many front servers as you like without having to worry about session replication and other difficult stuff. You don’t tie yourself to a certain web server and you get persistence and all other features databases provide for free. As far as I can tell, this works rather nicely for small to medium size websites.
The problem is the usual: The database server may become your bottleneck. In that case your best bet may be to take a suitcase full of money to Oracle or IBM and buy yourself a database cluster.
Reference: Saving Session Data in Web Applications
Session state inside a Caching service
Amazon ElastiCache offers fully managed Redis and Memcached. Seamlessly deploy, operate, and scale popular open source compatible in-memory data stores. Build data-intensive apps or improve the performance of your existing apps by retrieving data from high throughput and low latency in-memory data stores.
DynamoDB
Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. It is a fully managed cloud database and supports both document and key-value store models. Its flexible data model, reliable performance, and automatic scaling of throughput capacity.
Regardless the approach you use, a middleware must be deployed along with your app to manage the stored session state.
Middleware: Could be either a thrid-party solution or your own solution.
Resources
AWS Session Management
Amazon ElastiCache
Amazon DynamoDB
Middleware for session management (Google results)
So I'm digging into AppFog's pricing sheet and I am struggling to figure out which one I need. here
Several questions
What is a service instance?
How do I somehow translate "service instance" to web traffic capabilities?
I am deploying a web app soon and looking at something to handle serious traffic (around a million users a month)
Being in a similar situation as you I think that a possible hint is the Pagodabox traffic visualizer. Also from one of the Pagodabox user stories I've concluded that having 16 service instances with 400 MB RAM each can handle 1M users (assuming that you have a reliable DB, and probably using Memcached).
(In AppFog the Services menu in the admin portal is really misleading - count instances in the Applications menu)
Is there any difference for billing if application on AWS is web java application that communicate with outside world through web service or servlet or is i.e console java application with opened socket server and someone outside is connecting to it?
Assuming that all things are equal then the short answer is: No - there is no difference.
AWS bills you based on resource usage, so a web service, servlet or console application will still require an EC2 instance to run on. Assuming they can all run on a given instance type (e.g. micro) then you'll be billed at the hourly rate of the EC2 instance type.
If you wanted to get pedantic, then the solution that would send the least amount of data out would be the cheapest. When considering this remember that it doesn't matter if you connect in, or it connects to you, what matters is the number of bytes transmitted out from your instance. So a solution that support compression of the transmitted data would be better than a solution that didn't.