Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Does anyone know any free accessible clusters that are open to public and that use a Hadoop/MapReduce framework? There are plenty of tutorials of how to use MapReduce, but is there a way to test the examples without using my local single machine and installing the required framework?
Thanks!
Amazon EC2 has ready to use Hadoop cluster for per time rent, not very expensive even for play. Other way is to play with Cloudera Hadoop VM http://www.cloudera.com/downloads/virtual-machine/. You can run cluster on several virtual machines.
I will soon have a solution - it's not free, but it is VERY cheap.
I have built a small cluster for training and education (via web access) and will be live in May 2013.
I will rent out 4 node cluster for $2 a day or $10 a week.
Since the cluster is not very big, it will handle data sets of only 20-40GB, but will have full web access to run mapreduce, pig scripts.
Whilst I am asking for some money, it's not really a business - just hoping that I can pay the power bills!
http://jyrocluster.com
Regards,
Serge
You could also use Apache Whirr to deploy your own test cluster on Amazon EC2. This gives you more control than Elastic Map Reduce. It should be cheap if you are using it only to test map reduce jobs for short periods of time.
You can give CloudxLab a try. Though it is not free, it is quite affordable. It provides a complete environment to practice Hadoop, Spark, Kafka, Hive, Pig, HBase, Oozie, Zookeeper, Flume, Sqoop, Mahout, R, Linux, Python, Scala, NumPy, Scipy, scikit-learn etc. You will not have to install or configure any software on your local machine to use CloudxLab. Many of the popular trainers are already using CloudxLab.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
Improve this question
I've been thinking with my team which solution is the best for deploying Apache Airflow on AWS in terms of cost and performance. We did some research and found out some solutions, among them using Kubernetes (EKS), a machine on EC2 and using ECS (Fargate). But, on Google there isn't so much detailed contents about it. Also, we did some estimates based on our calculations, however, we're not so sure about that. We are looking for a discussion about the trade off of each solution.
So, my question is: is there someone who is going through this or someone who has been through this? And, which is, if exists, the best solution?
In late 2020, AWS announced Amazon Managed Workflows for Apache Airflow (MWAA). It is a fully managed service that makes it easy to run open-source versions of Apache Airflow (including v2) on AWS.
I'd suggest having a read through the documentation to find out more and determine if it meets your requirements.
From my personal experience: I had previously managed an Airflow stack using EC2 and ECS worker pools. Moving over to MWAA has definitely been a better solution & provided a much better user experience.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
As someone who has a laptop with insufficient processing power, I am having a hard time trying to train my neural network models. So, I thought of Amazon Web Services as a solution. But I have a few questions. First of all, as far as I know, Amazon SageMaker supports TensorFlow. I could not understand if the service is free or not though. I have heard some people say that it is free for a specific time, others say that it is free unless you surpass a limit. I would be more than happy if someone could clarify or put forward other alternatives that would help me out.
Thanks a lot!
Google cloud has similar options and they give $300 credit to developers.
Since google is the creator of tensorflow, I am guessing their cloud would be the one most up to date the latest. Try it out.
https://cloud.google.com/ml-engine/docs/pricing
They have a free tier, and this is all well documented at https://aws.amazon.com/sagemaker/pricing/
You should look into EC2 Spot Instances.
There is a market for AWS computing resources with prices rising and falling with supply and demand. You can set a max price as long as you are flexible on the availability. When the prices fall (usually at night), you can take advantage of (big data) computing resources at 90% off.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot-instances-work.html
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
Well, I want to know the basic of cloud infrastructure for Amazon cloud.
Can anyone help me how I can move ahead with this? and what would be the best for me?
Below mentioned is my requirement:
Project: Java EE based architecture
Deployment Server: Tomcat
DataBase: MySQL
Instance: Amazon ec2 and AWS Elastic Beanstalk (However I'm not sure what is
good for Java related project)
Space: 100 GB for now and it should be salable on the based on instant requirement.
Hosting Server: Linux
Here I want to know every possible things which can be good for initial setup for my production server.
Also, I would like to know what are the services that I need to purchase based on my requirement, and suggest me for the same, also guide me the best prices as well for the specific service.
Looking forward to hear from you everyone guys,
Have a nice time ahead!
Kuldeep
I would recommend you to start with Amazon EC2. For mysql, you can use Amazon RDS as well as it handles all the maintenance activity for DB. Also, you can start with m3.xl machines initially and can upgrade based on requirement.
As per storage is concerned, start with 100GB SSD EBS first . You can anytime attach more storage as needed.
Here are some useful links regarding machine types and costing:
https://aws.amazon.com/ec2/instance-types/?sc_channel=PS&sc_campaign=acquisition_SG&sc_publisher=google&sc_medium=ec2_b&sc_content=sitelink&sc_detail=ec2%20instance&sc_category=ec2&sc_segment=instance_types&sc_matchtype=p&sc_country=SG&s_kwcid=AL!4422!3!154807442051!p!!g!!ec2%20instance&ef_id=WG8jbAAAAIccH41L:20170703093155:s
https://calculator.s3.amazonaws.com/index.html
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Been looking around on the web for one of these, but I'm only finding node.js/ruby hosted implementations.
I'm keen to have a DynamoDB app, something like that of the navicat ilk that comes with an msi installer to get running quickly.
Maybe there's a valid reason as to why one of these doesn't exist yet?
If you are using DynamoDB local for development you can use the built-in DynamoDB shell: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.Shell.html
http://localhost:8000/shell
UPDATE: Amazon does also provide an official client: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html
I'm not aware of a dedicated desktop application for using Amazon DynamoDB from Windows yet.
However, the AWS Toolkits for Eclipse and Visual Studio both allow you to create tables, insert and edit data, initiate table scans, and more, straight from your local development environment (see the introductory post AWS Toolkits for Eclipse and Visual Studio Now Support DynamoDB).
So depending on your use case this might be all you need (or even better, if this is targeting a development scenario).
Update
The AWS team has just announced the option to Explore Your DynamoDB Tables Using the AWS Management Console as well:
You can now view and modify the contents of your DynamoDB tables from
within the AWS Management Console. With the addition of this new
feature, you can learn by doing -- trying out a number of DynamoDB
features without even writing any code. You can create a table, add
some items, and see them in the table, all through a very clean and
simple user interface.
This is obviously a very convenient addition to the rich client options referenced in my initial answer above, please check out the post itself for a short illustrated tour to help you get started.
RazorSQL Database GUI Tool for Amazon DynamoDB
http://razorsql.com/articles/razorsql_amazon_dynamodb.html
Amazon provides a local client for DynamoDb. It's a java jar and it runs on Linux and Windows.
It starts a server that binds to port 8000 at localhost (or you can specify another port when you start the server).
Download and unpack the desktop client, change the endpoint inside your app to localhost:8000, and then you can use the desktop client for DynamoDb.
Here you can download and read about installing the dynamodb desktop client.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 11 days ago.
Improve this question
We are planning to move the hosting to Amazon.
Has any one experienced their AWS Premium Support?
I am currently doing all admin stuff for the company and I was wondering if I signed up for this service it might ease my job a little.
Can they setup the whole hosting environment and install all the things like php mysql memcached load-balancing etc.?
How much of support can they give me?
Having used AWS for over a year now, I would say no. It is very expensive and whilst they are good for having a look if your server has gone down at hardware level or their general network status they are not so into actual server administration.
I would also say if you are not comfortable setting up the hosting architecture yourself than AWS may well not be for you. But if you want to get stuck in then do it properly as you can do a lot with it.
If you did want to setup something with all the features you require it is actually easier than you may think. I've listed out the software you want to run and how best to go about it (from experience)
php - Look at Amazon Community AMIs
(Amazon Machine Image -
http://aws.amazon.com/amis) - There
are loads setup with a LAMP stack pre
configured making it easy to get
going
mysql - If you want a dedicated MySQL
Server I seriously recommend Amazon's
RDS service (
http://aws.amazon.com/rds/ ) - it is
a great solution and really easy to
setup, it also automatically handles
things like backups, availability
etc.
memcached - Again plenty of community
AMIs with memcached pre-installed, if
you wanted this as a dedicated server
there are standalone AMIs aswell as
AMIs on a LAMP configuration
load-balancing - Amazon's Elastic
Load Balancing Service (ELB -
http://aws.amazon.com/elasticloadbalancing/)
makes load balancing really easy,
just select the ec2 instances you
want to put behind the load balancer
and it does the rest
The documentation is great and with the support on their user help board I have never had to use the paid support.
If you'd like to give AWS a trial before getting stuck in they have a free tier for developers
So in a nutshell, give it a go first before committing to any support packages. I think you'll be plesantly surprised at the ease of it.
One thing you should definitely look into is going to the AWS Tech Summits that they host. They're free and have great talks/tutorials from begginers to advanced users.