I have about 200Gb of files on a Windows server file share that i want to upload in to S3. From looking at the documentation, it looks like i have to write my own API to do.
I'm an networking and server guy - I have no coding experience. Can anyone point me in the right direction for getting started? Has anyone ever done this before, who could maybe give me the high level overview of the steps involved, so that I can go an research each step? Any info will be greatly appreciated. Thanks.
Related
I'm working on a project where I am tasked to use google cloud services to process and visualize fitness data. For example, I have exported some apple health data from my watch, and it is in .xml format. From a high level, I envision this .xml file starting off in object storage, and being converted to .csv through a cloud function (triggered by the creation of the .xml object in storage) and stored again in object storage (different bucket). Then I see these .csv files being processed by a DataFlow pipeline, which will reformat the data to the template schema that I would like the data to be organized with. This pipeline will output the resultant .csv to BigQuery, which will then be designated as a data source for Data Studio. I will then configure Data Studio to produce some simple reports that compare the health data to recommended values. I would like for this report to be accessible as a .pdf in object storage potentially as well. Am I on the right track, or am I missing some key services to accomplish this?
Also, I'm new to posting on StackOverflow, so if this question is against the rules or not welcome, please let me know.
Any feedback is greatly appreciated, as I have not been able to bounce these ideas off of other experienced cloud architects/developers.
This question is currently off-topics by the rule of StackOverflow, as it does not contain any problems to resolve. See point 4-5.
As a high-level advice, I do not see why it should not be possible based on the services you mentioned but you would need to implement it and try it on your side and evaluate the features of each service in your workflow.
In terms of solution or architecture advice, those are generally paid services and you would most likely find little help here for those unless you have a specific problem to solve with said services. You might find some help on the internet as well. ie.Cloud Solutions, Built it on GCP, etc
You might find this interesting to review as well as it mimics your solution. Hope this helps.
I have a website that I am trying to have the web form connect to a MySQL database running on Amazon's RDS to post and retrieve information. I'm an absolute beginner with code but have managed to get myself this far (creative3c.org). I've had coworkers and friends offer some help, but their knowledge doesn't extend to everything I was told I would need (AWS API Gateway, Lambda--anything else?).
I've been pulling my hair out for a week looking through tutorials, articles, and step-by-step guides but so many presume extensive knowledge on the viewer or they all talk about what I don't need (like phpmyadmin--and php won't work for S3 or Lambda).
Am I jumping too far into the really complex stuff? The person that told me to go the AWS route is certified and brilliant with code--but unfortunately they are fickle, busy, and aren't a good teacher to distill their knowledge. I don't know if I should have gone with something simpler. If you view the website, you'll probably understand how basic it is.
I'm stuck and really stressed about finishing this website and appreciate the help to get me in the right direction! I feel I'm so close! I'm really good at scaling up from a small example of exactly what I need--I just need that initial example!
I'm pleased to hear that you've learnt so quickly. All the terminology floating around can be very confusing. Just remember: AWS is just the platform you deploy to. It can be as simple and complicated as you want it to be
I'm not an AWS expert but here's my birdseye view
You could build an entire running website on your laptop then simply deploy that wholesale to a LAMP server that you've created up in AWS. Now you have a web application running in AWS, without using any of the AWS jargon (beanstalks, lambdas...)
Thats when you would follow this link to provision your server: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-LAMP.html
Or you could put the database piece of your application into RDS (a database on the cloud) then put the web application piece in a seperate web server then configure those two servers to talk to each other.
You have a web site but it's now running on two seperate machines
Or (I'm a bit hazy on this) for the web app you could instead deploy bits of your web code to lambda and stick them all together
In all cases you can apply 'elastic beanstalk' to automatically grow and shrink the computers running your site.
Like I said it can be as simple and complicated as you want it to be - and you don't need it to be complicated so the BlueHost option is fine.
I've been a mobile developer for a few years, and Im looking to expand to cloud integration with my apps. Im looking into AWS solutions to fill this need. I don't know a ton about servers or cloud capabilities, so I'm trying to get pointed in the right direction, and maybe be introduced to some good resources.
My goal is to be able to upload some images to AWS and manipulate these images in the cloud. I'm sure that I'll need S3 to store my images, but is an EC2 instance the correct thing to use to perform the manipulation? This is where my lack of knowledge of servers is holding me back.
I think that the best answer I could get would be a comment on whether my needs from AWS are what I listed above, and a point in the right direction towards articles to tutorials of how to get things up and running.
Thanks much for the help!
What I ended up doing was using AWS Lambda to accomplish what I needed. Running a node.js based lambda function with ffmpeg-like manipulation on the images/media that I was uploading worked out quite well.
Side note :: The processing that I was doing was fairly lightweight, so it worked well with lambda. If things scale up any further I might consider switching the processing to an EC2 instance.
I need to write a C/C++ program that uploads videos to ScreenCast.com. For uploading videos to YouTube there is a great manual that explains how it can be done with an http request. I can't find something like that about Screencast.
Maybe someone has experience with this or some related tasks. I'm quite new to programming, and would be very grateful for any advice or thoughts.
Thanks!
The right answer is that for now it is impossible. I have contacted screencast.com support, and they told me that they are going to make a puplic API somewhen in future
I'm working with Amazon's Product Advertising API and need some help on a few issues. The questions I have posted on SO and Amazon developer forums have gone unanswered. Where can I find help?
Looks like your best bet is going to be to pay amazon for support: http://aws.amazon.com/premiumsupport/
My guess is that there aren't many people using that API hence the problem getting regular programmers to help you. This is evidenced by a notice on the main AWS page (https://affiliate-program.amazon.com/gp/advertising/api/detail/main.html) about a number of discontinued API operations and responses due to low usage.
"low usage" in this scenario is usually marketing speak meaning "no one except that one guy in Minnesota".