I was hoping I could get some direction about creating a website using AWS that will run a python script. I created an EC2 Instance running Ubuntu and made it talk with a relational database made with the same account.
In a nutshell, the site I am creating is a YouTube Library of captions. The user will input a title and AWS will retrieve links to XML documents that contains the captions to the related videos from YouTube. I would like to know where and how to run a Python script to scrape the text from these XML documents every time a user makes a request.
My research has taken me in multiple directions, but I am not sure what is best for my purpose. For example, I am trying to run a remote script from GitHub, but don't know if there's a better way to store the script?
It's my first time working with AWS so please keep explanations simple. Thanks!
Related
I have started exploring AWS SageMaker starting with these examples provided by AWS. I then made some modifications to this particular setup so that it uses the data from my use case for training.
Now, as I continue to work on this model and tuning, after I delete the inference endpoint once, I would like to be able to recreate the same endpoint -- even after stopping and restarting the notebook instance (so the notebook / kernel session is no longer valid) -- using the already trained model artifacts that gets uploaded to S3 under /output folder.
Now I cannot simply jump directly to this line of code:
bt_endpoint = bt_model.deploy(initial_instance_count = 1,instance_type = 'ml.m4.xlarge')
I did some searching -- including amazon's own example of hosting pre-trained models, but I am a little lost. I would appreciate any guidance, examples, or documentation that I could emulate and adapt to my case.
Your comment is correct - you can re-create an Endpoint given an existing EndpointConfiguration. This can be done via the console, the AWS CLI, or the SageMaker boto client.
https://docs.aws.amazon.com/cli/latest/reference/sagemaker/create-endpoint.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_endpoint
I need last 3 months AWS billing data in graph using python and python Flask.
i found some articles for this, i just created environment in my local machine. after i dont know how to take billing data using python & Python Flask script. Any one have idea please help me.. Thanks in advance.
You can use Boto to connect to AWS using Python.
In general you need to cover several items:
Authentication - You'll need to set up your credentials to be able to connect to AWS. Check the Documentation
Enable DBR or CUR reports in the billing console. This will export your monthly billing information to S3. Check the Documentation
Once ready use Boto3 to download the reports and import them to DB, ES, whatever you're working with to process large excel files. Check the Documentation
Good luck!
EDIT:
For previous months you can just go to console -> bills and download the reports from the console directly, then process them in your application.
I have a Foswiki wiki on a server. Is it possible to script the following without FTP access (for various reasons I can't use it):
Download a topic's wikitext, modify it locally, then upload it again (overwriting the topic)
Upload wikitext to a new topic
I've been doing these tasks manually, but I'd like to automate them. I've looked into the Foswiki API and a few plugins, but nothing seems capable of doing this.
Is there a way? (any programming language)
If you have web access, you could drive the bin/view and bin/save scripts remotely from a script.
Take a look at our BuildContrib upload target for an example. It gets a strikeone key and downloads the original topic to recover any form data. It then uploads the topic text, creating a new version. It's written in perl, and uses LWP.
https://github.com/foswiki/distro/blob/master/BuildContrib/lib/Foswiki/Contrib/BuildContrib/Targets/upload.pm
The following isn't(!) the right solution (sure exists an nice Foswiki-way approach), but if you know perl, you can do anything with the:
Install Firefox
install MozRepl addon into it
Install the WWW::Mechanize::Firefox perl module
Now, you can script anything what you can do directly from the browser, e.g. logging into the Foswiki, click buttons, save topics, etc..etc. Drawback - it isn't an easy way - you need to know many details.
Myself using this technique for testing.
I am looking for a example where a fully completed web app can be embedded into amazon mechanical turk. I am working on a "game-like" activity that does not really belong to a form structure.
Here is my game/activity:
http://52.91.100.69:3030/
I would like to embed such tasks inside mechanical turk. My code accepts url parameters such as assignmentId, workerId etc (which I have found form the aws mturk docks)
For example:
http://52.91.100.69:3030/?assignmentId=23423&workerId=34&hitId=455
Basically, I am handling all the data logging etc, I plan to generate codes for users to enter upon completion of a number of tasks.
I would like to know how I cam accomplish this? Preferably in python (Boto)?
I looked at this tutorial: http://kaflurbaleen.blogspot.com/2014/06/in-which-i-battle-mturk-external-hits.html
Using this I made this boto file: https://gist.github.com/arendu/631a416e4cb17decb9dd
When I run it I dont see any errors, but I can't seem to find out whether the hit is available? I checked my aws mturk requester console (looked at manage HITs individually) but no hits are present.
What am I doing wrong?
I am building a small financial web app with django. The app requires that the database has a complete history of prices, regardless of whether someone is currently using the app. These prices are freely available online.
The way I am currently handling this is by running simultaneously a separate python script (outside of django) which downloads the price data and records it in the django database using the sqlite3 module.
My plan for deployment is to run the app on an AWS EC2 instance, change the permissions of the folder where the db file resides, and separately run the download script.
Is this a good way to deploy this sort of app? What are the downsides?
Is there a better way to handle the asynchronous downloads and the deployment? (PythonAnywhere?)
You can write the daemon code and follow this approach to push data to DB as soon as you get it from Internet. Since your daemon would be running independently from the Django, you'd need to take care of data synchronisation related issues as well. One possible solution could be to use DateTimeField in your Django model with auto_now_add = True, which will give you idea of time when data was entered in DB. Hope this helps you or someone else looking for similar answer.