I want to write a script a to migrate data from one table to another using gin golang. The dataset is quite big so I can't create an API endpoint and do it as it will get timed out. I am looking an equivalent of django admin commands for golang that can help me to connect to the database easily and perform these task.
Related
I am new to Docker so I wonder if I have 2 containers, a streamer that pushes data to the queue and vernemq which is the message queue.
How can I access the data to work with it in my django rest api app?
I was wondering is there a way to export or the data and create a separate database in django with it or it's not how it works?
Or is there a way to directly access the data in docker from django app.
Note the django rest api app is not in a docker Image.
depends on how your current Django REST API is configured... if is looking at a Database? is it looking at a series of other your companies RESI APIs, etc?
I can edit this with more info once I get a little more details. Don't have enough reputation to write a comment.
BUT I would say the best way if the Django app is looking at a database is to write a script to transfer code from the messaging queue and save to the database.
What I would say though is instead of using a message queue to just have the streamer push it directly to where you have your Django app looking.
I am not able to solve a problem:
I want to run a python script which is scraping information from the web according to an input.
This input will come from Wordpress (Either from the frontend or database of Wordpress)
After getting input from, I figured out that AWS Lambda can do the scraping automatically.
Now the problem is, how can I connect my database of Wordpress with Lambda. API Getaway would get information from that Wordpress frontend, save it in DynamoDB and Lambda could then run the Python code. BUT, the input is user specific. So the user should be able to change that input when he wants it.
I am having a flask application which uses airflow data.I would like to access the airflow database from my flask application and query the data.I am able to run raw SQL query.But i should have a solution how to use SQL alchemy query technique in it. For example if i need the dag table data. I should be able to run Dag.query.all() to get all dag data.
Airflow uses SQLAlchemy (currently 1.1.x for the Airflow 1.10 trunk) to manage its models, so you can interact with and query them using the standard SQLAlchemy API as documented in the tutorial.
https://docs.sqlalchemy.org/en/rel_1_1/orm/tutorial.html
You can access your database using a standard database connection string from any Python code, such as your Flask app, as described in the above tutorial as well.
There are examples of various the database connection URL formats supported by create_engine() here.
https://docs.sqlalchemy.org/en/rel_1_1/core/engines.html#database-urls
You can find more info such as the fields on a particular model in Airflow's models.py file.
https://github.com/apache/incubator-airflow/blob/master/airflow/models.py
You can find more SQLAlchemy documentation here.
https://docs.sqlalchemy.org/en/rel_1_1/
I'm working on a Django Rest Framework project. I successfully implemented Elasticsearch dsl within my project. I indexed and synced all my models with Elasticsearch.
Now I can fetch my data both with Elasticsearch (and make searches on it) and with Django Rest Framework.
I want to know what is better to fetch data : DRF or Elasticsearch.
Thx
Depends... Elasticsearch is the quickest way to fetch data as it's NoSQL but to perform complex query you might want to use DRF/Python.
It relies on your requirement.
Is there a way to use GQL and Google App engine data store with Django? If yes then how to use it, I have searched on web but I am unable to find any satisfying answers to this problem?
In case of both cloud-sql and django-nonrel we dont use GQL as query language. They use SQL and MySql. But I wanted to use GQL in Django. So when I looked at documentation of Google app data store and GQL for google app engine webapp2 I found out that it has nothing to do with webapp2, instead it was for python. So only thing which I changed to make it working for Django was the method of rendering the request.