I have successfully set up and used Django with MySQL on my local machine, but now if I put my project on say GitHub what should the requirements be for any other person to be able to run it?
I am asking this because my project uses a database that I have stored in my local machine and while uploading the project on GitHub I have not uploaded the database. in sqlite3 there is a database file inside the project itself but this does not happen for MySQL whose database is stored in a different location.
I mean Django accesses the database from a different location(var/lib/MySQL) and when I try to copy the database from there to the project folder and specify its location in settings.py, I get an access denied error.
So how can I solve this?
You would typically have a seed file for others to use. Others will create a database on their own systems and use your seed file to get started with the project.
It should not be necessary to copy the database files. Also, you should not just copy the MySQL directory like that. If you copy the whole directory then you might replace what somebody already has on their system, but if you copy only some of the files then you might be missing things like the MySQL user accounts. Besides, that is a backup procedure, not a deployment or distribution procedure.
For somebody else to get started with your project the normal process is:
manually create the appropriate MySQL user and database (or provide a script to automate it)
Run migrations: python manage.py migrate
Import initial data:
This can be with fixtures: python manage.py loaddata my_data.json
Or with a custom management command: python manage.py load_my_data
However, if you really need to provide somebody with an almost ready database, you can use mysqldmp which will produce a SQL text file, but the other person still needs to create the user account manually.
I want to add some with himank that if you need to provide some additional data for database you can up your fixture-datalink in fixture folder. Then other person will able to load those manually with command or even able to run a script link to populate initially data to database.
Related
I'm using a 3rd party dependency modeltranslation-lokalise which requires me to run python manage.py makemigration. This will generate a migration file that sits within the package directory once generated, and therefore are not source controlled.
For all other migration files I'm keeping them in source control and would like to do the same for these, as it will allow us to track changes to the database in the environments we use, ensuring they can be easily replicated and that operations are not duplicated on the same database (e.g. by having the same operation exist in differently-named migrations).
My question: How are these 3rd party migrations meant to be handled when building and deploying to production (e.g. via a Dockerfile in a build pipeline?
My guess would be to find a way to reliably extract the migration files from the package and add it to source control.
The alternative would be: run makemigrations within the Dockerfile that builds the deployed image, however that could result in the same migration being run twice on the database should the name of that migration change (e.g. if there's a timestamp in the name). So I'm thinking best to avoid that.
Note: this question is similar but it considers the use of the third part optional and doesn't seem to consider the above alternatives.
I solve this using the answer of this question:
Django migration file in an other app?
By using the MIGRATIONS_MODULE settings, I was able to point django to create the migrations in my source directory, and thereby commit them to source control.
For my job interview, I got assignment to create CRUD with Django and Postgresql. I created database locally and finished my assignment.
Now I have to upload my code to github. The problem is, they asked for some exampled for CRUD. Developer that is reviewing my code obviously can't access my local DB.
What can I do to solve this issue? Do I have to upload my DB to cloud (if so what is the best way to do that)? Or is there any other way to resolve this?
Thanks in advance
When they download your code they would need to create their own local db, run python manage.py makemigrations and python manage.py migrate, then all the db tables will be created. However, there won't be any initial data.
I recommend downloading your code and running through every step it takes to get your project up and running. This would include things like create an admin user, etc. Then create a basic README with all the steps to make it as easy as possible for them to get it up and running.
Alternatively, you could Dockerize your application and provide a Dockerfile, but that's a bit overkill for a take home interview in my opinion. It may impress the interviewer nonetheless. They may not even want to download and run your project, just review your code in Github.
To provide initial data, you would want to look into writing fixtures. Then have them run python manage.py dumpdata to populate the db.
Just upload it to Heroku or something similar. You already have postgres as database so it's valid way. It is pretty straightforward with Heroku's official guide for django applications. I had same issue in my recruitment process and it was a solution that satisfied recruiter.
Database obviously will be empty unless you prepare some fixtures, which is very good idea. Django docs have something for that.
I have this small project that specifies sqlite as the database choice.
For this particular project, the framework is Django, and the server is hosted by Heroku. In order for the database to work, it must be set up with migration commands and credentials whenever the project is deployed to continuous integration tools or development site.
The question is, that many of these environments do not actually use the my_project.sqlite3 file that comes with the source repository, which we version control with git. How do I incorporate changes to the deployed database? Is a script that set up the database suitable for this scenario? Meanwhile, it is worth notice that there are security credentials that should not appear in a script in unencrypted ways, which makes the situation tricky.
that many of these environments do not actually use the my_project.sqlite3 file that comes with the source repository
If your deployment platform does not support your chosen database, then your development environment should probably be moved to using one of the databases they do support. It is possible to run different databases in development and production, but just seems like the source of headaches.
I have found a number of articles that state that Heroku just doesn't support SQLite in production and instead recommends Postgres.
How do I incorporate changes to the deployed database? Is a script that set up the database suitable for this scenario?
I assume that you are just extracting data from one database to give to another, so yes,as long as that script is a one time batch operation each time the code is updated, then it should be fine. You will want something else if you are adding/manipulating data in production and then exporting it to your git.
Meanwhile, it is worth notice that there are security credentials that should not appear in a script in unencrypted ways
An environment variable should solve that. You set your host machine to have environment variables with your credentials and then just retrieve them within the script. You are looking to have something like this:
# Set environment vars
os.environ['USER'] = 'username'
os.environ['PASSWORD'] = 'password'
# Get environment vars
USER = os.getenv('USER')
PASSWORD = os.environ.get('PASSWORD')
I have a Django 1.9-driven website run on Ubuntu, and I very often face a strange issue that some error vanishes when I run the clone of the project locally from my PC using 127.0.0.1:8000 url. Locating the error in such cases is EXTREMELY time consuming and I wonder what are the best practices for debugging a large-scale project, especially when the website is already partially in use.
Just to be as specific as possible, I provide a step-by-step description of what goes wrong.
Step 1. I type some url, say, 10.8.0.1:8000/show_students/
Step 2. Do some action on the page, say, save a student profile. The operation does not end successfully, yielding an error.
Step 3. I copy-paste the project directory located on the remote server onto a local directory on my PC and try to run the CLONE. I see that the error does not take place.
Real-life example,
task_email_recipients = TaskEmailRecipients.objects.get(task_type =
task_instance.type, legal_entity_own = legal_entity_own_instance)
This line throws exception saying that LegalEntityOwn has no field named (yes, I did not omit anything. It is empty string after "field named")
If I run the same view from 127.0.0.1, the error goes away.
What should be my actions ?
BTW, I use Eclipse if this makes any difference. And I have MS Windows 10 on my local PC.
Summing up, my goal is to debug the project run from 10.8.0.1
UPDATE for – Paul Becotte's comment
I've always ignored this warning, but when running the project, it gives a warning
You have unapplied migrations; your app may not work properly until
they are applied. Run 'python manage.py migrate' to apply them.
So, let me explain a few concepts.
A. Source Control (Git) lets you keep track of all the changes to your source code. This is pretty important so that you can feel confident that you are running the same version of your code on your development machine as your deployed server without trying to do something like copy the files back and forth. A command like git status can show you if you changed something and maybe give you tips on what is different between the two environments. If you are not using git, you should immediately start!
B. Migrations are like source control for the schema of your database. A SQL database like Mysql or Postgres has a fixed schema- you have THIS many tables, with THESE names, and table A has three columns with one of them called Name and one called ID and so forth. Migrations are designed to give you visibility into what these schema are- instead of logging into the database and running CREATE TABLE A ... you build a migration file that contains the necessary commands, and then stamps the database with a version number. Then you run those command files so that if the databases are on the same version, you know they have the same structure (which allows you to get your local database to match your deployed one). Django has a helpful migration system all built in... manage.py migrate is a command to apply all of the migration files to the current database. If you are getting the error message you listed, there is basically no chance that your app IS going to work properly, because your database schema, somewhere, is out of sync with your model files. Based on your very limited description, you added a field to a model that now exists in your local database but does not exist in your production database.
C. I mentioned a deploy script- this is a single command you can run to get your code running on your remote server so that you are sure it happens the same way every time. In this case it might be something like...
ssh production
git pull
python manage.py migrate
uwsgi
Set up a script like that so that you know what is going on, and you can rule out accidentally skipping steps as an error vector.
I'm trying to develop a Django website with Heroku. Having no previous experience with databases (except the sqlite3 one from the tutorial), it seems to me a good idea to have the following file structure:
Projects
'-MySite
|-MySite
'-MyDB
I'm finding it hard to figure out how to do it, with psql commands preferring to put the databases in some obscure directory instead. Perhaps it's not such a good idea?
Eventually I want to be able to test and develop my site (it'll be just a blog for a while, I'm still learning) locally (ie. add a post, play with the CSS) and sync with the Heroku repository, but I also want to be able to add posts via the website itself occasionally.
The underlying data files (MyDb) has nothing to do with your project files and should not be under your project.
EDIT
added two ways to sync your local database with the database ON the Heroku server
1) export-import
This is the most simple way, do the following steps every now and then:
make an export on the Heroku server by using the pg_dump utility
download the dump file
import the dump into your local database by using the psql utility
2) replication
A more sophisticated way for keeping your local db in sync all the time is Replication. It is used in professional environments and it is probably an overkill for you at the moment. You can read more about it here: http://www.postgresql.org/docs/9.1/static/high-availability.html