I have two instances of Apex, Development and Production.
My current process involves making updates on Development, exporting the app and then importing and overwriting the app on Production. But if I make any schema changes, I am also having to go into the live production database and make those same changes as well.
Doing these changes on a live database is surely bad practice. I've used with frameworks in the past that have elegant database migration tools that allow for rollback, etc. And provide a lot of peace of mind.
Does anything like this exist for Apex? Or is this the only way of deploying between separate instances?
Thanks!
I can think of three reasons why:
providing users with the flexibility on "when" to commit model changes
debugging modularity
perhaps resource consumption in larger
databases
However, it does seem that migrate always follows shortly after migration (tutorials/youtube videos).
so is there a philosophy behind this that I'm missing?
Ofcourse there are some reasons.
First of all, 'makemigrations' doesn't touch real DB, it just tells django how models(db scheme) have changed so you can see what's going on when you do 'migrate'.
and this makes django more safe.
This also provides to make default options for new fields or db changes..
Other reason is 'revert'.
If you want to roll-back db scehme with specific migrations, you can just tell django to roll back to specific migration file.
Another reason is 'reusable-app' principle.
If you create app with django and it could be reusable with no-db-interaction. It means if you deploy your app(or project, too!) to another project or server, it just needs 'migrations' files not real db.
I have a mobile app frontend using Graphcool as the backend. I currently have two Graphcool project, one for production and the other one for dev. Now I need to migrate the dev version of the database to the production one. To be specific, I need to do the followings:
Update ENUM
Update Schema and make sure it won't break the current data
Update the permission and rules
Update functions
I have tried to copy and paste the new schema from dev to production, but I find out there are many inconsistencies I need to solve manually. Meanwhile, since the project export function currently has no way to backup the permissions and functions, I have to manually change them once step 1 and 2 are finished.
The question is my production project has many live requests and I don't want to have any down time or requests that potentially can break data consistency. What will be the best way to deal with this kind of migration? Are there any guidelines, best practises? Many thanks.
Since I last asked the question, Graphcool has posted a doc to introduce the multi staging workflow. The new cli has included most of the questions I asked.
https://www.graph.cool/docs/tutorials/cli-multi-staging-workflow-ex4wo4zaep/
I have an application which use Zend Framework and Doctrine.
I want to change for a module the database from the default settings.
I have created an alternative connection for doctrine.
When creating/updating the tables using,
./vendor/bin/doctrine-module orm:schema-tool:update --force
the tables are created in the first configuration of database.
Basically what I want to update the second configured database tables.
Can someone help me with an working example ?
Thanks,
Bogdan
To my knowledge, the schema-tool binary only works with the orm_default database.
Now, there's certainly nothing stopping you from having modules that add additional named connections. See this documentation for doing that:
https://github.com/doctrine/DoctrineORMModule/blob/master/docs/configuration.md#how-to-use-two-connections
But, the tooling around managing those additional databases might be a little "roll your own". The good news is all the pieces are there (Doctrine's underlying SchemaTool classes), you would just need to wire them up and build a cli command that acts on multiple schemas.
All that being said, if you find yourself using multiple unique schemas in the same database engine (unique being the key word to account for things like doctrine sharding), I worry your application design might be potentially troublesome. It could be possible that your multiple storage domains should actually live as separate applications.
On Heroku, as soon as you push new code, the web-serving instances restart... even if the underlying database schema additions/changes (via syncdb or south migrate) haven't yet been applied.
In many cases, this might just cause harmless errors undtil the syncdb/migrate is run soon afterward. But I'm concerned that in some cases, new code might half-work making unexpected changes in the pre-migration database.
What's the right way to be safe against this risk?
One technique might be to add the syncdb/migrate to the Procfile so it's run before web restart. But, in the case of multiple instances, or maybe even a case where the one old-code-instance is left running until the moment the one new-code-instance is known-up, there's still a variant of the issue where code is talking to a DB with a mismatched schema.
Is there a 'hold all web instances' feature (or common best practice) for letting the migrate complete without web traffic?
Or am I being overly concerned about a risk that is negligible in practice?
The safest way to handle migrations of this nature, Heroku or no, is to strictly adopt a compatibility approach with your schema and code:
Every additive or transformative schema change must be backwards-compatible;
Every destructive schema change must be performed after the code that depends on it has been removed;
Every code change must either be:
durable against the possibility that associated schema changes have not yet been made (for instance, removing a model or a field on a model) or
made only after the associated schema change has been performed (adding a model or a field on a model)
If you need to make a significant transformation of a model, this approach might require the following steps:
Create a new database table to hold your new model structure, and deploy that migration
Create a new model with the new structure, and code to copy changes from the old model to the new model when the old model changes, and deploy that code
Execute a migration or code action to copy all old model data to the new model
Update your codebase to use the new model rather than the old model, deleting the old model, and deploy that code
Execute a migration to delete the old model structure from the database
With some thought and planning, it can be used for more drastic changes as well:
Deploy code that completely removes dependence on a section of the database, presumably replacing those sections of the site with maintenance pages
Deploy a migration that makes drastic changes that would not for whatever reason work with the above dual-model workflow
Deploy code that brings the affected sections back with the new model structure supported
This can be hard to organize and requires strict discipline and firm understanding of your code's interaction with your database, but in practice, it does allow for most changes to be made with no more downtime than the server restart itself imposes.
Looks like fast-database changeovers are the way to go, but it requires a dedicated database.
http://devcenter.heroku.com/articles/fast-database-changeovers
Alternatively, here's a tutorial for copying the data from one database (e.g., production) to another database (e.g., staging), doing the schema/data migration (e.g., using django/south), then switching the app to use the newly-updated database instance.
http://devcenter.heroku.com/articles/migrating-data-between-plans
Seems reasonable, but potentially slow if there's a large amount of data.
The recommended method is this:
Add database changes for your new features to your existing code
Make the existing code compatible with the new schema
Deploy
Add the new features to your codebase
Deploy
This means that your database changes are already in place when the code starts to require them.
However....
There's a couple of issues with this. First that I know of no development shop that is organised enough to be able to handle this, as features just get built ad-hoc, and secondly that you're not really saving anything.
Generally speaking, unless your making big changes to a massive database your changes won't take long to apply and are usually over in a couple of seconds which a developer can work around quite happily issuing restarts etc when needed. The risk being that a user might get an error page. If the changes are larger, you have some alternatives. One is using maintenance mode to turn the site off for a few seconds.
To be honest, there is no clear cut way for how to handle this nicely as by definition your code needs to be in place for your database changes to start. The best way I've found to approach the problem is to look at each change individually and work out the smoothest path for each on a case by case basis.
Rehearsing deployments on a staging environment will mitigate the risk of a deploy going bad, and give you an idea of the impact.
Heroku recently released "buildpacks" which are the scripts they use to set up an environment for your application, from managing dependencies to restarting the instances. Essentially it's a more comprehensive Procfile which you can customize.
You can fork the Python buildpack and modify the script to run in the sequence you want. Append the command you run to syncdb to the end of bin/steps/django. Commit and put this repo on Github.
Unfortunately as of now it's not possible to modify the buildpack of an existing Heroku app, so you'll have to delete it and recreate one that points to your buildpack repo:
heroku create --stack cedar --buildpack git#github.com:...
This is the best solution because it
Doesn't cost anything at all
Doesn't require you to adapt your code to Heroku
Only syncs the db once per deployment
Hope this helps.