I'm currently using Dropwizard Migrations, which uses the open source Liquibase to perform DB migrations. There is an enterprise paid version of liquibase called Datical, which brings a lot of extra features.
However, I'm wondering if the Dropwizard Migrations extension will work nicely with Datical, since I cannot test it without first paying for it.
Has anyone had this experience? Can I use Liquibase (in my case through Dropwizard Migrations) to run the changesets in development and then use Datical for production?
I have not used DropWizard, so I am not positive, but if we assume that DropWizard uses the standard Liquibase APIs, then this should work. Note that I am one of the developers at Datical, so I do have some expertise in this.
If you want to see if it really does work, I would recommend that your organization contact the Datical sales folks, who can set up a Proof-Of-Concept to make sure that it works for your organization.
Related
I have two instances of Apex, Development and Production.
My current process involves making updates on Development, exporting the app and then importing and overwriting the app on Production. But if I make any schema changes, I am also having to go into the live production database and make those same changes as well.
Doing these changes on a live database is surely bad practice. I've used with frameworks in the past that have elegant database migration tools that allow for rollback, etc. And provide a lot of peace of mind.
Does anything like this exist for Apex? Or is this the only way of deploying between separate instances?
Thanks!
I have a mobile app frontend using Graphcool as the backend. I currently have two Graphcool project, one for production and the other one for dev. Now I need to migrate the dev version of the database to the production one. To be specific, I need to do the followings:
Update ENUM
Update Schema and make sure it won't break the current data
Update the permission and rules
Update functions
I have tried to copy and paste the new schema from dev to production, but I find out there are many inconsistencies I need to solve manually. Meanwhile, since the project export function currently has no way to backup the permissions and functions, I have to manually change them once step 1 and 2 are finished.
The question is my production project has many live requests and I don't want to have any down time or requests that potentially can break data consistency. What will be the best way to deal with this kind of migration? Are there any guidelines, best practises? Many thanks.
Since I last asked the question, Graphcool has posted a doc to introduce the multi staging workflow. The new cli has included most of the questions I asked.
https://www.graph.cool/docs/tutorials/cli-multi-staging-workflow-ex4wo4zaep/
I've been looking at Flyway as a database migration tool.
The one thing that I have been unable to find a definite answer for is the following:
Can I force Flyway to run all as-of-yet unapplied migrations in a single transaction, instead of having each migration be its own transaction?
In a dev environment it's not an issue, but in a production environment where you would potentially perform multiple migrations from one update to the next, one of the migrations failing would leave the database in a 'half-migrated' state, where some migrations were committed and some not - quite a bad thing.
A workaround would be to simply cram all the SQL required in a single file, but there are issues with that:
The production migrations and dev migrations would end up being performed differently, since you cannot know in advance what will be in the migration on dev environment. I guess you could always do a clean and then a new migrate, but this seems to be against the spirit of the flyway design with regard to incremental migrations.
The checksums will be different as soon as a new change is added.
Does Flyway still not support such a feature? Does Liquibase, or any other migration tool?
There is no such feature out of the box. It's a great question though and I'd bet it was thought about since Flyway provides the transactional boundaries per migration - hopefully Axel Fontaine will chime in on the technical / design considerations that resulted in this not being a feature.
The FAQ have this and this to say regarding downgrading / failures. The policy boils down to:
Maintain backwards compatibility between the DB and all versions of
the code currently deployed in production.... Have a well tested,
backup and restore strategy.
In my case, we have being using Flyway for almost 3 years and have abided by the quoted policy. At any given deployment we could have 100 or more migrations running against many databases and happy to say have never had anything untoward happen in production. This all comes down to minimizing the opportunity for failure in your release process.
I used Liquibase on a much smaller project prior to that and don't recall any such feature apart from providing the rollback procedures.
I have setup a git repository and cloned open source code which I am planning on modifying from github to start development. I committed the codebase in our repository.
I now have added few users and posts and other "stuff" into database.
I want to commit this change as well so that my teammate can check out and we have same settings and database throughout.
Is this possible by using south migrations? i.e will database bot contents and schema be in sync as well?
I have the project where I am writing the code as well the actual app. Should I commit both of them.
What should the github repository look like after doing the "right" thing
Data and database structure
This is possible using south migrations, data migrations and fixtures.
The easiest way for development is to just use a SQLite database, which is a binary file that you can commit. The test_project of django-autocomplete-light demonstrates such a possibility: http://django-autocomplete-light.readthedocs.org/en/latest/demo.html
you must use south anyway !
Apps in the project repo
I think you should keep apps as small and loosely coupled as possible.
If sound, make another repository and python package for the app:
In some cases it makes sense at the beginning, ie. a "blog" app that you know you will reuse,
In some cases it makes sense later, ie. you tought your app was really project specific but then you want to reuse it in another project,
In some cases it never makes sense (ie. the app is only useful to that particular project).
Best practice
As for best practices, there is http://12factor.net, http://lincolnloop.com/django-best-practices/ and pinax projects which are really interresting.
If you're going to reuse and extend external apps, then maybe this article on best practice reusing apps can help.
If there is data that every developer needs, these can be provided by initial_fixtures. I have just begun to use south, but I think it is mainly for migrations, (changing your models, without having to delete your data, and resync) not for sharing data. Schema should always be in sync using south, because a developer can pull the south migration, and apply it.
During development, I like the idea of frameworks like Entity Framework 4.3 Migrations (although I need it to work with sql scripts instead of Migration classes) keeping all developer databases up-to-date. Someone does an update to get the latest source, they try to run the application and get an error that they need to update their database to the latest migration (or have the migration happen automatically). Migration files are timestamped so devs don't have to worry about naming two files the same or the sequence the files need to be executed in.
When I get ready to build a WebDeploy deployment package, I want the package to contain the scripts required to move the production database to the latest db version. So somehow, MSBuild or WebDeploy need to make a decision about which scripts must be packaged. For deployment I don't want the application to try to update itself like what EF offers. I want to either hand the package to the IT department or do an auto-deploy through a deployment server.
Some of my questions are:
Can EF 4.3 work with sql scripts instead of DBMigration classes for my development needs (I'm already using it for my ORM so it would be nice if it could)?
Does MSBuild or WebDeploy understand the concept of database migrations (e.g. does it recognize the EF. 4.3 migrationHistory table) or do I have to make sure to give it just the scripts it needs to run that will bring my prod db to the latest migration? Manually deciding which scripts should be pakaged is not something I want to do so is there a MS WebDeploy extension that understands migrations?
Are my concerns and ideas valid? I'm just researching this stuff so I don't really know.
I think that your concerns are valid. During development anything that keeps the developer machines in sync will do. When deploying, more control is needed.
In my project we've decided to only use code-based migrations, to ensure that all databases are migrated in the same order with the same steps. Automatic migration and db creation is disabled by a custom initialization strategy that only checks that the db exists and is valid.
I've written more about the details in Prevent EF Migrations from Creating or Changing the Database. I've also looked a bit into merge conflicts which will eventually happen with multiple developers.
You can run SQL in the classes by calling the Sql method, or generate SQL from a migration by using the -script parameter with the update-database command.
No. They were looking at adding support for webdeploy but apparently decided against it before rtm. They did however release a command line app (and powershell script obviously) which you could call from either.
What I do in my projects is create an startup module in my applications and run any migrations that haven't been deployed automatically - Triggering EF migration at application startup by code. It's not perfect, developers have to run the app after getting latest before changing the db, but it works for both get latest and deployment.