I'd like to use alembic in offline mode, and execute the SQL migrations from an external software.
The external software tracks the DB version, so I don't need the alembic_version table at all.
Do you know if I can generate SQL migrations without creating an alembic_version table ? Is it a supported feature ?
Thanks
I don't think generating migrations off line is an available feature till now as #zzzeek wrote in 'Writing Migration Scripts to Support Script Generation' section in the official tutorial https://alembic.zzzcomputing.com/en/latest/offline.html . I do hope the new feature will come as my project can't connect to the database server directly but generating sql scripts off line will save my time by preventing writing SQL DDL and SqlAlchemy objects both.
Related
I am using flyway database migration tool. Let's assume I have 100 sql scripts under a folder and I migrated them by applying on a server. Later I added 50 more sql scripts. Can I be able to move these 100 old sql scripts away and archive them somewhere (artifactory or remote share). That way I can only have 50 sql scripts which are new and needed for current migration.
Is this possible? Or all sql scripts has to be present under the directory?
It would be useful if you could edit your question to state the reason why you want to only include the 50 scripts for the current migration, as this would help me recommend an approach.
There are two ways that I know of that you could use.
Create subfolders for each release, rather than have one single folder with an unmanageable list of scripts. This approach means that all migrations remain in the project, which has advantages such as being able to rebuild a test database from scratch (using flyway clean migrate) and also being able to reuse the project later on to repeat earlier deployments. If you do it this way, you will need to reference each subfolder using the flyway.locations parameter.
Remove the sql scripts that have already been run, as you have suggested. If you do this, you will need to run flyway baseline on any targets. This command will ensure that the targets don't expect migrations below a specific version, so the missing migration files won't confuse flyway.
My personal preference is the first option. If you can't use this for whatever reason, we'd be interested to understand why.
I'm working on a project, which is planned to use/manage 3 different databases [MySQL, SQL Server, Oracle] as the back-end. Now I have finished with my database design and everything I have set in with MySQL database.
Now I have to migrate / clone to SQL Server & Oracle as well. Also, I need the system to update automatically, whatever changes I will make in MySQL database [structural, not data] to be replicated in SQL Server & Oracle as well.
I couldn't find anything worthy from the documentations of FlywayDB and Liquibase, rather than a database source control mechanism.
So can I go for / get support from FlyWayDB or Liquibase to do this? Or is there an alternative for this task? Please advise.
Liquibase is easier to manage different kind of databases because it use XML to describe structure.
Flyway use SQL rather than XML to keep the syntax as simple as possible
In your case, you probably have to adapt your data structure to be compatible with your 3 databases. Oracle is one of the most restrictive.
Flyway could be a better solution because you don't care about historic modifications and because you have a better knowneldge of sql
I've got an existing production DB, and a development DB with some schema differences. Neither have used Liquibase before. How can I achieve the following:
Compute a difference between production and development.
Apply the delta to production,
End up with both production (and dev) schema under control of Liquibase.
Here is what I ended up with ($LIQUIBASE expands to the Liquibase command line tool configured for the particular DB I was using).
Generate a baseline changelog from the current state of the Production DB:
$LIQUIBASE --url=$PROD_URL --changeLogFile=changeLog.xml generateChangeLog
Record Production as being in sync with the change log:
$LIQUIBASE --url=$PROD_URL --changeLogFile=changeLog.xml changeLogSync
Calculate the differences between Development and Production, and append them to the change log file:
$LIQUIBASE --url=$PROD_URL --referenceUrl=$DEV_URL --changeLogFile=changeLog.xml diffChangeLog
Bring Production in sync with Development:
$LIQUIBASE --url=$PROD_URL --changeLogFile=changeLog.xml update
Record Development as being in sync with the change log:
$LIQUIBASE --url=$DEV_URL --changeLogFile=changeLog.xml changeLogSync
You would start by generating a changelog for the development database, using the generateChangelog command, documented here: http://www.liquibase.org/documentation/generating_changelogs.html
When you generate the changelog, Liquibase will also create and initialize the two database tables (DATABASECHANGELOG and DATABASECHANGELOGLOCK) that it uses to keep track of what changesets have been applied to the database.
Next, you want to diff the two databases and have liquibase generate the differences as a separate changelog using the diffChangelog command:
http://www.liquibase.org/documentation/diff.html
The diff changelog can be included as is, or copied into an existing change log. If the diff command is passed an existing change log file, the new change sets will be appended to the end of the file.
Finally, you deploy the changelog to the production environment.
5.3 part of this tutorial answers your question I guess.
http://www.baeldung.com/liquibase-refactor-schema-of-java-app
This uses maven in java but I think plain liquibase commands also can do it.
I have few questions about this plugin.
1- what does it do?
Is it for exchanging databases between teams or changing their schema or creating tables based on models or something else?
2- if it is not meant to create tables based on models where can I find a script that does this?
3-can it work under windows?
thanks
The Migrations plugin allows versioning of your db changes. Much like is available in other PHP frameworks and Rails.
You essentially start with your original schema and create the initial migration. Each time you make a change you generate a 'diff' that gets stored in the filesystem.
When you run a migration, the database is updated with the changes you made. Think deployment to a staging or production server where you want the structure to be the same as your code is moved from environment to environment.
We are starting to look at this plugin so we can automate our deployments, as the DB changes are done manually right now.
During development, I like the idea of frameworks like Entity Framework 4.3 Migrations (although I need it to work with sql scripts instead of Migration classes) keeping all developer databases up-to-date. Someone does an update to get the latest source, they try to run the application and get an error that they need to update their database to the latest migration (or have the migration happen automatically). Migration files are timestamped so devs don't have to worry about naming two files the same or the sequence the files need to be executed in.
When I get ready to build a WebDeploy deployment package, I want the package to contain the scripts required to move the production database to the latest db version. So somehow, MSBuild or WebDeploy need to make a decision about which scripts must be packaged. For deployment I don't want the application to try to update itself like what EF offers. I want to either hand the package to the IT department or do an auto-deploy through a deployment server.
Some of my questions are:
Can EF 4.3 work with sql scripts instead of DBMigration classes for my development needs (I'm already using it for my ORM so it would be nice if it could)?
Does MSBuild or WebDeploy understand the concept of database migrations (e.g. does it recognize the EF. 4.3 migrationHistory table) or do I have to make sure to give it just the scripts it needs to run that will bring my prod db to the latest migration? Manually deciding which scripts should be pakaged is not something I want to do so is there a MS WebDeploy extension that understands migrations?
Are my concerns and ideas valid? I'm just researching this stuff so I don't really know.
I think that your concerns are valid. During development anything that keeps the developer machines in sync will do. When deploying, more control is needed.
In my project we've decided to only use code-based migrations, to ensure that all databases are migrated in the same order with the same steps. Automatic migration and db creation is disabled by a custom initialization strategy that only checks that the db exists and is valid.
I've written more about the details in Prevent EF Migrations from Creating or Changing the Database. I've also looked a bit into merge conflicts which will eventually happen with multiple developers.
You can run SQL in the classes by calling the Sql method, or generate SQL from a migration by using the -script parameter with the update-database command.
No. They were looking at adding support for webdeploy but apparently decided against it before rtm. They did however release a command line app (and powershell script obviously) which you could call from either.
What I do in my projects is create an startup module in my applications and run any migrations that haven't been deployed automatically - Triggering EF migration at application startup by code. It's not perfect, developers have to run the app after getting latest before changing the db, but it works for both get latest and deployment.