I am trying to migrate DB2 LUW 9.7 databases and data to PostgreSQL 9.3. Any suggestion on which will be the best approach to do it? Which will be the best tool or any open source tool available to perform this?
The db2look utility can reverse-engineer your DB2 tables into DDL statements that will serve as a good starting point for your PostgreSQL definitions. To unload the data from each table, use the EXPORT command, which dumps the results of any SQL SELECT to a delimited text file. Although the db2move utility can handle both of those tasks, is not going to be of much help to you because it extracts the table data into IBM's proprietary PC/IXF format.
If you're moving off of DB2 because of price, IBM provides a free-as-in-beer version called DB2 Express-C, which shares the same core database engine as paid editions of DB2. Express-C is a first-rate, industrial strength DBMS that does not have the sort of severe limitations that other commercial vendors impose on their no-cost engines.
Related
I would like to build an admin dashboard in Django framework.
So far I have only worked with sqlite3 databases in Django.
However, the admin dashboard should read statistics from an MSSQL database and display them accordingly. (Sales figures as a graph, other sales in a table, etc.)
The turnover figures are very extensive. There are several thousand entries per month and the client would like to be able to filter by any date period.
Only data from the MSSQL database should be read. Writing or updating the data is not desired.
So far this is no problem, but I wonder what is the better solution to implement the project.
Should I connect the MSSQL database directly to Django or should I read the MSSQL database in certain intervals and cache the "results" in a sqlite3 database?
Caching seems to me to be the nicer solution, as we don't need real-time data and the performance of the MSSQL server might not suffer as a result. But I would have to build an additional connector to transfer the data from MSSQL to sqlite3.
How would you approach such a project?
Short version: I´d like to display data in django-framework App, should I read directly from MSSQL-Server or should I import the MSSQL-Dataset to local sqlite3-database?
Thanks in advance for your answers.
From official SQLite page:
Situations Where SQLite Works Well:
Application file format
SQLite is often used as the on-disk file format for desktop
applications such as version control systems, financial analysis
tools, media cataloging and editing suites, CAD packages, record
keeping programs, and so forth.
Cache for enterprise data
Many applications use SQLite as a cache of relevant content from an
enterprise RDBMS. This reduces latency, since most queries now occur
against the local cache and avoid a network round-trip. It also
reduces the load on the network and on the central database server.
And in many cases, it means that the client-side application can
continue operating during network outages.
I am new to Big data and related tools/technologies. I was going through the documentation of impala.
Is it true to say that Impala is a clustered columnar database?
and Impala needs heavy memory to compute/transform the data?
Impala is not a Database.
Impala is a MPP (Massive Parallel Processing) SQL query Engine. It is an interface of SQL on top of HDFS structure. You can build a file structure over Parquet files, that are columnar files that allow you fast read of data.
According Impala documentation:
Impala provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS, HBase, or the Amazon Simple Storage Service (S3). In addition to using the same unified storage platform, Impala also uses the same metadata, SQL syntax (Hive SQL), ODBC driver, and user interface (Impala query UI in Hue) as Apache Hive. This provides a familiar and unified platform for real-time or batch-oriented queries.
Impala uses Hive Metastore to store the file structure and Schema of each file. Impala allows you to run SQLs queries in your files, and it will be responsible to parallelize the data in your cluster.
About the uses of Memory, you are partially right. Impala uses memory bound for execution, Hive uses disk based in classical map reduce over Tez execution. In newer version of Impala this allow you to use Disk Spill, that will help you with data that doesn't fit your memory.
Impala integrates with the Apache Hive meta store database, to share databases and tables between both components. The high level of integration with Hive, and compatibility with the HiveQL syntax, lets you use either Impala or Hive to create tables, issue queries, load data, and so on.
Impala is not database.
Impala is not based on Map-Reduce algorithms. It implements a distributed architecture based on daemon processes that are responsible for all the aspects of query execution that run on the same machines.
I am not sure if this is the correct forum, but any help on this would be really appreciated.
Can someone please share some links/references which could help me in analyzing the feasibility of database migration from IBM Netezza to Amazon Redshift?
Kamlesh,
There are a lot of similarities between both technologies: IBM Pure Data/Netezza and AWS Redshift.
Some developers who worked on the first version of Netezza also worked on the first version of ParAccel DB. AWS Redshift utilizes the same core engine as ParAccel DB. ParAccel has been sold and the product has been re-branded as Actian Matrix. Still, the core engine is the same.
Both databases are MPP implementations, with a shared nothing architecture. Both share a PostgreSQL "heritage". AWS Redshift truly is a "columnar" database, while Netezza is not.
There are a few differences in SQL Syntax and also some differences in functionality. There are several features/capabilities that AWS Redshift does not yet have. Some of the most "noteworthy" differences is the fact that Redshift does not support Stored Procs, User Defined Functions or Sequences.
Amazon AWS lists the differences between AWS Redshift and PostgreSQL in this document. While this is not a comparison between Netezza and Redshift it will give you a good idea of "what to expect" in terms of differences since both Netezza and Redshift were both originally based on postgreSQL.
For increasing performance we want to duplicate part of our postgresql data to in-memory sql storage. At this moment I am searching for suggested in-memory relational database, it may run on either windows or linux. I will really appreciate any suggestions based on experience of usage or good references.
If you need an In-Memory RDBMS for extreme SQL OLTP and analytics performance then try Oracle TimesTen In-Memory Database.
TimesTen runs on both Linux and Windows [and Solaris, AIX and HP-UX].
TimesTen runs as either a system or record [RDBMS] or as a read/write cache for the Oracle RDBMS.
Oracle will also soon release the shared nothing scalout version of TimesTen called TimesTen Scaleout
DB layer of my product uses MyBatis to execute queries on Oracle database. Wanted to test the DB layer and be able to run the SQLs which are Oracle specific and return the mock data.
From the search I did, I found HSQL having support for Oracle Syntax, does JavaDB support similar feature ? for javaDB, it comes with > java6 so don't need to add more dependency. Which one will be more suitable if my SQL's evolve to use new Oracle specific features etc. I also use Oracle Sequences in my queries.
Why I want to test SQL's ?
I want to be able to catch logical bugs in SQL's # development time than in production.
This is a feature comparison request and will probably get closed by moderators.
Does JavaDB support Oracle Syntax? Obviously not. Oracle 10g has several times as many systax features and functions than the latest JavaDB. Does it support a small subset of Oracle Syntax? Yes.
Please read the Guide for these databases and see which one covers the features you need from Oracle Syntax.