I am working on a project that needs to be able to create dynamic queries into an H2 database. This also includes a full text search with built-in H2 logic, tables, and triggers.
I have been trying to figure out how to add that full-text search into my CriteriaQuery but keep running into the road block that the tables used aren't entities in my model. I could add them as entities, but I don't want them created automatically by EclipseLink when a new database file is created since there is a function in H2 that creates the tables and does other necessary housekeeping.
I had tried the path of creating a view to query the full text tables to give me the information I need in the format I need. But I still keep running into the same problem that that view is not an Entity.
Has anyone encountered this situation before and/or figured out a way around it?
Thanks!
Related
I have lots of models in my project like Advertisements, UserDetails etc. Currently I have to delete the entire database every year so as to not create any conflicts between this year data and previous year data.
I want to implement a feature that can allow me to switch between different years. What can be the best way to implement this?
I think you could switch schemas in PostgreSQL. It's not completely straightforward. There are several ways to do that you can look into. The way I did it was to use a default search path for the Django database user account (e.g. user2018, user2019, etc) that only included the schema I wanted to use. I can't check the exact settings right now because my office network is down. You can also do it in settings.py or in each individual model using db_table according to what I've read, although both those solutions seem more convoluted that using the search path.
You would have to shutdown, change the database username in settings.py (or change the search path in PostgreSQL, change the schema over to a new one, and then run migrate to create the tables again. If you have reference data in any of the tables then schema-to-schema copies are easy to do.
Try searching for change django database schema postgresql to see what options there are for specifying the schema.
I'm a beginner at Django, and as a practice project I would like to create a webpage with a dashboard to track investments in a particular p2p platform. They do not have a nice dashboard (but provide excel file with all data). As I see it, main steps that I need to do in this project are as follow:
Create login so that users would have account where they upload their excel files.
Make it possible to import excel file to a database
Manipulate/calculate data for it to be later used in dashboard
Create dashboard.
Host webpage.
After some struggle I have implemented point no. 2, and will deal with 1 and 5 later. But number 3 is my biggest issue now.
I'm completely unsure what I need to do, and google did not help. I need to calculate data before I can make dashboard from it. Union two of the tables, and then join them together with a third table, creating some additional needed calculated fields. Do I create a view in the database and somehow fetch this data to Django? Or do I need to create some rules so that new table would be created at the time of the import? I think having table instead of a view would have better performance. Or maybe I'm doing it completely wrong, and should take completely different approach for this kind of task? Also, is SQLite a good database for a task (I'm using it, because it was a default in Django)?
I assume for vizualization part I will need to do it with some JavaScript library, such as D3? Which then would use data from step 3.
For part 3 there is 2 way, either do these stuff and save the result in your database or you can do it when you need it using django model features like annotation, aggregation and etc.
Option 1 requires to add a table for you calculation which is Models in django.
Option 2 requires to create a doing the annotations in a view or model managers and then using them in views.
Django docs: Aggregation
Which is the best is depended on how big your data is, how complicated the calculation is and how often you need them.
And for database; SQLite is just a database for development use not the production and surly not with a lot of data and a lot of calculations. The recommended database for django is postgresql which is pretty good at handling millions and even billions of data and doing heavy calculation.
And for vizualization you should handle it on the template side which is basically HTML, CSS and JS.
I am trying to use django's database routers to shard my database, but I am not able to find a solution for that.
I'd like to define two databases, create the same table in both and then save the even rows in one db, the odd ones in the other one. The examples in the documentation show how to write to a master db and to read from the readonly slaves, which is not what I want, because I don't want to store the whole dataset in both dbs.
Do know any webpage explaining what I am trying to do?
Thank you
PS: I am using Postgresql and I know there are tools to achieve the same goal at DB level. My goal is to study if it can also be done in django and to explore if there are some advantages by doing this.
We are building a Sitecore site that will pull in some product data from an external database. On a nightly basis we will query the external database and either Add, Update or Archive/Delete/Remove product content items in Sitecore as needed. Our data template has some fields that will be populated directly from the external database (and will be read-only for content authors) and other fields that they will populate themselves. Included in our custom fields will be the SKU of the item from the external database. It is possible that over time a product could disappear from the external database. In this case we would want Sitecore to somehow remove this item from our list of products, but not completely delete it. The reason for this is that the products that have been removed could reappear in the future and we would not want to lose all of the data that had been added to other custom fields on the item. I can think of a number of different approaches for this:
Use Archiving/Recycling features of Sitecore. When we find that there is a product item in Sitecore that no longer appears in the external database, then we could archive it. That works well. However I can't seem to figure out a way to restore that item later if it reappears in the external database. I don't have any access to any custom fields when an item is archived (from what I have read online). So when I come across a SKU in the external database that is not in Sitecore, I have no way of figuring out if there is an archived item that has that SKU.
Use a custom status field on each product content item. I could set each product content item to "active" or "inactive". This would make it easy to reactivate items that reappear in the external database. However I worry about things like search and publishing. It seems messy to me to have some content items that are inactive in the folder of all products in the master database. It could be confusing to content authors and I worry that they will find their way in to the web database, etc. It seems like I would have to do a lot of custom coding to make sure that those products do not show up on any pages, etc.
When a product disppears from the external database I could then move those content items to a different location in Sitecore. Then when they reappear I could move them back. This also feels messy.
I just wonder if there is some better solution that I am missing. Thanks in advance for any help.
I would go with option 2 "Setting status field on each product "Active" or "Inactive", as its more clear and keep the data in one place.
Additional thing to do (as suggested by Vasiliy) is to set the "Publishable" checkbox on product to "False", this way the product will disappear from web database, hence no extra filter in your search methods.
You can implement custom content editor warning to inform content editor that the current product is "inactive":
Creating Custom Content Editor warnings
Hope this helps
Just a thought what if you just unpublished the items that were removed from the external database and set the ones in the authoring db unpublishable until they reappear again. With this scenario, you could also have a task running archiving items that have been unpublished and not republished for a given period of time.
The best solution really depends on the number and frequency of items appearing / disappearing and the cost benefit of keeping those items in the authoring database vs. deleting them.
I need to move a series of tables from one datasource to another. Our hosting company doesn't give shared passwords amongst the databases so I can't write a SQL script to handle it.
The best option is just writing a little coldfusion scripty that takes care of it.
Ordinarily I would do something like:
SELECT * INTO database.table FROM database.table
The only problem with this is that cfquery's don't allow you to use two datasources in the same query.
I don't think I could use a QoQ's either because you can't tell it to use the second datasource, but to have a dbType of 'Query'.
Can anyone think of any intelligent ways of getting this done? Or is the only option to just loop over each line in the first query adding them individually to the second?
My problem with that is that it will take much longer. We have a lot of tables to move.
Ok, so you don't have a shared password between the databases, but you do seem to have the passwords for each individual database (since you have datasources set up). So, can you create a linked server definition from database 1 to database 2? User credentials can be saved against the linked server, so they don't have to be the same as the source DB. Once that's set up, you can definitely move data between the two DBs.
We use this all the time to sync data from our live database into our test environment. I can provide more specific SQL if this would work for you.
You CAN access two databases, but not two datasources in the same query.
I wrote something a few years ago called "DataSynch" for just this sort of thing.
http://www.bryantwebconsulting.com/blog/index.cfm/2006/9/20/database_synchronization
Everything you need for this to work is included in my free "com.sebtools" package:
http://sebtools.riaforge.org/
I haven't actually used this in a few years, but I can't think of any reason why it wouldn't still work.
Henry - why do any of this? Why not just use SQL manager to move over the selected tables usign the "import data" function? (right click on your dB and choose "import" - then use the native client and permissions for the "other" database to specify the tables. Your SQL manager will need to have access to both DBs, but the db servers themselves do not need access to each other. Your manager studio will serve as a conduit.