I work with play 1.2.5,
I want use the db evolution system with oracle 11g Release 2.
But when I have a multi line request,
# --- la table champ pointe directement sur la table categorie (pas de relation n, n )
# --- !Ups
DROP TABLE CATEGORIE_CHAMP;
ALTER TABLE CHAMP ADD (ID_CATEGORIE NUMBER(38));
ALTER TABLE CHAMP ADD CONSTRAINT FK_CHAMP_CATEGORIE FOREIGN KEY (ID_CATEGORIE) REFERENCES CATEGORIE(ID_CATEGORIE);
# --- !Downs
CREATE TABLE "CATEGORIE_CHAMP"
( "ID_CATEGORIE" NUMBER(38,0),
"ID_CHAMP" NUMBER(38,0),
CONSTRAINT "Categorie_Champ PK" PRIMARY KEY ("ID_CHAMP", "ID_CATEGORIE")
USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS NOCOMPRESS LOGGING
TABLESPACE "USERS" ENABLE,
CONSTRAINT "CATEGORIE_CHAMP_CATEGORIE" FOREIGN KEY ("ID_CATEGORIE")
REFERENCES "RENTIT"."CATEGORIE" ("ID_CATEGORIE") ENABLE,
CONSTRAINT "CATEGORIE_CHAMP_CHAMP" FOREIGN KEY ("ID_CHAMP")
REFERENCES "RENTIT"."CHAMP" ("ID_CHAMP") ENABLE
) SEGMENT CREATION DEFERRED
PCTFREE 10 PCTUSED 40 INITRANS 1 MAXTRANS 255 NOCOMPRESS LOGGING
TABLESPACE "USERS" ;
ALTER TABLE CHAMP DROP COLUMN ID_CATEGORIE;
This request don't work and do an issue :
ORA-00911: caractère non valide
[ERROR:911, SQLSTATE:22019]
invalid character. The character is the ;.
What is the solution for do a multi line request?
For information I have resolve the issue, I have change the class play.db.SQLSplitter.
I have change the ligne 195 and 196 of :
++i;
CharSequence ret = sql.subSequence(prev, i);
in :
CharSequence ret = sql.subSequence(prev, i);
++i;
So the final ; is not used, And the request work.
Related
We are facing a weird issue with Redshift and I am looking for help to debug it please. Details of the issue are following:
I have 2 tables and I am trying to perform left join as follows:
select count(*)
from abc.orders ot
left outer join abc.events e on **ot.context_id = e.context_id**
where ot.order_id = '222:102'
Above query returns ~7000 records. Looks like it is performing default join as we have only 1 record in [Orders] table with Order ID = ‘222:102’
select count(*)
from abc.orders ot
left outer join abc.events e on **ot.event_id = e.event_id**
where ot.order_id = '222:102'
Above query returns 1 record correctly. If you notice, I have just changed column for joining 2 tables. Event_ID in [Events] table is identity column but I thought I should get similar records even if I use any other column like Context_ID.
Further, I tried following query under the impression it should return all the ~7000 records as I am using default join but surprisingly it returned only 1 record.
select count(*)
from abc.orders ot
**join** abc.events e on ot.event_id = e.event_id
where ot.order_id = '222:102'
Following are the Redshift database details:
Cutdown version of table metadata:
CREATE TABLE abc.orders (
order_id character varying(30) NOT NULL ENCODE raw,
context_id integer ENCODE raw,
event_id character varying(21) NOT NULL ENCODE zstd,
FOREIGN KEY (event_id) REFERENCES events_20191014(event_id)
)
DISTSTYLE EVEN
SORTKEY ( context_id, order_id );
CREATE TABLE abc.events (
event_id character varying(21) NOT NULL ENCODE raw,
context_id integer ENCODE raw,
PRIMARY KEY (event_id)
)
DISTSTYLE ALL
SORTKEY ( context_id, event_id );
Database: Amazon Redshift cluster
I think, I am missing something essential while joining the tables. Could you please guide me in right direction?
Thank you
My goal is to calculate if a building have at least one shared wall with a building of another estate. I used a PostGIS query to do so but it is really slow. I have tweaked this for two weeks with some success but no breakthrough.
I have two tables:
Estate (a piece of land)
CREATE TABLE IF NOT EXISTS public.front_estate
(
id integer NOT NULL DEFAULT nextval('front_estate_id_seq'::regclass),
perimeter geometry(Polygon,4326),
CONSTRAINT front_estate_pkey PRIMARY KEY (id),
)
CREATE INDEX IF NOT EXISTS front_estate_perimeter_idx
ON public.front_estate USING spgist
(perimeter);
Building
CREATE TABLE IF NOT EXISTS public.front_building
(
id integer NOT NULL DEFAULT nextval('front_building_id_seq'::regclass),
type character varying(255) COLLATE pg_catalog."default",
footprint integer,
polygon geometry(Polygon,4326),
shared_wall integer,
CONSTRAINT front_building_pkey PRIMARY KEY (id)
)
CREATE INDEX IF NOT EXISTS front_building_polygon_idx
ON public.front_building USING spgist
(polygon)
TABLESPACE pg_default;
CREATE INDEX IF NOT EXISTS front_building_type_124fcf82
ON public.front_building USING btree
(type COLLATE pg_catalog."default" ASC NULLS LAST)
TABLESPACE pg_default;
CREATE INDEX IF NOT EXISTS front_building_type_124fcf82_like
ON public.front_building USING btree
(type COLLATE pg_catalog."default" varchar_pattern_ops ASC NULLS LAST)
TABLESPACE pg_default;
The m2m relation:
CREATE TABLE IF NOT EXISTS public.front_estate_buildings
(
id integer NOT NULL DEFAULT nextval('front_estate_buildings_id_seq'::regclass),
estate_id integer NOT NULL,
building_id integer NOT NULL,
CONSTRAINT front_estate_buildings_pkey PRIMARY KEY (id),
CONSTRAINT front_estate_buildings_estate_id_building_id_863b3358_uniq UNIQUE (estate_id, building_id),
CONSTRAINT front_estate_buildin_building_id_fc5c4235_fk_front_bui FOREIGN KEY (building_id)
REFERENCES public.front_building (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
DEFERRABLE INITIALLY DEFERRED,
CONSTRAINT front_estate_buildings_estate_id_2c28ec2a_fk_front_estate_id FOREIGN KEY (estate_id)
REFERENCES public.front_estate (id) MATCH SIMPLE
ON UPDATE NO ACTION
ON DELETE NO ACTION
DEFERRABLE INITIALLY DEFERRED
)
CREATE INDEX IF NOT EXISTS front_estate_buildings_building_id_fc5c4235
ON public.front_estate_buildings USING btree
(building_id ASC NULLS LAST)
TABLESPACE pg_default;
CREATE INDEX IF NOT EXISTS front_estate_buildings_estate_id_2c28ec2a
ON public.front_estate_buildings USING btree
(estate_id ASC NULLS LAST)
TABLESPACE pg_default;
To have a shared wall a building must touch another building which is not the same estate.
The final data set will have around 100 millions rows. Right now my developpement building table has 2 millions rows.
Here is the query I used to get all relations between buildings and estates:
SELECT b.id as b_id, rel.estate_id as e_id, swb.id as swb_id, sw_rel.estate_id as swe_id
FROM front_building b
JOIN front_building swb ON swb.id < b.id AND ST_Intersects(b.polygon, swb.polygon)
JOIN front_estate_buildings rel ON rel.building_id = b.id
JOIN front_estate_buildings sw_rel ON sw_rel.building_id = swb.id
ORDER BY b.id ASC;
Here is the EXPLAIN ANALYZE given by pgAdmin:
1. Limit (rows=500 loops=1)
2. Nested Loop Inner Join (rows=500 loops=1)
3. Nested Loop Inner Join (rows=695 loops=1)
4. Nested Loop Inner Join (rows=2985 loops=1)
5. Index Scan using front_estate_buildings_building_id_fc5c4235 on front_estate_buildings as rel (rows=2985 loops=1) 2985 1
6. Memoize (rows=1 loops=2985)
Buckets: Batches: Memory Usage: 715 kB
7. Index Scan using front_building_pkey on front_building as b (rows=1 loops=2751)
Index Cond: (id = rel.building_id)
8. Index Scan using front_building_polygon_idx on front_building as swb (rows=0 loops=2985)
Filter: ((id < b.id) AND st_intersects(b.polygon, polygon))
Index Cond: (polygon && b.polygon)
Rows Removed by Filter: 2
9. Index Scan using front_estate_buildings_building_id_fc5c4235 on front_estate_buildings as sw_rel (rows=1 loops=695)
Index Cond: (building_id = swb.id)
On my dev machine (MBP M1 - 16GB RAM) it completes in 20 minutes for 2M rows which is not good but is ok. On my production machine (Linode - 8 CPU Cores - 16 GB RAM) the CPU goes throufh the roof (continuous 250% capacity) and the query seems to never end.
Do you have any clue on how to proceed ? Change the query ? The db struct ? Use multiprocessing ?
In Redshift, is there a way to add multiple constraint to an already created table?
Step ONE, created the table (say month ago)
CREATE TABLE test_user
(
account_id BIGINT DEFAULT NULL encode mostly32,
ingest_timestamp BIGINT DEFAULT NULL ENCODE mostly32,
ingest_date TIMESTAMP DEFAULT NULL ENCODE delta32k,
user_id BIGINT NOT NULL encode mostly32,
group_id BIGINT NOT NULL encode mostly32,
department_id BIGINT NOT NULL encode mostly32,
name VARCHAR(255) DEFAULT NULL ENCODE LZO,
mobile VARCHAR(15) DEFAULT NULL ENCODE LZO,
phone VARCHAR(15) DEFAULT NULL ENCODE LZO,
user_language VARCHAR(15) DEFAULT NULL ENCODE runlength,
deleted INT DEFAULT NULL ENCODE mostly8,
created_at TIMESTAMP DEFAULT NULL ENCODE delta32k,
updated_at TIMESTAMP DEFAULT NULL ENCODE delta32k,
PRIMARY KEY (user_id)
)
sortkey (account_id)
Step TWO, created more tables like 'test_group','test_department'
Step THREE, after a month, I am planning to add multiple constraints to 'test_user' table like this(but in separate query)
ALTER TABLE test_user ADD CONSTRAINT FK_1 FOREIGN KEY (group_id) REFERENCES test_group (group_id);
ALTER TABLE test_user ADD CONSTRAINT FK_1 FOREIGN KEY (department_id) REFERENCES test_department (department_id);
The above succeeds.
However, Is there a way to add multiple constraints in single query?
Tried like this,
option 1 -> ALTER TABLE test_user ADD CONSTRAINT FK_1 FOREIGN KEY (group_id) REFERENCES test_group (group_id),ADD CONSTRAINT FK_1 FOREIGN KEY (department_id) REFERENCES test_department (department_id);
option 2 -> ALTER TABLE test_user ADD CONSTRAINT FK_1 FOREIGN KEY (group_id) REFERENCES test_group (group_id), FK_1 FOREIGN KEY (department_id) REFERENCES test_department (department_id);
But got this error for both case,
An error occurred when executing the SQL command:
ALTER TABLE test_user ADD CONSTRAINT FK_1 FOREIGN KEY (group_id) REFERENCES test_group (group_id), FK_2 FOREIGN KEY (department_id) REFERENCES test_department (department_id); REFERENCES helpdesk_user (user_id)
ERROR: syntax error at or near "FK_2"
Is there a way to add multiple constraint to an already created table?
Please help
It doesn't matter when you define them, or even if you define them at all.
Redshift ignores constraints. Defining them has no effect, except that the optimizer may use them as additional information when creating the access plan.
However, I have created and used massive (billions of rows) databases in redshift, and I didn't define a single constraint of any kind and it all ran amazingly fast.
Answer: Don't bother.
The ALTER TABLE documentation does not indicate that it is possible to specify multiple constraints in one statement:
ALTER TABLE table_name
{
ADD table_constraint |
DROP CONSTRAINT constraint_name [ RESTRICT | CASCADE ] |
OWNER TO new_owner |
RENAME TO new_name |
RENAME COLUMN column_name TO new_name |
ADD [ COLUMN ] column_name column_type
[ DEFAULT default_expr ]
[ ENCODE encoding ]
[ NOT NULL | NULL ] |
DROP [ COLUMN ] column_name [ RESTRICT | CASCADE ] }
where table_constraint is:
[ CONSTRAINT constraint_name ]
{ UNIQUE ( column_name [, ... ] ) |
PRIMARY KEY ( column_name [, ... ] ) |
FOREIGN KEY (column_name [, ... ] )
REFERENCES reftable [ ( refcolumn ) ]}
On a side note, Redshift Pitfalls And How To Avoid Them has an interesting examination of the impact of constraints on Redshift queries (see "Beware of Constraints" section).
Moi guys, Matt here. I'm having trouble with a relatively complicated view. I have a parts and service table that each have unique identifiers for a given part/service. I'm trying to link these to a service invoice table and subsequent view as a M:N relationship, so I've set up intermediary relational tables, with both the invoice number (invoice primary key) and part/service number (part/service primary key) as the combined primary key. Here's my code for the whole relationship and view:
CREATE TABLE service_invoice
( servinv_Num VARCHAR2(10) CONSTRAINT serv_snum_PK PRIMARY KEY,
servinv_EmpID NUMBER(6) CONSTRAINT serv_empnum_FK REFERENCES employee(empID),
servinv_CustID NUMBER(6) CONSTRAINT serv_custid_FK REFERENCES customer(custID),
servinv_VIN VARCHAR2(25) CONSTRAINT serv_VIN_FK REFERENCES vehicle(vehicle_vin),
servinv_Terms VARCHAR2(6) CONSTRAINT serv_trms_NN NOT NULL,
servinv_Date DATE );
CREATE TABLE Parts
( PartID VARCHAR2(10) CONSTRAINT Part_PartID_PK PRIMARY KEY,
PartDesc VARCHAR2(50) CONSTRAINT Part_PartDesc_NN NOT NULL,
PartCharge NUMBER(4,2) CONSTRAINT Part_PartCharge_NN NOT NULL );
CREATE TABLE Service
( ServiceID VARCHAR2(10) CONSTRAINT Serv_ServID_PK PRIMARY KEY,
ServDesc VARCHAR2(50) CONSTRAINT Serv_ServName_NN NOT NULL,
ServCharge NUMBER(4,2) CONSTRAINT Serv_ServCharge_NN NOT NULL );
CREATE TABLE Serv_SI_Rel
( SI_num VARCHAR2(10) CONSTRAINT ServSI_SInum_FK REFERENCES service_invoice(servinv_Num),
ServiceID VARCHAR2(10) CONSTRAINT ServSI_ServID_FK REFERENCES Service(ServiceID),
CONSTRAINT ServSI_SInum_ServID_PK PRIMARY KEY(SI_num, ServiceID) );
CREATE TABLE Parts_SI_Rel
( SI_num VARCHAR2(10) CONSTRAINT PartSI_SInum_FK REFERENCES service_invoice(servinv_Num),
PartID VARCHAR2(10) CONSTRAINT PartSI_PartID_FK REFERENCES Parts(PartID),
CONSTRAINT PartSI_SInum_PartID_PK PRIMARY KEY(SI_num, PartID) );
CREATE OR REPLACE VIEW ServiceInvoiceDoc
AS
(
SELECT si.servinv_Num, si.servinv_Date, si.servinv_Terms,
es.empName,
sc.custName, sc.custHouse, sc.custCity,
sc.custState, sc.custZIP, sc.custPhone, sc.custEmail,
sv.vehicle_VIN, sv.vehicle_mileage,
srel.ServiceID,
prel.PartID,
s.ServDesc, s.ServCharge,
p.PartDesc, p.PartCharge,
SUM(s.ServCharge) TotalServCharges,
SUM(p.PartCharge) TotalPartsCharges,
( SUM(s.ServCharge)+SUM(p.PartCharge) ) SubTotalCharges,
( SUM(s.ServCharge)+SUM(p.PartCharge) )*0.0825 Taxes,
( SUM(s.ServCharge)+SUM(p.PartCharge) )*1.0825 TotalCharges
FROM service_invoice si
JOIN employee es
ON (es.empID = si.servinv_EmpID)
JOIN customer sc
ON (sc.custID = si.servinv_CustID)
JOIN vehicle sv
ON (sv.vehicle_VIN = si.servinv_VIN)
LEFT OUTER JOIN Serv_SI_Rel srel
ON (srel.SI_Num = si.servinv_Num)
LEFT OUTER JOIN Parts_SI_Rel prel
ON (prel.SI_num = si.servinv_Num)
JOIN Parts p
ON (prel.PartID = p.PartID)
JOIN Service s
ON (srel.ServiceID = s.ServiceID) );
The error I get has to do with summing the individual parts and service charges in the M:N relationship. Here's the error code from the run:
ORA-00937: not a single-group group function
I've tried fixing with a group by command, but the grouping identifier (service invoice) isn't included on the part or service tables, and the joins don't seem to link these up for a group. e.g. I tried calling GROUP BY si.servinv_Num
Can this be resolved at all or is it completely wrong? I have the option of dropping the M:N relationship as a 1:M and simply making a separate invoice for each part/service charge, but I would prefer to keep it compact and professional.
Any help would be greatly appreciated. Thank you so much for your time!
a) wrong tag
b) I'd imagine you would need to list all columns in group by clause that aren't aggregated, as per Oracle
...
group by si.servinv_Num, si.servinv_Date, si.servinv_Terms,
es.empName,
sc.custName, sc.custHouse, sc.custCity,
sc.custState, sc.custZIP, sc.custPhone, sc.custEmail,
sv.vehicle_VIN, sv.vehicle_mileage,
srel.ServiceID,
prel.PartID,
s.ServDesc, s.ServCharge,
p.PartDesc, p.PartCharge
I tried with normal sql query
SELECT activity_shares.id FROM `activity_shares`
INNER JOIN (SELECT `activity_id` FROM `activity_shares`
GROUP BY `activity_id`
HAVING COUNT(`activity_id`) > 1 ) dup ON activity_shares.activity_id = dup.activity_id
Which gives me record id say 10 and 11
But same query I tried to do in Doctrine query builder,
$qb3=$this->getEntityManager()->createQueryBuilder('c')
->add('select','c.id')
->add('from','MyBundleDataBundle:ActivityShare c')
->innerJoin('c.activity', 'ca')
// ->andWhere('ca.id = c.activity')
->groupBy('ca.id')
->having('count(ca.id)>1');
Edited:
$query3=$qb3->getQuery();
$query3->getResult();
Generated SQL is:
SELECT a0_.id AS id0 FROM activity_shares a0_
INNER JOIN activities a1_ ON a0_.activity_id = a1_.id
GROUP BY a1_.id HAVING count(a1_.id) > 1
Gives only 1 record that is 10.I want to get both.I'm not getting idea where I went wrong.Any idea?
My tables structure is:
ActivityShare
+-----+---------+-----+---
| Id |activity |Share| etc...
+-----+---------+-----+----
| 1 | 1 |1 |
+-----+---------+-----+---
| 2 | 1 | 2 |
+-----+---------+-----+---
Activity is foreign key to Activity table.
I want to get Id's 1 and 2
Simplified SQL
first of all let me simplify that query so it gives the same result :
SELECT id FROM `activity_shares`
GROUP BY `id`
HAVING COUNT(`activity_id`) > 1
Docrtrine QueryBuilder
If you store the id of the activty in the table like you sql suggests:
You can use the simplified SQL to build a query:
$results =$this->getEntityManager()->createQueryBuilder('c')
->add('select','c.id')
->add('from','MyBundleDataBundle:ActivityShare c')
->groupBy('c.id')
->having('count(c.activity)>1');
->getResult();
If you are using association tables ( Doctrine logic)
here you will have to use join but the count may be tricky
Solution 1
use the associative table like an entitiy ( as i see it you only need the id)
Let's say the table name is activityshare_activity
it will have two fields activity_id and activityshare_id, if you find a way to add a new column id to that table and make it Autoincrement + Primary the rest is easy :
the new entity being called ActivityShareActivity
$results =$this->getEntityManager()->createQueryBuilder('c')
->add('select','c.activityshare_id')
->add('from','MyBundleDataBundle:ActivityShareActivity c')
->groupBy('c.activityshare_id')
->having('count(c.activity_id)>1');
->getResult();
the steps to add the new identification column to make it compatible with doctrine (you need to do this once):
add the column (INT , NOT NULL) don' t put the autoincrement yet
ALTER TABLE tableName ADD id INT NOT NULL
Populate the column using a php loop like for
Modify the column to be autoincrement
ALTER TABLE tableName MODIFY id INT NOT NULL AUTO_INCREMENT
Solution2
The correction to your query
$result=$this->getEntityManager()->createQueryBuilder()
->select('c.id')
->from('MyBundleDataBundle:ActivityShare', 'c')
->innerJoin('c.activity', 'ca')
->groupBy('c.id') //note: it's c.id not ca.id
->having('count(ca.id)>1')
->getResult();
I posted this one last because i am not 100% sure of the output of having+ count but it should word just fine :)
Thanks for your answers.I finally managed to get answer
My Doctrine query is:
$subquery=$this->getEntityManager()->createQueryBuilder('as')
->add('select','a.id')
->add('from','MyBundleDataBundle:ActivityShare as')
->innerJoin('as.activity', 'a')
->groupBy('a.id')
->having('count(a.id)>1');
$query=$this->getEntityManager()->createQueryBuilder('c')
->add('select','c.id')
->add('from','ChowzterDataBundle:ActivityShare c')
->innerJoin('c.activity', 'ca');
$query->andWhere($query->expr()->in('ca.id', $subquery->getDql()))
;
$result = $query->getQuery();
print_r($result->getResult());
And SQL looks like:
SELECT a0_.id AS id0 FROM activity_shares a0_ INNER JOIN activities a1_ ON a0_.activity_id = a1_.id WHERE a1_.id IN (SELECT a2_.id FROM activity_shares a3_ INNER JOIN activities a2_ ON a3_.activity_id = a2_.id GROUP BY a2_.id HAVING count(a2_.id) > 1