Cannot dump Django database using pg_dump due to permissions - django

I want to dump PostgreSQL database to an SQL file. Here is the pg_dump DB_NAME > dump.sql command result:
pg_dump: [archiver (db)] query failed: ERROR: permission denied for relation django_migrations
pg_dump: [archiver (db)] query was: LOCK TABLE public.django_migrations IN ACCESS SHARE MODE
When I want to list relations:
postgres=# \d
No relations found.
List of schemas:
List of schemas
Name | Owner | Access privileges | Description
--------+----------+----------------------+------------------------
public | postgres | postgres=UC/postgres+| standard public schema
| | =UC/postgres +|
| | root=U/postgres +|
| | secretusr=U/postgres |
(1 row)
Database list:
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+------------------------
db_secret | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres +
| | | | | postgres=CTc/postgres +
| | | | | secretusr=CTc/postgres+
| | | | | root=CTc/postgres
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
Where is the problem?

Related

PowerBI filter Table on 2 different columns whose values are obtained from another table

I am new to PowerBI. I am trying to implement the following scenario in PowerBI
I have the following 2 tables -
Table 1:
| ExtractionId | DatasetId | UserId |
| -- | --- | --- |
| E_ID1 | D_ID1 | sta#example.com |
| E_ID2 | D_ID1 | dany#example.com |
| E_ID3 | D_ID2 | dany#example.com |
Table 2:
| DatasetId | Date | UserId | Status |
| --| --- | --- | --- |
| D_ID1 | 05/30/2021 | sta#example.com | Completed |
| D_ID1 | 05/30/2021 | dany#example.com | Completed |
| D_ID1 | 05/31/2021 | sta#example.com | Partial |
| D_ID1 | 05/31/2021 | dany#example.com | Completed |
| D_ID2 | 05/30/2021 | sta#example.com | Completed |
| D_ID2 | 05/30/2021 | dany#example.com | Completed |
| D_ID2 | 05/31/2021 | sta#example.com | Partial |
| D_ID2 | 05/31/2021 | dany#example.com | Completed |
I am trying to create a PowerBI report where, given an extraction id (in a slicer), we need to identify the corresponding DatasetId and UserID from Table 1 and use those fields to filter Table 2 and provide a visual of user status on the given date range.
When I am trying to implement the above scenario, I am creating a Many-Many relationship between DatasetID columns of Table1 and Table2, but cannot do the same for UserID column simultaneously as I get the following error :
You can't create a direct active relationship between Table1 and Table2 because an active set of indirect relationship already exists.
Because of this, given an extractionId, I can filter on DatasetID but not UserId and vice versa. Could you please help me understand what mistake I am doing here and how to resolve the same?
Thanks in advance
This case you said too. You can only merge two or more columns. Than you will create relationships.

Format AWS CLI output as *markdown* table

I am trying to format the AWS CLI table output so that it shows as a 'nice' formatted table in markdown in typora, github md files, etc...
For example, the original table-formatted output from the AWS CLI command
$ aws ec2 describe-subnets --query "Subnets[*].{CIDR:CidrBlock,Name:Tags[?Key=='Name']|[0].Value,AZ:AvailabilityZone}" --output table
is
----------------------------------------------------------------------
| DescribeSubnets |
+------------+------------------+------------------------------------+
| AZ | CIDR | Name |
+------------+------------------+------------------------------------+
| eu-west-3c| 10.1.103.0/24 | vpc-acme-test-public-eu-west-3c |
| eu-west-3b| 172.31.16.0/20 | None |
| eu-west-3a| 10.1.101.0/24 | vpc-acme-test-public-eu-west-3a |
| eu-west-3c| 10.1.3.0/24 | vpc-acme-test-private-eu-west-3c |
| eu-west-3b| 10.1.2.0/24 | vpc-acme-test-private-eu-west-3b |
| eu-west-3a| 172.31.0.0/20 | None |
| eu-west-3c| 172.31.32.0/20 | None |
| eu-west-3a| 10.1.1.0/24 | vpc-acme-test-private-eu-west-3a |
| eu-west-3b| 10.1.102.0/24 | vpc-acme-test-public-eu-west-3b |
+------------+------------------+------------------------------------+
Based on assorted markdown tutorials and tests, the output that would render properly as a table in typora and github is something like:
| AZ | CIDR | Name |
|------------|------------------|------------------------------------|
| eu-west-3c| 10.1.103.0/24 | vpc-acme-test-public-eu-west-3c |
| eu-west-3b| 172.31.16.0/20 | None |
| eu-west-3a| 10.1.101.0/24 | vpc-acme-test-public-eu-west-3a |
| eu-west-3c| 10.1.3.0/24 | vpc-acme-test-private-eu-west-3c |
| eu-west-3b| 10.1.2.0/24 | vpc-acme-test-private-eu-west-3b |
| eu-west-3a| 172.31.0.0/20 | None |
| eu-west-3c| 172.31.32.0/20 | None |
| eu-west-3a| 10.1.1.0/24 | vpc-acme-test-private-eu-west-3a |
| eu-west-3b| 10.1.102.0/24 | vpc-acme-test-public-eu-west-3b |
(the text above does not render as a table in stackoverflow. Below a screenshot of this table rendered in typora:
I could not find any AWS CLI option, but the following unix-like series of filters does the job.
Pipe the output of the AWS command to:
sed s/'+'/'|'/g | tail -n +4 | head -n -1
The full CLI command is:
$ aws ec2 describe-subnets --query "Subnets[*].{CIDR:CidrBlock,Name:Tags[?Key=='Name']|[0].Value,AZ:AvailabilityZone}" --output table | sed s/'+'/'|'/g | tail -n +4 | head -n -1
Other suggestions welcome!

Some TextFields created in database with NOT NULL YES/NO

I have the following Model:
class SystemMessage(Model):
subject = TextField(default=None)
message = TextField(default=None)
And the following output from explain
| id | int(11) | NO | PRI | NULL | auto_increment |
| language_code | varchar(5) | NO | | NULL | |
| subject | longtext | NO | | NULL | |
| message | longtext | YES | | NULL | |
I have no outstanding migrations or makemigrations
Is there any reason why message is NULLable but subject is not? I would like both to be NOT NULLable.
I'm provided a default value for each field to ensure IntegrityError when saving the strings with no value.

Two outer joins in django's queryset (for language fall back case)

I have two models, Version and Description.
class Version(models.Model):
version_name = models.CharField(max_length=100)
version_value = models.IntegerField()
url = models.CharField(max_length=240)
class Description(models.Model):
version = models.ForeignKey(Version)
lang = models.CharField(max_length=8)
content = models.TextField()
And a DescriptionSerializer.
class DescriptionSerializer(serializers.ModelSerializer):
version_name = serializers.RelatedField(source='version')
class Meta:
model = Description
fields = ('version_name', 'content')
They stored the descriptions of different versions in different languages.
E.g.
Version
+----+--------------+---------------+---------------------+
| id | version_name | version_value | url |
+----+--------------+---------------+---------------------+
| 1 | 1.0.0 | 1 | http://abc.net.tw/ |
| 2 | 1.0.1 | 2 | http://abc.net.tw/2 |
| 3 | 1.0.2 | 3 | http://abc.net.tw/3 |
| 4 | 1.0.3 | 4 | http://abc.net.tw/4 |
| 7 | 1.1.0 | 5 | http://abc.net.tw/5 |
| 8 | 1.1.1 | 6 | http://abc.net.tw/6 |
+----+--------------+---------------+---------------------+
Description
+------------+-------+---------+
| version_id | lang | content |
+------------+-------+---------+
| 1 | en_US | English |
| 1 | zh_TW | Chinese |
| 1 | es_ES | Spanish |
| 2 | en_US | English |
| 2 | zh_TW | Chinese |
| 2 | es_ES | Spanish |
| 3 | en_US | English |
| 3 | zh_TW | Chinese |
| 3 | es_ES | Spanish |
| 4 | en_US | English |
| 7 | en_US | English |
| 8 | en_US | English |
| 4 | es_ES | Spanish |
| 7 | es_ES | Spanish |
+------------+-------+---------+
I'm using django rest framework to implement a web API that returns the description of each version in certain language. If a description of certain language doesn't exist, use English version instead.
I can use following SQL to retrieve the desired result. I've read DRF's docs on relatedField and reverse relation. But I still can't figure out how to use django's ORM to do the same thing and to use it with django rest framework's serializer.
select
coalesce(d.id, d2.id), coalesce(d.version_id, d2.version_id), coalesce(d.lang, d2.lang), coalesce(d.content, d2.content)
from
version v
left outer join description d on v.id = d.version_id and d.lang='zh_TW'
left outer join description d2 on v.id = d2.version_id and d2.lang='en_US'
Please advise how to do it in django.
You can't use django orm for everything. There are numerous things you can't do with django. For those cases you either use straight up SQL (from django.db import connection, transaction etc...) or if the query results can be worked into objects you have described - then you can use raw queries (link)

Doctrine 2 memory hogging

I'm using Doctrine 2 with my ZF2 project, but i'm getting some weird problem with my server CPU and memory. And my server simply crashes.
I'm getting a lot of sleep state querys and they seem not to get cleaned.
mysql> show processlist;
+---------+--------------+-----------+------------------+----------------+------+--------------------+------------------------------------------------------------------------------------------------------+
| Id | User | Host | db | Command | Time | State | Info |
+---------+--------------+-----------+------------------+----------------+------+--------------------+------------------------------------------------------------------------------------------------------+
| 2832346 | leechprotect | localhost | leechprotect | Sleep | 197 | | NULL |
| 2832629 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832643 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832646 | db_user | localhost | db_exemple | Sleep | 3 | | NULL |
| 2832664 | db_user | localhost | db_exemple | Sleep | 154 | | NULL |
| 2832666 | db_user | localhost | db_exemple | Sleep | 153 | | NULL |
| 2832669 | db_user | localhost | db_exemple | Sleep | 152 | | NULL |
| 2832674 | db_user | localhost | db_exemple | Sleep | 7 | | NULL |
| 2832681 | db_user | localhost | db_exemple | Sleep | 1 | | NULL |
| 2832683 | db_user | localhost | db_exemple | Sleep | 4 | | NULL |
| 2832690 | db_user | localhost | db_exemple | Sleep | 149 | | NULL |
(.......)
Also, it seems php GC is not cleaning all the objects from memory, or even kill processes.
Is there a way to disable the cache system? Would it improve the use of my resorces=
Most my querys are similar to:
$query = $this->createQueryBuilder('i');
$query->innerJoin('\Application\Relation', 'r', 'WITH', 'r.child = i.id');
$query->innerJoin('\Application\Taxonomy', 't', 'WITH', 't.id = r.taxonomy');
$query->where('t.type = :type')->setParameter('type', $relation);
$query->groupBy('i.id');
$items = $query->getQuery()->getResult(2);
Thanks in advance.
Firstly check the mysql's wait_timout variable. From the documentation:
Wait_timeout : The number of seconds the server waits for activity on
a noninteractive connection before closing it.
In normal flow (which not using persistent connections), php closes the connection automatically after script execution. To ensure there are no sleeping threads; at the end of your script simply close the connection:
$entityManager->getConnection()->close();
If these queries are running in a big while/for loop, you might want to read doctrine 2 batch processing documentation.