How to deal with django migration postgres deadlock? - django

So, I was deploying the django app to production and then the infamous postgres deadlock situation happened. This happens during the django migration.
Django version: 3.2
Postgres 13
Google cloud sql postgres.
OperationalError
deadlock detected
DETAIL: Process 112506 waits for AccessExclusiveLock on relation 29425 of database 27145; blocked by process 112926.
Process 112926 waits for RowShareLock on relation 29381 of database 27145; blocked by process 112506.
HINT: See server log for query details.
I ran this query to get the process info:
SELECT 29425::regclass,29381::regclass from pg_locks;
and result:
regclass | regclass
"custom_requestlog" "auth_user"
"custom_requestlog" "auth_user"
I am not sure how to proceed ahead, as the pgaudit has been enabled but it doesnt show anything and also the query insights is not that helpful. Attached is image of query insights.
Any help would be helpful please!
Update:
The query from log explorer in Google cloud, gave this query just after deadlock detected error:
2022-04-29 13:51:36.590 UTC [6445]: [732-1] db=xyz_prod,user=backend-prod DETAIL: Process 6445 waits for AccessExclusiveLock on relation 29425 of database 27145; blocked by process 9249.
Process 9249 waits for RowShareLock on relation 29381 of database 27145; blocked by process 6445.
Process 6445: SET CONSTRAINTS "custom_requestlog_user_id_3ff3f1cf_fk_some_user_id" IMMEDIATE; ALTER TABLE "custom_requestlog" DROP CONSTRAINT "custom_requestlog_user_id_3ff3f1cf_fk_some_user_id"
Process 9249: INSERT INTO "custom_requestlog" ("user_id", "ip_addr", "url", "session_key", "method", "headers", "query", "body", "cookies", "timestamp", "status_code", "response_snippet") VALUES (NULL, 'xx.xxx.xx.xxx'::inet, '/version/', NULL, 'GET', '{"HTTP_HOST": "api.some.com", "HTTP_ACCEPT": "*/*", "HTTP_ACCEPT_ENCODING": "deflate, gzip", "HTTP_USER_AGENT": "GoogleStackdriverMonitoring-UptimeChecks(https://cloud.google.com/monitoring)", "HTTP_X_CLOUD_TRACE_CONTEXT": "xxxxxx/9771676669485105781", "HTTP_VIA": "1.1 google", "HTTP_X_FORWARDED_FOR": "xx.xxx.xx.xxx, xx.xxx.xx.xxx", "HTTP_X_FORWARDED_PROTO": "https", "HTTP_CONNECTION": "Keep-Alive"}', '{}', '\x'::bytea, '{}', '2022-04-29T13:48:46.844830+00:00'::timestamptz, 200, NULL) RETURNING "custom_requestlog"."id"

Related

Refresh Materialized Views Redshift

I am running into the following error
SQL Error [XX000]: ERROR: Assert
Detail:
-----------------------------------------------
error: Assert
code: 1000
context: mv_query != "" - Expected non-empty MV CTAS
query: 0
location: refresh.cpp:1570
process: padbmaster [pid=14118]
whenever I execute the SQL statement REFRESH MATERIALIZED VIEW <viw_name> in AWS Redshift. The views themselves are not empty so I do not really know how to resolve this. Would appreciate some help here
When you encounter the error, best fix is to drop your views, recreate them and then re-run the refresh materialized views statement.

After success validation also getting error message but its processing in oracle apex

Dynamic Action ->When:click button->If:True->submit page.here i am doing a validation with a sql query.
SELECT * FROM TAB_MASTER
WHERE REQUEST_ID=:P4_SELECTED_REQ_ID AND (upper(USER_NAME)=upper(:APP_USER) OR
EXISTS (SELECT NULL FROM USER_ACCOUNT WHERE USER_ROLE='ADMIN' AND upper(USER_NAME)=upper(:APP_USER)));
means request can be confirmed by same user or another user with ADMIN role.if the above query will return 1 row only should process otherwise error message (no process).If there is no record(another user and no admin role) its working fine.but when it return 1 record(another user and admin role) its processing but error message is getting pop up what used for FALSE condition.
Also tried to set value to the output sql query to a hidden field then used validation as plsql/expression :HIDDEN_FIELD=1,but no luck.

Empty User-Agent causes sql exception column "os" violates not-null constraint

Having WSO2AM (API Manager) 2.1.0 with an Analytics server (on PostgreSQL) we see following issue:
if a client call an API with no User-Agent header, the analytics server is unable to complete processing and in the logs of the analytics server (wso2carbon.log) there are following exceptions:
TID: [-1] [] [2017-10-10 10:55:17,292] ERROR {org.apache.spark.executor.Executor} - Exception in task 0.0 in stage 41522.0 (TID 32785) {org.apache.spark.executor.Executor}
java.sql.BatchUpdateException: Batch entry 0 INSERT INTO API_REQ_USER_BROW_SUMMARY (api, version, apiPublisher, tenantDomain, total_request_count, year, month, day, requestTime, os, browser) VALUES ('agenda.brussels', '0.0.2', 'admin', 'carbon.super', 1, 2017, 10, 10, 1507625491000, NULL, NULL) ON CONFLICT (api,version,apiPublisher,year,month,day,os,browser,tenantDomain) DO UPDATE SET total_request_count=EXCLUDED.total_request_count, requestTime=EXCLUDED.requestTime was aborted: ERROR: null value in column "os" violates not-null constraint
Detail: Failing row contains (agenda.brussels, 0.0.2, admin, carbon.super, 1, 2017, 10, 10, 1507625491000, null, null). Call getNextException to see other errors in the batch.
at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:145)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2156)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:460)
at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:793)
at org.postgresql.jdbc.PgPreparedStatement.executeBatch(PgPreparedStatement.java:1659)
...
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "os" violates not-null constraint
Detail: Failing row contains (agenda.brussels, 0.0.2, admin, carbon.super, 1, 2017, 10, 10, 1507625491000, null, null).
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155)
... 16 more
Apparently the "os" field is assumed as not empty. The APIs and clients are public, so we have no way to enforce the request sent from the clients.
I'd see temporary measures
set some User-Agent value if empty in the http proxy level (I am not sure if we are allowed to)
is it feasible to update the processing script to use coalesce postgresql function? (returning some default value of null argument is used).
Questions:
is it a bug (known/unknown) or we missed something? If so, what would be the best way to resolve it?
Thank all for any hint

Loopback accessToken not persisted to database

The access tokens created are not being persisted to the postgresql database. I created extended users and access_tokens models using the strongloop controller interface using the User and AccessToken models as parents. On login the user is successfully queried from the database, login is successful, and the access token is created. However the access token is not persisted in the database and so essentially the login fails.
Inside my model_config.json I have the following entries:
"access_tokens": {
"dataSource": "postgresdb",
"public": true
},
"users": {
"dataSource": "postgresdb",
"public": true
}
When investigating, the create function is called and the accessToken.beforeCreate() properly triggered and successfully returned. However the database does not store the token. When altering the access_tokens database to use db (memory) the access token created still cannot be used for authentication.
I am certain the database settings are correct because the access_tokens table is on the same db as the users and the users successfully grabs all the user information.

error using django-admin with django-mssql

I am using django 1.6 with django-mssql
My django admin site works perfectly except for one part.
Whenever I click an "add user" link or otherwise navigate to /admin/auth/user/add I get the following error
(-2147352567, 'Exception occurred.', (0, u'Microsoft OLE DB Provider for SQL Server', u'Cannot issue SAVE TRANSACTION when there is no active transaction.', None, 0, -2147467259), None)
Command:
SAVE TRANSACTION [s3612_x1]
Parameters:
[]
I've tried various seetings in my database config such as "use_mars": true but there has been no change.
I am able to create users in code by using the User model without issue.
I have not encountered this error anywhere else.
Fixed by setting 'ATOMIC_REQUESTS': True for my db connection, which I should have been doing anyway.