Today something strange happened. I'm importing data from a payment gateway after request:
for signature in response.json:
Signature.objects.get_or_create(**signature)
json example:
[
{'id': 1, 'plan': 1, 'customer': 1},
{'id': 31, 'plan': 12, 'customer': 22}
{'id': 2, 'plan': 3, 'customer': 50},
{'id': 3111, 'plan': 12, 'customer': 22},
{'id': 222, 'plan': 12, 'customer': 22},
]
Yes, my client didn't follow an ID sequence registering signatures manually on payment service so, I'm importing and keeping the same pk.
This code works as expected and data is now synced with payment service (all objects imported).
Now the strange behavior:
I'm using Django Rest Framework and after a POST (check validated_data) in my API the following error raises at this line:
Signature.object.create(**self.validated_data)
duplicate key value violates unique constraint
"plans_signature_pkey" DETAIL: Key (id)=(1) already exists.
validated data:
{
"plan": "3", # This is a foreign key to plan 3
"payer_only": False,
"schedule": "09:00",
"payment_method: "CREDIT_CARD"
}
There is no 'pk': 1 or 'id': 1 in validated data
Django is trying to create an object with an existing key?
Debugging code, I called the Subscription.create() line 31 times then:
duplicate key value violates unique constraint "plans_signature_pkey"
DETAIL: Key (id)=(1) already exists.
....
duplicate key value violates unique constraint "plans_signature_pkey"
DETAIL: Key (id)=(31) already exists.
On call 32 this works. So, am I missing something? This looks a weird behavior to me.
After a long research I've found the solution.
I'm using POSTGRES and Django uses PostgreSQL’s SERIAL data type to store auto-increment keys. A manual primary key assign stops the pk auto-increment.
The solution: sqlsequencereset
$ django-admin sqlsequencereset app_label
"Use this command to generate SQL which will fix cases where a sequence is out of sync with its automatically incremented field data."
Related
I'm trying to use Boto3 to get the number of vulnerabilities from my images in my repositories. I have a list of repository names and image IDs that are getting passed into this function. Based off their documentation
I'm expecting a response like this when I filter for ['imageScanFindings']
'imageScanFindings': {
'imageScanCompletedAt': datetime(2015, 1, 1),
'vulnerabilitySourceUpdatedAt': datetime(2015, 1, 1),
'findingSeverityCounts': {
'string': 123
},
'findings': [
{
'name': 'string',
'description': 'string',
'uri': 'string',
'severity': 'INFORMATIONAL'|'LOW'|'MEDIUM'|'HIGH'|'CRITICAL'|'UNDEFINED',
'attributes': [
{
'key': 'string',
'value': 'string'
},
]
},
],
What I really need is the
'findingSeverityCounts' number, however, it's not showing up in my response. Here's my code and the response I get:
main.py
repo_names = ['cftest/repo1', 'your-repo-name', 'cftest/repo2']
image_ids = ['1.1.1', 'latest', '2.2.2']
def get_vuln_count(repo_names, image_ids):
container_inventory = []
client = boto3.client('ecr')
for n, i in zip(repo_names, image_ids):
response = client.describe_image_scan_findings(
repositoryName=n,
imageId={'imageTag': i}
)
findings = response['imageScanFindings']
print(findings)
Output
{'findings': []}
The only thing that shows up is findings and I was expecting findingSeverityCounts in the response along with the others, but nothing else is showing up.
THEORY
I have 3 repositories and an image in each repository that I uploaded. One of my theories is that I'm not getting the other responses, such as findingSeverityCounts because my images don't have vulnerabilities? I have inspector set-up to scan on push, but they don't have vulnerabilities so nothing shows up in the inspector dashboard. Could that be causing the issue? If so, how would I be able to generate a vulnerability in one of my images to test this out?
My theory was correct and when there are no vulnerabilities, the response completely omits certain values, including the 'findingSeverityCounts' value that I needed.
I created a docker image using python 2.7 to generate vulnerabilities in my scan to test out my script properly. My work around was to implement this if statement- if there's vulnerabilities it will return them, if there aren't any vulnerabilities, that means 'findingSeverityCounts' is omitted from the response, so I'll have it return 0 instead of giving me a key error.
Example Solution:
response = client.describe_image_scan_findings(
repositoryName=n,
imageId={'imageTag': i}
)
if 'findingSeverityCounts' in response['imageScanFindings']:
print(response['imageScanFindings']['findingSeverityCounts'])
else:
print(0)
Hi Stackoverflow I'm trying to conditionally put an item within a DynamoDB table. The DynamoDB table has the following attributes.
ticker - Partition Key
price_date - Sort Key
price - Attribute
Every minute I'm calling an API which gives me a minute by minute list of dictionaries for all stock prices within the day so far. However, the data I receive from the API sometimes can be behind by a minute or two. I don't particularly want to overwrite all the records within the DynamoDB table every time I get new data. To achieve this I've tried to create a conditional expression to only use put_item when there is a match on ticker but there is a new price_date
I've created a simplification of my code below to better illustrate my problem.
import boto3
from boto3.dynamodb.conditions import Attr
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('stock-intraday')
data = [
{'ticker': 'GOOG', 'price_date': '2021-10-08T9:30:00.000Z', 'price': 100},
{'ticker': 'GOOG', 'price_date': '2021-10-08T9:31:00.000Z', 'price': 101}
]
for item in data:
dynamodb_response = table.put_item(Item=item,
ConditionExpression=Attr("ticker").exists() & Attr("price_date").not_exists())
However when I run this code I get this error...
What is wrong with my conditional expression?
Found an answer to my own problem. DynamoDB was throwing an error because my code WAS working but with some minor changes.
There needed to be a TRY EXCEPT block but also since the partition key is already evaluated only the price_date needed to be included within the condition expression
import boto3
from boto3.dynamodb.conditions import Attr
from botocore.exceptions import ClientError
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('stock-intraday')
data = [
{'ticker': 'GOOG', 'price_date': '2021-10-08T9:30:00.000Z', 'price': 100},
{'ticker': 'GOOG', 'price_date': '2021-10-08T9:31:00.000Z', 'price': 101}]
for item in data:
try:
dynamodb_response = table.put_item(Item=item,
ConditionExpression=Attr("price_date").not_exists())
except ClientError as e:
if e.response['Error']['Code'] == 'ConditionalCheckFailedException':
pass
Hi I am trying to receive one last message per user. I have filtered out messages but I am getting many messages for one user. How can I get last message of user.
queryset=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
return queryset
I am getting this list from filtering.
[
{
"message": "Hi b do you want to buy? 2",
"sUser": 1,
"rUser": 15,
"sent": true,
"pk": 8,
"user": {
"pk": 15
}
},
{
"message": "a to c",
"sUser": 1,
"rUser": 16,
"sent": true,
"pk": 7,
"user": {
"pk": 16
}
},
{
"message": "No thanks not buying this time",
"sUser": 15,
"rUser": 1,
"sent": false,
"pk": 6,
"user": {
"pk": 15
}
},
{
"message": "c to a",
"sUser": 16,
"rUser": 1,
"sent": false,
"pk": 4,
"user": {
"pk": 16
}
},
{
"message": "Hi b do you want to buy?",
"sUser": 1,
"rUser": 15,
"sent": true,
"pk": 1,
"user": {
"pk": 15
}
}
]
I want to get message with pk 8 and 7 only because it is the last message to user 15 and 16 but I am getting all messages for 15 and 16.
Ok I guess you are going for type like facebook messenger's dashboard. First get the related message of the user
relatedMessage=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
Add the first message to a list
convoList.append(relatedMessage.latest('id')) #since first message is unique
Then find out the friend from each one so that you can distinct who exactly the friend is (either sender or receiver which may be you or friend, since we are filtering message sender or receiver being you or your friend).
friend = message.sUser
if message.sUser==self.request.user:
friend=message.rUser
Great! You found which one your friend is.
Now for each message in relatedMessage check if message of you friend exist in your convoList or not. If it does not exist then add it to the convolist. Finally create serializer and return its data or queryset whichever you prefer.
This is preferred because it just makes one query to the database hence creating calculating part in the server. If you had first got related user from database then gone for again filtering message, it would be much costly due to huge query twice in database.
you have to
queryset=MessageModel.objects.filter(
Q(sUser=self.request.user) | Q(rUser=self.request.user)
).order_by('-id')
if queryset.exists():
return queryset[0]
else:
return None
Hope this will work for you
I'm not sure the Q expressions in your original query match what you asked for, but let's assume you want the most recent received message for every user and that you can depend on the ordering of id such that the higher the id the more recent.
If you are using a database backend that supports DISTINCT ON you can do the following:
most_recent_messages = MessageModel.objects.all().order_by('rUser', '-id').distinct('rUser')
You can add a filter instead of all if you want to filter the queryset further to particular users for example.
I am writing a Django command to seed an existing table,
I need to truncate the table before seeding, but there are foreign key constraints on that table.
because of that, I am getting django.db.utils.IntegrityError while truncating the table,
How do I turn the Foreign Key Checks off temporarily in Django?
I saw SET FOREIGN KEY CHECK = 0 but don't know where to put them :(
The Django Command class:
class Command(BaseCommand):
help = "Command to seed the aws regions"
regions = [
{
'name': 'Us East (N. Virginia)',
'region': 'us-east-1',
},
{
'name': 'US West (Oregon)',
'region': 'us-west-2',
},
{
'name': 'EU (Ireland)',
'region': 'eu-west-1',
},
]
def handle(self, *args, **options):
self.stdout.write('seeding regions...')
AwsRegions.objects.all().delete() # this is where i get errors
for name, region in self.regions:
self.stdout.write(region)
AwsRegions.objects.create(name, region)
self.stdout.write('done seeding regions')
Got the solution.
I had to disable the Triggers on the table to stop the foreign key constraint check.
Disable Triggers
def disable_triggers(self):
with connection.cursor() as cursor:
cursor.execute('ALTER TABLE "Table Name" DISABLE TRIGGER ALL;')
Enable Triggers
def enable_triggers(self):
with connection.cursor() as cursor:
cursor.execute('ALTER TABLE "Table Name" ENABLE TRIGGER ALL;')
Important Notes:
According to this doc link, you can pass a list as the second argument to the execute() method (eg: you might want to pass the table name dynamically), but this will automatically escape the variables and you might end up forming a syntactically wrong PostgreSQL query (which took a lot of my time to fix it)
Make sure you turn the triggers back on properly
If you are getting a Permission denied error Then you might want to check the DB user permissions, I just turned on superuser permissions from PgAdmin, which was ok for me. and everything back to work. How to do it ?
To disable triggers for all tables (useful when you need to stop it for multiple tables):
SET session_replication_role TO 'replica'
And to restore:
SET session_replication_role TO 'origin'
from django.db import connection
with connection.constraint_checks_disabled():
do_stuff()
Credit goes to https://stackoverflow.com/a/11926432/2558400
I'm on a page which sends a request to server side for a list of comments of current topic.
like this
localhost:8000/getComments/567 , while 567 is the topic id.
then, in my django view, like this:
def getProjectComments(request, pId):
format = 'json'
mimetype = 'application/javascript' #'application/xml' for 'xml' format
comments = PrjComment.objects.filter(prj=pId)
data = serializers.serialize(format, comments)
return HttpResponse(data,mimetype)
Now , the question is ,
when I try to use jQuery.parseJSON(data) at the browser side.
[
{
"pk": 10,
"model": "app.prjcomment",
"fields":
{
"status": 1,
"time_Commented": "2011-12-11 17:23:56",
"prj": 1,
"content": "my comment 1",
"user": 25
}
},
{
"pk": 9,
"model": "app.prjcomment",
"fields": {
"status": 1,
"time_Commented": "2011-12-11 17:23:51",
"prj": 1,
"content": "comment \u4e00\u4e2a",
"user": 33
}
} ..
I need to use some detail information of user object. (user is a Foreign Key in model PrjComment)
such as user.first_name to display on the comment list.
but here it is only an id for user.("user": 33)
How can I do that? Anyone who can kind help?
Thank you very much
the User is the Django auth_user.
The easy solution would be to specify the values you need:
comments = PrjComment.objects.filter(prj=pId) \
.values('status','time_commented','prj','content','user__id',
'user__username','user__first_name')
Update:
To access your userprofile information use the table name as defined in your AUTH_PROFILE_MODULE setting. In my case the table is called userprofile, and I'd access the data like this:
values('user__userprofile__moreinfo')
where moreinfo is the field you're interested in