Djanjo Unique Constraint - Upload BUT Skip Duplicates - django

Question

I have a django unique constraint where im using the django admin site to import .csv files. The constraint is working as expected but I would like to just skip over the duplicates and still add the valid records. Is there a method to get this behavior?
def data_upload(request):
template = "data_upload.html"
data = ScanData.objects.all()
prompt = {
'order': 'Order of the CSV should be CVE, CVSS, Risk, Host, Hostname, Project_Assigned, Component, Owner, Environment, Location, Notes, Protocol, Port, Name, Synopsis, Description, Solution, Plugin_Output',
'scandata': data
}
if request.method == "GET":
return render(request, template, prompt)
csv_file = request.FILES['file']
if not csv_file.name.endswith('.csv'):
messages.error(request, 'THIS IS NOT A CSV FILE')
data_set = csv_file.read().decode('UTF-8')
io_string = io.StringIO(data_set)
next(io_string)
for column in csv.reader(io_string, delimiter=','):
_, created = ScanData.objects.update_or_create(
CVE=column[0],
CVSS=column[1],
Risk=column[2],
Host=column[3],
Hostname=column[4],
Project_Assigned=column[5],
Component=column[6],
Owner=column[7],
Environment =column[8],
Location=column[9],
Notes=column[10],
Protocol=column[11],
Port=column[12],
Name=column[13],
Synopsis=column[14],
Description=column[15],
Solution=column[16],
Plugin_Output=column[17],
)
context = {}
return render(request, template, context)

Answer 1

You can work with .bulk_create(…) [Django-doc] and set the ignore_conflicts=True parameter. As the documentation says:
On databases that support it (all but Oracle), setting the ignore_conflicts parameter to True tells the database to ignore failure to insert any rows that fail constraints such as duplicate unique values. Enabling this parameter disables setting the primary key on each model instance (if the database normally supports it).
You thus make a list of your model objects (but you do not save these yet), and then you use .bulk_create(list_of_objects, ignore_conflicts=True). This thus looks like:
m1 = MyModel(field1=value11, field2=value12)
m2 = MyModel(field1=value21, field2=value22)
m3 = MyModel(field1=value31, field2=value32)
MyModel.objects.bulk_create(
[m1, m2, m3],
ignore_conflicts=True
)