df['new_Comments'] = df['Coments'].apply(lambda x: remove_punct(str(x)))
df.head(1)
KeyError Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3360 try:
-> 3361 return self._engine.get_loc(casted_key)
3362 except KeyError as err:
4 frames
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()
KeyError: 'Coments'
The above exception was the direct cause of the following exception:
KeyError Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
3361 return self._engine.get_loc(casted_key)
3362 except KeyError as err:
-> 3363 raise KeyError(key) from err
3364
3365 if is_scalar(key) and isna(key) and not self.hasnans:
KeyError: 'Coments'
Can anybody tell me what I am doing wrong in this code.
Related
I would lioke some help to unit test a function that deletes an object in a Django app
The problem
I display a list of values, it includes a bin icon to delete one value, and my view seems to work fine (at least according to the test I made).
How can I unit test it? I'm not able to find out the right way to do yet.
I searched the web and found the DoesNotExist exception, but I'm afraid I'm not able to use it, as I got a matching query does not exist error.
Could you please advise me on how to proceed?
What I tried
Here is my current whole code for the test:
class TestAdmUsers(TestCase):
def setUp(self):
self.company = create_dummy_company("Société de test")
self.usr11 = create_dummy_user(self.company, "user11")
self.usr13 = create_dummy_user(self.company, "user13")
self.usr14 = create_dummy_user(self.company, "user14")
self.client.force_login(self.user_staff.user)
def test_delete_user(self):
test_usercomp_id = self.usr13.id
url = reverse("polls:adm_delete_user", args=[self.company.comp_slug, self.usr13.id])
response = self.client.get(url, follow=True)
self.assertRaises(UserComp.DoesNotExist, UserComp.objects.get(id=test_usercomp_id))
The test log is the following:
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
E
======================================================================
ERROR: test_delete_user (polls.tests_admin.TestAdmUsers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\Mes documents\Informatique\Developpement\Votes AG\projet_votes\polls\tests_admin.py", line 136, in test_delete_user
self.assertRaises(UserComp.DoesNotExist, UserComp.objects.get(id=test_usercomp_id))
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\query.py", line 408, in get
self.model._meta.object_name
polls.models.UserComp.DoesNotExist: UserComp matching query does not exist.
----------------------------------------------------------------------
Ran 1 test in 1.763s
FAILED (errors=1)
Destroying test database for alias 'default'...
I made other tests like this one:
def test_delete_user(self):
url = reverse("polls:adm_delete_user", args=[self.company.comp_slug, self.usr13.id])
response = self.client.get(url)
self.assertContains(response, self.usr12.user.username)
self.assertNotContains(response, self.usr13.user.username)
that leads to this:
======================================================================
FAIL: test_delete_user (polls.tests_admin.TestAdmUsers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\Mes documents\Informatique\Developpement\Votes AG\projet_votes\polls\tests_admin.py", line 136, in test_delete_user
self.assertNotContains(response, self.usr13.user.username)
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\test\testcases.py", line 465, in assertNotContains
self.assertEqual(real_count, 0, msg_prefix + "Response should not contain %s" % text_repr)
AssertionError: 1 != 0 : Response should not contain 'user13'
----------------------------------------------------------------------
Or this one:
def test_delete_user(self):
test_usercomp_id = self.usr13.id
url = reverse("polls:adm_delete_user", args=[self.company.comp_slug, self.usr13.id])
response = self.client.get(url, follow=True)
self.company.refresh_from_db()
self.usr13.refresh_from_db()
users = UserComp.get_users_in_comp(self.company.comp_slug)
self.assertContains(response, self.usr12.user.username)
self.assertNotContains(users, self.usr13)
with the following result:
======================================================================
ERROR: test_delete_user (polls.tests_admin.TestAdmUsers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\Mes documents\Informatique\Developpement\Votes AG\projet_votes\polls\tests_admin.py", line 137, in test_delete_user
self.usr13.refresh_from_db()
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\base.py", line 628, in refresh_from_db
db_instance = db_instance_qs.get()
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\query.py", line 408, in get
self.model._meta.object_name
polls.models.UserComp.DoesNotExist: UserComp matching query does not exist.
----------------------------------------------------------------------
And this latest one that uses a class method:
def test_delete_user(self):
current_username = self.usr13.user.username
url = reverse("polls:adm_delete_user", args=[self.company.comp_slug, self.usr13.id])
response = self.client.get(url, follow=True)
self.company.refresh_from_db()
self.usr13.refresh_from_db()
self.assertContains(response, self.usr12.user.username)
self.assertNotContains(UserComp.get_users_in_comp(self.company.comp_slug), self.usr13)
but the result is still an error:
`======================================================================
ERROR: test_delete_user (polls.tests_admin.TestAdmUsers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\Mes documents\Informatique\Developpement\Votes AG\projet_votes\polls\tests_admin.py", line 137, in test_delete_user
self.usr13.refresh_from_db()
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\base.py", line 628, in refresh_from_db
db_instance = db_instance_qs.get()
File "C:\Users\Christophe\.virtualenvs\projet_votes-onIieQ0I\lib\site-packages\django\db\models\query.py", line 408, in get
self.model._meta.object_name
polls.models.UserComp.DoesNotExist: UserComp matching query does not exist.
----------------------------------------------------------------------
Related application code
Here is the code for the view:
def adm_delete_user(request, comp_slug, usr_id):
del_usr = User.objects.get(pk=usr_id)
msg = "Utilisateur {0} {1} supprimé.".format(del_usr.last_name, del_usr.first_name)
User.objects.get(pk=usr_id).delete()
messages.success(request, msg)
return redirect("polls:adm_users", comp_slug=comp_slug)
And the model:
class UserComp(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE, verbose_name="Utilisateur")
company = models.ForeignKey(Company, on_delete=models.CASCADE, verbose_name="Société")
phone_regex = RegexValidator(regex=r'^0[0-9]([ .-]?[0-9]{2}){4}$', message=("Format de numéro de téléphone invalide"))
phone_num = models.CharField("numéro de téléphone", validators=[phone_regex], max_length=14, null=True, blank=True)
is_admin = models.BooleanField("administrateur", default=False)
def __str__(self):
return '%s %s' % (self.user.last_name, self.user.first_name)
class Meta:
verbose_name = "Liens Utilisateurs / Sociétés"
verbose_name_plural = "Liens Utilisateurs / Sociétés"
#classmethod
def create_usercomp(cls, user, company, phone_num='', is_admin=False):
""" Create a new UserComp """
usr_comp = UserComp(user=user, company=company, phone_num=phone_num, is_admin=is_admin)
usr_comp.save()
return usr_comp
#classmethod
def get_users_in_comp(cls, comp_slug):
user_list = cls.objects.filter(company__comp_slug=comp_slug)
return user_list
Complementary search results
Finally, I found some posts here that oriented my tests, but I wasn't able to find the solution:
The most important to me leads me to assertRaise() test
Unfortunately, the one that asked how to use this exception has been closed, and referred to the previous one, but actually there is no information on how to proceed
This other one asking about the exception, and this one related to the error message did not give me any additional information.
assertRaises takes a callable as its (optional) second argument.
Since .get(...) is not a callable, you should use the context manager form instead:
with self.assertRaises(UserComp.DoesNotExist):
UserComp.objects.get(id=test_usercomp_id)
I'm creating delete request to webserver and trying to catch the exceptions like below:
try:
response = req.delete(path, auth=HTTPBasicAuth(config.user(), config.password()), params=params, headers=headers, verify=True)
except requests.HTTPError as he:
raise SystemExit(he)
except requests.ConnectionError as ce:
raise SystemExit(ce)
except requests.URLRequired as ue:
raise SystemExit(ue)
except requests.Timeout as te:
raise SystemExit(te)
except requests.RequestException as err:
print('Undefined error: ', err)
print(response.status_code)
But delete is not processed and response.status_code prints 400, but error handling does not work. Any ideas why the error handling is not working there?
If you want to catch http errors (e.g. 401 Unauthorized) to raise exceptions, you need to call Response.raise_for_status. That will raise an HTTPError, if the response was an http error.
try:
response = req.delete(path, auth=HTTPBasicAuth(config.user(), config.password()), params=params, headers=headers, verify=True)
except requests.HTTPError as he:
raise SystemExit(he)
I've recently updated my Django to version 2.2.4 and python to 3.7,
and I encounter this error randomly:
Traceback (most recent call last):
File "/foo/venv/lib/python3.7/site-packages/django/db/models/query.py", line 73, in __iter__
obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
File "/foo/venv/lib/python3.7/site-packages/django/db/models/base.py", line 511, in from_db
for f in cls._meta.concrete_fields
File "/foo/venv/lib/python3.7/site-packages/django/db/models/base.py", line 511, in <listcomp>
for f in cls._meta.concrete_fields
StopIteration
I tried to debug the code to find out what's happening, and it seems the from_db function in django.db.models.base.py is causing this error:
# /foo/venv/lib/python3.7/site-packages/django/db/models/base.py
...
#classmethod
def from_db(cls, db, field_names, values):
if len(values) != len(cls._meta.concrete_fields):
values_iter = iter(values)
values = [
next(values_iter) if f.attname in field_names else DEFERRED
for f in cls._meta.concrete_fields
]
new = cls(*values)
new._state.adding = False
new._state.db = db
return new
...
next(values_iter) is raising this error and it seems that Django devs should surround that with try except block to make this work in python 3.7,
but my question is, is there a way to overcome this issue as a temp fix or not?
thanks.
Update#1:
I've found out when exactly this error happens, When I call .values_list('myfield', flat=True) on a QuerySet, the query changes to SELECT myfield from ... and this breaks everything.
from multiprocessing import Pool
def f(arg):
if arg == 1:
raise Exception("exception")
return "hello %s" % arg
p = Pool(4)
res = p.map_async(f,(1,2,3,4))
p.close()
p.join()
res.get()
Consider this contrived example where I am creating a process pool of 4 workers and assigning work in f(). My question was:
How can I retrieve the successful work that was done for arguments 2,3,4 (and at the same time do exception handling for argument 1) ?
As is the code just gives me:
Traceback (most recent call last):
File "process_test.py", line 13, in <module>
res.get()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
Exception: exception
You can also just do the error handling in the work function
def do_work(x):
try:
return (None, something_with(x))
except Exception as e:
return (e, None)
output = Pool(n).map(do_work, input)
for exc, result in output:
if exc:
handle_exc(exc)
else:
handle_result(result)
You can use the imap function.
iterator = p.imap(f, (1,2,3,4,5))
while True:
try:
print next(iterator)
except StopIteration:
break
except Exception as error:
print error
I have a cronjob which processes emails and stores them in the system. It'd been working well for over a year but now all of a sudden it has started giving these weird transaction errors randomly every now and then.
2010-09-02-01:45:11 ERROR Exception occured during mail processing
Traceback (most recent call last):
File "/var/www/example.com/project/scripts/cronjobs.py", line 453, in process_msg
source, email, 'i' if rejection_reason else 'v', rejection_reason)
File "/usr/lib/python2.5/site-packages/django/db/transaction.py", line 267, in _commit_manually
leave_transaction_management()
File "/usr/lib/python2.5/site-packages/django/db/transaction.py", line 77, in leave_transaction_management
raise TransactionManagementError("Transaction managed block ended with pending COMMIT/ROLLBACK")
TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK
It happens without any reason and there isn't any other context information available to me. Here is the structure of my process_msg function:
#commit_manually
def save_email( u,l, d, t, s, e, status, reason):
try:
# ... bla bla bla ...
commit()
exception, e:
rollback()
raise e
def process_msg(m):
try:
#....
save_email(u, l, d, t
source, email, 'i' if rejection_reason else 'v', rejection_reason)
#....
except Exception, e:
logger.error('Error while processing email', exc_info=True )
return None
else:
return True
How how I probe this issue?
It appears as if the error occurred before entering save_email(). With no commit() or rollback() directive in process_msg() a TransactionError is raised.
You could try to circle the error with a debugger:
def process_msg(m):
try:
import pdb # import python debugger
pdb.set_trace() # begin debugging
#....
save_email(u, l, d, t,
[…]
You can find information on how to use the debugger on the Python website.