I'm trying to perform testing with coverage on a django application within pycharm itself. I'm using django-nose / nose as the test runner and I have these settings in test_settings.py
# nose settings
TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'
NOSE_ARGS = [
'--with-coverage',
'--cover-package=myapp',
]
This all works very nicely if I go to the command line (or the builtin pycharm terminal) activate my virtualenv and run... django-admin.py test --settings=myapp.tests.test_settings... I get the output I expect.
However when I try to use my test run config inside Pycharm, I no longer get the coverage output, all I get is:
Testing started at 16:37 ...
There is no such manage file manage
nosetests --with-coverage --cover-package=dynamicbanners --verbosity=1
Creating test database for alias 'default'...
..
Process finished with exit code 0
(I don't have a manage.py as it is a standalone app, hence the manage file message I assume).
The tests still "pass" but there is no output and no .coverage file created. I am running on windows temporarily, is it possible that windows is denying access to writing the .coverage file and thus nothing is being displayed? Or is Pycharm eating the output but failing to display it?
I would use the builtin pycharm "Run with coverage" option, but it seems to have a problem with
NOSE_ARGS = [
'--with-coverage',
'--cover-package=myapp',
]
because when I run it with those options it spits out an error which I believe is due to two instances of coverage being initialised
Testing started at 16:41 ...
There is no such manage file manage
nosetests --with-coverage --cover-package=myapp --verbosity=1
Creating test database for alias 'default'...
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm 3.0.1\helpers\run_coverage.py", line 34, in <module>
main()
File "C:\Users\ptinkler\venvs\myapp_venv\lib\site-packages\coverage\cmdline.py", line 720, in main
status = CoverageScript().command_line(argv)
File "C:\Users\ptinkler\venvs\myapp_venv\lib\site-packages\coverage\cmdline.py", line 438, in command_line
self.do_execute(options, args)
File "C:\Users\ptinkler\venvs\myapp_venv\lib\site-packages\coverage\cmdline.py", line 580, in do_execute
self.coverage.stop()
File "C:\Users\ptinkler\venvs\myapp_venv\lib\site-packages\coverage\control.py", line 410, in stop
self.collector.stop()
File "C:\Users\ptinkler\venvs\myapp_venv\lib\site-packages\coverage\collector.py", line 294, in stop
assert self._collectors[-1] is self
AssertionError
Process finished with exit code 1
This is despite me having unchecked "use bundled coverage.py" in settings -> coverage
Would appreciate any assistance, I'd rather not have to create a local settings file to get rid of those NOSE_ARGS just to get coverage displaying within pycharm. If I can only end up with standard coverage output in the pycharm testrunner console that's not the end of the world. Otherwise I guess I'll have to stick to commandline.
Related
I'm trying to setup running google tests on a C++ repository using Github Actions that are running on Windows Latest.
The building process completes, but when it comes to running the tests - it is stuck and doesn't execute the executable, that is generated from the Visual Studio Project via msbuild. Generally the tests should execute in a close to none amount, since there are not a lot of tests. I have also waited and 30 minutes for the action to execute, but no luck.
What I have tried is:
Running the executable straight from the actions.yml file like:
Build/Tests.exe
Build/Tests
./Build/Tests
./Build/Tests.exe
cd Build && Tests.exe
etc.
Some of these commands run and execute locally, but are stuck on execute on the Github Actions Runner.
Now I have switched to a python script running these commands for maybe getting more insight on the process and seems when calling subprocess.Popen, the action is still stuck, but on wait.
Workflow callstack after canceling the workflow:
Run python Tools/test.py
python Tools/test.py
shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
env:
CMAKE_VERSION: 3.21.1
NINJA_VERSION: 1.10.2
pythonLocation: C:\hostedtoolcache\windows\Python\3.10.2\x64
^CTraceback (most recent call last):
File "D:\a\Lib\Lib\Tools\test.py", line 48, in <module>
sys.exit(main())
File "D:\a\Lib\Lib\Tools\test.py", line 44, in main
return run(cmd_arguments)
File "D:\a\Lib\Lib\Tools\test.py", line 39, in run
return process.communicate()[0]
File "C:\hostedtoolcache\windows\Python\3.10.2\x64\lib\subprocess.py", line 1141, in communicate
self.wait()
File "C:\hostedtoolcache\windows\Python\3.10.2\x64\lib\subprocess.py", line 1204, in wait
return self._wait(timeout=timeout)
File "C:\hostedtoolcache\windows\Python\3.10.2\x64\lib\subprocess.py", line 1485, in _wait
result = _winapi.WaitForSingleObject(self._handle,
KeyboardInterrupt
I have a following question:
I have a test function written in Pytest for Django project:
#pytest.mark.django_db
def test_class():
path = Path(r'ascertain\tests\csv')
handler = DatabaseCSVUpload(path, delimiter=',')
handler()
ascertain\tests\test_upload_csv.py:19 (TestUploadCSVtoDatabaseNegative.test_class)
Traceback (most recent call last):
File "C:\Users\hardcase1\PycharmProjects\telephone_numbers\ascertain\tests\test_upload_csv.py", line 27, in test_class
handler()
File "C:\Users\hardcase1\PycharmProjects\telephone_numbers\ascertain\handle_csv.py", line 129, in __call__
for file in self.get_csv_files():
File "C:\Users\hardcase1\PycharmProjects\telephone_numbers\ascertain\handle_csv.py", line 48, in get_csv_files
raise EmptyFolder()
telephone_numbers.custom_exceptions.EmptyFolder:
It basically expects *.csv files be inside directory ‘path’ and handle them to upload to DB.
Problem is that Pytest can’t see any files in this directory as well as in other directories. I have tried multiple times with different folders.
In fact file is there:
list(Path(r'ascertain\tests\csv').glob('*.csv'))
[WindowsPath('ascertain/tests/csv/test.csv')]
Same test function written in unitests works correctly:
from rest_framework.test import APITestCase
class TestCase(APITestCase):
def test_open(self):
path = Path(r'ascertain\tests\csv')
handler = DatabaseCSVUpload(path, delimiter=',')
handler()
System check identified no issues (0 silenced).
Destroying test database for alias 'default'...
Process finished with exit code 0
Question is -what I need to do to make Pytest see this file?
Thank you!
I'm not sure this problem relates to Pytest vs Unittest, rather I think it has to do with where you executing these files from. I think you can resolve this by using an absolute path instead of the relative path you included in the example.
An absolute path will start at your systems root directory instead of starting at the directory you are executing the file from. I'm not a Windows user, but from what I can see online the path to this data should look something like this C:/users/path/to/data.
I have done quite some spark job in Java/Scala, where I can run some test spark job directly from main() program, as long as I add the required spark jar in the maven pom.xml.
Now I am starting to work with pyspark. I am wondering if I could do something similar? For example, I am using pycharm to run a the wordCount job:
If I just run the main() program, I got the following error:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/profiler/run_profiler.py", line 145, in <module>
profiler.run(file)
File "/Applications/PyCharm.app/Contents/helpers/profiler/run_profiler.py", line 84, in run
pydev_imports.execfile(file, globals, globals) # execute the script
File "/Users/edamame/PycharmProjects/myWordCount/myWordCount.py", line 6, in <module>
from pyspark import SparkContext
ImportError: No module named pyspark
Process finished with exit code 1
I am wondering how do I import pyspark here? so I could run some test job from the main() program like I did in Java/Scala.
I also tried to edit the interpreter path:
and my screenshot from Run -> Edit Configuration:
Last is my project structure screen shot:
Did I miss anything here? Thanks!
I finally got it work following the steps in this post. It is really helpful!
https://medium.com/data-science-cafe/pycharm-and-apache-spark-on-mac-os-x-990af6dc6f38#.jk5hl4kz0
I added the py4j-x.x.x-src.zip and pyspark.zip under $SPARK_HOME/python/lib to the project structure (preferences > Project> Project Structure and then do "+ Add Content Root") and it worked fine.
PS: Pycharm already had $PYTHONPATH and $SPARK_HOME read from the os env, which was set in .bashrc/.bash_profile
I am new to twisted and I have a twisted unit test in python, and I want to debug in pycharm with trial.
I can run the tests in command line fine (for e.g. like :~ nathan$ trial smoke_tests ) but would like to step through the test in an IDE
in another question
How debuging twisted application in PyCharm
It has been suggested that "configure the "Script" setting to point to that twistd" . so for 'trial' I tried pointing to /Library/Python/2.7/site-packages/twisted/trial/runner.py , but that fails.
Script
Assuming your working directory is /home/myself/mypythonproject/myworkingdirectory ,
Create a python file in your working directory with name trial_try.py. This is a copy of /usr/local/bin/trial. So use a copy of the version you have.
import os, sys
try:
import _preamble
except ImportError:
try:
sys.exc_clear()
except AttributeError:
# exc_clear() (and the requirement for it) has been removed from Py3
pass
# begin chdir armor
sys.path[:] = map(os.path.abspath, sys.path)
# end chdir armor
sys.path.insert(0, os.path.abspath(os.getcwd()))
from twisted.scripts.trial import run
run()
Configuration
Create a new Run Configuration in Pycharm.
for Script Enter
/home/myself/mypythonproject/myworkingdirectory/trial_try.py
for Parameters Enter you Parameters that you would use when running trial on the command line
for e.g. test_smoke
for Working directory Enter
/home/myself/mypythonproject/myworkingdirectory
You should be all Set !
i am using pycharm with django. When i do the runserver command, my project starts up and everything is fine.
if i use the pycharm run command - that green arrow at the top - then i get problems.
The problems are:
runnerw.exe C:\development\python\python.exe manage.py runserver 127.0.0.1:8000
Traceback (most recent call last):
File "manage.py", line 11, in
import settings
File "C:\development\PycharmProjects\dumpstown\settings.py", line 185, in
add_to_builtins('gravatar.templatetags.gravatar')
File "C:\development\python\lib\site-packages\django\template\base.py", line 1017, in add_to_builtins
builtins.append(import_library(module))
File "C:\development\python\lib\site-packages\django\template\base.py", line 963, in import_library
raise InvalidTemplateLibrary("ImportError raised loading %s: %s" % (taglib_module, e))
django.template.base.InvalidTemplateLibrary: ImportError raised loading
gravatar.templatetags.gravatar: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
Process finished with exit code 1
And stem from my use of add_to_builtins here: (this is in the settings.py file)
#gravatar stuff here.
add_to_builtins('gravatar.templatetags.gravatar')
I know this is the problem, because if i remove this line in the settings.py file? everything works fine.
Is there a way to remedy this problem for pycharm?
You need to set your Django settings module in Settings | Django Support | Settings
http://www.jetbrains.com/pycharm/webhelp/django-support.html
UPDATED
The problem with django-gravatar is that it's templatetags import django.contrib.auth.models.User which relies on settings module while setting module is loaded in Django by DJANGO_SETTINGS_MODULE environment variable, which is set internally by execute_manager function call in manage.py, which is executed AFTER import of settings. So when you use undocumented add_to_builtins feature just in settings.py at that point you have no DJANGO_SETTINGS_MODULE env variable set. So that is not a problem of PyCharm, but a problem of unset environment variable and usage of undocumented Django feature add_to_builtin.
When I ran the same project from Unix console I get the same error.
Probably you have DJANGO_SETTINGS_MODULE set in your environment if it works from console.
So to make it work in PyCharm you need to set up the variable in Django run configuration.
You can read here about that (see Environment variable section).
As i answered here: https://stackoverflow.com/a/11299516/1061426
pycharm is broken and doesn't work with add_to_builtins. The two obvious solutions are:
don't use pycharm, use some free django plugins for eclipse, or old school text editing?
use pycharm, just don't use add_to_builtins. This is the route i've gone down - it is annoying fixing all the template's to import a module, but it was a lot simpler in my case than the hassle of porting across to a new IDE.