I am trying to create documentation using Sphinx in PyCharm and it does not work. I already tried using the terminal command make html and it seems to work just fine. When I try to run Sphinx task in PyCharm it raises this error:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/rest_runners/sphinx_runner.py", line 5, in <module>
from sphinx import cmdline
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/sphinx/cmdline.py", line 24, in <module>
from sphinx.application import Sphinx
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/sphinx/application.py", line 33, in <module>
from sphinx.environment import BuildEnvironment
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 45, in <module>
from sphinx.util.websupport import is_commentable
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/sphinx/util/websupport.py", line 11, in <module>
from sphinxcontrib.websupport.utils import is_commentable # NOQA
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/sphinxcontrib/websupport/__init__.py", line 13, in <module>
__import__('pkg_resources').declare_namespace(__name__)
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 3017, in <module>
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 3003, in _call_aside
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 3045, in _initialize_master_working_set
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 2577, in activate
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 2151, in declare_namespace
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 2091, in _handle_ns
File "/Users/wtekimam/PycharmProjects/doctor/myvenv/lib/python3.6/site-packages/setuptools-28.8.0-py3.6.egg/pkg_resources/__init__.py", line 2120, in _rebuild_mod_path
AttributeError: '_NamespacePath' object has no attribute 'sort'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/rest_runners/sphinx_runner.py", line 7, in <module>
raise NameError("Cannot find sphinx in selected interpreter.")
NameError: Cannot find sphinx in selected interpreter.
note: I use virtual environment and I already configure PyCharm to use the same virtual environment that I use to run make html in terminal
There are two requirements:
Install Sphinx into your project's interpreter.
Create a run configuration for a Sphinx task with the appropriate settings as specified in PyCharm's documentation.
Here's a typical configuration with screenshot.
Command: html
Input: path/to/reST/source/files
Output: path/to/reST/build/files
Python Interpreter: Project Default (or whatever you select)
Working directory: path/to/conf.py
The solution is to update all packages that are used in the interpreter.
In particular, update the setuptools package.
This worked for me, but I do not know why, I am sorry. None of the answers above helped.
Related
i create a project folder, than create a venv and do $ django-admin startproject trydjango.
now when i try to run runserver it gives me an error that it couldnt find manage.py, so i have to move it to the original project folder (which now has venv, trydjango, and manage.py). this is a problem i have seen online.
however, when i try to run runserver now it says:
"ModuleNotFoundError: No module named 'trydjango.settings'"
(full traceback below)
the weird thing is, i have opened a project file just like this before, but now its not working. i have done it in the exact same way, the exact way that tutorials show. the only thing that changed is the django version - the tutorial said to use 2.0.7 in order to be able to follow the tutorial.
full traceback:
Traceback (most recent call last):
File "E:\Python\projects\try_django_project\manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "E:\Python\projects\try_django_project\venv\lib\site-packages\django\core\management\__init__.py", line 371, in execute_from_command_line
utility.execute()
File "E:\Python\projects\try_django_project\venv\lib\site-packages\django\core\management\__init__.py", line 317, in execute
settings.INSTALLED_APPS
File "E:\Python\projects\try_django_project\venv\lib\site-packages\django\conf\__init__.py", line 56, in __getattr__
self._setup(name)
File "E:\Python\projects\try_django_project\venv\lib\site-packages\django\conf\__init__.py", line 43, in _setup
self._wrapped = Settings(settings_module)
File "E:\Python\projects\try_django_project\venv\lib\site-packages\django\conf\__init__.py", line 106, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\importlib\__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'trydjango.settings'
I will suggest you doing such way:
Create a virtualenvironment named env or venv or as your wish and activate it.
Then install django via pip install django
now run this command django-admin startproject trydjango You can
see you project structure such way:
-env
-trydjango
--trydjango
--manage.py
Now enter the project directory where manage.py stays and run python manage.py runserver
Hope this will work for you.
So I want to build a Docker image with PyTorch Lightning that can be used with AWS lambda. However, when the function is invoked it raises an OS Error, that claims it uses a Read-only file system and wandb.py wants to write something.
I tried these things:
Overwrite the wandb.py file of pytroch lightning, that it does not init wandb --> Raises error
Execute a python script in Dockerfile, that the files are created on docker build and exist, when invoking the lambda function --> Same OS error
Does someone know a way to skip the wandb.py?
This is the error message:
START RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772 Version: $LATEST
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
[ERROR] OSError: [Errno 30] Read-only file system: '/home/sbx_user1051'
Traceback (most recent call last):
File "/var/lang/lib/python3.8/imp.py", line 234, in load_module
return load_source(name, filename, file)
File "/var/lang/lib/python3.8/imp.py", line 171, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 702, in _load
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/var/task/inference.py", line 5, in <module>
import pytorch_lightning as pl
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/__init__.py", line 63, in <module>
from pytorch_lightning.callbacks import Callback
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/callbacks/__init__.py", line 25, in <module>
from pytorch_lightning.callbacks.swa import StochasticWeightAveraging
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/callbacks/swa.py", line 26, in <module>
from pytorch_lightning.trainer.optimizers import _get_default_scheduler_config
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/trainer/__init__.py", line 18, in <module>
from pytorch_lightning.trainer.trainer import Trainer
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 30, in <module>
from pytorch_lightning.loggers import LightningLoggerBase
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/loggers/__init__.py", line 31, in <module>
from pytorch_lightning.loggers.wandb import _WANDB_AVAILABLE, WandbLogger # noqa: F401
File "/var/lang/lib/python3.8/site-packages/pytorch_lightning/loggers/wandb.py", line 34, in <module>
import wandb
File "/var/lang/lib/python3.8/site-packages/wandb/__init__.py", line 131, in <module>
api = InternalApi()
File "/var/lang/lib/python3.8/site-packages/wandb/apis/internal.py", line 17, in __init__
self.api = InternalApi(*args, **kwargs)
File "/var/lang/lib/python3.8/site-packages/wandb/sdk/internal/internal_api.py", line 73, in __init__
self._settings = Settings(
File "/var/lang/lib/python3.8/site-packages/wandb/old/settings.py", line 25, in __init__
self._global_settings.read([Settings._global_path()])
File "/var/lang/lib/python3.8/site-packages/wandb/old/settings.py", line 105, in _global_path
util.mkdir_exists_ok(config_dir)
File "/var/lang/lib/python3.8/site-packages/wandb/util.py", line 687, in mkdir_exists_ok
os.makedirs(path)
File "/var/lang/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/var/lang/lib/python3.8/os.py", line 213, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/var/lang/lib/python3.8/os.py", line 223, in makedirs
mkdir(name, mode)
END RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772
REPORT RequestId: ddae284d-4f32-4dc6-8160-d1fa62ba9772 Duration: 27000.33 ms Billed Duration: 27001 ms Memory Size: 10240 MB Max Memory Used: 241 MB
Unknown application error occurred
You need to make sure you have write access somewhere.
Then you can use wandb environment variables to modify the default location where files are saved locally, in particular look at WANDB_DIR, WANDB_CONFIG_DIR and WANDB_CACHE_DIR.
I'm trying to use HDFS sensor in airflow. But am getting the below error
[2020-06-30 07:22:57,636] {dagbag.py:246} ERROR - Failed to import: /airflow/dags/test_s3_file.py
Traceback (most recent call last):
File "/airflow/airflow-staging/lib/python3.6/site-packages/airflow/models/dagbag.py", line 243, in process_file
m = imp.load_source(mod_name, filepath)
File "/airflow/airflow-staging/lib64/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/airflow/dags/test_s3_file.py", line 3, in <module>
from S3HDFSObjectSensor import S3HDFSObjectSensor
File "/airflow/plugins/S3HDFSObjectSensor.py", line 6, in <module>
from airflow.hooks.hdfs_hook import HDFSHook
File "/airflow/airflow-staging/lib/python3.6/site-packages/airflow/hooks/hdfs_hook.py", line 24, in <module>
from snakebite.client import Client, HAClient, Namenode, AutoConfigClient
File "/airflow/airflow-staging/lib/python3.6/site-packages/snakebite/client.py", line 1473
baseTime = min(time * (1L << retries), cap);
^
SyntaxError: invalid syntax
From google, found that we will have this issue if we are using python3. Could you please let me know if there is any work around for this other than using python2.
from airflow.plugins.S3HDFSObjectSensor import S3HDFSObjectSensor
t1 = S3HDFSObjectSensor(
task_id='t1',
poke_interval=sensor_poke_interval,
timeout=sensor_timeout,
object_path=check_file_in_hdfs,
trigger_rule='one_failed',
dag=dag)
Below is my AIRFLOW_HOME directory structure
airflow
+-- dags
+-- plugins
+-- S3HDFSObjectSensor.py
Thanks
It worked after installing 'pip3 install snakebite-py3'
On my first Django project, I got started with a sqlite database, and now I'm switching to Postgres. The first part of the code that I wrote and tested was an import management command, to read records in from a file and create database records for them, and that's been working great. So to start making the change, I've updated the default database info in settings.py, and I ran migrations for it like so:
python3 manage.py migrate --database=default
I can see the correct table structure when I view the new database in pgAdmin 4 (just with no rows populated yet)- and the field max values appear to all match what's in the model- hooray! I also added some code to get the ENV_ROLE and database password from environment variables, as recommended here: https://ultimatedjango.com/learn-django/lessons/handling-sensitive-keys/
Next I run my import management command, and the problem: I can run this successfully from the terminal, but when I try running it from within PyCharm I'm getting errors I never saw while running it using the sqlite database:
No manage.py file specified in Settings->Django Support
/usr/local/bin/python3.8 /Library/Frameworks/Python.framework/Versions/3.8/bin/django-admin.py importgedcom Test_tree.ged
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/management/__init__.py", line 224, in fetch_command
app_name = commands[subcommand]
KeyError: 'importgedcom'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/django-admin.py", line 5, in <module>
management.execute_from_command_line()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/core/management/__init__.py", line 231, in fetch_command
settings.INSTALLED_APPS
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/conf/__init__.py", line 76, in __getattr__
self._setup(name)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
self._wrapped = Settings(settings_module)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/conf/__init__.py", line 142, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mysite.settings'
Process finished with exit code 1
I see a few interesting bits in there, likely multiple/different issues in my setup running from PyCharm vs from the terminal:
No manage.py file specified in Settings->Django Support. My configuration for the management command has always included this: DJANGO_SETTINGS_MODULE=mysite.settings; I did make one update to the configuration, just to add ENV_ROLE=development
ModuleNotFoundError: No module named 'mysite.settings'. I didn't rename the project or app, only updated settings.py (default database and environment logic)
UPDATE: I can run this to completion while stepping through in debug mode, I only see the error when I hit PyCharm's green play button to run it.
These errors happen before the code even does anything (I don't make it to print statements in the add_arguments or handle methods), so there's some problem with the setup. Any advice on what to check/do next, or is there more info I can give that would help figure out next steps?
The target is to create a pip package of a Django project.
The goal is to make the web application quickly installable anywhere just by doing pip install.
The use cases are 2 that I can think of:
Manually start the application using a console_scripts like client-web-up that should be the equivalent to type manage.py runserver.
This is what I'm focused on right now.
create some sort of script to generate configuration files for apache, nginx etc... for later.
Steps taken
Created a main folder client_web for my pip package.
Moved all my project into client_web folder.
Created a setup.py as shown below.
Edited manage.py file so its content can be executed by an function named entry().
Publish the pip package which downloads and installs correctly.
from setuptools import setup
setup(
name="client-web",
...,
version="2018.08.03.3",
packages=[
"client_web",
"client_web.controller",
"client_web.measurement",
"client_web.webapp",
"client_web.controller.migrations",
"client_web.measurement.migrations",
],
include_package_data=True,
install_requires=[
...
],
entry_points={
"console_scripts": [
"client-web-up=client_web.manage:entry"
]
}
)
The problem
If i type the following in the terminal:
$ client-web-up
Traceback (most recent call last):
File "/usr/local/bin/client-web-up", line 26, in <module>
sys.exit(entry())
File "/usr/local/lib/python3.7/site-packages/client_web/manage.py", line 33, in entry
main();
File "/usr/local/lib/python3.7/site-packages/client_web/manage.py", line 30, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 377, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 352, in execute
django.setup()
File "/usr/local/lib/python3.7/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 89, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.7/site-packages/django/apps/config.py", line 116, in create
mod = import_module(mod_path)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'webapp'
But if I do:
$ pip3 show client-web
Name: client-web
Version: 2018.8.3.3
Location: /usr/local/lib/python3.7/site-packages
Requires: ...
Required-by: ...
$ python3 /usr/local/lib/python3.7/site-packages/client_web/manage.py
Available subcommands:
[auth]
changepassword
createsuperuser
[contenttypes]
remove_stale_contenttypes
[django]
...
Could someone help me find a way to make the console_script work?
solved by adding a sys.path like so:
sys.path.append(os.path.dirname(os.path.realpath(__file__)))