classic dataflow template failing - google-cloud-platform

I am uploading a classic data flow template with them setup.py and requirements.txt file. I have a dofn method which has below syntax
import apache_beam as beam
class Post(beam.DoFn):
def process(self,element):
<code>
<code>
yield beam.pvalue.TaggedOutput('success', alarm_success)
yield beam.pvalue.TaggedOutput('failure', alarm_failure)
NameError: name 'beam' is not defined [while running 'Alarm posting to ESB/ParDo(PostToESB)-ptransform-63']
Though I have this import on my main.py file, i am getting this error.

Related

Django+Cython import cython module in django app views [duplicate]

This question already has an answer here:
Python Package "No module named..."
(1 answer)
Closed 2 years ago.
a newbie to django and Cython. I am creating an app in django and need to import function in views.py from cythonized module. following is views.py inside my app.
from django.shortcuts import render
import sys
import numpy as np
import random
import math
from cython_node_val import node_val
def home(request):
return render(request,'Home.html',{"name":"user"})
def shortest_path1(request):
K=int(request.POST['number of layers'])
if ((K%2!=0) or (K < 0)):
return render(request,"shortest_path1.html",{'shortest_path1':"K must be an even integer"})
else:
......
Node_val=node_val(Hash,C,K) #node_val is from cython_node_val which is a .pyx file, Hash C and K
are defined in body after else statement.
sPath=np.zeros((K,3))
sPath[K-1,:]=Node_val[n-1,:]
for m in range(K-2,-1,-1):
sPath[m,:]=Node_val[int(sPath[m+1,1])]
return render(request,"shortest_path1.html",{'shortest_path1':sPath[:,3]})'''
the directory of my project is like following:
my app directory looks like this
cython_node_val.pyx works fine when importing into a normal .py file, but when doing the same inside views.py in my app it throws me following error
File "C:\Users\amit\projects\application_shortest_path\shortest_path\DS2P\urls.py", line 9, in <module>
from . import views
File "C:\Users\amit\projects\application_shortest_path\shortest_path\DS2P\views.py", line 6, in <module>
from cython_node_val import node_val
ModuleNotFoundError: No module named 'cython_node_val'
I believe if views.py is a python file and we can do operations, it should pull cython_node_val and associated functions. Where am i wrong?
Thanks for your time.
Use os.getcwd() to debug where you are running from your script views.py:
import os
print(os.getcwd())
ModuleNotFoundError: No module named 'cimport'
Then adjust the path to your needs
Is usually an error when you are trying to reference something that isn't in the python running path script + the path that you are giving inside your script.

Initialising flask environment variable issues

I am trying to create a an application using Flask. I have done this before successfully, however, I am not sure why this is not working this time. Everything seems to be in the right order. I have searched for answers, however, I still can't determine what's going wrong as everything seems logical to me? Yet, it is still going wrong?
ZXM934/
app/
__innit__.py
views.py
venv/
run.py
The following are each files contents:
run.py
# Importing app object which was created in __innit__.py file into app.py
from app import app
if __name__ == "__main__":
app.run()
__innit__.py
# This class will ultimately bring our entire application together.
from flask import Flask
# Creating Flask app.
app = Flask(__name__)
# Importing views file to avoid circular import.
from app import views
view.py
# This class represents the UI of our website.
# Importing app directory. As __innit__.py file is apart of this directory,
# this import treats it as a package.
from app import app
#app.route("/")
def public_home():
return "Homepage"
#app.route("/login")
def login():
return "<h1 style='color: red'>Login</h1>"
I set the environment variables as following within the console:
export FLASK_APP=run.py
export FLASK_ENV=development
I then run the following command:
flask run
The following error occurs:
flask.cli.NoAppException: While importing "run", an ImportError was raised:
Traceback (most recent call last):
File "/Users/zahidmalik-ramzan/Desktop/zxm934/venv/lib/python3.7/site-packages/flask/cli.py", line 240, in locate_app
__import__(module_name)
File "/Users/zahidmalik-ramzan/Desktop/zxm934/run.py", line 2, in <module>
from app import app
ImportError: cannot import name 'app' from 'app' (unknown location)
I don't understand what I am doing wrong?
Your problem is in the file name __innit__.py.
For python to understand that a folder is an actual package within your project structure you need a special file inside, i.e. __init.py__ ps: no double n.
A regular package is typically implemented as a directory containing an __init__.py file. When a regular package is imported, this __init__.py file is implicitly executed, and the objects it defines are bound to names in the package’s namespace.
Conclusion: change your __innit.py__ to __init.py__

Django - Error importing storages.backends

I have created a custom storage backend, the file is called storages.py and is placed in an app called core:
from django.conf import settings
from storages.backends.s3boto import S3BotoStorage
class S3StaticBucket(S3BotoStorage):
def __init__(self, *args, **kwargs):
kwargs['bucket_name'] = getattr(settings, 'static.mysite.com')
super(S3BotoStorage, self).__init__(*args, **kwargs)
In settings.py, I have the follwing:
STATICFILES_STORAGE = 'core.storages.S3StaticBucket'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
When I try to do python manage.py collectstatic it shows the following error:
django.core.exceptions.ImproperlyConfigured: Error importing storage module core.storages: "No module named backends.s3boto"
And when I run python manage.py shell and try to import the same:
>>>
>>> from django.conf import settings
>>> from storages.backends.s3boto import S3BotoStorage
>>>
Any idea what I'm doing wrong?
There is a namespace conflict; the storage absolute name clashes with a storage local name. It may be unintuitive, but you can import from module in itself:
// file my_module/clash.py
import clash
print clash.__file__
Now we run python shell in a dir containing a my_module:
$ python
>>> import my_module.clash
my_module.clash.py
In short, your module tries to import a backend from itself.
You need an absolute import - Trying to import module with the same name as a built-in module causes an import error.
I had this same issue, but for me it turns out that despite django-storages being installed, boto was not. A simple pip install boto fixed the error in my scenario.
I had another type of issue that can help others, I used to have another file named storages.py but I deleted that file days ago, and still getting the Exception... the thing is I didn't had deleted the file storages.pyc !
Typo error.
Change:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
TO:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3Boto3Storage'

Calling a model's method from outside Django

I have a Django model with some static methods. I'd like to call the methods from outside the application (cronjob).
The model I have:
class Job(models.Job):
#Irrelevant information
#staticmethod
def methodIwantToCall():
#statements
I have the following python file that I'm using for the cron job:
#!/usr/bin/python
# -*- coding: UTF-8 -*-
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
from myapp.models import Job
Job.methodIwantToCall()
At first, I was having an error about DJANGO_SETTINGS_MODULE not being set and I fixed that, however, now I have the following error: No module named myapp.utils
I feel like I'm doing something that I'm not supposed to do. So how do I call that static method the way I want it to be called?
EDIT: It looks like the paths are getting messed up when I'm importing from outside Django. For example, I have an import in my models file, when I call the cron file it fails importing with the message ImportError: No module named myapp.utils even though it's working.
The proper solution is to create custom manage.py command.
Assuming your cron job code resides in the same directory as your settings file, use the following setup code at the beginning:
from django.core.management import setup_environ
import settings
setup_environ(settings)

Tests for django app produces template not found error for "render_to_string" when executed via Fabric

When I run tests on my remote server using fabric, I get an error saying:
File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 138, in find_template
raise TemplateDoesNotExist(name)
TemplateDoesNotExist: index.html
I am trying to render the template as a string using "render_to_string()"
If I login to the server and run tests manually (python manage.py test app), it is working properly. This error occurs while running through fabric.
Here is my fabric code:
from __future__ import with_statement
from fabric.api import local
import os
from fabric.api import *
env.hosts = ['server.com']
production_project_path = '/path/to/production/app/'
def run_remote_test():
run("python %s/manage.py test app"%production_project_path)
Did I miss something?
Note: I am not using virtual environment
Then let's make this official. ;)
In this case, the problem was the fact that manage.py expects to be ran from the project directory, so rewriting the abovestanding as:
from __future__ import with_statement
from fabric.api import local
import os
from fabric.api import *
env.hosts = ['server.com']
production_project_path = '/path/to/production/app/'
def run_remote_test():
with cd(production_project_path):
run("python manage.py test app")
has fixed the issue.