Custom runtime for non-flexible environment app? - python-2.7

I don't think that my gae python app has a flexible environment because I created it many years ago. Now I want to try and create a module that has another runtime than python and keep the python app running python alongside a new runtime, custom or just another. Maybe mix PHP and python or similar. I don't need it but I want to learn and explore the possibilities. I'm also interested in learning Erlang and deploy Erlang code with appengine. I see there is questions about it already
erlang on google app engine?
And issue 125 in the tracker.
But how should we actually do it? If we make our own runtime provided that is allowed.
My app.yaml looks like
application: montaoproject
version: newsearch
runtime: python27
api_version: 1
threadsafe: true
module: default
instance_class: F1
automatic_scaling:
min_idle_instances: 5
max_idle_instances: automatic
min_pending_latency: automatic
max_pending_latency: 30ms
max_concurrent_requests: 50
default_expiration: "14d 5h"
env_variables:
GAE_USE_MONTAO : 'anyvalue'
KOOL_VERSION : '17a'
includes:
- br.yaml # Brazil
- in.yaml # India
- us.yaml # USA
- pk.yaml
- search.yaml # search pages
- admin.yaml # admin pages
- providers.yaml # auth providers
- statics.yaml # static content
handlers:
- url: /(business|ai|newindia|insert-ad.html)
script: montao.app
- url: /blobview.*
script: kool_update.app
login: admin
- url: /market.*
script: main.app
- url: /
script: montao.app
- url: /(index.html|sign-up.html|login.html)
script: montao.app
- url: /(login.*|login|googlogin|googlogout|create/)
script: login.app
- url: /(customer_service.htm|contactfileupload|support.html|faq.html)
script: customer_service.app
- url: /stats.*
script: google.appengine.ext.appstats.ui.app
# All other URLs use main.app
- url: /.*
script: main.app
inbound_services:
- mail
builtins:
- remote_api: on
- deferred: on
#- appstats: on
error_handlers:
- file: default_error.html
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
- name: setuptools
version: latest
- name: markupsafe
version: latest
- name: django
version: latest
- name: PIL
version: latest
- name: webob
version: latest
- name: lxml
version: latest
- name: ssl
version: latest

Yes, your app.yaml file is a standard env one (it doesn't have vm:true or env:flex in it).
Yes, it's possible to mix and match services/modules in different languages and with different environments inside the same app. You can even switch the language and environment of the same module in a different version of that module. That's because modules offer complete code isolation, see Comparison of service isolation and project isolation. Related post: Upload a Java and node.js project to Google AppEngine at once
I always try to structure a multi-service/module app with each service in its own subdir, as described in Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
So first I'd create a default subdirectory of your app dir and move all your existing default module specific files into it, with the exception of the app-level config, which I'd keep at the top level and symlink inside the default dir as described in that post. Then I'd verify that the default module still works as expected.
Then I'd create a new subdirectory for every new module I need to add and add the code for it as needed.
Side note: sharing code via symlinks as described in the post mentioned above works for standard env modules, but it probably doesn't work with flexible ones, see Sharing code between modules in a GAE project

Related

Google cloud error : Cannot find module '/workspace/server.js'

I'm facing a very common issue on Google cloud after deployment. I'm getting Cannot find module '/workspace/server.js'. In local it's working fine but not on gcloud. Tried lot of solutions from google but not able to get it. Please do provide me if any suggestion to make it work.
Here is my directory and config:
Build directory
Yaml config
runtime: nodejs16 # or another supported version
instance_class: B1
service: adminoper
basic_scaling:
max_instances: 1
idle_timeout: 10m
handlers:
- url: /
static_dir: build
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
Google cloud error
I got few answers in google to add entrypoint path but that's not working. Please do suggest.

Django unable to create multiple databases for test in circleci

I have a Django project which uses 6 databases. For CI/CD, I am using CircleCI. I have written some unit test cases which works fine on my machine (local environment). But when I try to run it in the CircleCI environment, it fails. The reason of failing is Django creates only one database from the six (that also random i.e different one every time). I am not sure what I am doing wrong.
Here is the config, I am using for CircleCI
version: 2.1
orbs:
python: circleci/python#0.2.1
jobs:
test-job:
docker:
- image: circleci/python:3.8
environment:
DATABASE_URL: mysql://root#127.0.0.1:3306/db0
DB1_DATABASE_URL: mysql://root#127.0.0.1:3306/db1
DB2_DATABASE_URL: mysql://root#127.0.0.1:3306/db2
DB3_DATABASE_URL: mysql://root#127.0.0.1:3306/db3
DB4_DATABASE_URL: mysql://root#127.0.0.1:3306/db4
DB5_DATABASE_URL: mysql://root#127.0.0.1:3306/db5
ALLOWED_HOSTS: localhost
CORS_ORIGIN_WHITELIST: http://localhost:8080
CONN_MAX_AGE: 150
DEBUG: False
QRCODE_URL: http://test.com/
- image: circleci/mysql:8.0.18
command: [--default-authentication-plugin=mysql_native_password]
environment:
MYSQL_DATABASE: db0
steps:
- checkout
- python/load-cache
- python/install-deps
- python/save-cache
- run:
command: python manage.py test
name: Run Test
workflows:
main:
jobs:
- test-job:
filters:
branches:
only:
- add_test_to_circleci
Any help would be highly appreciated. Thanks in advance!

Deploying Django Web App using Devops CI/CD onto Azure App Service

I'm trying to deploy simple django web ap to Azure App Service using CI/CD pipeline (the most basic one that is offered by Microsoft for app deployment- no changes from me). However I'm getting the following error:
2021-03-08T16:55:51.172914117Z File "", line 219, in _call_with_frames_removed
2021-03-08T16:55:51.172918317Z File "/home/site/wwwroot/deytabank_auth/wsgi.py", line 13, in
2021-03-08T16:55:51.172923117Z from django.core.wsgi import get_wsgi_application
2021-03-08T16:55:51.172927017Z ModuleNotFoundError: No module named 'django'
I checked other threads and tried doing all the things mentioned but it did not help, or I am missing something:
In wsgi.py I added:
import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/..' )
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../licenses_api')
sys.path.append(os.path.dirname(os.path.abspath(__file__)) + '/../deytabank_auth')
from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'deytabank_auth.settings')
application = get_wsgi_application()
But still getting the same error, where django is not recognized. I can see that reuqirements.txt is being installed successfully and it has all the neccessary libraries there (including Django)
My CI/CD yaml file looks like this:
# Python to Linux Web App on Azure
# Build your Python project and deploy it to Azure as a Linux Web App.
# Change python version to one thats appropriate for your application.
# https://learn.microsoft.com/azure/devops/pipelines/languages/python
trigger:
- develop
variables:
# Azure Resource Manager connection created during pipeline creation
azureServiceConnectionId: '***'
# Web app name
webAppName: 'DeytabankAuth'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Environment name
environmentName: 'DeytabankAuth'
# Project root folder. Point to the folder containing manage.py file.
projectRoot: $(System.DefaultWorkingDirectory)
# Python version: 3.7
pythonVersion: '3.7'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: BuildJob
pool:
vmImage: $(vmImageName)
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(pythonVersion)'
displayName: 'Use Python $(pythonVersion)'
- script: |
python -m venv antenv
source antenv/bin/activate
python -m pip install --upgrade pip
pip install setup
pip install -r requirements.txt
workingDirectory: $(projectRoot)
displayName: "Install requirements"
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(projectRoot)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
displayName: 'Upload package'
artifact: drop
- stage: Deploy
displayName: 'Deploy Web App'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: DeploymentJob
pool:
vmImage: $(vmImageName)
environment: $(environmentName)
strategy:
runOnce:
deploy:
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(pythonVersion)'
displayName: 'Use Python version'
- task: AzureWebApp#1
displayName: 'Deploy Azure Web App : DeytabankAuth'
inputs:
azureSubscription: $(azureServiceConnectionId)
appName: $(webAppName)
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
Maybe I need to configure something in the Azure App Service? But i am not sure exactly what.
I have met this issue before, and the problem might be your deployment method. Not sure which one you use, but the classic deployment center below is being deprecated, try use the new deployment center.
Checked your workflow with the one worked on my side, there is nothing different. So I will post the correct step worked on my side for you to refer.
check your project locally to make sure it could run successfully.
Create a new web app (this is to make sure no damage on your web app) and navigate to the Deployment center page.
Go to your GitHub and navigate to GitHub Action page to see the log.
Test your web app and check the file structure on KuDu site: https://{yourappname}.scm.azurewebsites.net/wwwroot/
You could test by click the browse button like what I did.
If you want to run command, go to this site: https://{yourappname}.scm.azurewebsites.net/DebugConsole
By the way, I post this link if you need deploy using DevOps.
The possible reason for this question is that you don't have Django installed.
In the Microsoft-hosted agent ubuntu-latest, Django is not pre-installed. That is, you need to install it manually.
pip install Django==3.1.7
Click this document for detailed information about downloading Django.

cf v3-push with manifest and variable substitution

I have a v3 app that I want to deploy to 2 different environments. The app name and some definitions vary from env to env, but the structure of the manifest is the same. For example:
# manifest_test.yml
applications:
- name: AppTest
processes:
- type: web
command: start-web.sh
instances: 1
- type: worker
command: start-worker.sh
instances: 1
# manifest_prod.yml
applications:
- name: AppProd
processes:
- type: web
command: start-web.sh
instances: 3
- type: worker
command: start-worker.sh
instances: 5
Instead of keeping duplicate manifests with only minor changes in variables, I wanted to use a single manifest with variable substitution. So I created something like this:
# manifest.yml
- name: App((env))
processes:
- type: web
command: start-web.sh
instances: ((web_instances))
- type: worker
command: start-worker.sh
instances: ((worker_instances))
However, it seems like cf v3-apply-manifest doesn't have an option to provide variables for substitution (as cf push did).
Is there any way around this, or do I have to keep using a separate manifest for each environment?
Please try one of the cf v7 cli beta releases. I didn't test it but the output from cf7 push -h has a flag for using --vars and --vars-file. It should also use the v3 APIs so it will support things like rolling deploy.
For what it's worth, if you're looking to use CAPI v3 features you should probably use the cf7 beta releases going forward. That is going to get you the latest and greatest support for the CAPI v3.
Hope that helps!

Using Google endpoints in different modules of the same app

I'm quite new to development with Google App engine and other Google services of the Cloud platform and I'd like to create an app with different modules (so they can have their own lifecycle) which use endpoints.
I'm struggling with api paths because I don't know how to route requests to the good module.
My directory tree is like that:
/myApp
/module1
__init__.py
main.py
/module2
__init__.py
main.py
module1.yaml
module2.yaml
dispatch.yaml
module1.yaml
application: myapp
runtime: python27
threadsafe: true
module: module1
version: 0
api_version: 1
handlers:
# The endpoints handler must be mapped to /_ah/spi.
# Apps send requests to /_ah/api, but the endpoints service handles mapping
# those requests to /_ah/spi.
- url: /_ah/spi/.*
script: module1.main.api
libraries:
- name: pycrypto
version: 2.6
- name: endpoints
version: 1.0
module2.yaml
application: myapp
runtime: python27
threadsafe: true
module: module2
version: 0
api_version: 1
handlers:
# The endpoints handler must be mapped to /_ah/spi.
# Apps send requests to /_ah/api, but the endpoints service handles mapping
# those requests to /_ah/spi.
- url: /_ah/spi/.*
script: module2.main.api
libraries:
- name: pycrypto
version: 2.6
- name: endpoints
version: 1.0
dispatch.yaml
dispatch:
- url: "*/_ah/spi/*"
module: module1
- url: "*/_ah/spi/.*"
module: module2
So I'd like my endpoints to be called with the name of the corresponding module somewhere ('_ah/api/module1' or 'module1/_ah/api'). I don't know what to put in the different .yaml files. I don't even know if what I'm doing is right, or possible.
Thanks for your answers.
You can host different endpoints on different modules (now called services); the way to correctly address them is as follows:
https://<service-name>-dot-<your-project-id>.appspot.com/_ah/api
Now, let's say you have—as per your description—module1 and module2, each one hosting different endpoints. You will call module1 APIs by hitting:
https://module1-dot-<your-project-id>.appspot.com/_ah/api
And in a similar fashion, module2 APIs:
https://module2-dot-<your-project-id>.appspot.com/_ah/api
If you want to dig deeper into how this URL schema works (including versions, which are another important part of the equation here), go read Addressing microservices and the immediately following section Using API versions