Django unable to create multiple databases for test in circleci - django

I have a Django project which uses 6 databases. For CI/CD, I am using CircleCI. I have written some unit test cases which works fine on my machine (local environment). But when I try to run it in the CircleCI environment, it fails. The reason of failing is Django creates only one database from the six (that also random i.e different one every time). I am not sure what I am doing wrong.
Here is the config, I am using for CircleCI
version: 2.1
orbs:
python: circleci/python#0.2.1
jobs:
test-job:
docker:
- image: circleci/python:3.8
environment:
DATABASE_URL: mysql://root#127.0.0.1:3306/db0
DB1_DATABASE_URL: mysql://root#127.0.0.1:3306/db1
DB2_DATABASE_URL: mysql://root#127.0.0.1:3306/db2
DB3_DATABASE_URL: mysql://root#127.0.0.1:3306/db3
DB4_DATABASE_URL: mysql://root#127.0.0.1:3306/db4
DB5_DATABASE_URL: mysql://root#127.0.0.1:3306/db5
ALLOWED_HOSTS: localhost
CORS_ORIGIN_WHITELIST: http://localhost:8080
CONN_MAX_AGE: 150
DEBUG: False
QRCODE_URL: http://test.com/
- image: circleci/mysql:8.0.18
command: [--default-authentication-plugin=mysql_native_password]
environment:
MYSQL_DATABASE: db0
steps:
- checkout
- python/load-cache
- python/install-deps
- python/save-cache
- run:
command: python manage.py test
name: Run Test
workflows:
main:
jobs:
- test-job:
filters:
branches:
only:
- add_test_to_circleci
Any help would be highly appreciated. Thanks in advance!

Related

Redirect error while deploying nuxt app with aws amplify

When I try to deploy the app the build completes without errors. When I go to the link provided (https://master.d3m2wky0hslwhr.amplifyapp.com/) I get ERR_TOO_MANY_REDIRECTS .
My build config:
version: 1
frontend:
phases:
preBuild:
commands:
- npm ci
build:
commands:
- npm run build
artifacts:
# IMPORTANT - Please verify your build output directory
baseDirectory: .nuxt
files:
- '**/*'
cache:
paths:
- node_modules/**/*
Inside the app I have (index.vue, login.vue and register.vue)
I think it may be because im beeing redirected to url/index.html and that file does not exist in the proyect.
Solved it.
Change -npm run build to -npm run generate
(This creates the index.html by default and converts everything to static)
Change baseDirectory to dist

Configure REDIS in Gitlab CI/CD for usage in Django

I had a working pipeline on GitLab which started failing after I added tests involving REDIS. I've applied (?) what is written inside GitLab docs but still REDIS is not discovered by my tests.
Here's my gitlab-cy.yml file:
pep8:
image: python:latest
services:
- postgres:10-alpine
- redis:latest
variables:
POSTGRES_DB: ci
DATABASE_URL: "postgresql://postgres:postgres#postgres:5432/$POSTGRES_DB"
REDIS_URL: redis
stage: test
script:
- python -V
- pip install -r ./requirements/gitlab.txt
- pytest --pep8
cache:
paths:
- ~/.cache/pip/
And here's how I'm using it inside Django:
REDIS_URL = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
cache = redis.StrictRedis.from_url(url=REDIS_URL)
I've also set Environment Variables inside CI/CD settings in GitLab REDIS_URL to redis however still when the test executes None gets assigned to host and the test fails ....-> self = Connection<host=None,port=6379,db=0>
Any idea how to connect with Redis on GitLab?
Finally figured it out, so I'm posting it here if you are also having this problem :)
I removed the REDIS_URL variable from gitlab-ci.yml and I have added the following variable in GitLab CI Environment variables settings:
REDIS_URL -> redis://redis:6379/0
which solved the problem :)

Integrating Selenium with Gitlab CI

I have created an automated selenium test script which works perfectly fine.
My task now is to set up Gitlab CI and try to automatically run this selenium script when I make a push to git.
Is it possible to make the selenium script automatically execute and inform the user if the script runs successfully or it fails?
Thank you
How to automatically run Automation Tests on Gitlab Ci with Selenium and specflow with a .net Project ?
If this is something ,you are looking for then .
Here is the core part which is to setup the gitlab-ci.yml file :
Here is how the sample gitlab-ci.yml should look :
image: please give your own docker which can download .net stuff
variables:
DOCKER_DRIVER: overlay2
SOURCE_CODE_DIRECTORY: 'src'
BINARIES_DIRECTORY: 'bin'
OBJECTS_DIRECTORY: 'obj'
NUGET_PACKAGES_DIRECTORY: '.nuget'
stages:
- Build
- Test
before_script:
- 'dotnet restore ${SOURCE_CODE_DIRECTORY}/TestProject.sln --packages ${NUGET_PACKAGES_DIRECTORY}'
Build:
stage: Build
script:
- 'dotnet build $SOURCE_CODE_DIRECTORY/TestProject.sln --no-restore'
except:
- tags
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
expire_in: 2 hr
Test:
stage: Test
services:
- selenium/standalone-chrome:latest
script:
- 'export MSBUILDSINGLELOADCONTEXT=1'
- 'export selenium_remote_url=http://selenium__standalone-chrome:4444/wd/hub/'
- 'export PATH=$PATH:${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- 'dotnet test $SOURCE_CODE_DIRECTORY/ExpressTestProject.sln --no-restore'
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
Thats it .When you set up your project with this .git-lab-ci.yml ,90 % of your job is done .
The tests will run automatically in Gitlab ,whenever you commit something in your source tree or Tfs.
Thanks

Running CircleCI 2.0 workflows on tags, REGEX

I have a circleci integrated github repository, I am running CircleCI 2.0
I want one workflow, test, to run on all branches and tags, so long as the tag does not match the pattern v*.. with an optional ending. And anything matching the pattern v*.. should be run through the deploy workflow. For example these tags should, v0.0.1 be run through deploy, v0.0.1beta should be run through deploy, v0.1beta or betav0.1.1 should be run through test instead. What would the appropriate regex to match this be, and if you are familiar with circleci 2.0, what changes do I need to make to this section of the configuration file in order for this to work.
workflows:
version: 2
test:
jobs:
- build:
filters:
tags:
ignore:
- /^v[0-9]\.[0-9]\.[0-9].+/
branches:
only:
- /.*/
- lint:
requires:
- build
deploy:
jobs:
- build
filters:
tags:
only:
- /^v[0-9]\.[0-9]\.[0-9].+/
branches:
ignore:
- /.*/
- lint:
requires:
- build
- deploy:
requires:
- lint
Try this:
^v\d\.\d\.\d\w{0,5}$

Custom runtime for non-flexible environment app?

I don't think that my gae python app has a flexible environment because I created it many years ago. Now I want to try and create a module that has another runtime than python and keep the python app running python alongside a new runtime, custom or just another. Maybe mix PHP and python or similar. I don't need it but I want to learn and explore the possibilities. I'm also interested in learning Erlang and deploy Erlang code with appengine. I see there is questions about it already
erlang on google app engine?
And issue 125 in the tracker.
But how should we actually do it? If we make our own runtime provided that is allowed.
My app.yaml looks like
application: montaoproject
version: newsearch
runtime: python27
api_version: 1
threadsafe: true
module: default
instance_class: F1
automatic_scaling:
min_idle_instances: 5
max_idle_instances: automatic
min_pending_latency: automatic
max_pending_latency: 30ms
max_concurrent_requests: 50
default_expiration: "14d 5h"
env_variables:
GAE_USE_MONTAO : 'anyvalue'
KOOL_VERSION : '17a'
includes:
- br.yaml # Brazil
- in.yaml # India
- us.yaml # USA
- pk.yaml
- search.yaml # search pages
- admin.yaml # admin pages
- providers.yaml # auth providers
- statics.yaml # static content
handlers:
- url: /(business|ai|newindia|insert-ad.html)
script: montao.app
- url: /blobview.*
script: kool_update.app
login: admin
- url: /market.*
script: main.app
- url: /
script: montao.app
- url: /(index.html|sign-up.html|login.html)
script: montao.app
- url: /(login.*|login|googlogin|googlogout|create/)
script: login.app
- url: /(customer_service.htm|contactfileupload|support.html|faq.html)
script: customer_service.app
- url: /stats.*
script: google.appengine.ext.appstats.ui.app
# All other URLs use main.app
- url: /.*
script: main.app
inbound_services:
- mail
builtins:
- remote_api: on
- deferred: on
#- appstats: on
error_handlers:
- file: default_error.html
libraries:
- name: webapp2
version: latest
- name: jinja2
version: latest
- name: setuptools
version: latest
- name: markupsafe
version: latest
- name: django
version: latest
- name: PIL
version: latest
- name: webob
version: latest
- name: lxml
version: latest
- name: ssl
version: latest
Yes, your app.yaml file is a standard env one (it doesn't have vm:true or env:flex in it).
Yes, it's possible to mix and match services/modules in different languages and with different environments inside the same app. You can even switch the language and environment of the same module in a different version of that module. That's because modules offer complete code isolation, see Comparison of service isolation and project isolation. Related post: Upload a Java and node.js project to Google AppEngine at once
I always try to structure a multi-service/module app with each service in its own subdir, as described in Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
So first I'd create a default subdirectory of your app dir and move all your existing default module specific files into it, with the exception of the app-level config, which I'd keep at the top level and symlink inside the default dir as described in that post. Then I'd verify that the default module still works as expected.
Then I'd create a new subdirectory for every new module I need to add and add the code for it as needed.
Side note: sharing code via symlinks as described in the post mentioned above works for standard env modules, but it probably doesn't work with flexible ones, see Sharing code between modules in a GAE project