What is the right way of running the unit test cases for django as part of build process ?
We use Jenkins pipeline for building the docker image and container will be started by using a startup script.
Do i need to call the manage.py test before the nginx container was started ?
or
Do i need to add a post build task in the Jenkins build pipeline to run the tests after the container was started ?
So basically looking for best practice to run tests.. Do we need to run unit tests before the server has started or after the server has started ?
I know that it makes more sense to run unit tests before we start the nginx, but won't it create an increased time with more n more test cases being added in future?
Depends on your test cases. If you are running unit tests only you don't need to. If you are doing something more in your tests like for example calling your apis (functional testing, etc) a good approach (in my opinion) is to create different stages in your jenkinsfile where you first build the docker image, then run the unit tests, then decide what to do depending on the test results. I see this as a good thing because you will be running tests over your app inside the same container (same conditions) it will be running in a production environment. Another good practice would be to add some plugins to Jenkins and have some reports (i.e. coverage).
Related
I'm building a CI/CD for a django application. We dockerized the application and now our goal is to automate the building process triggered by push on a github repository. We are now building the GitHub Actions side. The project requires all containers to be running. I'm wondering where I should be running tests.
Running them in the docker file seems useless as there are several tests that would fail if the other containers are not running (postgres container for example, or rabbitmq ). The approach I was thinking was to maybe setup a job in GitHub actions, build and start all containers with compose and then run the tests ? What is the recommended approach ?
I set up a new Flask Python server and I created a Dockerfile with all my codes. I've written some unit tests and I'm executing them locally. When should I execute them if I want to implement a CI/CD?
I also need to write integration tests (to test if I'm querying the database correctly, to understand if the endpoint is exposed correctly, and so on), when should I execute them in a CI/CD?
I was thinking to execute them during the docker build so to put the execution of the tests in the Dockerfile. Is it correct?
Unit tests: Outside of Docker, before you run your docker build. Within your CI pipeline, after checking out the source code and running any setup steps like installing package dependencies.
Integration tests: Launched from outside of Docker; depending on how complex your setup is, either late in your CI pipeline or as part of your CD pipeline.
This assumes a true "unit test" that has no external dependencies; it depends only on the application/library code, and where it needs things like databases, it either mocks out those dependencies or uses something like an embedded SQLite. (Some frameworks are especially bad at this workflow and make it impossible to start up the application at all if the database isn't available. But Rails doesn't run on Python.)
Running unit tests in a Dockerfile will last until it's midnight, you have a production outage, and either your quick fix that will bring the site back up happens to break one obscure unit test, or you can't wait the 5-minute cycle time to run the whole unit-test suite. Since there shouldn't be dependencies on the Docker-or-not environment in your unit tests, I'd just run them outside Docker.
Often you can stand up enough infrastructure to be able to run your application "for real" with a couple of docker run commands or a simple Docker Compose setup. In that case, it makes sense to run an integration test towards the end of your CI pipeline. With a more complex setup (maybe one involving Kubernetes) you might need to actually deploy into a test environment, and if you have separate CI and CD tools, this would turn into "test deploy", "integration test", "pre-production deploy".
As a developer I find having tools not-in-Docker vastly easier to manage than tools that only run in Docker. (I don't subscribe to the "any binary other than /usr/bin/docker is bad" philosophy.) I'd rather just run pytest or curl than remember the 4-line docker run invocation to do some specific task.
Locally I'm running meteor test --driver-package practicalmeteor:mocha --port 4001 to run my unit tests, which is refreshing each time I am changing files. So this is ok for developing.
In my CI process I would like to do first unit tests, and if they are passing, I do a meteor build and deploy the application to a docker container. This is working, but running meteor test --once --driver-package dispatch:mocha as a CI stage will always take some minutes (5-15 minutes) each time to start up a meteor instance, which is not very practical for unit tests. Working, but takes way too much time...
So my idea would be if it is technically possible to run a kind of meteor test server as a docker container, which is already running. In the CI unit test stage I would pull the repository (files) and run the tests with those immediately.
But is this possible with meteor at all?
I'm trying to get started with Docker for developing a web application with Clojure and am unsure which way to go. From what I've read so far and also looking at the offical Docker Clojure repo, there are basically two possible ways:
call lein ring server (interactively or as a CMD in a Dockerfile) or
use a Dockerfile to compile your application into an uberjar and use java -jar as the CMD on the resulting jar file.
The former seems to me to be problematic in the sense that the dev environment is not as close as possible to the production environment, given that we're probably using a :dev leiningen profile adding stuff that one would strictly not want in production (providing as few tools and "information", i.e. code on an exposed production server is always a good idea). The latter, however, seems to have the exact opposite problem: Now every change requires basically a rebuild of the image (think edit-compile-run cycle), so you would lose lein rings nice re-compile on modification functionality.
How are people using this combination in practice?
PS: I'm aware that there might be some other modes of operation in practice (e.g. using Immutant or Tomcat as the deployment target or using a CI server like Hudson etc.). I'm asking about the most basic setup first here.
My team and I have opted to optimize rapid feedback while developing and minimize the number of moving parts in our deploys. As a result we've opted to use lein ring server in development and opt to ship an uberjar for our deployment. I've done this with code running in docker containers and without them.
I wouldn't want to go back to using a development workflow that didn't enable seeing the results of changing code as quickly as possible. In my mind, the rapid feedback far outweighs the risk of the running services slightly differently between my local machine and production.
Also, nothing stops me from changing a couple lines of code and then starting up a local service that is running much closer to my production setup (either running a built docker image or building an uberjar locally).
There's nothing stopping you from running in production mode with Leiningen. Just use:
lein with-profile production ring server
I've used both approaches successfully, although we've settled on the uberjar approach because it gives faster startup times.
I use the second option java -jar ... to deploy my web application to production (not using Docker yet). This creates an edit-compile-run cycle as you said. But I don't recompile for every change. Only when I'm ready to release I create the uberjar. Of cource CI is always recommended.
I am writing an app that records time of events. For unit testing I would usually use a monkey-patch to replace datetime.time with a fake so I can test it properly. I am trying to do end-to-end tests with Selenium, with the test cases in a separate program, not using python manage.py test. Therefore I can't do a patch. I did try using manage.py but it did not seem to help.
I'm sure this is a solved problem. How should I be doing it? Is Selenium just not the right tool for this sort of testing? Am I missing how to get the test case to talk to the application?
Selenium talks to a full webserver and has no access to the python interpreter running inside that webserver. Even if you are scripting SeleniumRC with python, the script instance of the interpreter is separate from the webserver instance.
If you are running the test webserver via manage.py runserver, you could write your own management command to replace 'runserver' with a version that patches datetime.time. This won't be easy, so you may consider either revising your Selenium-driven tests to cope with events happening in realtime, or convert you time-sensitive tests to django client tests so you can use the mock library.