If we upgrade package version using GCP OS Patch Management but you want to downgrade package version.
Is there any option available with OS Patch Management?
Related
I am currently running Google Cloud Composer with a Composer version 2.0.9 and airflow version 2.1.4. I am trying install the most recent version of dbt (1.0.4 for core and 1.0.0 for the BigQuery plugin). Because cloud composter images has specific packages installed, I am getting conflicting PyPI dependency issues. When I try to fix one dependency another issue occurs. Does anyone know the specific set of packages installed that would resolve this issue? I have read the following posts by the community but I wanted to know if anyone has a solution for just using composer?
How to run DBT in airflow without copying our repo
How to set up dbt with Google Cloud Composer?
I was able to reproduce the behaviour you are seeing. Below are the dependency conflicts I saw in the Cloud Build logs. These conflicts are occurring between the dbt-core requirements and the pre-installed package requirements in Composer.
Pre-installed package requirements:
hologram 0.0.14 has requirement jsonschema<3.2,>=3.0, but you have jsonschema 3.2.0. ##=> can be installed manually
flask 1.1.4 has requirement click<8.0,>=5.1, but you have click 8.1.2.
apache-airflow 2.1.4+composer has requirement markupsafe<2.0,>=1.1.1, but you have markupsafe 2.0.1.
looker-sdk 22.4.0 has requirement typing-extensions>=4.1.1, but you have typing-extensions 3.10.0.2.
dbt-core requirements:
hologram 0.0.14 has requirement jsonschema<3.2,>=3.0, but you have jsonschema 3.2.0. ##=> can be installed manually
dbt-core 1.0.4 has requirement click<9,>=8, but you have click 7.1.2.
dbt-core 1.0.4 has requirement MarkupSafe==2.0.1, but you have markupsafe 1.1.1.
dbt-core 1.0.4 has requirement typing-extensions<3.11,>=3.7.4, but you have typing-extensions 4.1.1.
I tried downgrading the pre-installed packages, but subsequent package installations fail and it is not recommended as well.
Therefore, I would suggest using an external solution as stated in this thread you have linked. Quoting the workarounds given in #Ryan Yuan's answer here.
Using external services to run dbt jobs, e.g. Cloud Run.
Using Composer's KubernetesPodOperator(updated Composer 2 link). My colleague has put up a nice article on dbt discourse here going through the setup process.
Ignoring Composer's Dependency conflicts by setting Composer's environmental variable IGNORE_PYPI_DEPENDENCY_CONFLICTS to True.
However, I don't recommend this as it may cause potential issues.
Creating a Python virtual environment in Composer and install the dbt packages.
As mentioned by #Kabilan Mohanraj, the current version of dbt (1.0.4) and a more recent version of Composer has dependency issues (Composer version 2.0.9 and Airflow version 2.1.4). Therefore an alternative solution is needed. In my case, I played around and searched for a solution from other people in the community and found one person using a certain version of Composer and dbt that only had mimimal dependency issues. However, as mentioned by #Kabilan Mohanraj, Google does not recommend downgrading preinstalled packages, so this would not be a viable solution for something in production.
create composer through gcloud to use an older version that is not available via the Composer UI
gcloud composer environments create my_airflow_dbt_example
--location us-central1
--image-version composer-1.17.9-airflow-2.1.4
requirements
dbt-bigquery==0.21.0
jsonschema==3.1.1
packaging==20.9
For this specific composer version, you are downgrading jsonschema from 3.2.0 to 3.1.1 and packaging from 21.3 to 20.9
I am using Cloud Foundry's nodejs profile and my nodejs package.json requires chartjs-node-canvas. That package uses node-canvas and node-canvas is based on Cairo. The node-canvas site says I have to add the cairo-devel package to Linux (apt-get) in order for canvas to be installed.
Is it possible to add software to the OS image running on cloud foundry? If so, how?
You can do that by vendoring the dependencies. When you vendor them, you'll build locally in an Ubuntu Bionic Linux container or VM. Node will build everything that's required and you will no longer need the cairo-devel package (it's only needed to build).
The process to vendor dependencies is documented here.
The other option is to use the Apt Buildpack which is described on this SO post. That can be used to install any apt packages.
According to GCP's OS patch management, patch jobs can be created for the Compute Instance VMs that have the OS config agent installed in them.
This document explains how to install the agent if it is not there already in some VMs.
Operating systems it covers are:
Windows Server / Ubuntu / Debian / RHEL / CentOS SLES / openSUSE
How to create a patch job for compute instances with Google container optimized os (COS)?
I am not able to find a way to install the OS config agent in these COS instances neither an option to create a "OS Patch Deployment". Is this even possible?
Unfortunately, you're not able to use OS patch management for VM instances with Google Container Optimized OS.
Please have a look at the documentation Creating patch jobs section Supported operating systems:
Debian 9
Ubuntu 16.04 and 18.04
CentOS 6, 7, and 8
Red Hat Enterprise Linux (RHEL) 6, 7, and 8
Windows Server 2012R2, 2016, 2019, and semi-annual releases 1803 and 1809
SUSE Enterprise Linux Server (SLES) 12 and 15, openSUSE Leap 15
Also, at section Setting up your VM:
To use the OS patch management service, you need to set up the OS
Config service API and install the OS Config agent. For detailed
instructions, see Managing your operating systems.
and as you can see at the documentation Managing your operating systems section Installing the OS Config agent there's no option for Google COS.
You can try to file a feature request at Google Issue Tracker if you need this feature.
I have a specific version of google-cloud-core in my server and I don’t know how to choose the other libraries in a way to make them suit with my google cloud core version.
I can’t just move to the latest versions because old programs cannot run into new versions for example BigQuery library.
My specific need is to know « how to know » which version of Google Cloud Storage should I choose according to the 0.26.0 core version.
Is there some repositories where we can find packages grouped by google-cloud-core versions?
In my case the version 1.6 of google-cloud-storage works but I found it just by downgrade and try again method !
Best regards
What you can do as a workaround is create a virtual environment, install a specific library - like google-cloud-storage - and check the dependencies installed with that library version. I made a quick test and installed a few versions of google-cloud-storage. For version 1.3.0, the google-cloud-core 0.26.0 dependency was installed.
You can do so by following these steps:
virtualenv env-name
source env-name/bin/activate
pip freeze (to check there is nothing there)
pip install google-cloud-storage==1.3.0
pip freeze (again)
Once finished you’ll see google-cloud-core 0.26.0 was installed.
Is it possible to install API Manager 1.6.0 Feature on wso2esb 4.8.1 .I tried to install using Feature Management,but it is showing dependency problem..
While trying to install using feature management,i am getting the below error
Your original install request has been modified. org.wso2.carbon.registry.core.server.feature.group-4.2.1 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.security.mgt.feature.group-4.2.2 is already installed, so an update will be performed instead. org.wso2.carbon.module.mgt.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.rest.api.server.feature.group-4.2.1 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.endpoint.server.feature.group-4.2.1 will be ignored because a newer version is already installed. org.wso2.carbon.registry.community.features.server.feature.group-4.2.1 will be ignored because it is already installed. org.wso2.carbon.event.server.feature.group-4.2.1 will be ignored because it is already installed. org.wso2.carbon.application.mgt.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.mediators.server.feature.group-4.2.1 will be ignored because a newer version is already installed. org.wso2.carbon.sequences.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.datasource.server.feature.group-4.2.1 will be ignored because it is already installed. org.wso2.carbon.application.mgt.synapse.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.mediation.templates.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.relay.server.feature.group-4.2.0 will be ignored because it is already installed. org.wso2.carbon.localentry.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.logging.mgt.feature.group-4.2.1 is already installed, so an update will be performed instead. org.wso2.carbon.security.mgt.server.feature.group-4.2.2 is already installed, so an update will be performed instead. org.wso2.carbon.registry.resource.properties.feature.group-4.2.0 is already installed, so an update will be performed instead. org.wso2.carbon.service.mgt.server.feature.group-4.2.1 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.event.common.feature.group-4.2.0 will be ignored because it is already installed. org.wso2.carbon.task.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.throttle.feature.group-4.2.0 will be ignored because it is already installed. org.wso2.carbon.rule.mediation.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.core.common.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.core.runtime.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. org.wso2.carbon.identity.authenticator.saml2.sso.server.feature.group-4.2.0 is already installed, so an update will be performed instead. org.wso2.carbon.identity.authenticator.saml2.sso.ui.feature.group-4.2.1 will be ignored because it is already installed. org.wso2.carbon.core.server.feature.group-4.2.0 is already present because other installed software requires it. It will be added to the installed software list. Cannot complete the install because of a conflicting dependency. Software being installed: WSO2 Carbon - Webapp Management Feature 4.2.1 (org.wso2.carbon.webapp.mgt.feature.group 4.2.1) Software being installed: WSO2 Carbon - Webapp Management Core Feature 4.2.0 (org.wso2.carbon.webapp.mgt.server.feature.group 4.2.0) Only one of the following can be installed at once: WSO2 Carbon - Webapp Management Core Feature 4.2.2 (org.wso2.carbon.webapp.mgt.server.feature.jar 4.2.2) WSO2 Carbon - Webapp Management Core Feature 4.2.1 (org.wso2.carbon.webapp.mgt.server.feature.jar 4.2.1) WSO2 Carbon - Webapp Management Core Feature 4.2.0 (org.wso2.carbon.webapp.mgt.server.feature.jar 4.2.0) Cannot satisfy dependency: From: WSO2 Carbon - Webapp Management Feature 4.2.1 (org.wso2.carbon.webapp.mgt.feature.group 4.2.1) To: org.wso2.carbon.webapp.mgt.server.feature.group [4.2.1] Cannot satisfy dependency: From: WSO2 Carbon - Webapp Management Core Feature 4.2.0 (org.wso2.carbon.webapp.mgt.server.feature.group 4.2.0) To: org.wso2.carbon.webapp.mgt.server.feature.jar [4.2.0] Cannot satisfy dependency: From: WSO2 Carbon - Webapp Management Core Feature 4.2.1 (org.wso2.carbon.webapp.mgt.server.feature.group 4.2.1) To: org.wso2.carbon.webapp.mgt.server.feature.jar [4.2.1]
Can anyone help me on this please....
Thanks in advance,,,,