In previous api-manager version when we create an api, api synapse files were recorded in Users/bilal/test/api-manager/wso2am-2.6.0/repository/deployment/server/synapse-configs/default/api. But in 410 only default apis are there and my apis' synapse file is not there. But the api can be called without issue. Same thing for sequences.
I need to check and change the mediation that was applied. Always changing and uploading from the publisher is too much of work.
Adding to what pubudu has mentioned above. From APIM 3.2.0 onwards, we have introduced inbuilt artifact synchronizer and this will store the API artifacts such as synapse api definition, mediation sequences and endpoints to the database. It will then use events to distribute these artifact across gateways.
Previously, artifacts were saved to the file system and if you had multiple gateway nodes, file based synchronizers such as NFS, rsync had to be used to deploy these artifacts across gateway cluster.
Because of this new architecture, we removed the artifacts being saved to the file system and provided a gateway rest API, that you can use to view the deployed artifacts or redeploy artifacts in each gateway. If your requirement is to verify the deployed artifact, you can use this API.
And from APIM 4.1.0, we have changed the mediation policies feature to the new API policies framework.
If your requirement is to change the meditation sequence manually, you still can create a mediation sequence as you have created for APIM 2.6.0 and use the correct naming convention <API_NAME>:<VERSION>--<Direction> (eg: PizzaShackAPI:v1.0.0--In) and add it to the sequence directory. This will deploy the policy in the gateway and will be engaged when you invoke the API. However since we no longer keep artifacts in the file system, every restart will clear these directories and only keep the whitelisted files. That's where pubudu's above configurations are used for and you need to add your sequences' name to this configuration under sequences.
Please note that never to use these file based artifacts in your production deployment and only use them for testing progress. This is because whenever you add a policy from the publisher and deploy it in the gateway, this mediation policy will be undeployed and that file will deploy. At the server startup, this will create a inconsistent behaviour. so my recommendation is, you can use this apporach during the sequence development phase and once you have satisfied with it, use the correct apporach and add it as an API policy.
In the latest API Manager version, you can't find those files as those are not written into those files. Those data is kept in the memory.
You can add the following config to the deployment.toml to provide files from the file system.
[apim.sync_runtime_artifacts.gateway.skip_list]
apis = ["api1.xml","api2.xml"]
endpoints = ["endpoint1.xml"]
sequences = ["post_with_nobody.xml"]
local_entries = ["file.xml"]
Related
Env : We using GCP cloud composer to run airflow dags.
Aim : To use apache airflow stable apis to trigger dags externally using REST.
We tried to use airflow experimental apis to trigger dags externally by setting airflow override configuration in cloud composer: airflow.api.auth.backend.default and making IAP request. It worked fine. We followed steps described in https://cloud.google.com/composer/docs/how-to/using/triggering-with-gcf.
Now starting from Airflow 2.0.0 + , airflow experimental apis are deprecated and new stable apis are introduced. The new stable api doesn't supports airflow.api.auth.backend.default. Hence it is throwing :403 FORBIDDEN.
Also we tried with auth_backend : airflow.api.auth.backend.basic_auth , but this also not working as when we pass user/password in Authorization header, we are not able to pass IAP Bearer token.
Please share the step to access new airflow apis similar to steps shown to access experimental apis in https://cloud.google.com/composer/docs/how-to/using/triggering-with-gcf
Do we need to do double Auth i.e first Authorization with IAP and then with Airflow ?
TL'DR version:
In order to make Airflow Stable API work at GCP Composer:
Set "api-auth_backend" to "airflow.composer.api.backend.composer_auth"
Make sure your service account email length is <64 symbols
Make sure your service account has required permissions (Composer User role should be sufficient)
Longread:
We are using Airflow for a while now, and started with version 1.x.x with "experimental" (now deprecated) API's.
To Authorize, we are using "Bearer" token obtained with service account:
# Obtain an OpenID Connect (OIDC) token from metadata server or using service account.
google_open_id_connect_token = id_token.fetch_id_token(Request(), client_id)
# Fetch the Identity-Aware Proxy-protected URL, including an
# Authorization header containing "Bearer " followed by a
# Google-issued OpenID Connect token for the service account.
resp = requests.request(
method, url,
headers={'Authorization': 'Bearer {}'.format(
google_open_id_connect_token)}, **kwargs)
Now we are migrating to Airflow 2.x.x and faced with exact same issue:
403 FORBIDDEN.
Our environment details are:
composer-1.17.3-airflow-2.1.2 (Google Cloud Platform)
"api-auth_backend" is set to "airflow.api.auth.backend.default".
Documentation claims that:
After you set the api-auth_backend configuration option to airflow.api.auth.backend.default, the Airflow web server accepts all API requests without authentication.
However, this does not seem to be true.
In experimental way, we found that if "api-auth_backend" is set to "airflow.composer.api.backend.composer_auth", Stable REST API (Airflow 2.X.X) starting to work.
But there is other caveat to this: for us, some of our service accounts did work, and some did not.
The ones that did not work were throwing "401 Unauthorized" error.
We figured out that accounts having email length > 64 symbols were throwing error. Same was observed at this answer.
So after setting "api-auth_backend" to "airflow.composer.api.backend.composer_auth" and making sure that our service account email length is <64 symbols - our old code for Airflow 1.x.x started to work for Authentication. Then we needed to make changes (API URLs and response handling) and stable Airflow (2.x.x) API started to work for us
in the same way as it was for Airflow 1.x.x.
UPD: this is a defect in Airflow and will be fixed here: https://github.com/apache/airflow/pull/19932
Cloud Composer's support for Airflow2 stable API starts from composer-1.17.0-preview.12 onwards.
The how-to documentation is here:
Access Airflow REST API
Trigger DAG with Cloud Function
I have an API that I have setup in WSO2's API Manager Publisher tool. All the calls work fine when called correctly with an Access Token.
But I have one url that I want anyone to be able to call. (It is a customized Swagger UI page (different from the one built into the WSO2 tooling).)
To do this I add the Swagger UI url as a resource. But when I go there it says:
Required OAuth credentials not provided. Make sure your API invocation call has a header: 'Authorization : Bearer ACCESS_TOKEN'
Generally, I really like this default. (I want all my other API resources to require an Access Token.) But for this one I don't have an access token and don't expect to.)
Is it possible to publish one resource and not need any authentication?
Also it can be done in 2.x versions of API Manager through disabling for each endpoint the security in publisher tool. In manage tab, resources section, for each resource/endpoint set the value x-auth-type to "None". Default value is "Application & Application User". This disable security and then authentication required for accesing to the selected API resources.
This can be done as mentioned in https://stackoverflow.com/a/61133046/3176125
Basically you can turn off for one of the resources while you allow security for the rest of the resources.
I am working on one scenario where I have add a sequence in the API using restAPI of APIM2.6.0 [https://docs.wso2.com/display/AM260/apidocs/publisher/#!/operations#MediationPolicyCollection#apisApiIdPoliciesMediationPost]
Once the mediation policy is added to the respective API, do we have to publish the API once again. As when I am doing it from publisher once the sequence is added to direction IN, then I am saving it to get it reflected to synapse.
So, I believe if I am adding the same from restAPI, then also I have to republish it again to get it reflected to synapse.
Please let me know if the understanding is correct.
Thanks
Yes, you need to republish the API again to reflect the changes applied. You can check the code when adding a mediation policy for the API in [1].
[1] - https://github.com/wso2/carbon-apimgt/blob/85d02e1864bf77bd53bd269445995ab8e8e9641f/components/apimgt/org.wso2.carbon.apimgt.rest.api.publisher/src/main/java/org/wso2/carbon/apimgt/rest/api/publisher/impl/ApisApiServiceImpl.java#L796
Yes.
Normally, the once the API is published, all the changes will be auto deployed.
So, adding the mediation policies should also reflect without re-publishing the API.
Update
The answer above is for when using the UI. In REST API, we should republish the API.
How to chain multiple APIs on the same URL using Azure API management?
I have two APIs the user store and that user recources api.
I want to build rest api so resources will related to user.
Mycompany/users/dani#gmail.com/resources.
User store api is mycompany/users
Resources api is mycompany/resources.
I dont want to build additional service that will consolidate this two apis.
Thanks
Additionally you can use 'send-request' policy, that will call another API endpoint and store the result in declared variable. This is what worked for me.
In my case I had hit another URL after getting response of first APIs. Same can be done in inbound policy also before getting response and similarly you can hit multiple APIs.
Policy snippet.
You can create a single API in API Management that points to multiple different backend APIs. Use the set-backend-service policy to change the backend API for certain operations.
I followed steps provided in this https://www.c-sharpcorner.com/blogs/api-aggregation-using-azure-apim link and able to combine two API responses within one API call.
It tells how to aggregate or chain more than one API responses in API management
I've created a simple API and I'm trying to publish it using WSO2's API Publisher (aka API Cloud). I've gone through all the steps, but it seems to require an Authorization header to access my endpoint. In older documentation, it says that I can change the "Auth Type" at the resource level.
https://docs.wso2.com/display/AM160/API+Resources
However, this option doesn't seem to be there in the current version. I tried to make it so the Authorization header was not required. Unfortunately, I still get the following error:
<ams:fault xmlns:ams="http://wso2.org/apimanager/security">
<ams:code>900902</ams:code>
<ams:message>Missing Credentials</ams:message>
<ams:description>Required OAuth credentials not provided</ams:description>
</ams:fault>
Is it possible to disable authentication for my API? I don't need it at this point in my project.
The document you have referred is from APIM 1.6. From APIM 1.7, the APIM team changed the API creation process to a 3-step process. It involves API Design, Implement and Manage. I think you have experienced this by now. In the Manage section, at the very bottom, it lists down the available resources of the API, their auth type, allowed tier and the scope allowed.
Default auth type is application & application user. If you click on that, you will get a drop down where you will see "None" as an option. If you set the auth type as none, you will be able to invoke the API without providing the OAuth token.
See the following screenshot where I have selected different Auth types when creating an API.
Open the configuration related to your API in ${AM_HOME}/repository/deployment/server/synapse-configs/default/api/ and remove the following part.
<handler class="org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler"/>