Adding Service Provider to WSO2 Identity Server via file is not working - wso2-identity-server

Question

I want to configure within the WSO2 IDS a service provider which is available from the start. To do this I followed the following instructions: Adding a service provider
However when i boot the IDS and attempt to initiate a call to retrieve a token I get the following response:
{
"error_description": "A valid OAuth client could not be found for client_id: service-provider-fuga",
"error": "invalid_client"
}
and the log within the terminal of WSO2 IDS shows the following:
[2021-08-05 14:06:55,111] [0d5f9d6c-5f87-4dc3-a87f-cb473cd4127c] DEBUG {org.wso2.carbon.identity.oauth2.OAuth2Service} - Error while finding application state for application with client_id: 1ou1fLDyFA9BEqywVtrR6vAxc48a org.wso2.carbon.identity.oauth.common.exception.InvalidOAuthClientException: Cannot find an application associated with the given consumer key : 1ou1fLDyFA9BEqywVtrR6vAxc48a
at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.handleRequestForANonExistingConsumerKey(OAuthAppDAO.java:1154)
at org.wso2.carbon.identity.oauth.dao.OAuthAppDAO.getAppInformation(OAuthAppDAO.java:354)
at org.wso2.carbon.identity.oauth2.util.OAuth2Util.getAppInformationByClientId(OAuth2Util.java:1887)
The request I initiated is as follows: https://localhost:9443/oauth2/token?grant_type=password&client_id=service-provider-fuga&client_secret=...&username=user&password=...
The service provider file which is put in /home/wso2carbon/wso2-config-volume/repository/conf/identity/service-providers/service-provider.xml is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<ServiceProvider>
<ApplicationName>service-provider-fuga</ApplicationName>
<Description>Service Provider configuration for FUGA</Description>
<JwksUri/>
<InboundAuthenticationConfig>
<InboundAuthenticationRequestConfigs>
<InboundAuthenticationRequestConfig>
<InboundAuthKey>1ou1fLDyFA9BEqywVtrR6vAxc48a</InboundAuthKey>
<InboundAuthType>oauth2</InboundAuthType>
<InboundConfigType>standardAPP</InboundConfigType>
<inboundConfiguration><![CDATA[<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<oAuthAppDO>
<oauthConsumerKey>1ou1fLDyFA9BEqywVtrR6vAxc48a</oauthConsumerKey>
<oauthConsumerSecret>...</oauthConsumerSecret>
<applicationName>service-provider-fuga</applicationName>
<callbackUrl></callbackUrl>
<oauthVersion>OAuth-2.0</oauthVersion>
<grantTypes>refresh_token password </grantTypes>
<scopeValidators/>
<pkceSupportPlain>true</pkceSupportPlain>
<pkceMandatory>false</pkceMandatory>
<state>ACTIVE</state>
<userAccessTokenExpiryTime>3600</userAccessTokenExpiryTime>
<applicationAccessTokenExpiryTime>3600</applicationAccessTokenExpiryTime>
<refreshTokenExpiryTime>86400</refreshTokenExpiryTime>
<idTokenExpiryTime>3600</idTokenExpiryTime>
<audiences/>
<bypassClientCredentials>true</bypassClientCredentials>
<renewRefreshTokenEnabled>true</renewRefreshTokenEnabled>
<requestObjectSignatureValidationEnabled>false</requestObjectSignatureValidationEnabled>
<idTokenEncryptionEnabled>false</idTokenEncryptionEnabled>
<idTokenEncryptionAlgorithm>null</idTokenEncryptionAlgorithm>
<idTokenEncryptionMethod>null</idTokenEncryptionMethod>
<tokenType>JWT</tokenType>
</oAuthAppDO>
]]></inboundConfiguration>
<Properties/>
</InboundAuthenticationRequestConfig>
</InboundAuthenticationRequestConfigs>
</InboundAuthenticationConfig>
<LocalAndOutBoundAuthenticationConfig>
<AuthenticationSteps>
<AuthenticationStep>
<StepOrder>1</StepOrder>
<LocalAuthenticatorConfigs>
<LocalAuthenticatorConfig>
<Name>FugaAuthenticator</Name>
<DisplayName>FUGA Authenticator</DisplayName>
<IsEnabled>true</IsEnabled>
<Properties/>
</LocalAuthenticatorConfig>
</LocalAuthenticatorConfigs>
<FederatedIdentityProviders/>
<SubjectStep>false</SubjectStep>
<AttributeStep>false</AttributeStep>
</AuthenticationStep>
</AuthenticationSteps>
<AuthenticationType>local</AuthenticationType>
<alwaysSendBackAuthenticatedListOfIdPs>false</alwaysSendBackAuthenticatedListOfIdPs>
<UseTenantDomainInUsername>false</UseTenantDomainInUsername>
<UseUserstoreDomainInRoles>true</UseUserstoreDomainInRoles>
<UseUserstoreDomainInUsername>false</UseUserstoreDomainInUsername>
<SkipConsent>false</SkipConsent>
<skipLogoutConsent>false</skipLogoutConsent>
<EnableAuthorization>false</EnableAuthorization>
</LocalAndOutBoundAuthenticationConfig>
<RequestPathAuthenticatorConfigs/>
<InboundProvisioningConfig>
<ProvisioningUserStore/>
<IsProvisioningEnabled>false</IsProvisioningEnabled>
<IsDumbModeEnabled>false</IsDumbModeEnabled>
</InboundProvisioningConfig>
<OutboundProvisioningConfig>
<ProvisioningIdentityProviders/>
</OutboundProvisioningConfig>
<ClaimConfig>
<RoleClaimURI/>
<LocalClaimDialect>true</LocalClaimDialect>
<IdpClaim/>
<ClaimMappings/>
<AlwaysSendMappedLocalSubjectId>false</AlwaysSendMappedLocalSubjectId>
<SPClaimDialects/>
</ClaimConfig>
<PermissionAndRoleConfig>
<Permissions/>
<RoleMappings/>
<IdpRoles/>
</PermissionAndRoleConfig>
<IsSaaSApp>true</IsSaaSApp>
<ImageUrl/>
<AccessUrl/>
<IsDiscoverable>true</IsDiscoverable>
</ServiceProvider>
When I attempt to upload the file manually via the management console of WSO2 IDS is get an error that the application already exists.
When I boot the IDS without the service provider flow and upload it manually the authentication request is working.
The version of WSO2 IDS on which this occurs is 5.10.

Answer 1

WSO2 IS does not support adding the OAuth application configuration through file inside /repository/conf/identity/service-providers/. Because for the oAuth application, we need the entries in the database to manage the tokens issued for the applications. So file-based storage will not work for OAuth applications.
When I attempt to upload the file manually via the management console
of WSO2 IDS is get an error that the application already exists.
This is kind of expected, even though WSO2 IS does not support OAuth applications from file-based configuration. Having the file in /repository/conf/identity/service-providers/ will be considered as an application in the system (because WSO2 IS support multiple inbound protocols for same application - SAML or OAuth)

Answer 2

Answering my own question with the approach we took. Might be of benefit to others.
Since we are deploying the WSO2 identity server with Helm into a kubernetes environment we decided to create a job which inserts the service provider via the WSO2 management API. The created job look as follows:
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .Release.Name }}-identityserver.service-provider-fuga
labels:
system: identity-service
spec:
template:
spec:
restartPolicy: OnFailure
initContainers:
- name: wait-for-ids
image: groundnuty/k8s-wait-for:v1.3
args:
- "pod"
- "-ldeployment=identityserver"
containers:
- name: import-service-provider-fuga
image: curlimages/curl:7.72.0
args:
- /bin/sh
- -ec
- "curl --location --request POST 'http://{{ .Release.Name }}-identityserver-service:9763/api/server/v1/applications/import' --header 'Authorization: Basic YWRtaW46c3VwZXJTZWNyZXQ=' --form 'file=#\"/service-provider.xml\"'"
volumeMounts:
- name: identity-server-conf
mountPath: /service-provider.xml
subPath: service-provider.xml
volumes:
- name: identity-server-conf
configMap:
name: {{ .Release.Name }}-identityserver.cm
The wait-for-ids container makes the job wait until all IDS pods are running. The central part of the job calls the IDS management API to import the service provider. The service provider xml file is stored within a configuration map.

Answer 3

We have eventually added the service provider via a kubernetes job after the IDS has been booted. The job looks as follows:
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .Release.Name }}-identityserver.service-provider-fuga
labels:
system: identity-service
spec:
template:
spec:
restartPolicy: OnFailure
initContainers:
- name: wait-for-ids
image: groundnuty/k8s-wait-for:v1.3
args:
- "pod"
- "-ldeployment=identityserver"
containers:
- name: import-service-provider-fuga
image: curlimages/curl:7.72.0
args:
- /bin/sh
- -ec
- "curl --location --request POST 'http://{{ .Release.Name }}-identityserver-service:9763/api/server/v1/applications/import' --header 'Authorization: Basic {{ (printf "%s:%s" .Values.identityserver.management.admin.user .Values.identityserver.management.admin.password) | b64enc }}' --form 'file=#\"/service-provider-fuga.xml\"'"
volumeMounts:
- name: identity-server-conf
mountPath: /service-provider-fuga.xml
subPath: service-provider-fuga.xml
volumes:
- name: identity-server-conf
configMap:
name: {{ .Release.Name }}-identityserver.cm
The configuration of the service provider is shared via a configuration map.