WSO2 APIM connecting to backend service secured with QWAC certificate - wso2

Question

I am implementing an API gateway for a backend service which requires QWAC certificate.
I followed the instructions at: https://apim.docs.wso2.com/en/3.1.0/administer/product-security/mutual-ssl-between-api-gateway-and-backend/ and imported the public key to client keystore in WSO APIM.
When I try to reach the endpoint in question I am getting the following error response:
{"errorCode":"bad_request","errorText":"400 - {\"status\":\"INVALID\",\"errorCode\":\"unspecified_error\",\"errorText\":\"Mapping error\"}"}, which seems to be coming from the
backend service.
Here is the output from wso2carbon wire logs:
2 Message direction=IN Server name=localhost Timestamp=1587116916556 Service name=__SynapseService Operation Name=mediate
TID: [-1] [] [2020-04-17 11:48:36,823] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "GET /api/slsp/sandbox/v1/psd2-ais/v1/accounts HTTP/1.1[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:36,890] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "Authorization: ewogICJ0eXBlIjogInRva2VuIiwKICAibmFtZSI6ICJTTFNQIGNsaWVudDEiLAogICJzZXNzaW9uVVVJRCI6ICIyMzI1YzFkMS01ZTMwLTQ2NGQtOGM0Ni1kYzc5Y2E2NTkzMDAiLAogICJzY29wZXMiOiBbXSwKICAiY29uc2VudCI6IFsKICAgIHsKICAgICAgImlkIjogIjExMTExIiwKICAgICAgImNvbnRlbnQiOiAibm9uZSIKICAgIH0KICBdLAogICJsaW1pdHMiOiB7CiAgICAiYWNjZXNzU2Vjb25kcyI6IDM2MDAsCiAgICAicmVmcmVzaFNlY29uZHMiOiA3Nzc2MDAwCiAgfSwKICAiYWNjZXNzVHlwZSI6ICJudWxsIiwKICAiZXhwaXJhdGlvbiI6ICIyMDIwLTA0LTE3VDA5OjUxOjI2LjQ1MVoiCn0=[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:36,954] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "activityID: 490325399145411914682[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,017] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "web-api-key: b5830b00-772f-4e94-8a4a-be370d4e5481[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,082] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "accept: application/json[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,145] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "Host: webapi.developers.erstegroup.com[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,208] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "Connection: Keep-Alive[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,273] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,336] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 << "[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,642] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "HTTP/1.1 400 [\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,706] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Date: Fri, 17 Apr 2020 09:48:37 GMT[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,771] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Server: Apache[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,835] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Strict-Transport-Security: max-age=31536000; includeSubDomains[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,900] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "cz-transactionId: 197173439577254[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:37,966] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Content-Type: application/json;charset=utf-8[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:38,031] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Content-Length: 140[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:38,095] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Set-Cookie: 48f65e4d401373b3b03cb2a02b953e21=425c12b91ee874d67b6799357c467562; path=/; HttpOnly; Secure[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:38,158] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "Connection: close[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:38,221] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "[\r][\n]"
TID: [-1] [] [2020-04-17 11:48:38,286] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Sender I/O dispatcher-2 >> "{"errorCode":"bad_request","errorText":"400 - {\"status\":\"INVALID\",\"errorCode\":\"unspecified_error\",\"errorText\":\"Mapping error\"}"}"
I have tried to reach the same service with Postman, after I imported the client certificate in postman , the service was responding without errors.
So it looks like the isssue is not with the certificate itself, as the SSL connection was established with the backend server, but what could have gone wrong? (When the OAuth2.0 token expires I get the following error "OAUTH2 failed to TOKEN_INFO with response: {\\"active\\":false}", which is the same as what I get with Postman.)
Here is the swagger spec from WSO2 APIM:
paths:
/accounts:
get:
parameters:
-
name: "withBalance"
in: "query"
required: false
style: "form"
explode: true
schema:
type: "string"
-
name: "web-api-key"
in: "query"
required: true
style: "form"
explode: true
schema:
type: "string"
-
name: "access_token"
in: "query"
required: true
style: "form"
explode: true
schema:
type: "string"
responses:
200:
description: "ok"
security:
-
default: []
x-auth-type: "None"
x-throttling-tier: "Unlimited"
components:
securitySchemes:
default:
type: "oauth2"
flows:
implicit:
authorizationUrl: "https://test.com"
scopes: {}
x-wso2-auth-header: "Authorization"
x-throttling-tier: "Unlimited"
x-wso2-cors:
corsConfigurationEnabled: false
accessControlAllowOrigins:
- "*"
accessControlAllowCredentials: false
accessControlAllowHeaders:
- "authorization"
- "Access-Control-Allow-Origin"
- "Content-Type"
- "SOAPAction"
accessControlAllowMethods:
- "GET"
- "PUT"
- "POST"
- "DELETE"
- "PATCH"
- "OPTIONS"
x-wso2-sandbox-endpoints:
urls:
- "https://webapi.developers.erstegroup.com/api/slsp/sandbox/v1/psd2-ais/v1"
type: "http"
x-wso2-basePath: "/slsp_ais/1.0"
x-wso2-transports:
- "http"
I tried to pass the 2 mandatory parameters in HTTP headers as well, but I get the same results:
curl -X GET "http://localhost:8280/slsp_ais/1.0/accounts" -H "accept: application/json" -H "web-api-key: b5830b00-772f-4e94-8a4a-be370d4e5481" -H "Authorization: Bearer ewogICJ0eXBlIjogInRva2VuIiwKICAibmFtZSI6ICJTTFNQIGNsaWVudDEiLAogICJzZXNzaW9uVVVJRCI6ICIyMzI1YzFkMS01ZTMwLTQ2NGQtOGM0Ni1kYzc5Y2E2NTkzMDAiLAogICJzY29wZXMiOiBbXSwKICAiY29uc2VudCI6IFsKICAgIHsKICAgICAgImlkIjogIjExMTExIiwKICAgICAgImNvbnRlbnQiOiAibm9uZSIKICAgIH0KICBdLAogICJsaW1pdHMiOiB7CiAgICAiYWNjZXNzU2Vjb25kcyI6IDM2MDAsCiAgICAicmVmcmVzaFNlY29uZHMiOiA3Nzc2MDAwCiAgfSwKICAiYWNjZXNzVHlwZSI6ICJudWxsIiwKICAiZXhwaXJhdGlvbiI6ICIyMDIwLTA0LTE3VDA5OjUxOjI2LjQ1MVoiCn0=" -H "apikey: eyJ4NXQiOiJaalJtWVRNd05USmpPV1U1TW1Jek1qZ3pOREkzWTJJeU1tSXlZMkV6TWpkaFpqVmlNamMwWmc9PSIsImtpZCI6ImdhdGV3YXlfY2VydGlmaWNhdGVfYWxpYXMiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImFwcGxpY2F0aW9uIjp7Im93bmVyIjoiYWRtaW4iLCJ0aWVyIjoiVW5saW1pdGVkIiwibmFtZSI6IlRlc3RfYXBwIiwiaWQiOjIsInV1aWQiOiIyNDUxZWMyNy02ZDllLTRjN2YtODcwYi0xOWIxZGNiMDI2MzQifSwidGllckluZm8iOnsiVW5saW1pdGVkIjp7InN0b3BPblF1b3RhUmVhY2giOnRydWUsInNwaWtlQXJyZXN0TGltaXQiOjAsInNwaWtlQXJyZXN0VW5pdCI6bnVsbH19LCJrZXl0eXBlIjoiU0FOREJPWCIsInN1YnNjcmliZWRBUElzIjpbeyJzdWJzY3JpYmVyVGVuYW50RG9tYWluIjoiY2FyYm9uLnN1cGVyIiwibmFtZSI6IkFQSU9CIiwiY29udGV4dCI6Ilwvb3Blbi1iYW5raW5nXC92My4xXC9haXNwXC92My4xLjUiLCJwdWJsaXNoZXIiOiJhZG1pbiIsInZlcnNpb24iOiJ2My4xLjUiLCJzdWJzY3JpcHRpb25UaWVyIjoiVW5saW1pdGVkIn0seyJzdWJzY3JpYmVyVGVuYW50RG9tYWluIjoiY2FyYm9uLnN1cGVyIiwibmFtZSI6Ik1vY2tUYXJnZXRBUEkiLCJjb250ZXh0IjoiXC9BcGlnZWVfTW9ja19UYXJnZXRcLzEuMC4wIiwicHVibGlzaGVyIjoiYWRtaW4iLCJ2ZXJzaW9uIjoiMS4wLjAiLCJzdWJzY3JpcHRpb25UaWVyIjoiVW5saW1pdGVkIn0seyJzdWJzY3JpYmVyVGVuYW50RG9tYWluIjoiY2FyYm9uLnN1cGVyIiwibmFtZSI6IlNMU1BfUFNEMl9BSVMiLCJjb250ZXh0IjoiXC9zbHNwX2Fpc1wvMS4wIiwicHVibGlzaGVyIjoiYWRtaW4iLCJ2ZXJzaW9uIjoiMS4wIiwic3Vic2NyaXB0aW9uVGllciI6IlVubGltaXRlZCJ9XSwiaWF0IjoxNTg3MTEzMTgzLCJqdGkiOiI0ZmFhYTQ4Ny0yODNjLTQ3ZjMtYTdlNi05OGQ1YmI2NjFmN2YifQ==.QJ8-ODdRueTtDKDfWYVFeI3I6YJGfCtRGIg64nGdewQP9jW8KzyFLmkt14i7OGXkKpA4e2Yowa9lidxN0qrdRmUjJLKpZmBOn6TjN5auE8TcvxyeSlOigK0N-J-eLB6DuHnqg6Rf918d2oJS2bJBmqbzqs0BPMuEj5Y9ImS7F1CdMcRaDTOYt6G-GxmwpScU4dlxOrxZGu8uD5Nnz2SHikXSqGcrF-KLmNUFJuFKTitEMEaHz8N9M-MYsTDlOnvu0BeEFiW60NRCPumzCOzs5wL7dMTcCXOGd40-OKcUkS2KpH-YEh7cl0ALz9wi0vgFRqN0V2CAndbCUwppmkzo9w=="
{"errorCode":"bad_request","errorText":"400 - {\"status\":\"INVALID\",\"errorCode\":\"unspecified_error\",\"errorText\":\"Mapping error\"}"}
I also intercepted the working Postman request via Burp:
GET /api/slsp/sandbox/v1/psd2-ais/v1/accounts?web-api-key=b5830b00-772f-4e94-8a4a-be370d4e5481&access_token=ewogICJ0eXBlIjogInRva2VuIiwKICAibmFtZSI6ICJTTFNQIGNsaWVudDEiLAogICJzZXNzaW9uVVVJRCI6ICIyMzI1YzFkMS01ZTMwLTQ2NGQtOGM0Ni1kYzc5Y2E2NTkzMDAiLAogICJzY29wZXMiOiBbXSwKICAiY29uc2VudCI6IFsKICAgIHsKICAgICAgImlkIjogIjExMTExIiwKICAgICAgImNvbnRlbnQiOiAibm9uZSIKICAgIH0KICBdLAogICJsaW1pdHMiOiB7CiAgICAiYWNjZXNzU2Vjb25kcyI6IDM2MDAsCiAgICAicmVmcmVzaFNlY29uZHMiOiA3Nzc2MDAwCiAgfSwKICAiYWNjZXNzVHlwZSI6ICJudWxsIiwKICAiZXhwaXJhdGlvbiI6ICIyMDIwLTA0LTE3VDA5OjUxOjI2LjQ1MVoiCn0= HTTP/1.1
User-Agent: PostmanRuntime/7.24.1
Accept: */*
Cache-Control: no-cache
Postman-Token: b925ae09-0b5b-440f-a1e9-98bc5f79b043
Host: webapi.developers.erstegroup.com:443
Accept-Encoding: gzip, deflate
Connection: close
Here is the whole thing via Postman console:
GET /api/slsp/sandbox/v1/psd2-ais/v1/accounts?web-api-key=b5830b00-772f-4e94-8a4a-be370d4e5481&access_token=ewogICJ0eXBlIjogInRva2VuIiwKICAibmFtZSI6ICJTTFNQIGNsaWVudDEiLAogICJzZXNzaW9uVVVJRCI6ICI4MWJlZDMwMS1lMGFkLTQwMzAtODMxMC0wNThmZDViYWIyMDkiLAogICJzY29wZXMiOiBbXSwKICAiY29uc2VudCI6IFsKICAgIHsKICAgICAgImlkIjogIjExMTExIiwKICAgICAgImNvbnRlbnQiOiAibm9uZSIKICAgIH0KICBdLAogICJsaW1pdHMiOiB7CiAgICAiYWNjZXNzU2Vjb25kcyI6IDM2MDAsCiAgICAicmVmcmVzaFNlY29uZHMiOiA3Nzc2MDAwCiAgfSwKICAiYWNjZXNzVHlwZSI6ICJudWxsIiwKICAiZXhwaXJhdGlvbiI6ICIyMDIwLTA0LTE3VDExOjU0OjQ5LjA4OFoiCn0%3D HTTP/1.1
User-Agent: PostmanRuntime/7.24.1
Accept: */*
Cache-Control: no-cache
Postman-Token: fc30b165-7571-4efe-96fe-e23b1cf1c20e
Host: webapi.developers.erstegroup.com:443
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
HTTP/1.1 200 OK
Date: Fri, 17 Apr 2020 10:55:37 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
accept: */*
Access-Control-Allow-Origin: *
correlation-id: 6b27116c-15e6-4410-8ff7-87afd9bbd92b
forwarded: for=10.198.136.200;host=webapi.prod.eapihub.microp.cs.eb.lan.at;proto=https;proto-version=
ip-address: 178.41.84.88
origin-transaction-id: 185078296373260
postman-token: fc30b165-7571-4efe-96fe-e23b1cf1c20e
TPP-QWAC-Body: MIIFSTCCAzGgAwIBAgIEAQIDBDANBgkqhkiG9w0BAQsFADCBgDELMAkGA1UEBhMCQ1oxDjAMBgNVBAcTBUN6ZWNoMRMwEQYDVQQKEwpFcnN0ZUdyb3VwMRUwEwYDVQQLEwxFcnN0ZUh1YlRlYW0xETAPBgNVBAMTCEVyc3RlSHViMSIwIAYJKoZIhvcNAQkBFhNpbmZvQGVyc3RlZ3JvdXAuY29tMB4XDTE5MDkxNzA4NTA1MFoXDTIyMTIxNzA4NTA1MFowUjEaMBgGA1UEYRMRUFNEQ1otQ05CLTEyMzQ1NjcxCzAJBgNVBAYTAkNaMRYwFAYDVQQDEw1UUFAgVGVzdCBRV0FDMQ8wDQYDVQQKEwZNeSBUUFAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDJZunh6Nj5aBtbhJJ9eaSDjDiiERBOZfVrmEwz9Ea5ldLAf8NUy+t71etzGeHtCKyTuSlhj1Clhla+iG/r1uz25H3T6wUQbmw1+pFsaSovkamQaKy+2GJSPCx66li6z2JBv0I66DtoGl6QXcy5JO2sBjZaO4m11bcFFVkvo9lh2QCC2x68w8bHeBuPUMnU6rupVQfgPPWMH+WqqaPgxoQr5KmQ03ItY2/TBqoX6xTTbL6+B8OMbX41Lxah+g+5XPOlDoC64HiBO2T+FL62W+51ehUCORuuUt6/AYzXcCHSu1FXsk25KeObe9Na2AfobGNL83I68Bl0K2WtjbEtHs1FAgMBAAGjgfcwgfQwCwYDVR0PBAQDAgHGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjCBrwYIKwYBBQUHAQMEgaIwgZ8wCAYGBACORgEBMAsGBgQAjkYBAwIBFDAIBgYEAI5GAQQwEwYGBACORgEGMAkGBwQAjkYBBgMwZwYGBACBmCcCMF0wTDARBgcEAIGYJwEBDAZQU1BfQVMwEQYHBACBmCcBAgwGUFNQX1BJMBEGBwQAgZgnAQMMBlBTUF9BSTARBgcEAIGYJwEEDAZQU1BfSUMMBUVyc3RlDAZBVC1FUlMwFAYDVR0RBA0wC4IJbXl0cHAuY29tMA0GCSqGSIb3DQEBCwUAA4ICAQCRs+caFpc9s1v1bK1kTQI0aA7lbyuzcd+zMkDOVvyN1h6d6BCv41XksB25tgzn8WeUl+2p08rxQJzobjm+e210fgqDC8KVjbnzzbsut11aFjIZN8qgs5ZBQrujABXGCSOyo4R+AxTpSBmM++gdrR+Kp4H+BV3bpFjtGfrzyN2pfPXj5M6t/CAMk6mAmHzZT7MXSRMltM+xbEkMniUW856lLo1YxPBF8zEIg8bicTyV2fJkDCGxnaJpJ2+vSt9s8WDB4qw5/hRKDcpT/TyZ6136SldlsbsMNYAC0Vd//2CSeT5yPG3970skf797zTtLNWtWjZUZtXaHHCfJDkVCMjFSEw/+LKDhmorQmv+4ZYgKVUDMCcjjaAOPF1Gwq2U57Jg96E3gBY0h/jMhzQPCDQfR7jBvYRYtpaykNBNXsITCvrXYn88FKEJK9qqLmAifFB/QF+EkXN2PUgl+VXC4ly2eHj6JTI7Xy0MbiF/m861mRQ1i6YOsB+GG3X1gPCMxbyXMYwDB9/22JftRjo2d9h/sNjpXdowJokFNLXAzq4shmasUvaIE+BpGScVuO0/3gnMW06MbSB2kSR1+m7r4pFs/ND8/WcckteSrKXD94b8feClzA5H83t+uI0EOXzO7qpbZOkAHPfioA1F4P4JewDJ9HpKu96VO5uZUN6M0DK4dnQ==
transaction-id: 185078296373260
web-api-correlation-id: 6b27116c-15e6-4410-8ff7-87afd9bbd92b
web-api-transaction-id: 185078296373260
x-forwarded-for: 178.41.84.88, 178.41.84.88
x-forwarded-host: webapi.prod.eapihub.microp.cs.eb.lan.at
x-forwarded-port: 443
x-forwarded-proto: https
x-forwarded-server: webapi.developers.erstegroup.com
X-Traits: TPP_ONLY;PSD2_QWAC;DELEGATE_QSEAL_VALIDATION
x-webapi-client-ip: 178.41.84.88
x-webapi-message-id: 185078296373260
Content-Type: application/json;charset=utf-8
Vary: Accept-Encoding
Content-Encoding: br
Content-Length: 276
Keep-Alive: timeout=60, max=99
Connection: Keep-Alive
{"accounts":[{"resourceId":"CCA4F9863D686D04","iban":"SK5409000000005037706253","currency":"EUR","name":"Mag. A. M. Tester","cashAccountType":"CACC","status":"enabled","bic":"GIBASKBX","_links":{"detail":{"href":"/v1/accounts/CCA4F9863D686D04"},"balances":{"href":"/v1/accounts/CCA4F9863D686D04/balances"},"transactions":{"href":"/v1/psd2-ais/v1/transactions"}}},{"resourceId":"AF500F1000071A0A0","iban":"SK0209000000005037645497","currency":"USD","name":"Adam Tester","cashAccountType":"CACC","status":"enabled","bic":"GIBASKBX","_links":{"balances":{"href":"/v1/accounts/AF500F1000071A0A0/balances"},"transactions":{"href":"/v1/accounts/AF500F1000071A0A0/transactions"}}}]}
I would appreciate any feedback. Thank you for your help.

Answer 1

In the keystore in WSO2-AM side you need to import the private key, not just the certifiate
Refer to the sample at https://apim.docs.wso2.com/en/3.1.0/administer/product-security/mutual-ssl-between-api-gateway-and-backend/
<!-- For Mutual SSL Handshake configure both trust store and key store-->
<profile>
<servers>10.100.5.130:9444</servers>
<TrustStore>
<Location>repository/resources/security/client-truststore.jks
</Location>
<Type>JKS</Type>
<Password>wso2carbon</Password>
</TrustStore>
<KeyStore>
<Location>repository/resources/security/wso2carbon.jks</Location>
<Type>JKS</Type>
<Password>xxxxxx</Password>
<KeyPassword>xxxxxx</KeyPassword>
</KeyStore>
</profile>
</parameter>
In the keystore file repository/resources/security/wso2carbon.jks you need to have private key of your client certificate.