WSO2 API Manager Config issue with Surf OAuth as Third Party KeyManager - wso2

Question

I am trying to use Surf OAuth as third party key manager with WSO2 API Manager. Following the documentation at https://docs.wso2.com/display/AM1100/Configuring+a+Third-Party+Key+Manager
My issue is that when I add the configuration for Surf OAuth in api-manager.xml the WSO2 API Manager blows up with NPE while trying to read config. It looks like its blowing up on some system property such as CARBON_HOME and throwing NPE.
Has anybody else faced similar issue with WSO2 API Manager and Surf OAuth integration? I am intrigued why its not working even after following step by step direction
Here is my api-manager.xml config
<APIKeyManager>
<KeyManagerClientImpl>nl.surfnet.demo.SurfOAuthClient</KeyManagerClientImpl>
<Configuration>
<RegistrationEndpoint>http://myserver:8080/admin/resourceServer/101</RegistrationEndpoint>
<AccessToken>4a3aa6a5-8f37-4267-8911-0e3049ccee86</AccessToken>
<IntrospectionURL>http://myserver:8080/v1/tokeninfo</IntrospectionURL>
<ConsumerKey>1ff100f0-3e10-43cc-af2f-129d6a638739</ConsumerKey>
<ConsumerSecret>3a79061f-4fa9-4d4e-bb6b-3af38e6a6733</ConsumerSecret>
</Configuration>
</APIKeyManager>
And the stack trace
[2016-03-22 16:07:36,381] ERROR - APIKeyMgtServiceComponent Failed to initialize key management service.
org.wso2.carbon.apimgt.api.APIManagementException: Unexpected error occurred while parsing configuration: /opt/wso2am-1.10.0/repository/conf/api-manager.xml
at org.wso2.carbon.apimgt.impl.APIManagerConfiguration.load(APIManagerConfiguration.java:112)
at org.wso2.carbon.apimgt.keymgt.internal.APIKeyMgtServiceComponent.activate(APIKeyMgtServiceComponent.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.identity.thrift.authentication.internal.ThriftAuthenticationServiceComponent.activate(ThriftAuthenticationServiceComponent.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:451)
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:514)
at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.registerService(BundleContextImpl.java:433)
at org.eclipse.equinox.http.servlet.internal.Activator.registerHttpService(Activator.java:81)
at org.eclipse.equinox.http.servlet.internal.Activator.addProxyServlet(Activator.java:60)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.init(ProxyServlet.java:40)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.init(DelegationServlet.java:38)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1284)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1197)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1087)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5262)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5550)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.wso2.carbon.apimgt.impl.utils.APIUtil.replaceSystemProperty(APIUtil.java:3946)
at org.wso2.carbon.apimgt.impl.APIManagerConfiguration.readChildElements(APIManagerConfiguration.java:160)
at org.wso2.carbon.apimgt.impl.APIManagerConfiguration.readChildElements(APIManagerConfiguration.java:265)
at org.wso2.carbon.apimgt.impl.APIManagerConfiguration.load(APIManagerConfiguration.java:92)
... 83 more

Answer 1

The Surf-oauth client implementation which is provided in the documentation is not supported with the APIM latest version(1.10.0). Therefore you need to do the following changes in order to fix the issue.
Change the SurfOAuthClient.java file to import org.wso2.carbon.apimgt.impl.AbstractKeyManager class instead of org.wso2.carbon.apimgt.keymgt.AbstractKeyManager class.
import org.wso2.carbon.apimgt.impl.AbstractKeyManager;
Change the version of "org.wso2.carbon.apimgt" group ids to 5.0.3 in the pom.xml file as follows.
<dependency>
<groupId>org.wso2.carbon.apimgt</groupId>
<artifactId>org.wso2.carbon.apimgt.impl</artifactId>
<version>5.0.3</version>
</dependency>
<dependency>
<groupId>org.wso2.carbon.apimgt</groupId>
<artifactId>org.wso2.carbon.apimgt.gateway</artifactId>
<version>5.0.3</version>
</dependency>
<dependency>
<groupId>org.wso2.carbon.apimgt</groupId>
<artifactId>org.wso2.carbon.apimgt.keymgt</artifactId>
<version>5.0.3</version>
</dependency>
<dependency>
<groupId>org.wso2.carbon.apimgt</groupId>
<artifactId>org.wso2.carbon.apimgt.hostobjects</artifactId>
<version>5.0.3</version>
</dependency>
Also please refer [1] for the fixed code. Build the code, generate the war file and jar file.
Remove the current gateway.client-1.0.0.jar file from /repository/components/lib and /repository/components/dropins folders and copy the generated jar file to /repository/components/lib folder.
Use the generated war file and do the configurations again.
[1] https://github.com/jaadds/surf-oauth-demo/tree/v1.10.0