SageMaker multimodel and RandomCutForest - amazon-web-services

Question

I am trying to invoke a MultiModel Endpoint with a RandomCutForest Model. I receive error though, 'Error loading model'. I can invoke the endpoint with models given from the examples.
Am I missing something e.g. limitations on what models I can use?
For MultiModel inspiration I am using below:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/multi_model_xgboost_home_value/xgboost_multi_model_endpoint_home_value.ipynb
https://aws.amazon.com/blogs/machine-learning/save-on-inference-costs-by-using-amazon-sagemaker-multi-model-endpoints/
I am trying to deploy the outputted 'model.tar.gz' from below RCF example in the MultiModel endpoint:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/random_cut_forest/random_cut_forest.ipynb
model_name = 'model'
full_model_name = '{}.tar.gz'.format(model_name)
features = data
body = ','.join(map(str, features)) + '\n'
response = runtime_sm_client.invoke_endpoint(
EndpointName=endpoint_name,
ContentType='text/csv',
TargetModel=full_model_name,
Body=body)
print(response)
Cloudwatch log Error:
> 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - Error loading model: Unable
> to load model: invalid load key, '{'. [17:28:59]
> /workspace/src/learner.cc:334: Check failed: fi->Read(&mparam_,
> sizeof(mparam_)) == sizeof(mparam_) (25 vs. 136) : BoostLearner: wrong
> model format 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - Stack trace: 2020-04-27
> 17:28:59,005 [INFO ] W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (0)
> /miniconda3/xgboost/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x24)
> [0x7f37ce1cacb4] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9 com.amazonaws.ml.mms.wlm.WorkerThread
> - Backend response time: 0 2020-04-27 17:28:59,005 [INFO ] W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (1)
> /miniconda3/xgboost/libxgboost.so(xgboost::LearnerImpl::Load(dmlc::Stream*)+0x4b5)
> [0x7f37ce266985] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (2)
> /miniconda3/xgboost/libxgboost.so(XGBoosterLoadModel+0x37)
> [0x7f37ce1bf417] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (3)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c)
> [0x7f37ee993ec0] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (4)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call+0x22d)
> [0x7f37ee99387d] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (5)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
> [0x7f37eeba91de] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (6)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0x12c14)
> [0x7f37eeba9c14] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (7)
> /miniconda3/bin/python(_PyObject_FastCallKeywords+0x48b)
> [0x563d71b4218b] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (8)
> /miniconda3/bin/python(_PyEval_EvalFrameDefault+0x52cf)
> [0x563d71b91e8f] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 2020-04-27 17:28:59,005
> [WARN ] W-9003-b39b888fb4a3fa6cf83bb34a9
> com.amazonaws.ml.mms.wlm.WorkerThread - Backend worker thread
> exception. java.lang.IllegalArgumentException: reasonPhrase contains
> one of the following prohibited characters: \r\n: Unable to load
> model: Unable to load model: invalid load key, '{'. [17:28:59]
> /workspace/src/learner.cc:334: Check failed: fi->Read(&mparam_,
> sizeof(mparam_)) == sizeof(mparam_) (25 vs. 136) : BoostLearner: wrong
> model format Stack trace: [bt] (0)
> /miniconda3/xgboost/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x24)
> [0x7f37ce1cacb4] [bt] (1)
> /miniconda3/xgboost/libxgboost.so(xgboost::LearnerImpl::Load(dmlc::Stream*)+0x4b5)
> [0x7f37ce266985] [bt] (2)
> /miniconda3/xgboost/libxgboost.so(XGBoosterLoadModel+0x37)
> [0x7f37ce1bf417] [bt] (3)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c)
> [0x7f37ee993ec0] [bt] (4)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call+0x22d)
> [0x7f37ee99387d] [bt] (5)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
> [0x7f37eeba91de] [bt] (6)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0x12c14)
> [0x7f37eeba9c14] [bt] (7)
> /miniconda3/bin/python(_PyObject_FastCallKeywords+0x48b)
> [0x563d71b4218b] [bt] (8)
> /miniconda3/bin/python(_PyEval_EvalFrameDefault+0x52cf)
> [0x563d71b91e8f]

Answer 1

SageMaker Random Cut Forest is part of the built-in algorithm library and cannot be deployed in multi-model endpoint (MME). Built-in algorithms currently cannot be deployed to MME. XGboost is an exception, since it has an open-source container https://github.com/aws/sagemaker-xgboost-container.
If you really need to deploy a RCF to a multi-model endpoint, one option is to find a reasonably similar open-source implementation (for example rrcf looks reasonably serious: based on the same paper from Guha et al and with 170+ github stars) and create a custom MME docker container. The documentation is here and there is an excellent tuto here