Azure Webjob: Could not load 'System.Net.Http' - azure-webjobs

I am getting the following error:
[04/27/2017 14:18:56 > 2dcb9f: INFO] Warning: Only got partial types from assembly: WebJob2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
[04/27/2017 14:18:56 > 2dcb9f: INFO] Exception message: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.RuntimeModule.GetTypes()
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.Assembly.GetTypes()
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.DefaultTypeLocator.FindTypes(Assembly assembly, IEnumerable`1 extensionAssemblies)
[04/27/2017 14:18:56 > 2dcb9f: INFO] Warning: Only got partial types from assembly: WebJob2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
[04/27/2017 14:18:56 > 2dcb9f: INFO] Exception message: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.RuntimeModule.GetTypes()
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.Assembly.GetTypes()
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.DefaultTypeLocator.FindTypes(Assembly assembly, IEnumerable`1 extensionAssemblies)
[04/27/2017 14:18:56 > 2dcb9f: INFO] Error indexing method 'Functions.ProcessCopyProjectQueueMessage'
[04/27/2017 14:18:56 > 2dcb9f: INFO] Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexingException: Error indexing method 'Functions.ProcessCopyProjectQueueMessage' ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
[04/27/2017 14:18:56 > 2dcb9f: INFO] --- End of inner exception stack trace ---
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.CustomAttribute._CreateCaObject(RuntimeModule pModule, IRuntimeMethodInfo pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.CustomAttribute.CreateCaObject(RuntimeModule module, IRuntimeMethodInfo ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.RuntimeMethodInfo.GetCustomAttributes(Type attributeType, Boolean inherit)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Attribute.GetCustomAttributes(MemberInfo element, Type type, Boolean inherit)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Attribute.GetCustomAttribute(MemberInfo element, Type attributeType, Boolean inherit)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](MemberInfo element)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.TypeUtility.IsAsync(MethodInfo methodInfo)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.TypeUtility.IsAsyncVoid(MethodInfo methodInfo)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsyncCore>d__14.MoveNext()
[04/27/2017 14:18:56 > 2dcb9f: INFO] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsync>d__13.MoveNext()
[04/27/2017 14:18:56 > 2dcb9f: INFO] --- End of inner exception stack trace ---
[04/27/2017 14:18:56 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsync>d__13.MoveNext()
[04/27/2017 14:18:56 > 2dcb9f: INFO] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:56 > 2dcb9f: INFO] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: INFO] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: INFO] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexTypeAsync>d__9.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ]
[04/27/2017 14:18:57 > 2dcb9f: ERR ] Unhandled Exception: Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexingException: Error indexing method 'Functions.ProcessCopyProjectQueueMessage' ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.Net.Http, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of inner exception stack trace ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.CustomAttribute._CreateCaObject(RuntimeModule pModule, IRuntimeMethodInfo pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.CustomAttribute.CreateCaObject(RuntimeModule module, IRuntimeMethodInfo ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeMethodInfo method, RuntimeType caType, Boolean inherit)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.RuntimeMethodInfo.GetCustomAttributes(Type attributeType, Boolean inherit)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Attribute.GetCustomAttributes(MemberInfo element, Type type, Boolean inherit)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Attribute.GetCustomAttribute(MemberInfo element, Type attributeType, Boolean inherit)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Reflection.CustomAttributeExtensions.GetCustomAttribute[T](MemberInfo element)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.TypeUtility.IsAsync(MethodInfo methodInfo)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.TypeUtility.IsAsyncVoid(MethodInfo methodInfo)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsyncCore>d__14.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsync>d__13.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of inner exception stack trace ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexMethodAsync>d__13.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.<IndexTypeAsync>d__9.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexProvider.<CreateAsync>d__11.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexProvider.<GetAsync>d__10.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Executors.JobHostContextFactory.<CreateAndLogHostStartedAsync>d__5.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.Host.Executors.JobHostContextFactory.<CreateAndLogHostStartedAsync>d__4.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.JobHost.<CreateContextAndLogHostStartedAsync>d__44.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.JobHost.<StartAsyncCore>d__27.MoveNext()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] --- End of stack trace from previous location where exception was thrown ---
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.JobHost.Start()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at Microsoft.Azure.WebJobs.JobHost.RunAndBlock()
[04/27/2017 14:18:57 > 2dcb9f: ERR ] at WebJob2.Program.Main() in d:\a\1\s\WebJob2\Program.cs:line 33
[04/27/2017 14:18:57 > 2dcb9f: SYS ERR ] Job failed due to exit code -532462766
[04/27/2017 14:18:57 > 2dcb9f: SYS INFO] Process went down, waiting for 60 seconds
[04/27/2017 14:18:57 > 2dcb9f: SYS INFO] Status changed to PendingRestart
[04/27/2017 14:18:59 > 2dcb9f: SYS INFO] Detected WebJob file/s were updated, refreshing WebJob
[04/27/2017 14:18:59 > 2dcb9f: SYS INFO] Status changed to Stopping
[04/27/2017 14:19:00 > 2dcb9f: SYS INFO] Status changed to Stopped
Package.config:
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="EntityFramework" version="6.1.3" targetFramework="net452" />
<package id="Microsoft.Azure.KeyVault.Core" version="2.0.4" targetFramework="net452" />
<package id="Microsoft.Azure.WebJobs" version="2.0.0" targetFramework="net452" />
<package id="Microsoft.Azure.WebJobs.Core" version="2.0.0" targetFramework="net452" />
<package id="Microsoft.Azure.WebJobs.Extensions" version="2.0.0" targetFramework="net452" />
<package id="Microsoft.Data.Edm" version="5.8.2" targetFramework="net452" />
<package id="Microsoft.Data.OData" version="5.8.2" targetFramework="net452" />
<package id="Microsoft.Data.Services.Client" version="5.8.2" targetFramework="net452" />
<package id="Microsoft.Tpl.Dataflow" version="4.5.24" targetFramework="net452" />
<package id="Microsoft.Web.WebJobs.Publish" version="1.0.13" targetFramework="net46" />
<package id="Microsoft.WindowsAzure.ConfigurationManager" version="3.2.3" targetFramework="net452" />
<package id="ncrontab" version="3.3.0" targetFramework="net452" />
<package id="Newtonsoft.Json" version="10.0.2" targetFramework="net452" />
<package id="System.ComponentModel.EventBasedAsync" version="4.3.0" targetFramework="net452" />
<package id="System.Dynamic.Runtime" version="4.3.0" targetFramework="net452" />
<package id="System.Linq.Queryable" version="4.3.0" targetFramework="net452" />
<package id="System.Net.Requests" version="4.3.0" targetFramework="net452" />
<package id="System.Spatial" version="5.8.2" targetFramework="net452" />
<package id="WindowsAzure.Storage" version="8.1.1" targetFramework="net452" />
</packages>
I even tried to set CopyLocal to True. However, none of these is working. I still getting the above error and webjobs is in a continuous PendingRestart state.
NOTE: The webjob doesn't even work locally. The target version is .Net 4.6.1

Related

SageMaker multimodel and RandomCutForest

I am trying to invoke a MultiModel Endpoint with a RandomCutForest Model. I receive error though, 'Error loading model'. I can invoke the endpoint with models given from the examples.
Am I missing something e.g. limitations on what models I can use?
For MultiModel inspiration I am using below:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/multi_model_xgboost_home_value/xgboost_multi_model_endpoint_home_value.ipynb
https://aws.amazon.com/blogs/machine-learning/save-on-inference-costs-by-using-amazon-sagemaker-multi-model-endpoints/
I am trying to deploy the outputted 'model.tar.gz' from below RCF example in the MultiModel endpoint:
https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/random_cut_forest/random_cut_forest.ipynb
model_name = 'model'
full_model_name = '{}.tar.gz'.format(model_name)
features = data
body = ','.join(map(str, features)) + '\n'
response = runtime_sm_client.invoke_endpoint(
EndpointName=endpoint_name,
ContentType='text/csv',
TargetModel=full_model_name,
Body=body)
print(response)
Cloudwatch log Error:
> 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - Error loading model: Unable
> to load model: invalid load key, '{'. [17:28:59]
> /workspace/src/learner.cc:334: Check failed: fi->Read(&mparam_,
> sizeof(mparam_)) == sizeof(mparam_) (25 vs. 136) : BoostLearner: wrong
> model format 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - Stack trace: 2020-04-27
> 17:28:59,005 [INFO ] W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (0)
> /miniconda3/xgboost/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x24)
> [0x7f37ce1cacb4] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9 com.amazonaws.ml.mms.wlm.WorkerThread
> - Backend response time: 0 2020-04-27 17:28:59,005 [INFO ] W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (1)
> /miniconda3/xgboost/libxgboost.so(xgboost::LearnerImpl::Load(dmlc::Stream*)+0x4b5)
> [0x7f37ce266985] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (2)
> /miniconda3/xgboost/libxgboost.so(XGBoosterLoadModel+0x37)
> [0x7f37ce1bf417] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (3)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c)
> [0x7f37ee993ec0] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (4)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call+0x22d)
> [0x7f37ee99387d] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (5)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
> [0x7f37eeba91de] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (6)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0x12c14)
> [0x7f37eeba9c14] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (7)
> /miniconda3/bin/python(_PyObject_FastCallKeywords+0x48b)
> [0x563d71b4218b] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - [bt] (8)
> /miniconda3/bin/python(_PyEval_EvalFrameDefault+0x52cf)
> [0x563d71b91e8f] 2020-04-27 17:28:59,005 [INFO ]
> W-9003-b39b888fb4a3fa6cf83bb34a9-stdout
> com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 2020-04-27 17:28:59,005
> [WARN ] W-9003-b39b888fb4a3fa6cf83bb34a9
> com.amazonaws.ml.mms.wlm.WorkerThread - Backend worker thread
> exception. java.lang.IllegalArgumentException: reasonPhrase contains
> one of the following prohibited characters: \r\n: Unable to load
> model: Unable to load model: invalid load key, '{'. [17:28:59]
> /workspace/src/learner.cc:334: Check failed: fi->Read(&mparam_,
> sizeof(mparam_)) == sizeof(mparam_) (25 vs. 136) : BoostLearner: wrong
> model format Stack trace: [bt] (0)
> /miniconda3/xgboost/libxgboost.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x24)
> [0x7f37ce1cacb4] [bt] (1)
> /miniconda3/xgboost/libxgboost.so(xgboost::LearnerImpl::Load(dmlc::Stream*)+0x4b5)
> [0x7f37ce266985] [bt] (2)
> /miniconda3/xgboost/libxgboost.so(XGBoosterLoadModel+0x37)
> [0x7f37ce1bf417] [bt] (3)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c)
> [0x7f37ee993ec0] [bt] (4)
> /miniconda3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call+0x22d)
> [0x7f37ee99387d] [bt] (5)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(_ctypes_callproc+0x2ce)
> [0x7f37eeba91de] [bt] (6)
> /miniconda3/lib/python3.7/lib-dynload/_ctypes.cpython-37m-x86_64-linux-gnu.so(+0x12c14)
> [0x7f37eeba9c14] [bt] (7)
> /miniconda3/bin/python(_PyObject_FastCallKeywords+0x48b)
> [0x563d71b4218b] [bt] (8)
> /miniconda3/bin/python(_PyEval_EvalFrameDefault+0x52cf)
> [0x563d71b91e8f]

SageMaker Random Cut Forest is part of the built-in algorithm library and cannot be deployed in multi-model endpoint (MME). Built-in algorithms currently cannot be deployed to MME. XGboost is an exception, since it has an open-source container https://github.com/aws/sagemaker-xgboost-container.
If you really need to deploy a RCF to a multi-model endpoint, one option is to find a reasonably similar open-source implementation (for example rrcf looks reasonably serious: based on the same paper from Guha et al and with 170+ github stars) and create a custom MME docker container. The documentation is here and there is an excellent tuto here

Unable to start embedded Tomcat whe all Report Services are deployed on AWS FARGATE ECS Containers

I am using AWS FARGATE ECS to host Report Portal. We have hosted Postgres in AWS RDS. Out of 11 services, we able to deploy 10 services in ECS. Last remaining service is API . To deploy these, we have created Cluster, Task Definitions with containers. The, we create Tasks and start them.
Getting error as unable to start embedded tomcat.
API Service: 5.1.0
Built with ♡ by EPAM Systems
Spring Boot (v2.2.5.RELEASE)
2020-05-02 02:22:16.755 INFO 11 --- [ main] c.e.t.r.core.configs.ReportPortalApp : Starting ReportPortalApp on ip-10-0-1-59.ap-south-1.compute.internal with PID 11 (/service-api-5.1.0-exec.jar started by root in /)
2020-05-02 02:22:16.836 INFO 11 --- [ main] c.e.t.r.core.configs.ReportPortalApp : The following profiles are active: default
2020-05-02 02:22:25.338 INFO 11 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2020-05-02 02:22:26.647 INFO 11 --- [ main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 1299ms. Found 30 JPA repository interfaces.
2020-05-02 02:22:37.056 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.security.config.annotation.configuration.ObjectPostProcessorConfiguration' of type [org.springframework.security.config.annotation.configuration.ObjectPostProcessorConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.145 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'objectPostProcessor' of type [org.springframework.security.config.annotation.configuration.AutowireBeanFactoryObjectPostProcessor] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.150 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.security.access.expression.method.DefaultMethodSecurityExpressionHandler#1cc9cfb2' of type [org.springframework.security.access.expression.method.DefaultMethodSecurityExpressionHandler] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.157 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'userRoleHierarchy' of type [com.epam.ta.reportportal.auth.UserRoleHierarchy] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.160 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'securityConfiguration' of type [com.epam.ta.reportportal.core.configs.SecurityConfiguration$$EnhancerBySpringCGLIB$$2bca0c92] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.238 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'permissionEvaluator' of type [com.epam.ta.reportportal.auth.permissions.PermissionEvaluatorFactoryBean] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.340 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'projectManagerPermission' of type [com.epam.ta.reportportal.auth.permissions.ProjectManagerPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.450 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSourceConfig' of type [com.epam.ta.reportportal.config.DataSourceConfig$$EnhancerBySpringCGLIB$$7b53bf4d] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:37.654 INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2020-05-02 02:22:39.056 INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2020-05-02 02:22:39.058 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSource' of type [com.zaxxer.hikari.HikariDataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.260 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.452 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.535 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'ACLContext' of type [com.epam.ta.reportportal.core.configs.ACLContext$$EnhancerBySpringCGLIB$$6e333ceb] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.852 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'coffeinCache' of type [org.springframework.cache.caffeine.CaffeineCache] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.948 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'permissionGrantingStrategy' of type [org.springframework.security.acls.domain.DefaultPermissionGrantingStrategy] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.955 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclAuthorizationStrategy' of type [com.epam.ta.reportportal.auth.acl.ReportPortalAclAuthorizationStrategyImpl] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:39.957 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclCache' of type [org.springframework.security.acls.domain.SpringCacheBasedAclCache] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.046 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'lookupStrategy' of type [org.springframework.security.acls.jdbc.BasicLookupStrategy] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.059 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclService' of type [com.epam.ta.reportportal.auth.acl.ReportPortalAclService] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.147 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclPermissionEvaluator' of type [org.springframework.security.acls.AclPermissionEvaluator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.151 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclFullPermission' of type [com.epam.ta.reportportal.auth.permissions.AclFullPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.236 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'aclReadPermission' of type [com.epam.ta.reportportal.auth.permissions.AclReadPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.243 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'assignedToProjectPermission' of type [com.epam.ta.reportportal.auth.permissions.AssignedToProjectPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.250 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'notCustomerPermission' of type [com.epam.ta.reportportal.auth.permissions.NotCustomerPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.252 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'reporterPermission' of type [com.epam.ta.reportportal.auth.permissions.ReporterPermission] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.255 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'permissionEvaluator' of type [com.epam.ta.reportportal.auth.permissions.ReportPortalPermissionEvaluator] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.256 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'securityConfiguration.MethodSecurityConfig' of type [com.epam.ta.reportportal.core.configs.SecurityConfiguration$MethodSecurityConfig$$EnhancerBySpringCGLIB$$998bc8f7] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.437 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'methodSecurityMetadataSource' of type [org.springframework.security.access.method.DelegatingMethodSecurityMetadataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:40.560 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'jacksonConfiguration' of type [com.epam.ta.reportportal.core.configs.JacksonConfiguration$$EnhancerBySpringCGLIB$$3275d909] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:42.437 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'objectMapper' of type [com.fasterxml.jackson.databind.ObjectMapper] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:43.139 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'jsonConverter' of type [org.springframework.http.converter.json.MappingJackson2HttpMessageConverter] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:43.158 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'stringConverter' of type [org.springframework.http.converter.StringHttpMessageConverter] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:43.242 INFO 11 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'mvcConfig' of type [com.epam.ta.reportportal.core.configs.MvcConfig$$EnhancerBySpringCGLIB$$fe4171f8] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-05-02 02:22:47.353 INFO 11 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8585 (http)
2020-05-02 02:22:47.448 INFO 11 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2020-05-02 02:22:47.448 INFO 11 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.31]
2020-05-02 02:22:47.849 INFO 11 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2020-05-02 02:22:47.849 INFO 11 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 30799 ms
2020-05-02 02:22:52.435 INFO 11 --- [ main] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [name: default]
2020-05-02 02:22:53.036 INFO 11 --- [ main] org.hibernate.Version : HHH000412: Hibernate ORM core version 5.4.12.Final
2020-05-02 02:22:54.335 INFO 11 --- [ main] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
2020-05-02 02:22:58.444 INFO 11 --- [ main] org.hibernate.dialect.Dialect : HHH000400: Using dialect: com.epam.ta.reportportal.commons.JsonbAwarePostgresDialect
2020-05-02 02:23:09.639 INFO 11 --- [ main] o.h.e.t.j.p.i.JtaPlatformInitiator : HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
2020-05-02 02:23:09.650 INFO 11 --- [ main] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2020-05-02 02:23:10.151 INFO 11 --- [ main] org.quartz.impl.StdSchedulerFactory : Using default implementation for ThreadExecutor
2020-05-02 02:23:10.243 INFO 11 --- [ main] org.quartz.core.SchedulerSignalerImpl : Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
2020-05-02 02:23:10.244 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : Quartz Scheduler v.2.3.2 created.
2020-05-02 02:23:10.256 INFO 11 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : Using db table-based data access locking (synchronization).
2020-05-02 02:23:10.258 INFO 11 --- [ main] o.s.s.quartz.LocalDataSourceJobStore : JobStoreCMT initialized.
2020-05-02 02:23:10.258 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : Scheduler meta-data: Quartz Scheduler (v2.3.2) 'reportportal' with instanceId 'api:3d399a8ccb9a8c7800a1c62f702319d1'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 1 threads.
Using job-store 'org.springframework.scheduling.quartz.LocalDataSourceJobStore' - which supports persistence. and is clustered.
2020-05-02 02:23:10.258 INFO 11 --- [ main] org.quartz.impl.StdSchedulerFactory : Quartz scheduler 'reportportal' initialized from an externally provided properties instance.
2020-05-02 02:23:10.258 INFO 11 --- [ main] org.quartz.impl.StdSchedulerFactory : Quartz scheduler version: 2.3.2
2020-05-02 02:23:10.258 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : JobFactory set to: com.epam.ta.reportportal.core.configs.SchedulerConfiguration$1#3c20e9d6
2020-05-02 02:25:22.440 ERROR 11 --- [ main] o.s.b.web.embedded.tomcat.TomcatStarter : Error starting Tomcat context. Exception: org.springframework.beans.factory.BeanCreationException. Message: Error creating bean with name 'servletEndpointRegistrar' defined in class path resource [org/springframework/boot/actuate/autoconfigure/endpoint/web/ServletEndpointManagementContextConfiguration$WebMvcServletEndpointManagementContextConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.endpoint.web.ServletEndpointRegistrar]: Factory method 'servletEndpointRegistrar' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'infoEndpoint' defined in class path resource [org/springframework/boot/actuate/autoconfigure/info/InfoEndpointAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.actuate.info.InfoEndpoint]: Factory method 'infoEndpoint' threw exception; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'infoContributorComposite' defined in URL [jar:file:/service-api-5.1.0-exec.jar!/BOOT-INF/classes!/com/epam/ta/reportportal/info/InfoContributorComposite.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'analyzerInfoContributor' defined in URL [jar:file:/service-api-5.1.0-exec.jar!/BOOT-INF/classes!/com/epam/ta/reportportal/info/AnalyzerInfoContributor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'managementTemplate' defined in class path resource [com/epam/ta/reportportal/core/configs/rabbit/AnalyzerRabbitMqConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.epam.ta.reportportal.core.analyzer.auto.client.RabbitMqManagementClient]: Factory method 'managementTemplate' threw exception; nested exception is org.springframework.web.client.ResourceAccessException: I/O error on PUT request for "http://15.206.28.47:15672/api/vhosts/analyzer": Connect to 15.206.28.47:15672 [/15.206.28.47] failed: Connection timed out (Connection timed out); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to 15.206.28.47:15672 [/15.206.28.47] failed: Connection timed out (Connection timed out)
2020-05-02 02:25:22.548 INFO 11 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2020-05-02 02:25:22.635 WARN 11 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
2020-05-02 02:25:22.635 INFO 11 --- [ main] o.s.s.quartz.SchedulerFactoryBean : Shutting down Quartz Scheduler
2020-05-02 02:25:22.635 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : Scheduler reportportal_$_api:3d399a8ccb9a8c7800a1c62f702319d1 shutting down.
2020-05-02 02:25:22.635 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : Scheduler reportportal_$_api:3d399a8ccb9a8c7800a1c62f702319d1 paused.
2020-05-02 02:25:22.753 INFO 11 --- [ main] org.quartz.core.QuartzScheduler : Scheduler reportportal_$_api:3d399a8ccb9a8c7800a1c62f702319d1 shutdown complete.
2020-05-02 02:25:22.762 INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2020-05-02 02:25:22.834 INFO 11 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
2020-05-02 02:25:22.943 INFO 11 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-05-02 02:25:22.950 ERROR 11 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.context.ApplicationContextException: Unable to start web server; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:156)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at com.epam.ta.reportportal.core.configs.ReportPortalApp.main(ReportPortalApp.java:37)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat

Based on the error:
Exception: I/O error on PUT request for
"http://15.206.28.47:15672/api/vhosts/analyzer":
Connect to 15.206.28.47:15672 [/15.206.28.47] failed: Connection timed out
(Connection timed out);
your RabbitMQ instance is not accessible or VHOST analyzer is not created.

It looks like the problem is that it can't connect to 15.206.28.47 during startup.

Cannot get response 200 with spring boot

Hi i am testing my controllers methods with chrome's reslet client and i have an error 404, after checking everything in my project i built a new one and created just one service and one controller and still all i get is error 404.
any ideas?
this is the url im calling:
http://localhost:8080/rest/api/encryptor/checkAutorization
that is my PomXML with al the dependencys :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.serverside</groupId>
<artifactId>app</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>app</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
this is the Controller:
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import Services.GeneralResponse;
import Services.TokenEncryptor;
#RestController
#RequestMapping("rest/api/encryptor")
public class EncryptionController {
#Autowired
private TokenEncryptor tokenEncryptor;
#GetMapping("/checkAutorization")
public GeneralResponse checkAutorization() throws Throwable {
try {
// byte[] byteToken = token.getBytes(StandardCharsets.UTF_8);
return new GeneralResponse(tokenEncryptor.encryptToken(new byte[]
{}));
} catch (Exception AccessDeniedException) {
return new GeneralResponse(AccessDeniedException);
}
}
}
Main method:
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Configuration;
#Configuration
#SpringBootApplication
public class AppApplication {
public static void main(String[] args) {
SpringApplication.run(AppApplication.class, args);
}
}
General Response Class:
import java.text.SimpleDateFormat;
import java.util.Date;
import com.fasterxml.jackson.annotation.JsonInclude;
#JsonInclude(JsonInclude.Include.NON_NULL)
public class GeneralResponse {
private Object error;
private Object result;
public GeneralResponse() {
}
public GeneralResponse(Object result) {
this.result = result;
}
public GeneralResponse(Exception e) {
this.result = null;
this.error = e;
StackTraceElement[] stackTrace = e.getStackTrace();
System.out.println("The exception occured in method " + stackTrace[0].getMethodName());
System.out.println("The class name is " + stackTrace[0].getClassName());
System.out.println("at line number: " + stackTrace[0].getLineNumber());
System.out.println(e.toString() + " \n" + "error cause: " + e.getCause() + "\n" + "at: " + formatTime(System.currentTimeMillis()));
}
public Object getError() {
return error;
}
public void setError(Object error) {
this.error = error;
}
public Object getResult() {
return result;
}
public void setResult(Object result) {
this.result = result;
}
private String formatTime(Long currentTime) {
SimpleDateFormat sdf = new SimpleDateFormat("MMM dd,yyyy HH:mm");
Date resultdate = new Date(currentTime);
return (sdf.format(resultdate));
}
}
Console startup log :
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.1.1.RELEASE)
2018-12-17 13:40:20.763 INFO 21872 --- [ restartedMain] com.serverside.app.AppApplication : Starting AppApplication on LAPTOP-0GNLE6KN with PID 21872 (C:\Users\omert\eclipse-workspace\app\target\classes started by omert in C:\Users\omert\eclipse-workspace\app)
2018-12-17 13:40:20.766 INFO 21872 --- [ restartedMain] com.serverside.app.AppApplication : No active profile set, falling back to default profiles: default
2018-12-17 13:40:20.814 INFO 21872 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2018-12-17 13:40:20.815 INFO 21872 --- [ restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG'
2018-12-17 13:40:21.546 INFO 21872 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data repositories in DEFAULT mode.
2018-12-17 13:40:21.569 INFO 21872 --- [ restartedMain] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 15ms. Found 0 repository interfaces.
2018-12-17 13:40:21.946 INFO 21872 --- [ restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration$$EnhancerBySpringCGLIB$$365e8361] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2018-12-17 13:40:22.515 INFO 21872 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
2018-12-17 13:40:22.541 INFO 21872 --- [ restartedMain] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2018-12-17 13:40:22.542 INFO 21872 --- [ restartedMain] org.apache.catalina.core.StandardEngine : Starting Servlet Engine: Apache Tomcat/9.0.13
2018-12-17 13:40:22.554 INFO 21872 --- [ restartedMain] o.a.catalina.core.AprLifecycleListener : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [C:\Program Files\Java\jre1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:/Program Files/Java/jre1.8.0_191/bin/server;C:/Program Files/Java/jre1.8.0_191/bin;C:/Program Files/Java/jre1.8.0_191/lib/amd64;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\nodejs\;C:\Program Files\Git\cmd;C:\Users\omert\AppData\Local\Microsoft\WindowsApps;C:\Users\omert\AppData\Roaming\npm;C:\Program Files\Microsoft VS Code\bin;C:\Program Files\nodejs\node.exe;;C:\WINDOWS\system32;;.]
2018-12-17 13:40:22.697 INFO 21872 --- [ restartedMain] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
2018-12-17 13:40:22.697 INFO 21872 --- [ restartedMain] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 1882 ms
2018-12-17 13:40:22.915 INFO 21872 --- [ restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2018-12-17 13:40:23.063 INFO 21872 --- [ restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2018-12-17 13:40:23.109 INFO 21872 --- [ restartedMain] o.hibernate.jpa.internal.util.LogHelper : HHH000204: Processing PersistenceUnitInfo [
name: default
...]
2018-12-17 13:40:23.178 INFO 21872 --- [ restartedMain] org.hibernate.Version : HHH000412: Hibernate Core {5.3.7.Final}
2018-12-17 13:40:23.181 INFO 21872 --- [ restartedMain] org.hibernate.cfg.Environment : HHH000206: hibernate.properties not found
2018-12-17 13:40:23.338 INFO 21872 --- [ restartedMain] o.hibernate.annotations.common.Version : HCANN000001: Hibernate Commons Annotations {5.0.4.Final}
2018-12-17 13:40:23.482 INFO 21872 --- [ restartedMain] org.hibernate.dialect.Dialect : HHH000400: Using dialect: org.hibernate.dialect.MySQL5Dialect
2018-12-17 13:40:23.713 INFO 21872 --- [ restartedMain] o.h.t.schema.internal.SchemaCreatorImpl : HHH000476: Executing import script 'org.hibernate.tool.schema.internal.exec.ScriptSourceInputNonExistentImpl#77c8a0a2'
2018-12-17 13:40:23.716 INFO 21872 --- [ restartedMain] j.LocalContainerEntityManagerFactoryBean : Initialized JPA EntityManagerFactory for persistence unit 'default'
2018-12-17 13:40:23.731 INFO 21872 --- [ restartedMain] o.s.b.d.a.OptionalLiveReloadServer : LiveReload server is running on port 35729
2018-12-17 13:40:23.983 INFO 21872 --- [ restartedMain] o.s.s.concurrent.ThreadPoolTaskExecutor : Initializing ExecutorService 'applicationTaskExecutor'
2018-12-17 13:40:24.038 WARN 21872 --- [ restartedMain] aWebConfiguration$JpaWebMvcConfiguration : spring.jpa.open-in-view is enabled by default. Therefore, database queries may be performed during view rendering. Explicitly configure spring.jpa.open-in-view to disable this warning
2018-12-17 13:40:24.325 INFO 21872 --- [ restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2018-12-17 13:40:24.329 INFO 21872 --- [ restartedMain] com.serverside.app.AppApplication : Started AppApplication in 3.869 seconds (JVM running for 4.415)

I am getting 200 response code
POM : http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
org.gjp.springboot
test
0.0.1-SNAPSHOT
jar
test
Test Spring Boot Web Application
org.springframework.boot
spring-boot-starter-parent
2.0.0.RELEASE
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.8</java.version>
</properties>
#Service
public class TokenEncryptor {

vSphere vs. terraform VM Customization Failure with Network not being connected

When I'm trying to customize the network interface post provision of a Windows VM, terraform errors, VM is left running with no NIC attached.
Version
Terraform = v0.11.7
vSphere Version = 6.7.0, 8169922
Machine clone of Windows 2012 R2
Investigation so far;
Within vCenter and the VM settings the "Network Adapter 1" is not marked to start on power up. It's as if during the customization that this setting is lost which is resulting in lack of comms to the VM. If I power the server off then check the box to connect NIC on start up then when the server comes back online it has its network configuration correct as defined in the customization section. I have also stripped back the customization to bare minimum and still get the same fault.
terraform console output
Error dumps below.
Error: Error applying plan:
1 error(s) occurred:
* vsphere_virtual_machine.Clone: 1 error(s) occurred:
* vsphere_virtual_machine.Clone:
Virtual machine customization failed on "/Data Center/vm/AIS1EM-SAMSON":
An error occurred while customizing VM AIS1EM-SAMSON. For details reference the log file C:/Windows/TEMP/vmware-imc/guestcust.log in the guest OS.
Customfile log from VM
[2018-05-22T17:35:13: : INFO] GuestCustUtil invoked.
[2018-05-22T17:35:13: : INFO] Params: customize -sealparam "/quiet /generalize /oobe /reboot" -nics 4000 -schedulenativeunobfusc
[2018-05-22T17:35:13: : DEBUG] Registry value 'CustomRescheduleFlags' is not set. Returning default value: 0
[2018-05-22T17:35:13: : INFO] Checking for Windows Sysprep Respecialize operation.
[2018-05-22T17:35:13: : DEBUG] Sysprep process is not running.
[2018-05-22T17:35:13: : INFO] Windows Sysprep Respecialize operation was not detected.
[2018-05-22T17:35:13: GuestCustUtil: DEBUG] Deleting status marker file C:/Windows/.post-gc-status
[2018-05-22T17:35:13: GuestCustUtil: DEBUG] Status marker file C:/Windows/.post-gc-status doesn't exist
[2018-05-22T17:35:13: : INFO] Extended customization is not enabled!
[2018-05-22T17:35:13: : DEBUG] HandleCustomizationStarted
[2018-05-22T17:35:13: : DEBUG] Customization in progress set to 1 at 2018-May-22 17:35:13
[2018-05-22T17:35:13: : DEBUG] GOScPostRebootStatus feature is enabled
[2018-05-22T17:35:13: GuestCustUtil: DEBUG] SetVmxProperty: key-'guestinfo.gc.notification', value-'Started'
[2018-05-22T17:35:13: WindowsOsAbstractionLayer: INFO] VMware tools installation path=C:\Program Files\VMware\VMware Tools\
[2018-05-22T17:35:13: WindowsOsAbstractionLayer: INFO] Executing command: '"C:\Program Files\VMware\VMware Tools\vmtoolsd.exe" --cmd "info-set guestinfo.gc.notification Started"'
[2018-05-22T17:35:13: WindowsOsAbstractionLayer: INFO] Command execution succeeded, output: '
'
[2018-05-22T17:35:13: : DEBUG] Rpci: Sending request='deployPkg.update.state 4 0 C:/Windows/TEMP/vmware-imc/guestcust.log'
[2018-05-22T17:35:13: : DEBUG] Rpci: Sent request='deployPkg.update.state 4 0 C:/Windows/TEMP/vmware-imc/guestcust.log', reply='', len=0, status=1
[2018-05-22T17:35:13: : DEBUG] Temporary sysprep dir path: C:\sysprep
[2018-05-22T17:35:13: : DEBUG] Copying file/directory from 'sysprep' to 'C:\sysprep'
[2018-05-22T17:35:13: : INFO] Sysprep File is C:\Windows\Panther\unattend.xml
[2018-05-22T17:35:13: : INFO] Password unobfuscation in progress
[2018-05-22T17:35:13: : DEBUG] Decrypting passwords
[2018-05-22T17:35:13: : DEBUG] Password found and decrypted for <AdministratorPassword><EncryptedValue>
[2018-05-22T17:35:13: : DEBUG] preTag <Credentials> not found
[2018-05-22T17:35:13: : DEBUG] preTag <Password> not found
[2018-05-22T17:35:13: : DEBUG] Tag -de not found after <RunSynchronousCommand
[2018-05-22T17:35:13: : INFO] Password unobfuscation successful
[2018-05-22T17:35:14: : DEBUG] select * from win32_networkadapter where Manufacturer != 'Microsoft' and ServiceName != 'VMnetAdapter' and manufacturer is not null and MACAddress is not null
[2018-05-22T17:35:14: : DEBUG] Found 0 objects. Pointer 0. return code -2147217398(0x8004100a)
[2018-05-22T17:35:14: : ERROR] WinMgmt :
[2018-05-22T17:35:14: : DEBUG] HandleCustomizationFailed
[2018-05-22T17:35:14: : DEBUG] Customization in progress set to 0 at 2018-May-22 17:35:14
[2018-05-22T17:35:14: : INFO] Cleaning up obfuscation details from registry
[2018-05-22T17:35:14: GuestCustUtil: DEBUG] Status marker file C:/Windows/.post-gc-status doesn't exist
[2018-05-22T17:35:14: : DEBUG] GOScPostRebootStatus feature is enabled
[2018-05-22T17:35:14: GuestCustUtil: DEBUG] SetVmxProperty: key-'guestinfo.gc.notification', value-'WinMgmt : '
[2018-05-22T17:35:14: WindowsOsAbstractionLayer: INFO] VMware tools installation path=C:\Program Files\VMware\VMware Tools\
[2018-05-22T17:35:14: WindowsOsAbstractionLayer: INFO] Executing command: '"C:\Program Files\VMware\VMware Tools\vmtoolsd.exe" --cmd "info-set guestinfo.gc.notification WinMgmt : "'
[2018-05-22T17:35:14: WindowsOsAbstractionLayer: INFO] Command execution succeeded, output: '
'
[2018-05-22T17:35:14: : DEBUG] Rpci: Sending request='deployPkg.update.state 4 100 C:/Windows/TEMP/vmware-imc/guestcust.log#WinMgmt : '
[2018-05-22T17:35:14: : DEBUG] Rpci: Sent request='deployPkg.update.state 4 100 C:/Windows/TEMP/vmware-imc/guestcust.log#WinMgmt : ', reply='', len=0, status=1
[2018-05-22T17:35:14: : INFO] GuestCustUtil exiting.
[2018-05-22T17:35:23: : INFO] GuestCustUtil invoked.
[2018-05-22T17:35:23: : INFO] Params: restoreMountedDevices
[2018-05-22T17:35:23: : DEBUG] Registry value 'CustomRescheduleFlags' is not set. Returning default value: 0
[2018-05-22T17:35:23: : INFO] Checking for Windows Sysprep Respecialize operation.
[2018-05-22T17:35:23: : DEBUG] Sysprep process is not running.
[2018-05-22T17:35:23: : INFO] Windows Sysprep Respecialize operation was not detected.
[2018-05-22T17:35:23: SysprepDriveLettersWorkaround: DEBUG] Restoring HKLM MountedDevices to fix sysprep issue
[2018-05-22T17:35:23: : DEBUG] Enabling SE_RESTORE_NAME privilege
[2018-05-22T17:35:23: : DEBUG] Retrieving process access token for SeRestorePrivilege
[2018-05-22T17:35:23: : DEBUG] Getting LUID
[2018-05-22T17:35:23: : DEBUG] Adjusting privileges
[2018-05-22T17:35:23: : DEBUG] Restoring SYSTEM\MountedDevices from C:\sysprep\MountedDevicesBackup
[2018-05-22T17:35:23: SysprepDriveLettersWorkaround: WARN] Failed to restore MountedDevices: The system cannot find the file specified.
. Ignoring
[2018-05-22T17:35:23: : INFO] GuestCustUtil exiting.
[2018-05-22T17:35:23: : INFO] GuestCustUtil invoked.
[2018-05-22T17:35:23: : INFO] Params: flagComplete
[2018-05-22T17:35:23: : DEBUG] Registry value 'CustomRescheduleFlags' is not set. Returning default value: 0
[2018-05-22T17:35:23: : INFO] Checking for Windows Sysprep Respecialize operation.
[2018-05-22T17:35:23: : DEBUG] Sysprep process is not running.
[2018-05-22T17:35:23: : INFO] Windows Sysprep Respecialize operation was not detected.
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] Status marker file C:/Windows/.post-gc-status doesn't exist
[2018-05-22T17:35:23: WindowsOsAbstractionLayer: INFO] Executing command: 'C:\Windows\system32\schtasks.exe /query /tn "VMware Guest OS Customization Task"'
[2018-05-22T17:35:23: WindowsOsAbstractionLayer: WARN] Command execution failed with exit code: 1, output: 'ERROR: The system cannot find the file specified.
'
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] SchedulePostCustomizationActivity: Run Command-'HandleCustomizationCompleted<\|/>'
[2018-05-22T17:35:23: GuestCustUtil: INFO] Scheduling post-customization activity to run after reboot...
[2018-05-22T17:35:23: WindowsOsAbstractionLayer: INFO] Executing command: 'C:\Windows\system32\schtasks.exe /query /tn "VMware Guest OS Customization Task"'
[2018-05-22T17:35:23: WindowsOsAbstractionLayer: WARN] Command execution failed with exit code: 1, output: 'ERROR: The system cannot find the file specified.
'
[2018-05-22T17:35:23: GuestCustUtil: INFO] Task 'VMware Guest OS Customization Task' exists and will be deleted: false
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] Artifact full path: C:\sysprep\libeay32.dll
[2018-05-22T17:35:23: GuestCustUtil: INFO] Copying C:\sysprep\libeay32.dll to C:\Windows\TEMP\vmw857B.tmp.
[2018-05-22T17:35:23: : DEBUG] Copying file/directory from 'C:\sysprep\libeay32.dll' to 'C:\Windows\TEMP\vmw857B.tmp'
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] Artifact full path: C:\sysprep\ssleay32.dll
[2018-05-22T17:35:23: GuestCustUtil: INFO] Copying C:\sysprep\ssleay32.dll to C:\Windows\TEMP\vmw857B.tmp.
[2018-05-22T17:35:23: : DEBUG] Copying file/directory from 'C:\sysprep\ssleay32.dll' to 'C:\Windows\TEMP\vmw857B.tmp'
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] Artifact full path: C:\sysprep\guestcustutil.exe
[2018-05-22T17:35:23: GuestCustUtil: INFO] Copying C:\sysprep\guestcustutil.exe to C:\Windows\TEMP\vmw857B.tmp.
[2018-05-22T17:35:23: : DEBUG] Copying file/directory from 'C:\sysprep\guestcustutil.exe' to 'C:\Windows\TEMP\vmw857B.tmp'
[2018-05-22T17:35:23: GuestCustUtil: DEBUG] Schedule command: 'C:\Windows\TEMP\vmw857B.tmp\guestcustutil.exe post-run HandleCustomizationCompleted<\|/>'
[2018-05-22T17:35:23: : INFO] Scheduling 'VMware Guest OS Customization Task'
[2018-05-22T17:35:23: : DEBUG] Executing command C:\Windows\system32\schtasks.exe /create /sc ONSTART /rl HIGHEST /np /z /v1 /ru SYSTEM /tr "C:\Windows\TEMP\vmw857B.tmp\guestcustutil.exe post-run HandleCustomizationCompleted<\|/>" /tn "VMware Guest OS Customization Task"
[2018-05-22T17:35:23: GuestCustUtil: INFO] Post-customization activity successfully scheduled.
[2018-05-22T17:35:23: : INFO] GuestCustUtil exiting.
[2018-05-22T17:35:23: : INFO] GuestCustUtil invoked.
[2018-05-22T17:35:23: : INFO] Params: deleteContainingFolder
[2018-05-22T17:35:23: : DEBUG] Registry value 'CustomRescheduleFlags' is not set. Returning default value: 0
[2018-05-22T17:35:23: : INFO] Checking for Windows Sysprep Respecialize operation.
[2018-05-22T17:35:23: : DEBUG] Sysprep process is not running.
[2018-05-22T17:35:23: : INFO] Windows Sysprep Respecialize operation was not detected.
[2018-05-22T17:35:23: : DEBUG] Deleting "C:\sysprep" ...
[2018-05-22T17:35:23: : DEBUG] Deleted folder "C:\sysprep" .
[2018-05-22T17:35:23: : INFO] GuestCustUtil exiting.
[2018-05-22T17:35:47: : INFO] GuestCustUtil invoked.
[2018-05-22T17:35:47: : INFO] Params: post-run HandleCustomizationCompleted<\|/>
[2018-05-22T17:35:47: : DEBUG] Registry value 'CustomRescheduleFlags' is not set. Returning default value: 0
[2018-05-22T17:35:47: : INFO] Checking for Windows Sysprep Respecialize operation.
[2018-05-22T17:35:47: : DEBUG] Sysprep process is not running.
[2018-05-22T17:35:47: : INFO] Windows Sysprep Respecialize operation was not detected.
[2018-05-22T17:35:47: : INFO] Post-customization run operation: script 'HandleCustomizationCompleted<\|/>'
[2018-05-22T17:35:47: : INFO] post-run HandleCustomizationCompleted
[2018-05-22T17:35:47: : INFO] Deleting post-customization task to avoid re-execution
[2018-05-22T17:35:47: WindowsOsAbstractionLayer: INFO] Executing command: 'C:\Windows\system32\schtasks.exe /query /tn "VMware Guest OS Customization Task"'
[2018-05-22T17:35:48: WindowsOsAbstractionLayer: INFO] Command execution succeeded, output: '
Folder: \
TaskName Next Run Time Status
======================================== ====================== ===============
VMware Guest OS Customization Task N/A Running
'
[2018-05-22T17:35:48: GuestCustUtil: INFO] Task 'VMware Guest OS Customization Task' exists and will be deleted: true
[2018-05-22T17:35:48: : INFO] Deleting 'VMware Guest OS Customization Task'
[2018-05-22T17:35:48: : DEBUG] Executing command C:\Windows\system32\schtasks.exe /delete /f /tn "VMware Guest OS Customization Task"
[2018-05-22T17:35:48: : DEBUG] HandleCustomizationCompleted
[2018-05-22T17:35:48: : DEBUG] Customization status previously reported, nothing to do
[2018-05-22T17:35:48: : DEBUG] Deleting "C:\Windows\TEMP\vmw857B.tmp" ...
[2018-05-22T17:35:48: : DEBUG] Deleted folder "C:\Windows\TEMP\vmw857B.tmp" .
[2018-05-22T17:35:48: : INFO] GuestCustUtil exiting.
Terraform file
/*
Secrets
*/
provider "vault" {
address = "https://<removed>:8200"
token = "<removed>"
}
data "vault_generic_secret" "terraform_secrets" {
path = "secret/serviceaccounts/default_group"
}
provider "vsphere" {
user = "${data.vault_generic_secret.terraform_secrets.data["username"]}#${data.vault_generic_secret.terraform_secrets.data["domain_fqdn"]}"
password = "${data.vault_generic_secret.terraform_secrets.data["password"]}"
vsphere_server = "<removed>"
}
/*
Variables
*/
#The template to use for creation of new Virtual Machine
variable "Template" { default = "W2012R2_RELEASE" }
#The Resource Pool where the new VM should be created
variable "ResourcePool" { default = "Self-Service" }
#The folder where the new VM should be located
variable "Folder" { default = "AIS1EM" }
/*
VM Build
*/
data "vsphere_datacenter" "dc" {
name = "Data Center"
}
data "vsphere_datastore" "datastore" {
name = "vsanDatastore"
datacenter_id = "${data.vsphere_datacenter.dc.id}"
}
data "vsphere_resource_pool" "pool" {
name = "${var.ResourcePool}"
datacenter_id = "${data.vsphere_datacenter.dc.id}"
}
data "vsphere_distributed_virtual_switch" "dvs" {
name = "DSwitch0"
datacenter_id = "${data.vsphere_datacenter.dc.id}"
}
data "vsphere_network" "network" {
name = "DPortGroup-VM-VLAN59"
datacenter_id = "${data.vsphere_datacenter.dc.id}"
}
data "vsphere_virtual_machine" "template" {
name = "${var.Template}"
datacenter_id = "${data.vsphere_datacenter.dc.id}"
}
# clone from template
resource "vsphere_virtual_machine" "Clone" {
name = "AIS1EM-SAMSON"
resource_pool_id = "${data.vsphere_resource_pool.pool.id}"
datastore_id = "${data.vsphere_datastore.datastore.id}"
wait_for_guest_net_timeout = false
num_cpus = 2
memory = 4096
guest_id = "${data.vsphere_virtual_machine.template.guest_id}"
scsi_type = "${data.vsphere_virtual_machine.template.scsi_type}"
network_interface {
network_id = "${data.vsphere_network.network.id}"
adapter_type = "vmxnet3"
}
disk {
label = "AIS1EM-SAMSON.vmdk"
size = "${data.vsphere_virtual_machine.template.disks.0.size}"
eagerly_scrub = "${data.vsphere_virtual_machine.template.disks.0.eagerly_scrub}"
thin_provisioned = "${data.vsphere_virtual_machine.template.disks.0.thin_provisioned}"
}
clone {
template_uuid = "${data.vsphere_virtual_machine.template.id}"
customize {
windows_options {
computer_name = "AIS1EM-SAMSON"
#join_domain = "<removed>"
#domain_admin_user = "<removed>"
#domain_admin_password = "<removed>"
admin_password = "<removed>"
}
network_interface {
ipv4_address = "192.168.59.50"
ipv4_netmask = "24"
dns_server_list = ["192.168.72.167", "192.168.72.168"]
#dns_domain = "emisais.com"
}
ipv4_gateway = "192.168.59.1"
}
}
}
output "ID" {
value = "${data.vsphere_network.network.id}"
}

You may want to ensure your VM Template hasn't already been sysprepped.
I went through the same issues you're having, once using a template which hadn't been sysprep'd (I believe Terraform will sysprep it) the network adaptors would connect at the correct times during customization.

How to upload file to aws S3 from EC2 instance with private IP

Normally I can download and upload files to S3 instance using:
using (var fileTransferUtility = new TransferUtility(new AmazonS3Client(credentials, regionEndpoint)))
{
await fileTransferUtility.UploadAsync(filePath, bucketName, directoryName);
}
It work's as expected from local computer and EC2 instances with public IP.
But when I launch instance only with private IP - only download works but on upload it throws an exception:
at Amazon.Runtime.Internal.WebExceptionHandler.HandleException(IExecutionContext executionContext, WebException exception) in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\_bcl\WebExceptionHandler.cs:line 44
at Amazon.Runtime.Internal.ExceptionHandler`1.Handle(IExecutionContext executionContext, Exception exception) in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\ExceptionHandler.cs:line 38
at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception) in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\ErrorHandler.cs:line 204
at Amazon.Runtime.Internal.ErrorHandler.<InvokeAsync>d__5`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\ErrorHandler\ErrorHandler.cs:line 104
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CallbackHandler.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.CredentialsRetriever.<InvokeAsync>d__7`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CredentialsRetriever.cs:line 98
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\RetryHandler\RetryHandler.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at Amazon.Runtime.Internal.RetryHandler.<InvokeAsync>d__10`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\RetryHandler\RetryHandler.cs:line 137
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CallbackHandler.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.CallbackHandler.<InvokeAsync>d__9`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\CallbackHandler.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.S3.Internal.AmazonS3ExceptionHandler.<InvokeAsync>d__1`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\Internal\AmazonS3ExceptionHandler.cs:line 62
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.ErrorCallbackHandler.<InvokeAsync>d__5`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\ErrorCallbackHandler.cs:line 58
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.Runtime.Internal.MetricsHandler.<InvokeAsync>d__1`1.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\Handlers\MetricsHandler.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Amazon.S3.Transfer.Internal.SimpleUploadCommand.<ExecuteAsync>d__10.MoveNext() in E:\JenkinsWorkspaces\v3-stage-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\_bcl+coreclr+pcl\Transfer\Internal\_async\SimpleUploadCommand.async.cs:line 42
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at ..StorageService.<UploadFileAsync>d__13.MoveNext() in ..\StorageService.cs:line 100
What should I set up on S3 or while launching EC2 with private IP? Or I must always enable public IP?

Set up VPC Endpoint for Amazon S3