Apache Ignite autoscaling in AWS (linux) - amazon-web-services

Question

Currently we host Apache Ignite node in AWS using Apache Ignite Image with 16 RAM.
We want to dynamically add new nodes while load on cache increases.
For this purpose we need to somehow trigger that node will run out of memory soon and we need to add additional node. Is there any way to track that?
I've tried to load cache with random data, cache failed with OutOfMemoryException when java process took over 30-40% of RAM.
Here is default-config.xml from {IGNITE_HOME}\config:
<?xml version="1.0" encoding="UTF-8"?>
...
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="cacheConfiguration">
<list>
<!-- Partitioned cache example configuration (Atomic mode). -->
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="default"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="backups" value="1"/>
</bean>
</list>
</property>
<!-- Enabling Apache Ignite Persistent Store. -->
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="false"/>
<property name="metricsEnabled" value="true"/>
<property name="maxSize" value="#{10L * 1024 * 1024 * 1024}"/>
</bean>
</property>
</bean>
</property>
<!-- Explicitly configure TCP discovery SPI to provide list of initial nodes. -->
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder">
<property name="awsCredentialsProvider" ref="aws.creds"/>
<property name="bucketName" value="dev-apache-ignite"/>
</bean>
</property>
</bean>
</property>
</bean>
<!-- AWS credentials. Provide your access key ID and secret access key. -->
<bean id="aws.creds" class="com.amazonaws.auth.InstanceProfileCredentialsProvider">
<constructor-arg value="false" />
</bean>
</beans>
Sorry if this question was alsready answered in documentation.
Is there any predefined guidelines to configure AWS autoscaling for ignite?

Answer 1

Amount of RAM taken by JVM is not relevant. You will see IgniteOutOfMemoryException when you run out of data region - in your case, 10G.
You may divide DataRegionMetrics.getOffheapUsedSize() by data region size to understand how much runway do you have left.
Then you can probably use GridGain K8S Operator to scale your cluster:
https://www.gridgain.com/docs/latest/installation-guide/operator/how-tos