Logrotation for flink not working with logrotate.d - google-cloud-platform

Question

I would like to enable log rotation for flink but didn't see an option to reload the process.
Tried to enable log rotation using "logrotate.d" with copytruncate option but leading to the creation of sparse files.
Is there any option to enable log rotation for flink taskmanager without restarting the process.

Answer 1

Have you tried something like inside log4j.properties in flink/conf installation? log4j.appender.file=org.apache.log4j.RollingFileAppender log4j.appender.file.File=${log.file} log4j.appender.file.MaxFileSize=1000MB log4j.appender.file.MaxBackupIndex=0 log4j.appender.file.append=false log4j.appender.file.layout=org.apache.log4j.PatternLayout log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n

Answer 2

It has been a couple years since the original answer. I needed to do the same thing on EMR for Flink 1.13. I figure EMR is a common enough platform that I'd share what worked for me.
I believe the main difference between my answer and the earlier one is that Flink 1.13 uses Log4j 2. I suspect the earlier example is using Log4j version 1 because there's this line, for example:
log4j.appender.file.MaxBackupIndex=0
but in my code below, the same would be
appender.main.strategy.max = 0 (I actually use 1000 in the code, but wanted to show an apples-to-apples comparison)
That makes sense, because in Migration from log4j to log4j2:the equivalent attribute to MaxFileSize and MaxBackupIndex it says:
In Log4j 2 those values are associated with the triggering policy or RolloverStrategy
Anyhow, the configuration file on the EMR master node is /etc/flink/conf.dist/log4j.properties.
Here is the complete file contents (minus the boilerplate license at the top).
Each place I made a change to the content of the file to support log rolling has a rolling_file_change comment:
# This affects logging for both user code and Flink
rootLogger.level = INFO
rootLogger.appenderRef.file.ref = MainAppender
# Uncomment this if you want to _only_ change Flink's logging
#logger.flink.name = org.apache.flink
#logger.flink.level = INFO
# The following lines keep the log level of common libraries/connectors on
# log level INFO. The root logger does not override this. You have to manually
# change the log levels here.
logger.akka.name = akka
logger.akka.level = INFO
logger.kafka.name= org.apache.kafka
logger.kafka.level = INFO
logger.hadoop.name = org.apache.hadoop
logger.hadoop.level = INFO
logger.zookeeper.name = org.apache.zookeeper
logger.zookeeper.level = INFO
# Log all infos in the given file
appender.main.name = MainAppender
# rolling_file_change: original line
# appender.main.type = File
# rolling_file_change: replacement line
appender.main.type = RollingFile
appender.main.append = false
appender.main.fileName = ${sys:log.file}
# rolling_file_change: net new line
appender.main.filePattern = ${sys:log.file}.%i
appender.main.layout.type = PatternLayout
appender.main.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n
# rolling_file_change: new block start
appender.main.policies.type = Policies
appender.main.policies.size.type = SizeBasedTriggeringPolicy
appender.main.policies.size.size=10MB
appender.main.strategy.type = DefaultRolloverStrategy
appender.main.strategy.max = 1000
# rolling_file_change: new block end
# Suppress the irrelevant (wrong) warnings from the Netty channel handler
logger.netty.name = org.apache.flink.shaded.akka.org.jboss.netty.channel.DefaultChannelPipeline
logger.netty.level = OFF