Constant database disk write activity when Task Scheduling enabled with Email task activated

Matthis · June 13, 2022, 6:45pm

Hi, using iotop I found out that I have constant disk write activity when Task Scheduling in set to true with Email sending scheduled task activated.

iotop output:

TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
291612 be/4 postgres 0.00 B/s 7.06 K/s 0.00 % 2.09 % postgres: 12/main: postgres hjportal 127.0.0.1(45102) idle

As soon as I deactivate the task or turn scheduling to false, this entry disappears.

I found this a bit disturbing… is it expected / should I worry about it?

Related info:
Cuba platform version: 7.1.9
Postgres: 12.11
Tomcat (installed from Ubuntu): 9.0.31-1ubuntu0.2
Task: cuba_Emailer -> processQueuedEmails()

Strace on the writing postgres process returns repeated errors:
recvfrom(9, 0x559d5029f523, 5, 0, NULL, NULL) = -1 EAGAIN (Resource temporarily unavailable)

Thank you for any help!

albudarov · June 14, 2022, 6:49am

Hi,
CUBA scheduling uses SQL queries like “select from SYS_SCHEDULED_TASK … for update” for coordination. It leads to write activity.
Also, CUBA scheduling writes to the SYS_SCHEDULED_EXECUTION table, if you enabled “singleton” or “log start”, “log finish” flags in the task.

Matthis · June 14, 2022, 8:35am

Hi,

Noted that this is the expected behavior, thank you for your explanation. I get write activity almost each second - does this seem reasonable? (This is constant regardless of the scheduling interval).

albudarov · June 14, 2022, 9:39am

This is regulated by the cuba.schedulingInterval application property (milliseconds), which is 1000 by default.

See

github.com

cuba-platform/cuba/blob/master/modules/core/src/com/haulmont/cuba/spring.xml#L201


      
              <property name="daemon" value="true"/>
              <property name="poolSize" value="10"/>
              <property name="statisticsAccumulator" ref="cuba_MiddlewareStatisticsAccumulator"/>
              <property name="threadNamePrefix" value="cuba_core_scheduler-"/>
          </bean>
          
          
<task:scheduled-tasks scheduler="scheduler">
              <task:scheduled ref="cuba_UserSessions" method="processEviction" fixed-rate="10000"/>
              <task:scheduled ref="cuba_LockManager" method="expireLocks" fixed-rate="60000"/>
              <task:scheduled ref="cuba_Scheduling" method="processScheduledTasks"
                              fixed-rate="${cuba.schedulingInterval?:1000}"/>
              <task:scheduled ref="cuba_TriggerFilesProcessor" method="process"
                              fixed-delay="${cuba.triggerFilesCheckInterval?:5000}"/>
              <task:scheduled ref="cuba_MiddlewareStatisticsAccumulator" method="gatherParameters" fixed-rate="5000"/>
              <task:scheduled ref="cuba_QueryResultsManager" method="deleteForInactiveSessions" fixed-rate="${cuba.deleteOldQueryResultsInterval?:600000}"/>
          </task:scheduled-tasks>
          
          
<!-- Email setup -->
          
          
<bean id="mailSendTaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
              <property name="corePoolSize" value="5"/>

Matthis · June 15, 2022, 1:54am

Alexander, thank you for your detailed reply.

I changed cuba.schedulingInterval to 60,000, but I still get database disk write activity constantly (or at least every second as monitored by iotop). Is this also expected?

Thank you,

albudarov · June 15, 2022, 6:54am

I would suggest to enable “eclipselink.sql” debug logging in the CUBA app, and enable PostgreSQL logging for all SQL queries, to see what queries does PostgreSQL server receive.

Note that due to the MVCC or other reasons it is completely normal for PostgreSQL server to write some disk pages even during execution of READ queries or even when the server is idle.
See:

Matthis · June 15, 2022, 7:20am

Thank you for all this useful information!!