Search Engine: Elastic

Article ID: 131582, created on Oct 9, 2017, last review on Oct 9, 2017

Symptoms

APS 2.0 tasks are failing:

Network error: SSL connection unexpectedly closed. Application endpoint: 192.0.2.2. Request: DELETE ...

where 192.0.2.2 is the application instance endpoint node.

The /var/log/httpd/error_log file contains Segmentation fault error messages corresponding to task failures by time.

Cause

httpd service is unstable and faces segmentation faults during processing APS 2.0 REST requests.

Resolution

As a workaround, restart Apache service to stop the errors:

# systemctl restart httpd

To allow future root cause diagnostics:

  • change core pattern:

    # mkdir /tmp/coredumps
    # sysctl -w kernel.core_pattern="/tmp/coredumps/core-%e-sig%s-user%u-group%g-pid%p-time%t"
    # echo 'kernel.core_pattern="/tmp/coredumps/core-%e-sig%s-user%u-group%g-pid%p-time%t"' >> /etc/sysctl.conf
    
  • create /etc/security/limits.d/core.conf file with some reasonable limit, e.g. 10 core dumps:

    *       hard        core        10
    *       soft        core        10
    
  • modify /etc/systemd/system.conf and add:

    DefaultLimitCORE=infinity
    
  • modify /etc/systemd/system/multi-user.target.wants/httpd.service and add:

    LimitCORE=infinity
    
  • commit the changes:

    # systemctl daemon-reexec
    # systemctl stop httpd
    # systemctl start httpd
    

Email subscription for changes to this article
Save as PDF