> Source: [sk82420](https://support.checkpoint.com/results/sk/sk82420)

# sk82420 - Troubleshooting "Too many open files" error

| Property | Value |
|----------|-------|
| Solution ID | sk82420 |
| Date Created | 2012-08-16 |
| Last Modified | 2018-12-02 |
| Technical Level | General |
| Products | Security Gateway, Security Management Server, Multi-Domain Security Management Server, Endpoint Security |
| Versions | R82.10, R82, R81.20, Cloud, R82.20, E89.X, E88.X |

## Solution

File Descriptors
================

There is a limit to the number of HTTP connections that one daemon can handle. This is because there is a UNIX limit to the number of open file descriptors that any one process may have. When started, *in.ahttpd* daemon will attempt to reserve the maximum number of file descriptors allowed by the OS. An HTTP connection going through the Security Server requires two sockets: one for the connection from the client and one for the client to the server. Each socket requires a file descriptor. **When the maximum number of file descriptors has been reached, a "*Too many open files* " error is logged in the *FWDIR/log/ahttpd.log* file.**

For a more comprehensive discussion, see [sk40782: How to improve the performance of the HTTP Security Server](http://supportcontent.checkpoint.com/solutions?id=sk40782).

Various Scenarios
=================

There are quite a number of scenarios, in which you may encounter the "*Too many open files*" error. The scenarios that we have encountered and dealt with are detailed below:

1. "`Too many open files`" error on Security Gateway running SecurePlatform OS
2. *$FWDIR/log/fwm.elg* shows "`Failed to open file '`*File_Name*`': Too many open files`"
3. Policy compilation fails with "`Failed to Download Security Policy on XXX: Too many open files`"
4. "`fwasync_accept: accept(9) failed: Too many open files`" in *cpca.elg* file

*** ** * ** ***

### Scenario 1: "`Too many open files`" error on Security Gateway running SecurePlatform OS {#Scenario 1}

**Product:** Security Gateway

**OS:** SecurePlatform 2.4, SecurePlatform 2.6

**Symptoms:**

* "`Too many open files`" error on SecurePlatform.

**Cause:**

The system has run out of available file handles to open files.

**Solution:**

*Introduction:*

Under some circumstances limit of available file handles to open files can be reached. It is possible that the system is simply very busy, or there might be an underlying problem, where the system is leaving files open and not returning the file handle to the pool.

*Procedure:*

**Check the number of currently opened files:**

*\[Expert@HostName\]# cat /proc/sys/fs/file-nr*

This read-only file contains three numbers:

* First (left-most) = the number of allocated file handles
* Second (in the middle) = the number of free file handles
* Third (right-most) = system-wide limit on the number of open files for all processes - the maximum number (total available) of file handles

**Note:** Linux 2.6 always reports 0 as the number of free file handles - this is not an error, it just means that the number of allocated file handles exactly matches the number of used file handles.

The kernel allocates file handles dynamically, but it does not free them again. If the number of allocated files is close to the maximum, consider increasing the maximum.

When the number of free file handles is large, it means that a peak was encountered in the usage of file handles and probably the maximum does not need to be increased.

**To increase the maximum number of file handles *on-the-fly*, run this command:**

*\[Expert@HostName\]# echo 'NEW_FILE_MAX_NUMBER_IN_DEC_FORMAT' \> /proc/sys/fs/file-max*

**To increase the maximum number of file handles *permanently*:**

Edit the value of '*fs.file-max* ' parameter in the */etc/sysctl.conf* file and reboot the machine..

***Example*:**

The maximum number of file handles is set to 65535 (the default), and it should be doubled - increased to 131070.

**To increase the maximum number of file handles *on-the-fly*, run this command:**

*\[Expert@HostName\]# echo '131070' \> /proc/sys/fs/file-max*

**To increase the maximum number of file handles *permanently*:**

Set the value of '*fs.file-max* ' parameter in the */etc/sysctl.conf* file to be 131070 and reboot the machine.

**Reference:**

* <http://linux.die.net/man/5/proc>

*** ** * ** ***

### Scenario 2: *$FWDIR/log/fwm.elg* shows "`Failed to open file '`*File_Name*`': Too many open files`" {#Scenario 2}

**Product:** Security Management Server

**OS:** SecurePlatform 2.4

**Symptoms:**

* *$FWDIR/log/fwm.elg* shows "`Failed to open file '`*File_Name*`': Too many open files`"
* Running '`cpstop`' and then '`cpstart`' commands solves the issue only temporarily.

**Cause:**

Each time that the user connects to the Security Management Server with SmartConsole clients, several file descriptors are opened for */var/opt/CPshrd-R65/conf/*, and they are not closed when SmartConsole clients disconnect.

**Solution:**

**Note:** Before you implement the solution, you should be able to see that the opened file descriptors number is increasing each time that you connect with SmartConsole clients, and that many of those file descriptors are linked to */opt/CPshrd-R65/conf*/ by running:

1. *\[Expert@HostName\]# cpwd_admin list \| grep -E "PID\|FWM"*
2. *\[Expert@HostName\]# ls -la /proc/\<PID_of_FWM_daemon\>/fd \| wc -l*

To resolve the issue, proceed as follows:

1. Backup the Check Point Registry file:

   *\[Expert@HostName\]# cp $CPDIR/registry/HKLM_registry.data $CPDIR/registry/HKLM_registry.data_orig*   

2. Stop the Security Management Server:

   *\[Expert@HostName\]# cpstop*   

3. Edit the Check Point Registry file:

   *\[Expert@HostName\]# vi $CPDIR/registry/HKLM_registry.data*   

4. Change the line with "*CertPath*":

   from *:CertPath ("/opt/CPshrd-R65/conf")*
   to *:CertPath ("/opt/CPshrd-R65/conf/**sic_cert.p12**")*
5. Start the Security Management Server:

   *\[Expert@HostName\]# cpstart*

*** ** * ** ***

### Scenario 3: Policy compilation fails with "`Failed to Download Security Policy on XXX: Too many open files`" {#Scenario 3}

**Product:** Security Management

**OS:** SecurePlatform 2.4, SecurePlatform 2.6

**Symptoms:**

* Policy installation fails with:  
  "`Failed to Download Security Policy on XXX: Too many open files`  
  `Compilation failed`"
* Output of "*ls -l /proc/\<PID_of_FWM_daemon\>/fd* " command shows that 4 open file descriptors are added */var/opt/CPshrd-R77/conf/* during each policy installation, each opening / saving of a policy.

**Cause:**

Management certificate has partial corruption and is causing file descriptor leak.

**Solution:**

Check if opened file descriptors number is increasing:

1. *\[Expert@HostName\]# cpwd_admin list \| grep -E "PID\|FWM"*
2. *\[Expert@HostName\]# ls -la /proc/\<PID_of_FWM_daemon\>/fd \| wc -l*

If the number is close to "1024" (the default limit), then this is probably the cause.

To resolve the issue, recreate the ICA:

1. On the SmartCenter, reset Internal certificate:

   *\[Expert@HostName\]# fwm sic_reset*   

2. Recreate Internal certificate:

   1. *\[Expert@HostName\]# cpconfig*   

   2. Select "*Certificate Authority*"

   <br />

   <br />

3. Restart SmartCenter:

   *\[Expert@HostName\]# cpstop ; cpstart*

*** ** * ** ***

### Scenario 4: "fwasync_accept: accept(9) failed: Too many open files" in *cpca.elg* file {#Scenario 4}

**Product:** Security Management

**OS:** SecurePlatform, Gaia OS

**Platform:** VMware

**Symptoms:**

* High CPU with *cpca* and *fw_ful* processes
* *cpca.elg* file contains the following messages:  
  * "fwasync_accept: accept(9) failed: Too many open files"
  * "SvcSk_new_connection: accept(9) failed: Too many open files"

**Cause:**

High logging causing processes to run high.

**Solution:**

* Reduce total logging going to the VMware environment.
* Upgrade the VMware robust hardware.

---

# Agent Instructions

This content is from the Check Point Support Center (https://support.checkpoint.com), the official knowledge base for Check Point cybersecurity products.

## Navigating This Knowledge Base

- **Complete index**: [llms.txt](https://support.checkpoint.com/llms.txt)
- **All SK articles**: [SecureKnowledge Sitemap](https://support.checkpoint.com/sitemaps/secureknowledge-sitemap-index.xml)
- **SK article URL pattern**: `https://support.checkpoint.com/results/sk/{skId}`
- **Markdown responses**: AI bot User-Agents automatically receive `text/markdown` content