> Source: [sk132193](https://support.checkpoint.com/results/sk/sk132193) # sk132193 - What is the "Custom Intelligence Feeds" feature? | Property | Value | |----------|-------| | Solution ID | sk132193 | | Date Created | 2018-07-19 | | Last Modified | 2026-08-05 | | Technical Level | General | | Products | Security Gateway | | Versions | R82.10, R82, R81.20, R81.10 (EOS), R81 (EOS) | | OS | Gaia | ## Solution **Table of Contents** * Introduction * Availability * Requirements * Supported Feed Formats * CSV Format (\*.csv) * Custom CSV Format (\*.csv) * STIX Format * Check Point Format * Snort Format * Known Feed Examples * Fetching New Feed in CLI (ioc_feeds) * Making Sure Specific Observable was Loaded (ioc_search) * IP Allow List (Exception List) * IoC Feed Enforcement Behavior * Managing Custom Intelligence Feeds from the Security Management Server * Known Limitations * Troubleshooting * Related Solutions Introduction {#TOC01} --------------------- **Custom Intelligence (IoC) Feeds** feature provides an ability to add custom cyber intelligence feeds into the Threat Prevention engine. It allows fetching feeds from a third-party server directly to the Security Gateway to be enforced by Anti-Virus, Anti-Bot, and IPS Software Blades. The Custom Intelligence Feeds feature also assists customers with the operational and engineering management challenges they face handling indicators: Managing and monitoring of the custom intelligence feeds is done with minimal operational overhead. **Indicator** is a pattern of relevant observable malicious activity in an operational cyber domain, with relevant information on how to interpret it and how to handle it. Indicators are derived from intelligence, self-analysis and/or governments, partners etc. **Observable** is an event or a stateful property that can be observed in an operational cyber domain. For example: IP address, MD5 file signature, URL, Mail sender address. Availability {#TOC02} --------------------- The support for IoC feeds is integrated in the versions R80.30 and higher. **Note:** To import external Custom Intelligence Feeds in SmartConsole **R81 and higher** , refer to the [Threat Prevention Administration Guide](https://support.checkpoint.com/product/417#f-commonsource=C.%20Documentation) for your version \> Chapter "Common Features in Custom Threat Prevention and Autonomous Threat Prevention" \> Section "Configuring Threat Indicators" \> Section "Importing External Custom Intelligence Feeds" \> Section "Importing External Custom Intelligence Feeds in SmartConsole". Requirements {#TOC03} --------------------- * If IoC feeds contain URLs and are hosted on an HTTPS server, you must enable the HTTPS Inspection Software Blade in the Security Gateway / Cluster object. This is because URL is sent as part of the HTTP request, and it can only be seen when traffic is clear (not encrypted), or when HTTPS Inspection is enabled. * To use IoC feeds with Snort rules (supported in the Management Server R82 and higher and Security Gateway R82 and higher), you must enable the IPS Inspection Software Blade in the Security Gateway / Cluster object. * IoC feeds with authentication are supported in the Management Server and SmartConsole R81.20 and higher (PMTR-73043). Supported Feed Formats ---------------------- Each IoC feed file must be in one of the supported formats as described below. ### Supported Feed Formats - CSV Format (\*.csv) {#TOC04_1} Show / Hide this section * Delimiter between fields is comma ',' * Delimiter between records is new line '\\n' * Fields are plain-text * Each record must contain the same number of comma-separated fields * Quoted fields are accepted * The first line/record, prefixed with the pound character '#', is a header that contains column names in each of the fields * If severity or confidence is specified in a record, the Security Gateway assigns the default severity / confidence to be "Medium". Example: > > ``` > #! DESCRIPTION = indi file,,,,,, > "#! REFERENCE = Indicator Bulletin; Feb 20, 2014",,,,,, > # FILE FORMAT:,,,,,, > "# All lines beginning ""#"" are comments",,,,,, > "# All lines beginning ""#!"" are metadata read by the SW",,,,,, > "# UNIQ-NAME,VALUE,TYPE,CONFIDENCE,SEVERITY,PRODUCT,COMMENT",,,,,, > observ1,8d9b6b8912a2ed175b77acd40cbe9a73,MD5,medium,medium,AV,FILENAME:WUC Invitation Letter Guests.doc > observ2,76700f862a0c241b8f4b754f76957bda,MD5,high,high,AV,FILENAME:essais~.swf| > NOTE:FWS type Flash file > observ7,http://somemaliciousdomain.com/uploadfiles/upload/exp.swf?info= > 789c333432d333b4d4b330d133b7b230b03000001b39033b&infosize=00840000 > ,URL,high,high,AV,IPV4ADDR:196.168.25.25 > observ8,svr01.passport.ServeUser.com,Domain,low,high,AB,TCP:80| > IPV4ADDR:172.18.18.25|NOTE:Embedded EXE Remote C&C and Encoded Data > observ9,somemaliciousdomain2.com,Domain,,low,AV,TCP:8080|IPV4ADDR:172.22.14.10 > observ10,http://www.bogusdomain.com/search?q=%24%2B%25&form=MOZSBR&pc= > MOZI,URL,low,low,AB,IPV4ADDR:172.25.1.5 > observ11,http://somebogussolution.com/register/card/log.asp?isnew=-1&LocalInfo= > Microsoft%20Windows%20XP%20Service%20Pack%202&szHostName= > ADAM-E512679EFD&tmp3=tmp3,URL,medium,,AB, > observ14,172.16.47.44,IP,high,medium,AB,TCP:8080 > observ15,172.16.73.69,IP,medium,medium,AB,TCP:443|NOTE:Related to Flash > exploitation > observ16,abc@def.com,mail-to,,high,AV,"NOTE:truncated; samples have appended to > the subject the string ""PH000000NNNNNNN"" where NNNNNNN is a varying number" > observ34,stamdomain.com,domain,,,AB, > observ35,stamdomain.com,mail-from,high,medium,AV, > observ37,xyz.com,mail-from,medium,medium,AB, > observ38,@xyz.com,mail-from,medium,medium,AB, > ``` ### Supported Feed Formats - Custom CSV Format (\*.csv) {#TOC04_2} Show / Hide this section The IoC Feeds feature supports different kinds of CSV structure files. For the fetch to succeed, you must define the file's format, delimiter, and the comment lines to skip. **Syntax Notes:** * The supported fields are: * `name` * `value` * `type` * `confidence` * `severity` * `product` * `comment` * To use a field value from the original file (must be one of the supported fields listed above), you must specify its location in the CSV row by using *#index*. For example, if you want to take the 3rd index in your CSV file to be your observable's comment in the new file, then use: `--format [comment:#3]` * To use a default value for all observables, use: `--format [type:domain]` * The "`Value`" field is mandatory and must be taken from the original file. * The "`Type`" field is mandatory and must be taken from the original file or be sent as a default value for all observables. * All other fields are optional and can either be taken from the original file or sent as a default value for all observables. * When the IoC feed's resource is a remote source (the transport is HTTP or HTTPS) - every time the Security Gateway fetches the IoC feed, it parses it based on the format that has been specified for this feed. **Examples:** * **Original CSV structure is a list of IP addresses:** Syntax: `ioc_feeds add --feed_name ip_list --transport http --resource "http://blocklist.greensnow.co/greensnow.txt" --format [value:1,type:ip]` Structure (shows only 3 rows): `104.156.155.12` ` 125.34.171.202` ` 120.55.99.107` ` ... ... ...` * **Original CSV structure is a list of Domains:** Syntax: `ioc_feeds add --feed_name domain_list --transport https --resource "https://urlhaus.abuse.ch/downloads/text/" --format [type:domain,value:1] --comment "#, Site"` Structure (shows only 4 rows): `# This list consists of High Level Sensitivity website URLs` ` # Columns (tab delimited):` ` #` ` # (1) site` ` #` ` Site` ` 4kqd3hmqgptupi3p.k7oud1.top` ` stgg5jv6mqiibmax.torshop.li` ` googlepetkavanis4.pw` ` jazmany.cu.ma` ` ... ... ... ` This structure can hold a sub-domain in the "`Site`" column. For example, adding a row with the Site value "`cu.ma`" will enforce any domain containing this subdomain "`jazmany.cu.ma`", "`a.b.cu.ma`", and so on. * **Original CSV structure is a list of IP addresses in CIDR format and comment lines are marked as ';':** Syntax: `ioc_feeds add --feed_name ip_cidr_list_with_delimiter --transport https --resource "https://www.spamhaus.org/drop/drop.txt" --format [value:1,type:ip] --delimiter ";" --comment ";"` Structure (shows only 9 rows): ` ; Spamhaus DROP List 2024/12/09 - (c) 2024 The Spamhaus Project SLU` ` ; https://www.spamhaus.org/drop/drop.txt` ` ; Last-Modified: Mon, 09 Dec 2024 11:54:42 GMT` ` ; Expires: Mon, 09 Dec 2024 13:09:09 GMT` ` 1.10.16.0/20 ; SBL256894` ` 1.19.0.0/16 ; SBL434604` ` 1.32.128.0/18 ; SBL286275` ` 2.56.192.0/22 ; SBL459831` ` 2.57.122.0/24 ; SBL636050` ` 2.57.168.0/24 ; SBL628448` ` 2.57.232.0/23 ; SBL538946` ` 2.57.234.0/23 ; SBL538947` ` 5.42.92.0/24 ; SBL625300` ` ... ... ... ` * **Original CSV structure is a list of IP addresses separated by the '\|' delimiter, and comment lines are marked as '#':** Syntax: `ioc_feeds add --feed_name ip_list_with_spaces --transport local_file --resource "/home/admin/ioc/ip_list_with_spaces.txt" --format [value:3,comment:#2,type:ip] --comment [#] --delimiter "|"` Structure (shows only 3 rows): `# category Descriptive tag name for this entry. For this report,` ` # the text sshpwauth will appear.` ` #` ` # A commented footer section show an aggregate cound of ASNs and` ` # addresses seen in the current report.` ` #` ` NA | NA | 186.249.213.118 | 2018-09-15 19:43:37 | sshpwauth` ` 3 | MIT-GATEWAYS - Massachusetts I | 18.30.10.26 | 2018-09-15 08:16:39 | sshpwauth` ` 111 | BOSTONU-AS - Boston University | 128.197.31.119 | 2018-09-14 17:00:57 | sshpwauth` ` ... ... ...` * **Original CSV structure is a list of different types separated by the ',' delimiter, and comment lines are marked as '#' or 'Site':** Syntax: `ioc_feeds add --feed_name try_custom_csv --transport http --resource http://192.168.13.13/ioc/bad_csv_format.csv --format [type:#1,value:#3,name:#6,comment:#7,product:av] --comment [#, Site] --delimiter ,` Structure (shows only 6 rows): `domain,example2,www.ynet.co.il/articles/,example3,example4,observ1,Ynet articlesAccess,AV,medium,example5,example6,example7#comment1` ` url,example2,www.ynet.co.il/articles/,example3,example4,observ2,Ynet articlesAccess,AV,medium,example5,example6,example7#comment2` ` ip,example2,192.168.4.44,example3,example4,observ3,Ynet articlesAccess,AB,medium,example5,example6,example7` ` ip range,example2,192.168.4.44-192.168.4.45,example3,example4,observ4,Ynet articlesAccess,AB,medium,example5,example6,example7` ` md5,example2,806d61dca2f872b64d1500fdd129aa40,example3,example4,observ5,Ynet articlesAccess,AB,medium,example5,example6,example7` ` Mail-subject,example2,malicious-email@gmail.com,example3,example4,observ6,Ynet articlesAccess,AB,medium,example5,example6,example7` * **Original CSV structure is a list of different types separated by the ',' delimiter:** Syntax: `ioc_feeds add --feed_name ioc_feed --transport http --resource "/home/admin/ioc/txtmalwaredomainlist.com/updatecsv.php" --format [value:3,comment:#2,type:ip] --delimiter ,` ### Supported Feed Formats - STIX Format {#TOC04_3} Show / Hide this section STIX? is a collaborative community-driven effort to define and develop a standardized language to represent structured cyber threat information. For more information, refer to [STIX 1.x Archive Website](https://stixproject.github.io). Supported Observable Types: * URL * Domain * This type of observable support also super-domain enforcement. For example, adding "com" as super-domain will enforce any domain containing it - "a.b.com", or "a.com". * IP * IP Range * MD5 * Mail-subject * Mail-from * Mail-to * Mail-cc * Mail-reply-to * SHA1 (requires Security Gateway versions R80.40 and higher) * SHA256 (requires Security Gateway versions R80.40 and higher) ### Supported Feed Formats - Check Point Format {#TOC04_4} Show / Hide this section **Check Point format has the following structure:** > `#UNIQ-NAME,VALUE,TYPE,CONFIDENCE,SEVERITY,PRODUCT,COMMENT` > ` > observ1,www.websitename.com,URL,medium,low,AV,Website articles Accessobserv2,192.168.38.187,IP,high,high,AB,C&C server IP` > `observ4,b04552c6ede6c7f47327f304a6ac1a20,MD5,low,high,AV,Security.doc` **Check Point format syntax in R81.20 and higher:** > Check Point format syntax includes only the combination of type and blade according to the list below (observable types and blades). > > In case of incorrect blade for the observable type, the current observable will not be loaded. |-----------------|----------------|--------------------------| | Observable Type | Software Blade | Full Software Blade Name | | URL | AV/AB | Anti-Virus / Anti-Bot | | Domain | AV/AB | Anti-Virus / Anti-Bot | | IP | AB | Anti-Bot | | IP Range | AB | Anti-Bot | | MD5 | AV | Anti-Virus | | Mail-subject | AV/AB | Anti-Virus / Anti-Bot | | Mail-from | AV/AB | Anti-Virus / Anti-Bot | | Mail-to | AV/AB | Anti-Virus / Anti-Bot | | Mail-cc | AV/AB | Anti-Virus / Anti-Bot | | Mail-reply-to | AV/AB | Anti-Virus / Anti-Bot | | SHA1 | AV | Anti-Virus | | SHA256 | AV | Anti-Virus | | Snort | IPS | IPS | ### Supported Feed Formats - Snort Format {#TOC04_5} Show / Hide this section Syntax: See the official documentation at [snort.org](https://snort.org/). Requirements: 1. Management Server R82 and higher. 2. Security Gateway / Cluster / Scalable Platform Security Group R82 and higher. Configuration: 1. Enable the **IPS** Software Blade in the Security Gateway / Cluster object. 2. Configure the IoC Feed object. Click "**Test Feed**" to see how many supported observables can be loaded onto the Security Gateway / Cluster Members. 3. Install the Threat Prevention policy. Known Feed Examples (using the Custom CSV format) {#TOC05} ---------------------------------------------------------- |--------------------------|--------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **Name** | **URL** | **Command Line** | | Alienvault IP Reputation | http://reputation.alienvault.com/reputation.data | *ioc_feeds add --feed_name reputation --transport http --resource "http://reputation.alienvault.com/reputation.data" --format \[type:ip,value:#1,comment:#4\] --delimiter "#"* | | Domains | https://www.botvrij.eu/data/ioclist.hostname.raw | *ioc_feeds add --feed_name domains --transport https --resource "https://www.botvrij.eu/data/ioclist.hostname.raw" --format \[type:domain,value:#1\]* | | Spam List | http://www.ipspamlist.com/public_feeds.csv | *ioc_feeds add --feed_name spam_list --transport https --resource "https://www.ipspamlist.com/public_feeds.csv" --format \[type:ip,value:#3,comment:#4\] --comment \["#", "first_seen"\] --delimiter ","* | | Cybercrime hash list | http://cybercrime-tracker.net/ccamlist.php | *ioc_feeds add --feed_name hash_list --transport http --resource "http://cybercrime-tracker.net/ccamlist.php" --format \[type:sha1,value:#1\]* | Fetching New Feed in CLI (ioc_feeds) {#TOC06} --------------------------------------------- IoC Feed's resource can be one of these: * URL - HTTP/HTTPS (example: *--transport http --resource "http://10.0.0.1/my_feeds/stix_feed.xml"*) * File on the machine (example: *--transport local_file --resource "/home/admin/my_feed.csv"*) * Directory on the machine (example: *--transport local_directory --resource "/home/admin/my_feed_folder"*) **Syntax:** |--------------------------------| | `ioc_feeds {-h | --help}` | | `ioc_feeds ` | **Actions and Options:** Enter the string to filter this table: |-------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Action / Option | Description | Example | | `-h` `--help` | Shows the built-in help. | *ioc_feeds --help* | | `add` | Adds a new feed. |-------------------------------------------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Sub-Parameter | Type | Description | | `--feed_name ` | Mandatory | Configures the feed name. | | `--transport {http | https | local_file | local_directory}` | Mandatory | Configures the feed transport: * `http` Downloads the feed over HTTP * `https` Downloads the feed over HTTPS * `local_file` Loads the feed from a file (must be inside the `/home/` directory) * `local_directory` Loads the feeds from a local directory (must be inside the `/home/` directory) | | `--resource {url | full_path}` | Mandatory | Configures the feed source: * `url` Downloads the feed from a URL * `full_path` Load the feed from a local path (must be inside the `/home/` directory) | | `--state {true | false}` | Optional | Configures the feed state: * `true` Feed is active (default) * `false` Feed is inactive | | `--feed_action {prevent | detect}` | Optional | Configures the Security Gateway action: * `prevent` Blocks traffic (default) * `detect` Allows traffic **Important Note:** * In Security Gateway versions R82 and higher, the final Security Gateway action is based on the confidence and severity in the Threat Prevention Profile. * In Security Gateway versions R81.20 and lower, the final Security Gateway action is based on the IoC Feed action. | | `--user_name ` | Optional | Configures a username for the remote feed (shows a prompt for a password). | | `--proxy :` | Optional | Configures a proxy a server for fetching the feed. This proxy overrides the proxy configured for the Security Gateway. | | `--proxy_user_name ` | Optional | Configures a username for a proxy server for fetching the feed (shows a prompt for a password). | | `--test true` | Optional | Performs a test run - tries to fetch and parse the feed. |
| *ioc_feeds add --feed_name LocalCsvFeed --transport local_file --resource /home/admin/my_feed.csv*
*ioc_feeds add --feed_name LocalStixFeed --transport local_file --resource "/home/admin/my_feed.xml"*
*ioc_feeds add --feed_name RemoteCsvFeed --transport http --resource "http://10.10.1.100/ioc/my_feed.csv" --feed_action prevent*
*ioc_feeds add --feed_name RemoteStixFeed --transport http --resource http://10.0.0.1/ioc/my_feed.xml --proxy 127.10.10.1:8080 --state false --feed_action detect --user_name user1@example.com*
*ioc_feeds add --feed_name RemoteStixFeed --transport http --resource "http://www.public_indicators.com/ioc/my_feed.xml" --test true* | | `delete ` | Deletes the existing feed. | *ioc_feeds delete --feed_name MyLocalFeed* | | `modify` | Modifies the existing feed. Fields that are not specified in the syntax will continue to use the previous values. | *ioc_feeds modify --feed_name MyLocalFeed --state true* | | `push` | Pushes the feeds immediately. | *ioc_feeds push* | | `show [--feed_name ]` | Shows all configured feeds, or the specified feed. | *ioc_feeds show* *ioc_feeds show --feed_name MyLocalFeed* | | `export` | Exports the configured IoC feeds and saves them in the file `export_ioc.tar.gz` in the current working directory. | *ioc_feeds export* | | `import ` | Imports the IoC feeds from the exported file - `export_ioc.tar.gz` | *ioc_feeds export /home/admin/export_ioc.tar.gz* | | `show_interval` | Shows the fetching interval for the configured IoC feeds. | *ioc_feeds show_interval* | | `set_interval ` | Configures the interval for fetching all the configured IoC feeds (in seconds). | *ioc_feeds set_interval 1000* | | `show_scanning_mode` | Shows whether the scanning mode for external indicators is enabled (on) or disabled (off). | *ioc_feeds show_scanning_mode* | | `set_scanning_mode {on | off}` | Enables (`on`) or disables (`off`) the scanning. | *ioc_feeds set_scanning_mode off* | | `--yes` | Prevents all user interactions - configures the answer "*yes* " for all "*yes / no*" questions. This allows to run the `ioc_feeds` command in shell scripts. | | | `--no_proxy` | Disables all configures proxy servers for fetching feeds (the proxy configured for the Security Gateway and the proxy configured for the specific feeds). | | | `--self_sign_certificate ` | Configures the local path to a self-signed public certificate file. | | | `--format []` | Configures the format of the IoC feed file. | *ioc_feeds add --feed_name MyLocalFeed --transport localfile --resource /home/admin/IoC/MyLocalFeed.txt --format value:1,type:ip* *ioc_feeds add --feed_name MyLocalFeed --transport localfile --resource /home/admin/IoC/MyLocalFeed.txt --format \[type:#1,value:#3,name:#6,comment:#7,product:av\]* | | `--delimiter ""` | Configures the delimiter character that is used in the IoC feed file in a custom format. | *--delimiter "#"* | | `--comment []` | Configures the comment character that is used in the IoC feed file in a custom format. | *--comment "#"* *--comment \[#, Site\]* | | `--feed_file_type {cp_csv | stix_1.x | custom_csv | snort}` | Configures the type of a feed file: * `cp_csv` - Check Point CSV * `custom_csv` - Custom CSV * `stix_1.x` - STIX v1.x * `snort` - Snort | | | `--severity {1 | 2 | 3 | 4}` | Configures the severity for a feed of Snort rules: * 1 = Low * 2 = Medium * 3 = High * 4 = Critical | | | `--confidence {1 | 2 | 3 | 4 | 5}` | Configures the confidence for a feed of Snort rules: * 1 = Low * 2 = Medium-Low * 3 = Medium * 4 = Medium-High * 5 = High | | | `--performance_impact {1 | 2 | 3 | 4 | 5}` | Configures the performance impact for a feed of Snort rules: * 1 = Very-Low * 2 = Low * 3 = Medium * 4 = High * 5 = Critical | | {#Unique_ID_CLI_Table} **Important Note for Security Groups on Maestro / Scalable Chassis R80.20SP and higher:** The local feed content is refreshed periodically based on the feed fetching interval (see "`--set_interval`" above). When you modify the local feed source file, you must distribute the updated file to all Security Group Members with the "`asg_cp2blades`" command in the Expert mode: 1. `vi /path/to/file.extension` 2. `asg_cp2blades /path/to/file.extension` Making Sure Specific Observable was Loaded (ioc_search) {#TOC07} ---------------------------------------------------------------- Starting in Security Gateway R82, you can use the `ioc_search` command to check if an observable from external feeds was loaded correctly. The command output shows the components that are related to the specific observable type. The search starts from checking the parsed product of the IoC feed, so for more accurate result use the whole value of the observable as it appears in the feed. **Syntax:** |----------------------------------------------------------------| | `ioc_search {ip | hash | snort | mail | url | domain} ` |
**CLI Parameters:** |---------------|------------------------------------------------------------|----------------------------------------------------| | Parameter | Description | Example | | `-h` `--help` | Shows the built-in help. | *ioc_search -h* | | `ip` | Searches for observables of the type "IP Address". | *ioc_search ip 1.1.1.1* | | `hash` | Searches for observables of the type "File Hash". | *ioc_search hash 1a79a4d60de6718e8e5b326e338ae533* | | `snort` | Searches for observables of the type "Snort Rule Message". | *ioc_search snort "Block pdf files over HTTP"* | | `mail` | Searches for observables of the type "Email Address". | *ioc_search mail user@example.com* | | `url` | Searches for observables of the type "URL". | *ioc_search url http://example.com/file.exe* | | `domain` | Searches for observables of the type "Domain Address". | *ioc_search domain example.com* | {#Unique_ID_CLI_Table} **R82.10 AND BELOW:** For example, IP enforcement relies on SecureXL ("FWACCEL"). So, if you want to check if the IP observable was loaded correctly for the specified IP address, you will get this output: `[Expert@MyGateway:0]# ioc_search ip 1.1.1.1` `Component Exists` `-------------------------------` `Feed........................YES` `DB..........................YES` `FWACCEL.....................YES` **Known limitations of the 'ioc_search' tool:** * The 'ioc_search' tool supports only external IoC feeds. * The 'ioc_search' tool does not support the IP Range observable. IP Allow List (Exception List) {#TOC08} --------------------------------------- The IP Allow List (Exception List) is available in these versions: * R82 and R82.10 ONLY * [Jumbo Hotfix Accumulator for R81.20](https://sc1.checkpoint.com/documents/Jumbo_HFA/R81.20/Default.htm) starting from Take 43 The IP allow list provides a convenient way to allow certain IP addresses to bypass the enforcement actions, that have been defined by threat intelligence feeds. Follow these steps to managing IP addresses within the IP allow list: 1. **Edit the File**: `vi $FWDIR/conf/ip_whitelist.eng` 2. **Append IP Addresses**: Add the desired IP addresses, one IP address in each line. 3. **Save the Changes**: Ensure you save the file after adding the IP addresses. **Fetch the Feeds** After adding the needed IP's to the list use # ioc_feeder -f (on each cluster member) to update the ip_whitelist.eng file into the ioc_feeder engine.
**Exemption from Enforcement** : IP addresses listed in the `$FWDIR/conf/ip_whitelist.eng` file will not be subjected to enforcement actions even if they appear in any of the threat intelligence feeds. IoC Feed Enforcement Behavior {#TOC09} -------------------------------------- IoC Feed enforcement is intended to enforce indicators from external or local intelligence feeds through the Threat Prevention engine. For IP and IP Range observables, IoC Feed enforcement is based on the SecureXL DoS deny-list infrastructure. This enforcement works in a stateless manner at the accelerator layer. The enforcement decision is based on a packet match against the loaded IoC observable. It does not inspect the connection state, session origin, or traffic direction in the same way as the Access Control rulebase. As a result, when an IP address is included in an IoC Feed and enforcement is enabled, an incoming packet from that IP can be dropped on the applicable interface even if the packet is: * A new inbound connection initiated by the IoC-listed IP address. * Return traffic for a connection that was initiated by an internal host. * Part of an already established connection. **Important:** IoC Feeds do not provide stateful Access Control behavior. IoC Feeds cannot distinguish between a new inbound connection from an IoC-listed IP address and return traffic from the same IP address for a connection that was initiated by an internal host. **Example:** * An IoC Feed contains the IP address `8.8.8.8`. * An internal client initiates a connection to `8.8.8.8`. * The outbound traffic can be accepted according to the policy and configuration. * The return traffic from `8.8.8.8` can still be prevented when it arrives on an interface where IoC Feed enforcement applies. This behavior is expected by design. **Unsupported use cases:** * Using IoC Feeds as a stateful replacement for Network Feed objects in the Access Control rulebase. * Blocking only new inbound sessions from listed IP addresses while allowing return traffic from the same IP addresses for internally initiated sessions. * Configuring IoC Feeds to allow only established or return traffic while still blocking new inbound traffic from the same listed IP addresses. * Using the IP Allow List to allow only return traffic for internally initiated connections while still blocking new inbound traffic from the same IP addresses. If the required behavior is to block new inbound sessions from listed IP addresses, but still allow internal clients to initiate connections to those IP addresses and allow the return traffic, use Network Feed objects in the Access Control rulebase or another stateful Access Control design (please consult your Check Point local office on rule base tailoring and design). If Network Feed scale is a concern, review the required architecture with the local Check Point office or account team to evaluate supported design alternatives. Managing Custom Intelligence Feeds from the Security Management Server {#TOC09} ------------------------------------------------------------------------------- * In the Management Server versions R81 and higher, to manage IoC Feeds in SmartConsole, refer to the [Threat Prevention Administration Guide](https://support.checkpoint.com/product/417#f-commonsource=C.%20Documentation) for your version \> Chapter "Configuring Advanced Threat Prevention Settings" \> Section "Configuring Threat Indicators" \> Section "Importing External Custom Intelligence Feeds" \> Section "Importing External Custom Intelligence Feeds in SmartConsole". * In the Management Server versions R80.30 and R80.40 only, you can distribute the IoC feeds from the Management Server with these steps: 1. Run on one of your Security Gateways, on which the IoC feeds were fetched successfully: `ioc_feeds export` This command saves the required configuration in the file `export_ioc.tar.gz` in your home directory. 2. Move the `/home/admin/export_ioc.tar.gz` file from the Security Gateway to the Security Management Server. 3. Connect with SmartConsole to the Security Management Server. 4. In the left bottom corner of the navigation panel, click **Command Line**. 5. Run the required commands (copy these commands and paste them one by one): 1. `put-file file-path "/home/admin" file-name "export_ioc.tar.gz" file-content @"/home/admin/export_ioc.tar.gz" --treat-value-as-file-by-prefix @ targets.1 "corporate-gateway" targets.2 "Backup-GW"` 2. `run-script script-name "Import Custom feeds" script "ioc_feeds import /home/admin/export_ioc.tar.gz" targets.1 "corporate-gateway" targets.2 "Backup-GW"` 6. Close the **Command Line** window. 7. You can configure the feed fetch interval in SmartConsole (applies to all IoC feeds): 1. From the left navigation panel, click **Manage \& Settings** 2. Click **Blades** 3. In the **Threat Prevention** section, click **Advanced Settings** 4. Click the **External Feed** page 5. Configure the applicable interval 6. Click **OK** 7. Install the Threat Prevention Policy ![](https://sc1.checkpoint.com/sc/SolutionsStatics/sk132193/feed_interval202309131429081.png) Known Limitations {#TOC10} -------------------------- * Observables of IP addresses: * In the versions R81 and higher: Observables of IP addresses can hold IPv4 and IPv6 values. * In the versions R80.40 and lower: Observables of IP addresses can hold IPv4 values only. IP feeds are enforced only for the destination IP addresses. * Observables of IP Ranges: * In the versions R81 and higher: Observables of IP Ranges can hold IPv4 and IPv6 values. * In the versions R80.40 and lower: Observables of IP Ranges can hold IPv4 values only. IP feeds are enforced only for the destination IP addresses. * The Anti-Bot Software Blade cannot enforce the MD5, SHA1, and SHA256 observables. To enforce these observables, enable the Anti-Virus Software Blade. * Only these versions for Scalable Platforms (Maestro and Chassis) support IoC feeds: * R81.10 and higher (PMTR-65769). * [R80.20SP Jumbo Hotfix Accumulator](https://support.checkpoint.com/results/sk/sk155832), Take 283 or higher (MBS-13295). * Inbound traffic to a host behind the Security Gateway does not get blocked (PRHF-30512). Example: If an IP address that is configured in a feed, sends ICMP Request to a host behind the Security Gateway, then the Security Gateway does not block this traffic. This problem was fixed. The fix is included starting from: * [Jumbo Hotfix Accumulator for R81.10](https://sc1.checkpoint.com/documents/Jumbo_HFA/R81.10/Default.htm) starting from Take 131 (PRJ-49877) * Large feeds can take a long time to load on the **ext3** filesystem. For additional information, see [sk141432](https://support.checkpoint.com/results/sk/sk141432). * In the versions R82 and higher: * IoC Feed objects support Snort version 2.9 or lower. * IoC Feed with Snort rules is limited to 3000 observables and 6000 rules in total. The Security Gateway will not load a larger IoC Feed. * In the versions R81.20 and higher: To prevent the CPU overload, the feed is not loaded if its size exceeds one of these: * 80% of the total free disk space * 50% of the free RAM * In the versions R81.10 and lower: There is a 1-million-character limit for the number of observables. * The "`ioc_feeds export`" command is supported only in the R80.30 and R80.40 versions. * IOC External feeds are not supported over MTA * In the versions R82 and higher: Observables are loaded only if at least one of the supported Software Blades for that observable type is enabled on the Security Gateway. * The IP exclusion feature is not supported in SmartConsole. * TAXII (Trusted Automated eXchange of Intelligence Information) format is not supported. * SmartConsole R81.10 and R81 shows the error "`Unauthorized: Access is denied due to invalid credentials (401)`" when clicking "**Test Connectivity** " in an IoC Feed object. This issue PMTR-73043 was resolved in the R81.20 SmartConsole. Note - The button "Test Connectivity" was renamed to "Test Feed". * In R81.20, the number of IP observables supported in External IoC has increased significantly compared with R81.10. We currently certify support for up to 2 million IP addresses without affecting system resources. As an immediate mitigation, you can limit the number of observables loaded by adjusting the relevant parameter in the `malware_config` file. * For unsupported SNORT syntax please refer to [Snort Signature Support](https://sc1.checkpoint.com/documents/R82.10/WebAdminGuides/EN/CP_R82.10_ThreatPrevention_AdminGuide/Content/Topics-TPG/SNORT-Signature-Support.htm?Highlight=snort%20protections). * SNORT related limitations: * Opening a Snort protection link from a SmartConsole log may result in an error. * The configured Feed Action does not apply to Snort rules; enforcement is determined only by the Threat Prevention profile configuration. * After policy installation, Snort feed enforcement starts only after all feeds have fully loaded, especially after adding a new Snort feed. * A standby member cannot download Snort feeds, meaning it cannot fetch the feed from the configured URL. Troubleshooting {#TOC11} ------------------------ **Symptom:** The Security Gateway / Cluster log in SmartConsole / SmartView shows "`Feed load failed`". Example: ![](https://sc1.checkpoint.com/sc//SolutionsStatics/sk181208/fail_log202307090932482.png) **Solution:** 1. Connect to the command line on the Security Gateway / Cluster Member. 2. Log in to the Expert mode. 3. Fetch IoC feeds in the debug mode and examine the output: `$FWDIR/bin/ioc_feeder -d -f` 4. Make sure the configuration file `$FWDIR/conf/ioc_feeder.conf` exists and that it is not corrupt: `cat $FWDIR/conf/ioc_feeder.conf` If the file is corrupt, then in SmartConsole: 1. Delete the IoC Feed object. 2. Configure the IoC Feed object again. 3. Install the Threat Prevention policy. 5. Examine these log files on the Security Gateway / Cluster Member for errors: `$FWDIR/log/ioc_feeder.elg*` ### General Troubleshooting on the Security Gateway / Cluster Members * Run this command on the Security Gateway to fetch feeds in the debug mode: `$FWDIR/bin/ioc_feeder -d -f` Examine this log file: *$FWDIR/log/ioc_feeder.elg* * Make sure the configuration file ***$FWDIR/conf/ioc_feeder.conf*** exists and is not corrupt: `cat -n $FWDIR/conf/ioc_feeder.conf` If the file is corrupt, delete the IoC feed and add it again with the required changes. * Make sure there are no errors in these log files: * ***$FWDIR/log/ioc_feeder.elg*** * ***$FWDIR/log/ext_ioc_push.elg*** * For remote IoC feeds (over HTTP / HTTPS): * Web servers might return files with unauthorized or forbidden headers as a response. Make sure that file is fetched as expected and contains the correct information, whether it is a STIX file or a CSV file, in this directory: ***$FWDIR/external_ioc/\*** * For HTTPS remote IoC feeds, if the certificate update process fails, you can skip the SSL certificate verification on the Security Gateway with the special environment variable: * To skip the SSL certificate verification **in the current session** (does not survive reboot), run: 1. `export EXT_IOC_NO_SSL_VALIDATION=1` 2. `ioc_feeds