Recorded Future Collective Insights for Exabeam

Introduction
Recorded Future's Collective Insights is a new type of Recorded Future analytics, providing clients with a complete view of what threats matter to an organization. Collective Insights enables Recorded Future’s clients to analyze detected incidents to create an intelligence resource for use in two ways:

• Collective Insights provides clients with a comprehensive view of detections across their infrastructure and controls.
• Anonymized data can be used to create visualizations and analytics comparing threat vectors and the entire threat landscape for your enterprise anonymously compared to specific industries and geographies.

For more information on Recorded Future’s Collective Insights, please see the Getting Started with Collective Insights page for more information.

Overview
The following script will ingest events from Exabeam matching the filter is_ioc. The script runs an initial search based on the 'lookback' -L command line argument specifying the time in hours to search the Exabeam API (or 1 hour by default). Once the script executes for the first time a timestamp is saved to file that is then referenced in future script executions depending on the events returned. If events are returned then the timestamp saved is the timestamp of the latest Exabeam event. Otherwise the timestamp saved will be the script execution time. Following executions add one millisecond to this reference timestamp when searching the next round of documents to send to Collective Insights.

• To overwrite the saved timestamp with the command line argument lookback you must first delete the file at config/latest_timestamp.txt and then run the python script.

Fields Collected from each incident by Recorded Future:

• id - Event ID
• time - Timestamp of the detected event
• ioc_fields - Exabeam event field listing the indicators found in the event. Recorded Future only collects fields matching the types in ip, domain, hash, url, vulnerability
• mitre_labels - Specific MITRE ATT&CK techniques and tactics used in a particular security incident
• malware_family - Specific family or group of malware associated with an event or alert

Prerequisites
The following items be installed/gathered before the setup of the integration script

1. Python v3.10 or greater must be installed
2. Client must have Exabeam Search API access
3. Client must be able to provide an environment to run the script locally with access to the internet.
   1. Server/Workstation for script to run on schedule
   2. Internet Access to Recorded Future API & Exabeam API
   3. Recommend whitelisting api.recordedfuture.com
4. Client must have Recorded Future API Token with access to Collective Insights API
   1. This can be provided by your IntServ Consultant or PS Engineer
5. Client must have the following from Exabeam:
   1. Instance Region
   2. API Client ID
   3. API Client Secret

Set-up Instructions

1. Setup a new virtual environment to install dependencies and run the script from: python3 -m venv venv
2. Activate new virtual environment: source venv/bin/activate
3. Install dependencies from requirements.txt: pip3 install -r requirements.txt
4. Run python script to confirm successful: python3 run_exabeam_ci.py -h 
   1. usage: run_exabeam_ci.py [-h] [-k RF_API_KEY] [-c CLIENT_ID] [-s CLIENT_SECRET] [-r {us-west,us-east,canada,europe,singapore,japan,australia}] [-L LOOKBACK] [--debug] [-l {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
5. Setup script to run on schedule daily to ingest events and send to collective insights:
   1. Example cron schedule to run at 00:15 every night: 15 0 * * * <FILE_DIR>/venv/bin/python3 <FILE_DIR>/run_exabeam_ci.py -k RF_API_KEY

Troubleshooting
The below section is for providing assistance with troubleshooting when having issues running the script.

• Script is failing due to modules not installed.
  • Confirm the requirements.txt was installed properly with pip3
  • Confirm the virtual environment where the python packages are installed is activated
• Not authorized to collect events from Exabeam
  • Confirm the Exabeam Region, Client ID, and Client Secret are accurate for the input parameters
• Not authorized to submit to Recorded Future Collective Insights API
  • Confirm that the Recorded Future API token has the correct Collective Insights API permissions activated
• Why am I ingesting more events from Exabeam than are being submitted to Collective Insights?
  • Collective Insights has filtering for unique indicators when submitting via the api.