Introduction

Recorded Future Collective Insights is a new type of analytic, providing clients a complete view of what threats matters to an organization. Collective Insights can aggregate detections across all client integrations to show trends across all detections to help Threat Intelligence and SecOps Intelligence users better prevent and protect client networks by prioritizing their actions based on which detections and TTPs are most common across their networks. This article will walk through how to setup Collective Insights for Microsoft 365 Defender using the Recorded Future Collective Insights API

In this page, we will look into the installation steps for deploying Recorded Future Collective Insights for Microsoft 365 Defender for sending detections to Recorded Future and identify trends, patterns and insights across your enterprise.

Overview

Recorded Future can collect information on incidents from Defender 365 into Collective Insights allowing you to track indicators and TTPs from incidents across platforms. Specifically, Recorded Future can collect from the following Defender Security tools:

• Microsoft Defender For Endpoint
• Microsoft Defender For Identity
• Microsoft Defender For CloudApps,
• Microsoft Defender For Office365
• Microsoft 365Defender
• AzureAd Identity Protection
• Microsoft App Governance
• Microsoft Defender For Cloud

The following fields are collected from each incident by Recorded Future: 

• createdDateTime - Time the incident was created
• title - Title of the Incident
• Id - unique ID of the incident
• serviceSource - Defender tool (from list above) the incident came from
• detectionSource - Detection technology or sensor that identified the notable component or activity
• mitreTechniques - TTPs associated with the incident

There are two ways to implement the Defender Collective Insights integration. You can use an Azure logic app, or deploy and run a python script

Authentication

To authenticate a connection to the Graph API, you will need to create an app registration in Azure Active Directory. The app registration needs to have the application (not delegated) permissions SecurityAlert.Read.All. A global administrator will need to grant these permissions. Record the Client ID of the app, and generate a secret and record that as well. The Client ID and Client Secret are required for the integration

For the logic app (not the python script), it is also possible to authorize access to the Graph API via a managed identity instead of an app registration. You will need to add the permissions SecurityAlert.Read.All to the managed identity. Note that this is not possible through the Azure UI - you will either need to use Powershell or the Microsoft Graph API to grant this access. 

To authenticate to the Recorded Future API, you will need an API token. See Requesting API Tokens for more information.

Installation

Python Script

Before you run this script, you must:

• Have python3.8 installed
• Have the python module requests installed (pip install requests)

Secrets can be passed to the script as environment variables or command line arguments.

Environment variables:

• RF_API_KEY: Recorded Future API Key
• MS_CLIENT_ID: Azure AD Client ID
• MS_CLIENT_SECRET: Azure AD Client Secret
• MS_TENANT: Azure Tenant ID

Alternatively, as command line arguments:

• -k: Recorded Future API Key
• -cid: Azure AD Client ID
• -cs: Azure AD Client Secret
• -t: Azure Tenant ID

The script also supports additional command line arguments to configure the integration:

• -l: The number of days to lookback to fetch incidents from. If running on a daily cadence, should set to 1
• -fs: filter string. Add a custom ODATA filter string to filter which incidents will be ingested into Collective Insights
• -ll: Log level of the integration

Example usage: python3 365_collective_insights.py -k=<Recorded Future API Token> -cid=< Client ID> -cs=<Client Secret> -t=<tenant ID>   -lb 1 -fs="serviceSource eq 'microsoftDefenderForEndpoint'"

Logic app

You can install the logic app using the attached ARM template by following the steps provided here.