Ingest Microsoft Azure activity logs

Supported in:

This document describes the steps required to ingest Microsoft Azure activity logs (AZURE_ACTIVITY) into Google Security Operations.

Configure a Storage Account

Complete the following steps to configure a Storage account:

  1. In the Azure console, search for Storage accounts.
  2. Click Create.
  3. Select the Subscription, Resource Group, region, performance (recommend Standard), and Redundancy (recommend GRS or LRS) needed for the account, enter a name for the new Storage Account.
  4. Click Review + create, review the overview of the account and click Create.
  5. On the Storage Account Overview page, select Access keys from the left navigation of the window.
  6. Click Show keys and make a note of the shared key for the storage account.
  7. Select Endpoints from the left navigation of the window.
  8. Make a note of the Blob service endpoint. (https://<storageaccountname>.blob.core.windows.net/)

Configure Azure activity logging

Complete the following steps to configure Azure activity logging:

  1. In the Azure console, search for Monitor.
  2. Click the Activity log link in the left navigation of the page.
  3. Click the Export Activity Logs at the top of the window.
  4. Click Add diagnostic Setting.
  5. Select all the categories you wish to export to Google SecOps.
  6. Under Destination details select Archive to a storage account.
  7. Select the subscription and storage account you created in the previous step.
  8. Click Save.

Set up feeds

There are two different entry points to set up feeds in the Google SecOps platform:

  • SIEM Settings > Feeds > Add New
  • Content Hub > Content Packs > Get Started

How to set up the Microsoft Azure activity feed

  1. Click the Azure Platform pack.
  2. Locate the Microsoft Azure Activity feed.

  3. Specify values for the following fields:

    • Source Type: Microsoft Azure Blob Storage V2
    • Azure URI: enter the Blob Service endpoint value you recorded earlier, suffixed with insights-activity-logs (for example, https://acme-azure-chronicle.blob.core.windows.net/insights-activity-logs)
    • Source Deletion Option: specify whether to delete files and directories after transferring.
    • Maximum File Age: Includes files modified in the last number of days. Default is 180 days.
    • Shared key: enter the shared key value you captured earlier.

    Advanced options

    • Feed Name: A prepopulated value that identifies the feed.
    • Asset Namespace: Namespace associated with the feed.
    • Ingestion Labels: Labels applied to all events from this feed.

  4. Click Create feed.

For more information about configuring multiple feeds for different log types within this product family, see Configure feeds by product.

Field mapping reference

This parser code first initializes a large number of fields to empty strings, then performs a series of string manipulations and JSON parsing operations to extract relevant information from the Azure Activity log message. Finally, it maps the extracted data to the Unified Data Model (UDM) fields, categorizing the event type and enriching it with additional details like severity, principal information, and network data.

UDM Mapping Table

Log Field UDM Mapping Logic
category read_only_udm.security_result.category_details Directly mapped from the "category" field in the raw log.
callerIpAddress read_only_udm.principal.asset.ip, read_only_udm.principal.ip Directly mapped from the "callerIpAddress" field in the raw log.
correlationId read_only_udm.security_result.detection_fields.correlationId Directly mapped from the "correlationId" field in the raw log.
data.callerIpAddress read_only_udm.principal.asset.ip, read_only_udm.principal.ip Directly mapped from the "callerIpAddress" field within the "data" object in the raw log.
data.correlationId read_only_udm.security_result.detection_fields.correlationId Directly mapped from the "correlationId" field within the "data" object in the raw log.
data.DeploymentUnit read_only_udm.target.resource.name Directly mapped from the "DeploymentUnit" field within the "data" object in the raw log.
data.details read_only_udm.metadata.description Directly mapped from the "details" field within the "data" object in the raw log, only if the "details" field is not "Unknown".
data.entity read_only_udm.additional.fields.entity Directly mapped from the "entity" field within the "data" object in the raw log.
data.EventName read_only_udm.metadata.product_event_type Directly mapped from the "EventName" field within the "data" object in the raw log.
data.hierarchy read_only_udm.additional.fields.hierarchy Directly mapped from the "hierarchy" field within the "data" object in the raw log.
data.identity.authorization.action read_only_udm.security_result.detection_fields.action Directly mapped from the "action" field within the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.evidence.principalId read_only_udm.principal.user.product_object_id, read_only_udm.principal.resource.product_object_id, read_only_udm.principal.group.product_object_id Directly mapped from the "principalId" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log. The specific UDM field it maps to depends on the value of the "principalType" field. If "principalType" is "User" or "ServicePrincipal", it maps to principal.user.product_object_id. If "principalType" is "Group", it maps to principal.group.product_object_id. If "principalType" is "ServicePrincipal", it maps to principal.resource.product_object_id.
data.identity.authorization.evidence.principalType read_only_udm.principal.resource.resource_subtype Directly mapped from the "principalType" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.evidence.role read_only_udm.principal.user.role_name Directly mapped from the "role" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.evidence.roleAssignmentId read_only_udm.principal.resource.attribute.labels.roleAssignmentId Directly mapped from the "roleAssignmentId" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.evidence.roleAssignmentScope read_only_udm.principal.resource.attribute.labels.roleAssignmentScope Directly mapped from the "roleAssignmentScope" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.evidence.roleDefinitionId read_only_udm.principal.resource.attribute.labels.roleDefinitionId Directly mapped from the "roleDefinitionId" field within the "evidence" object of the "authorization" object of the "identity" object in the raw log.
data.identity.authorization.scope read_only_udm.security_result.detection_fields.scope Directly mapped from the "scope" field within the "authorization" object of the "identity" object in the raw log.
data.identity.claims.aio read_only_udm.security_result.detection_fields.aio Directly mapped from the "aio" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.appid read_only_udm.security_result.detection_fields.appid Directly mapped from the "appid" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.appidacr read_only_udm.security_result.detection_fields.appidacr Directly mapped from the "appidacr" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.aud read_only_udm.security_result.detection_fields.aud Directly mapped from the "aud" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.exp read_only_udm.security_result.detection_fields.exp Directly mapped from the "exp" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.http://schemas.microsoft.com/identity/claims/identityprovider read_only_udm.security_result.detection_fields.identityprovider Directly mapped from the "http://schemas.microsoft.com/identity/claims/identityprovider" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.http://schemas.microsoft.com/identity/claims/objectidentifier read_only_udm.security_result.detection_fields.objectidentifier Directly mapped from the "http://schemas.microsoft.com/identity/claims/objectidentifier" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.http://schemas.microsoft.com/identity/claims/tenantid read_only_udm.security_result.detection_fields.tenantid Directly mapped from the "http://schemas.microsoft.com/identity/claims/tenantid" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier read_only_udm.security_result.detection_fields.nameidentifier Directly mapped from the "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.iat read_only_udm.security_result.detection_fields.iat Directly mapped from the "iat" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.iss read_only_udm.security_result.detection_fields.iss Directly mapped from the "iss" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.nbf read_only_udm.security_result.detection_fields.nbf Directly mapped from the "nbf" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.rh read_only_udm.security_result.detection_fields.rh Directly mapped from the "rh" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.uti read_only_udm.security_result.detection_fields.uti Directly mapped from the "uti" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.ver read_only_udm.security_result.detection_fields.ver Directly mapped from the "ver" field within the "claims" object of the "identity" object in the raw log.
data.identity.claims.xms_tcdt read_only_udm.security_result.detection_fields.xms_tcdt Directly mapped from the "xms_tcdt" field within the "claims" object of the "identity" object in the raw log.
data.identity.UserName read_only_udm.principal.user.user_display_name Directly mapped from the "UserName" field within the "identity" object in the raw log.
data.level read_only_udm.security_result.severity, read_only_udm.security_result.severity_details Directly mapped from the "level" field within the "data" object in the raw log. The "level" field is also used to determine the value of the severity field. If "level" is "Information" or "Informational", severity is set to "INFORMATIONAL". If "level" is "Warning", severity is set to "MEDIUM". If "level" is "Error", severity is set to "ERROR". If "level" is "Critical", severity is set to "CRITICAL".
data.___location read_only_udm.target.___location.name Directly mapped from the "___location" field within the "data" object in the raw log.
data.operationName read_only_udm.metadata.product_event_type Directly mapped from the "operationName" field within the "data" object in the raw log.
data.properties.EventChannel read_only_udm.additional.fields.properties EventChannel Directly mapped from the "EventChannel" field within the "properties" object of the "data" object in the raw log.
data.properties.EventSource read_only_udm.additional.fields.properties EventSource Directly mapped from the "EventSource" field within the "properties" object of the "data" object in the raw log.
data.properties.EventId read_only_udm.metadata.product_log_id Directly mapped from the "EventId" field within the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.cause read_only_udm.security_result.detection_fields.cause Directly mapped from the "cause" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.clientIPAddress read_only_udm.principal.asset.ip, read_only_udm.principal.ip Directly mapped from the "clientIPAddress" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.compromisedHost read_only_udm.principal.asset.hostname, read_only_udm.principal.hostname Directly mapped from the "compromisedHost" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.currentHealthStatus read_only_udm.security_result.detection_fields.currentHealthStatus Directly mapped from the "currentHealthStatus" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.previousHealthStatus read_only_udm.security_result.detection_fields.previousHealthStatus Directly mapped from the "previousHealthStatus" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.type read_only_udm.security_result.detection_fields.type Directly mapped from the "type" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.User read_only_udm.principal.user.userid Directly mapped from the "User" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log.
data.properties.eventProperties.userName read_only_udm.principal.user.user_display_name Directly mapped from the "userName" field within the "eventProperties" object of the "properties" object of the "data" object in the raw log, after removing the "SECURE\" prefix.
data.properties.ipAddress read_only_udm.principal.asset.ip, read_only_udm.principal.ip Directly mapped from the "ipAddress" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyChannels read_only_udm.security_result.detection_fields.legacyChannels Directly mapped from the "legacyChannels" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyEventDataId read_only_udm.security_result.detection_fields.legacyEventDataId Directly mapped from the "legacyEventDataId" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyResourceId read_only_udm.security_result.detection_fields.legacyResourceId Directly mapped from the "legacyResourceId" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyResourceGroup read_only_udm.security_result.detection_fields.legacyResourceGroup Directly mapped from the "legacyResourceGroup" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyResourceProviderName read_only_udm.security_result.detection_fields.legacyResourceProviderName Directly mapped from the "legacyResourceProviderName" field within the "properties" object of the "data" object in the raw log.
data.properties.legacyResourceType read_only_udm.security_result.detection_fields.legacyResourceType Directly mapped from the "legacyResourceType" field within the "properties" object of the "data" object in the raw log.
data.properties.legacySubscriptionId read_only_udm.security_result.detection_fields.legacySubscriptionId Directly mapped from the "legacySubscriptionId" field within the "properties" object of the "data" object in the raw log.
data.properties.operationId read_only_udm.security_result.detection_fields.operationId Directly mapped from the "operationId" field within the "properties" object of the "data" object in the raw log.
data.properties.result read_only_udm.security_result.action_details Directly mapped from the "result" field within the "properties" object of the "data" object in the raw log.
data.properties.statusCode read_only_udm.network.http.response_code Directly mapped from the "statusCode" field within the "properties" object of the "data" object in the raw log.
data.properties.suspiciousCommandLine read_only_udm.target.process.command_line Directly mapped from the "suspiciousCommandLine" field within the "properties" object of the "data" object in the raw log.
data.properties.suspiciousProcess read_only_udm.target.process.file.full_path Directly mapped from the "suspiciousProcess" field within the "properties" object of the "data" object in the raw log.
data.properties.suspiciousProcessId read_only_udm.target.process.pid Directly mapped from the "suspiciousProcessId" field within the "properties" object of the "data" object in the raw log.
data.properties.tlsVersion read_only_udm.network.tls.version Directly mapped from the "tlsVersion" field within the "properties" object of the "data" object in the raw log.
data.properties.userAgent read_only_udm.network.http.user_agent, read_only_udm.network.http.parsed_user_agent Directly mapped from the "userAgent" field within the "properties" object of the "data" object in the raw log.
data.properties.userAgentHeader read_only_udm.network.http.user_agent, read_only_udm.network.http.parsed_user_agent Directly mapped from the "userAgentHeader" field within the "properties" object of the "data" object in the raw log.
data.properties.userId read_only_udm.target.user.product_object_id Directly mapped from the "userId" field within the "properties" object of the "data" object in the raw log.
data.ReleaseVersion read_only_udm.metadata.product_version Directly mapped from the "ReleaseVersion" field within the "data" object in the raw log.
data.resourceId read_only_udm.target.resource.name Directly mapped from the "resourceId" field within the "data" object in the raw log.
data.resourceType read_only_udm.additional.fields.resourceType Directly mapped from the "resourceType" field within the "data" object in the raw log.
data.resultDescription read_only_udm.metadata.description Directly mapped from the "resultDescription" field within the "data" object in the raw log.
data.resultSignature read_only_udm.additional.fields.resultSignature Directly mapped from the "resultSignature" field within the "data" object in the raw log.
data.resultType read_only_udm.security_result.action_details, read_only_udm.additional.fields.resultType Directly mapped from the "resultType" field within the "data" object in the raw log.
data.RoleLocation read_only_udm.target.___location.name Directly mapped from the "RoleLocation" field within the "data" object in the raw log.
data.time read_only_udm.metadata.event_timestamp The "time" field within the "data" object in the raw log is parsed to extract the timestamp, which is then mapped to event_timestamp.
data.uri read_only_udm.network.http.referral_url Directly mapped from the "uri" field within the "data" object in the raw log.
read_only_udm.extensions.auth.mechanism INTERACTIVE Set to "INTERACTIVE" if the "isInteractive" field within the "properties" object of the "data" object in the raw log is "true". Otherwise, it is set to "MECHANISM_OTHER".
read_only_udm.extensions.auth.type MACHINE Set to "MACHINE" if the "category" field in the raw log is "NonInteractiveUserSignInLogs", "ManagedIdentitySignInLogs", or "ServicePrincipalSignInLogs".
read_only_udm.metadata.log_type AZURE_ACTIVITY Hardcoded to "AZURE_ACTIVITY".
read_only_udm.metadata.vendor_name Microsoft Hardcoded to "Microsoft".
read_only_udm.principal.platform WINDOWS, MAC, LINUX, ANDROID Determined based on the value of the "properties.test.deviceDetail.operatingSystem" field. If it contains "Win", platform is set to "WINDOWS". If it contains "Mac", platform is set to "MAC". If it contains "Lin", platform is set to "LINUX". If it contains "Android", platform is set to "ANDROID".
read_only_udm.principal.resource.type SERVICE_ACCOUNT, UNSPECIFIED Determined based on the value of the "identity.authorization.evidence.principalType" field. If it is "ServicePrincipal", type is set to "SERVICE_ACCOUNT". Otherwise, it is set to "UNSPECIFIED".
read_only_udm.security_result.action ALLOW, BLOCK, UNKNOWN_ACTION Determined based on the values of the "resultType", "status_errorcode", and "statusText" fields. If "resultType" is one of "Success", "success", "Succeeded", "Started", "Resolved", "Active", "Updated", "Start", "Accept", "Accepted", "0", or if "status_errorcode" is 0, or if "statusText" is "Success", action is set to "ALLOW". If "resultType" is one of "Failure", "Failed", or if "status_errorcode" is not empty, or if "resultType" is not empty, action is set to "BLOCK". Otherwise, it is set to "UNKNOWN_ACTION".
read_only_udm.target.cloud.environment MICROSOFT_AZURE Hardcoded to "MICROSOFT_AZURE".