3. Option 1 – Snapshot Report (e.g. a one-time assessment or a Proof of Concept)
Firewall/Proxy Upload Single
Log File to
MCAS
Create snapshot reports of Cloud Discovery cloud app use |
Microsoft Docs
4. Option 2 – Microsoft Defender for Endpoint
Win 10 device
running Microsoft
Defender for
Endpoint
Integrate Microsoft Defender for Endpoint with Cloud App
Security | Microsoft Docs
Automatic Upload to
MCAS
5. Option 3 – Secure Web Gateway (SWG)
Supported SWGs
Auto uploads traffic
data to MCAS
Integrate Cloud App Security with Zscaler | Microsoft Docs
Integrate Cloud App Security with iboss | Microsoft Docs
Integrate Cloud App Security with Corrata | Microsoft Docs
Integrate Cloud App Security with Menlo Security | Microsoft Docs
6. Option 4 – Cloud Discovery API (REST API)
MCAS REST API
Auto uploads traffic
data to MCAS
Cloud App Security Cloud Discovery API | Microsoft Docs
7. Option 5 – Automatic Log Upload for Continuous Reports (aka “Log Collector”)
Firewall/Proxy
Auto upload to
MCAS
Configure automatic log upload for continuous reports in Cloud App Security | Microsoft Docs
Firewall/Proxy (Data Source)
Syslog or FTP
Logs
Log Collector(s)
OPTIONAL: Firewall/proxy logs from a
Security Information & Event
Management system (SIEM)
8. How does a Log Collector work?
• Receives logs over Syslog or FTP from firewalls/proxies
• Each log is automatically processed, compressed, and transmitted to MCAS
• FTP logs are uploaded to MCAS after the file finished the FTP transfer to the Log
Collector
• For Syslog, the Log Collector writes the received logs to the disk. Then the collector
uploads the file to Cloud App Security when the file size is larger than 40 KB.
• After a log is uploaded to Cloud App Security, it's moved to a backup directory
• The backup directory stores the last 20 logs
• When new logs arrive, the old ones are deleted.
• Whenever the log collector disk space is full, the log collector drops new logs until it has
more free disk space. You'll receive a warning on the Log collectors tab in the MCAS
portal.
• IMPORTANT: The log collector will compress data before it’s uploaded. Outbound traffic
is 10% the size of the original log.
9. What are the prerequisites?
• Logs must be forwarded in their original format
• Logs must either be sent over Syslog or FTP
• The more verbose in the log, the better the visibility in MCAS. MCAS requires web-traffic data with the following attributes:
• Date of the transaction
• Source IP
• Source user - highly recommended
• Destination IP address
• Destination URL recommended (URLs provide higher accuracy for cloud app detection than IP addresses)
• Total amount of data (data information is highly valuable)
• Amount of uploaded or downloaded data (provides insights about the usage patterns of the cloud apps)
• Action taken (allowed/blocked)
• It is recommended to ensure the log contains upload/download transaction sizes, usernames and target URL. The more verbose
the log, the better!
• Log events aren't more than 90 days old.
• The log file is valid and includes outbound traffic information.
• Log data source is supported.
10. Network Requirements
To enable Cloud Discovery features using a log collector and detect Shadow IT in your organization, open the following items:
• Allow the log collector to receive inbound FTP and Syslog traffic.
• Allow the log collector to initiate outbound traffic to the MCAS portal (for example contoso.cloudappsecurity.com) on port 443.
• Allow the log collector to initiate outbound traffic to the Azure blob storage on port 443:
• US1 https://adaprodconsole.blob.core.windows.net/
• US2 https://prod03use2console1.blob.core.windows.net/
• US3 https://prod5usw2console1.blob.core.windows.net/
• EU1 https://prod02euwconsole1.blob.core.windows.net/
• EU2 https://prod4uksconsole1.blob.core.windows.net/
• Gov US1 https://gprd1usgvconsole1.blob.core.usgovcloudapi.net/
• Note: If your firewall requires a static IP address access list and does not support allowing based on URL, allow the log collector to
initiate outbound traffic to the Microsoft Azure datacenter IP ranges on port 443.
• If you didn't specify a proxy when you set up the log collector, you need to allow http connections on port 80 for the URLs listed on
the Azure TLS certificate changes page. This is used for checking certificate revocation status when you connect to the Cloud App
Security portal.
11. Supported firewalls/proxies
• Barracuda - Web App Firewall (W3C)
• Blue Coat Proxy SG - Access log (W3C)
• Check Point
• Cisco ASA with FirePOWER
• Cisco ASA Firewall (For Cisco ASA firewalls, it's necessary to set the
information level to 6)
• Cisco Cloud Web Security
• Cisco FWSM
• Cisco IronPort WSA
• Cisco Meraki – URLs log
• Clavister NGFW (Syslog)
• ContentKeeper
• Corrata
• Digital Arts i-FILTER
• Websense - Web Security Solutions - Investigative detail report (CSV)
• Websense - Web Security Solutions - Internet activity log (CEF)
• Forcepoint
• Fortinet Fortigate
• iboss Secure Cloud Gateway
• Juniper SRX
• Juniper SSG
• McAfee Secure Web Gateway
• Menlo Security (CEF)
• Microsoft Forefront Threat Management Gateway (W3C)
• Palo Alto series Firewall
• Sonicwall (formerly Dell)
• Sophos SG
• Sophos XG
• Sophos Cyberoam
• Squid (Common)
• Squid (Native)
• Stormshield
• WatchGuard
• Zscaler
Deploy Cloud Discovery | Microsoft Docs
12. Supported firewalls/proxies (cont’d)
• If your log isn't supported, or if you are using a newly
released log format from one of the supported data
sources and the upload is failing:
• Select Other as the Data source and specify the appliance and
log you're trying to upload.
• Your log will be reviewed by the Cloud App Security cloud
analyst team and you'll be notified if support for your log type
is added.
• Alternatively, you can define a custom parser that
matches your format. For more information, see Use a
custom log parser.
• Note: Generic CEF,LEEF,W3C formats are also supported.
Deploy Cloud Discovery | Microsoft Docs
13. What is a Log Collector?
• A Docker server that runs on Windows, Ubuntu, Red Hat Enterprise or Cent
OS
• Can be located on-premises or in Azure
• Minimum requirements:
• Disk space: 250 GB
• CPU: 2
• RAM: 4 GB
• Each OS will have additional minimum requirements. See the
following for more details:
• Docker on Windows on-premises
• Docker on Linux on-premises
• Docker on Linux in Azure
14. Log Collector Performance
The Log collector can successfully handle log capacity of up to 50 GB per hour. The main
bottlenecks in the log collection process are:
• Network bandwidth - Your network bandwidth determines the log upload speed.
• I/O performance of the virtual machine - Determines the speed at which logs are written
to the log collector's disk.
• The log collector has a built-in safety mechanism that monitors the rate at which logs arrive and
compares it to the upload rate.
• In cases of congestion, the log collector starts to drop log files.
• If your setup typically exceeds 50 GB per hour, it's recommended that you split the traffic between
multiple log collectors.
• If more than 10 data sources, consider splitting the sources among multiple log collectors
15. How do I deploy a Log Collector?
• Docker on Windows on-premises
• Docker on Linux on-premises
• Docker on Linux in Azure