Confessions of a Serial CISO: Enterprise Security Metrics

As a CISO, I struggled quite a bit to build my security metrics and dashboards. I invested a lot of time/effort, resources and budget on them:

• Read all the four leading books on the subject, and collated the best best-practices
• Considered leveraging reporting dashboards of and/or data from GRC tools, other leading security platforms (SIEM, Vuln mgt, DLP etc)
• Included both technology metrics as well as process and people metrics (HR security, physical security, BCP/DR, risk assessments, audit/compliance findings, change mgt, incidnet mgt etc)
• Deployed my team as well as vendor/partner security teams on building tailored enterprise security dashboards that fed from all the above and rolled up in a couple of layers of reporting that could make sense to middle and higher management including the board

I am not sure if building security metrics for enterprises is any easier now than what I encountered couple of years ago. But some of the areas of challenge are better served now - visualization, integration, and automation:

1. Building customized security metrics was challenging while working on excel, tableau etc. One had to deal with building visualization templates as well as integrating data from multiple sources while the visualization tool itself did not come with any built in security reporting logic.
2. Integrating the reporting from multiple tools/platforms, and adding process/people metric data into it was always a huge challenge.
3. Automating reporting and more importantly downstream action/remediation mostly remained a good to have wish.

On all these fronts, there are a lot of advantages to adopting the Microsoft 365 (M365) Security and Compliance suites, and the Azure Security platform. PowerBI reporting is built into most of these, and are in the horizon for all. So, it becomes quite effortless to build integrated security metrics by aggregating and collating PowerBI reporting from all the tools in M365 and Azure. Further, by plugging-in data into this aggregated Power BI from other third party security solutions, it can become an enterprise level security reporting tool for Msft and all your other key partners. Furthermore, reporting capabilities in Msft 365 and Azure include both usage and security information; so the enterprise security dashboard can be elevated to include usage metrics when required. Of course, there are a lot of details to consider in terms on what reporting is available by default in which workload, what reporting can be turned on, and what is in the roadmap etc. But there is an integrated security (& usage) reporting story waiting to be explored in detail.

Having taken a detailed look at some of these capabilities, I feel confident to report that security reporting capabilities in M365 and Azure are rich and layered. They can be somewhat dispersed and may need a bit of integration to turn them into an enterprise security dashboard. I think it would pay great dividends to unify and integrate all these reporting capabilities which are integral part of M365 and Azure (so no additional burden to TCO); and they help bring great transparency to enterprise risk management besides being excellent props to have a strategic security conversation with senior execs and the Board. They can also be easily drilled down for operational audiences if/when required. 

From my humble learning from CISO tenures and numerous CISO interaction, here are some key considerations to building high quality Enterprise Security Metrics:

• Architecture: Build them right with the following components, and even if some of them are blank for a while - remember, information gaps tell a story as well:
• Modular: They should represent all major security domains (whether one uses ISO 27001, NIST, ISF, PCI-DSS or any other major global/industry standard)
• Hierarchical: There should be multiple layers of details - at least 2, preferably 3 or more. The highest layer would be suitable for executive consumption and lower layers for the lower levels of management. One should be able to dive deep into the data to the level of raw data from the contributing platform where relevant.
• Growing: They should have a means of demonstrating growth in maturity of the Enterprise Security program whether on CMM or any customized maturity level program - which should be elaborately documented.
• Threshold: Each metric should be able to be compared against a known good or expected threshold to indicate basic success and desirable levels of success. This is also connected to the previous aspect of Growth.
• Customized: There should be a way to customize on all the four aspects above - i.e. add/delete/modify a domain, a layer of detail, and a level of maturity.
• User Interface: Make them easy to consume, attractive, intuitive, and help them tell a story to a variety of user groups - Board of Directors, executive leadership, senior management, auditors, operational management, business functions and user cohorts. 

Mega-trends in Enterprise Information Security

The enterprise perimeter has become punctuated with multiple technology and business factors including advent of mobility and connected devices (BYOD, and IoT), adoption of cloud technologies and solutions (IaaS, PaaS and SaaS), and proliferation of shadow IT (budgets for technologies managed outside of IT such as for marketing and digital initiatives). Consequently, the enterprise information security mandates and expectations need to be commensurate with these phenomenal changes. There is complete acknowledgement of this need which is being witnessed in the form of some key changes in the component domains of information and cyber security. 

From a strategic perspective, security governance, risk management and compliance have been the key domains - referred to as GRC. Technology platforms in this space have slowly matured over the last decade plus but rarely kept in pace with the needs of an enterprise to evaluate their specific security risks, assess those risks internally and track deployment of risk mitigation measures. Meanwhile, the GRC space itself has been transformed to something which Gartner calls Integrated Risk Management (IRM). Existing GRC technology platforms are no surprise front runners to fulfil this space but we are likely to see a similar lag between the enterprise ask of IRM and why technology platform providers can offer.

On the operational side, two areas have solidified as critical to enterprise information security - Identity & Access Management (IAM) and Security Operations (SOC). 

• IAM got disected to several specialization domains such as Identity Provisioning/Deprovisioning, SSO, Identity Governance & Administration, Access Management & Certification, Privileged Identity Management (PIM) etc and we are seeing a return consolidation of all of these domains back to IAM, and there is considerable possibility and appetite to consider outsourcing of the IAM platforms.
• SOC has been highly outsourced from the beginning  through service offerings called Managed Security Services (MSS) provided by MSSPs (MSS provides) based on two core security platforms - one for logging and correlation of security event logs called Securiry Information and Event Management (SIEM). SIEM, and another for tracking leakage of information called Data Leakage Prevention (DLP). Both platforms have gone through significant metamorphosis over the last decade plus. From pure log collection, to focussed correlation, to a combination of both and inclusion of behavioural analytics, SIEM has been through quite a bit of transition and evolution. DLP has been a monolithic platform to look at data traversing a perimeter whether that of a device or a network, and matching certain criteria that makes such data inappropriate to traverse though that perimeter. With the enterprise boundaries vanishing literally into the cloud, we have had to look at managing data with in the numerous repositories it can get created and stored at, and preventing such data from going across trust boundaries from such repositories. The new class of managed service providers in this space are now said to be providing Managed Detection & Response (MDR) Services.

It is worth examining each of these mega trends above which I will attempt to do in subsequent blogs.

Meanwhile, we are continuing to witness paradigm shifts in enterprise security where pointed capabilities for management of specific security capabilities are being replaced by broad spectrum solutions at scale, and often hosted in the cloud. There is likely to be further consolidation in this space where existing players broaden their scope while large enterprise players traditionally not known for security emerge as key players and new ones emerge in the horizon leap frogging on Artificial Intelligence and Machine Learning.