As a CISO, I struggled quite a bit to build my security
metrics and dashboards. I invested a lot of time/effort, resources and budget
on them:
- Read all the four
leading books on the subject, and collated the best best-practices
- Considered leveraging
reporting dashboards of and/or data from GRC tools, other leading security
platforms (SIEM, Vuln mgt, DLP etc)
- Included both technology
metrics as well as process and people metrics (HR security, physical
security, BCP/DR, risk assessments, audit/compliance findings, change mgt,
incidnet mgt etc)
- Deployed my team as well as vendor/partner security teams on building tailored enterprise security dashboards that fed from all the above and rolled up in a couple of layers of reporting that could make sense to middle and higher management including the board
I am not sure if building security metrics for enterprises
is any easier now than what I encountered couple of years ago. But some of the
areas of challenge are better served now - visualization, integration, and
automation:
- Building customized
security metrics was challenging while working on excel, tableau etc. One
had to deal with building visualization templates as well as integrating
data from multiple sources while the visualization tool itself did not
come with any built in security reporting logic.
- Integrating the
reporting from multiple tools/platforms, and adding process/people metric
data into it was always a huge challenge.
- Automating reporting and more importantly downstream action/remediation mostly remained a good to have wish.
On all these fronts, there are a lot of advantages to
adopting the Microsoft 365 (M365) Security and Compliance suites, and the Azure Security platform. PowerBI
reporting is built into most of these, and are in the horizon for all. So, it
becomes quite effortless to build integrated security metrics by aggregating
and collating PowerBI reporting from all the tools in M365 and Azure. Further,
by plugging-in data into this aggregated Power BI from other third party
security solutions, it can become an enterprise level security reporting tool
for Msft and all your other key partners. Furthermore, reporting
capabilities in Msft 365 and Azure include both usage and security information;
so the enterprise security dashboard can be elevated to include usage metrics
when required. Of course, there are a lot of details to consider in terms on
what reporting is available by default in which workload, what reporting can be
turned on, and what is in the roadmap etc. But there is an integrated security
(& usage) reporting story waiting to be explored in detail.
Having taken a detailed look at some of these capabilities, I feel confident to report that security reporting capabilities in
M365 and Azure are rich and layered. They can be somewhat dispersed and may
need a bit of integration to turn them into an enterprise security dashboard. I
think it would pay great dividends to unify and integrate all these reporting
capabilities which are integral part of M365 and Azure (so no
additional burden to TCO); and they help bring great transparency to enterprise risk management besides being excellent props to have a strategic
security conversation with senior execs and the Board. They can also be easily
drilled down for operational audiences if/when required.
From my humble learning from CISO tenures and numerous CISO interaction, here are some key considerations to building high quality Enterprise Security Metrics:
- Architecture: Build them right with the following components, and even if some of them are blank for a while - remember, information gaps tell a story as well:
- Modular: They should represent all major security domains (whether one uses ISO 27001, NIST, ISF, PCI-DSS or any other major global/industry standard)
- Hierarchical: There should be multiple layers of details - at least 2, preferably 3 or more. The highest layer would be suitable for executive consumption and lower layers for the lower levels of management. One should be able to dive deep into the data to the level of raw data from the contributing platform where relevant.
- Growing: They should have a means of demonstrating growth in maturity of the Enterprise Security program whether on CMM or any customized maturity level program - which should be elaborately documented.
- Threshold: Each metric should be able to be compared against a known good or expected threshold to indicate basic success and desirable levels of success. This is also connected to the previous aspect of Growth.
- Customized: There should be a way to customize on all the four aspects above - i.e. add/delete/modify a domain, a layer of detail, and a level of maturity.
- User Interface: Make them easy to consume, attractive, intuitive, and help them tell a story to a variety of user groups - Board of Directors, executive leadership, senior management, auditors, operational management, business functions and user cohorts.
