Submitted by Angela Walters on Tue, 31/03/2020 - 11:08
In 2019, the Centre for Digital Built Britain, as part of the Construction Innovation Hub and in partnership with the UK BIM Alliance, commissioned a project to evaluate existing tools that assess BIM maturity and identify the benefits (return on investment) of using BIM in projects. The results of the research are provided below. If you would like to know more about the original call and tender award click here.
We welcome the research undertaken by Northumbria University and support the recommendations, particularly around the use of the ISO 19650 framework and bringing about greater consistency of approach. As a next step, will now integrate the findings of this research into the Hub’s wider transformative programme.
Executive Summary
There is a lack of effective tools for assessing Building Information Modelling (BIM) maturity, or for evaluating its benefits.
These are among the findings of a new report, Building Information Modelling: Evaluating Tools for Maturity and Benefits Measurement, commissioned by the Centre for Digital Built Britain (CDBB) for the Construction Innovation Hub, with funding provided through the Government’s modern industrial strategy by Innovate UK, part of UK Research and Innovation.
The project was instigated and managed working in conjunction with the UK BIM Alliance (UKBIMA). The research team and report were led by the University of Northumbria at Newcastle (UNN).
The authors examined 25 different ways of evaluating Building Information Modelling (BIM) maturity and benefits available to industry. This included 15 maturity tools, four “maturity” methods for organisations and projects, and six benefits tools.
None of the maturity assessment tools examined were aligned with the new ISO 19650 series of standards, as they were all developed prior to its release in 2018. The report recommended that a framework be developed for maturity assessment tools, aligned with the ISO 19650 series, to ensure a common approach.
The study found that BIM benefits evaluation tools have value in helping to promote BIM and encourage collaborative working but recommended that these tools be extended to look at benefits throughout the whole supply chain. Some of the findings are highlighted here.
The purpose of the study
BIM and the adoption of a standards-based approach to information management across the built environment is crucial to the government’s modern industrial strategy. This work is being supported by the UK BIM Framework, comprising the UK BIM Alliance (UKBIMA), the British Standards Institution (BSI) and the Centre for Digital Built Britain (CDBB), which provides guidance for industry on how to implement good-practice BIM using standards.
With the launch of the ISO 19650 series in 2018, the international standard for managing information over the whole life cycle of an asset CDBB decided it was a sensible time to review the various tools used for assessing BIM maturity and measuring BIM benefits, to understand what the gaps were.
To that end, CDBB working with the UK BIM Alliance, commissioned the University of Northumbria at Newcastle (UNN) to undertake this study, following a competitive tender process. UNN led a collaborative team that examined a total of 25 tools available to industry that can be used to assess BIM maturity and measure benefits, to establish how useful the tools are for organisations and projects in the construction industry. They also examined the extent to which BIM maturity and benefits are being measured within the UK construction and asset management industries currently.
Methodology
The research project team included UNN, BIM Academy, the Royal Institute of British Architects (RIBA) and HKA.
The project was led by Associate Professor Mohamad Kassem, who also led the authoring of the report with Jennifer Li of UNN. Contributing authors were Professor Bimal Kumar and Richard Watson of UNN, Adrian Malleson of RIBA, Dr Graham Kelly of BIM Academy and Dr David-John Gibbs of HKA.
The team used five different methods to conduct its research and evaluation as follows:
- Desktop research of the available tools and methods. Unpublished tools used by some organisations were made available for the study. While not an exhaustive list, the 25 tools and methods identified for the study include industry recognised tools used in the UK and internationally;
- Analysis of project BIM maturity tools against the ISO 19650 series;
- Three industry workshops (one in Newcastle-upon-Tyne and two in London) held by the report’s authors with experts from the UK construction sector to understand the current applications of the tools in organisations and projects;
- Eight interviews conducted with experts from the UK construction sector to address the same objectives as the workshops;
- An online survey held between 30 August 2019 and 15 October 2019, with 184 responses.
The team identified that organisations could benefit from improved precision in the language used and the following terminology has been used in the report:
- BIM compliance: the ability of organisations / project teams to fulfil prescribed or mandated requirements (e.g. a standard, client requirements or industry guidelines).
- BIM readiness: the preparatory activities an organisation / project team undertakes prior to the adoption of BIM.
- BIM capability: minimum ability required to engage with a BIM process or deliver an outcome (for example, the availability of BIM tools / protocols within an organisation / project).
- BIM maturity: the extent of BIM capabilities within organisations and project teams. It is usually measured on an ordinal scale with levels such as ‘ad-hoc’, ‘defined’, ‘managed’, ‘integrated’, and ‘optimised’.
Key findings
The study found that BIM maturity assessment tools are not widely used. Only 28% of the 184 survey respondents use a tool to measure BIM maturity. Another 18% measure BIM maturity without a tool.
The research showed gaps between the characteristics of existing BIM maturity assessment tools and the industry requirements. Criticisms include:
- The tools are typically rigid, with binary (yes/no) inputs from users and are largely focused on readiness and capability, rather than maturity;
- The tools tend to lack granularity, which means the depth of the assessment is low or moderate. Where the granularity is higher, assessments take significantly longer to conduct;
- A lack of precision limits the usefulness of some tools; there is insufficient distinction made between readiness, capability and maturity.
- Maturity assessments don’t reach far enough down into the supply chain, instead focusing on tier one contractors and lead designers;
- The gaps and limitations in existing tools are driving organisations to develop their own internal maturity assessments. Some 45% of respondents have already done this, according to a survey conducted by the report’s authors;
- The depth of assessment enabled by most tools is low, offering limited understanding of BIM maturity of organisations or projects;
- Maturity assessment is currently geared towards complying with clients’ BIM requirements, but these differ from client to client.
The survey identified several positive reasons for measuring BIM maturity, including that it helps organisations identify their BIM implementation challenges and develop improvement strategies.
As with maturity assessment tools, the report found that BIM benefits evaluation tools are not commonly used. Of the respondents to the survey, just 16% use a tool to measure BIM benefits, with 35% saying they measure benefits without a tool, and 49% saying they don’t measure benefits at all.
The task of measuring BIM benefits is complex: it is hard to eliminate alternative explanations for the relationship between a BIM capability and an end benefit; there is a lack of benchmarking data and there is variation in the knowledge of those inputting data. There is a further challenge in communicating benefits.
If organisations are to adopt BIM benefits evaluation methodologies, they need to ensure that the benefits sought are material and relevant over long project lifespans.
Some 92% of survey respondents agree that measuring BIM benefits encourages a collaborative way of working, while 77% agree there is a need for better measurement tools.
Recommendations
Gaps identified by the report in the assessment of both BIM maturity and benefits evaluation tools need to be addressed, to fulfil the requirements and expectations of the construction and asset management industry.
Maturity
All BIM maturity assessment tools should be aligned with the ISO 19650 series.
At an organisational level, a multi-level framework for effective BIM maturity assessment should be developed to provide a common approach within the industry. This should identify a range of BIM competencies required and propose metrics for their assessment. The framework should have one common level, relevant to all disciplines within construction and asset management, which is also adaptable to suit specific organisations. Additional levels should be specific to different disciplines.
At a project level, a BIM maturity assessment method should be developed based on the UK BIM Framework, including the ISO 19650 series.
BIM competencies, including maturity, should play a greater role at the “invitation to tender”, “tender response”, “appointment”, and “mobilisation” stages of a project. The competencies should be extended beyond readiness and capability to include maturity. This process can be assisted by adopting the ISO 19650 series which sets out enabling requirements throughout the tender process for each appointment.
There is a perception that maturity in the execution of BIM processes is not sufficiently valued by clients to justify expenditure on professional development. For there to be significant take-up of these training and development opportunities, there will need to be a parallel programme to create a demand for those skills.
Benefits
BIM benefits evaluation tools should be extended to include the broader benefits of embracing a digital culture and driving digital maturity in the supply chain.
At a project level, the tools should establish metrics at the outset of a project and consistently measure against targets.
These metrics need to address challenges that are commonly experienced with construction projects, including: lengthy project lifetime; time lag between an activity and its corresponding benefit; variation in a project’s scope; and the continuing evolution of technology and processes.
The study identified a need for targets to be informed by good practice using BIM processes.
What next?
CDBB and the Construction Innovation Hub are working to better understand industry progress toward information management and digital ways of working becoming ‘business as usual’ in the construction sector. Focused on promoting the methodology of information management and digital ways of working, rather than technical solutions, they are developing the guidance, case studies, tools and support needed to better design, build and operate our buildings and infrastructure.
Together with the BSI and UK BIM Alliance, CDBB supports the UK BIM Framework. For more details see www.ukbimframework.co.uk. This website is devoted to implementing BIM in the UK using the ISO19650 standard.
For stay up to date with CDBB, the Construction Innovation Hub and the UK BIM Framework go to www.cdbb.cam.ac.uk.
This research forms part of Centre for Digital Built Britain’s work within the Construction Innovation Hub. The funding was provided through the Government’s modern industrial strategy by Innovate UK, part of UK Research and Innovation.