The sum of all thepartial scores after considering the weighting factorsrepresents the total score of BIM performance of anorganization. The questionnaire is meant to be filledin by a BIM consultant based on an observation ofthe organization and an in-depth interview with theperson in charge of BIM. This means that the expertopinions of the BIM consultant contribute to thejustification of the answers. This also means thatmisinterpretation of the KPIs or the questions bynon-experts can be avoided since all BIMconsultants carrying out the BIM Quick Scan arecertified by TNO, an independent research institute,after being trained and after receiving clearguidelines about the content and methodology ofthe assessment tool. The expert opinions becomeeven more valuable for the analysis when the BIMconsultant is experienced with the businessoperation of the assessed organization.After analysing the results of the scan, anorganization will receive a total score. A certainscore always represents a certain level ofperformance. For benchmarking purposes, thescores are always consistent, that is, twoorganizations with the same score are directlycomparable in terms of BIM performance. The upperlimit of the score is open, which means that the maximum score can be higher as the standard levelsof performance increase along with theimprovement of knowledge and technologies. Asimple example can be given as follows. After ascan in 2010, Company A gets a score of ‘8’ whilethe maximum score in that year is 10. Based on thelatest BIM development in the following year, themaximum score is determined at 12. The score ofCompany A (an 8) will still be valid, but it means thatthe performance gap with the state of the art growslarger.The scores for each chapter and the total score aremapped in a radar diagram (Figure 2). This diagramvisualizes the level of performance related todifferent BIM aspects. Underperformance ofparticular aspects can, therefore, be easily pointedout and put into perspective in comprehensiverelation with the other aspects.During the development of the BIM Quick Scantool, the process for applying and disseminating thetool in practice was taken into account. Theassessments are performed by a certified BIMconsultant upon a formal request from anorganization. After analysing the results, the BIMconsultant generates a report using a standard TNOtemplate. The report contains the score, explanationof the BIM performance according to the score, andrecommendations for improvement. All the resultswill be stored in the centralized online database foranonymous benchmarking purposes. An organizationcan repeat the assessment over time in order tomonitor its own progress of BIM performance.Based on the assumption of the average timerequired for the adoption of new technology andworking methods, the minimum interval periodbetween two assessments in the same organizationis six months.Next to achieving the objective of being a standardbenchmarking instrument, the BIM Quick Scan toolis also expected to help stimulate BIM innovation inthe Netherlands. In order to accelerate this process,the BIM Quick Scan should be accepted by as manyorganizations as possible in the Dutch constructionindustry. Therefore, major clients and leading BIMconsultants should endorse this tool. Major clients,both public and private, are expected to use this toolto determine the BIM qualification of the candidatepartners during the selection or the tender processof a project. Leading BIM consultants are expectedto commit to use BIM Quick Scan as the standardassessment tool to define the most appropriateadvice or assistance based on their clients’ currentBIM performance level. The tool will also bepromoted through various collaboration platformsand communities of practice.
As a way to disseminate and continually improvethe tool with active participation of the users, anannual BIM Quick Scan congress will be organised.The congress has three main purposes: first, toupdate the construction industry on the current BIMlevel in the Netherlands; second, to upgrade theassessment criteria and maximum score based on the feedback from performed assessments overthe preceding years and the state-of-the-art BIMdevelopment; and third, to monitor the qualificationsof the certified BIM consultants and to equip themwith the latest guidelines about BIM Quick Scan. Theguidelines are restricted for use by the certified BIMconsultants. The guidelines are meant to clarify therationale and purpose of the assessment criteria andthe justification of possible answers; to clarify theweighting factors related to the different criteria incase one BIM aspect is given a higher prioritycompared with another; and to present supportingexamples or cases.For raising BIMawareness among the broad rangeof organisations in the construction industry, asimplified version of the BIM Quick Scan tool will bemade available free of charge on the Internet. Anorganization can perform a self-scan and get anindicative score without further analysis. It isexpected that the indicative result will trigger theorganization to learn more about BIM performancelevels, and to pursue more detailed assessment andadvice from a certified BIM consultant listed on thewebsite. In order to guarantee the accessibility andaffordability of the BIM Quick Scan, especially forsmaller enterprises, TNO will set a maximum pricefor having the scan performed by a certified BIMconsultant.VERIFICATION AND PRACTICALVALIDATION OF THE NEW TOOLThe prototype BIM Quick Scan tool was put intoverification and validation processes. The verificationwas done by an expert panel. The panel consistedof 15 experts, as follows: three researchersrepresenting the knowledge areas of BIMtechnology, BIM implementation process,benchmarking and tool development; one universityprofessor on architectural design systems; onebusiness consultant; and 10 experienced BIMconsultants. All the experts involved possessedpractical and theoretical knowledge of thedevelopment and application of instruments tomeasure BIM performance. The verification processcomprised reviews and expert meetings. Theapproach, structure, analytical method and KPIs ofthe BIM Quick Scan tool were subjected toscrutinized reviews by the experts. Each expertdelivered his critical comments and, during theexpert meetings, these comments and proposals forimprovement were then discussed and decidedupon. This process was repeated in two rounds.The validation was carried out through two pilotcases. The first case represented a fictional firm; itsprofile and performance was created to reflect thewhole spectrum of BIM aspects and practices to beevaluated. The second case focused on a real firm; itwas a medium-sized design and consulting firm thathad recently adopted BIM. Ten BIM consultantswere independently sent to both firms to executethe BIM Quick Scan. All findings were presented andanalysed in the subsequent expert meeting.The pilot cases showed that, after independentassessments by the 10 BIM consultants, 80% of thetotal of 50 assessment criteria delivered equalresults. This confirmed that the BIM Quick Scan toolwas coherent and reliable. Variations in the results ofthe other 20% of the assessment criteria werestudied in detail. Most variations were caused byambiguous interpretations. Two solutions weretaken to tackle this problem: (a) providing clearerdefinitions and more extensive explanations withexamples in the standard guidelines to theconsultants and (b) adjusting the weighting factorsbased on the reconsiderations of the level ofimportance and the degree of accuracy of the KPIs.A lower weighting factor was assigned to certainKPIs that were necessary for clarification, yet maylead to wider variations in score. By doing this,subjectivity in expert opinions was allowed only to alimited extent, without a major influence on theobjectivity of the analysis. Several other variationswere caused by lack of relevance of the assessmentcriteria, for example, with regard to novel conceptsthat were not yet commonly accepted. These criteriawere then removed and stored in the ‘ideas box’ forpossible inclusion in the future updates of the tool.The impact of the BIM performance assessmentto the particular company was also investigatedbased on feedback from the representatives of thereal company that was used as a pilot case.The impact turned out to be positive, both at themanagement level, as well as at the expert stafflevel of the company. The prototype tool was thus proven to achieve an adequate readiness level interms of technical content, analytical method,user-friendliness and practical impact. A remarkshould be made, though, that neither the verificationnor the validation process was designed to meet afull scientific standard.CONCLUDING REMARKSThis article describes the applied research to generatean instrument for benchmarking BIM performance.The instrument aims to provide insight into thecurrent BIM performance level of organizations inthe Dutch construction industry. BIM性能基准测试工具英文文献和中文翻译(4):http://www.youerw.com/fanyi/lunwen_51796.html