Accuracy, Precision, Tolerance We understand the issues in this digital age? Abstract Survey4BIM has put a challenge down to the industry that geo-spatial accuracy is not properly defined in BIM systems. Accuracy is considered one of the fundamental issues in peoples use and confidence in data and information systems. Historically the geospatial community knows how to determine accuracy, but how can this be translated into a digital environment? If we can define it properly and communicate it to users it can add huge value to decision making and risk management. So read on to get to grips with the issues. After reading we would appreciate if you could pass on a few comments using the fillowing link: https://www.surveymonkey.co.uk/r/bk7gdj5 Introduction Survey4BIM is a group of like-minded professionals and organisations which are involved in the collection, processing, management and delivery of geospatial services within a BIM context. The group works under the umbrella of the UK BIM Alliance. The group s mission is to provide a forum for survey organisations and industry professionals to collaborate and share their journey putting "BIM into practice" and to provide guidance documents on survey matters relating to BIM. Big 5 Challenges While there is a clear vision outlined in numerous documents what BIM Level 2 entails and how we have to collaborate to make it work, some important details pertaining to geospatial matters are not readily understood. Those details are technical and need to be embraced not only by survey professionals but also by all users of the system. The Survey4BIM Group has identified those technical details and grouped them under five headings, namely: Accuracy Interoperability Metadata Level Of Detail Generalisation Page 1
The headings have been named the Big Five Challenges by the group and their solution is being likened to the building blocks of a lighthouse. This analogy has been described in some detail in several publications by the Survey4BIM group. The Five Challenges are not new; in fact they have always been understood at some level by those who commissioned a topographical or other survey. The problem is that this understanding is paper based and we need to move this forward into the digital age. Purpose of "Problem Statement" The first challenge to be addressed is "accuracy". Everybody deep down has a feeling of knowing the meaning of that word. But on detailed examination we find slightly different interpretations from different people even from within the same industry. While "accuracy" in a geospatial sense is specific and well defined there is a need to translate that meaning into the wider BIM community. This process represents the first problem in our accuracy statement, namely: Challenge Part 1: The aim is to reach a common understanding of the frequently vaguely-defined terms, accuracy, precision and tolerance, their implications for surveyors, their stakeholders and if and how they should be shown in BIM models and data. While defining "accuracy" the group came across terms such as "Absolute Accuracy" and "Relative Accuracy" and also "Construction Tolerance". This represents the second problem of our statement. Challenge Part 2: How do you describe absolute and relative accuracy in a BIM system for survey model and data and how can it be related to the intentions of the design? How is the distinction made between accuracy and tolerance in a BIM system? Definition of Accuracy According to the Oxford Dictionary the technical definition of accuracy is: "The degree to which the result of a measurement, calculation, or specification conforms to the correct value or standard." If we leave the wording for specification out and replace correct value with true value, then we arrive at a definition which conforms to the general understanding within the geospatial industry. Thus the definition of accuracy in a geospatial sense would be: "The degree to which the result of a measurement and/or calculation conforms to a true value." Some people also use the word accurate to describe the conformance of some work to a predefined standard or specification as per definition of the Oxford English Page 2
Dictionary. Usually they refer to some missing items within a topographical survey that make the work inaccurate. In a geospatial sense the work has not become inaccurate for the overall accuracy of surveyed items remains the same. It would therefore be better to describe the work as incomplete. To keep our definition of accuracy simple and in line with geospatial thinking it has been decided to treat completeness separately. The subject is therefore being covered under the "Level of Detail" challenge. Fundamental Difficulties In examining the above definition the first question that arises is the meaning of "true value". Of course we never really know a true value of any measurement since all observations produce slightly different answers. If we observe a dimension several times we are likely to obtain a large amount of slightly different answers. Intuitively we believe that the arithmetic mean value of all those readings is nearest to the truth. We describe this as the most probable value. This idea can be extended to a whole range of observations all connected via some geometry. The combination of that geometric configuration in conjunction with multiple surplus observations allows for the application of some statistical calculations. The outcome of those calculations is the most likely dimensional relationship between all observed points and their individual accuracy. This accuracy is described as "Standard Deviation" or, that with 68% probability; all measured differences shall fall within the stated margins. The dimensional relationship between points is best described using a coordinate system. For most people this is a right angled Cartesian system of even scale in X, Y and Z direction. For some professionals the origin of the system is also not important since to them it is the vector relation between described points and objects that matters most. In land surveying the coordinate system is realised using coordinated control stations. Those can be physical markers or the location is expressed in terms of Global Navigation Satellite Systems. Either way it is important to know the nature of the coordinate system in use. It can be a simple Cartesian system as described above, a map projection covering a large part of the country or a global system which models the entire earth. Surveying is a two way process. In the first instance data is collected for the purpose of design/asset mapping and in the second instance a design or modification of an asset has to be established on the ground to enable construction. Both of those instances require the use of coordinated control stations that fix the system in use. The coordinated control stations will have had their positional accuracy determined as part of the calculation process. This positional accuracy is also influenced by the chosen coordinate system. For instance a national system like Ordnance Survey in the UK will contain historical inaccuracies that feed right into the calculation, making the resulting positions less accurate as could have been the case in a different system. It follows that any transformation between systems will change the accuracy Page 3
of the transformed points and objects. Any change of system during the design process should therefore be avoided. Location seems to be obvious because we can see it on the ground. But the same location in different coordinate systems will produce different results. This is yet another reason to ensure that the coordinates are well defined. Points and objects of detail are usually collected from otherwise determined survey control stations. The control stations contain positional inaccuracies and added to this are the observational and instrumentational inaccuracy of the observed detail. Therefore any observed detail will be less accurate than the point it was observed from. It is possible to describe those instrumentational and observational inaccuracies empirically. In turn any object can have a location accuracy assigned. It is often more important to know the likely accuracy of a calculated distance between points and objects than their absolute location accuracy within a defined coordinate system. So far we have been talking about how accurate some object is within a coordinate system. It should be possible to work out a relative accuracy from the previously determined absolute accuracies between objects. The question is can this easily be done in a BIM system. While talking about relative accuracy and working out some distance between two points from given coordinates it is important to realise that this calculated distance may not be the same as the one that would have been measured on the ground. The difference between calculated and directly measured can be considerably greater than the determined relative accuracy. This difference may have to do with the fact that the rules of the coordinate system have not been taken into consideration. For instance if we work in a system that is a projection and our project area lies some distance elevated above the calculation surface then our calculated distance will be on the calculation surface. To obtain a ground distance we need to scale up to the right elevation and make adjustments for the projection scale at our location. All this may sound trivial to the land surveyor but it can be a minefield to the designer and other users of the BIM system. It appears therefore important that the choice of coordinate system is understood and agreed by all parties at the beginning of the BIM process. Absolute Accuracy By now it has become apparent that absolute accuracy is defined within the realms of a coordinate system. So in geospatial terms absolute accuracy is the measure to which an object or point conforms to its true location within a chosen coordinate system Absolute accuracy can be expressed as the three dimensional coordinate or individual coordinate component difference between a surveyed position in a defined survey coordinate system (i.e. x, y or z) and its true position in that coordinate system with an associated probability. For example: there is a 68% probability that the z coordinate of point A is accurate to within 0.005m. Page 4
The accuracy of a location is expressed in the direction of the coordinate axis of the system. As can be established from the diagram below, those axis errors are not necessarily providing the maximum error possible. In order to understand the direction and magnitude of the uncertainties it is necessary to construct an error ellipse as in the given 2D example. Error ellipsoids can also be constructed from resulting 3D information. Besides providing critical information regarding the precision of adjusted point positions, a major advantage of error ellipses is that they offer a method of making a visual comparison of the relative precision between any two points or objects. Relative Accuracy Relative accuracy may describe the resultant accuracy of calculated directions and or distances between two points which may or may not have been directly observed. Thus in geospatial terms relative accuracy is the comparison of a derived measurement between specific surveyed points or objects and their true measure. Relative accuracy is more complex because it is dependent upon selecting specific features to derive a measurement from, and, how those measurement observations are related, in range and capture methodology (i.e. how far apart, how many measures and how related they are to each other). For example if two features are captured from in the same point cloud from a single instrument setup, at the same time and under the same conditions, the derived measurement between those features is likely be more accurate than a derived measurement between two points separated by several rooms and different instrument setups which have been joined together by a coordinated traverse. This can also be expressed in x, y and z terms with an associated probability. What is the pictorial representation of relative accuracy? Page 5
In this diagram the relative accuracy between points 'A' and 'B' are shown in relation to point 'A'. A similar ellipse can be drawn from 'B' to 'A' which will most likely produce a different directional uncertainty. It is common that the relative accuracy ellipse is shown in the centre of a selected line between two points. In the above example error ellipses are shown at 95% confidence level for adjusted points 2, 4, 5 and 6. The network has been calculated using a variety of horizontal angle and distance measurements. The observed lines are shown in cyan. The line between point 2 and 4 has never been observed directly but it is still possible to obtain relative accuracy information in the form of an ellipse. Page 6
Possible Solutions What are the issues in practice and how could absolute and relative accuracy be expressed in a BIM system. For a long time the survey profession has been able to calculate error ellipses as part of a survey control network. But in the eyes of other professions survey control is only a means to an end. In other words they are not interested or only notionally interested. What other professions are interested in are the detail and the associated accuracy. Another question occurs, and that is the adjustment methods for survey control. It is customary that new survey control is adjusted in relation to other existing control stations and those are held fixed for the purpose of the adjustment. In the previous example this can be identified with station 1 and station 3. Normally when holding stations fixed that means they have zero error designed to it, therefore our adjustment is relative to those two stations. Strictly speaking those stations also contain errors around which they can float. This of course will produce larger overall error ellipses. So it appears to be necessary to state which stations are held fixed in any adjustment. In order to make any meaningful statement in a BIM system it is necessary to express the accuracies of detail locations within a chosen survey network. An example on how that might look is given below. One can clearly make out that the size of the ellipses depends largely on the distance accuracy and on the accuracy of the point being measured from. The angular influence in this case is relatively small since the distance to the objects is less than 100m. An important observation is that they all have different orientations and therefore no blanket accuracy statement can be made for a group of points observed from the same setup. Page 7
The relative accuracy is determined from elements of the Cofactor Matrix in a Least Square Adjustment. Therefore the network accuracy determines the relative accuracy and associated ellipses. In other words, the relative accuracy relates to the fixed control stations. The above diagram shows relative error ellipses at 95% for lines 1A-1B, 5A-5B, 2A- 2B, 2B-5A, 2A-5B, 5B-1A and 1B-2A. The sizes of the ellipses are enlarged, to put this into perspective the actual dimensions are given below. We can conclude from this analysis what we already instinctively know, namely that the accuracy of single observational data decreases considerably in comparison to multi-fix control stations. Subsequently the reliability of any dimensions between those secondary points is also not that great. But at least it is possible to quantify those accuracies and make an appropriate statement. Would it be sensible to make those calculations available in a BIM system? The orientation and size of both relative and absolute error ellipses are calculated from elements of the Cofactor Matrix in a Least Square Adjustment. To my knowledge the only way to reproduce this would be to carry out another adjustment. It would be surely not a good idea to make those tools available to people who do not understand the principles. A structural engineer may object to a lay person messing about with his static calculations. Page 8
It might be more sensible for the land surveyor to make some general statements about accuracy in selected critical areas. If at a later stage some further information is needed then it should be easy enough for designers etc. to request specific updates. Tolerance Tolerance is specified by the designers and represents the maximum permitted difference between the design and reality. So in geospatial terms, and in a BIM system, we can see tolerance (design intent) as complementary to accuracy (asbuilt). In geospatial terms, absolute tolerance is the maximum permitted difference between the actual (built / manufactured) position and the design position of an object within a defined coordinate system and relative tolerance is the maximum permitted difference between the actual and the designed spatial relationship between two or more objects. Absolute tolerance is often less important to the designer than to the land surveyor. We establish building and other positions from the available project control. The maintenance of that project control is very important during the construction process because it guaranties that any setting out can be re-established with a high degree of confidence. Setting out is also done with an increased amount of additional dimension checking such as diagonals and lines along the building. All this leads to increased internal accuracy of the established building than would be obtained from a single bearing and distance measurement demonstrated earlier. The only thing that is not done is to put those measurements to the test by carrying out a Least Square Adjustment. Relative Tolerance can be used to specify alignment of specific objects and straightness, level, distance etc. between them. The question is how this can be related to the relative accuracy discussed so far. Page 9
As can be seen from the above diagram, the relative error ellipse between 5B and 2A has a major axis of ±32mm and a minor axis of ±16mm. The major axis is predominantly along the axis of the line and the minor axis provides the uncertainty in parallel direction to the line. One can work out from the orientation of the ellipse an actual length error of the line and a potential offset error. In our case, because of the skewed arrangement give a resulting distance error of ±27mm and an offset error of ±17mm all at 95% confidence. We can readily compare the obtained relative accuracies to given tolerances. Sometimes tolerances are given as zero to +10mm meaning that the arrangement can't be any smaller than the given dimension but is allowed to exceed it by 10mm. If we add 5mm to the given dimension then the resulting tolerance will be ±5mm which is comparable to our relative accuracy. All this is possible using existing commercial software. What is necessary is to develop the right reporting formats to make the results available in a BIM system. Conclusion The above examination uses well established technics described in numerous text books and other publications and made available in commercial software packages. It is possible that not all packages on the market offer the flexibility required for carrying out the detailed analysis demonstrated in this article. We also do not always have neat solutions like those provided through a traditional control network. How do we deal with moving trajectories as control as those calculated when using mobile mapping techniques? There is always room to think a little further and come up with a general working solution that is fitting for today s survey techniques. Page 10