PERFORMANCE MEASUREMENT AND MONITORING

Transcription

1 The University of Auckland New Zealand PERFORMANCE MEASUREMENT AND MONITORING 15 May 2014 Dr. Theunis F.P. Henning Dr Seosamh Costello Prof Susan Tighe

2 The world I come from

3 Background Road Length Total length 94,244km Sealed length 62,843km Rural length 74,688km $ Road Fund State Highways 100% Local roads 40-60%

4 New Zealand Roads

5 Core Messages Data organisation and confidence is a key for successful performance monitoring Link performance to the agency objectives and goals Measures have to be able to tell an effective story A basic statistical understanding of the network and network changes is essential

6 Definitions 5% 25% 50% 75% 95% Level of Service Intervention/Trigger Levels Public Level of Service Level of Service / Performance Monitoring

7 Monitoring Pavement Health Physical Signs Rutting Roughness (IRI and Wavelet) Structural Indices Network Vulnerability Risk Index Probability to Failure Cost balance Cost Efficiency Routine Maintenance

8 Data Types Element Road Inventory Pavement Structures Traffic Finance Activity Resources Aspects Network/Location Geometry Furniture Environs Pavement structure Pavement Condition Structures Inventory Bridge Condition Volume Loadings Accidents Unit Costs Budget Revenue Projects Interventions Commitments Institutional Materials Equipment Location and Network extent Composition physical Condition functional Use What does it cost to maintain

9 Network Segmentation Source World Bank

10 Recommended Section Length Purpose Recommended FHWA NZ PMS Performance Monitoring Performance Based Contracts Homogenous treatment length Fixed lengths 100m-500m 500m (World Bank) 0.5km- 15km Summary interval 100m.. 800m-2.5km 200m 500m or, TL and 50m exceptions

11 Using CUSUM for Section Identification

12 Proposed Confidence Level Framework Confidence Rating Confidence Factor Very Low (1) Low (2) Medium (3) High (4) Equipment Automated non Visual Sophistication laser Automated Laser Automated Laser Calibrated Calibration Contractual No calibration Internal Calibration According to NZTA Standard Calibration Process SH standards Quality Assurance No Evidence of QA Internal QA Calibrated Contractual QA According to NZTA e.g. loop method SH standards Consider individual Post Survey Compare over- all sections and Benchmark with None Confirmation network trends exception LTPP Sites reporting

13 Good Guidelines

14 Linking to Objective or Strategic Goals Source Henning et al, 2011

15 Adopting a Tiered Approach AUDIENCE INDICATOR High Level (Strategic Level) Politicians / Policy / Strategy / Industry Steering Groups or Governing Organisations / Funding Mid Level (Tactical Level) Asset Managers / Industry Steering Groups or Governing Organisations / Funding / Research Low Level Asset Managers / Engineers / Research / Audit High Level (Overview) e.g. PII (pavement) / CII (surface) / STE (user) / RISK Mid Level (Descriptive) Descriptors e.g. Condition / Construction Performance / Road Profile / Data Quality Indication Low Level (Detail) Individual measures / Faults / Data quality indication

16 Using Stats Appropriately data lognormal distribution normal distribution mean mode median mean Roughness (NRM) Source Kadar and Henning 2009 % exceeding limit mode % exceeding 110 NRM log-normal % exceeding 110 NRM normal distribution Years Roughness (NRM)

17 Interpretation of Data

18 Examples Trend Monitoring Rutting Distribution National Rut (mm) Survey Year Source NZTA

19 Example Surface Performance

20 Example Maintenance Efficiency Good Agency ABC Outcome Indicator Poor Cost Agency XYZ Effectiveness measures the ability of an organisation to meet their objectives/outcomes. Efficiency refers to the ability of an organisation to meet those objectives/outcomes (i.e. to be effective) utilising resources to their optimum.

21 Example Outputs - Perceived Vs. Actual Data

22 Source Henning et al World Bank Example Outputs Comparing Apples with Apples $ % Average Bus Fare (USD) $1.80 $1.60 $1.40 $1.20 $1.00 $0.80 $0.60 $0.40 $0.20 $0.00 Colombo Beijing Bucharest Singapore Vancouver Auckland Average Bus Fare Cost as percentage of per capita income 0.007% 0.006% 0.005% 0.004% 0.003% 0.002% 0.001% 0.000% Bus Fare as a Percentage of per Capita Income

23 Show me the evidence!

24 Comparing Two Networks PSMC006 Waikato Mostly rural roads Thin flexible pavements, chip seals Poor subgrade conditions volcanic origin Manukau City Mostly urban network Flexible pavements, surfaced with AC and chip Variability in subgrade conditions

25 Surface Condition PSMC006 Surface Condition PSMC006 Surface Condition Trend P50 P95 P25 P75 Manukau SC Manukau Surface Condiiton Index P50 P95 P25 P75

26 Structural Indices Si Value Network Structural Indices for PSMC006 (2010) Si Rut Si Rough Si Flex Si Shear P50 P95 P25 P75 Si Value Network Structural Indices for MCC (2010) Si Rut Si Rough Si Flex Si Shear P50 P95 P25 P75

27 Rutting PSMC006 Rutting Index (%) SH PSMC006 Rutting Index Trend P50 P95 P25 P75 Manukau Rutting Index (%) Hsd_Rut Depth, mm MCC hsd_rutting Trend MCC Rutting Index Trend P50 P95 P25 P75 P50 P95 P25 P75

28 Vehicle Operating Cost VoCi PSMC SH PSMC 006 Vehicle Operating Cost Index VoCi RIOfAvg Roughness (IRI) VoCi Manukau Manukau Vehicle Operating Cost Index VoCi RIOfAvg Roughness (IRI)

29 Pavement Failure Risk PSMC006 SH PSMC006 Risk Index Distribution 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% RiskIndex_Crack RiskIndex_Rut RiskIndex_Shear RiskIndex_Overall < =1.0 Manukau

30 Telling the Story PSMC006 Surface condition The rutting The failure risk index network prone to rutting The anomaly of the 2008 HSD survey is apparent on this network Well maintained network Manukau Surface condition The rutting Structural indices shows network prone to roughness developing The anomaly of the 2010 HSD survey is apparent on this network Well maintained network, surfacing may be an issue

31 Findings/Recommendations Data issues and availability Data at appropriate granularity Value of information is a function of data quality Understand uncertainty around data Have a plan to improve Value of KPIs Link to objectives Normalisation is important Contextual data is important Motivation/buy in from the top Framework is key

32