A comparison between the hourly autoscaled and manually scaled characteristics from the Chilton ionosonde from 1996 to 2004

Transcription

1 RADIO SCIENCE, VOL. 43,, doi: /2005rs003401, 2008 A comparison between the hourly autoscaled and manually scaled characteristics from the Chilton ionosonde from 1996 to 2004 R. A. Bamford, 1 R. Stamper, 1 and L. R. Cander 1 Received 3 October 2005; revised 4 May 2007; accepted 21 August 2007; published 4 January [1] Most monitoring ionosonde stations now rely on automatic processing rather than manual processing to provide ionospheric characteristics. A systematic assessment was made of the quality of all the key ionospheric characteristics scaled automatically from hourly ionograms from the midlatitude Chilton ionosonde in the United Kingdom, by comparing them with the definitive values produced by manual scaling. The data for the study were a nearly continuous series of hourly values covering the majority of solar cycle 23 from 1996 to the end of This period includes solar maximum and minimum conditions and quiet and disturbed intervals, and the investigation is the first comprehensive examination of the performance of automatic scaling without any data preselection over such an extended period. The accuracy of autoscaled values during storm periods was examined against the global storm index Dst for the whole 9-year data set. Geomagnetic conditions were found to have only a small effect on autoscaling performance, with the most important identifiable cause of error being the truncation of automatic layer traces due to broadcast interference. Overall, the performance of the autoscaling algorithms was found to be acceptable, with the characteristics fof2, h 0 E, M(3000)F2, and MUF(3000)F2 within defined error bounds for more than 90% of the time and all characteristics within these bounds more than 80% of the time. Citation: Bamford, R. A., R. Stamper, and L. R. Cander (2008), A comparison between the hourly autoscaled and manually scaled characteristics from the Chilton ionosonde from 1996 to 2004, Radio Sci., 43,, doi: /2005rs Introduction [2] Most ionosonde stations now rely on automatic scaling for obtaining regular ionospheric characteristics, and it is rare for a skilled person, or scaler, to check each ionogram routinely and manually correct or determine the values of the characteristics. Despite the complexity of ionogram data, automated processing has the obvious advantages of speed and reduced cost. This allows ionospheric sounding data to be made available on the Web in near real time, thus enabling the development of the nowcasting and forecasting capabilities required of space weather services (e.g., ac.uk/, [Stamper et al., 2004a]. 1 Space Science and Technology Department, Rutherford Appleton Laboratory, Didcot, UK. Copyright 2008 by the American Geophysical Union /08/2005RS [3] The present study addresses the question of whether this change to rely on automatic scaling has been at the expense of accuracy in the scaled characteristics. The midlatitude Chilton ionosonde in the United Kingdom (51.6 N, E) has maintained a high standard of manual scaling, with experienced scalers used during its operation over many decades. In particular, the manually scaled data used in this investigation were all produced by a single person. The ionosonde is one of the most widely used types of instrument, the Digital Portable Sounder, model DPS-1, also known as a Digisonde and produced by the University of Massachusetts, Lowell (UML) [Reinisch, 1996]. At all times the Chilton ionosonde has kept up with equipment and software updates, making it an ideal representative station for investigating the performance of automatic scaling algorithms, specifically the ARTIST software developed at UML [Reinisch and Huang, 1983]. [4] Unlike previous studies [Gilbert and Smith, 1988; Fox and Blundell, 1989; Igi et al., 1992; Pezzopane and Scotto, 2005], this study covers all the regularly scaled characteristics (fof2, h 0 F2, foe, h 0 E, MUF3000F2, fof1, 1of11

2 Figure 1. year An example of a contour plot of the annual, diurnal variation in the error in fof2 for the h 0 F1, and fmin) for the continuous period from 1996 to the end of This encompasses sunspot minimum and maximum, storms and quiet periods, and changes in ionosonde equipment and software [McNamara, 2006]. [5] This study was conducted as a result of participation in the COST 271 Action Effects of the upper atmosphere on terrestrial and Earth-space communications [Zolesi and Cander, 2006] and the resulting creation of a real-time ionospheric database of European sounders, available at html [Stamper et al., 2004b]. 2. Scope of Analysis [6] Our analysis concentrated on eight of the more important ionospheric characteristics. These were (1) foe, fof1, and fof2, the critical frequencies of the principal ionospheric layers; (2) h 0 E and h 0 F2, the minimum virtual heights of the E and F2 layers; (3) fmin, indicative of ionospheric absorption; and (4) M(3000)F2 and MUF(3000)F2, the standard radio propagation characteristics used to indicate the maximum transmission frequency possible on a 3000-km circuit. [7] For each of these the quantity investigated was the error in the autoscaled value, defined as the difference between the autoscaled and manually scaled values from the same sounding. This assumes the correctness of the manually scaled values [Piggott and Rawer, 1972]. For the Chilton sounder this is a reasonable assumption because all the ionograms in the study were scaled by the same person, who has more than 15 years experience of scaling ionograms. 3. Diurnal Variation of Autoscaling Errors [8] In order to visualize such a long time series as 9 years of hourly soundings for each characteristic the data are presented as contour plots of the time of day against day number for that year. This format highlights the diurnal variations of the error in the automatic value compared with the manual value during the year. The color scale indicates the magnitude of the error, where a lighter color indicates a larger error. Figure 1 shows an example of a contour plot of the error of the autoscaled fof2 value for a year of high solar activity, namely The time of local sunrise and sunset (at ground level) is shown as a thin white line. The Chilton ionosonde makes its soundings on the hour. Figure 1 shows that the majority of the larger errors in the automatically scaled value of fof2 (light patches) consistently occur just after dusk. There are occasional larger errors during daylight but these are intermittent and rarely last longer than 2 to 3 h. There is a suggestion of marginally greater inaccuracy in the automatic value around the autumn equinox. [9] Figure 2 shows multiple panels of the absolute error in automatically scaled fof2 values for the years 1996 to 2004, inclusive. The earliest year is shown in Figure 2, bottom left, with successive plots following a bottom to top then left to right order, enabling a column of panels to be readily interpreted as a single continuous plot. The contour colors are on a fixed scale from 0 to 2of11

3 Figure 2. Contour plots of the annual, diurnal variation in the error in fof2 for 1996 to MHz. Similar plots were generated for each of the other seven characteristics, and those for foe and M(3000)F2 are reproduced here in Figures 3 and 4. The most striking feature of these plots is a step change in the size of the errors near day 140 of 1999 (20 May). The size of the errors in some characteristics before this point, and the abrupt nature of the change, excludes the possibility that this change is due to natural variation of the geophysical environment. This leaves environmental or instrumental causes as the remaining possibilities. [10] The most probable cause is instrumental change. UML released a new version of the ARTIST software in May 1999, although the engineering log book for the Chilton instrument records this as having been installed there only in January The evidence from the data presented here is that it was actually installed soon after release in May This is possible, despite the record in the log book, because at that time engineers at UML had the capability to modify parameters on the Chilton instrument remotely. Because of this instrumental effect, subsequent analysis concentrated on the data from the year 2000 onward. 4. Pointwise Analysis of Automatic and Manually Scaled Characteristics [11] A scatterplot of automatically scaled against manually scaled fof2 values for all the hourly soundings in the year 2001 is shown in Figure 5. The fitted line, the parameters for which are shown at the top of the diagram, indicates that the fit is generally good, with a slope close to 1 and only a small intercept. Nonetheless, Figure 5 shows prominent ledges extending to the right of the ideal y = x line, where a frequency is scaled preferentially by the autoscaling algorithm when the true value ranges from that value up to about 3 MHz higher. 3of11

4 Figure 3. Contour plots of the annual, diurnal variation in the error in foe for 1996 to Looking at the panels in other years (2000 onward in Figure 6), this phenomenon can be seen to reoccur at frequencies of roughly 6 MHz, 9.5 MHz, and 12 MHz. [12] The cause of the problem becomes clear when the raw ionogram data are examined in more detail (see Figure 7). The cause was identified as interference from other radio sources. The Digisonde checks the noise (interference) level for each sounding frequency and retains only signals that are larger than the noise threshold. The presence of an external signal at just above 6 MHz would significantly raise the noise floor and would therefore cause a gap in the echo trace seen in Figure 7. [13] Figure 7 shows a typical raw ionogram selected from one of the intervals where this occurred. The figure shows the swept frequency of 1 to 15 MHz on the x axis and the virtual height (corresponding to the time of flight) of the returned echo on the y axis. The red pixels indicate the O polarization, and the green pixels indicate the X trace. The pale blue line is the true height electron density profile generated by the ARTIST automaticscaling software. It is clear that there is a gap between about 6 MHz and 6.3 MHz in both the O and X traces due to interference. Although the ionogram traces continue, it is clear that the ARTIST O-trace scaling prematurely halts at the gap. ARTIST determines fof2 as the maximum frequency on the scaled O-trace, which is clearly incorrect when that trace is ended prematurely as in this example. This is what produces the cluster of errors near specific interference frequencies on the annual plots of automatic against manual fof2. Clearly, this error also affects the h 0 F2 and fof1 characteristics and the HF propagation characteristics of M(3000)F2 and MUF(3000)F2, but has no effect on the lower-frequency characteristics like fmin, foe, and h 0 E. [14] To assess the overall quality of the agreement between autoscaled and manually scaled values, a linear fit was carried out for each characteristic for each year. A 4of11

5 Figure Contour plots of the annual, diurnal variation in the error in M(3000)F2 for 1996 to summary plot of the correlation coefficients for these regressions for each ionospheric characteristic in each of the 9 years is shown in Figure 8. This figure demonstrates the overall improvement in the accuracy of the automatic scaling, after May 1999, for all the characteristics. It also shows that the ARTIST automatic-scaling software performs best at determining fof2 and MUF(3000)F2 and less well with fmin and M(3000)F2. The overall accuracy of ARTIST has improved considerably since 1999/2000, principally because of the instrument changes discussed previously and clearly demonstrated in Figures 2, 3, and 4. [15] Similarly good correlations between the automatic and manual values hold for the other characteristics foe, h 0 F2, and MUF(3000)F2 from the year 2000 onward. The annual scatterplots for h 0 F2 are shown in Figure 9, and those for foe and MUF(3000)F2 show an equivalent Figure 5. All the hourly autoscaled fof2 values for 2001 plotted against the true, manually verified, values. 5of11

6 Figure 6. Correlation between manually scaled and autoscaled values of fof2. Figure 7. Chilton ionogram for 1900 UT on 3 March of11

7 Figure 8. Correlation coefficients between the manually scaled and autoscaled values for each characteristic, calculated for each year. Figure 9. Correlation between manually scaled and autoscaled values of h 0 F2. 7of11

8 Figure Errors in hourly fof2 plotted against the corresponding Dst index values for the year pattern. Generally, the virtual height h 0 F2 is determined less reliably than the critical frequencies. 5. Impact of Geomagnetic Activity [16] In order for ionosondes to continue to be a primary source of real-time data on the state of the ionosphere for HF communications and other applications affected by space weather, the automatic-scaling software has to be able to function reliably during ionospheric disturbances throughout a solar cycle. To assess the reliability of the ARTIST software at Chilton in varying conditions, the errors in the autoscaled hourly values were compared with the geomagnetic Dst index. Classification of geomagnetic storms is frequently based on the Dst index, which can be obtained promptly from the World Data Center for Geomagnetism, Kyoto University, Japan, at wdc/sec3.html. This index, derived from a network of near-equatorial observatories, responds readily to the intensity of the globally symmetrical equatorial electrojet (the ring current ) and is therefore called the Disturbance Storm Time (Dst) index [Mayaud, 1980]. [17] Figure 10 shows a comparison between the errors in autoscaled value against Dst value for each hour in the year 2001, at high solar activity. Similar plots for the years are significantly affected by the instrumental problems discussed previously, but these do not affect the later years for which the results were broadly similar to those from 2001 (see Figure 11). [18] Inspection of the plots for each parameter for a year does not reveal any obvious relationship between the Dst index and the errors in any of the autoscaled values. In all cases the largest errors are observed with Dst values close to zero, indicating a geomagnetically quiet period. Comparing the marginal distribution of Dst values for the example year of 2001 (Figure 12) with the form of the autoscaling error distributions in the panel plots makes it clear that the error distributions are largely what one would expect to see if Dst and autoscaling errors were uncorrelated. Indeed, examination of the correlation coefficient, r, between Dst and each of the characteristics for each year reveals that in no case does the magnitude of r exceed 0.17, indicating essentially uncorrelated variables. [19] However, to investigate the possibility that this distribution of Dst values was obscuring any dependence on Dst of the autoscaling errors, another form of plot was produced for the errors in fof2 values for To compensate for the much greater number of observations with near-zero Dst, the observations were binned by Dst value, ensuring that each bin contained at least 200 observations, and that all observations for any given Dst value lay inside a single bin. The resulting plot is shown in Figure 13, where the x axis values are the average Dst value in each bin, and the y axis values are the mean of the magnitude of the fof2 autoscaling errors; 8of11

9 Figure 11. Errors in hourly fof2 plotted against corresponding Dst index values for the years 1996 to the error bars show the standard deviation of the magnitudes in each bin. [20] The plot suggests that there may be a correlation between the level of geomagnetic disturbance and the errors in autoscaled values, but that the correlation is weak and leads to only a small increase in errors. In the case of the 2001 fof2 values the total range of error magnitudes is only roughly 0.15 MHz, which is not significant in the context of the much larger errors shown in Figure 11 for In summary, although the geomagnetic conditions may have an impact on the accuracy of autoscaling, that impact is small and outweighed by other factors. 6. Summary of Analysis [21] Some specific failings of the autoscaling algorithms have been identified associated with interference and early versions of the automatic scaling software. This should not be permitted to give a misleading Figure 12. Distribution of Dst values for the year of11

10 impression of the overall accuracy of the autoscaling process. The improvement in accuracy following the change to the sounder parameters in 1999 is made very clear, and the relative stability of the performance from 2000 to 2004 is also demonstrated. It should also be noted that a new version of the ARTIST software was released in 2005 which significantly reduces the sensitivity to trace gaps [Reinisch et al., 2005]. [22] A useful indication of the overall quality of the performance of the autoscaling can be gained by examining the International Union of Radio Science (URSI) guidance on when ionospheric characteristics are too uncertain to merit quoting a numerical value [Piggott and Rawer, 1972]. Although this guidance is intended to be used at the time of scaling to indicate doubtful values, it gives a benchmark by which the seriousness of the errors in autoscaled values can be judged. The limits are quoted in terms of both absolute deviations (0.25/0.5 MHz or 2/5 km for E/F layer, respectively) and percentage deviations (20%). For simplicity we adopted a single, on average slightly more stringent, condition for each parameter as set out in Table 1. [23] Using the data from 2000 onward, the eight scaled characteristics studied were found to fall into two groups, according to how frequently the errors remain below these limits: (1) >90% reliable fof2, h 0 E, M(3000)F2, and MUF(3000)F2 and (2) >80% reliable fmin, foe, fof1, and h 0 F2. For the vast majority of soundings the accuracy of characteristics is therefore acceptable, with particularly good reliability when scaling the widely used characteristics fof2 and MUF(3000)F2. In general, the scaling is less good for any characteristics likely to be affected by difficulties in distinguishing the relevant traces in the relatively busy low-frequency portion Figure 13. Distribution of binned fof2 autoscaling errors for the year All bins contain at least 200 points. Table 1. Maximum Acceptable Errors in Ionospheric Characteristics Type of Parameter E Layer Limit F Layer Limit Frequency 0.2 MHz 0.7 MHz (1.0 for MUF) Virtual height 20 km 50 km M factor 0.4 of an ionogram: hence the lesser accuracy with fmin, foe, fof1, and h 0 F2. [24] Acknowledgments. The authors wish to thank Rita Blake, the Chilton ionosonde scaler, for all her diligent hours scaling ionograms over the years. Thanks also to Sarah James, Chris Davis, and John Smith of the Ionosonde Group at RAL. This work was funded by the Radio Communications Agency of the U.K. Government, now part of Office of Communications, Ofcom. References Fox, M. W., and C. A. Blundell (1989), Automatic scaling of digital ionograms, Radio Sci., 24(6), Gilbert, J. D., and R. W. Smith (1988), A comparison between the automatic ionogram scaling system ARTIST and the standard manual method, Radio Sci., 23(6), Igi, S., H. Minakoshi, and M. Yoshida (1992), Automatic ionogram processing system: 2. Automatic ionogram scaling, J. Commun. Res. Lab., 39(2), Mayaud, P. N. (Ed.) (1980), Derivation, Meaning and Use of Geomagnetic Indices, Geophys. Monogr. Ser., vol. 22, AGU, Washington, D. C. McNamara, L. F. (2006), Quality figures and error bars for autoscaled Digisonde vertical incidence ionograms, Radio Sci., 41, RS4011, doi: /2005rs Pezzopane, M., and C. Scotto (2005), The INGV software for the automatic scaling of fof2 and MUF (3000)F2 from ionograms: A performance comparison with ARTIST 4.01 from Rome data, J. Atmos. Sol. Terr. Phys., 67(12), Piggott, W. R., and K. Rawer (1972), U.R.S.I. Handbook of Ionogram Interpretation and Reduction, 2nded.,Rep. UAG-23, World Data Cent. A, Boulder, Colo. Reinisch, B. W. (1996), Modern ionosonde, in Modern Ionospheric Science, edited by H. Kohl, R. Rüster, and K. Schlegel, pp , Eur. Geophys. Soc., Katlenburg-Lindau, Germany. Reinisch, B. W., and X. Huang (1983), Automatic calculation of electron density profiles from digital ionograms: 3. Processing of bottomside ionograms, Radio Sci., 18(3), Reinisch, B. W., X. Huang, I. A. Galkin, V. Pznukhov, and A. Kozlov (2005), Recent advances in real-time analysis of 10 of 11

11 ionograms and ionospheric drift measurements with digisondes, J. Atmos. Sol. Terr. Phys., 67, Stamper, R., et al. (2004a), Nowcasting, forecasting and warning for ionospheric propagation: Tools and methods, Ann. Geophys., suppl. S, 47(2 3), Stamper, R., J. Lilensten, and N. Jakowski (2004b), Nowcasting, forecasting and warning for ionospheric propagation: Supporting databases, Ann. Geophys., suppl. S, 47(2 3), Zolesi, B., and L. R. Cander (2006), Effects of the upper atmosphere on terrestrial and Earth-space communications: Final results of the EU COST 271 action, Adv. Space Res., 37(6), R. A. Bamford, L. R. Cander, and R. Stamper, Space Science and Technology Department, Rutherford Appleton Laboratory, Didcot OX11 0QX, UK. 11 of 11