Assimilation of ionosonde profiles into a global ionospheric model

Transcription

1 RADIO SCIENCE, VOL. 46,, doi: /2010rs004457, 2011 Assimilation of ionosonde profiles into a global ionospheric model Leo F. McNamara, 1,2 Gregory J. Bishop, 1 and Judith A. Welsh 1 Received 11 June 2010; revised 22 November 2010; accepted 7 January 2011; published 17 March [1] The Utah State University Global Assimilation of Ionospheric Measurements (GAIM) ionospheric model has been run for multiday intervals in November 2008 and February/ March 2009, to investigate the model s ability to assimilate plasma frequency profiles provided by Digisondes. Ionosondes are currently the only type of assimilation data that can provide information on the profile below the peak of the F2 layer. Attention has been focused on the Republic of South Africa, which has four Digisondes and thus offers a unique validation environment. The model has been run for multiple assimilation data scenarios, some of which include GPS total electron content (TEC) observations, in order to provide benchmarks for testing the profile assimilation. The Hermanus Digisonde was set aside to provide the ground truth, in particular the values of fof2, hmf2, and the width of the F2 layer. The values of these characteristics were also tested at the other three ionosonde sites, since the data assimilation procedures do not usually reproduce the assimilated data exactly. It was found that assimilation of ionosonde data did not improve the accuracy of the GAIM values of fof2 at Hermanus (or Grahamstown) beyond that provided by the GPS TEC data, but these TEC only errors were already relatively small. However, the ionosonde only errors were smaller than the relatively large TEC only errors for Louisvale and Madimbo. Assimilation of ionosonde data did not provide any significant increases in the accuracy of the model values of hmf2 and width of the F2 layer. Citation: McNamara, L. F., G. J. Bishop, and J. A. Welsh (2011), Assimilation of ionosonde profiles into a global ionospheric model, Radio Sci., 46,, doi: /2010rs Introduction [2] The Utah State University Global Assimilation of Ionospheric Measurements (GAIM) [Scherliess et al., 2004; Schunk et al., 2004; Thompson et al., 2006] model of the ionosphere is run on a routine basis by the Air Force Weather Agency (AFWA), which has tasked the Air Force Research Laboratory (AFRL) with the model s validation. The starting point for the assimilation is a background physics based model that is driven by real time geophysical indices, the Ionospheric Forecast Model (IFM [Schunk et al., 1997]). The IFM itself relies on several empirical models such as of the thermosphere, thermospheric winds and electric fields [Sojka et al., 2003]. GAIM assimilates observations of such things as slant 1 Air Force Research Laboratory, Hanscom Air Force Base, Bedford, Massachusetts, USA. 2 Now at Air Force Research Laboratory, Kirtland Air Force Base, New Mexico, USA. This paper is not subject to U.S. copyright. Published in 2011 by the American Geophysical Union. total electron content (TEC) made at ground based GPS sites, and thus generates an updated three dimensional worldwide specification of the ionosphere from 92 to 1380 km. The current version of GAIM implemented at AFWA uses a Gauss Markov technique to assimilate the observations, and generates a new specification of the ionosphere every 15 min between ±60 geographic. This model is identified as GAIM GM, and we have tested version v [3] AFRL has performed extensive validations of the GAIM GM model against ground truth [see, e.g., McNamara et al., 2007; Decker and McNamara, 2007; McNamara et al., 2008, 2010]. These validations were mostly concerned with assimilation of GPS TEC and DMSP/SSIES in situ electron density observations. [4] The plasma frequency (or electron density) versus altitude profiles provided by the Digisonde are a unique type of assimilation data, since they are the only data type that currently provides height (profile) information. (GAIM GM v2.4.3 does not assimilate radio occultation data, which does provide height information. RO data will be assimilated by v2.8.1, which is currently being vali- 1of14

2 Table 1. Locations of the Local GPS Ground Sites That Provided Assimilation Data ID Location Country Latitude Longitude harb Pretoria South Africa hrao Krugersdorp South Africa pre1/2 Pretoria South Africa rbay Richardsbay South Africa simo Simonstown South Africa suth Sutherland South Africa sutm Sutherland South Africa wind Windhoek Namibia zamb Lusaka Zambia closed dated by AFRL.) The observed slant TEC and UV radiances are all integrated quantities, with no height information, while the in situ DMSP/SSIES electron densities are for a single altitude ( 840 km). The assimilation of Digisonde profiles was addressed in a limited study by McNamara et al. [2007], who showed that GAIM GM favored the TEC observations from a nearby GPS site over the Digisonde information. [5] The assimilation by GAIM GM of both GPS TEC and ionosonde profiles has previously been investigated by Thompson et al. [2006]. These authors assimilated profiles from 19 globally distributed ionosondes, and used the Bear Lake Observatory (BLO) ionosonde as ground truth. Assimilation of both slant TEC data (from 339 GPS sites) and ionosonde profiles was shown to produce the best comparison to the BLO observations of fof2, while maintaining the fidelity of the global TEC comparison. [6] The present analysis takes advantage of the 4 Digisonde network and multiple GPS sites in the Republic of South Africa (RSA). The four Digisondes offer a regional validation capability that is not available elsewhere. With just one Digisonde, there is the invidious choice of either assimilating the profile information, or using it for validation purposes. We have run GAIM GM for multiple scenarios. For example, we include/exclude GPS TEC observations from within the RSA, GPS TEC observations from global sites external to the RSA, and Digisonde profile information from up to three sites. At least one Digisonde (usually Hermanus) is set aside to provide ground truth. However, the assimilation procedures do not guarantee a perfect fit at the other Digisonde sites, so we also investigate the GAIM GM specifications at those sites. The main study covered an interval in November 2008, with a supplementary study for February/March 2009 to investigate issues that arose from the November study. The ionograms were all autoscaled by ARTIST 5 [Galkin et al., 2007], as distinct from the earlier and less reliable versions of ARTIST used by Thompson et al. [2006]. The profiles are provided to GAIM GM as edp2 files, with an altitude spacing of 1 km. (The current version of GAIM GM uses only the 10 km values.) The new network of Digisonde DPS 4D ionosondes currently being deployed by AFWA will all use ARTIST 5. [7] Our particular concern in this paper is to investigate the improvements in the GAIM GM specifications that accrue from having ionosonde profiles available for assimilation, with and without the usual GPS TEC data. We run GAIM GM with GPS TEC and SSIES in situ electron densities in basically the same manner as it is run by AFWA. These specifications provide a bench mark against which to measure the advantages provided by assimilating ionosonde profiles. All TEC quality control issues are addressed internally by GAIM GM. However, we have been very careful with the ionosonde profiles that are assimilated, since that data source is our main interest. We are well aware of the limitations of autoscaled ionogram data [McNamara, 2006]. However, we wish to separate the issue of poor assimilation data from that of how much advantage GAIM GM actually takes of ionosonde profiles. [8] Section 2 discusses the Digisonde observations and the plasma frequency profiles that are assimilated by GAIM GM. The ionograms are processed automatically (autoscaled), and then further processed so that unreliable profiles are not passed to GAIM GM. Section 3 discusses the correlation between values of DfoF2 and of DhmF2 for different pairs of Digisondes (D is deviation from the monthly median), as well as autocorrelation coefficients for each Digisonde. The utility of the ionosonde information can be expected to be greater when these correlation coefficients are higher. Various assimilation scenarios, with different combinations of assimilation data are described in section 4. Section 5 describes some of the key results for the accuracy of the GAIM GM values of fof2 at Hermanus, the assigned ground truth Digisonde, while section 6 does the same for hmf2. Section 7 discusses the GAIM GM values of the F2 layer width or thickness. All calculations presented in this paper are based on edp2 files generated by the program QualScan [McNamara, 2006]. Section 8 compares edp2 files generated by QualScan and ARTIST 5, and discusses their differences. Section 9 presents a summary of the results of the study, while section 10 discusses their implications. 2. The Assimilated Observations [9] Table 1 lists the GPS sites and their locations, while Figure 1 provides a simple map of the locations. The Lusaka (zamb) site is listed just to note that this potentially useful site seems to have been closed. Simonstown (simo) has also been closed, but not until after November There are two Sutherland sites that are essentially collocated. The same holds for Pretoria. [10] The Digisonde locations are listed in Table 2, and plotted (separately) in Figure 2. The Hermanus, Louisvale 2of14

3 Table 2. Locations of the Four RSA Digisondes Station URSI ID Latitude Longitude Grahamstown GR13L 33.3 S 26.5 E Hermanus HE13N 34.4 S 19.2 E Louisvale LV12P 28.5 S 21.2 E Madimbo MU12K 22.4 S 30.9 E Figure 1. Locations of the IGS sites in the South African region. The stations actually used are listed in Table 1. and Grahamstown Digisondes form a roughly equilateral triangle with sides of 700 km, so the values of fof2 are expected to be reasonably well correlated [McNamara, 2009]. Actual correlation coefficients for the study intervals are given in section 3. Madimbo is 1200 km from Grahamstown and Louisvale, so the correlation coefficients would be expected to be relatively low, with the Madimbo edp2 data having little effect at Hermanus, which we use solely to provide ground truth. [11] The Hermanus Digisonde started up in July The first validation period was chosen to be after this date, and at a time when the more remote Digisondes (Louisvale and Madimbo) were actually operating. Thus the first validation interval ran from November 2008 (days ). This interval included a storm related doubling of the electron densities at all sites on day 330 (25 November 2008). This doubling would not have been unexpected on the basis of the kp and Ap indices, and illustrates the value of real time assimilative models. The second validation interval ran from 11 February through 11 March 2009, after which the productivity of the remote stations decreased substantially. [12] All ionograms were specifically autoscaled using ARTIST 5 [Galkin et al., 2007] by Ivan Galkin at the University of Massachusetts Lowell. At the time, only the Hermanus ionograms were routinely scaled using this latest version of ARTIST. The Hermanus and Grahamstown Digisondes generate ionograms every 15 min, while the other two have a 30 min cadence. [13] The Digisonde profile information is provided to GAIM GM in the form of a table of altitude (1 km grid), plasma frequency, electron density, and an estimated uncertainty in the last two. The data files have a standard extension of edp2, which identifies them to GAIM GM as ionogram profile files. The second line of an edp2 file contains the values of fof2 and hmf2, along with the estimated uncertainties (or confidences). The values of fof2 and hmf2 are not specifically used by GAIM GM as the peak of the profile because the values given by earlier versions of ARTIST were too unreliable when the GAIM algorithms were being developed. The ARTIST 5 values are much more reliable. [14] For the present analysis, the edp2 files were generated by the program QualScan [McNamara, 2006], using the ARTIST 5 autoscaled data. QualScan inverts (i.e., obtains the plasma frequency profile from) the autoscaled ionogram trace using the program POLAN [Titheridge, 1985], but only after performing reasonableness checks (since a nonphysical trace can cause POLAN to fail gracelessly). QualScan assigns a confidence (uncertainty in plasma frequency) at each altitude that is equal to half the difference between the NHPC and POLAN plasma frequencies. (NHPC is the part of ARTIST that deduces the electron density profiles [see Reinisch and Huang, 1983; Huang and Reinisch, 1996].) QualScan is currently an element in the preprocessing of ionograms that provide plasma frequency profiles to AFWA. [15] Some edp2 files contained only a header, either because ARTIST 5 classified the ionogram autoscaling as unreliable, or because QualScan did so. The Hermanus daytime ionograms often have sections of the trace missing on either side of the daytime fof1 cusp. This caused problems with an earlier version of ARTIST 5 and its profiles, as well as with POLAN. The ARTIST 5 issues were quickly resolved, but the missing part of the trace Figure 2. Locations of the four RSA Digisondes. 3of14

4 to have only an average effect (no space weather effects), if any. [17] McNamara [2009] cautioned against using autoscaled values to determine the correlation lengths, but in the assimilation world the autoscaled data represents the only option. We have therefore calculated the correlation coefficients for the autoscaled data at each UT hour for both fof2 and hmf2 for all station pairs, to estimate the utility of the Digisonde data. Actually, the correlation coefficients are for the deviations of fof2 and hmf2 from the monthly median values. We consider both cross correlation (between ionosondes) and autocorrelation (versus time lag for the same ionosonde). Figure 3. Correlation coefficients for deviations of fof2 at Grahamstown and Hermanus, November often leads to nonphysical POLAN profiles, so such ionograms are often rejected by QualScan. QualScan produced valid edp2 files (the others were created, but contained only a header) for only 80% of the RSA ionograms. The long time gaps between edp2 files (partly from missing ionograms) mean that GAIM GM has to model the temporal decay of the isolated observations. The percentage of Grahamstown ionograms accepted by QualScan fell to 65% between about 0600 and 1200 UT for both study intervals. 3. Correlation Coefficients for fof2 and hmf2 [16] In a simple picture, the effects of an observation such as that of fof2 and hmf2 will fall off horizontally as exp( d/d), where D is some correlation distance or length. Mostly for convenience, the correlation length is usually set to the distance at which the correlation coefficient drops to 0.7 [Klobuchar and Johanson, 1977]. Using manually scaled ionograms, McNamara [2009] found that the correlation length for September/October 2006 for the RSA stations was generally greater than 735 km (the side length for the Hermanus, Louisvale and Grahamstown triangle), and certainly less than 1200 km, the distance from Louisvale and Grahamstown to Madimbo. Thus Louisvale and Grahamstown should contribute to the GAIM GM specifications at the ground truth station (Hermanus), but Madimbo would be expected 3.1. Cross Correlation Coefficients for fof2 [18] Figure 3 shows the cross correlation coefficients (blue upper curve) for deviations of fof2 at Grahamstown and Hermanus, the stations with 15 min cadences, for November The red lower curve shows the counts for each hour (by 0.01) for which QualScan provided a value of fof2 for both locations. [19] The correlation coefficients exceed 0.8 (a comfortably high value; a value of 0.7 is usually taken as a good value) for most of the 24 h, but not for around dawn. With autoscaled data, some of the low correlation coefficients arise from a single outlier value caused by an autoscaling blunder. The fact that the ionogram traces are often spread at night could also lead to low correlation coefficients, since ARTIST could handle the spread traces at different sites a little differently. The low value of at 0330 UT is actually due to a relatively large uncertainty in the values of fof2 compared with the day to day variability at that time, which is only ±0.5 MHz. At 1200 UT, the correlation coefficient is 0.945, and the points for each day lie close to the line of best fit. [20] The fof2 correlation coefficients for Hermanus Louisvale and Grahamstown Louisvale generally exceed 0.8 from 0800 to 2400 UT (from 1000 to 0200 LT), but are very noisy and below 0.6 at other times. For Grahamstown Madimbo, the correlation coefficients are also very noisy. They reach 0.6 in the middle of the day, but drop below 0.2 at night. These low values are consistent with the large separation ( 1200 km). [21] It should be noted that the correlation coefficients paint a somewhat pessimistic picture of how much a Digisonde can contribute to the GAIM GM specification at the location of the ground truth site, which is Hermanus. This is because Hermanus itself has autoscaling uncertainties, somewhat compromising the concept of ground truth. [22] The high correlation coefficients for fof2 should manifest themselves in the form of accurate specifications of fof2 at Hermanus and Louisvale when the only data 4of14

5 Hermanus can potentially lead to large GAIM GM errors in hmf2 at Hermanus. Figure 4. Correlation coefficients for deviations of hmf2 at Grahamstown and Hermanus, November assimilated is the Grahamstown edp2 data (case R18, as described in section 5.3) Cross Correlation Coefficients for hmf2 [23] The correlation coefficients for hmf2 are significantly lower than those for fof2. Figure 4 shows the November 2008 correlation coefficients for deviations in hmf2 at Grahamstown and Hermanus, the stations with 15 min cadences. [24] The correlation coefficients are even more erratic and lower for Grahamstown Louisvale and Grahamstown Madimbo. This low correlation is understandable, given that hmf2 is a derived parameter that relies on the validity of the whole autoscaled ionogram trace and the successful inversion of the trace to derive the profile. The accuracy of the inversion process is limited by a lack of specific knowledge of the distribution of ionization below the lowest ionogram frequency, fmin, especially at night, and in the daytime E F1 valley. Model distributions must be applied in both cases. There is in fact some current concern with the model of the nighttime underlying ionization that is used in QualScan, since it does not match observed Arecibo incoherent scatter profiles (B. W. Reinisch, private communication, 2009). While this issue has yet to be resolved, it is not relevant to the present study because the assimilated and ground truth profiles are all derived by QualScan, so the analysis is self consistent. The noise level in the values of hmf2 at 3.3. Autocorrelation Coefficients for fof2 and hmf2 [25] The autocorrelation coefficients for deviations of fof2 from the median value for the four Digisondes are shown in Figure 5, as a function of the lag in hours, for November The autocorrelation coefficients are important because they influence the success with which GAIM GM can account for missing ionograms. [26] Figure 5 shows that the autocorrelation coefficient decreases linearly with the lag, starting at 0.9 for 0.25 h (sequential ionograms) and passing through 0.7 at 1.5 h. The Madimbo points consistently lie below those for the other Digisondes. The coefficients are somewhat lower for the February/March 2009 interval, passing though 0.7 at 1.0 h. For the same interval, the correlations are significantly higher for nighttime ionograms than for daytime ones. For the November 2008 interval, the correlations are similar for day and night, except for Madimbo, which has lower correlations at night. [27] Figure 6 shows the autocorrelation coefficients for deviations of hmf2 from the median value for November As with the fof2 autocorrelation, the decrease of the hmf2 autocorrelation with increasing lag is approximately linear. However, the coefficient never exceeds the desired value of 0.7. The coefficients are even lower for February/March They are lower at night Figure 5. Autocorrelation coefficients for deviations of fof2 for the four Digisondes, November of14

6 Figure 6. Autocorrelation coefficients for deviations of hmf2 for the four Digisondes, November than during the day for both study intervals. The lowest daytime autocorrelations are for Hermanus, which is probably a result of the gaps in the ionogram trace around fof1that cause problems for POLAN (as mentioned at the end of section 2). These low autocorrelation coefficients for DhmF2 suggest that the GAIM GM values of hmf2 will not be a substantial improvement over the values obtained without assimilating edp2 data, especially at night. This point is discussed in section The Assimilation Scenarios [28] The various assimilation scenarios discussed here are listed in Table 3, in which GR13L is Grahamstown, LV12P is Louisvale, and MU12K is Madimbo. Some of the earlier case numbers were later made irrelevant. Recall that we need to consider cases for which only GPS TEC and SSIES electron densities are assimilated in order to provide a benchmark for the contributions of the ionosonde profiles. A more detailed study of GAIM GM s performance when these other data types are assimilated without ionosonde profiles is given by McNamara et al. [2008]. Ionosondes were simply used in that paper to provide ground truth. [29] The logic of these scenarios is as follows. GPS TEC is the main type of assimilation data used by GAIM GM. In many regions of the world, it is the only type of data available for assimilation (apart from the DMSP/SSIES electron densities at 840 km, which have minimal impact at the F2 peak). All GPS indicates that the data from the RSA GPS sites was assimilated, along with that from another 45 GPS sites around the world. Case R02 thus represents the typical situation. Case R03 represents an ideal case, in which the full set of TEC observations is complemented by three sets of edp2 data. These results might be expected to be the most accurate of all, but the picture is actually quiet complex, as discussed in section 5.2. [30] For cases R06 and R08, the TEC data from the local GPS sites (those in the RSA) is not assimilated. These cases are representative of areas with no GPS sites. Case R06 includes the edp2 data from three Digisondes, whereas case R08 does not. The case R08 results can be expected to be the least accurate. The Grahamstown edp2 files would be expected to have the most impact at Hermanus, the ground truth site, since they have a 15 min cadence and few data gaps. [31] For cases R16 and R18, no TEC data is assimilated at all, and GAIM GM is given only the edp2 data from different Digisondes. The cases considered for the second analysis interval, February/March 2009, are R02, R03, R06, and R18. Note that the DMSP/SSIES in situ electron densities are always assimilated, so we do not keep referring to them. 5. GAIM GM Values of fof2 [32] The validations discussed here are restricted to the parameters of the F2 peak, fof2, hmf2, and the F2 width. The GAIM GM profiles below 200 km are basically just the IFM profiles, and do not agree well with Digisonde profiles (unpublished AFRL studies) Assimilation of GPS TEC Only [33] Figure 7 shows the observed and GAIM GM case R02 values of fof2 at Hermanus for each 15 min for days , November It can be seen from Figure 7 that the GAIM GM values of fof2 are quite accurate, except that they are too low near the predawn minimum. They obviously track fof2 during the disturbed day (330) very well. However, this type of display tends to hide some of the errors that occur during rapid changes of fof2. For example, it is not obvious that the GAIM GM values of fof2 take a few days to return to the correct levels Table 3. The Various Assimilation Scenarios Case ID GPS TEC Digisonde edp2 Files R02 All GPS R03 All GPS 3 GR13L, LV12P, MU12K R06 No RSA GPS 3 GR13L, LV12P, MU12K R08 No RSA GPS R16 No GPS at all 3 GR13L, LV12P, MU12K R18 No GPS at all 1 GR13L 6of14

7 Figure 7. Observed (black pluses) and GAIM GM (blue line) values of fof2 for Hermanus, days , November Case R02. after the large disturbance on day 330. Figure 8 therefore shows the diurnal variation of the average error in the GAIM GM values of fof2 at Hermanus, for case R02 (GPS TEC only) and case R08 (No RSA TEC; No edp2 data). [34] The R02 errors (blue 2 in Figure 8) are about 0.2 ± 0.6 MHz. The R08 curve (red 8 ) shows some larger errors, especially during the night. This is an effect of not having any local GPS TEC observations, a common scenario in the real world. [35] The GAIM GM values of fof2 for Grahamstown have an accuracy that is very similar to that of the Hermanus values. However, the errors are significantly larger for Louisvale and Madimbo, the two northern stations. (The GAIM GM assimilation algorithms do not attempt to match the assimilated data exactly, but consider the data along with all other available information. This means that the GAIM GM specifications at Grahamstown, Louisvale and Madimbo will not necessarily match the assimilated data. It is therefore legitimate to compare the GAIM GM and assimilated values of the F2 peak parameters.) [36] Figure 9 shows that the GAIM GM R02 values of fof2 for Louisvale are systematically high on most days. The data gaps in Figure 9 also emphasize the days that had no ionograms, an occupational hazard for remote locations. The average errors in fof2 at both Louisvale and Madimbo were about 0.6 ± 0.6 MHz for case R02. The R08 errors (no local TEC) for Louisvale and Madimbo were similar to those for Hermanus and Grahamstown, Figure 8. Diurnal variation of the average error in the GAIM GM values of fof2 for Hermanus, for cases R02 and R08, November which is as expected since the remote rest of the world (together with the background IFM) is determining the GAIM GM specifications in the RSA. [37] GAIM GM generally uses the GPS TEC observations to good effect in deriving the values of fof2 at nearby Figure 9. Observed (black pluses) and GAIM GM (blue line) values of fof2 for Louisvale, days , November Case R02. 7of14

8 Figure 10. Diurnal variation of the average errors in GAIM GM Hermanus values of fof2 for cases R02 and R03, November locations. This was noted by Decker and McNamara [2007], who found that the GAIM GM predictions of fof2 at locations in Australia were more accurate when there was a nearby GPS site. In the present study, the TEC only values of fof2 (case R02) are quite accurate for Hermanus and Grahamstown, but there are systematic errors for Louisvale and Madimbo (discussed further in section 5.4). Figure 11. Diurnal variation of the average errors in GAIM GM Hermanus values of fof2 for cases R02 and R03, February/March variation can be expected to be different for different data sets. [40] The scatter about the average is of course also important. Figure 12 shows (mainly as an example) the 5.2. Assimilation of GPS TEC and Digisonde Profiles [38] Case R03 (All GPS, 3 edp2 files) should provide the most accurate values of fof2, since all of the relevant data is assimilated. Figures 10 and 11 show the diurnal variation of the average GAIM GM errors in fof2 at Hermanus for cases R02 (All GPS, No edp2) and R03, for the two study intervals. The difference is that the edp2 data is assimilated for case R03. [39] Apart from the large errors near 0600 UT for February/March 2009 (Figure 11), there really is not much to choose between the R02 and R03 errors, which are generally smaller than 0.5 MHz. The observed values of Hermanus fof2 (November 2008) plotted in Figure 7 provide a useful reference level. During the day, the average errors are 0.4/6.0 or 7% of fof2. We return to the 0600 UT peak in the discussion following Figure 13. The different diurnal variations of the fof2 results for the two study intervals (Figures 10 and 11) indicate that this Figure 12. Diurnal variation of the standard deviation of the errors in GAIM GM Hermanus values of fof2 for cases R02 and R03, February/March of14

9 Figure 13. Diurnal variation of the average error in the GAIM GM values of fof2 for Louisvale, for cases R02 and R18 (the red 8), February/March Only the Grahamstown edp2 data were assimilated. standard deviation of the GAIM GM errors in fof2 for cases R02 and R03, February/March [41] The standard deviation is 0.6 ± 0.2 MHz, and about the same for the two cases. For the November 2008 data, the R02 and R03 standard deviations are almost identical, with a lower value of 0.35 ± 0.15 MHz. Recall that the fof2 cross correlation coefficients were higher for the November 2008 study interval. The diurnal variation of the standard deviations would be expected to be inversely related to the cross correlation coefficients, but there is no obvious relationship between them Assimilation of edp2 Data Only [42] It is not intended that GAIM GM be run with only edp2 data, since there are usually GPS TEC sites in the same region as the ionosondes. However, validation has its own rules, since one of its aims is to stress GAIM GM and see if it fails. We have specifically studied several cases, including cases R16 and R18, which assimilate only edp2 data. Note that these GAIM GM specifications would not be expected to be valid outside the RSA, since the correlation lengths for fof2 are less than 1000 km. [43] Figures 10 and 11 showed that the GAIM GM values of the Hermanus fof2 for case R02 (GPS TEC only) are quite accurate. The same holds for Grahamstown. However, the errors are larger (and positive) at Louisvale and Madimbo. We have therefore investigated whether or not the GAIM GM values of fof2 at Louisvale and Madimbo would have been more accurate if the GPS TEC had not been assimilated. Figure 13 illustrates the average error in the GAIM GM values of fof2 at Louisvale, for February 2009, for cases R02 and R18. For case R18, the only data assimilated is the Grahamstown edp2 data. [44] If we initially ignore the large case R18 errors near 0600 UT, Figure 13 shows that assimilating the edp2 data by itself produces smaller errors than those provided by case R02 (GPS TEC only). The results for Madimbo are very similar, except that the R02 overestimates of fof2 during the day reach 1.5 MHz. A 0600 UT error peak also appeared for R03 at Hermanus (Figure 12) but with smaller amplitude. Plotting the diurnal variations of the individual values of fof2 shows in fact a spuriously high peak in the R18 values of fof2 near 0600 UT that does not appear in the observations or in the R02 values. There is also a small peak in the R18 curve at 1400 UT, which may be associated with the low correlation coefficients that occur from 1400 to 1700 UT. For Madimbo, the 1400 UT peak is higher than the 0600 UT peak. [45] For the November 2008 interval, the R02 errors for Louisvale were about 1.0 ± 0.5 MHz, while the R18 errors were about 0.25 ± 0.5 MHz. Thus the Grahamstown edp2 data provided overall more accurate values of the Louisvale fof2 than those provided by the GPS TEC data for both study intervals. The large errors near 0600 UT for case R18 confirm the stabilizing value of TEC observations, although they can lead to larger errors overall. The question of suitable relative weights for GPS TEC and ionosonde assimilation data has not yet been resolved. The quality of the ionosonde data is an important aspect of this question. The November 2008 errors did not exhibit the large peak seen at 0600 UT for the February/March 2009 interval F2 Slab Thickness [46] We have interpreted the overestimates of fof2 at Louisvale and Madimbo that occur for both study intervals in terms of an incorrect GAIM GM slab thickness. Most likely this deficiency is the result of an incorrect effective modeling of the slab thickness in the underlying physicsbased IFM model. (The slab thickness is not modeled specifically, but its use simplifies the discussion.) [47] The ionospheric slab thickness is simply the ratio of TEC to NmF2. In terms of fof2 (MHz), the slab thickness in km is given by. h i S ¼ TEC 1:24E10ðf of2þ 2 The TEC is expressed in TECU units, or el/m. [48] Figures 14 and 15 show the slab thickness plots for corresponding Hermanus and Louisvale Digisonde values of NmF2 and the vertical TEC values for the closest GPS TEC site, which is SUTH for both cases. The values of 9of14

10 Figure 14. Observed/GAIM GM values of fof2 (squared) versus observed vertical TEC for Hermanus (Digisonde) and SUTH (GPS TEC), February/March See the text for more detail. vertical TEC were mapped from the slant TEC observations in the usual way, with a lower elevation limit of 40. [49] The (blue) points in Figures 14 and 15 are the corresponding case R02 values of the GAIM GM (fof2) 2 and the observed vertical TEC. The dot dash (red) lines are the least squares fit (LSF) lines through these points. The dashed (black) lines are the LSF lines through the corresponding observed values of NmF2 and vertical TEC. These points are not shown. [50] The first point to note about these two plots is the good agreement between the LSF lines for Hermanus, to which we can attribute the accurate GAIM GM values of fof2 when only TEC data is assimilated edp2 data (case R02). The slopes of the dashed lines (the observations) for all four Digisondes sites are very similar, indicating that the ionosphere is correlated over the modeled region. However, the slopes of the GAIM GM LSF (dot dash) lines are significantly higher than for the observations for Louisvale (Figure 15). This suggests that GAIM GM will tend to overestimate the values of NmF2 corresponding to an observed value of vertical TEC. This is confirmed by the R02 overestimate of fof2 for Louisvale that can be seen in Figure 13 (the blue 2 ). 6. GAIM GM Values of hmf2 [51] When only GPS TEC data is assimilated, the GAIM GM subpeak profiles (i.e., below hmf2) are basically the IFM profiles, apart from adjustments to match the assimilated values of TEC that have little effect on hmf2. The IFM values of hmf2 tend to be too high by km at midlatitudes during the day (unpublished AFRL studies), and this bias remains in the GAIM GM profiles. [52] For case R02, the full range of the GAIM GM values of hmf2 is only 30 km, much less than the range of the Digisonde values, which exceeds 100 km. This is illustrated in Figure 16 for Grahamstown (the Hermanus plot is similar, but more cluttered). [53] The GAIM GM values (blue curve) of hmf2 are generally too high. The lowest observed values of 225 km occur at 0400 UT, while the highest values of 275 km occur at night (about UT). On the disturbed day 330, the observed values of hmf2 between about 0400 and 1100 UT were 300 km, and significantly higher than the median values. [54] Figure 17 shows the diurnal variation of the average error in the GAIM GM values of hmf2 for Hermanus, for cases R02 (All GPS) and R03 (All GPS; 3x Edp2) for February/March Figure 17 shows an improvement of a few km in the values of hmf2 during the day when edp2 data is assimilated, but there is no systematic overall improvement. [55] The basic diurnal curve of the height errors is a property of the IFM, and differs with location and epoch. In the present examples, the errors in hmf2 are a little lower at around 1600 UT when the edp2 data is assimilated Figure 15. Observed/GAIM GM values of fof2 (squared) versus observed vertical TEC for Louisvale (Digisonde) and SUTH (GPS TEC), February/March See the text for more detail. 10 of 14

11 were assimilated for those studies. This suggests that the shape of the GAIM GM subpeak F2 profile is basically just that of the IFM. [58] The issue here is whether or not GAIM GM takes advantage of the ionogram profiles that are provided in the edp2 files to adjust the F2 layer shape (the thickness at 0.8 fof2 in particular). Figure 18 shows in fact that it does not. [59] Figure 18 shows that there is virtually no correlation between the Hermanus GAIM GM and POLAN values of the F2 thickness for the case in which only the Grahamstown edp2 data is assimilated. The average values agree (47.7and 47.5 km for POLAN and GAIM GM), but the day to day and diurnal variability of the GAIM GM values is only about one third of the observed (POLAN) variability. We deduce therefore, that GAIM GM does not specifically adjust the IFM subpeak profile in response to the shape of the assimilated profiles. The profiles are mainly adjusted to match the TEC observations. Figure 16. Observed (black pluses) and GAIM GM (blue line) values of hmf2 for Grahamstown, days , November Case R02. (case R03), but the errors are generally higher at night. Recall that the autocorrelation and cross correlation coefficients for hmf2 are very low during the night. For the November 2008 interval, the errors for cases R02 and R03 are virtually identical. 7. GAIM GM Values of F2 Layer Thickness [56] The thickness of the F2 layer is the third key characteristic of the subpeak F2 layer. The width or thickness of the F2 layer is usually characterized by the difference between hmf2 and the height at which the electron density drops to some fraction of its peak value. Two common choices for the lower height are for 0.5 NmF2 and (1/e) NmF2. Neither of these choices is suitable for the present solar minimum analysis because the corresponding height during the day is often in the F1 or even the E region. We have therefore chosen to use as the lower height the height at which the plasma frequency is equal to 0.8 fof2, which is about km below hmf2 for the present analysis. [57] Earlier (unpublished) AFRL validation studies have shown that the shape of the GAIM GM F2 layer is in reasonable agreement with the average of the ARTIST/ NHPC (or POLAN) profiles at 0.8 fof2, but that the GAIM GM profiles have much less day to day variability than the NHPC profiles. Only GPS TEC observations 8. Using ARTIST 5 edp2 Files [60] Most of the edp2 files currently assimilated operationally by GAIM GM are generated by QualScan. However, the new Digisondes being deployed by AFWA will provide edp2 files generated by ARTIST 5 [Reinisch et al., 2009]. The ARTIST 5 edp2 files have confidences with a better physical basis than those provided by QualScan. However, these confidences currently do not Figure 17. Diurnal variation of the average error in the GAIM GM values of hmf2 for Hermanus, for cases R02 and R03, February/March of 14

12 Figure 18. Corresponding Digisonde/POLAN and GAIM GM case R18 values of the F2 layer thickness at 0.8 fof2. For case 18, the only data assimilated are the Grahamstown edp2 data. November include the difference between the POLAN and NHPC profiles at each altitude, which are legitimate differences that arise because of the different assumptions about the missing ionization that are made by QualScan and NHPC. In a future version, ARTIST 5 will spawn a version of QualScan to obtain the POLAN profile and if necessary include the difference between the NHPC and POLAN plasma frequencies into its estimates of the confidences. With these changes to the ARTIST 5 edp2 files, there appears to be no need to use the QualScan edp2 files for ionograms scaled by ARTIST 5. (QualScan is still required for ionograms scaled by earlier versions of ARTIST.) [61] As expected, using QualScan and ARTIST 5 edp2 files (separately) yields similar errors in the GAIM GM values of fof2 at Hermanus for case R18 (the only case considered). The diurnal variations of the two sets of average errors agree in the times of maximum and minimum errors, but not in their detailed values. The errors would not be expected to be identical because QualScan rejects some ionograms that ARTIST 5 accepts. [62] The QualScan and NHPC profiles are very similar during the day, but there are some significant differences at night, when the scaled trace starts at 1.5 MHz and an altitude of 200 km. Figure 19 shows three pairs of Digisonde 1 km plasma frequency profiles for Grahamstown as given by QualScan/POLAN and ARTIST5/NHPC, November 2008, 1800 UT. Figure 19. Digisonde plasma frequency profiles for Grahamstown given by QualScan/POLAN and ARTIST5/ NHPC, November 2008, 1800 UT. [63] The corresponding profiles usually agree quite well at the peak (fof2, hmf2). However, there are obvious differences at the lowest scaled frequency of 2 MHz (as mentioned in section 3.2). The differences are perhaps best Figure 20. Altitude dependence of the confidence in the plasma frequency for Grahamstown given by QualScan/ POLAN and ARTIST5/NHPC, November 2008, 1800 UT. 12 of 14

13 seen in Figure 20, which shows the altitude dependence of the error/confidence in plasma frequency for the same situation. [64] For example, the QualScan error bars (black pluses) for the lowest curve show that the POLAN and NHPC profiles differ by MHz at 200 km. The QualScan errors bars are obviously much larger than the ARTIST 5 error bars. In fact, the QualScan error bars are always constrained to have a minimum value of 0.1 MHz, since smaller values seem unrealistic. The ARTIST 5 confidences are so small that they are probably less than the GAIM GM representation errors. (Because GAIM GM represents a whole spatial grid cell by a single point, account must be taken of the expected change in electron density across the cell, which is expressed as a representation error for that point.) 9. Discussion [65] The present study must be considered as preliminary, in that there are known issues that have not yet been addressed. For example, we have not considered (1) the penalty of having GAIM GM include ionograms that were incorrectly scaled and labeled as acceptable by QualScan, (2) the penalty of having GAIM GM exclude ionograms that were wrongly labeled as unacceptable by QualScan, (3) how GAIM GM copes with strings of missing ionograms, or (4) the impact of using ionograms scaled by the less reliable versions of ARTIST (we have used ARTIST 5 only). [66] We have also investigated just one geographic location for 2 months at solar minimum, although this has been sufficient to define the significant limitations of GAIM GM s use of the ionosonde profiles. [67] The utility of the edp2 data relative to the GPS TEC data depends partly on the reliability of the Digisonde profiles. During the early development of GAIM GM, the Digisonde ionograms were processed by ARTIST 4, and the profiles sometimes rendered invalid because the autoscaling tended to stop at the lower edge of a prohibited frequency band (over which the Digisonde could not transmit). This issue was resolved with ARTIST 4.5. The newer ARTIST 5 uses a different approach, and gaps that occur in the trace because of prohibited frequency bands are not an issue. ARTIST 4.5 gives poor results for daytime equatorial ionograms (unpublished AFRL study), but ARTIST 5 does not. edp2 files from ionograms scaled by all three versions of ARTIST are currently being assimilated at AFWA. [68] The utility of the GPS data, and thus the validity of the GAIM GM specifications of fof2 based upon GPS data, will be compromised by inaccuracies in the IFM profiles, as discussed in section 5.4. This is a further instance of the need for the IFM to be as good as possible, not just for regions for which there is no assimilation data. [69] The GAIM GM model is currently being modified in a somewhat ad hoc fashion to cope with its various limitations as determined by ongoing validations. We have seen, for example, that it does not generally produce accurate values of the peak height and thickness of the F2 layer even when it assimilates edp2 data. USU has already produced its full physics model (GAIM FP [Scherliess et al., 2009]), which is expected to obviate many of the limitations found with the GAIM GM model when it is installed at AFWA. Full use of the information provided by the Digisonde profiles will probably have to wait for GAIM FP to replace GAIM GM. 10. Conclusions [70] We have found that GAIM GM successfully assimilates Digisonde profile information, either separately or together with GPS TEC data, but that it does not take full advantage of the profile information. The following are some specific results: [71] 1. For Hermanus (the ground truth site), both study intervals, the average errors in the GAIM GM values of fof2 for cases R02 (assimilation of all GPS TEC data) and R03 (assimilation of all GPS TEC data, plus edp2 data from Grahamstown, Louisvale, and Madimbo) are usually smaller than 0.5 MHz. Assimilation of edp2 data did not decrease the errors in any consistent fashion. See Figures 10 and 11. The results are similar for Grahamstown (recall that the assimilation procedures do not fit the assimilated data exactly, so it is legitimate to consider assimilation stations). [72] 2. For the two northern stations, Louisvale and Madimbo, GAIM GM tends to overestimate fof2 when GPS TEC data is assimilated without edp2 data. See Figure 9 for Louisvale November 2008, and Figure 13 for Louisvale February/March 2009 (the blue 2 ). [73] 3. For the two northern stations, Louisvale and Madimbo, assimilation of edp2 data by itself gives values of fof2 with smaller errors than those given when only GPS TEC data is assimilated, but with some caveats associated with invalid GAIM GM values of fof2 around 0600 UT. See Figure 9 for Louisvale, November [74] 4. When only GPS TEC data is assimilated (case R02), the accuracy of the GAIM GM values of fof2 depends on the accuracy of the IFM s slab thickness values (simply put). See section 5.4. [75] 5. Assimilation of edp2 data has limited impact on the accuracy of the GAIM GM values of hmf2, and then only during the day. See Figure 17 for Hermanus, February/March The correlation coefficients for DhmF2 are particularly low at night (Figure 4), so the lack of improvement is not surprising. [76] 6. The GAIM GM values of the F2 width are not affected by assimilated edp2 data. 13 of 14

14 [77] 7. The utility of the edp2 data is controlled to some extent by the size of the cross correlation (between locations) and autocorrelation (different time lags at the one location) of the deviations DfoF2 and DhmF2 from the monthly median values. See section 3. [78] 8. GAIM GM took several days to recover from the doubling of the electron densities on day 330, November A careful look at Figure 7 reveals that GAIM GM overestimates the daytime values of fof2 for case R02 on days 331 and 332. The decay back to small errors seems to be exponential. A similar situation held for case R16 (assimilation of only edp2 data). [79] Recall again that the validation results are flavored somewhat by the noise in the autoscaled F2 peak parameters at the ground truth site (Hermanus). [80] Acknowledgments. This work was performed as part of an AFRL validation effort in support of AFWA. We greatly appreciate the continuing cooperation provided by Utah State University (Don Thompson) and University of Massachusetts Lowell (Ivan Galkin and Bodo Reinisch). This investigation would not have been possible without the good quality ionograms provided by the RSA Digisonde network operated by the Hermanus Magnetic Observatory. References Decker, D. T., and L. F. McNamara (2007), Validation of ionospheric weather predicted by Global Assimilation of Ionospheric Measurements (GAIM) models, Radio Sci., 42, RS4017, doi: /2007rs Galkin, I. A., G. M. Khmyrov, A. V. Kozlov, B. W. Reinisch, X. Huang, and V. Paznukhov (2007), The ARTIST 5, in Radio Sounding and Plasma Physics, AIP Conference Proceedings, edited by P. Song, Am. Inst. of Phys., Melville, N. Y. Huang, X., and B. W. Reinisch (1996), Vertical electron density profiles from the Digisonde network, Adv. Space Res., 18, , doi: / (95) Klobuchar, J. A., and J. M. Johanson (1977), Correlation distance of mean daytime electron content, Rep. AFGL TR , Air Force Geophys. Lab., Bedford, Mass. McNamara,L.F.(2006),Qualityfiguresanderrorbarsfor autoscaled Digisonde vertical incidence ionograms, Radio Sci., 41, RS4011, doi: /2005rs McNamara, L. F. (2009), Spatial correlations of fof2 deviations and their implications for global ionospheric models: 2. Digisondes in the United States, Europe, and South Africa, Radio Sci., 44, RS2017, doi: /2008rs McNamara, L. F., D. T. Decker, J. A. Welsh, and D. G. Cole (2007), Validation of the Utah State Univ. Global Assimilation of Ionospheric Measurements (GAIM) model predictions of the maximum usable frequency for a 3000 km circuit, Radio Sci., 42, RS3015, doi: /2006rs McNamara, L. F., C. R. Baker, and D. T. Decker (2008), Accuracy of USU GAIM specifications of fof2 and M(3000)F2 for a worldwide distribution of ionosonde locations, Radio Sci., 43, RS1011, doi: /2007rs McNamara,L.F.,J.M.Retterer,C.R.Baker,G.J.Bishop, D. L. Cooke, C. J. Roth, and J. A. Welsh (2010), Longitudinal structure in the CHAMP electron densities and their implications for global ionospheric modeling, Radio Sci., 45, RS2001, doi: /2009rs Reinisch, B. W., and X. Huang (1983), Automatic calculation of electron density profiles from digital ionograms: 3. Processing of bottomside ionograms, Radio Sci., 18(3), , doi: /rs018i003p Reinisch, B. W., G. S. Sales, V. Paznukhov, I. A. Galkin, Q. Zong, G. Khmyrov, V. Galushko, and D. F. Altadill (2009), section 3 Automatic Ionogram Scaling, ARTIST, Equator and High Latitude Ionosphere to Magnetosphere Research, annual report, Cent. for Atmos. Res., Univ. of Mass. Lowell, 4 Dec. Scherliess, I., R. W. Schunk, J. J. Sojka, and D. C. Thompson (2004), Development of a physics based reduced state Kalman filter for the ionosphere, Radio Sci., 39, RS1S04, doi: /2002rs Scherliess, L., D. C. Thompson, and R. W. Schunk (2009), Ionospheric dynamics and drivers obtained from a physicsbased data assimilation model, Radio Sci., 44, RS0A32, doi: /2008rs Schunk, R. W., J. J. Sojka, and J. V. Eccles (1997), Expanded capabilities for the ionospheric forecast model, final report, AFRL VS HA TR , Schunk, R. W., et al. (2004), Global Assimilation of Ionospheric Measurements (GAIM), Radio Sci., 39, RS1S02, doi: /2002rs Sojka, J. J., D. C. Thompson, R. W. Schunk, J. V. Eccles, J. J. Makela, M. C. Kelley, S. A. Gonzalez, N. Aponte, and T. W. Bullett (2003), Ionospheric data assimilation: Recovery of strong mid latitudinal density gradients, J. Atmos. Sol. Terr. Phys., 65, , doi: /j.jastp Thompson, D. C., L. Scherliess, J. J. Sojka, and R. W. Schunk (2006), The Utah State University Gauss Markov Kalman filter of the ionosphere: The effect of slant TEC and electron density profile data on model fidelity, J. Atmos. Sol. Terr. Phys., 68(9), , doi: /j.jastp Titheridge, J. E. (1985) Ionogram analysis with the generalised program POLAN, Rep. UAG 93, World Data Cent. A for Sol. Terr. Phys., Boulder, Colo. G. J. Bishop and J. A. Welsh, Air Force Research Laboratory, Hanscom Air Force Base, Bedford, MA 01731, USA. L. F. McNamara, Air Force Research Laboratory, Kirtland Air Force Base, NM 87117, USA. (leo.mcnamara@kirtland. af.mil) 14 of 14