EVALUATION REPORT

Initial Evaluation Of The Effectiveness Of
Mr. Shan R. Shanmugasundaram's
Sun Shadow Deviation Measurement Based
Earthquake Forecasting Procedure
And It’s Significance to Earth Science,
Geophysics & Seismology

Part-2

By:

Chen I-wan, Advisor
Committee of Natural Hazards Prediction
of the China Geophysics Society
(messenger@xrg-china.com)

Evaluation Criteria for Failure, Success, Practical Value & Scientific Value

The criteria I have used for these evaluations is as follows:

Ratings

Days

Lat./Long. (Degrees)

Magnitude (Ms)

Failure

>/=3

>/=3

>/=1.0

Success

</=2

</=2

</=0.5

Practical

>2

>2

>0.5

Value

<3

<3

<1.0

Scientific

>2

>3

>0.5

Value

<3

<4

<1.0

Total Value = Success + Practical Value + Scientific Value

 

Practical Value: The prediction could be used to provide seismic hazard monitoring departments within or close to the concerned region with information that would help them enhance their seismic monitoring efforts around and during the time windows for the expected EQs.

Scientific Value: The prediction would not be accurate enough to be useful as an EQ warning. However, it would be accurate enough to provide evidence that the observed SSD was linked with a temporary shift – tilting in the Earth’s North – South Pole rotation axis.

Format of the Excel Evaluation File:

Shan’s EQ forecasts:  

Forecast Code

Location

Lat.

Long.

From

Until

Ms

20030815SSS-A

Hindu Kush Region, Afghanistan

36.62

71.17

2003.08.14

2003.08.22

Ms5

20030815SSS-B

Andaman Islands

12.14

93.52

2003.08.14

2003.08.22

Ms5

20030815SSS-C

Southern Sumatera

-2.2

105

2003.08.14

2003.08.22

Ms5

20030815SSS-D

Southern Xinjiang, China

40.5

80.5

2003.08.14

2003.08.22

Ms5

20030815SSS-E

East Coast of Kamchatka

51.26

158.87

2003.08.14

2003.08.22

Ms5

20030815SSS-F

Sumba Region

-9.95

119.13

2003.08.14

2003.08.22

Ms5

20030815SSS-G

Banda Sea

-7.4

128.4

2003.08.14

2003.08.22

Ms5

20030815SSS-H

Irian Jaya, Indonesia

-1.6

134.3

2003.08.14

2003.08.22

Ms5

20030815SSS-I

New Britain Region

-5.54

148.94

2003.08.14

2003.08.22

Ms5

 

Forecast Code – Actual EQs Parameters:

Forecast Code

Actual Date

Lat.

Long.

 

 

Location

Source

20030815SSS-A

2003.08.15

37.6

70.4

4.9

mb

Afghanistan- Tajikistan

REDPUMA

EMS

20030815SSS-B

2003.08.21

12.962

93.037

4.7

Ms

Andaman Islands, India Region

NEIC

 

20030815SSS-C

2003.08.21

2.289

96.549

5.2

Ms

Simeulue, Indonesea

USGS

 

20030815SSS-D

2003.08.23

41.6

79

4.7

mb

Kyrgyzstan-Xinjiang Border

REDPUMA

EMS

20030815SSS-E

2003.08.14

55.4

162.4

4.9

Ms

Near East Coast of Kamchatka

REDPUMA

NEI

20030815SSS-F

N/A

N/A

N/A

 

 

N/A

N/A

N/A

20030815SSS-G

2003.08.20

-6.492

129.748

4.8

Ms

Banda Sea

USGS

 

20030815SSS-H

2003.08.22

-0.49

132.36

4.5

Ms

Near North Coast of  Irian Jaya

NEIC

 

20030815SSS-I

2003.08.16

-4.584

151.76

5.3

Ms

New Britain Region, P.N.G.

NEIC

 

 

Forecast Code –Evaluation Statistics Calculation:  

 

Forecast/Actual EQ Error

Evaluation Statistics Calculation

Forecast Code

Days

Lat.

Long.

Mag.

Total

Failure

Success

Practical Value

Scientific Value

20030815SSS-A

0

0.98

-0.77

-0.1

1

　

1

　

　

20030815SSS-B

0

0.822

-0.483

-0.3

1

　

1

　

　

20030815SSS-C

0

4.489

-8.451

0.2

1

1

　

　

　

20030815SSS-D

1

1.1

-1.5

-0.3

1

　

1

　

　

20030815SSS-E

0

4.14

3.53

-0.1

1

1

　

　

　

20030815SSS-F

　

　

　

　

1

1

　

　

　

20030815SSS-G

0

0.908

1.348

-0.2

1

　

1

　

　

20030815SSS-H

0

1.11

-1.94

-0.5

1

　

1

　

　

20030815SSS-I

0

0.956

2.82

0.3

1

　

　

1

　

Notes on the evaluation Excel file:

The above mentioned Excel file containing all the data used in the evaluation is available. I will forward this Excel file to anybody who makes such request by email to me via messenger@xrg-china.com. People can examine those data and draw their own conclusions. Current data can also be obtained from Shan’s Website (http://EQ.itgo.com/today.htm).

Notes on “Forecast / Actual EQ Error”:

Error of Days = Actual Date – Time Window:

l         If actual date of EQ falls within Shan’s predicted time window, then Error of Days = 0;

l         If actual date of EQ is 1 day later than last day of Shan’s time window, then Error of Days = 1;  

Error of Latitude (or Long.) = Actual Latitude (or Long.) – Predicted Latitude (or Long.)

Ø    When an actual EQ occurs, multiple seismic monitoring stations equipped with massive modern equipment determine the longitudes/latitudes of the epicenter. Upon review of the longitudes/latitudes of the same actual EQs announced by NEIC, EMSC, and RedPuma, different degree of differences is found in many cases, and neither network is able to judge and state that the longitudes/latitudes determined by them is the only correct one.  

Ø    Accordingly, in my evaluation process, the longitudes/latitudes closest to the predicted longitudes/latitudes of the same actual EQs announced by NEIC, EMSC, and RedPuma was selected for the evaluation.  

Ø    As all international seismic monitoring networks equipped with massive modern equipment accept and respect the existence of errors they determine for the actual EQs after their occurrence, the above evaluation approach should be considered fair and scientific for the purpose of evaluating Shan’s EQ forecasts conducted at only one site by only one person’s personal efforts based on only one means of SSD observation.  

Error of Magnitude = Magnitude of Actual EQ – Predicted Magnitude:

l        If actual magnitude falls within Shan’s predicted magnitude range, then Error of Magnitude = 0;  

l        If actual magnitude of EQ is 0.5 greater than max. value of Shan’s predicted magnitude range, then Error of     Magnitude = 0.5;  

l        If actual magnitude of EQ is 0.5 less than min. value of Shan’s predicted magnitude range, then Error of Magnitude = - 0.5;  

l        When Shan’s predicted magnitude is stated as Ms4 – Ms5+, the max. value is considered to be Ms5.5; When Shan’s predicted magnitude is stated as Ms5 – Ms6.5+, the max. value is considered to be Ms6.75.  

l       The  scale of magnitude of the actual EQs determined by different networks/stations is different, i.e. Ms, Ml, mb, MW, which contains different degrees of errors. Accordingly, when calculating the Error of Magnitude, only the values are considered, and different scales and the value difference caused by the different scales is ignored.  

Notes on Evaluation Statistics Calculation:

Ø     For Shan’s predictions rated as Failure, or Success, or Practical Value, they are calculated only once and thus Total Predictions = Failure + Success + Practical Value.  

Ø     However, as Shan’s predictions rated as Scientific Value are also rated as Failure with regard to their value as an EQ forecast. Thus Failure + Success + Practical Value + Scientific Value > Total Predictions.  

Notes on adopting the ratings of “Practical Value” and “Scientific Value”:

Ø    EQ prediction evaluation criteria standards adopted by official seismology departments in certain countries take a simple either Success or Failure approach in evaluation of EQ predictions. However, at the same time, with a network of seismic monitoring stations with all their modern equipment and personnel resources and government funds, they have to accept and respect the difficulties and differences they have with regard to accurately determining the latitude/longitude of the epicenter, and the magnitude, and even the exact timing of the occurrence of the actual EQ.  

Ø    Different kinds of geophysical instruments and EQ forecasting techniques/methods, each have their own advantages and limitations. Accurate and reliable EQ predictions as explained here thus requires the approach of multiple techniques/methods, including a network of multiple observation stations formed by both stations in nearby areas as well in remote locations, equipped with multiple different type of geophysical monitoring instruments/techniques.  

Ø    The above mentioned simple Success or Failure approach adopted by official seismology departments in certain countries for evaluation of EQ predictions adopts a degree of “fairness” in face value, i.e. treating all EQ predictions the same. But such a practice ignores the above situation. It tends to simply reject most, if not all, effective EQ forecasting techniques/methods conducted at only one site by only one person’s personal efforts based on only one means of observation. The reason is that such EQ forecasting techniques/methods cannot reach the standard of accuracy and reliability achievable with a network of multiple observation stations formed by both stations in nearby areas as well in remote locations, and equipped with multiple types of geophysical monitoring instruments/techniques.  

Ø     Accordingly, the above-mentioned simple Success or Failure approach adopted by official seismology departments in certain countries for evaluation of EQ predictions is neither a fair nor scientific approach. And it has become another obstacle blocking the recognition of most, if not all, effective EQ forecasting techniques/methods conducted at only one site by only one person’s personal efforts based on only one means of observation.  

Ø    Such an approach is scientifically misleading not only to official seismologists themselves, but also misleading to government officials, the media and the general public.  

Ø    Taking the above into account, to improve scientific fairness, in my evaluation of Shan’s SSD based EQ forecasting method, as explained above, I have adopted two additional scales of rating: Practical Value and Scientific Value.  

Ø     In addition to the simpler Success or Failure approach, by adopting these two extra scales of rating the evaluation result can provide all concerned with a scientifically fair and comprehensive better understanding of not only the effectiveness of Shan’s SSD observation based method for EQ forecasting, but also its significance in provided further scientific evidence to deepen our understandings in basic Earth science, geophysics and seismology theories.  

[ Evaluation Report : Part 1 || Part 3 || Part 4 ]

Please send your comments and suggestions to Chen I-wan <<>> Shan


HOME

CONCEPT

METHODOLOGY

HISTORY

ACK

PREPAREDNESS

FORECAST

Top of this page