3.5 Fundamental limits from chaos on instability time predictions in compact planetary systems

In the paper “Predicting the long-term stability of compact multiplanet systems” (Tamayo et al. 2020) an interesting approach to replace simulation runs by a machine learning algorithm.

Astrophysics is a science which has a long history of handling huge amount of data, no wonder that there are also ML applications in this field. This application helps to determine whether or not a planetary system will be stable. Until now the stability is determined by calculating an astonishing \(10^9\) orbits. Reducing simulation time by factor of 10,000 by combining analytical understanding of resonant dynamics in two-planet systems with machine learning was accomplished.

Reducing simulation time:

  • Combine
    • analytical understanding
    • machine learning (XGBoost)
    • calculation of only \(10^4\) orbits instead of 1\(0^9\)

The following graph shows the flow of computation and the utilization of machine learning to reduce the number of necessary simulations.

Figure from (Hussain and Tamayo 2020)

Training the model:

  • simulate 100,000 initial conditions simulated
    • 80% training set
    • 20% test set
    • machine learning (XGBoost)

3.5.1 Comparison between SPOCK and previous models

The comparison between SPOCK and previous models are shown below. At an FPR of 10%, SPOCK correctly classifies 85% of stable systems. According to the shown results SPOCK is a huge advance from previous models.

Comparison between SPOCK and previous models:

  • At a false positive rate: 10%
    • SPOCK has true positive rate: 85%
    • N-body has true positive rate: 100%
    • MEGNO, AMD, and Hill < 50%

with SPOCK being \(10^5\) faster than N-body

Figure from (Hussain and Tamayo 2020)

An explanation of ROC is given at chapter


Hussain, Naireen, and Daniel Tamayo. 2020. “Fundamental Limits from Chaos on Instability Time Predictions in Compact Planetary Systems.” Monthly Notices of the Royal Astronomical Society 491 (4): 5258–67.

Tamayo, Daniel, Miles Cranmer, Samuel Hadden, Hanno Rein, Peter Battaglia, Alysa Obertas, Philip J Armitage, et al. 2020. “Predicting the Long-Term Stability of Compact Multiplanet Systems.” Proceedings of the National Academy of Sciences 117 (31): 18194–18205.