Study Analysis Groups (SAGs) and Science Interest Groups (SIGs). Much of the ExoPAG's work is conducted by Study Analysis Groups (SAGs) and Science Interest Groups (SIGs), which focus on specific exoplanet topics or goals. The exoplanet community is encouraged to participate in the ExoPAG and in SAGs or SIGs of interest.
*SAGs 3, 4, 6, and 7 were initiated but later deactivated; often their topic areas were assigned to other SAGs.
Topic: Debris Disks & Exozodiacal Dust
Leads: Aki Roberge
Status: Report completed; paper published in PASP, 2012, 124, 799-808
Products: "The Exozodiacal Dust Problem for Direct Observations of Exo-Earths"
Overview: Debris disks arise from collisions between and evaporation of extrasolar asteroids and comets, the same processes that produce the zodiacal dust in the Solar System.
Many nearby stars harbor an outer debris disk that is orders of magnitude brighter than that of the Solar System. At the moment, we know little about debris dust in the habitable zones of nearby stars (aka. exozodiacal dust). In direct images and spectra of terrestrial planets, zodiacal and exozodiacal background emission will likely dominate the planet signals. Teams designing future space telescopes aimed at direct observations of habitable planets have studied the impact of these backgrounds on mission performance. Within the range of current uncertainty, the impact could be severe. Apparently, the distribution of exozodi brightness levels around nearby stars is as crucial to such a mission’s success as is ??. Several vital questions remain unanswered:
- How well must we know exozodi brightness levels to determine the performance of various types of direct imaging/spectroscopy exoplanet missions?
- How is the problem complicated by possible asymmetries and other complex morphologies in exozodiacal dust disks?
- What are the exozodiacal dust levels around nearby stars?
These are all large questions requiring much additional work to fully answer. However, we believe an ExoPAG study group can contribute in three important ways:
- Collect existing information on the impact of exozodiacal background on various exoplanet mission concepts. Describe what additional analyses need to be done, and work with members of past or existing mission concept teams to see that they are performed in a uniform fashion.
- Collect reliable information on the expected sensitivity of all upcoming facilities to debris dust at different distances from various types of nearby stars.
- Determine how many stars we must observe with what exozodi sensitivity to confidently predict the number of feasible targets for direct exoplanet imaging/spectroscopy. Start by organizing a theoretical study that will produce a plausible expected distribution of exozodi brightness levels.
From this foundation, the community may be able to plan the required observations of debris disks, and narrow the range of possible scenarios for teams studying future mission performance.
Topic: Potential for Exoplanet Science Measurements from Solar System Probes
Leads: Dave Bennett, Dan Coulter
Status: Completed. Topic explored in detail at Kavli Institute workshop, Santa Barbara CA, May 2010
Products: "KISS/KITP Workshop: Exoplanet Science Measurements from Solar System Probes"
Overview: This SAG (Science Analysis Group) would bring together scientists, engineers, theorists, and NASA leadership from the exoplanetary and planetary/solar system communities to investigate the scientific potential and practicality of doing exoplanet science using spacecraft that primarily study our own Solar System.
This could involve observations during cruise phase or an extended mission and possibly the addition of low-impact instruments to missions in development. Past examples of successful collaborations include the measurements of Earth made from afar by the recent EPOXI mission and by the TES instrument aboard Mars Global Surveyor in 2004. Future examples of such exoplanet science could include zodiacal light studies, microlensing observations, transits, etc. The product of this SAG would be a report on potential exoplanet science that could be performed from the platform of a solar system mission including: top level science goals, top-level instrument concepts, potential solar system missions which could accommodate exoplanet science instruments, and preliminary assessment of impacts of exoplanet science instruments on solar system missions. In connection with this SAG, the ExoPAG plans to hold a session on exoplanet collaboration at the DPS (Division of Planetary Sciences) meeting next October and/or request a talk at DPS to discuss this topic with the planetary science community.
Topic: Planetary Measurements Needed for Exoplanet Characterization
Leads: Lisa Kaltenegger
Overview: Understanding the physical and chemical states of exoplanets requires knowledge of a number of parameters that may be gleaned from observations with current and future remote sensing observatories.
The difficulty of making these observations spans a large range, and the finite set of current and near-future practical techniques will yield different precisions on different parameters. This SAG is convened to study what parameters must be known to characterize large and small planets, what precisions are required to understand different aspects of planetary science, what measurements are needed, and what ground- and space-based techniques are most promising for obtaining them. Specific questions to be addressed by this study group are:
- What planetary and host star parameters should we know, in order to characterize the physical and chemical states of exoplanet interiors, surfaces, and atmospheres? What observations can distinguish whether a planet or moon is habitable or currently sustaining life as we know it? What will this information tell us about a planet's formation, past, and likely future? What precision in these parameters would enable us to resolve each question or ambiguity about a planet's state? These parameters are expected to include, but not be limited to, planetary size, mass, rotation, surface heterogeneity, atmospheric composition, surface features, cloud properties, surface and atmosphere temperature profiles.
- What measurements can provide this information, and what sensitivity would be needed? How are the measurements combined with each other and with models to provide the needed information? On what time scales are they likely to become available? We anticipate that these measurements will include, but not be limited to, planetary and stellar motions, photometry, and atmospheric spectra. Specific measurements and their ranges required to understand different planetary properties should be explained.
- What information is required for understanding giant planets, and what information is required for understanding terrestrial planets and moons? The differences should be clearly delineated for all aspects of this SAG's investigations.
Topic: Exoplanet Flagship Requirements and Characteristics
Leads: Charley Noecker, Tom Greene
Status: Report completed; posted on ArXiv:1303.6707, March 26, 2013
Products: "Flagship Exoplanet Imaging Mission Science Goals and Requirements Report"
Overview: This group will work toward a unified view of the science requirements for a combined flagship exoplanet imaging and general astrophysics mission for the 2020s.
We will discuss and evaluate possible features of a decision process for NASA HQ to select either an internal coronagraph or an external occulter (starshade) as the favored mission architecture for exoplanet direct detection. NASA's selected architecture will be presented to the Decadal Survey Implementation Advisory Committee (DSIAC) in 2015, will guide technology development in the years 2015-2020, and eventually will be presented to the Astrophysics Decadal Survey in 2020. We will assemble science requirements from the Exoplanet Exploration Program and Cosmic Origins Program, forming a unified set of requirements for a combined mission; and we will add a list of technical and programmatic criteria which should be considered in the decision. Our report will contain the elements which are deemed relevant to a robust decision between the two architecture families in the future.
Topic: Requirements and Limits of Future Precision Radial Velocity Measurements
Leads: Dave Latham, Peter Plavchan
Status: Report completed; posted on ArXiv:1503.01770, March 5, 2015
Products: "Radial Velocity Prospects Current and Future"
Overview: Precision radial velocity measurement has been the workhorse technique in the exoplanet field, for both detection and characterization of exoplanets.
This group will evaluate the future role of RV measurements in the exoplanet field, both scientific and programmatic, and will attempt to assess the resources required fulfill this goal. Key science questions include:
- What are the near-term, medium-term, and long-term needs for Doppler measurements to support NASA science objectives, i.e., how many measurements, of what precision and what time baseline, and for how many stars of what magnitudes and spectral types?
- What are the astrophysical limitations on radial velocity precision for measurements of nearby stars?
- How does this precision vary as a function of stellar type and wavelength?
- What are the implications of these limitations for new ensemble survey science goals and for finding the nearest low-mass exoplanets for future characterization?
Instrument/technical questions include:
- What approaches can improve radial-velocity instrumental precision to the astrophysical limits?
- What can be done to increase the efficiency and sensitivity of radial-velocity facilities?
- What potential exists for red/near-infrared radial velocity precision?
Programmatic questions to be considered include:
- What are the benefits or disadvantages of increased investment in telescope time (and for which class of telescope)?
- How should we prioritize increased investment in existing telescope resources versus investment in new, dedicated facilities and/or technology development for precision calibration/stabilization?
- What competitive opportunities exist in the short and long term in the context of existing and planned US and global facilities?
Topic: Exoplanet Probe to Medium Scale Direct-Imaging Mission Requirements and Characteristics
Leads: Rémi Soummer
Status: Report completed. Posted on http://exep.jpl.nasa.gov/exopag/ on 8/18/2015
Products: "Exoplanet Probe to Medium Scale Direct Imaging Mission Requirements and Characteristics - Final report" https://exep.jpl.nasa.gov/files/exep/ExoPAG-SAG9-Final.pdf
Overview: The ExoPAG Study Analysis Group 9 (SAG-9) will define metrics by which the science yield of various exoplanet probe-scale to medium-scale direct-imaging mission designs can be compared and evaluated in order to facilitate a well-informed decision process by NASA.
SAG-9 will focus on mission sizes that can be considered on shorter timescales than a flagship, with a particular emphasis on missions with probe-scale costs (under $1B). The work will build on the methodology developed by SAG-5 (Exoplanet Flagship Requirements and Characteristics), defining science goals, objectives and requirements, further detailed into "Musts" and "Discriminators". SAG-9 will establish the minimum science thresholds ("Musts") for such missions, and develop quantitative metrics to evaluate the marginal performance increase beyond the threshold science using "Discriminators". Key questions to be studied by this group include:
- What is the minimum threshold science to justify an exoplanet probe-scale direct imaging mission?
- What are the additional science goals that can be used as "discriminators" to evaluate science performance beyond the minimum thresholds?
- What are the possible achievements from the ground by plausible launch date, and overlapping the expected mission lifetime?
- What quantitative metrics for these "discriminators" can we provide to help define the weighting process to be used in the comparison of mission concepts?
Topic: Characterizing the Amtospheres of Transiting Planets with JWST and Beyond
Leads: Nick Cowan
Status: Report completed; posted on JSTOR: 10.1086/680855, March 3, 2015
Products: "Characterizing Transiting Planet Atmospheres through 2025"
Overview: The past decade has seen rapid progress in the study of exoplanet atmospheres. It has become common --if not routine-- to measure albedo, atmospheric composition, as well as the vertical and horizontal temperature structure on these distant worlds; in a few cases we can even observe a planet's seasonal response to varying insolation.
Such large-scale averages of atmospheric temperature and composition are probes of exoplanet climate. In short, the study of planetary climate is no longer limited to the Solar System, but also to planets orbiting nearby stars. Much of this explosion in scope is due to the unexpected existence of short-period planets, which can be studied without spatially resolving them from their host star. In the near-term, the most readily detected and characterized temperate terrestrial planets will likely transit nearby M-Dwarfs. NASA's James Webb Space Telescope (JWST) promises to be a powerful tool for characterizing the climate of transiting planets. This infrared space observatory will be capable of measuring the transits, eclipses and phase variations of short-period planets. Nevertheless, many strategic questions remain. In concert with the relevant instrument teams, our group will attempt to answer these questions, including:
- What is the full diversity of planet properties (mass, atmosphere, insolation, orbits, etc.) needed to characterize and understand the climate of short-period exoplanets?
- Which measurement suites and how much observing time are needed to characterize the climate of transiting planets?
- Will JWST be able to characterize the atmospheres of transiting terrestrial planets?
- Which critical measurements will be too expensive or inaccessible to JWST, and can these be obtained with planned observatories (eg: dedicated ground-based spectrographs, balloon-borne experiments, SOFIA, exoplanet-specific Explorer missions)?
Topic: Preparing for the WFIRST Microlensing Survey
Leads: Jennifer Yee
Status: Report Completed
Products: "NASA ExoPAG Study Analysis Group 11: Preparing for the WFIRST Microlensing Survey" arxiv.org/abs/1409.2759
Overview: Although the launch of the WFIRST mission is still many years off, it is nevertheless vitally important to consider what activities must be carried out in the near future in order to retire any scientific risks associated with, and maximize the returns from, the WFIRST microlensing survey.
In particular, there may be projects that require a long time baseline and/or might affect the final mission design, and thus must be undertaken soon. This SAG will bring together members of the microlensing community to identify scientific programs that will benefit the WFIRST microlensing mission. Of particular interest are mission-critical observational programs that must be completed before the launch of WFIRST. Specifically, the major question this SAG will address is: "What scientific programs can be undertaken now to ensure the success of the WFIRST mission and maximize its scientific return?" In the process of answering this question, the SAG will:
- Identify both mission critical and mission enhancing programs,
- Identify immediate science to come out of each program, as well as the program's direct impact on the WFIRST mission,
- For each proposed program, quantify the improved scientific return for the WFIRST mission,
- Emphasize programs that can be executed using existing (NASA) resources.
Topic: Scientific potential and feasibility of high-precision astrometry for exoplanet detection and characterization.
Leads: Eduardo Bendek
Status: Report Completed
Overview: High-precision astrometry has the potential to play an important role in the detection and characterization of exoplanets. High-precision astrometry can complement high-contrast direct imaging surveys by allowing for improved yields, as well as measurement of planet masses.
Sub-microarcsecond astrometry would allow the measurement of the mass and orbital parameters of Earth-mass planets orbiting stars within 10pc. In particular, such astrometry may ultimately be needed to measure the masses of planets of potentially habitable planets orbiting nearby solar type stars. Finally, astrometric surveys provide an important complementary tool for characterizing the demographics of nearby planetary systems, as it is sensitive to planets with arbitrary inclination, and its sensitivity increases with semi-major axis, in contrast to radial velocity and transit surveys. Key questions and goals that this group will address are:
- What is the scientific potential of astrometry for different precision levels? What types of planets can be studied with astrometry? How effective is astrometry to confirm planet candidates?
- What are the technical limitations to achieving astrometry of a given precision? Can we implement observational strategies or post processing to improve the astrometry? What are the hardware changes that would enable high-precision astrometry on planned missions?
- Identify mission concepts that are well suited for astrometry and study potential collaboration with current and future European astrometry missions.
Topic: Exoplanet Occurrence Rates and Distributions
Leads: Rus Belikov
Status: Report completed
Topic: Characterization of Stars Targeted for NASA Exoplanet Missions
Leads: Keivan Stassun
Overview: Fabienne Bastien, Alan Boss, Tabetha Boyajian, Lars Buchhave, David Ciardi, Courtney Dressing, Shawn Goldman, Natalie Hinkel, Stephen Kane, Andrew Mann, Peter Plavchan, Maggie Turnbull, Kaspar von Braun, Lucianne Walkowicz, Russel White The accuracy with which we can determine the basic properties of any planet from any exoplanet mission depends fundamentally on our knowledge of the host stars.
In particular, a planet’s mass, size, density, composition, insolation, age, and past and future evolutionary history can only be determined via the known properties of the host star and their respective uncertainties. Thus, the star properties have a direct influence on our knowledge of both exoplanet physics (parameters such as mass and radius) and exoplanet astrophysics (history of formation, evolution, migration, etc.). Yet, very often, the exoplanet host stars turn out to be inadequately characterized, which has become a bottleneck for the field despite the trove of planet discoveries delivered by NASA missions such as Kepler and K2. “Know the star, know the planet” has become a cri de coeur for the exoplanet science community. Recently, the community has turned the discussion towards the large gap between the rapid pace of exoplanet discovery and the slow pace of follow-up stellar characterization. For example, the majority of Kepler Objects of Interest (KOIs) still remain poorly characterized because the host stars remain poorly characterized. Indeed, stellar characterization efforts in the era of Kepler have in general been
- in a constant state of backlog,
- largely uncoordinated in terms of target prioritization, data quality, and analysis methods, and
- usually not independent of the planet discovery teams.
The TESS mission is now set to imminently provide an additional large number of small planets that will need to be accurately characterized for the primary mission deliverables. With additional exoplanet missions planned for the near future, it is vitally important to determine what activities will reduce scientific risk and maximize return from TESS and from other near-term exoplanet missions. In particular, there are projects that can be undertaken soon to capitalize on extant observing resources, to quantify stellar characteristics en masse and rapidly fulfill core science from upcoming exoplanet missions. Importantly, we recognize that the results from the Gaia mission promise to address many stellar characterization issues needed on the TESS timescale. Even so, there may still be gaps in the additional data products needed to characterize the host stars, such that the community will not be ready to the extent required for ultimate mission success and maximum output for the scientific community and for the public in general. This SAG will bring together experts in the field, whose specialties include a number of multidimensional approaches to stellar characterization, in order to identify scientific programs that will benefit TESS and follow-on exoplanet missions. The SAG will also build on the previous (Jan 2015) ExoPAG minisymposium on stellar characterization. Of particular interest are mission-critical observational programs that must be completed before the launch of TESS or before the end of its primary data collection mission. Specifically, the major question this SAG will address is: "What exoplanet host-star characterization programs can be undertaken now to ensure the success of the TESS mission and its follow-on missions, and maximize their scientific return?" In the process of answering this question, the SAG will:
- Identify both mission critical and mission enhancing programs -- including observational programs and analysis methodologies -- especially in light of known gaps from previous and current missions, including in particular gaps that will remain after the expected deliverables from Gaia,
- Identify immediate science to come out of each program, as well as the program's direct impact on the TESS mission and on TESS follow-on missions,
- For each proposed program, quantify the improved scientific return for the TESS mission and for planned follow-on exoplanet missions,
- Emphasize programs that can be executed using existing (NASA) resources, and that represent an opportunity for scientific independence from the mission and/or exoplanet discovery teams.
Topic: Exploring Other Worlds: Observational Constraints and Science Questions for Direct Imaging Exoplanet Missions
Leads: Daniel Apai
Status: Report Completed
Overview: Future direct imaging missions may allow observations of flux density as a function of wavelength, polarization, time (orbital and rotational phases) for a broad variety of exoplanets ranging from rocky sub-earths through super-earths and neptunes to giant planets.
With the daunting challenges to directly imaging exoplanets, most of the community’s attention is currently focused on how to reach the goal of exploring habitable planets or, more specifically, how to search for biosignatures. Arguably, however, most of the exoplanet science from direct imaging missions will not come from biosignature searches in habitable earth-like planets, but from the studies of a much larger number of planets outside the habitable zone or from planets within the habitable zone that do not display biosignatures. These two groups of planets will provide an essential context for interpreting detections of possible biosignatures in habitable zone earth-sized planets. However, while many of the broader science goals of exoplanet characterization are recognized, there has been no systematic assessment of the following two questions:
- What are the most important science questions in exoplanet characterization apart from biosignature searches?
- What type of data (spectra, polarization, photometry) with what quality (resolution, signal-tonoise, cadence) is required to answer these science questions?
We propose to form SAG15 to identify the key questions in exoplanet characterization and determine what observational data obtainable from direct imaging missions is necessary and sufficient to answer these. The report developed by this SAG will explore high-level science questions on exoplanets ranging from gas giant planets through ice giants to rocky and sub-earth planets, and — in temperatures — from cold (~200 K) to hot (~2,000 K). For each question we will study and describe the type and quality of the data required to answer it. For example, the SAG15 could evaluate what observational data (minimum sample size, spectral resolution, wavelength coverage, and signal-to-noise) is required to test that different formation pathways in giant planets lead to different abundances (e.g. C/O ratios). Or the SAG15 could evaluate what photometric accuracy, bands, and cadence is required to identify continents and oceans in a habitable zone Earth-sized or a super-earths planet. As another example, the SAG15 could evaluate what reflected light data is required to constrain the fundamental parameters of planets, e.g. size (distinguishing earth-sized planets from superearths), temperature (cold/warm/hot), composition (rocky, icy, gaseous), etc. SAG15 will not attempt to evaluate exoplanet detectability or specific instrument or mission capabilities; instead, it will focus on evaluating the diagnostic power of different measurements on key exoplanet science questions, simply adopting resolution, signal-to-noise, cadence, wavelength coverage as parameters along which the diagnostic power of the data will be studied. Decoupling instrumental capabilities from science goals allows this community-based effort to explore the science goals for exoplanet characterization in an unbiased manner and in a depth beyond what is possible in a typical STDT. We envision the SAG report to be important for multiple exoplanet sub-communities and specifically foresee the following uses:
- Future STD teams will be able to easily connect observational requirements to missions to fundamental science goals;
- By providing an overview of the key science questions on exoplanets and how they could be answered, it may motivate new, dedicated mission proposals;
- By providing a single, unified source of requirements on exoplanet data in advance of the Decadal Survey, the science yield of various missions designs can be evaluated realistically, with the same set of assumptions.
Our goal is to carry out this SAG study by building on both the EXOPAG and NExSS communities. We aim to complete a report by Spring 2017 and submit it to a refereed journal, although this timeline can be adjusted to maximize the impact of the SAG15 study for the ongoing and nearfuture STDTs and other mission planning processes. Synergy with a potential future SAG proposed by Shawn Domagal-Goldman: While the SAG proposed here will include studies of habitable zone rocky planets, it will focus on planets without significant biological processes. A future SAG may be proposed by Shawn DomagalGoldman to explore biosignatures; if such a SAG is proposed, we envision a close collaboration on these complementary, but distinct problems.
Topic: Exoplanet Biosignatures
Leads: Shawn Domagal-Goldman, Nancy Kiang, Niki Parenteau
Products: "SAG16 Proposal"
Overview: The future of exoplanet observations will begin a shift from the physical and astronomical characterization of planet size and orbital properties towards the characterization of planet chemical composition, habitability, and inhabitance. We are proposing a SAG to explore the last of these issues, focused on biosignatures.
The future of exoplanet observations will begin a shift from the physical and astronomical characterization of planet size and orbital properties towards the characterization of planet chemical composition, habitability, and inhabitance. We are proposing a SAG to explore the last of these issues, focused on biosignatures. Due to the interdisciplinary nature of biosignature research, it is paramount that the astrobiology and exoplanet communities come together for this effort. This SAG will bring these groups together, in the pursuit of three goals: 1) review the existing state of biosignature science, 2) develop a plan for uncovering novel biosignatures, and 3) list the features of existing biosignatures as an input to mission development and planning activities. In order to most comprehensively achieve these goals, we will organize a workshop to bring together scientists across disciplines, collect notes from that activity to draft a report, and then circulate that report to the community to obtain feedback for a final report. This process will engage a broad range of experts from the NASA Astrobiology Institute (NAI), the Nexus for Exoplanet System Science (NExSS), NASA’s Exoplanet Exploration Program (ExEP), and the community served by the ExoPlanet Assessment Group (ExoPAG) and planetary AGs.
Topic: Community Resources Needed for K2 and TESS Planetary Candidate Confirmation
Leads: David Ciardi
Products: SAG 17 Charter
Overview: K2, operating since 2013 and expected to continue operations through 2017, is producing hundreds of candidate planets (approximately 50 – 100 per field).
Additionally, TESS, when launched in 2017, will produce thousands of candidates from the selected TESS targets, and potentially hundreds of thousands of candidates from the full‐frame images. In order to confirm these candidates, follow‐up observations, from either the ground or space, are required. Spectroscopy is needed for stellar characterization; radial velocity observations are needed to determine companion masses, and imaging (both seeing‐limited and high‐resolution) is needed to ascertain the target blending and hence determine accurate planetary radii and possible false positives. Some amount of triage work can also be done by time‐series photometric follow‐up with higher angular resolution. SAG 17 will study and enumerate the resources needed by the community to effectively and efficiently validate as many K2 and TESS candidates as possible, and propose methods to allow the community to coordinate and self‐organize the process. This SAG is geared more towards the validation efforts needed rather than the characterization of the systems, but the two efforts are related, and as such, This SAG is complementary to previous and ongoing SAGs (8: RV; 10: Atmospheres; 12: Astrometry; 14: TESS Stars; 15: Directing Imaging Science; 16: Biosignatures). Finally, the purpose of this SAG is not to define what is needed by the TESS project to satisfy their level 1 science requirements, but rather what is needed by the community to validate and study the bounty of the full range of planetary candidates being discovered by K2 and will be discovered by TESS. The following are specific goals of SAG 17: Identify needed follow‐up observations for K2 and TESS including but not limited to imaging, spectroscopy, and time‐series follow‐up Identify telescopes, instrument, and financial resources available to the US community Identify how archival resources can be utilized (e.g., Gaia) Identify how the community can be organized and communication facilitated particularly with regards TESS full frame images, candidate identification, single transiting events, and candidate prioritization. Identify needs to ensure efficient and effective characterization with JWST (and WFIRST) Identify connections to other SAG efforts (e.g., SAGs 15 and 16) Identify synergies of resources with non‐exoplanet science.
Topic: Metrics for Direct-Imaging with Starshades
Leads: Tiffany Glassman, Margaret Turnbull
Status: Report Completed
Products: Metrics for Direct-Imaging with Starshades - Closeout
Overview: The use of starshades for future direct-imaging missions is being studied and developed by various groups. Extensive testing of starshade performance has been conducted over a wide range of scales and test conditions.
One missing piece has been a clear set of metrics to standardize the starlight-suppression performance of a test and requirements of a flight mission. There have been informal definitions of contrast as the amount of residual starlight at the location of an exoplanet of interest and of suppression as the total amount of residual starlight entering the telescope. But more precise definitions are needed to compare test results across groups and to then define flight requirements using these same definitions. An agreed-upon set of metrics would allow unbiased comparisons between separate tests and between tests and flight requirements. A second SAG (SAG19, chaired by Dimitri Mawet) will be starting at the same time and working in parallel to answer similar questions but focused more on signal detection theory. We will coordinate closely with this SAG and develop compatible standards as much as possible. We propose to form SAG18 to identify the areas of starshade performance where standardized metrics would be beneficial and to create rigorous definitions of key terms. Some questions that may be answered by this SAG are list below. Refining these questions and goals will be the first task for the SAG group. We expect that the outputs from this SAG will be published so that the results can be used in the continuing starshade development work. The minimum outputs will be a glossary of terms as they are used in the community. 1. How can contrast or suppression be used as metrics of starshade performance (pros and cons)? Contrast is directly linked to planet detectability, but is not flight-like if image is over-resolved. Suppression is independent of telescope/ resolution, but it doesn’t take into account the spatial distribution of light, so it’s a very worst-case assessment. 2. How should contrast be defined? The contrast should be calculated using a standard pixel location in the image, such as an annulus near the petal tips. For test where there is no off-axis object in the field, a contrast limit must be calculated. This could be the average light in the chosen pixels, a statistical measure of the noise in the pixels (e.g. StDev), or a simulation of a point source detectability limit in the image. If the test image is over-resolved compared to a flight-like configuration, the image could be post-processed to compensate. A method of compensation should be determined. New starshade metrics should be based off of existing metrics where possible. Coronagraph groups have developed methods of defining contrast achieved in their testbeds and contrast required for planet detection. In additional, standard astronomical techniques for detecting faint sources will be referenced. The starshade metrics will be unique only where some aspect of the residual stray light from starshades requires a new approach. 3. How should suppression be defined? If a test only measures the focal plane, then the suppression must be calculated by summing over an area in that image. Create a standard for what area of the image is included, including what features can be masked off and what radius the area should extend to. If there is a smooth background present in the image, this could be subtracted either as a constant level or a smooth distribution.
Topic: SAG 19: Exoplanet Imaging Signal Detection
Theory and Rigorous Contrast Metrics
Leads: Dimitri Mawet, Rebecca Jensen-Clem
Products: SAG 19 Charter
Overview: As planning for the next generation of high contrast imaging instruments (e.g. WFIRST, HabEx, and LUVOIR, TMT-PFI, EELT-EPICS) matures, and second-generation ground-based extreme adaptive optics facilities (e.g. VLT-SPHERE, Gemini-GPI) are halfway through their large main surveys...
It is imperative that the performance of different designs, post-processing routines,
observing strategies, and survey results be compared in a consistent, statistically robust
framework. SAG19, exoplanet imaging signal detection theory and rigorous contrast metrics, is
overarching to all direct imaging instrument, strategies, and methods. The scope of SAG19 is: 1- To go back to the basics of Bayesian Signal Detection Theory (SDT). Bayesian SDT implies H0:signal absent / H1:signal present hypothesis testing, and invokes well-known concepts such as: the confusion/contingency matrix, false positive (type I error), false negative (type II error), true positive, and true negative fractions, and useful combinations of these quantities such as sensitivity (or completeness) and specificity. 2- To rebuild a solid set of usual definitions used for or in lieu of “contrast” in different contexts, such as astrophysical contrast or ground truth, instrumental contrast used for coronagraph/instrument designs, and the measured on-sky data-driven contrast. Bayesian, hypothesis testing SDT will automatically force our community to be inclusive of all possible aspects of exoplanet detection, and signal-to-noise ratio (SNR) metrics, including signal-related parameters: planet-star contrast, SED, polarization, variability; instrument parameters: throughput, bandwidth, Strehl ratio/encircled energy, background (sky/thermal, or astrophysical), detector characteristics; noise characteristics as affected by the starlight suppression technique (in a very broad sense): mean intensity, RMS pixel intensities, RMS resolution element (resel, of characteristic size wavelength/telescope diameter) intensities, the probability density function (PDF) computed over pixels, the same PDF computed over resels, their nature and higher order moments, the sample zone and size, outlier management, etc. 3- To identify what we can learn and apply from communities outside our field (e.g. medical imaging). A good example is the widespread use of receiver operating characteristic curve (ROC) and area under the curve (AUC). ROC plots the true positive fraction against the false positive fraction, and is useful to capture the true performance of a given high contrast imaging instrument, or post-processing/detection algorithm. Other formalisms, alternative to the ROC curve, such as the precision-recall curve will also be considered. 4- To define precise contrast computation and ROC curve computation recipes, a new “industry standard”. The goal is to be able to compare results from surveys, instrument and/or telescope designs on a level-playing field. 5- To identify how the new metrics and recipes can be used to define confidence levels for detection (H1) and subsequently error bars for photometric, spectroscopic, astrometric characterization. Ancillary goals: better understanding what limits exoplanet characterization, not just detection. For instance, understanding the limiting precision of extracted planet spectra from algorithms that anneal the planet signal and gaining proper error assessments from spectral extraction. This is particularly important in cases where the prior on the signal/wavelength to be detected is unknown and iterative forward-modeling must be applied. 6- To perform a community data challenge before and after applying our proposed set of standardized SDT rules and recipes, and apply lessons learned.