Data Processing & Analysis Consortium
The Data Processing & Analysis Consortium ( DPAC ) is a team of around 450 scientists and software developers of various nationalities who have been preparing and evaluating the scientific data of the Gaia mission since 2006 .
In contrast to other missions, the raw data from Gaia cannot be used without further treatment. The ESA had to develop together with DPAC new software so that the data obtained efficiently processed, archived and can be recycled for use on the ground. In addition, the reception capacities and internet connections of the three 35-meter antennas of the ESTRACK network in New Norcia , Cebreros and Malargüe have been expanded so that they can effectively process and forward the large amounts of data from the Gaia mission.
The data from the antenna network is bundled by the European Space Flight Control Center ESOC in Darmstadt, forwarded to the Science Operations Center SOC of ESAC and stored there. The center of DPAC with the central pooling of all data is located in Villafranca del Castillo near Madrid in Spain and is provided and maintained by ESA. During the five years of the nominal mission duration, the probe is expected to produce a total of over a petabyte of data, which corresponds to the data capacity of 1.5 million CD-ROMs or 200,000 DVDs.
The subsequent scientific preparation is done by the "Data Processing & Analysis Consortium" (DPAC). DPAC is organized in nine Coordination Units (CUs). The costs for DPAC for the further processing of the data are borne by the national budgets or the budgets of the participating institutes, not by the ESA.
Participating countries and institutions
DPAC is a mainly, but not exclusively, European project supported by 21 nations and two international institutions. The participating countries and institutions are:
Locations
The data is processed at six locations with their own data centers in Villafranca, Barcelona, Cambridge, Geneva, Turin and CNES in Toulouse. The individual locations have their own financing and each develop their own way of fulfilling their tasks and decide for themselves which means they will use. Each work group (CU) is assigned to one or more locations. The locations are networked with 1 Gbit data lines, the capacity of which can be expanded if more bandwidth is required in the future. All data, intermediate results and final results of the respective processes are stored in the DPCE in Villafranca and can be called up from there by the working groups for the next steps.
The data processing centers (DPC) are located under the umbrella of existing institutions and have their own names. Each of these data centers operates its own data processing systems, uses the systems of the participating institutes or assigns the tasks to corresponding supercomputers.
DPCE
DPCE is part of the Scientific Mission Center (SOC) in ESAC Villafranca. The data center operates the software of CU1 with the tasks Main Data Base (main database, MDB), Mission Operations Center Interface Task (MIT), Decompression and Calibration Service (DCS), Payload Operations System (POS), Gaia Transfer System (GTS). For CU3, DPCE operates the modules Initial Data Treatment (IDT), First Look (FL), Astrometric Global Iterative Solution (AGIS). My own tasks include IDT / FL Database (IDTFLDB), Daily PipeLine (DPL), Gaia Observing Schedule Tool (GOST). DPCE also serves as a link between the MOC and the other data centers. The main database comprised almost 2.5 billion objects with the status of Gaia DR2, of which data on 1.7 billion objects were published with Gaia DR2.
DPCB
The Data Processing Center of Barcelona DPCB is part of the DPAC group of the University of Barcelona together with the Institute for Space Studies of Catalonia (IEEC), the Barcelona Supercomputing Center (BSC) and the Consorci de Serveis Universitaris de Catalunya (CSUC) in Barcelona. The hardware is provided by BSC, while the team from UB and IEEC is responsible for management, operation, development and software testing. For CU1, DPCB takes over the Gaia Transfer System (GTS) module for exchange with DPCE, for CU3 Intermediate Data Updating IDU and for CU2 data simulations for probe telemetry and for the Gaia catalogs. For these tasks, BSC intensively uses the MareNostrum supercomputer, which in 2017, equipped with 3000 computing nodes and 100.8 TB main memory, achieved a computing power of 1.1 petaflops. 389,245 processor hours were required for Gaia DR2, this number only counts the successful runs.
DPCC
DPCC is located at the CNES in Toulouse. CNES data center in Toulouse stores a complete record of all Gaia data as a backup at another location while the main database in Villafranca is held. DPCC processes the data from CU4 with the solar system objects and multiple star systems, from CU6 with the processing of the spectroscopic data and CU8 with the determination of the spectral classes of the objects. The data processing worked for Gaia DR2 with 172 computing nodes with 3500 processor cores and 19 TB main memory as well as 2.6 PB data memory. The computing capacity must be expanded along with the tasks and data volumes. In the end, the cluster will require around 5800 processor cores and 4.5 PB of data storage. The backup solution for the main database is based on temporary disk storage with 64 TB and a robot for tape storage.
DPCI
The DPCI is located at the Institute of Astronomy (IoA) at the University of Cambridge. DPCI operates the data processing for CU5 with PhotPipe. PhotPipe can start work when all data for a data segment has been collected and the results from IDU and AGIS are available. For Gaia DR2, 51.7 billion transits were processed in PhotPipe, with 40.9 billion remaining after filtering out 10.7 billion surplus objects for which no objects could be assigned from the database. The majority of these are apparent objects caused by overexposure to bright objects or effects of space radiation.
DPCG
The Data Processing Center in Geneva (DPCG) is located in the Faculty of Astronomy at the University of Geneva. The IntegralScience Data Center (ISDC) in Versoix near Geneva is used for data processing. DPCG conducts the investigation of the variability with the IntegratedVariability Pipeline (IVP) for CU7, thereby also statistical values are determined. The processing is carried out cyclically, data and results from CU3, CU4, CU5, CU6 and CU8 are used. Variable objects are recognized, periods are determined and the type of variable object is classified. The results are processed further by CU4, CU6, and CU8. A cluster with 850 processor cores is used for data processing. All 1.7 billion objects from Gaia DR2 were examined for photometric variability for the corresponding observation time.
DPCT
The data processing center in Turin (DPCT) operates the data processing for the Astrometric Verification Unit (AVU) of the CU3. The tasks include monitoring and analyzing the outputs of the astronometric instruments, monitoring the Basic Angle Monitoring System, which monitors the minimal changes in the base angle between the two telescopes. The group operates the Global Sphere Reconstruction (GSR) unit, and the AGIS results are verified. The group provides the calibration data for the daily processes from the analyzes of the instruments. One of the processes is on the Fermi supercomputer of CINECA executed. One of the modules removes the rotating movement of the telescopes and links the data with the Gaia Celestial Reference Frame .
Working groups
CU1 system / IT architecture
CU1 was responsible for defining the system architecture used by DPAC, for the hardware, software and technology required for the main database. She was also responsible for defining and monitoring system tests. The tasks included the adaptation to the ECSS standards and the MOC interface as well as the definition for quality assurance. CU1 determined which of the locations took over which function, how the data flow should proceed and how the results obtained would be collected and saved at the end.
CU2 simulations
CU2 was responsible for the simulations for testing hardware and software before use. The software was tested for reliability with extensive simulated data before it could be entrusted with processing the Gaia mission data. CU1 and CU2 were the first active working groups and began their work a long time before the start phase.
After successful development and start of the mission, the task is limited to supporting the software and issuing updates. The data simulator will be available for the entire duration of the mission and will continue to be maintained.
CU3 Astrometric core processing
CU3 does most of the arithmetic for the astrometric data and provides the position and direction of movement of objects in the sky. Daily tasks include decoding, unpacking, and processing the science data supplied by the Mission Center (MOC). The working group monitors the correct functioning of the probe and provides ultra-precise information on the work of the instruments to ESOC for monitoring the probe. CU3 assumes a first examination of the material (First Look), and returns the science alerts from when sudden events are detected.
CU3 works with AGIS (astrometric global iterative solution) and takes over most of the mathematical data reduction. In addition to the daily work, it takes over the periodic iterative recalculation of the telemetry data by recalculating the data with the help of improved calibration data. CU3 takes into account the calibration data of the measuring instruments, e.g. B. the tiny changes in the basic angle, as measured by the sensors of the Basic Angle Monitor, which are not yet taken into account in the daily data processes.
CU4 complex object processing
CU4 takes over the object calculations for more complex objects that cannot be dealt with in detail by CU3. CU4 deals with double and multiple star systems, the objects of the solar system and with exoplanets, as well as with extended objects that cannot be processed as points, e.g. B. galaxies and star clusters and with other extragalactic objects. If the object is a star, the object is measured several times at the same point; otherwise it is likely an asteroid or comet that has changed its position from the previous observations. All objects discovered in this way are compared with the orbits of known asteroids. If there is no match, it is potentially a newly discovered object. Movements of more than ~ 1 as / s can already be recognized by comparing the different results from the field of the astrometry sensors during an acquisition. The results are reported to the Gaia FUN SSO network, which tries to track the objects over a longer period of time in order to gain more orbit data and to prevent objects that have already been recorded from being lost again.
CU5 Photometric processing
CU5 concentrates on the photometric data and carries out the general calibration for the brightness, collects the information about the brightness in relation to the epoch. CU5 issues science alerts when objects show unusual fluctuations in brightness, e.g. B. when a supernova is discovered.
CU6 Spectroscopic processing
CU6 processes the spectroscopic data of individual stars. The spectroscopy enables the calculation of the radial speed and the rotational speed based on the Doppler shift. It determines the composition of individual stars. The tasks include the assignment of the spectra to the individual objects and the correction of the spectra against the radiation of the background. Another task is the validation of the data. The origins of the CU6 began in the planning phase towards the end of the 1990s. A small group of specialists worked on the concept of the Radial Velocity Spectrometer (RVS) of the Gaia mission and carried out various simulations. In 2001 the RVS working group was founded, which defined the scientific goals and supported the industry with the implementation in technical specifications. The work also included testing the RVS prototypes and simulating and evaluating the data in order to test the capabilities of the instrument. In 2006 it was determined that the concept could be integrated into the Gaia mission and subsequently DPAC was founded and the RVS working group was renamed Coordination Unit 6 (CU6). The team of scientists at various institutes in Europe (Meudon Observatory, MSSL, AIP, Universities of Antwerp, Bordeaux, Brussels, Liege, Ljubljana, Montpellier) was entrusted with the development of the appropriate evaluation methods and also had the task of creating the appropriate infrastructure for processing to provide these large amounts of data. CNES in Toulouse is responsible for data processing, together with setting up the spectroscopic pipeline. A particular challenge is connecting the pipeline with the results of the previous CUs. The pipeline started work in 2016 and the first results were published in Gaia DR2.
CU7 variability analysis
Variable stars are examined by CU7, their periods are examined and determined, and variability models are designed. The results of CU7 are in turn included in the calibration calculations, it also coordinates the external observations and announces newly recognized variable stars. The team examines the catalog data collected and searches for previously unrecognized variable stars.
CU8 Astrophysical characterization
CU8 undertakes the determination of extensive and non-punctiform celestial objects. It outputs the photometric and spectroscopic parameters, e.g. B. the spectral class. She takes care of star occultations and analyzes object clusters. It estimates the luminosity, age and mass of objects, divides all observed objects into certain classes and tries to identify groups of objects that belong together.
CU9 Archive and catalog access
CU9 was the last to be formed by all groups and will continue to work well beyond the life of the mission when all other groups have completed their tasks. CU9 takes care of the basic functions of archiving, database and interfaces for retrieval and the associated server for future generations of researchers, verifies and prepares the publication for the preliminary catalogs and the final catalog. The requirements for the catalog can change again and again with scientific progress, and future developments and questions must also be met.
CU9 is not allowed to release any data to individual scientists or institutes prior to publication in the official catalog. In individual cases, Gaia data were important for other space missions, e.g. B. the Gaia mission helped the New Horizons mission with precise data on the stellar occultations of (486958) Arrokoth . In these cases the data were published in advance.
CU9 Develops programs and applications for better accessibility and visualization of the data. CU9 supports the training and teaching of scientists and also serves general education and supports the presentation of the mission to the public.
Data packets of the Gaiamission
There are two basic types of data: on the one hand, data obtained by the space probe and, on the other hand, data that is collected during the mission. Doppler measurements, time of flight measurements and Delta-DOR measurements of the radio telescopes and the position data of the telescopes operated in the Ground-Based Optical Tracking Unit (GBOT), i.e. the data for determining the position of the probe, are such accompanying data.
Gaia itself produces a wide variety of raw data, which are prioritized and processed differently. Gaia generates data from different instruments and sensors with different data packages. There are three groups of data packets SP, ASD, SIF. Scientifically usable data are only contained in the so-called Star Packets (SP) and Ancillary Science Data Packets (ASD). While the Star Packets (SP) contain the actual data, the Ancillary Science Data Packets (ASD) are necessary for the reconstruction of positions and times. There are nine different Star Packets (SP1 to SP9) and 7 different Ancillary Science Data Packets (ASD1 to ASD7). There are also Service Interface Packets (SIF). These do not contain any scientific data, but contain data about the function of the service module, e.g. B. Power supply data, temperatures and other measured values. These data packages are necessary for the operation of the probe.
- SP1: The most common data packets. Every object that moves over the CCDs receives such a data packet. It contains the data from the Starmapper (SM), Astrometric Field (AF), Blue Photometer (BP) and Red Photometer (RP). Objects in this sense are all registered scan windows; it is possible that the scan window contains double stars or overlapping observations.
- SP2: The data packets are like SP1, if the object is bright enough for a measurement there is additional data from the Radial Velocity Spectrometer (RVS)
- SP3: These data packets contain objects that move clearly at right angles to the scanning direction. These are often objects of the solar system, mostly asteroids, which move comparatively strongly.
- SP4: The values of the Basic Angle Monitoring (BAM) CCDs for monitoring the basic angle are transmitted at certain intervals. It is not astronomical information, but it also comes from the sensor level.
- SP5: Data from the Wave Front Sensor (WFS) to monitor the focusing of the telescopes
- SP6 and SP7: are produced when a bright star appears, or when the position control system kicks in and the position has been changed briefly. It contains the sensor data from SM and AF1. These data are not taken into account in standard data processing.
- SP8 and SP9: Contain measurements from AF1 or SM of bright stars. This data type is only created under certain conditions and is not processed using standard data processing.
- ASD1: Position of the scan window across the scan direction for most CCDs, is created once per second for each of the 7 image processing systems (VPU).
- ASD2: Contains the electronic values of the unexposed sensor (pre-scan pixels), these are automatically generated approximately once per minute for each VPU.
- ASD3: Contains information about the RVS that the resolution has been changed. Originally this data package was supposed to be produced when a star is bright enough for the measurement with RVS. At the beginning of the mission, it was decided that the high resolution would always be used and not be changed. These data packets are therefore no longer generated in normal operation.
- ASD4: Statistical values and counter readings, e.g. B. About the number of objects, number of rejected objects, etc. Automatically generated every second.
- ASD5: Values about the times when Artificial Charge Injections occurred, generated every second with high precision
- ASD6: Information about the activation of gates on the CCDs. Is generated when a star brighter than G <12 is observed.
- ASD7: Contains the information about time and positions, when and where an SP1, SP2 or SP3 was detected by the VPU.
By December 31, 2018, the 1620th day since the start of scientific data collection on July 25, 2014, 116,023,842,167 objects had been recorded by the sensors, of which there were 1,143,663,587,072 astrometric measurements by the 62 astrometric and the 14th Skymapper CCDs. There were 231,915,375,586 photometric measurements by the 14 blue and red photometer CCDs. The RVS instrument for calculating the radial velocity recorded 22,300,583,616 spectra and 7,432,109,980 objects.
Work steps
During the daily downlink times, the Gaia data packets are transmitted to the ground station. In general, SIF and ASD data packets are transmitted as the first data, only then do the actual star packets (SP) follow, graded according to magnitude. This ensures that the function of the probe can be monitored in any case, that no calibration data is missing and that all received SPs can also be further processed, even if, for whatever reason, not all data have been transmitted. SP without corresponding calibration data and ASD would be useless and cannot be processed further. Processing can then begin as soon as the first SP is received and does not have to wait until all data are complete. In rare cases, more data is produced than can be stored and transmitted to the ground. In this case, the packages with the faintest objects are discarded in favor of brighter objects.
Once the data packets have reached the bottom, they are first processed further in a daily procedure. In addition to the daily work steps, there are epoch-related work steps for the complex recursive recalculation of the data. Some steps are run through only once and others recursively. There are algorithms that are only run through in daily procedures and others that only make sense with a certain amount of collected data.
The entire Gaiamission is divided into larger segments for processing, which can cover a period from several months to around a year. Some procedures use the database from a complete segment, so they can only be used after a segment has been completed. Each data release includes the results from a certain number of such segments.
| segment | begin | The End |
| 0 | 25-07-2014 | 03-06-2015 |
| 1 | 04-06-2015 | 16-09-2015 |
| 2 | 16-09-2015 | 23-05-2016 |
For the publication of Gaia DR1 the data segments 0 and 1 were used, for Gaia DR2 the segments 0 to 2, DR3 and the following catalogs will also include further segments.
Mission Operations Center Interface Task (MIT)
Immediately after the daily transmission, the data stream is checked for complete transmission, the various data packets are identified and initially stored in the correct place in the database. The data can come from different ground stations and have been transmitted on different days and therefore be mixed up. MIT is still being carried out by the MOC in Darmstadt and the data is stored in Villafranca, as well as a backup in the CNES data center in Toulouse. Data processing begins after a certain time, before all data packets have arrived and without it being certain that apparently missing data packets will ever arrive.
Initial Data Treatment (IDT)
This is the first preparation of the daily received data volume. One of the most important tasks is raw data reconstruction. In this procedure, the data packets are unpacked and details are reconstructed for each object: position, shape, the function of the gates, observation time, measurement of the amount of light, the data from the photometer and preliminary color, in the case of light objects also the results of the RVS. The exact calibration of the data is done later. At the same time, the orientation of the spacecraft and the telescopes and the basic angle BAM are determined with sufficient approximation that the objects can be identified as known objects in the object catalog, or else can be added as new objects.
Usually IDT happens once a day and is completed within a day. If parts of the data processing system are temporarily out of operation due to maintenance work, or if scans in particularly populated regions produce an above-average amount of data, the less relevant parts of the data can also be processed later.
The result of IDT is an approximation of the position data and a comparison with the already known objects, which is finally saved in the database. For the initial operation, the Attitude Star Catalog , which is contained in IGSL , was used to enable the first approximation of the positions, IGSL was the first set of objects on the basis of which the initial observations were compared. In the course of the observation, both IGSL and the Attitude Star Catalog were replaced by the object lists of the main database, which were created by Gaia itself.
On-ground Attitude Reconstruction (IOGA, ODAS, OGA1 and OGA2)
For the precise measurement and reconstruction of the position and alignment of the telescopes, the start trackers provide data with an accuracy of a few arc seconds. That is sufficient for the navigation of satellites and space probes. For Gaia, however, this method is too imprecise and one would like to achieve an accuracy of a few milli-arcseconds. There is an iterative model for this goal.
The first step is Initial On-Ground-Attitude (IOGA). The data from the position control of the probe are evaluated. OGA1 needs the information in the correct order as provided by IDT. The processing takes place in sections that z. B. be triggered by changes in the position of the probe by micrometeorites or by a "clank", a crack within the probe that is triggered by internal stresses in the components of the probe. The objects from different sections can therefore be offset from one another to a small extent. Sections can also be artificially created by the end of a data transmission or with the need to start a new calculation. The objects in a section are linked to one another by measurements and are processed together. Data from different sections are not linked to one another by measurements, the measurements have been interrupted for some reason, so that different sections are not processed together in order to achieve the highest possible consistency of the data.
The IOGA results form the output data of First On Ground Attitude (OGA1), which uses a Kalman filter to smooth the data and achieve an accuracy of 50 milli-arcseconds and 5 milli-arcseconds later in the mission. In addition to the preliminary IOGA data, OGA1 processes the transit identifier, the observation time and the observation angle, including the calibration data, as well as the object lists for comparison. The result of OGA1 is used for the daily One Day Astrometric Solution (ODAS).
Once the results have gone through the ODAS and First Look process there is an improvement in accuracy and the result is OGA2. The small shifts between the sections are eliminated. For the first two data segments, OGA2 achieved an accuracy of 50 mas, because it was still tied to the less precise Attitude Star catalog, but already achieved a consistency of the data among each other at a level of less than one mas, but is still against the ICRF in moved a rotational movement. It would be possible to use OGA1 data for AGIS, but OGA1 contains many gaps and interruptions, so using OGA2 simplifies further processing.
First Look (FL)
Immediately after IDT, the data is analyzed in ESAC. FL is essentially a quality check of the data. It is determined whether all systems and instruments function as desired and whether the data meet the scientific quality criteria. It is also determined when an intervention in the operation of the probe is necessary and at which points a new section is started. Part of the daily calibration and a second improved position determination from the ground takes place during First Look. A daily routine, partly independent of this, takes place in Turin, in particular to confirm the results of ESAC, which are important for the precision of the astronomical data.
First Look issues the science alerts and outputs data from newly found solar system objects for further tracking of the orbits with earth-based telescopes (SSO FUN).
Intermediate Data Updating (IDU)
The Intermediate Data Updating procedure is run through about once every six months. The files from a longer period of time, e.g. B. considered a data segment together. The data found at ODAS are checked, updated and recalibrated. In this context, calibration means recognizing and determining the systematic sources of error as well as calculating and minimizing systematic errors from the data. Calibration also means eliminating unsystematic errors, e.g. B. by recognizing erroneous measurements and outliers and collecting data through repeated measurements.
IDU has four main tasks:
- Updated comparison of objects based on the meanwhile improved object catalogs with improved position data and improved geometric calibration
- Improved calibration through additional data on the properties of the CCDs e.g. B. on the noise level and individual differences between the individual sensors, inclusion of the background brightness
- Improved calibration through better geometric evaluation of the scan window
- Improved calibration of the object positions through better known light flow of the objects
IDU uses both AGIS and PhotPipe, with the improved results from IDU being used by AGIS and PhotPipe. Gaia DR1 results are still based on IDT results, Gaia DR2 results are based on IDU and are therefore more accurate, more complete and more consistent.
This procedure is carried out by the MareNostrum supercomputer in Barcelona.
PhotPipe
PhotPipe is a procedure for handling the data to determine the G-band magnitude and the two magnitudes G BP and G RP of the blue (BP) and red (RP) photometers. For the G-band magnitude, the objects are evaluated during IDU with the help of point spread functions (PSF for 2D objects) or with line spread functions (LSF for 1D objects) and thus a total light flux over the entire sensitivity range of the Integrated sensors , called the G-band magnitude. These functions are also used to calculate the center of mass and to determine statistical values and characteristics for the quality of the measurement.
For the values of the two photometers G BP and G RP , however, the raw data from IDT is used and the evaluation and calibration is carried out directly by PhotPipe. Photpipe takes care of all processing steps from the raw data to the integration of the light flows and the calibration to the results. Photpipe uses the data synchronization of the IDU, in which various measurements are assigned to a specific object and artifacts due to overexposure to bright objects are eliminated. Photpipe uses the information about the background brightness, about geometric distortions caused by the optics and about the condition of the sensors, e.g. B. hot pixels or radiation damage in the sensors.
The large amount of data that has to be processed within a certain period of time requires a distributed system . Photpipe uses Apache Hadoop for this .
Astrometric Global Iterative Solution (AGIS)
AGIS is an iterative mathematical procedure that gradually assembles billions of data into a map of the entire Milky Way and the universe. The accuracy of the data improves in accordance with the number of measurements of an object and the computation requirement increases accordingly. Data from several observations on already known objects are compared. AGIS also converts the position data of the objects on Gaia's focal plane into barycentric coordinates. The Gaia mission uses several different coordinate systems, each of which has to be converted.
| Coordinate system | Zero point | rotation | Time system |
|---|---|---|---|
|
Barycentric Coordinate Reference System These coordinates coincide with the International Celestial Reference Frame |
Barycenter of the solar system | not rotating | Barycentric Coordinate Time (TCB) |
|
Center-of-Mass Reference System (CoMRS) This value describes the coordinates of the space probe in space |
Center of gravity of the Gaia spacecraft | not rotating | TCB |
|
Scanning Reference System (SRS) The coordinates are fixed in relation to Gaia's telescopes. This indicates the orientation of the telescope |
Center of gravity of the Gaia spacecraft | rotating with Gaia | TCB |
|
CCD pixel coordinates The pixel coordinates describe where an object is located on the CCDs at a certain time. |
CCD position with time stamp | Fixed axis, according to the CCD geometry | On-Board Mission Timeline (OBMT) |
AGIS uses some of the objects as primary objects. These are preferably well-observed objects that have a good distribution of position and magnitude measurements. These are single stars or quasars that are distributed as evenly as possible across the sky. There were 16 million primary objects for Gaia DR2, including around half a million quasars from the existing quasar catalogs and the ICRF catalogs. The remaining secondary objects are each assigned to such a primary object and their positions are determined relative to the primary location. There are approximately 150 secondary objects to a primary object, the number being variable.
AGIS goes through four phases:
- AGIS pre-processing : All input data are sorted according to their processing, e.g. B. all observations of an object are grouped and sorted by time. In doing so, data with implausible values are filtered out, also when the quality is not assured, i.e. in times when there is some anomaly. In the final step of the preparation, the primary object is selected for each object, which is used to determine the coordinates and the calibration.
- Primary source processing : In this phase, the positions of the primary objects, the position and orientation of the probe, the influence of the background brightness, the calibration values etc. are calculated with the best possible accuracy in an iterative process. The coordinates of the primary objects are then aligned on the ICRF. In this process, the data from different sections are merged
- Secondary source processing : In this phase the values for the secondary objects are calculated. The values for the primary objects from the primary source processing are used as calibration. The values of the secondary objects are calculated relative to the primary objects, which simplifies the calculation enormously. To ensure the consistency of the data, the primary objects are calculated a second time.
- AGIS post-processing : Finally, the results of primary and secondary processing are merged. The results are stored in the main database in a format that is easier for users to use. The times are converted into OBMT and additional information that is of no use outside of AGIS is removed.
Most of these AGIS calculations are done by the MareNostrum supercomputer in Barcelona. The control of AGIS is with CU3.
Results
The results of the working groups are collected and published in catalogs, each of which contains a number of sub-catalogs, and further catalogs have been announced. All data in the catalogs are freely accessible and can be used by everyone. The participating scientists use the results for their own research projects and can publish their findings in their own publications under their own name, but they cannot use any data for this that have not yet been published. For certain critical applications, data for individual objects can be published outside of the large publications. B. the case with certain space missions. For example, data for occultations were published in advance for New Horizons .
DPAC has published the following catalogs so far:
- Gaia DR1 , 1.1 billion objects, mostly with 2 parameters (position, G-magnitude )
- Gaia DR2 , 1.7 billion objects, mostly with 5 parameters (position, G-Magnitude, parallax, angular velocity) and additional photometer data G BP Magnitude / G RP Magnitude. ,
Expected for the end of 2020:
Web links
literature
- L. Lindegren, J. Hernandez, A. Bombrun, S. Klioner, U. Bastian: Gaia Data Release 2: The astrometric solution . In: Astronomy & Astrophysics . tape 616 , August 2018, ISSN 0004-6361 , p. A2 , doi : 10.1051 / 0004-6361 / 201832727 , arxiv : 1804.09366 .
Individual evidence
- ^ List of Institutes involved in DPAC. Retrieved November 9, 2017 .
- ↑ GAIA @SSDC. Retrieved December 26, 2017 .
- ↑ DPAC Consortium - Cosmos. Retrieved August 13, 2017 (UK English).
- ↑ a b c d e f Gaia Collaboration: Gaia Data Release 2; Documentation release 1.2 . Ed .: European Space Agency and Gaia Data Processing and Analysis Consortium. June 5, 2019, p. 73-84 ( esa.int [PDF]).
- ↑ GaiaDR2_CU6 - Gaia - Cosmos. Retrieved August 3, 2019 .
- ^ Delivering the promise of Gaia, Response to ESA's Announcement of Opportunity, Proposal for the Gaia Archive . ( esa.int ).
- ↑ a b Gaia Collaboration: Gaia Data Release 1; Documentation release 1.2 . Ed .: European Space Agency and Gaia Data Processing and Analysis Consortium. December 18, 2017, p. 70-72 ( esa.int [PDF]).
- ↑ At this point, a knowledge of the instruments and sensors of the Gaia space probe is necessary for full understanding. The relevant information and resolutions of the acronyms can be found in all details in the article Gaia (space probe) .
- ↑ Gaia Mission Status Numbers. ESA, accessed December 31, 2018 . This page is updated several times a day with new figures.
- ↑ a b Gaia Collaboration: Gaia Data Release 1; Documentation release 1.2 . Ed .: European Space Agency and Gaia Data Processing and Analysis Consortium. December 18, 2017, p. 103 ( esa.int [PDF]).
- ^ C. Fabricius et al .: Gaia Data Release 1; Pre-processing and source list creation . Ed .: A&A. November 24, 2016, doi : 10.1051 / 0004-6361 / 201628643 .
- ↑ M. Riello, F. De Angeli, DW Evans, G. Busso, NC Hambly, M. Davidson, PW Burgess, P. Montegriffo, PJ Osborne, A. Kewley, JM Carrasco, C. Fabricius, C. Jordi, C. Cacciari, F. van Leeuwen, G. Holland .: Gaia Data Release 2; Processing of the photometric data. Ed .: A&A. 616, A3, August 10, 2018, doi : 10.1051 / 0004-6361 / 201832712 .