Sustainability of Digital Formats: Planning for Library of Congress Collections

Flexible Image Transport System (FITS) was initially developed by astronomers in the USA and Europe in the late 1970s to serve the interchange of data between observatories and was brought under the auspices of the International Astronomical Union in 1982. In 2012, FITS is still in widespread use as a data interchange and archiving format by astronomers. FITS is a file format designed to store, transmit, and manipulate scientific images and associated data. The term "image" in the standard's name is loosely applied and FITS files often contain only non-image data. Astronomers view their images as data for analysis rather than simply as pictures to look at. From its beginning, FITS was seen as a transport format for more than a still image intended for printing or viewing in two dimensions. FITS was designed to facilitate the unambiguous transmission of n-dimensional regularly spaced data arrays, an n-cube. These multi-dimensional arrays may be 1-D spectra, 2-D images or data cubes of three or more dimensions. Two-dimensional tables containing rows and columns of data can also be stored in a FITS file. The substantial use of FITS for data that even astronomers would not class as image data led to consideration to change the full name for the FITS format, but the proposal was rejected. For the same reason, in this resource, FITS is categorized primarily as a dataset format, with use for image data as secondary.

When FITS is used for straightforward image data, it is often (but not always) in one of two forms: two-dimensional (X, Y) with single values at each point, perhaps some measurement of intensity in a particular spectral band (comparable to grayscale); or three-dimensional (X, Y, color), with the third axis representing different points on a spectral axis, as measured by wavelength, frequency, or some other appropriate scale. There is no assumption in astronomical analysis or in FITS of a defined colorspace (such as RGB). Astronomy images are often rendered in false color in order to make infra-red and ultra-violet radiation visible [see Notes below]. A dimension of the n-cube may also represent time steps, or indeed be used to represent other data characteristics.

FITS is also often used to store non-image data, such as spectra, photon lists, data cubes, or even structured data such as multi-table databases. FITS includes many provisions for describing photometric and spatial calibration details, together with image origin metadata.

FITS was designed with an eye towards long-term archival use, and the maxim "once FITS, forever FITS" (or "once FITS, always FITS") has guided the IAU to ensure that the format is backwards compatible as new features are added. Version 3 (published in 2012 and distinguished in this format resource as FITS_3) documented several features approved since the previous edition of the full specification: support for 64-bit integers (from an earlier maximum of 32 bits) in images and tables; tables with variable-length arrays; and adopted new World Coordinate Systems (WCS) conventions to map an element in a data array to standard physical coordinates on the sky. WCS conventions also exist to define mappings for a spectral coordinate into a standard frame of reference, allowing spectral shifts due to the Doppler effect and/or cosmic expansion to be accounted for. The WCS conventions are implemented through an extensive set of reserved keywords and values to be used in headers associated with image arrays. A WCS convention for specifying time and relative dimensions in space is in preparation as of early 2013.

A FITS file consists of one or more Header + Data Units (HDUs), where the first HDU is called the primary HDU, or primary array. The primary array may be empty or contain an N-dimensional array of pixels, such as a 1-D spectrum, a 2-D image, or a 3-D data cube. Data types supported in the primary array in FITS_3 are: unsigned 8-bit bytes, 16, 32, and 64-bit signed integers, and 32 and 64-bit single or double precision floating point reals (using the ANSI/IEEE-754 standard, approved by IEEE in 1985). FITS_3 documented support for 16 and 32-bit unsigned integers. Any number of additional HDUs may follow the primary array; these additional HDUs are called FITS extensions. A FITS file may have an empty primary array and comprise only extensions. Three standard types of extensions have been included in the FITS Standard since the mid-1990s:

• Image Extension - an N-dimensional array of pixels, as in a primary array
• ASCII Table Extension - rows and columns of data in ASCII character format
• Binary Table Extension - rows and columns of data in binary representation, with the same numeric data types supported as for the primary array

For more detail on the form of HDUs, see Notes, below.

FITS is maintained by the International Astronomical Union (IAU) FITS Working Group.  Version 3 of the specification document incorporates incremental changes made in earlier years, was approved in 2008, and published formally in 2010.  Note that the "once FITS, forever FITS" maxim has led the working group to avoid explicit identification of a version within a FITS file; thus versioning is not considered "strong."

The FITS file format is a well-established format for exchange and archiving of astronomical data. It is supported by the International Astronomical Union, NASA, and other national and international bodies engaged in astronomy and related fields. Data from major international programs, such as the Hubble Telescope, the Sloan Digital Sky Survey, and the orbiting Chandra X-ray Observatory are released in the FITS format. So are records from land-based observatories, such as the National Optical Astronomy Observatory (NOAO), the National Radio Astronomy Observatory (NRAO), and the Keck Observatory. In Europe, the land-based European Southern Observatory (ESO) and recent missions of the European Space Agency (ESA) release their data in the format.

Software libraries for reading and writing FITS files exists in many programming languages. See https://fits.gsfc.nasa.gov/fits_libraries.html. An important example is CFITSIO, a library of subroutines in C and Fortran maintained by NASA's High Energy Astrophysics Science Archive Research Center (HEASARC). Many other tools build on this library. The GDAL (Geospatial Data Abstraction Library) uses the CFITSIO library to support conversion among raster image formats, including FITS.

Image viewers and format converters that support FITS are listed at https://fits.gsfc.nasa.gov/fits_viewer.html. Utilities also exist to convert FITS images to image formats for publication and display. Examples are FITS Liberator and STIFF.

In March 2010, the Vatican Library announced that historical scans of ancient books and manuscripts in the Vatican's collections would be stored in the FITS format. A December 2011 press release from the European Space Agency similarly reported the Vatican Library's plan to take advantage of the long-lived format developed for astronomy.

The FITS format is primarily for use with astronomy images and/or data; it is not a format for raster images in general. Many FITS files have no image data, containing, for example a catalog of celestial objects. A useful description of FITS for an audience interested in the applicability of FITS for storing scanned images of cultural heritage materials, such as manuscripts, was given by Lucio Chiappetti at the 2012 European Week of Astronomy and Space Science [https://www.vatlib.it/moduli/Ammenti_EWASS2012.pdf] at a special session on long-term preservation of digital cultural resources.

Astronomy images in the FITS format are typically not "normal" images in one important sense; they are ready for analysis, not ready for human viewing. In particular, the treatment of color, or rather the equivalent of color extended outside the visible spectrum, is very different from that in the raster image formats used for digital photography, such as TIFF. In the most natural "sky" images, the X,Y dimensions are celestial coordinates and the Z dimension is related to intensity (e.g., flux, or number of photons). In FITS there is no default colorspace or standard way to identify that one of the conventional colorspaces (such as sRGB) has been used. See Notes below for a discussion on false color for astronomy images. Utilities exist to convert FITS images into TIFF_6, BigTIFF, or other image formats for publication and display. Examples are FITS Liberator and STIFF. A list of FITS Image Viewers and Format Converters is made available by the FITS Support Office. In addition, the simplest FITS image files can be imported into a few general graphics applications (such as ImageMagick, GIMP, or Graphic Converter) and support panning, zooming, image manipulation, etc. in the same way as other raster image formats.

Data in a FITS file is required to be stored in big-endian byte order. Numeric data types supported within FITS_3 are: signed (two's complement) integers of 16, 32, and 64 bits; and unsigned 8-bit integers; floating point (32-bit) and double-length (64-bit) binary numbers in IEEE-754 (1988) Basic formats. Unsigned integers of 16, 32, or 64 bits (and 8-bit signed integers) can be stored by shifting into the range of the corresponding signed integer and using the BZERO keyword to indicate the shift that must be applied to restore the original value. Character data is stored in one-byte characters and limited to 7-bit ASCII. Compared to a general dataset format, such as DBF, DBase Table File Format, this set of supported data types is limited.

FITS supports three basic data models, as exemplified in the specifications for the three extension types:

• An Image extension (or the primary array) is an n-cube of data, using a single data type (one of the six types listed above).
• An ASCII table extension provides a mean for storing catalogs and tables of astronomical data. Each row is a fixed length, divided into fixed-length fields, which can represent ASCII character strings, or numbers in character form. Numbers in ASCII tables can be integers, or use one of three Fortran-compatible representations for floating point numbers.
• A Binary table extension offers more features and more compact data storage for numeric data than an ASCII table. Rows have a fixed number of bytes, divided in a fixed way into fields. Variable-length data can be stored by using an array descriptor in the field in a row and storing the variable-length data for all rows in a heap area that follows the main data table. Not all applications supporting FITS data tables can handle variable-length arrays.

Form of FITS HDUs: The FITS format was first standardized in 1981 and terminology in the early specifications make clear its origin in an era of punched card images stored on magnetic tape.  The standard has evolved gradually since then, with proposed changes published as journal articles with opportunity for feedback over an extensive period.  However, 80-byte header records and other features of the format still reflect its early origin. Every HDU consists of a human-readable ASCII header unit followed by an optional data unit. Each header or data unit must be an exact multiple of 2880 bytes long. Unused space at the end is padded with zeroes or blanks as appropriate. Headers consist of 80-character keyword records which have the general form: KEYNAME = value / comment. Each header unit begins with a series of required keywords that specify the size and format of the following data unit. The required keywords may be followed by other optional keywords to describe various aspects of the data, such as the date and time of the observation. COMMENT or HISTORY keywords are also frequently added to further document the contents of the data file. The data unit, if present, immediately follows the last 2880-byte block in the header unit.

False color: In astronomy, the data value(s) associated with a "pixel" are measurements of intensity in a spectral frame of reference that is usually very different from the conventional colorspaces used for printing or screen display. Depending on sensors and equipment design, an image may represent intensity in a single spectral band (which may be narrow or wide), or intensities in separate spectral bands that may vary from radio waves (infra-red) to gamma radiation (ultra-violet). To be viewed, the images must be transformed by manipulating (by stretching, squeezing, cropping, etc.) the spectral histogram into one in the visible range. The result is known as "false color." See the tutorial Chandra Images and False Color for a good explanation of why "false" color is needed and how astronomers use different color approaches to convey different aspects of what has been observed. [Note: Chandra is an orbiting X-ray observatory operated by the Smithsonian's Astrophysical Observatory at Harvard.]

Application-specific conventions and extensibility: User communities, for example for sub-disciplines, have developed generic and application-specific conventions that refine the FITS standard for increased interoperability in specific contexts or for specific categories of astronomical data. The standard also defines a mechanism for adding conforming extensions to the file structure.

The FITS data format was developed in the late 1970s for interchange of astronomical image data between observatories. In 1981, the FITS specification was published in Astronomy and Astrophysics Supplement, Vol. 44, pp. 363-370. By then, FITS had become the de facto standard interchange format for astronomy data. In the 1990s several versions of the specification were published by NOST (NASA Office of Science and Technology): NOST 100-1.0 (June 1993); NOST 100-1.1 (September 29 1995); NOST 100-1.2 (April 1998); and NOST 100-2.0 (March 1999). FITS Version 2.1b was published in December 2005. Version 3.0 was approved in July 2008, and formally published in 2010. Version 4.0 was adopted in July 2016, and formally released in August 2018 after editing for language, not substance. A complete summary of the FITS 4.0 language editing is also available from the FITS Standard Document site. All versions since 1993 are available at https://fits.gsfc.nasa.gov/fits_standard.html.

The description of the specification for FITS Version 4.0 at https://fits.gsfc.nasa.gov/fits_standard.html indicates, "The main difference from the previous Version 3 of the document is that a number of FITS conventions are now officially recognized, including conventions for representing Time world coordinates, and conventions for compressing FITS images and tables. A complete list of all the changes can be found in appendices H.3 and H.4 of the document."

See https://fits.gsfc.nasa.gov/fits_overview.html for more details of the early history.