Main Panel ********** This is the central part of the VDAT window. It is divided in two parts: * a column on the left: it shows the reduction steps as horizontal tabs and, on the lower part, buttons to execute one or more commands; * a big area on the right: typically this part shows fits files for the directory and the step selected. .. figure:: /_static/vdat_main_panel.png :alt: a screenshot of the central VDAT panel Screenshot of the central VDAT panel, showing all the elements discussed in this section Configuration file ================== The content of the main panel is configurable by the user. The configuration file is written according to the `YAML `_ standard. VDAT ships a standard ``tasks.yml`` file, that can be retrieved using the ``vdat_config copy`` command. When starting, ``vdat`` will look for such a file in the current directory. A different file can be provided using the ``-t/--tasks-config`` command line options. The configuration file must contain a number of sections, one for each type of files that the user wants to visualize. The name of the sections correspond to the types that are shown in the :doc:`tree_view`. Each section contains a list of steps, each one corresponding to one of the horizontal buttons in the upper part of the left panel. The main structure is as in this example: .. code-block:: yaml zro: # configuration for the zero/bias shots - # step 1 [...] - # step n [...] cal: # configuration for the flat and arc shots - # step 1 [...] - # step n [...] sci: # configuation for the science shots - # step 1 [...] - # step n [...] Each step has three sections, like in this example: .. code-block:: yaml - step_name: Raw buttons: # description of the buttons to create - # button 1 [...] - # button n [...] tabs: # descriptions of the tabs to show - # tab 1 [...] - # tab n [...] * ``step_name`` (mandatory): this is the name that appears in the horizontal tab. Typically it represents one step of the reduction process. * ``buttons`` (optional): list of buttons to show in the lower part of the left panel. See :ref:`buttons` for further information. * ``tabs`` (optional): list of tab types to show in the right panel. See :ref:`tabs` for more info. .. _buttons: The buttons ----------- To each step can be associated one or more button, each of which can run one or more commands. Each command is interpreted and run as described in :doc:`../command_interpreter`. The buttons are defined as a list and each item is composed by: * ``name`` (mandatory): the name that appears in the button. * ``commands`` (mandatory): a string or a list of strings. Each string define a command that will pushed into a queue when the button is clicked. * ``tool_tip`` (optional): if provided, the text is used as tool tip for the button. As an example, the following will create two buttons, called ``Subtract Overscan`` and ``Reduce``, associated with one and two commands, respectively. Hovering over the first button the string ``Subtract the overscan from the frame`` will appear, while the second doesn't show any tooltip. .. code-block:: yaml buttons: - name: Subtract Overscan commands: subtractfits $args -o $biassec $fits tool_tip: 'Subtract the overscan from the frame' - name: Reduce commands: - subtractfits $args -o $biassec $fits - masterbias $args -o $target_dir/$masterbias_file $fits .. _tabs: The tabs -------- For each step and file type, the user might want to inspect files, logs and have tools to validate or compare the results. To allow this, we reserve the largest portion of the VDAT GUI. In this area multiple tabs can be added by the user via the aforementioned configuration file. Tabs are defined as a list under the ``tabs`` section. Each item must have the ``tab_type`` entry, specifying the type of tab to add, while all the other entries depend on the type of tab. Each tab type can create one or more tabs, according to its specifications. A generic configuration entry looks like the following example: .. code-block:: yaml :emphasize-lines: 2,5 tabs: - tab_type: type1 [all the options for the tab type 1] [...] - tab_type: typeN [all the options for the tab type N] VDAT ships with few tab types, described in :ref:`builtin_tabs`. Also it is possible to add extra tabs types using the plugin mechanism described in :ref:`new_tabs`. List available tab types ++++++++++++++++++++++++ VDAT provides a way to discover which tab types are available on the system, via the ``vdat_plugins`` executable. The command:: vdat_plugins tab_types lists all the available tab types names; the option ``-v\--verbose`` will show the doc strings for all the tab types. It is also possible to display only one type providing its name in the command line. E.g:: vdat_plugins tab_types exp_combined shows only the ``exp_combined`` type and its doc string (this command ignores the ``verbose`` settings). .. _builtin_tabs: Builtin tab types +++++++++++++++++ In this section are described the builtin types and the arguments required to use them. We will provide more types in the future. All the builtin tab types, unless otherwise noted, use the focal plane file provided via the ``[fplane]`` section main vdat configuration file to build the focal plane shown in the tabs. If the input file, given with the option ``fp_filename`` contains IFUs that you don't want to show, it is possible to list the undesired IFUSLOTs using the ``exclude_ifuslot`` option. Finally the ``respect_empty`` option allows to decide whether empty rows and columns in the focal plane file must be shown (``yes``) or can be hidden (``no``). Here is an example of the relevant configuration entries: .. code-block:: ini [fplane] # Settings related to the focal plane fp_filename = ${general:config_dir}/fplane.txt # (optional, default empty) comma separated list of IFUSLOTs that should not be # shown in the GUI. There is no harm listing IDs even if they are not in the # fplane file exclude_ifuslot = 000, 555, 056, 066 # (optional, default: ``yes``) If the options is set to ``no``, the focal plane # uses the maximum available space and all the empty columns and rows are not # shown. If ``yes``, shows empty internal rows and columns, but not outside the # range of existing IFUs. # Example: the focal plane has the following two IFUSLOT: 073 and 095. # * if ``no``: the two IFUs are shown at the upper left and lower right corners # of a 2x2 square, i.e. the empty column and row between them is not shown # * if ``yes``: the two IFUs are shown at the upper left and lower right # corners of a 3x3 square respect_empty = yes .. note:: the same focal plane file is used internally by the VDAT GUI to register the selected IFUs. .. _exp_tabs: ``exp_fits`` and ``exp_combined`` ................................. Each shot consists of a number of exposures. Also multiple calibration shots might be combined into a unique directory by VDAT. These tab types allow to show each exposure in one tab. The difference between the two types is that the ``exp_fits`` type shows the fits files, while the ``exp_combined`` adds a checkbox that can be used to display to a "quick reconstruction" image. Both types accept the options described below, together with the formatting names available for building file names and tab titles. .. autofunction:: vdat.gui.tabs.entry_points.exp_fits :noindex: .. autofunction:: vdat.gui.tabs.entry_points.exp_combined :noindex: As example the following configuration creates the tabs shown in the image below. .. code-block:: yaml :emphasize-lines: 2 - &cal_raw_exp_combined tab_type: exp_combined file_name: '{basename}_{ifuslot}{col}{row}_{orig_type}.fits' cols: ['L', 'R'] rows: ['U', 'L'] title: '({step}) {object}: {exp}' tool_tip: '({step}) {object}, {type}, {orig_type}: {exp}' header_keys: ['SPECID', 'IFUID', 'IFUSLOT', 'CCDPOS', 'CCDHALF'] .. figure:: /_static/fplane_exp_combined.png :alt: a screenshot of the tabs created by the exp_combined type Screenshot of the tabs created by the ``exp_combined`` type, when selecting a directory containing two cmp, one flt and one twi shots, each one with one exposure. The tab types create multiple tabs, one per each exposure in the selected directory. Each of the squares represent one IFU. If one shows a white "X" on black, it means that no file has being found. Otherwise the area is divided in parts, according to the number of ``cols`` and ``rows`` in the configuration, and each part is filled with the thumbnail of one file. In the example there are four files per IFU per exposure. Each file is displayed using the `zscale scaling algorithm `_. Since each IFU is represented by a rectangle only a few centimeters in size, we don't display the fits files themselves, but a much smaller thumbnail that is created, updated or removed at need. Users can remove thumbnails from the selected directory or from everywhere using the :ref:`file_menu`. If at least one file is shown, the user can change the scaling of the images using the radio buttons below the focal plane. ``Global`` repaint the thumbnails using the absolute minimum and maximum index from the ``zscale`` algorithm. The limits are shown in a tool tip that appears when hovering over the button for a few seconds. The ``Custom`` button allow to set user selected upper and lower limits; the limit submitted via the ``Rescale`` button are applied on all the IFUs. The three radio buttons can also be used to repaint the thumbnails. This is useful, e.g., when files has been created or removed. For the ``exp_combined`` type, the ``Reconstructed`` checkbox on the lower right part allows to switch between a view showing the fits files and one showing their combination. This use the :class:`~pyhetdex.het.reconstruct_ifu.QuickReconstructedIFU` to build an approximated image from the fits files. The image is created only if all the files required exists. The reconstruction algorithm needs some extra information to know where fibers located on the fits files and which part of the sky they see. The information are provided to VDAT via the ``vdat_setting.cfg`` file: .. code-block:: cfg [reconstruction] # The name of the IFUCen file ifucen = ${general:extra_files}/IFUcen_HETDEX.txt default_dist_r = ${general:extra_files}/defaultR.dist default_dist_l = ${general:extra_files}/defaultL.dist pixscale = 0.5 VDAT provides a copy of such files and, of course, the user can use other files as fit. The pixel scale gives the size of a pixel: the smaller the number the more pixels will be used in the reconstructed image and the more realistic it will look (with all the caveats given by the fact that the reference files might not be accurate for the files at hand). This comes at the cost of greater time needed to initialize the reconstruction. However we cache the object used to performed the reconstruction, so the impact of the pixel scale will be visible only when creating the first tab that requires it. In order to be able to run command, at least one IFU must be selected. This can be achieved by clicking on the desired IFUs. When selected, an IFU shows a blue border. Clicking on a selected IFU, will deselect it. It is also possible to select or deselect all the IFUs using :ref:`select_menu`. The tab type also offers the possibility to inspect both the fits files and the reconstructed image in a dedicated window. Double clicking will bring up a window with as many tabs as there are fits files in the IFU or, if clicking on the reconstructed image, with only one tab. The window shows both the fits files data and their header. Via the ``header_keys`` option in the yaml configuration, it is possible to enhance some header keywords by putting them on top of the list. The window is described with more details in :ref:`fits_viewer`. ``fits`` and ``fits_combined`` .............................. When creating master frames, the user combines multiple exposures into one single fits file. Thus the previous tab type is not suited to display these new files and the ``fits`` or ``fits_combined`` type can be used: .. autofunction:: vdat.gui.tabs.entry_points.fits :noindex: .. autofunction:: vdat.gui.tabs.entry_points.fits_combined :noindex: These types produce one or more tabs that look and behave like the ``exp_combined`` ones. E.g. .. code-block:: yaml :emphasize-lines: 4, 5, 15, 16 zro: step_name: Masterbias tabs: - &masterbias_tab tab_type: fits_combined file_name: 'masterbias_{ifuslot}_{col}{row}.fits' cols: ['L', 'R'] rows: ['U', 'L'] title: '{step}' tool_tip: 'Reduction {step}. Masterbias' cal: step_name: Masterframes tabs: - <<: *masterbias_tab orig_type: ['flat', 'arc', 'twi'] file_name: 'master{orig_type}_{ifuslot}_{col}.fits' rows: ['',] title: 'Master{orig_type}' Will create one tab (``Masterbias``) for a ``zro`` directory and three (``Masterflat``, ``Masterarc`` and ``Mastertwi``) for a ``cal`` directory. An example of the latter is shown in the figure below. .. figure:: /_static/fplane_fits_combined.png :alt: a screenshot of the tabs created by the fits_combined type Screenshot of the tabs created by the ``fits_combined`` type, when selecting a directory containing a master frame for arcs, flats and twilights. Without providing the ``orig_type`` keyword, VDAT would have used the information from the internal database and would have created three tabs with ``{orig_type}``, both in the file name and the tab title, replaced by ``cmp``, ``flt``, ``twi``. If for some reason ``orig_type`` from the example above does not suffice, it is possible to specify a custom list of file types. The following example will create three tabs called ``cmp, flt, twi round``, ``cmp, flt, twi bow`` and ``cmp, flt, twi 42``. .. code-block:: yaml :emphasize-lines: 4, 5 cal: tabs: - <<: *masterbias_tab file_types: custom_name custom_name: ['round', 'bow', '42'] file_name: 'master{custom_name}_{ifuslot}_{col}.fits' rows: ['',] title: '{orig_type} {custom_name}' Clicking and double clicking on each IFU behaves as described at the end of :ref:`exp_tabs`. .. _fits_cube: ``fits_cube`` ............. This tab type is almost identical to the ``fits`` one, except for the fact that is specialised for displaying data cubes. It accepts only one extra configuration option: .. autofunction:: vdat.gui.tabs.entry_points.fits_cube :noindex: This type produce one or more tabs that look and behave like the ``fits`` ones. As example take the following configuration entries: .. code-block:: yaml - &sci_cube_tab tab_type: fits_cube title: 'Cube 20' file_name: 'CuFeSpdsses_{ifuslot}.fits' z_indx: [null, 40] tool_tip: 'Compressed cube slice' - <<: *sci_cube_tab title: 'Cube 600' z_indx: [550, 650] This creates two tabs, called ``Cube 20`` and ``Cube 600``, each showing a compressed slice of the data cube; the former tab shows the data for z index in the range [1, 40] while the latter for [551, 650]. The compression is done using :func:`numpy.nanmedian`. If ``z_indx`` is not given or its value is ``[null, null]`` the full cube is compressed. The following image shows an example of the focal plane created by the above configuration .. figure:: /_static/fplane_fits_cube.png :alt: a screenshot of the tabs created by the fits_cube type Screenshot of the tabs created by the ``fits_cube`` type Clicking and double clicking on each IFU behaves as described at the end of :ref:`exp_tabs`. In addition in this case the :ref:`fits_viewer` also uses the ``z_indx`` option. .. _fits_multiext: ``fits_multiext`` ................. This tab type is almost identical to the ``fits`` one, except for the fact that it display the desired extensions from multi extensions fits files. It accepts one extra configuration option and provide one more formatting name, described in: .. autofunction:: vdat.gui.tabs.entry_points.fits_multiext :noindex: This type produce one or more tabs that look and behave like the ``fits`` ones. As example the following configuration: .. code-block:: yaml - &cal_fmod tab_type: fits_multiext orig_type: ['twi', ] file_name: 'master{orig_type}_{ifuslot}_{col}.fmod' rows: ['',] cols: ['L', 'R'] title: 'mastertwi fmod {ext}' extensions: ['AMPLITUDE', 'Y', 'SIGMA', 'H2', 'H3', 'EXP'] creates six tabs, called ``mastertwi fmod AMPLITUDE``, ``mastertwi fmod Y`` and so on. Each tab shows the one extension of the fits file called ``mastertwi_{ifuslot}_{col}.fmod``. The following configuration: .. code-block:: yaml - <<: *cal_fmod extensions: [1, 2, 3, 4, 5, 6] creates the same tabs, but with titles like ``mastertwi fmod 1`` and possibly with a different order. An example of the tabs can be seen in the image below: .. figure:: /_static/fplane_fits_multiext.png :alt: a screenshot of the tabs created by the fits_multiext type Screenshot of the tabs created by the ``fits_multiext`` type Clicking and double clicking on each IFU behaves as described at the end of :ref:`exp_tabs`. In addition in this case the :ref:`fits_viewer` also uses the ``ext`` option. ``reconstruct`` ............... Science shots usually consist of multiple dithered exposures, typically three for HETDEX observations. The ``reconstruct`` tab type allows to merge all the exposures of one shot into one image. The type creates always only one tab. The configuration options and formatting names are very similar to the previous two tab types: .. autofunction:: vdat.gui.tabs.entry_points.reconstruct :noindex: Reusing the configuration entries shown for ``exp_combined``, we can easily create one tab the exposure and add a ``reconstruct`` tab as shown here: .. code-block:: yaml :emphasize-lines: 2, 3, 4 tabs: - *cal_raw_exp_combined - <<: *cal_raw_exp_combined tab_type: reconstruct title: '{step} reconstructed' tool_tip: 'Reduction {step}. Object: {object}; type: {type}; original type: {orig_type}' The figure below shows the result for the ``reconstruct`` type. The tab behaves and feels like in the previous cases, with the exception that there is no ``"Reconstructed"`` button to switch between the fits files and the reconstructed image. .. figure:: /_static/fplane_reconstruct.png :alt: a screenshot of the tab created by the reconstruct type Screenshot of the tabs created by the ``reconstruct`` type Clicking and double clicking on each IFU behaves as described at the end of the ``exp_combined`` section. The ``text_file`` ............... Some reduction steps create text files and this tab type allow to show them. The number of lines is shown in the focal plane. The configuration options and formatting names are very similar to the previous tab types: .. autofunction:: vdat.gui.tabs.entry_points.text_file :noindex: An example of a configuration entry for the ``text_file`` tab is shown here: .. code-block:: yaml tabs: - tab_type: text_file file_name: 'dither_{ifuslot}.txt' title: 'Dither file' tool_tip: 'Reduction {step}. Show the dither file' The figure below shows a tab created with the above configuration. The tab behaves and feels like in the previous cases. However the content is very different: the upper line shows the total number of lines, while the one lower one shows within parenthesis the number of non comment lines. A comment line starts with ``#``. .. figure:: /_static/fplane_text_file.png :alt: a screenshot of the tab created by the text_file type Screenshot of the tab created by the ``text_file`` type Clicking on each IFU behaves as described at the end of the ``exp_combined`` section. On double click, a :ref:`text_file_viewer` window is opened, if the file exists. .. _dist_widget: ``dist`` ........ One of the products of the ``deformer`` step are distortion solutions. This tab type shows if distortion files have been produced (i.e. if the ``deformer`` step worked) and allow to display them. The configuration options and formatting names are : .. autofunction:: vdat.gui.tabs.entry_points.dist :noindex: An example of a configuration entry for the ``dist`` tab is shown here: .. code-block:: yaml tabs: - tab_type: dist file_name: 'mastertwi_{ifuslot}_{col}.dist' fits_names: ['mastertwi_{ifuslot}_{col}.fits', 'masterarc_{ifuslot}_{col}.fits'] rows: ['',] cols: ['L', 'R'] title: 'mastertwi distortion' The figure below shows a tab created with the above configuration. The tab behaves and feels like in the previous cases. However the content shows the version of the distortion file, in the image ``D: 17``, and whether the region file has been created (``R: yes`` or ``R: no``). .. figure:: /_static/fplane_dist.png :alt: a screenshot of the tab created by the dist type Screenshot of the tab created by the ``dist`` type Clicking on each IFU behaves as described at the end of the ``exp_combined`` section. On double click, a :ref:`dist_viewer` window is opened, if the at least one of the region files exist.