New tab types

This document provides guidance and examples on how to create new tab types and reuse some of the functionalities implemented in VDAT.

Entry point and packaging

Each and every tab type, included the ones shipped with VDAT, is a plugin that is loaded and executed when needed. This system offers various advantages, among which:

• new tab types from third party packages;
• clear decoupling between the container (the GUI) and the content (the tabs).

All the tab types are advertised and loaded using the entry point mechanism. VDAT expect them to be advertised in the vdat.tab_types entry point group.

An example is probably the best way to illustrate how to use this to feature to extend VDAT. Lets assume that a user needs a new tab type, that we will call cool_tab. The best way to implement it is to create a new package, e.g. vdat_ext with the following structure:

vdat_ext
  ├─ setup.py
  ├─ vdat_ext
  │    ├─ __init__.py
  │    └─ core.py
  └─ tests/

The setup.py file is need to tell tools like pip how the package is to be installed, which dependences it needs and so on. The minimal setup instruction looks like:

from setuptools import setup, find_packages

setup(
    name='vdat_ext',
    packages=find_packages(),
    entry_points={'vdat.tab_types':
                  ['cool_tab = vdat_ext.core:cool_tab_function'],
                 }
)

The tab type is implemented in vdat_ext/core.py following the template provided in vdat.gui.tabs.interface

in a function called cool_tab_function:

from vdat.gui.tabs.interface import FplaneTabTemplate

def cool_tab_function(target_dir, tab_dict, step_name, cache,
                      parent_widget):
    '''This function implements the ``cool_tab`` plugin. It receive a
    number of input parameters, create and/or setup the tab(s) and return
    it(them).

    If the tab type provides multiple tabs, create all of them in this
    function appropriately and then return a list containing all of them in
    the order they should appear'''
    tab_type = tab_dict['tab_type']
    # if using the cache, try to get one object from the cache. If
    # use_cache == False there is no need to do this
    obj = cache.from_cache(tab_type)
    if not obj:
        obj = CoolTab(tab_type, parent=parent_widget)
    # prepare the arguments and keyword arguments
    obj.setup(arguments, kw_arguments)

    return [obj, ]

class CoolTab(FplaneTabTemplate):
    '''Class implementing the tab(s) for the ``cool_tab`` tab_type'''
    def __init__(self, tab_type, parent=None):
        super(CoolTab, self).__init__(tab_type, parent=parent)
        self.use_cache = True  # cache the tabs when removing them
        # self.use_cache = False  # do not cache the tabs
        self.title = 'A nice name'  # name to associate with the tab
        # any extra code to run at instantiation time

    def setup(self, arguments, keyword_arguments):
        '''This method use the input arguments to setup a tab that can be
        used. When using the cache system, this method can be used to
        re-set the content of the tab'''
        # implement as needed

    # implement all the needed method/slots/etc

Once the package is implemented (and of course tested), it can be be installed with pip install. This way the plugin becomes available to VDAT and it’s possible to use it in the same way of the built-in tab types:

tabs:
    - tab_type: cool_tab
      [all the options for the cool_tab type]

The plugin anatomy

In this section we illustrate the plugin mechanism and the interface of the tabs.

• When selecting a directory or reduction step, the old tabs are removed from the GUI (see below for more details).

• The tabs section for the given directory type and step is retrieved from the configuration file and the code iterates through the items.

• For each item, get the tab_type and load the entry point(s) belonging to the vdat.tab_types group and with the name given in the aforementioned entry. Ideally there should be only one entry point per name, but identical names provided by different packages are not forbidden. In the future we will provide some tool to check existing plugin names (see issue #1613).

• For each entry point, load it. It should provide a function with a signature compatible with vdat.gui.tabs.interface.plugin_interface(). The input, output and what is should do are:

  • Input: target_dir, i.e. the directory selected by the user. tab_dict: the dictionary with the instructions to build the tab(s); except for the tab_type entry, all the remaining entries are specific for the plugin. step_name: name of the selected step. cache: a FplaneCache instance; see The tabs cache for more info. parent_widget the widget to which the tabs belong.
  • Implementation: each function can create zero or more widgets, as necessary. Here are the main steps and some hints for the implementation.
    • Retrieve from the cache, using the cache.from_cache method, or create a widget. We strongly suggest deriving the widget from FplaneTabTemplate or a child thereof [1]. See the FplaneTabTemplate documentation about the widget interface.
    • Setup the widget for use.
    • If creating multiple widgets, store or yield them.
    • target_dir is the absolute path to the selected directory. Information about the directory content can be obtained from the VDATDir and VDATExposures SQL tables. For info about querying the tables see the peewee documentation or the examples in the code.
  • Output: a list containing the widget(s) to add to the GUI as tabs

• For each widget in the output list:

  • The following signals <-> slots are connected

    Connections between custom signals and/or slots

    Signal	Slot	Explanation
    sig_ifuToggled	ifuToggled()	The signal should be emitted when a user select or deselect and IFU from the widget. VDAT reacts storing the selected IFUs. They are then used when running commands or switching step/directory.
    sig_ifuSelected	ifuSelected()	The signal is emitted multiple times from VDAT when switching steps/directory. Tabs can reimplement the slot to react to this, e.g. pre-selecting IFUs.
    sig_selectAllIFUs	selectAllIFUs()	The signal is emitted when all IFUs are selected at once, e.g. from a menu entry. Tabs should reimplement the slot to react to this, e.g. to show that all IFUs has being selected
    sig_deselectAllIFUs	deselectAllIFUs()	The signal is emitted when all IFUs are deselected at once, e.g. from a menu entry. Tabs should reimplement the slot to react to this, e.g. to show that all IFUs has being deselected

  • Add the widget to the GUI as a tab. The title for the tab is taken from the widget attribute title, that must be present.

  • If the widget has a tool_tip attribute, its value is used as tooltip when hovering over the tab name. This could be used to use more information or complete tab names in case of truncations.

• If error happens between loading the plugin and adding the tab widgets to the GUI, it is reported to the user via error boxes. If no tab can be shown, because of missing configurations or error, an overlay with the motivation will be shown.

• Going back to the beginning of the list: when a new directory or step is selected the tabs shown in the GUI are removed. For each widget:

  • all the signals shown in the previous table are disconnected;
  • call the cleanup method; the default implementation hides the widget; override it for more complex behaviour;
  • if the use_cache attribute of the widget is set to True, it is cached under the tab type name, that is saved in the tab_type attribute; widget must implement the use_cache attribute;
  • if the use_cache attribute of the widget is set to False, mark the widget for deletion.

The tabs cache

When an object representing a tab is popped, the code checks if the use_cache attribute set to True. In this case the tab is stored into a FplaneCache instance under the tab_type name.

The same instance is passed to every tab type function. This way, the function can use the from_cache() to retrieve an existing widget, if available in the cache, instead of creating a new one. If the cache for the given tab_type is empty, a None is returned and the function should take care of creating a new instance. If an object is returned, it is responsibility of the plugin modify its content according to the context.

Note

Currently the cache doesn’t have a size limits, so there could be memory leaks due to object put there but never reused. In general, we suggest to try to retrieve from the cache as in the example above, independently of the value of use_cache: this way if the cache needs to enabled or disable for some reasong, the only change to do is the value of the attribute.

The cache has been introduced to avoid memory leaks due to objects not correctly released when deleting them. If a tab types leads to memory leaks, using the cache might help.

Reuse the base classes

Since most of the tabs provided by VDAT share the same basic layout and functionality, we have created some base class implementing this. Here we show how it is possible to extend those classes to create new tab widgets. The classes are ifu_widget.BaseIFUWidget and tab_widget.BaseFplanePanel.

These two classes, together with the tab_widget.UpdateIFUTask thread, are designed together to provide a working, although not very useful, widget.

Here we will sketch some idea on how to extend the base classes and the guide line that should be followed.

Names

Let’s start with defining the new classes and entry point. We will provide the implementation extra pieces as we proceed.

class NewIFUWidget(BaseIFUWidget):
    pass

class NewFplanePanel(BaseFplanePanel):
    @property
    def ifu_widget_class:
        'we want to use the ``NewIFUWidget`` class'
        return NewIFUWidget

def new_tab(target_dir, tab_dict, step_name, cache, parent_widget):
    pass

Setup the classes

The base classes are very basic. Typically we want to pass to the fplane and the ifu widget some more information about what needs to be displayed. Let’s say that the target_dir and the tab_dict contain all the needed information. Some of those information need then to be pushed into the IFU widget. So we can do something like the following:

class NewIFUWidget(BaseIFUWidget):
    def setup(self, target_dir, file_template, cols, rows):
        '''Save the template of the file to save and the list of columns
        and rows. These values are used to create a ``len(cols) *
        len(rows)`` images that will be stitched and shown in the IFU.'''
        self.target_dir = target_dir
        self.file_template = file_template
        self.cols = cols
        self.rows = rows
        ## prepare all the sizes needed to deal with len(cols) and len(rows)
        ## images

class NewFplanePanel(BaseFplanePanel):
    def setup(self, target_dir, tab_dict):
        '''Get the selected directory and the configuration dictionary,
        manipulate the relevant entries and pass the to all the IFU widget.
        '''
        # from the target_dir extract the type of the file shown using
        # the database.
        file_type = query_database(target_dir)

        # set the title and the tooltip for the current tab
        self.title = tab_dict['title'].format(ftype=file_type)
        self.tool_tip = self.title
        file_template = tab_dict['fname'].format(ftype=file_type)

        for ifu in self.fplane.ifus:
            ## replace the {ftype} and the {ifuslot} in the file template
            ifu.setup(target_dir, file_template, tab_dict.get('cols', ['*', ]),
                      tab_dict.get('rows', ['*', ]))

With this we can now rewrite the entry point function:

def new_tab(target_dir, tab_dict, cache, parent_widget):
    '''Implement the ``new_tab`` tab type. It can be configured via the
    following entries in the configuration file:

    * fname (mandatory): template used to construct the name of the file(s)
      to show. Can expand the following placeholders:

      * {ftype}: type of the files in the directory
      * {ifuslot}: slot ID
      * {col}: replaced with the elements from ``cols``
      * {row}: replaced with the elements from ``rows``

      Once all the placeholders has been filled, the resulting name is
      joined with the ``target_dir`` and then the first file matching it is
      take. The matching is done via :func:`glob.glob`.
    * title (mandatory): title of the tab. Can expand the {ftype}
      placeholder
    * cols, rows (optional): when creating the image to show in the IFU,
      loop through the cols and rows, for each one replace the {col} and
      {row} placeholder in the file template and place the corresponding
      file in the correct position in the IFU. They both default to
      ``['*']``
    '''
    tab_type = tab_dict['tab_type']
    # try to get one object from the cache.
    tab_obj = cache.from_cache(tab_type)
    if not tab_obj:
        tab_obj = NewFplanePanel(tab_type)
    # reset the parent. It is not strictly necessary but, hey, better safe
    # that sorrow
    tab_obj.setParent(parent_widget)
    # pass all the relevant pieces to the object to be able to show the target
    # directory
    tab_obj.setup(target_dir, tab_dict)

    return [tab_obj, ]

A note on showing the tab

The tabs defined by tab_widget.BaseFplanePanel and derivatives are drawn only when showing them. We do this because some of the tabs require time and/or CPU consuming tasks to create what need to be shown in the IFUs, so it is possible to show many tabs and then fill only the ones that we actually want to see.

New one widget is shown, Qt checks that it implements the showEvent() and calls it. So we implement tab_widget.BaseFplanePanel.showEvent() and, if the tab_widget.BaseFplanePanel.initialized is set to False, build the table content and set it to True. Successive show events will do nothing. When the tab is removed and tab_widget.BaseFplanePanel.cleanup() is called, the tab widget is reset to the non initialized status. If derived classes reimplement this method, they should take care of calling the parent class one and also to make sure that the IFU widgets cleanup method clears the the images shown.

Create the images to show

Now we need to implement the functionalities to actually create and show the images in the IFUs. We trigger it with the tab_widget.BaseFplanePanel.update_ifus() method. We don’t have to override it, however to correctly build the IFUs we need to reimplement the ifu_widget.BaseIFUWidget.prepare_image() method to build the image to display.

class NewIFUWidget(BaseIFUWidget):
    [...]
    def prepare_image(self):
        '''Prepare the image to be shown in the GUI'''
        # first call the parent method to be sure that the IFU shows
        # something
        super(NewIFUWidget, self).prepare_image()
        image = QtGui.QImage(self.size, self.size,
                             QtGui.QImage.Format_RGB16)
        image.fill(QtCore.Qt.black)
        p = QtGui.QPainter()
        p.begin(img)
        has_file = False
        for (i, col), (j, row) in it.product(enumerate(cols),
                                             enumerate(rows)):
            fname = self.file_template(ifuslot=self.ifuslot,
                                       col=col, row=row)
            fname = os.path.join(self.target_dir, fname)
            try:
                fname = glob.glob(fname)[0]  # get the first file name
                has_file = True
            except IndexError:  # no matching file found
                continue
            # if the files are big you might want to make thumbnails
            # and show them. We are not going to show how to implement such
            # a function here
            thumbnail = self.get_thumbnail(fname)
            ## add the thumbnail to the image in position (i, j)
        p.end()
        # if at least one file was found, it is worth showing it
        # otherwise the default empty image will be created.
        if has_file:
            self.current_img = image

After the prepare_image method returns, the showing is triggered automatically.

Open a window on double click

The ifu_widget.BaseIFUWidget has a handler for dealing with double clicks, but it doesn’t do anything beside making sure that a double click won’t trigger two selections of the IFU. However you might want to use it to do more, like opening a window to better inspect the data.

class NewIFUWidget(BaseIFUWidget):
    [...]
    def mouseDoubleClickEvent(self, event):
        'open a window on double click'
        # make sure to call the parent class implementation to get the
        # single/double click distinction
        super(NewIFUWidget, self).mouseDoubleClickEvent(event)

        # prepare a window, pass it the file names to show and whatever
        # else is necessary.
        window = NewWindow(...)
        window.show()

If the window need some configuration option, pass the configuration dictionary to the NewIFUWidget.setup() method and store it.

Extras

Of course you might want to have more complex behaviours. To have ideas and examples give a look at the current implementation. We might extend this part in the future.

Have fun coding.

Footnotes

[1]	FplaneTabTemplate implements the the minimal interface needed by the plugin system. It provides the signals and do-nothing implementations of the slots that are connected with the rest of the GUI. Also will make easier to keep plugins up to date in future backward-compatible API changes.