fiat_image: advanced image widget#
Fiatlight provides an advanced image viewer and analyzer which enables to zoom, pan, look at pixel values and sync the zoom across images.
Example#
from fiatlight.fiat_kits.fiat_image import fiat_image_attrs_demo
fiat_image_attrs_demo.main()
In the “show_image” output, the options panel was opened
The “show_image_channels” output shows the image channels, and it zoom/pan is linked to “show_image”
The “show_image_different_zoom_key” image has a different zoom key, and the zoom/pan is not linked to “show_image”. It also zoomed at a high-level, so that pixel values are displayed.
the “show_image_only_display” image is displayed, and cannot be zoomed or panned (the widget may be resized however)
Fiat attributes available for the ImageWithGui widget#
The image widget provided with fiat_image is extremely customizable. Here is a list of all the possible customizations options:
%%bash
fiatlight gui ImageWithGui
GUI type: ImageWithGui
=======================
A highly sophisticated GUI for displaying and analysing images. Zoom/Pan, show channels, show pixel values, sync zoom accross images, etc.
Available custom attributes for fiat_image.ImageWithGui:
--------------------------------------------------------------------------------
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| Name | Type | Default | Explanation |
+=================================+=================+===========+===============================================+
| | | | **Main attributes for the image viewer** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| only_display | bool | False | Only display the image, no info displayed, no |
| | | | zoom, no pan |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| image_display_size | tuple[int, int] | (200, 0) | Initial size of the displayed image (width, |
| | | | height). One of them can be 0 |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| zoom_key | str | z | Key to zoom in the image. All images with the |
| | | | same zoom key will be zoomed together |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| is_color_order_bgr | bool | True | Color order is BGR (default is True). OpenCV |
| | | | uses BGR by default, unfortunately. |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| can_resize | bool | True | Can resize the image by dragging the mouse at |
| | | | the bottom right corner |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| | | | **Channels** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_channels | bool | False | Show channels |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| channel_layout_vertically | bool | False | Layout channels vertically |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| | | | **Zoom & Pan** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| pan_with_mouse | bool | True | Pan with mouse |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| zoom_with_mouse_wheel | bool | True | Zoom with mouse wheel |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| | | | **Info displayed on image** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_school_paper_background | bool | True | Show school paper background, when the image |
| | | | is unzoomed |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_alpha_channel_checkerboard | bool | True | Show alpha channel checkerboard |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_grid | bool | True | Show grid with the zoom level is high |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| draw_values_on_zoomed_pixels | bool | True | Draw values on pixels, when the zoom is high |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| | | | **Info displayed under the image** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_image_info | bool | True | Show image info, i.e image size and type |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_pixel_info | bool | True | Show pixel info, i.e. show pixel value and |
| | | | position under the mouse |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| | | | **Control buttons under the image** |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_zoom_buttons | bool | True | Show zoom buttons |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_options_panel | bool | True | Show options panel |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_options_button | bool | True | Show options button |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
| show_inspect_button | bool | True | Show the inspect button, that enables to open |
| | | | a large version of image in the Image |
| | | | Inspector |
+---------------------------------+-----------------+-----------+-----------------------------------------------+
Available custom attributes for AnyDataWithGui Generic attributes:
--------------------------------------------------------------------------------
+----------------+--------+---------------------+------------------------------------------------+
| Name | Type | Default | Explanation |
+================+========+=====================+================================================+
| | | | **Generic attributes** |
+----------------+--------+---------------------+------------------------------------------------+
| validate_value | object | None | Function to validate a parameter value (should |
| | | | return DataValidationResult.ok() .error() |
+----------------+--------+---------------------+------------------------------------------------+
| label | str | | A label for the parameter. If empty, the |
| | | | function parameter name is used |
+----------------+--------+---------------------+------------------------------------------------+
| tooltip | str | | An optional tooltip to be displayed |
+----------------+--------+---------------------+------------------------------------------------+
| label_color | ImVec4 | ImVec4(0.000000, | The color of the label (will use the default |
| | | 0.000000, 0.000000, | text color if not provided) |
| | | 1.000000) | |
+----------------+--------+---------------------+------------------------------------------------+
Code to test this GUI type:
----------------------------
```python
import typing
import fiatlight
@fiatlight.with_fiat_attributes(
# Main attributes for the image viewer
union_param__only_display = False,
union_param__image_display_size = (200, 0),
union_param__zoom_key = "z",
union_param__is_color_order_bgr = True,
union_param__can_resize = True,
# Channels
union_param__show_channels = False,
union_param__channel_layout_vertically = False,
# Zoom & Pan
union_param__pan_with_mouse = True,
union_param__zoom_with_mouse_wheel = True,
# Info displayed on image
union_param__show_school_paper_background = True,
union_param__show_alpha_channel_checkerboard = True,
union_param__show_grid = True,
union_param__draw_values_on_zoomed_pixels = True,
# Info displayed under the image
union_param__show_image_info = True,
union_param__show_pixel_info = True,
# Control buttons under the image
union_param__show_zoom_buttons = True,
union_param__show_options_panel = True,
union_param__show_options_button = True,
union_param__show_inspect_button = True,
# Generic attributes
union_param__validate_value = None,
union_param__label = "",
union_param__tooltip = "",
union_param__label_color = ImVec4(0.000000, 0.000000, 0.000000, 1.000000))
def f(union_param: typing.Union[fiatlight.fiat_kits.fiat_image.image_types.ImageU8_1, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_2, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_3, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_4, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_RGB, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_RGBA, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_BGRA, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_BGR, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_GRAY, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_1, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_2, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_3, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_4]) -> typing.Union[fiatlight.fiat_kits.fiat_image.image_types.ImageU8_1, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_2, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_3, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_4, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_RGB, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_RGBA, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_BGRA, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_BGR, fiatlight.fiat_kits.fiat_image.image_types.ImageU8_GRAY, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_1, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_2, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_3, fiatlight.fiat_kits.fiat_image.image_types.ImageFloat_4]:
return union_param
fiatlight.run(f)
```
Image types#
Fiatlight provides several synonyms for Numpy arrays that denote different types of images. Each of these types will be displayed by the image widget.
import fiatlight
from fiatlight.fiat_notebook import look_at_code
%look_at_python_file fiat_kits/fiat_image/image_types.py
"""This module defines several types you can use to annotate your functions.
The image types are defined as NewType instances, which are just aliases for numpy arrays.
All those types will be displayed in the GUI as images, using the ImmVision image viewer
(https://github.com/pthom/immvision)
Notes:
- The easiest way to display an image is to use the `Image` type, which is a union of all image types,
or to use the `ImageU8` type, which is a union of all UInt8 image types.
- any numpy array can be used to create an `Image`, and the viewer will try to display it
"""
from typing import Any, NewType
import numpy as np
from typing import Tuple, Union
# Define shape types for clarity
ShapeHeightWidth = Tuple[int, int]
ShapeHeightWidthChannels = Tuple[int, int, int]
# Define UInt8 as a dtype for numpy arrays
UInt8 = np.dtype[np.uint8]
AnyFloat = np.dtype[np.floating[Any]]
#
# UInt8 Images
#
# ImageU8 = NewType("ImageU8", np.ndarray[ShapeHeightWidthChannels | ShapeHeightWidth, UInt8])
# Type definitions for UInt8 images based on channel count
ImageU8_1 = NewType("ImageU8_1", np.ndarray[ShapeHeightWidth, UInt8])
ImageU8_2 = NewType("ImageU8_2", np.ndarray[ShapeHeightWidthChannels, UInt8])
ImageU8_3 = NewType("ImageU8_3", np.ndarray[ShapeHeightWidthChannels, UInt8])
ImageU8_4 = NewType("ImageU8_4", np.ndarray[ShapeHeightWidthChannels, UInt8])
ImageU8_WithNbChannels = Union[ImageU8_1, ImageU8_2, ImageU8_3, ImageU8_4]
# Type definitions based on the roles of the channels
ImageU8_RGB = NewType("ImageU8_RGB", ImageU8_3)
ImageU8_RGBA = NewType("ImageU8_RGBA", ImageU8_4)
ImageU8_BGRA = NewType("ImageU8_BGRA", ImageU8_4)
ImageU8_BGR = NewType("ImageU8_BGR", ImageU8_3)
ImageU8_GRAY = NewType("ImageU8_GRAY", ImageU8_1)
ImageU8_WithChannelsRoles = Union[ImageU8_RGB, ImageU8_RGBA, ImageU8_BGRA, ImageU8_BGR, ImageU8_GRAY]
# Generic type for any 8-bit image
ImageU8 = Union[ImageU8_WithNbChannels, ImageU8_WithChannelsRoles]
#
# Float Images
#
# Type definitions for float images based on channel count
ImageFloat_1 = NewType("ImageFloat_1", np.ndarray[ShapeHeightWidth, AnyFloat])
ImageFloat_2 = NewType("ImageFloat_2", np.ndarray[ShapeHeightWidthChannels, AnyFloat])
ImageFloat_3 = NewType("ImageFloat_3", np.ndarray[ShapeHeightWidthChannels, AnyFloat])
ImageFloat_4 = NewType("ImageFloat_4", np.ndarray[ShapeHeightWidthChannels, AnyFloat])
# Generic type for any float image
ImageFloat = Union[ImageFloat_1, ImageFloat_2, ImageFloat_3, ImageFloat_4]
#
# Generic Image Type
#
# Image is a union of all image types
Image = Union[ImageU8, ImageFloat]
# ---------------------------- Register image type factories ----------------------------
def _register_image_type_factories() -> None:
from fiatlight.fiat_togui.gui_registry import gui_factories
from fiatlight.fiat_kits.fiat_image.image_gui import ImageWithGui
prefix = "fiatlight.fiat_kits.fiat_image.image_types.Image"
gui_factories().register_factory_name_start_with(prefix, ImageWithGui)
gui_factories().register_factory_union(prefix, ImageWithGui)
`
Source code for the example#
%look_at_python_file fiat_kits/fiat_image/fiat_image_attrs_demo.py
"""Demo how to set custom presentation attributes for the Image Widget (ImageWithGui)
Notes:
- The custom attributes can be set using the decorator fl.with_fiat_attributes
- In these examples, we intend to set custom attributes for the output of the
functions, i.e. the returned value.
As a consequence, the custom attributes are set in the return__...
arguments of the decorator.
"""
import fiatlight as fl
from fiatlight.fiat_kits.fiat_image import ImageU8_3
import cv2
# Our demo image
demo_image: ImageU8_3 = cv2.imread(fl.demo_assets_dir() + "/images/house.jpg") # type: ignore
# A simple function that will use the Image Widget with its default settings.
def show_image(image: ImageU8_3 = demo_image) -> ImageU8_3:
return image
# A function whose output will initially show the channels
# Since it does not specify a zoom key,
# it will be zoomed and panned together with the image
# shown by "show_image"
@fl.with_fiat_attributes(return__show_channels=True)
def show_image_channels(image: ImageU8_3 = demo_image) -> ImageU8_3:
return image
# A function whose output will have a different zoom key:
# it can be panned and zoomed, independently of the other images
@fl.with_fiat_attributes(return__zoom_key="other")
def show_image_different_zoom_key(image: ImageU8_3 = demo_image) -> ImageU8_3:
return image
# A function that will use the Image Widget with custom attributes:
# - the image is displayed only (it cannot be zoomed or panned,
# and the pixel values are not shown)
# - the image is displayed with a height of 300 pixels
# (the width is automatically calculated)
# - the image cannot be resized
@fl.with_fiat_attributes(
return__only_display=True,
return__image_display_size=(0, 300),
return__can_resize=False,
)
def show_image_only_display(image: ImageU8_3 = demo_image) -> ImageU8_3:
return image
def main() -> None:
graph = fl.FunctionsGraph()
graph.add_function(show_image)
graph.add_function(show_image_channels)
graph.add_function(show_image_different_zoom_key)
graph.add_function(show_image_only_display)
fl.run(graph, app_name="fiat_image_fiat_attrs_demo")
if __name__ == "__main__":
main()