This document discusses Pixeye, a software tool that allows for rapid development, evaluation, and configuration of image processing pipelines. Pixeye provides an integrated environment for image processing, algorithm development, mass execution and configuration, image quality assessment, and camera module integration. It aims to increase productivity by standardizing development and providing a one-stop shop for tasks typically requiring separate tools. Pixeye can load, configure, and execute image processing blocks and supports integrating new custom blocks. It provides tools for image analysis, comparison, and algorithm execution and configuration.

1. Dec 8th, 2012

2.  Introduction
 What is the problem?
 Pixeye goal
 Pixeye methodology
 Pixeye feature set

3.  Pixeye offers an integrated suite for
◦ Image processing
◦ Algorithm development, mass execution and configuration
◦ Image quality assessment
◦ Camera module and external component integration
 Developed with algorithms developers and IQ engineers to
allow rapid development, evaluation and configuration of
image pipeline blocks.
 Pixeye usage model has a proven success in accelerating
development and integration processes, increasing
collaboration between developers and IQ engineers.

5.  Usually algorithm engineers develop blocks in Matlab and
sometimes convert it to compiled language (e.g., C, C++ etc.)
to decrease the run-time, and enable integration.
 Many times, the algorithms require some configuration to
control its behavior. Usually, the algorithms’ configuration
turns to be complex, require scalability and visibility for
additional post-development tasks such as IQ, integration
and configuration management.
 Algorithm developers usually build a GUI to allow the IQ
engineers to verify the block, comment on it and later to
configure it.
 Writing a GUI in Matlab is easy, but it’s a maintenance
nightmare when it comes to adding features.

6.  Algorithm, SW and IQC Team Flow
◦ Image capture using external camera module.
◦ Image extraction from an external stream.
◦ Building an algorithms chain (Blocks pipe).
◦ Algorithms blocks configuration.
◦ Execution.
◦ Intermediate results/Outputs analysis and evaluation.
◦ Outputs comparison.
Algorithm
IQC
SW

7. Can it be
Combined
Into One
Tool…?
Combining several
tools to see the
behavior of a
complete image
pipeline can also
be challenging

8.  In addition, as part of the algorithm development and the IQ
work, images created in the process are frequently compared
to reference images which requires saving the images after
each run and using other external tools (e.g. FastStone.).

10. One stop shop tool for
development, evaluation, configuration and
demonstration
 Increase productivity of algorithm and IQ engineers – decrease the
invested time on ad-hoc tools/GUIs/configuration tools.
 Standardize development of blocks and tools.
 Standardize a typed output of intermediate results and maps of any
execution block to assist developers and IQ engineers.
 Encapsulating all image quality activity inside a single tool, without
the need to use other tools for routine work (e.g., image
comparison).
 To allow using the same tool for demonstration purposes with the
options to hide or expose to the customers certain parameters of
the configuration.
Our Goal

11.  Pixeye can load, configure and execute any block which
accepts Bayer/RGB/Gray image and outputs a Bayer/RGB/Gray
images.
 Pixeye has a well defined API for custom blocks insertion,
based on XML and a managed .NET wrapper.
 Each block declares its configuration through an XML.
 The XML uses simple schema to declare its components, and
based on it, Pixeye renders a specific GUI for this block.
 Pixeye dynamically detects the available blocks on
initialization and allows the user to declare image pipelines
chains using it’s arsenal of blocks.
 Pixeye provide session context and workspace per user under
a project entity, allowing custom preferences, and
configuration management.

13.  Algorithms processing environment
◦ Manages algorithms in user projects.
◦ Dynamically constructs image pipeline processing blocks
into chains
◦ Runs multiple algorithms chains simultaneously.
◦ The framework is agnostic to algorithms implementation
and internal configuration.
◦  Can learn configuration and present it ‘on the fly’.
◦ Supports different pixel orders, image size, masks, header
lines etc.
◦ Controls images and blocks execution.
◦ Auto saves user environment.

14.  Processing blocks configuration
◦ Loads metadata of native processing blocks
◦ Presents various controls of different configuration types.
◦ Have rich UI to calibrate processing blocks.
◦ Load/Save blocks configuration.
◦ Supports intermediate outputs per algorithm
◦ Provides a way to configure the parameters of each
processing block under any chain.
 Algorithms execution
◦ Multithreaded execution.
◦ Partial image processing according to cropped area
selection on different source images.
◦ Batch execution.

15.  Images workspace
◦ Image quality analysis tools.
◦ Displays intermediate results.
◦ Loads reference images for benchmarking.
◦ Allows comparison between images.
◦ Lock multiple images to share same ROI and zoom settings.
◦ Analyze image in full screen mode.
◦ Full crop/pan/zoom capabilities

17. RAW
BAYER
BPC
Noise
Reduction
LSC
GrGb
Statistics
Gains & Offsets
New BlockDemosaicing
CCM
Gamma
Lateral Color
Scaling
RGB2YUV
SharpeningNew Algorithm
block can be
integrated
easily anywhere
inside the
image pipeline

25. KJS
LFD
Nominal

28.  Pixeye comes with a plugin which allow integration
with sensor.
 The current code supports SCOOBY2
interface, where all the sensor configuration is
flexible and loaded from XML.
 Other custom interfaces can be added on demand.
Scooby2 Platform for
Image Sensor Analysis

30.  Currently the tool is supplied with the
following ISP blocks:
◦ White balance
◦ CCM
◦ Gamma correction
◦ Black level substraction
◦ Simple Demosaic
◦ Advanced Demosaic

32. •SW installation package
•Support C++/C Algorithms
•Managed .NET API
•XML Reference
•C++/C Structures auto generation
•Bayer/RGB/Gray full support
•Documentations (e.g. User
manual, integration etc.))
Basic
•Algorithms native language
(Matlab/Lua/Java/R/etc..)
•Algorithm input/output types.
•Maps/Intermediate results
•Custom IQ analysis tools
•Images formats
•Platform related requirements
•External components/Camera modules
Definition
•Native algorithm wrapping (Managed layer)
•Supporting custom types.
•Integrating new maps formats.
•Adding custom tools
•Writing plug-ins for external tools
Integration