Home

QCV

Introduction

QCV is a set of libraries that provide tools for the development of computer vision applications. These tools include a 2D visualization window, an on-line parameter editor and handling, clock handling and an image sequence control tool.

There are two main concepts associated with the QCV framework: operator and drawing list. An operator receives an input, process the data and provides an output. It can also show visual data or text by filling a list with drawing elements such as images, lines or polygons. Such a list is called a drawing list.

Other three main characteristics of this framework is the clock handling, on-line parameter editor and handling, and an image sequencer interface to handle sequence of images. The following subsections address each component in detail.

Operators

QCV introduces the concept of operator. An operator is ab object that performs a specific task of an algorithm (e.g., computing derivative filters, hough transform, or image scaling). Operators can have one parent and multiple children forming this way a tree structure.

The output of an operator is provided to its parent. An operator must also be able to provide the input to its children. Thus, an algorithm can be decomposed by a set of operations in a hierarchical structure.

Operators in QCV are classes derived from the class COperator. The operator subclass can reimplement the cycle(), initialize(), show(), and exit() methods to perform the corresponding desired operation.

The QCV framework communicates with the root operator, calling a set of convenience virtual functions such as cycle(), reset(), initialize(), show(), and exit(). The root operator can call recursively the same virtual function for each of its children (by explicitly calling to COperator::cycle()). Each children can do the same, reaching this way all nodes of the tree.

QCV also communicates to the operators calling virtual functions such as keyPressed() and mouseMove(), so that the operator can handle user input as well.

There are a few operator already implemented for user reference in the modules/operators directory. The stereoExample application demonstrates how to instantiate and use an operator in the framework.

Input and Output of Operators

Operators get their inputs by calling the template function * Type * getInput<type>(std::string f_id_str) *, where * Type * is the type of the object that the operator requires as input and * f_id_str * is string used as identifier. If found, the return value is a pointer to the object, or NULL otherwise. The search for the object is performed by first checking if the required object is an output of the current operator. If not found, the same search operation is performed on the parent operator up to reaching the root operator. If the desired object was not found, NULL is returned.</type>

The output of an operator is provided to the system by calling the template function void * registerOutput<type>(std::string f_id_str, Type * f_obj_p)</type>, where Type is the type of the output object and * f_id_str * its corresponding desired identifier. Once the output has been defined, it will be visible to its parent operator and to the all operator descendants.

There are many similar functions defined in the COperator class, such as const Type &getInput ( const std::string f_id_str, const T_ &f_default ). In this version, the function will return a reference to the object, if found, or f_default otherwise.

The root operator gets its input from the system, as specified by the instantiated CSeqDeviceControler subclass (as specified in the main.cpp file).

With this input/output system, operators can communicate, obtaining inputs from its ancestors and providing output to its parent and children operators.

Drawing Lists

A drawing list is the instantiation of the class CDrawingLists. The CDrawingList class provides an interface with a large number of functions to add drawing elements in it. A drawing list has a name, a current line color, and fill color. It also has a screen location and a visibility flag.

An operator can instantiate as many drawing lists as required. Drawing lists will be automatically displayed by the 2D visualization tool if the visibility flag is set to true. The programmer can specify a default location or a fix location for the drawing list. The user can also change the position of the drawing list in the screen, and hide or show it at her/his own convenience.

The set of drawing lists in the system are displayed in a window in a tree structure. The parent of the drawing lists are the operators (which can also have other operators as parents). The user can drag elements from this list and drop them to the to the 2D visualization window, changing this way the position where they are displayed. Snapshots of the drawing list is also available if the user positions the mouse on a drawing list element a couple of seconds.

The elements of the drawing list are displayed using OpenGL, so the visualization of elements is really fast.

2D Visualization Tool

The visualization tool divides the display space into an array of screens. A screen is a rectangular region where data can be displayed. Drawing are displayed at their corresponding screen locations. The number of screen can be specified by the programmer or by the user using the corresponding user interface. The drawing lists can be drawn outside the boundaries of its assigned screen. It is up to the user to decide if a drawing list should surpass its screen area.

The user can zoom-in and move the zoomed-in area with the mouse and keyboard. The user has the option to render the current view into an image or SVG file, allowing the easy creation of videos.

The 2D visualization tool also provides information about size and cursor location. The input of the user on this window, such as key press and mouse clicks are passed to the operators so that each operator can define its own interaction with the user.

This window also provides the help of the system, which can be accessed by pressing the 'H' key.

Parameter Editor

A powerful feature of the QCV framework is the parameter editor. Each operator can specify the parameters that wants to make available to the user. In concrete, this means to define a set and get functions for the parameter in the operator class, and instantiate a new parameter object class that give access to the set and get methods.

A few macros have been defined to provide the operator's programmer a consistent and concise way of defining the get and set interface methods, as well as the instantiation of the parameter object.

The parameter editor window makes available the operator's parameters with an appropriate way of accessing it (i.e. input box if number or text, check box if boolean and so on). If the user changes the value of a parameter by using the GUI, the corresponding set function in the operator will be called, changing the parameter.

This feature allows an user to evaluate on-the-fly the effect that parameters have on the algorithms. So, question such as: which is the effect of window size has on a gaussian filter, can be realise an evaluated in a matter of few clicks.

Parameters can also be saved and read from an xml file. Following the same hierarchical structure of the operators, the parameter set for the whole tree (or subtree) can be stored and recover at once.

Clock Handling

The programmer can as well define clocks to measure the computation time of the algorithms. Default clocks for the operators are already handled directly by the framework. Clocks are available for the method cycle, initialize, and show. The programmer can add additional clocks by calling two simple functions available through the parent class COperatorBase: startClock ( "name" ) and stopClock ( "name" ).

All started clocks are displayed in a tree structure in the clock window, showing details about the number of cycles, total time and average time. The clocks can all be reset by the user using or be printed into the terminal by clicking on the corresponding buttons.

Sequencer

Finally, the framework offers a sequencer interface to handle sequence of images (call it image sequence, video file, or video streaming). The class interface allows the programmer to define the type of device (i.e. bidirectional such as in a video file or forward only such as in video streaming) and the way to provide the images (and whatever data want to be made available to the operators).

The sequencer classes provide a sequence control window to allow the user to play, pause, advance and stop sequences of images, and thus analyze the computer vision algorithm over time.

An example of a hard disk sequence device is already made available in the current release and we are planning to add live streaming from a webcam very soon. The stereoExample application shows how to instantiate and use this device.

Examples

The library is provided with multiple examples to demonstrate the power of the provided tools and the simplicity of use. Examples for C and C++ are provided. C examples only use the basic core library, for which only the visualization tool and clocks are available. The full framework can be tested by the C++ libraries, which additionaly includes the operator concept, the sequencer, and the parameter editor.

Examples go from the very typical hello world example, to more complex ones just as Sobel, surf, and stereo computation.

Additional Notes

Peformance:

The framework is ready for real-time processing. There is no much overhead, more than a few updates in the QT windows and the OpenGL rendering. In order to increase frame rate, it is advisable to reduce the number of images to show, since copying the image data to the graphics card is time consuming. It is also advisable to only draw elements into a drawing list if the list is currently being displayed (so add only the drawing elements if CDrawingList::isVisible() is true).

Project Admins:

• NaN