User Guide
Last update: 11/15/2004
Code release version: 0.7
ARJF © 2001-2004
|
|
User Guide
Last update: 11/15/2004
|
MFSM (Modular Flow Scheduling Middleware) is an architectural middleware implementing the SAI style. Its core element, the FSF library, is a set of extensible classes that can be specialized to define new data structures and processes, or encapsulate existing ones (e.g. from libraries). MFSM is an open source project, released under the GNU Lesser General Public License. A number of software modules regroup specializations implementing specific algorithms or functionalities. They constitute a constantly growing base of open source, reusable code, maintained as part of the MFSM project. The project also includes extensive documentation, including this user guide and a reference guide.
This user guide is an example-based, code-level tutorial on the use of the MFSM architectural middleware for implementing data stream manipulation applications designed in the SAI style.
The first section si an introduction to the SAI architectural style.
The second section is an introduction to the MFSM open source architectural middleware.
The third section introduces the basicsof design and implementation of applications performing data stream manipulation. These principles are illustrated at the code level with simple examples. The first example illustrates application setup and cleanup, application graph elements instantiation and connection. The implementation of a custom cell is illustrated with two examples, one used in the first example, and one used in a slightly more complex example involving shared memory access for incremental computations along the time dimension. The development of specialized nodes is explained with a study of the implementation of a generic image node (in the Image module).
The last section describes generic architectural patterns, used in the design of the open source modules.
The code appearing in this guide was developed by the author, and is part of open source libraries, modules and examples released under the GNU Lesser General Public License. They are available for download on the MFSM web site. This code is provided here in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. Complete license information is provided in the downloadable packages.
SAI (Software Architecture for Immersipresence) is a new software architecture style for designing, analyzing and implementing applications performing distributed, asynchronous parallel processing of generic data streams. The goal of SAI is to provide a universal framework for the distributed implementation of algorithms and their easy integration into complex systems that exhibit desirable software engineering qualities such as efficiency, scalability, extensibility, reusability and interoperability. SAI specifies a new architectural style (components, connectors and constraints). The underlying extensible data model and hybrid (shared repository and message-passing) distributed asynchronous parallel processing model allow natural and efficient manipulation of generic data streams, using existing libraries or native code alike. The modularity of the style facilitates distributed code development, testing, and reuse, as well as fast system design and integration, maintenance and evolution. A graph-based notation for architectural designs allows intuitive system representation at the conceptual and logical levels, while at the same time mapping closely to processes.
This section is an overview of the SAI architectural style, introducing the concepts, vocabulary and notation used in the remainder of this guide.
Graphical symbols are introduced to represent each element type. Together these symbols constitute a graph-based notation system for representing architectural designs. In addition, when available, the following color coding will be used: green for processing, red for persistent data, blue for volatile data. Figure 1 presents a summary of the proposed notation.
Figure 1: Summary of notation for SAI designs. Cells are represented as squares, sources as disks or circles. Source-cell connections are drawn as double or fat lines, while cell-cell connections are drawn as thin arrows crossing over the cells. When color is available, cells are colored in green (reserved for processing); sources, source-cell connections, passive pulses are in colored in red (persistent information); streams and active pulses are colored in blue (volatile information).
In the following, the distinction between an object type and an instance of the type will be made explicitly only when required by the context.
The SAI style defines two types of components: cells and sources. Cells are processing centers. They do not store any state data related to their computations. The cells constitute an extensible set of specialized components that implement specific algorithms. Each specialized cell type is identified by a type name (string), and is logically defined by its input data, its parameters and its output. Cell instances are represented graphically as green squares. A cell can be active or inactive, in which case it is transparent to the system. Sources are shared repository of persistent data. Source instances are represented as red disks or circles. Two types of connectors link cells to cells and cells to sources. Cell to source connectors give the cell access to the source data. Cell to cell connectors define data conduits for the streams. Note that the semantics of these connectors are relaxed compared to that of the pipes of the classical Pipes and Filters architectural style: where pipes are FIFO queues, SAI cell-cell connectors do not convey any constraint on the time ordering of the data flowing through them.
Cell and source instances interact according to the following rules. A cell must be connected to exactly one source, which holds its persistent state data. A source can be connected to an arbitrary number of cell, all of which have concurrent shared memory access to the data held by the source. A source may hold data relevant to one or more of its connected cells, and should hold all the relevant data for each of its connected cells (possibly with some overlap). Cell-source connectors are drawn as either double or fat red lines. They may be drawn across cells (as if cells were connected together by these links) for layout convenience. Volatile data flows in streams, which are defined by cell-to-cell connections. A cell can be connected to exactly one upstream cell, and to an arbitrary number of downstream cells. Streams (and thus cell-cell connections) are drawn as thin blue arrows crossing over the cells.
Each source contains a passive pulse, which encodes the instantaneous state of the data structures held by the source. Volatile data flows in streams, that are temporally quantized into active pulses. Pulses are represented graphically as a root (solid small disk) and a hierarchy of nodes (small circles); passive pulses may be rooted in the circle or disk representing the source.
When an active pulse reaches a cell, it triggers a series of operations that can lead to its processing by the cell (hence the ``active'' qualifier). Processing in a cell may result in the augmentation of the active pulse (input data), and/or update of the passive pulse (process parameters). The processing of active pulses is carried in parallel, as they are received by the cell. Since a cell process can only read the existing data in an active pulse, and never modify it (except for adding new nodes), concurrent read access will not require any special precautions. In the case of passive pulses, however, appropriate locking (e.g. through critical sections) must be implemented to avoid inconsistencies in concurrent shared memory read/write access.
Passive pulses may hold persistent data relevant to several cells. Therefore, before a cell can be activated, the passive pulse must be searched for the relevant persistent data. As data is accumulated in active pulses flowing down the streams through cells, it is also necessary for a cell to search each active pulse for its input data. If the data is not found, or if the cell is not active, the pulse is transmitted, as is, to the connected downstream cells. If the input data is found, then the cell process is triggered. When the processing is complete, then the pulse, which now also contains the output data, is passed downstream.
Searching a pulse for relevant data, called filtering, is an example of run-time data binding. The target data is characterized by its structure: node instances (type and name) and their relationships. The structure is specified as a filter or a composition hierarchy of filters. Note that the term filter is used here in its ``sieving'' sense. Figure 2 illustrates this concept. A filter is an object that specifies a node type, a node name or name pattern and eventual subfilters corresponding to subnodes. The filter composition hierarchy is isomorphic to its target node structure. The filtering operation takes as input a pulse and a filter, and, when successful, returns a handle or hierarchy of handles isomorphic to the filter structure. Each handle is essentially a pointer to the node instance target of the corresponding filter. When relevant, optional names inherited from the filters allow to identify individual handles with respect to their original filters.
Figure 2: Pulse filtering. Each cell is associated with its required volatile and persistent data structures, in the form of substructures called active and passive filters (respectively). Pulses are searched for these structures in an operation called filtering, which results in the creation of handles that can be used during processing for direct access to relevant nodes.
The notation adopted for specifying filters and hierarchies of filters is nested square brackets. Each filter specifies a node type, a node instance name or name pattern (with wildcard characters), an optional handle name, and an eventual list of subfilters, e.g.: [NODE_TYPE_ID ``Node name'' handle_id [...] ... ]. Optional filters are indicated by a star, e.g.: [NODE_TYPE_ID ``Node name'' handle_id]*.
When several targets in a pulse match a filter name pattern, all corresponding handles are created. This allows to design processes whose input (parameters or stream data) number is not fixed. If the root of the active filter specifies a pattern, the specialized process function is invoked for each handle generated by the filtering (sequentially, in the same thread). If the root of the passive filter specifies a pattern, only one passive handle is generated (pointing to the first encountered node satisfying the pattern).
A particular system architecture is specified at the conceptual level by a set of source and cell instances, and their inter-connections. The specialized cells may be accompanied by a description of the task they implement. Source and cell instances may be given names for easy reference. In some cases, important data nodes and outputs may be specified schematically to emphasize some design aspects.
A logical level description of a design requires to specify, for each cell, its active and passive filters and its output structure, and for each source, the structure of its passive pulse. The table below summarizes the notations for logical level cell definition.
| ClassName (ParentClass) | CELL_TYPE_ID |
| Active filter | [NODE_TYPE_ID "Node name" [...] ... ] |
| Passive filter | [NODE_TYPE_ID "Node name" [...] ... ] |
| Output | (NODE_TYPE_ID "default output base name--more if needed" (...) ... ) |
Filters and nodes are described using the nested square brackets and nested parentheses notations introduced above. By convention, in the cell output specification, (x) represents the pulse's root, (.) represents the node corresponding to the root of the active filter, and (..) represents its parent node.
MFSM (Modular Flow Scheduling Middleware) is an open source project, whose goal is to support and promote the design, analysis and implementation of applications in the SAI style. This goal is reflected in the different facets of the project.
Figure 3 shows the overall system architecture suggested by MFSM. The middleware layer provides an abstraction level between low-level services and applications, in the form of SAI software elements. At the core of this layer is the Flow Scheduling Framework (FSF) \cite{FrancoisThesis,FrancoisMed2000}, an extensible set of foundation classes that implement SAI style elements. The generic extensible data model allows to encapsulate existing data formats and standards as well as low-level service protocols and APIs, and make them available in a system where they can interoperate. The hybrid shared repository and message passing communication and parallel processing model supports control, asynchronous concurrent processing and synchronization of data streams. The application layer can host a data-stream processing software system, specified and implemented as instances of SAI components and their relationships.
Figure 3: MFSM system architecture model.
In its current implementation, the FSF library contains a set of C++ classes implementing SAI elements: the source, the nodes and pulses, the cells, the filters, the handles. It also contains classes for two implementation related object types: the factories and the System. A brief overview of each object type is proposed below. The online reference guide provides detailed interface description and implementation notes for all the classes defined in the FSF library.
The source class is fairly simple. It holds a single passive pulse, and keeps a list of pointers to connected cells. Note that connectors are not implemented as objects, but simply lists of pointers in the components. This results from the simplicity of the role played by connectors in the SAI style. In particular, they do not perform any computation. Sources also perform garbage collecting on their passive pulse when notified of a structural change by a connected cell.
The FSF library contains a base node class, that implements connectivity elements for building hierarchical structures. Specialized node classes, encapsulating or implementing specific data structures, will be derived from the base class. In FSF, from The base node class are derived two specific classes for active and passive pulse respectively. These serve as root nodes for active and passive pulse which are node hierarchies. Another node specialization is provided in FSF, that implements nodes holding atomic type values (integer, floating point, boolean, string, etc.). These are called TypeNodes, and are implemented as template instances of a template classe, itself derived from a non template TypeNodeBase class (to allow undetermined pointers to a TypeNode object).
The FSF library contains a base cell class, that implements cell-cell and cell-source connectivity mechanisms (lists of pointers). It also implements all the computation associated with the parallel processing of active pulses. In MFSM, in each cell, each active pulse is processed in a separate thread. Each passive filtering operation is also carried in a separate thread. As a result, some precautions must be taken for concurrent data access. Thanks to inheritance and virtual functions, when implementing a specialized cell derived from the base cell, only the specific computation must be coded, in an overloaded process virtual member funtion. All the events leading to the call to this process function, including filtering, will be automatically performed. A cell can be active or inactive. For efficiency purposes, passive filtering only occurs when the cell is activated (activation fails if passive filtering fails), and when explicitely notified by the source, after a cell notified the source of a structural change in the passive pulse.
Two sets of classes are implemented, for active and passive filters respectively. For each, a template class derived from a non template base class provides support for intantiating filters corresponding to any node type. The non template base class allows to have pointers to undetermined filters (e.g. in cells). The filter objects also implement the filtering functionality. Active filtering instantiates and returns a list of active handles, passive filtering instantiates and returns a list of passive handles.
Two sets of classes are implemented, for active and passive handles respectively. Active handle simply provide a pointer to a node in an active pulse. An optional handle ID, inherited during the filtering operation, allows to identify the node with respect to the filter. Passive handles play the additional role of locks for node deletion and garbage collecting (not required in active pulses). When a cell removes a node, it does not destroy it, but marks it as deleted, so it is not used in subsequent filtering operations. The cell must notify the source, which in turns notify all connected cells to re-filter the passive pulse, and initiates a garbage collecting process, which will physically deleted the node when is it free of handles from ongoing process.
Class factories are used to instantiate objects at run time, whose type is not known at compile time. Since cells and nodes constitute extensible sets of specialized object types derived from respective base classes, Cell and Node factories a necessary for instantiating cells and nodes at run time, and modify application graphs dynamically. Node factories also allow to instantiate filters for the corresponding node type.
By convention, any application using FSF must have a single instance of the System type. The corresponding class therefore implements the Singleton pattern, so that this unique instance is automatically created the first time it is needed. The unique system instance provides the reference clock, and holds the lists of node and cell factories available for building application graphs. In order to use a module in an application, its node and cell factories must be declared and registered with the system instance.
In this section, a simple example console application, called ex1, illustrates all the steps from design to implementation.
Suppose the goal of the program is to generate a stream at a given frame rate, and display the time stamp of each pulse in seconds and in h/m/s formats. Note that since the time stamp in the win32 vesrion of MFSM is the system time provided by MS Windows, the example is in fact a clock showing the time since Windows was last restarted.
FSF contains a fundamental cell used to generate empty pulses at a given frame rate: the Pulsar. Its specifications are as follows:
| fsf::CPulsar (fsf::CCell) | FSF_PULSAR |
| Active filter | (no input) |
| Passive filter | [FSF_INT64_NODE "Pulse delay"] |
| Output | (FSF_ACTIVE_PULSE "Root") |
A pulsar does not have any input. One parameter, of integral type, specifies the time delay between two consecutive output pulses.
In this section, a cell capable of looking up the time stamp of each incoming active pulse and printing it to the console in a given format, is supposed to be available. The actual implementation of this cell is addressed in the next section about custom cells. The cell specifications are as follows:
| myspace::CMyCell (fsf::CCell) | MY_CELL |
| Active filter | [FSF_ACTIVE_PULSE "Root"] |
| Passive filter | [FSF_PASSIVE_PULSE "Root"] |
| Output | (no output) |
It is straightforward to implement our simple application from from one instance of Pulsar and one instance of MyCell. Figure 4 shows the conceptual system graph.
Figure 4: Conceptual system graph for ex1
Two functional units can be identified:
The following subsections explain in detail how to code the simple application in C++: setup the system, build and run the application graph, clean-up the objects allocated. For information about creating a console application in MSVC++, please refer to the corresponding entry in the MFSM F.A.Q.
The unique instance of the system class is automatically created when first needed (Singleton pattern). First and foremost, the nodes and cells defined in each module used must be registered with the system. As a result, although FSF is highly multi-threaded, the unique instance will be allocated before multi-threading kicks in, and thus a double lock mechanism is probably not needed in the implementation of the CSystem class.
Factories can be registered in any order. They are not necessary for pre-coded application graphs, although they are used by the scripting module functions (see below). Factories are necessary for dynamic, run-time application graph building and/or modification, which is out of the scope of this introduction. Each module is required to declare and implement a function called RegisterFactories for registering factories in the system for all nodes and cells implemented in the module.
// register system factories
fsf::RegisterFactories();
// register myspace factories
myspace::RegisterFactories();
A scripting module (namespace scripting) provides shortcut functions to instantiate sources and cells, instantiate nodes and place them in source pulses, connect cells to other cells and to sources. The name ``scripting'' comes from the fact that the functions provided by this module are coding equivalents of user actions in an interactive system. In particular, the scripting module uses aspects of the MFSM implementation that are related to dynamic system evolution, such as class factories. Note that the scripting module itself does not implement any node or cell class and thus does not register any factory (there is no scripting::RegisterFactories).
The code for building an application graph instantiates and connects all the elements of the conceptual graph. In this simple example, the graph can be divided into two functional subgraphs: the pulsing unit, built around the Pulsar instance, and the display unit built around the MyCell instance. Each functional subgraph in this case corresponds to one source and one cell (minimal computing units).
Each minimal unit, consisting of one cell and one source whose pulse contains the cell's parameters, can be coded following these steps:
These principles can be used as guidelines and adapted to code any graph. The following code builds the graph for ex1, first the pulsing unit, then the display unit. Successful instantiation of all graph elements is checked, as failure to register the appropriate factories will result in the failure to instantiate a given cell or node.
// build graph
bool bSuccess=true;
////////////
// Pulsar //
////////////
// create the source
fsf::CSource *pPSource=new fsf::CSource;
bSuccess &= (pPSource!=NULL);
// parameter nodes
fsf::Int64Node *pPulseRate=static_cast<fsf::Int64Node*>(
scripting::CreateNode(std::string("FSF_INT64_NODE"),pPSource->GetPulse()));
bSuccess &= (pPulseRate!=NULL);
if(bSuccess)
{
// set name
pPulseRate->SetName(std::string("Pulse delay"));
// set parameter values
__int64 nPulseDelay=static_cast<__int64>(1000000000.0f/fPulseRate);
pPulseRate->SetData(nPulseDelay);
}
// cell
fsf::CCell *pPcell=static_cast<fsf::CCell*>(
scripting::CreateCell(std::string("FSF_PULSAR")));
bSuccess &= (pPcell!=NULL);
if(bSuccess)
{
// connect with source
scripting::ConnectSource(pPcell,pPSource);
}
/////////////
// My cell //
/////////////
// create the source
fsf::CSource *pPSource=new fsf::CSource;
bSuccess &= (pMySource!=NULL);
// cell
fsf::CCell *pMyCell=static_cast<fsf::CCell*>(
scripting::CreateCell(std::string("MY_CELL")));
bSuccess &= (pMyCell!=NULL);
if(bSuccess)
{
// connect with source
scripting::ConnectSource(pMyCell,pMySource);
// connect with Pcell
scripting::ConnectUpstreamCell(pMyCell,pPcell);
}
// Check everything went OK...
if(bSuccess==false)
{
cout << "Some elements in the graph "
<< "could not be instantiated..." << endl;
return (-1);
}
Once the graph is completed, the cells must be activated. The Pulsar instance is the origin of the active stream and starts generating empty pulses as soon as it is activated. The MyCell instance, once activated, will process incoming pulses in parallel, asynchronously. The cells can be started in any order.
// Run
pMyCell->On();
pPcell->On();
Although this aspect is not evident in this simple example, cells can be turned on and off at any time, elements (sources and cells) can be connected and disconnected at any time, new ones can be created and existing ones destroyed at any time.
The following code stops the cells, disconnects and destroys the different elements instantiated when building the graph.
// Stop the cells
cout << "Stopping the cells..." << endl;
pPcell->Off();
pMyCell->Off();
// Clean up
scripting::DisconnectSource(pPcell);
scripting::DisconnectStream(pPcell);
delete pPcell;
delete pPSource;
scripting::DisconnectSource(pMyCell);
scripting::Disconnectstream(pMyCell);
delete pMyCell;
delete pMySource;
The example implementation allows to specify the pulse rate on the command line (the default rate is 15 Hz). Below is a sample output from the program.
>ex1 1
Pulse: 65079 s = 18 h 4 min 39 s
Pulse: 65080 s = 18 h 4 min 40 s
Pulse: 65081 s = 18 h 4 min 41 s
Pulse: 65082 s = 18 h 4 min 42 s
Pulse: 65083 s = 18 h 4 min 43 s
Pulse: 65084 s = 18 h 4 min 44 s
Pulse: 65085 s = 18 h 4 min 45 s
If one of the goals of MFSM is to allow rapid development of applications from existing modules, one of its main strenghts is to allow easy development of custom elements that will interoperate seamlessly with existing or third party components. The example developed in the previous section, ex1, uses the cell MyCell to look up the time stamp of each incoming active pulse and print it to the console in a given format. This section details how to describe and implement such custom cells.
Any processing cell must be derived from the base Cell (CCell). It is characterized by an identification string (used for to link it to its factory). A complete description includes the active and passive filters, the process output, a list of data members and member functions with a short description.
A custom cell must implement the default constructor, and overload a number of virtual functions which characterize the cell:
The specifications of the cell used in ex1 are as follows:
Print time stamp of incoming pulse to the console in seconds and in h/m/s format. Member functionsConstructors, destructor and other functions that are part of any derived cell class
Active stream processing
|
The following code is the class declaration
class CMyCell : public fsf::CCell
{
public:
CMyCell();
// Factory mapping key
virtual void GetTypeID(std::string &str) { str.assign("MY_CELL"); }
// Specialized processing function
virtual void Process(fsf::CPassiveHandle *pPassiveHandle,
fsf::CActiveHandle *pActiveHandle,
fsf::CActivePulse *pActivePulse);
};
In the constructor, the default output name base is set, and both passive and active filters are instantiated from the corresponding template classes.
CMyCell::CMyCell()
: CCell()
{
// default output name
m_strOutputName.assign("Doesn't matter..."); // no output
// set the filters
m_pPassiveFilter=new fsf::CPassiveFilter<fsf::CPassivePulse>(std::string("Root"));
m_pActiveFilter=new fsf::CActiveFilter<fsf::CActivePulse>(std::string("Root"));
}
When the process function is executed, filtering of passive and active streams has succeeded. The active and passive handles are thus bound to the nodes satisfying the filters. When the filters are complex (hierarchies of filters), the passive and active handles point to the roots). For this very simple cell, no output is generated.
void CMyCell::Process(fsf::CPassiveHandle *pPassiveHandle,
fsf::CActiveHandle *pActiveHandle,
fsf::CActivePulse *pActivePulse)
{
// get input active node from active handle
fsf::CNode *pNode=static_cast<fsf::CNode*>(pActiveHandle->GetNode());
// compute the node time stamp in h/m/s format
long tsec=static_cast<long>(pNode->GetTime()/1000000000); // time unit is ns
long h=tsec/3600;
long m=(tsec-h*3600)/60;
long s=tsec-h*3600-m*60;
// print time stamp in s and in h/m/s on the console
cout << "Pulse: " << tsec << " s = "
<< h << " h " << m << " min " << s << " s " << endl;
}
Any module must define a RegisterFactories function that registers its node and cell factories with the system. Following is the code for the RegisterFactories function that registers the factory for CMyCell cell class.
// Factory registration
void RegisterFactories()
{
using namespace fsf;
CSystem *pSystem=CSystem::GetInstance();
std::string strAlex("Alexandre R.J. Francois");
////////////////////////////////////////////////
// Node factories
////////////////////////////////////////////////////////
////////////////////////////////////////////////
// Cell factories
////////////////////////////////////////////////////////
pSystem->RegisterCellFactory(std::string("MY_CELL"),
new CCellFactory<CMyCell>(std::string("My cell"),strAlex));
}
Here is a slightly more complex example, called ex2. The goal is now to display not the pulse time stamp, but the pulse frequency. This requires to compute the time delay between two consecutive pulses. Some data must therefore be shared between the threads processing each pulse: the time stamp of the last pulse processed is saved in a node on the passive pulse. Each time the process function is called, the last time stamp value is retrived from the node, and the node value is updated with the time stamp of the pulse being processed. The difference between the two time stamps is then computed and formatted to be output to the console. The corresponding cell will be called MyCell2.
Figure 5 shows the conceptual level system graph.
Figure 5: Conceptual system graph for ex2
The specifications for MyCell2 are as follows:
Compute and print pulse frequency to the console in Hz. Member functionsConstructors, destructor and other functions that are part of any derived cell class
Active stream processing
|
The class declaration is identical to that of MyCell. The only differences are to the passive filter in the constructor, and to the process function.
The code for the new constructor is as follows:
CMyCell2::CMyCell2()
: CCell()
{
// default output name
m_strOutputName.assign("Doesn't matter..."); // no output
// set the filters
m_pPassiveFilter=new fsf::CPassiveFilter<fsf::Int64Node>(std::string("Last time"));
m_pActiveFilter=new fsf::CActiveFilter<fsf::CActivePulse>(std::string("Root"));
}
Note the change in the passive filter: in order to compute a delay, the process function must access and update the value of the time stamp of the last pulse handled.
The code for the new process function is as follows:
void CMyCell2::Process(fsf::CPassiveHandle *pPassiveHandle,
fsf::CActiveHandle *pActiveHandle,
fsf::CActivePulse *pActivePulse)
{
// get pointer to last time node from passive handle
fsf::Int64Node *pLastTime=static_cast<fsf::Int64Node*>(pPassiveHandle->GetNode());
// get input active node from active handle
fsf::CNode *pNode=static_cast<fsf::CNode*>(pActiveHandle->GetNode());
if(m_bReset)
{
// set inital time stamp
pLastTime->SetData(static_cast<__int64>(pNode->GetTime()));
m_bReset=false;
}
else
{
// get last time value
fsf::Time tLastTime=static_cast<fsf::Time>(pLastTime->GetData());
// update node value with current pulse time stamp value
pLastTime->SetData(static_cast<__int64>(pNode->GetTime()));
// compute and print time difference and corresponding frequency
long dtms=static_cast<long>((pNode->GetTime()-tLastTime)/1000000);
cout << "Pulse interval: " << dtms << " ms <=> "
<< 1000.0f/dtms << " Hz" << endl;
}
}
The code for building the application graph for ex2 is very similar to that of ex1. The only difference
is in the output subgraph: the node storing the "last time stamp" value must be instantiated and placed on the
passive pulse held by pMySource. Of course, the cell to be instantiated now is of type MY_CELL2.
The corresponding code is as follows:
///////////////
// My cell 2 //
///////////////
// create the source
fsf::CSource *pPSource=new fsf::CSource;
bSuccess &= (pMySource!=NULL);
// last time stamp
fsf::Int64Node *pLastTime=static_cast<fsf::Int64Node*>(
scripting::CreateNode(std::string("FSF_INT64_NODE"),pMySource->GetPulse()));
bSuccess &= (pLastTime!=NULL);
if(bSuccess)
{
// set name
pLastTime->SetName(std::string("Last time"));
}
// cell
fsf::CCell *pMyCell=static_cast<fsf::CCell*>(
scripting::CreateCell(std::string("MY_CELL2")));
bSuccess &= (pMyCell!=NULL);
if(bSuccess)
{
// connect with source
scripting::ConnectSource(pMyCell,pMySource);
// connect with Pcell
scripting::ConnectUpstreamCell(pMyCell,pPcell);
}
The example implementation allows to specify the pulse rate on the command line (the default rate is 15 Hz). Below is a sample output from the program.
>ex2 15
Pulse interval: 68 ms <=> 14.7059 Hz
Pulse interval: 66 ms <=> 15.1515 Hz
Pulse interval: 67 ms <=> 14.9254 Hz
Pulse interval: 66 ms <=> 15.1515 Hz
Pulse interval: 66 ms <=> 15.1515 Hz
Pulse interval: 67 ms <=> 14.9254 Hz
Pulse interval: 66 ms <=> 15.1515 Hz
Pulse interval: 67 ms <=> 14.9254 Hz
Pulse interval: 66 ms <=> 15.1515 Hz
The definition and implementation of custom nodes is illustrated with a generic image node. The specifications for the image node image::CImage, available in the Image module, are as follows.
CImage (fsf::CCharBuffer) "IMAGE_IMAGE"Image node. [ImageModule.h] Data members
Member functionsConstructors, destructor and other functions that are part of any derived node class
Memory allocation
Data copy
Image parameters access
Image parameters setting
|
One solution when designing the image node was to encapsulate an existing image structure. Unfortunately, each image processing library comes with its own image structure. Committing to a given library might prevent access to other libraries, and prove restrictive in the long term. The image node defined in the Image Module provides a minimum representation to ensure its compatibility with existing image structures. However the image node does not contain any field specific of particular image formats, to ensure the widest compatibility. When needed, more specific image nodes may be derived from this base image node for leveraging specific library features. Because of inheritance properties, these specialized image nodes will be usable with all processes defined for the base image node.
Any node type specialization must be derived from the base node fsf::CNode or a derived node. A node type is characterized by an identification string (used to link it to its factory). A complete node type description includes a list of data members and member functions, and a short description of its semantics.
The image node is derived from the character buffer node fsf::CCharBuffer defined in the FSF library. An image buffer is indeed a character buffer. The smallest set of parameters needed to make the character buffer usable as an image buffer are the image width and height, the number of channels and the pixel depth. Since some libraries require data line alignment for optimal performance, the actual aligned width (width step) must also be stored. A utility protected member function is used to compute the aligned width.
class CImage : public fsf::CCharBuffer
{
protected:
int m_nNbChannels; // Number of channels
int m_nDepth; // Pixel depth IMAGE_DEPTH_*
int m_nWidth; // Image width
int m_nHeight; // Image height
int m_nWidthStep; // Aligned width (in bytes)
// utility protected member function
int ComputeWidthStep(bool bNoAlign=false);
Any custom node class must implement a number of constructors: the default constructor, all the constructors defined in the the base node class (these must define default values for the local data members), additional constructors for specifying local data members initial values, and the copy constructor. When necessary, the virtual destructor must also be overloaded.
public:
// Default constructor
CImage() : CCharBuffer(),
m_nNbChannels(3), m_nDepth(IMAGE_DEPTH_8U),
m_nWidth(0), m_nHeight(0), m_nWidthStep(0) {}
// Constructors with default values for local data members
CImage(fsf::CNode *pParent, fsf::Time tTime=0)
: CCharBuffer(pParent,tTime),
m_nNbChannels(3), m_nDepth(IMAGE_DEPTH_8U),
m_nWidth(0), m_nHeight(0), m_nWidthStep(0) {}
CImage(const string &strName,
fsf::CNode *pParent=NULL, fsf::Time tTime=0)
: CCharBuffer(strName,pParent,tTime),
m_nNbChannels(3), m_nDepth(IMAGE_DEPTH_8U),
m_nWidth(0), m_nHeight(0), m_nWidthStep(0) {}
// Constructors with local data members initial values input
CImage(int nWidth, int nHeight,
int nNbChannels=3, int nDepth=IMAGE_DEPTH_8U,
fsf::CNode *pParent=NULL, fsf::Time tTime=0)
: CCharBuffer(pParent,tTime),
m_nNbChannels(nNbChannels), m_nDepth(nDepth),
m_nWidth(nWidth), m_nHeight(nHeight), m_nWidthStep(0) {}
CImage(const string &strName, int nWidth, int nHeight,
int nNbChannels=3, int nDepth=IMAGE_DEPTH_8U,
fsf::CNode *pParent=NULL, fsf::Time tTime=0)
: CCharBuffer(strName,pParent,tTime),
m_nNbChannels(nNbChannels), m_nDepth(nDepth),
m_nWidth(nWidth), m_nHeight(nHeight), m_nWidthStep(0) {}
// Copy constructor
CImage(const CImage&);
No destructor overload is needed here, since the destructor for the character buffer parent class takes care of deallocating the buffer if needed.
The custom node class must also overload a number of virtual functions which characterize the node:
// Assignment operator
CImage& operator=(const CImage&);
// Cloning: necessary for run-time polymorphism
virtual fsf::CNode *Clone() { return new CImage(*this); }
// Factory mapping key
virtual void GetTypeID(string &str)
{ str.assign("IMAGE_IMAGE"); }
A set of member functions provides basic access to local data members (set and get operations). A memory allocation function and high level parameter and image data (buffer content) copy functions complete the set of tools offered by the image node.
void CopyParameters(const CImage&);
void CopyImageData(const CImage&);
// Image parameters setting
void SetWidth(int nWidth) { m_nWidth=nWidth; }
void SetHeight(int nHeight) { m_nHeight=nHeight; }
void SetNbChannels(int nNbChannels) { m_nNbChannels=nNbChannels; }
void SetPixelDepth(int nDepth) { m_nDepth=nDepth; }
void SetWidthStep(int nWidthStep) { m_nWidthStep=nWidthStep; }
// Image parameters access
int Width() const { return m_nWidth; }
int Height() const { return m_nHeight; }
int NbChannels() const { return m_nNbChannels; }
int PixelDepth() const { return m_nDepth; }
int WidthStep() const { return m_nWidthStep; }
// Memory allocation
void Allocate(bool bNoAlign=false);
};
When an image node instance is created, its parameters must be set. Constructors provide default values, set functions allow to change the values explicitly. The corresponding buffer must then be allocated by a call to the Allocate function. The image node instance can then be used for processing.
Any module must define a RegisterFactories function that registers its node and cell factories with the system. Following is the code for the image::RegisterFactories function. Apart from the image node image::CImage, the module also implements a number of cells that provide access to its various data members. Their description can be found in the Image module documentation. Since an example of cell factory registration is provided in the previous section, the code for cell factory registration has been ommitted below.
void image::RegisterFactories()
{
using namespace fsf;
using namespace image;
CSystem *pSystem=CSystem::GetInstance();
// Node factories
pSystem->RegisterNodeFactory(std::string("IMAGE_IMAGE"),
new CNodeFactory(std::string("Image node"),
std::string("Alexandre R.J. Francois")));
// Cell factories
...
}
This section describes architectural patterns for interfacing with external libraries or devices. The model, which englobes a set of patterns for data input and output, is named interface after the main data element through which communication with external entities takes place: the Interface node.
Interface nodes are a class of specialized nodes whose specific purpose is to provide a link to and from external data through specialized drivers or ibraries, whose API they encpasulate. In some cases, a same interface node class may handle both input and output.
The output pattern is used for exporting data from an SAI stream to an external entity. Examples include image display (export images to a display device), file output (export data to a file), network transmission (export data to a socket), etc.
Figure 6 shows the core output pattern, composed of an Output cell, connected to a source holding an Interface node. The Interface node need only provide a member function, called by the Output cell's Process function, that exports the data based on the specifics of the driver or library used, and after eventual encoding.
Figure 6: Output pattern.
The logical output cell definition for this pattern is of the form:
| COutput (fsf::CCell) | "OUTPUT" |
| Active filter | [DATA "Data"] |
| Passive filter | [INTERFACE "Interface"] |
| Output | (no output) |
Input patterns are used to import data from an external entity into an SAI stream. Examples include live video capture (import images from a camera), video file input (import images from a file), gamepad input (import state data from a gaming device), network reception (import data from a socket), etc.
The core pattern is composed of an Input cell, connected to a source holding an Interface node. Input cells/interfaces can operate according to two different modalities: push and pull.
In this approach, the Input cell is triggered regularly by a Pulsar, and queries the interface node through a method that performs data import. After encoding into an SAI compatible structure (node), the data is added to the active pulse.
Figure 7 shows the pull input pattern.
Figure 7: Pull input pattern.
This model is the simplest input pattern to implement.
The logical input cell definition for this pattern is of the form:
| CInput (fsf::CCell) | "INPUT" |
| Active filter | [FSF_ACTIVE_PULSE "Root"] |
| Passive filter | [INTERFACE "Interface"] |
| Output | (x (DATA "Data")) |
In this approach, the external entity triggers the creation of an active pulse when data is available. The interface node provides a callback function that is triggered by the external entity. This function calls a special function in the input cell whose purpose is to create an active pulse carrying the data, and placing it on the stream. Note that the cell's Process function is completely bypassed, and that the interface node must have a pointer to the cell to which it is associated.
Figure 8 shows the push input pattern.
Figure 8: Push input pattern.
This model is a little more complicated to implement, as it requires requires non trivial interface node functionalities and overloading of additional member functions in the input cell. However, it is the most efficient when dealing with discrete, irregular streams (typically messages, e.g. MIDI messages), and with some interface drivers (e.g. image capture from a camera).
A logical input cell definition for this pattern is of the form:
| CInput (fsf::CCell) | "INPUT" |
| Active filter | (no input) |
| Passive filter | [INTERFACE "Interface"] |
| Output | (x (DATA "Data")) |
The patterns described above assume that the interfacing has taken place. Interface management operations, such as connecting to and disconnecting from a device, opening or closing a connection or a file, may be hardcoded in the application (interface nodes must provide functions to perform these tasks). In general, dynamic management should be carried through specialized action cells, that provide a consistent SAI access to the nodes.
An action cell is typically created dynamically, and triggered once at a time. It takes as input a set of action parameters (e.g. a device ID, a file name, an IP address), and applies to an interface node instance. When triggered, the Process function calls the appropriate member function of the interface node instance with the input parameters. The output is a success status in the form of a boolean node added to the active stream.
Figure 9 illustrates the general pattern for action cells.
Figure 9: Action cell pattern.
Action cells can be created (and destroyed) and triggered dynamically. For example, they can provide consistent access to data stored in sources through GUI or programs not implemented in the SAI style.
A logical action cell definition is of the form:
| CAction (fsf::CCell) | "ACTION" |
| Active filter | [action parameters -- may be complex] |
| Passive filter | [INTERFACE "Interface"] |
| Output | (x (FSF_BOOL_NODE "Success")) |
Figure 10 shows a typical rendering pattern. A possibly dynamic but persistent model to be rendered, made of one or several standard or custom nodes, is stored in a source. A rendering cell, triggered by pulses generated by a Pcell, computes snaphots of the model. The source also holds a number of parameter nodes (here grouped under a generic node): a camera node and some parameters such as the image width and height, used by the renderer to produce its output images.
Figure 10: Rendering pattern.
A logical rendering cell definition is of the form:
| CRenderer (fsf::CCell) | "RENDERER" |
| Active filter | [FSF_ACTIVE_PULSE "Root"] |
| Passive filter | [scene graph, camera, parameters, etc.] |
| Output | (x (IMAGE_IMAGE "Image")) |
Note that from an implementation point of view, rendering into a bitmap that would then be displayed by a separate cell (e.g. from the Display module might not seem to be an optimal solution. A similar rendering cell, that uses OpenGL to render directly into a window context, proved significantly more computationally expensive. The solution presented here is thus not only closer to the componentization philosophy of SAI, it is also more efficient in its implementation.
Figure 11 shows the basic animation pattern.
Figure 11: Animation pattern.
A logical update cell definition is of the form:
| CUpdate (fsf::CCell) | "UPDATE" |
| Active filter | [FSF_ACTIVE_PULSE "Root"] |
| Passive filter | [object, parameters, etc.] |
| Output | (no output) |
Figure 12 shows a camera control pattern following the pull input interface model described above.
Figure 12: Camera control pattern (pull input interface model).
A logical camera control cell definition is of the form:
| CControl (fsf::CCell) | "CONTROL" |
| Active filter | [INPUT_DEVICE_DATA "Data"] |
| Passive filter | [object, parameters, etc.] |
| Output | (no output) |