It is All about Traffic, and Programming
Search
Archives
0 visitors online now
0 guests, 0 members

C/C++

Who touches my “for-loop” condition?

A for loop is a common control structure that allows repetitions of certain statements for a specific number of times. The syntax of a for-loop in C++ or C# is

for (initialization; condition; increment) {
   statement(s);
}

As simple as ABC. As easy as a no-brainer. As happy as Clam.

Now, look at the following C++ code. How many times would the the loop execute?

int main( )
{
   int N = 10;

   for ( int i = 0; i < N; i++ ) {
     N--;
     std::cout << "i = " << i 
               << ";"
               << "N= "  << N 
               << std::endl;
   }

   return 0;
} 

As the output shown below – The loop executes 5 times, even though we set N = 10.  Obviously, modifying N inside the loop body, i.e., N–, causes the termination of the loop prematurely.

Generally speaking, it is a bad practice to modify loop termination condition inside a for-loop. Bad, Bad, Bad!

Now, the same question for a similar Delphi code:

uses
  System.SysUtils;

var
  I, N: Integer;

begin
  try
    N := 10;

    for I := 0 to N do
    begin
      Dec(N);
      WriteLn( Format('I=%d, N=%d',[I, N]) );
    end;
  except
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);
  end;
end.

The output is not so much unsurprising – unlike the C++ counterpart, it does loop 11 times:

Interesting. It seems modifying N inside loop body doesn’t impact the number of intended loop execution times.

Let’s check the disassembled C++ binary:

As seen from the highlighted lines, N is referenced by address (i.e., ptr [N]) in loop body. This means, any change to N’s value will eventually impact the termination condition evaluation. Address [ebp-14h] is the local variable i stored in Stack, which is incremented with every repetition of the loop, before comparing to ptr[N] that is decremented with each loop.

Delphi compiler generates something different:

N is addressed at [0x004258a8]. At the beginning of the loop, N’s value is copied to [$ebp-$14]. Each loop just checks [$ebp-$14] for termination. Modifying the original Nwouldn’t impact the loop termination condition evaluation.

So, though it is generally not a good practice to change for-loop termination condition inside the loop,  Delphi obviously has some compiler caveat that may slip the mind of a careless programmer. Be warned!

The Move Constructor: A Linking Pitfall of Qt Library with Different Compilers

Move constructor is a C++11 feature from ISO/IEC14882:2011.  It enhanced the old-school C++ copy constructor by moving the resource of an object, instead of member-wise copying.  It is faster than a copy constructor.  Thus a C++11 compiler will generate six special member functions for a C++ object, including  move constructor and move assignment operator:

  • Default Constructor
  • Copy Constructor
  • Copy Assignment Operator
  • Destructor
  • Move Constructor
  • Move Assignment Operator

We are not going to discuss C++11 Move Constructor in details here –  better details and better explanations are there, and there.

The motivation of  this article,  is not to discuss the semantics or usage of Move Constructor,  but an interesting debugging experience, and the lessons learned which involves C++11 move constructor.

Long story short – I have been working on developing an advanced Qt-based C++ plugin DLL for a host software developed in Qt 4.8.7.   I was using Visual C++ 2010 compiler, and the Qt for VS2010 official release for commercial applications.

The plugin DLL is supposed to be dynamically loaded by the host via Qt’s QLibrary API (see my previous article on this subject), which internally will call Microsoft’s Win32 API LoadLibrary.  Somehow, I got the following error, when trying to load the DLL into the host software:

34f8:3518 @ 01570140 – LdrpGetProcedureAddress – INFO: Locating procedure “AcdssPluginFactory” by name
34f8:3518 @ 01570140 – LdrpNameToOrdinal – WARNING: Procedure “AcdssPluginFactory” could not be located in DLL at base 0x0B630000
34f8:3518 @ 01570140 – LdrpReportError – WARNING: Locating export “AcdssPluginFactory” for DLL “Unknown” failed with status: 0xc0000139
34f8:1168 @ 01570265 – LdrpReportError – ERROR: Locating export “??4QString@@QAEAAV0@$$QAV0@@Z” for DLL “C:\Program Files (x86)\…\plugins\acdssplugin.DLL” failed with status: 0xc0000139

The above debug messages were generated by GFlags Utility, which comes with Debugging Tools for Windows. You can find the most advanced debugging techniques including how to use GFlags utility from the excellent book Inside Windows Debugging.  In our case, we just need to set “Show Loader Snaps” flag, as shown in the figure.  This is a typical way of debugging LoadLibrary failures.

2016-10-12_21-41-26

Looking at the debug messages, it is simply pointing out that LoadLibrary fails because an exported function is missing from the image space – its C++ mangled name as  “??4QString@@QAEAAV0@$$QAV0@@Z”,  which can be interpreted as:


    ? = C++ mangled symbol
    ?4 = "operator="
    QString@ = the enclosing class, hence method name: QString::operator=
    @ = end of the scope
    Q = public
    A = no const and volatile qualifier
    E = __thiscall
    AAV0 = const ref to the first named class (QString) - the return value (const QString &)
    @ = end return type
    $$ = move constructor &&
    QAV0 = const pointer to first named class - QString * const
    @ = end parameter type
    @ = end parameters
    Z = function

So clearly, the host software was complaining that the Move Constructor of QString can not be located in any of the image space.  Upon further inspection of the host software, it is discovered it was compiled using VS2005.  QtCore4.dll does not have the Move Constructor generated with VC++ 2005 compiler.  

The solution is simple – just rebuilt the Qt 4.8.7 using VC++ 2005, and rebuilt the plugin DLL.  Above all, the lessons learned are:

Do NOT mix Qt library compiled with different compilers, EVEN with the same Qt Version.  Period.

Dissecting Vissim COM Internal from Inside Out (3)

In order to study the life cycle of CCOMVissim,  we have to track the calling stack of comsvr.dll.  This DLL is the host of CCOMVissim and other helper classes of Vissim COM functionalities. Please note it is not possible and not an option to perform static code analysis using a typical debugger. This is because CodeMeter hardware chip employs sophisticated anti-debugging mechanism, black-listing all known static/dynamic analysis debuggers such as OllyDBG or IDAPro .  We’d have to return to comsrv.DLL  – and recall in this post – the 58 exported functions, something like below:

vissim-com-export-index-1-30

As an effective approach (for interoperability among Vissim’s own various licensed modules),  let’s start creating a DLL project, using the same name “comsrv.DLL”,  as the one in the Exe folder of Vissim installation.

As a good background reading on exporting  C++ classes in a DLL,  here is an detailed post CodeProject: Export C++ classes from a DLL

Following this new Visual Studio C++ DLL project,  create a new class called CCOMVissim  – pay attention to the class definition, and the empty methods, and the declaration “__declspec(dllexport)“.  By default, the calling convention is “thiscall“.  Because we know Vissim is compiled in Visual C++ compiler apriori, thus name mangling wouldn’t be an issue (because of the same C++ compiler).

2015-06-05_14-59-13

Now, compile this project, and a dll called comsrv.dll is generated.  After this new DLL is generated,  de-compile it to verify the exported functions (see the figure below):

2015-06-05_15-02-18

We just created a “proxy” DLL with the same name as Vissim’s original comsrv.dll.  We emulated some of the original exported functions and classes.  The point is,  as long as the same list of  functions  are exported by this new dll, it is “loadable” by Vissim.  Therefore, we are  enableed to append additional code to track, manage, and manipulate the calling stack of those class methods or functions relevant to the life-cycle of IVissimPtr. This will help accomplish the objective of enabling interoperability of Vissim COM functionalities within Vissim’s various DLL-based modules,  including Signal Control API DLL, External Driver DLL and such.

If you could understand up to this point,  there should be no problem for you to move on to a final working solution to the question asked in the beginning.  The efforts are yours, and the omissions are mine.  One last hint:

Rename the original “comsrv.dll” to a different name, e.g., “comsrv_org.dll”.  Redirect calls from inside the proxy comsrv.dll to the original dll.

Dissecting Vissim COM Internal from Inside Out (2)

Let’s look at a typical Vissim COM programming code snippet in C++. It is taken from the Vissim manual, with some reformatting.

For Vissim end users, C++ is not an ideal choice of language for COM applications.  You’d have to handle all the complexities involved with COM  interface management. C++/COM programming has never been something as easy as a breeze with a steep learning curve.  On the other hand,  C++/COM is excellent that it does not hide COM implementation details,  while providing the best performance than any other languages. It is simply native to the COM world.

vissim-com-vc-snippet-1vissim-com-vc-snippet-2

In this beautiful C++ code snippet, Smart Pointer is used.  QueryAttach is a helper function shipped with this sample; it encapsulates the tedious QueryInterface process, and wraps up the returned Vissim interface as a smart pointer.

Clearly, the code illustrates the work-flow invoking Vissim as an automation server (see my another post on Vissim COM Instance Model).  When you call CreateInstance,  a Vissim process is started, where the default interface IVissim is instantiated and returned as the smart pointer IVissimPtr.

To say the least,  IVissimPtr is the “Open Sesame” to the wonderland of Vissim COM programming.

The  flow of all Vissim COM programming starts from getting an instance of IVissim interface.

It is fairly easy to get an IVissim interface by invoking Vissim as an automation server – just having an external COM client to call   CreateInstance.

VISSIMLIB::IVissimPtr pVissim declares the smart pointer to the default  IVissim interface.  COM smart pointer encapsulates COM operations including AddRef, Release etc. Don’t confuse yourself VISSIMLIB::IVissimPtr with IVissimIVissimPtr is generated by macro _COM_SMARTPTR_TYPEDEF(IVissim, __uuidof(IVissim)), while IVissim is declared as the following:

struct __declspec(uuid("a023e2ec-8002-4bb1-a584-b85ce59681ee"))
IVissim : IObjectBase
{
    //
    // Property data
    //

    __declspec(property(get=GetNet))
    INetPtr Net;
    __declspec(property(get=GetSimulation))
    ISimulationPtr Simulation;
    __declspec(property(get=GetEvaluation))
    IEvaluationPtr Evaluation;
    __declspec(property(get=GetGraphics))
    IGraphicsPtr Graphics;
    __declspec(property(get=GetPresentation))
    IPresentationPtr Presentation;

    //
    // Wrapper methods for error-handling
    //

    HRESULT New ( );
    HRESULT LoadNet (
        _bstr_t NetPath,
        VARIANT_BOOL Additive );
    HRESULT SaveNet ( );
    HRESULT SaveNetAs (
        _bstr_t NetPath );
    HRESULT LoadLayout (
        _bstr_t LayoutPath );
    HRESULT SaveLayout (
        _bstr_t LayoutPath );
    HRESULT ImportANM (
        _bstr_t NetPath,
        _bstr_t RoutesPath,
        _bstr_t InputPath,
        ImportType ImportType,
        int importOptions,
        int evaluationInterval );
    HRESULT BringToFront ( );
    HRESULT Exit ( );
    INetPtr GetNet ( );
    ISimulationPtr GetSimulation ( );
    IEvaluationPtr GetEvaluation ( );
    IGraphicsPtr GetGraphics ( );
    HRESULT ImportSynchro (
        _bstr_t synchroFile,
        _bstr_t workingDirectory,
        VARIANT_BOOL adaptiveImport );
    IPresentationPtr GetPresentation ( );
    HRESULT ExportVisum (
        _bstr_t NetPath,
        VisumExportType ExportType );
    HRESULT ImportResults (
        _bstr_t Path );
    HRESULT VissimTest (
        int TestNo,
        _bstr_t Param1,
        _bstr_t Param2,
        _bstr_t Param3 );
    HRESULT SuspendUpdateGUI ( );
    HRESULT ResumeUpdateGUI (
        VARIANT_BOOL enforceRedraw );

    //
    // Raw methods provided by interface
    //

      virtual HRESULT __stdcall raw_New ( ) = 0;
      virtual HRESULT __stdcall raw_LoadNet (
        /*[in]*/ BSTR NetPath,
        /*[in]*/ VARIANT_BOOL Additive ) = 0;
      virtual HRESULT __stdcall raw_SaveNet ( ) = 0;
      virtual HRESULT __stdcall raw_SaveNetAs (
        /*[in]*/ BSTR NetPath ) = 0;
      virtual HRESULT __stdcall raw_LoadLayout (
        /*[in]*/ BSTR LayoutPath ) = 0;
      virtual HRESULT __stdcall raw_SaveLayout (
        /*[in]*/ BSTR LayoutPath ) = 0;
      virtual HRESULT __stdcall raw_ImportANM (
        /*[in]*/ BSTR NetPath,
        /*[in]*/ BSTR RoutesPath,
        /*[in]*/ BSTR InputPath,
        /*[in]*/ ImportType ImportType,
        /*[in]*/ int importOptions,
        /*[in]*/ int evaluationInterval ) = 0;
      virtual HRESULT __stdcall raw_BringToFront ( ) = 0;
      virtual HRESULT __stdcall raw_Exit ( ) = 0;
      virtual HRESULT __stdcall get_Net (
        /*[out,retval]*/ struct INet * * Net ) = 0;
      virtual HRESULT __stdcall get_Simulation (
        /*[out,retval]*/ struct ISimulation * * Simulation ) = 0;
      virtual HRESULT __stdcall get_Evaluation (
        /*[out,retval]*/ struct IEvaluation * * Evaluation ) = 0;
      virtual HRESULT __stdcall get_Graphics (
        /*[out,retval]*/ struct IGraphics * * Graphics ) = 0;
      virtual HRESULT __stdcall raw_ImportSynchro (
        /*[in]*/ BSTR synchroFile,
        /*[in]*/ BSTR workingDirectory,
        /*[in]*/ VARIANT_BOOL adaptiveImport ) = 0;
      virtual HRESULT __stdcall get_Presentation (
        /*[out,retval]*/ struct IPresentation * * Presentation ) = 0;
      virtual HRESULT __stdcall raw_ExportVisum (
        /*[in]*/ BSTR NetPath,
        /*[in]*/ VisumExportType ExportType ) = 0;
      virtual HRESULT __stdcall raw_ImportResults (
        /*[in]*/ BSTR Path ) = 0;
      virtual HRESULT __stdcall raw_VissimTest (
        /*[in]*/ int TestNo,
        /*[in]*/ BSTR Param1,
        /*[in]*/ BSTR Param2,
        /*[in]*/ BSTR Param3 ) = 0;
      virtual HRESULT __stdcall raw_SuspendUpdateGUI ( ) = 0;
      virtual HRESULT __stdcall raw_ResumeUpdateGUI (
        /*[in]*/ VARIANT_BOOL enforceRedraw ) = 0;
};

In addition to invoking Vissim as an automation server,  In-Menu/Event-based COM scripting has a different mechanism.

In the In-Menu/Event-based COM script,  there is no explicit instantiation of IVissim. As a matter of fact,  there is a “Vissim” object already created, ready to be referenced in the script.

That is a mystery, isn’t?

To trace how and when “Vissim” object is created in the scripting context, we need to trace the life cycle of  the class CCOMVissim. That is the key to the original question – i.e.,  how to call all those COM functions inside an external DLL.

We’ll continue with our journey looking into the life cycle of the class CCOMVissim.  Right now, please just look at IVissim declaration again, and remember the following:

IVissim declaration stipulates nothing but just a memory blueprint. The sequence of all its virtual members are the slots of the v-table of IVissim’s implementation class.  This, is the “Open Sesame”.

Dissecting Vissim COM Internal from Inside Out (1)

There was an interesting discussion at PTV Vissim LinkedIn group – how to access Vissim COM interface inside a DLL, e.g., a Signal Control DLL,  an External Driver DLL etc.  To properly answer this question, some in-depth understanding of the COM technology,  C++ compiler internals, and Vissim binary level details are in order.

To begin with, Component Object Model (COM) is just a binary-level coding standard for out-of-process and in-process communication. It’s been out there for many years but somehow never become popular due to its daunting learning-curve for ordinary programmers,  and then Microsoft began promoting .NET, which is essentially similar idea revamped at the level of Common Language Run-time (CLR) and Intermediate Language (IL).

In the nostalgia good-old days,  COM/C++ programming required too steep learning curve to weed out sub-par programmers.  Nowadays programming is so much easier with various .NET languages or scripting languages that do not require low-level knowledge of hardware or compiler internals. The barrier to programing is quite low nowadays; anyone with common  sense can pick up programming,  develop some apps, and sell such on streets.

In the context of Vissim,  PTV wraps up the simulation elements and workflow into various interfaces. These interfaces are implemented (in C++) at binary level thus any language that supports COM (by their respective compilers) will be able to use Vissim’s COM interface to interact with Vissim functionality.

In the  earlier days of Vissim, its COM interfaces allow a client application to invoke Vissim as an automation server (as an out-of-process use case). Then PTV added In-Menu scripting, that supports invoking COM inside the active Vissim instance, using Python, VBS, or JS.

Starting Vissim 6, PTV completely refactored and redesigned Vissim, almost rebuild the software from scratch, inside out.  As part of this process, more gems are added,  one of such, is the so-called “event-based” COM scripting. “Event-based” COM scripting is a use case of in-process COM; it was added since Vissim v7.

Once again, with the In-Menu COM/Event-based Scripting,  and many other new designs/architectures, Vissim has set up a solid foundation significantly better than its v5 and any earlier generations, making itself readily primed for the future. A big congratulation is due to the PTV development team for this achievement, seriously.

Some of my earlier post discussed Vissim COM Internal as well.

As of today, Vissim 7-09 appears as the most impressive redemption of a legacy software (I mean, particularly, Vissim v4 and earlier versions) – and presents itself, with so many improvements and beef-ups, and as a powerful, sleek and increasingly popular tool for its users that have been yelling and crying for  years while its competitors eating up its market share.

Aside from COM interface, Vissim also provides various APIs e.g., Signal Control API, Emission API, External Driver API etc. These APIs have to be compiled in DLL to be loaded by Vissim host, which calls back the exported functions when certain events fire.

It is interesting that “event-based” COM scripting is capable of providing a unified interface for all the above APIs.  In future, it is possible that PTV might gradually phase out all these DLL-based APIs so everything is done through a unified event-based COM scripting. Granted, that is just my personal perspective.

Now, coming back to the million dollar question –  can we access COM interface from inside an API DLL? For example,  from inside External Driver DLL,  how can we access some, if not all of the COM functions?

The answer is: yes,  that is doable, but with some twist.

Now you start to have a faint of heart,  aren’t you?  Before we roll up our sleeves and jump to the 1-2-3 steps, I want you to take a look at the following snapshots – they are the key to the solution.

The following are the 58 exported functions from comsrv.dll, which is located in Vissim installation Exe folder. Most of these exported functions, are class methods of several classes, primarily:

  • CComVissim
  • CComNetObjHelper
  • CComVehicleContainer
  • CComSignalController
  • CComPedestrianContainer
  • CComException

💡 comsrv.dll is the “hub” of all Vissim COM functionality.

With the above in mind,   let me lead you to the interesting journey to figure out comsvr.dll  and Vissim COM invocation flow, till we reach the final solution to the original question.

vissim-com-export-index-1-30

vissim-com-export-index-31-58

 

(to be continued)

Brain-Wave Controlled Traffic Simulation: Control a Vissim Simulated Car (1)

Ever watched the movie “Carrie”, a classic produced in 1976 from Stephen King’s 1974’s horror novel?  An abused 17-year-old girl with telekinesis, gets pushed to the limit by a humiliating prank,  and finally took her revenge with her power – turning her school’s prom into a bloody night of killing.

You might want to develop telekinesis on your own,  so as to get back on whoever you get pissed of with – Good Luck on That!  Here is a nice tip on How to Develop Telekinesisjust let me know how it works!!

What we are going to talk about today is the so-called Brain-Computer-Interface (BCI), which is about using your brain wave patterns to train a computer program, while the latter directs some device doing some nice, dirty, and/or even horny (you wish!)  jobs for you   😆  

Here is an excerpt from Wikipedia:

A brain–computer interface (BCI), sometimes called a mind-machine interface (MMI), or sometimes called a direct neural interface (DNI), synthetic telepathy interface (STI) or a brain–machine interface (BMI), is a direct communication pathway between the brain and an external device. BCIs are often directed at assisting, augmenting, or repairing human cognitive or sensory-motor functions.

BCI uses Electroencephalography(EEG), which detects voltage fluctuations resulting from ionic current flows within neurons of brain. Whenever your neurons fire,  weak electrical signals in millisecond-range resolution propagates all the way to the scalp.This signal is at higher resolution than CT or MRI.

Therefore, BCI is pretty much about signal processing, machine learning and pattern recolonization recognition.

EEG was invented in early 1900’s, hence it is not something new.  However, when the time comes to 2010’s,  the industry has come out with cost-effective and portable EEG devices (unlike those used in medical applications).  Novel applications are thus enabled.  The following pictures (source:http://emotiv.com/) illustrate a conventional EEG and a portable EEG device.

compare1 compare2

The following picture shows Emotiv EPOC EEG device, manufactured by Emotiv, featuring

16 wet electrodes, 14 EEG electrodes read brain waves, two-axis gyroscope to read head movements, 4 mental states, 13 conscious thoughts or facial expressions, 4 processing suites

The wonderful thing is, this little device sells for only $399 desktop or $499 for Bluetooth Smart, with full programming SDKs. You can even integrate it with wearable devices (such as Google Glass).

epoc

And, that is what I am gonna do.  I am going to use this little gadget, and its SDKs to interface with VISSIM microscopic traffic simulator via its COM interface and Driver API – so I can:

  • Command a simulated car to brake, accelerate and stop,  using my brain wave – termed otherwise,  by simply staring at my computer screen like a dork 😯 .

That is right.  I am going to present an interesting demo here that Brain-Computer-Interface for the first time,  being used in traffic simulation to control a simulated car…… (I really wish one day I could use the same to command my boss to pay me more  😈 )

Stay tuned. It is going to be realllllly FUN.

(to be continued)

QThread: the Way of Do-It-Right

Recently I run into this blog of Bradley T. Hughes, a member of the Qt development team.   He put an interesting post about the “right” way of using QThread.  That is,  to sub-class QObject and use  moveToThread.  This is quite contrary to what is said in the existing Qt official documentation, which suggests subclassing QThread directly.

The reason of doing this is that QThread should be treated as an interface,  a control point or a manager of worker threads.  As such  it shouldn’t contain  any threaded code.   As a matter of fact,  QThread’s thread affinity by default is with its creating thread,  not the thread it spawns.  This means QThread by default shares the same thread message queue as its creating thread, which in most cases is the main thread.

There are a few discussions on the internet about subclassing QThread,  while calling moveToThread in the QThread sub-class’s constructor.  This is based on the misconception of QThread usage.  Sometimes it works as a work-around trick to change the default thread affinity from the creating thread to the thread it spawns.  This practice is not encouraged,  according to Bradley blog, because this  will result in not only a bad design violating OO principles but also problems very hard to debug later.  This is well explained in Bradley’s blog.

Here is a brief summary of the right (suggested) way of using QThread:

1. When no event-loop is involved,  we can continue with the “official” approach, like what is introduced in the Qt documentation, i.e.,

  • Subclass QThread;
  • Overload Run();
  • Use some termination token inside a while loop inside Run(), so it can exit gracefully.

2. When event-loop is involved, things are a little more complicated:

  • Subclass QObject, define relevant signals and slots.  Distribute the “threaded” code over the relevant slots;
  • Since QThread::run() automatically calls  QThread::exec() so no need to worry that (for Qt 4.4 and later);
  • Call QObject::moveToThead, so that the slots are executed in the context of the working threads spawned and managed by the QThread instance.

Maya Posch’s blog gives a nice example illustrating the the above:

class Worker : public QObject
{
    Q_OBJECT
public:
    Worker();
    ~Worker();
public slots:
    void process();
signals:
    void finished();
    void error(QString err);
private:
    // add your variables here
};

// --- CONSTRUCTOR ---
Worker::Worker()
{
   // you could copy data from constructor arguments to internal variables here.
}

// --- DECONSTRUCTOR ---
Worker::~Worker()
{
   // free resources
}

// --- PROCESS ---
// Start processing data.
void Worker::process()
{
   // allocate resources using new here
   qDebug("Hello World!");
   emit finished();
}

In Maya’s original example,  the following is posted:

QThread* thread = new QThread;
Worker* worker = new Worker();

worker->moveToThread(thread);

connect(worker, SIGNAL(error(QString)), this, SLOT(errorString(QString)));
connect(thread, SIGNAL(started()), worker, SLOT(process()));
connect(worker, SIGNAL(finished()), thread, SLOT(quit()));
connect(worker, SIGNAL(finished()), worker, SLOT(deleteLater()));

// The following line may cause memory leak.
connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
thread->start();

There are some nuances here –

First,  the highlighted line may cause leak of the worker instance,  because the the QThread instance may have already be deleted earlier. Then the thread context is no long existing for the following line of code since it is only executed in the context of the thread managed by QThread:

connect(worker, SIGNAL(finished()), worker, SLOT(deleteLater()));

Therefore,  this line

connect(thead, SIGNAL(finished()), thread, SLOT(deleteLater()));

should be changed to

connect(worker, SIGNAL(destroyed()), thread, SLOT(deleteLater()));

Second,  do NOT allocate resources in the ctor of QOjbect derived class.  This is because before calling moveToThread,  the thread affinity is still with the main thread (i.e., the thread where the instance of the derived class is created) hence it may cause some hidden hard-to-debug problems later with the switch of thread affinity.

REFERENCE LINKS 

  1. http://labs.qt.nokia.com/2010/06/17/youre-doing-it-wrong/
  2. http://labs.qt.nokia.com/blogs/2006/12/04/threading-without-the-headache/
  3. http://labs.qt.nokia.com/blogs/2007/07/05/qthreads-no-longer-abstract/
  4. http://gitorious.org/qthreadhowto/qthreadhowto/trees/master
  5. http://blog.exys.org/entries/2010/QThread_affinity.html
  6. http://thesmithfam.org/blog/2010/02/07/talking-to-qt-threads/
  7. http://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/
  8. http://hi.baidu.com/cyclone/blog/item/5fac3bc7ab1b90d1d10060f2.html

Advanced Aimsun API Programming (1)

Aimsun MicroAPIs are the programming interface of Aimsun microscopic traffic simulator. These APIs provided 10 predefined call back hooks, together with various other utility functions.

MicroAPIs allow advanced users to implement sophisticated external logic, for example, non-standard ramp metering, advanced traffic management strategies, adaptive signal control etc.. These are not readily available with standard off-the-shelf Aimsun features, but made possible with Aimsun MicroAPIs.

The Aimsun MicroAPIs are very flexible, as its final product is in the format of a Dynamic Link Library (DLL). The DLL is loaded by Aimsun host during run time before running a simulation replication, and unloaded after the simulation replication finishes. The supported programming language is C++ and Python. But a developer with sufficient knowledge of DLL programing can easily port the development in other languages such as Delphi, C#, Java, Fortran, if he or she desires so.

One nuance about Aimsun MicroAPIs programming  is the granular control level down to a single simulation step. This is no surprise since Aimsun Micro is a time-discretized space continuous simulator that all the simulation events have to be driven by incrementing simulation steps.  However,   this might pose some programming challenges or certain pitfalls when multi-threading is involved, or  when a TCP/IP based client-server structure is employed.  The tricky point is that at the simulation step level,  the control of flow is transferred from Aimsun host to MicroAPI DLL,  while NOT until the call back finishes,  Aimsun host is essentially “blocked”. This could possibly counter-balance any benefits obtained from the multi-threaded code implemented inside the API logic.

Overall, Aimsun MicroAPIs include two types of functions:

  1. Predefined DLL Exported Functions (hooks to the Aimsun Host);
  2. Utility Functions provided by the Aimsun Host to retrieve various run time information.

The utility functions can be further classified into three categories:

  1. External Control Interface Functions for traffic signal control (ramp metering, intersection signal control etc). These functions are usually prefixed with ECI, for example, ECIChangeParametersFlowAlineaMeteringById;
  2. Aimsun NG Objects Connection Interface Functions for accessing properties of Aimsun internal objects. These functions are usually prefixed with ANGConn, for example, ANGConnActivatePolicy;
  3. Aimsun Kernel Interface Functions to retrieve other simulation run time information. These functions are usually prefixed with AKI, for example, AKIDetGetDensityAggregatedbyId.

Predefined Callbacks

As mentioned above, Aimsun MicroAPIs provided 10 predefined call-back functions (hooks). These functions are exported by the MicroAPI DLL, so that the Aimsun host can load the DLL and call these functions at appropriate time during simulation. It is the task of the MicroAPI developers to design the external logic and implement the logic under the framework of these functions properly.

In a nutshell, Aimsun host will call back these functions on the following events:

  1. The MicroAPI DLL is loaded or unloaded by the Aimsun host;
  2. The external logic is registered with the Aimsun host;
  3. Before running, or after finishing the simulation;
  4. Before or after a simulation step is executed;
  5. A vehicle enters or exits the transportation network;
  6. Before a route choice calculation is performed.

In addition to the 10 predefined call-backs (See Table 1 below)  exported by the MicroAPI DLL, there are also many utility functions exported by the Aimsun host. This renders two way communication between the MicroAPI DLL and the Aimsun host for exchanging of information and control.

As to be explained in future post,  Aimsun’s two way communication is quite different from VISSIM, another popular traffic simulator developed and marketed by the Germany company PTV AG.  VISSIM also provides various APIs, e.g., VISSIM external driver APIs,  VISSIM signal control APIs etc.   Particularly, VISSIM provides an automation mechanism by virtue of Microsoft’s COM technology.   Regardless,  in terms of APIs,  VISSIM is merely one way communication.  The pros and cons of these two types of interfaces will be discussed later.

To use the utility functions exported by the Aimsun host, “angext.lib” must be specified for the linker.  However,  for some reason,  the corresponding DLLis not named angext.dll,  but a2kernel.dll.

The following table summaries the 10 predefined call-backs.

TABLE 1. Aimsun MicroAPIs 10 Call-backs (hooks)

Build log4cxx with Visual Studio 2010

Steps for building log4cxx with VS2005 and VS2008 can be found from here
http://www.dreamcubes.com/b2/software-development/28/log4cxx-for-win32-with-vs2005/

And to build log4cxx with Visual Studio 2010, some steps are advised here
http://permalink.gmane.org/gmane.comp.apache.logging.log4cxx.user/2842

These steps are simply copied below in italics fonts for convenience:

  1. Move all those LOG4CXX_LIST_DEF macros to before the class definition. in telnetappender.h you will also need to move the typedef that precedes this macro.
  2. If the compiler complains about KeySet not being member of LoggingEvent, just remove the scope (since we moved the type to outside the class in the previous step, these types no longer are inside the class) ex: change: LoggingEvent::KeySet set; to: KeySet set;
  3. If the compiler complains about insert_iterator not being in the namespace std, add #include to the include section of the source file
  4. For some reason the VS2010 converter ‘forgot’ to reference the lib files from apr,apr-util and xml, so i had to add them manually.

I followed these steps and have the following observations:

  • Step 4 didn’t happen to me;
  • Step 1 is a little complex depending on what compiler you use. If you use Visual Studio 2010’s default VC++ compiler 10, then you’ll get error C2252 : an explicit instantiation of a template can only occur at namespace scope; however, if you use Intel C++ compiler 2011 SP1, you’ll just get a warning. After moving the LOG4CXX_LIST_DEF macros to before the class definition, Intel C++ compiler 2011 SP1 will drop the warning but pop out a new warning about explicit instantiation more than once. In other words, with Intel Compiler, Step 1 is optional (as I’ve tested, you can just simply ignore these cute warnings);
  • Step 2 is necessary for both VC++ and Intel C++ compilers;
  • Step 3 is necessary for both VC++ and Intel C++ compilers.

Interestingly, log4cxx built with Intel C++ compiler with some Intel specific optimization switches on shows some what performance improvement by 5% ~10% (this is a rough estimate based on the before-and-after comparison of some program I wrote, you mileage may vary depending on the context, use case and what particular switches).

Also Intel C++ compiler seems to generate larger binary files.   Finally, the log4cxx Visual Studio 2010 solution files (with additional settings for Intel compilers) can be downloaded from the following link, so you don’t have to go through the troubles yourself:

log4cxx-vs2010

If you download it and find it helpful, please let me know and say “Thank you!” 🙂

FORTRAN Interop with C/C++

Assuming we have the following function type defined in C, i.e.,

[cpp]

typedef int (_stdcall * SetRelTurnFlowCbFunc)(DTAEnvironment *dtaEnv,
TASession *taSesn, int iterID,
int arrSize, EdgeDynaAttrRec **edgeDynaAttrArr);
[/cpp]

This type is defined to facilitate call-back using function pointers. The actual implementation of this function is defined using FORTRAN as

[cpp]

FUNCTION SetRelTurnFlowCbFunc(dtaEnv, taSesn, iterID, arrSize, edgeDynaAttr_C_Arr)
RESULT(f_result)
!DEC$ ATTRIBUTES STDCALL :: SetRelTurnFlowCbFunc

TYPE(C_PTR), VALUE, INTENT(IN) :: dtaEnv
TYPE(C_PTR), VALUE, INTENT(IN) :: taSesn
INTEGER(C_INT), VALUE, INTENT(IN) :: iterID
INTEGER(C_INT), VALUE, INTENT(IN) :: arrSize
TYPE(C_PTR), VALUE, INTENT(IN) :: edgeDynaAttr_C_Arr
….

CALL C_F_POINTER(edgeDynaAttr_C_Arr, edgeDynaAttr_F_Arr, [arrSize])

DO n = 1, arrSize
CALL C_F_POINTER(edgeDynaAttr_F_Arr(n), aEdgeDynaAttrRec)
END DO
RETURN
END FUNCTION SetRelTurnFlowCbFunc

[/cpp]

There are several subtleties here –

  1. Use compiler directive to tell the FORTRAN compiler the calling convention. You can leave this to the default setting of the IDE you use but I always do this explicitly.
  2. When converting from a C pointer to Fortran pointer, using C_F_POINTER, a explicit array size must be specified, so that the block of memory can be correctly addressed and interpreted by the FORTRAN compiler.
  3. For C pointer of pointer, an inner loop is needed. In the above example, aEdgeDynaArrRec is declared as a Fortran pointer to some data structure. In the inner loop, each C pointer is individually converted to the F-pointer.