Debugging Multi-Thread Application

Most of the application I develop are running more than one thread with different task for each thread. It might sound ok if the threads are running properly, all the critical sections are not dead locked, and memory are well controlled.

But, sometimes due to deadlines, we forget some parts of the codes and it leads to application crash, memory leak, and the most difficult part is data race problem. These are sorted based on difficulty in debugging.

These several months, I've been struggling hard to read others' code, interpret what they are trying to achieve with the code, how the threads work, how do they synchronize, and how to add another feature to the existing program. And here because it's almost the end of the year, I want to summarize what I learned so far this year.

1. Application crash is the easiest to debug among all of the three problems I mentioned above. This can be traced quite easily by setting debug points on object creations, every function calls, and object destructions. Usually the error messages gives a clue on what to debug, it takes experience to know what the problem is from the error messages. The more mistakes you make, the more experience you have in interpreting error codes :), but this is not a justification for making mistakes.
2. Memory leaks is more difficult when it comes to multi threaded applications because the problem might spread with the thread instantiation. The only way to debug this is by checking the source code line by line, making sure that there's always a delete or delete [] for every new or new[].
3. The data race can be traced by checking all the critical sections and writing a log file for every threads. This is difficult to check because the more threads in the software the more complicated the debugging process is. It become more complicated when the thread is dealing with arrays. Some compilers are not very good in debugging multi threaded application, because the application itself might crash. So, what I did is writing the algorithm into dll, then write some debugging string then use another software to catch the debug strings. In this way, the main threads are not interrupted during debugging and it's much easier to debug the software. The software I am using is called DebugView (https://technet.microsoft.com/en-us/sysinternals/debugview.aspx). It's very convenient to use in debugging algorithm, especially when it is run in thread.

Multiple Webcam using OpenCV 2.1 and Visual Studio 2008

I just found out that it is not difficult to connect to two webcam using OpenCV and do image processing on the captured images. I used OpenCV 2.1 and Visual C++ 2008 for this program, so those who are familiar with Visual C++ 2008 should be familiar with this code. The installation and configuration of OpenCV is explained clearly in their wiki and you can follow the steps written there.

We need to create two capture objects, one for each camera:

CvCapture* capture;
CvCapture* capture2;

Also, create two IplImage objects for the two cameras:

IplImage* image;
IplImage* image2;

The CvCapture and IplImage objects have to be initialized before use, add a button to your form and add these initialization lines:

int w = 640;
int h = 480;

// creating the capture fwom webcam #1
capture = cvCreateCameraCapture(0);
capture2 = cvCreateCameraCapture(1);

// parameter setting
cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, w);
cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, h);

cvSetCaptureProperty(capture2, CV_CAP_PROP_FRAME_WIDTH, w);
cvSetCaptureProperty(capture2, CV_CAP_PROP_FRAME_HEIGHT, h);

// initialization of iplimage object
image  = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 3);
image2 = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 3);

Then the next thing is to declare a function for capturing the images:

private: System::Void ProcessFrame(System::Object^ sender, System::EventArgs^ e)
{              // query the frame and capture
image = cvQueryFrame(capture);
image2 = cvQueryFrame(capture2);

cvShowImage("camera 1", image);
cvShowImage("camera 2", image2);
  
if(blnGrabImage)
{
cvSaveImage("image_left.bmp", image);
cvSaveImage("image_right.bmp", image2);
blnGrabImage = false;
}
}

We have to register this function to be called by the system, add these lines after the initialization of the capture and the images:

Application::Idle += gcnew EventHandler(this, &OpenCVImage::Form1::ProcessFrame);

There are two important parts in this line, the first one is

Application::Idle += gcnew EventHandler

which tell the system to add an event handler to the system when the system is idle and the other thing is this part:

&OpenCVImage::Form1::ProcessFrame

which tells the system to call this function every time there's a new event in the system, in this case OpenCVImage is the name of the project, Form1 is the name of the Form, and ProcessFrame is the function that is going to be called.

So, that's it, tidy up the code a little bit and ready to run. If you're successful you'll get something like this:
1. For single camera:
2. For multiple cameras:
- left camera:
- right camera:
Left and right are seen from the objects' viewpoint facing the camera...

Good luck!

Using Euresys Multicam Library

There are several different frame grabbers and industrial cameras available in my Lab for experiment. One of them is from Euresys (sorry, this is not an advertisement) and there are several other brands. However most of the time I'm dealing with this card either with analog cameras (standard analog interface) or digital cameras (with cameralink interface).

I found that this library is very convenient to use. The disadvantage is that like other available SDK's, I found that the guide of the SDK is not quite human readable. So I was trying hard to understand how to use this, and happens to be after a lot of attempts (try, fail, ask support, fail, try again, etc.) I can understand some this library.

Therefore I want to write how to use this library in this blog. So if I have to use it again in the future, I can read it from this blog and so can you :). If there's some part that is not correct, please give a feedback to me. Thanks.

The process of using this library is divided into three parts:

1. Initialization, includes the initialization of the card(s), camera(s) and the callback function.

2. Image acquisition.

3. Stop image acquisition and memory cleanup.

Let's start from the first part, initialization. Depending on your coding platform, you can write your own code. I'm using C++ on a Borland C++ Builder compiler, if you're using Visual C++ then you might need to adjust this code to your coding platform. I use the sample code provided from Euresys.

• Board and camera initialization

1. Initialization of driver

McOpenDriver(NULL);

2. Setting error message configuration

// Activate message box error handling

McSetParamInt(MC_CONFIGURATION, MC_ErrorHandling, MC_ErrorHandling_MSGBOX);

McSetParamStr(MC_CONFIGURATION, MC_ErrorLog, "error.log");

3. Setting board topology (check your board)

McSetParamInt(MC_BOARD + 1, MC_BoardTopology, MC_BoardTopology_MONO);

4. Creating a channel

McCreate(MC_CHANNEL, &m_Channel);

McSetParamInt(m_Channel, MC_DriverIndex, 1);

• Creating a connection to a camera

1. Choose the channel for the camera (depends on the port).

McSetParamStr(m_Channel, MC_Connector, "M");

2. Select the cam file, check your camera

McSetParamStr(m_Channel, MC_CamFile, "VCC-G22V31CL_P120SC");

3. Set the exposure time

McSetParamInt(m_Channel, MC_Expose_us, 10000);

4. Set the color format

McSetParamInt(m_Channel, MC_ColorFormat, MC_ColorFormat_Y8);

5. Set the acquisition mode

McSetParamInt(m_Channel, MC_AcquisitionMode, MC_AcquisitionMode_HFR);

• Setting the trigger mode

1. Choose the way the first acquisition is triggered

McSetParamInt(m_Channel, MC_TrigMode, MC_TrigMode_IMMEDIATE);

2. Choose the triggering mode for subsequent acquisitions

McSetParamInt(m_Channel, MC_NextTrigMode, MC_NextTrigMode_REPEAT);

3. Choose the number of images to acquire

McSetParamInt(m_Channel, MC_SeqLength_Fr, MC_INDETERMINATE);

• Setting up the frame parameters

1. Retrieve image dimensions

McGetParamInt(m_Channel, MC_ImageSizeX, &m_SizeX);

McGetParamInt(m_Channel, MC_ImageSizeY, &m_SizeY);

McGetParamInt(m_Channel, MC_BufferPitch, &m_BufferPitch);

2. The memory allocation for the images is automatically done by Multicam when activating the channel. We only set the number of surfaces to be created by MultiCam.

McSetParamInt(m_Channel, MC_SurfaceCount, EURESYS_SURFACE_COUNT);

• Enabling Multicam Signals

1. Enable MultiCam signals

McSetParamInt(m_Channel, MC_SignalEnable + MC_SIG_SURFACE_PROCESSING, MC_SignalEnable_ON);

McSetParamInt(m_Channel, MC_SignalEnable + MC_SIG_ACQUISITION_FAILURE, MC_SignalEnable_ON);

2. Register the callback function

McRegisterCallback(m_Channel, GlobalCallback, this);

• Preparing bitmap info

1. Build bitmap info Y8

m_pBitmapInfo = (BITMAPINFO *) new BYTE[sizeof(BITMAPINFO) + 255*sizeof(RGBQUAD)];

m_pBitmapInfo->bmiHeader.biSize = sizeof(BITMAPINFOHEADER);

m_pBitmapInfo->bmiHeader.biPlanes = 1;

m_pBitmapInfo->bmiHeader.biBitCount = 8;

m_pBitmapInfo->bmiHeader.biCompression = BI_RGB;

m_pBitmapInfo->bmiHeader.biSizeImage = 0;

m_pBitmapInfo->bmiHeader.biXPelsPerMeter = 0;

m_pBitmapInfo->bmiHeader.biYPelsPerMeter = 0;

m_pBitmapInfo->bmiHeader.biClrUsed = 0;

m_pBitmapInfo->bmiHeader.biClrImportant = 0;

for (int i = 0 ; i <>

{

m_pBitmapInfo->bmiColors[i].rgbBlue = (BYTE)i;

m_pBitmapInfo->bmiColors[i].rgbGreen = (BYTE)i;

m_pBitmapInfo->bmiColors[i].rgbRed = (BYTE)i;

m_pBitmapInfo->bmiColors[i].rgbReserved = 0;

}

m_pBitmapInfo->bmiHeader.biWidth = m_BufferPitch / (m_pBitmapInfo->bmiHeader.biBitCount/8) ;

m_pBitmapInfo->bmiHeader.biHeight = -(int)m_SizeY ;

• Declaring the callback function

void WINAPI GlobalCallback(PMCSIGNALINFO SigInfo)

{

if (SigInfo && SigInfo->Context)

{

TForm1* pTForm1 = (TForm1*) SigInfo->Context ;

pTForm1->Callback(SigInfo);

}

}

• Callback function

void TForm1::Callback(PMCSIGNALINFO SigInfo)

{

if (SigInfo->Signal == MC_SIG_SURFACE_PROCESSING)

{

// Update "current" surface address pointer

McGetParamInt(SigInfo->SignalInfo, MC_SurfaceAddr, (PINT32) &m_pCurrent);

//----------------------------------------

//

// Insert the eVision code here.

//

//----------------------------------------

// Post screen refresh message

RECT recpict;

recpict.left = 0;

recpict.top = 0;

recpict.right = m_SizeX-1;

recpict.bottom = m_SizeY-1;

InvalidateRect(Handle, &recpict, false);

}

else if (SigInfo->Signal == MC_SIG_ACQUISITION_FAILURE)

{

StatusBar1->SimpleText = m_StatusBarText.sprintf("Frame Rate: %.2f, Channel State: IDLE", 0);

MessageBox(NULL, "Acquisition Failure !", "PicoloVideoTrigger", MB_OK);

}

}

• Start and Stop Grabbing

1. Start an acquisition sequence by activating the channel

McSetParamInt(m_Channel, MC_ChannelState, MC_ChannelState_ACTIVE);

2. Stop an acquisition sequence by deactivating the channel

McSetParamInt(m_Channel, MC_ChannelState, MC_ChannelState_IDLE);

• Deactivating Channel and Cleaning Up

1. Set the channel to IDLE before deleting it.

McSetParamInt(m_Channel, MC_ChannelState, MC_ChannelState_IDLE);

2. Delete the channel

McDelete(m_Channel);

3. Terminate driver

McCloseDriver();

4. Delete bitmap info

if (m_pBitmapInfo) delete m_pBitmapInfo;

That's all, I will post the details in the next article because it's too long to be posted here.