Tuesday, April 14, 2009

Copy Constructor Update

Stumbled upon a problem today and understood that I was not implementing the copy constructor properly.
Actually, in my post "Copy Constructor" I wrote:

For the List type, for example, the following approach would work:

class Customer  
{
private List names;

// Copy constructor.
public Customer(Customer previousCustomer)
{
names = new List(previousCustomer.names.ToArray());
}

...
}

This is not true for the list of reference type objects.

The workaround I use now is to implement a copy constructor in the reference type and create a copy of the list in the following manner:

class Customer  
{
private List customerIDs;

// Copy constructor.
public Customer(Customer previousCustomer)
{
customerIDs = new List();
foreach(ID id in previousCustomer.customerIDs)
{
customerIDs.Add(new ID(id));
}
}

...
}

The syntax could be a lot more elegant in 3.5, but this application uses the 2.0 framework.

by . Also posted on my website

Friday, April 10, 2009

TFS Disaster Update

The decision was made to log an issue with Microsoft with a low priority as the TFS is not in our production environment yet. The first response we received just in about 2 hours after logging the issue.
They were interested in the following information:

1. The installation log file;

2. Accounts used for the installation: the setup /TFS service/reports /SQL Service account. Please describe briefly about them and make sure they meet the requirements listed in the installation guide;

3. Did you follow the “prerequisites for Team Foundation Server” section to prepare for the installation? The "install SQL Server Reporting Service" section documents how to prepare reporting service. Per this doc one should not configure reporting service before TFS installation. Do you confirm this is consistent with your installation?


After that they promised to get back to us ASAP and still did not, a full business day after I provided all the required information. It's the Thursday before the Easter Friday though, so I did not hold my breath.

by . Also posted on my website

Wednesday, April 8, 2009

TFS Disaster

Today I was performing a 'pre-production' install of the TFS, which is different from the install I've done before. The first install had application and data tiers on the same computer, this one was supposed to have two tiers on different computers. Should be nice and easy since I have some TFS installation experience and a competent database admin to work together with.

And the first issue that I came across was that .Net Framework 2.0 is required by the TFS, but it was not installed. But how could I not list it under the prerequisites when I was writing the installation procedure document? Well, the first installation had both the SQL Server and TFS on the same computer. And the .Net Framework is a prerequisite for the SQL Server, so by the time TFS had to be installed it was present.

That was the easy part.

To be able to place Reporting Services on the same machine as the data tier, we used the advice from this article

Reporting Services Flexibility (Orcas RTM Only)

and edited the msiproperty.ini according to that advice.

Next problem we came across happened during the system health check. The following message was generated:

"The System Health Check has detected a problem that may cause Setup to fail.

Description
SQL Server Analysis Services is not installed."

We used the workaround and happily proceeded further. That was the easy part too.

"Workaround / Remedy
SQL Server Analysis Services is a prerequisite for Visual Studio Team Foundation Server 2008. Install a supported version of SQL Server Analysis Services. For more information about supported versions of SQL Server and Team Foundation Server prerequisites, download the most recent version of the Team Foundation Installation Guide, which is available from the Microsoft Web site.

More information
For additional information and help please refer to: http://go.microsoft.com/fwlink/?LinkId=79226"

The next error came up during the actual Team Foundation Server installation process.


---------------------------
Microsoft Visual Studio 2008 Team Foundation Server Setup
---------------------------
Error 29109.Team Foundation Report Server Configuration: SQL Reporting Services configuration encountered an unknown error. Verify that you have sufficient permissions to configure SQL Reporting Services, and try again.
---------------------------
Retry Cancel
---------------------------

And the entry in the installation log was the following


TFRSConfig - Team Foundation Server Reporting Services Configuration Tool
Copyright (c) Microsoft Corporation. All rights reserved.

Connecting to SQL Server Reporting Services. Please wait...
Invalid namespace
Querying the following Windows Management Instrumentation (WMI) path: IIS://DATASERVERNAME/W3SVC.
System.Runtime.InteropServices.COMException (0x80070005): Access is denied.

at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
at System.DirectoryServices.DirectoryEntry.Bind()
at System.DirectoryServices.DirectoryEntry.get_IsContainer()
at System.DirectoryServices.DirectoryEntries.ChildEnumerator..ctor(DirectoryEntry container)
at System.DirectoryServices.DirectoryEntries.GetEnumerator()
at Microsoft.TeamFoundation.Admin.ReportingServices.WebSiteFinder.FindBestMatch(Uri searchUri)
at Microsoft.TeamFoundation.Admin.ReportingServices.InputArgs.EnsureIISSettings()
at Microsoft.TeamFoundation.Admin.ReportingServices.ReportingServicesConfigurator.Run()
at Microsoft.TeamFoundation.Admin.ReportingServices.Program.Main(String args)

Configuring SQL Server Reporting Services failed.

04/07/09 16:56:06 DDSet_Status: Process returned 2519
04/07/09 16:56:06 DDSet_Status: Found the matching error code for return value '2519' and it is: '29109'
04/07/09 16:56:06 DDSet_Error: 2519
MSI (s) (A4!B8) [08:44:23:998]: Product: Microsoft Visual Studio 2008 Team Foundation Server - ENU -- Error 29109.Team Foundation Report Server Configuration: SQL Reporting Services configuration encountered an unknown error. Verify that you have sufficient permissions to configure SQL Reporting Services, and try again.

04/08/09 08:44:23 DDSet_Status: Commandline: "d:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\Tools\TFRSConfig.exe" /setup /install /s "DATASERVERNAME" /u "NT Authority\NetworkService" /buildInIdentity /l "1033" /verify /ignoreExistingIISArtifacts /instancename "MSSQLSERVER" /appPoolName "Classic .NET AppPool" /reportServerUri "http://DATASERVERNAME/ReportServer" /reportManagerUri "http://DATASERVERNAME/Reports" /h "DATASERVERNAME"

And this is where we got stuck. We tried pretty much every solution we could find. Every possible account was given every possible permission on the data tier machine, the reporting services were perfectly accessible from the application tier machine, I could log on to the data tier machine from every account used in the TFS installation and configure Reporting Services if I wished to, but the error still did not go away.

I have to say that we are now stuck at this point.
There's couple options I can see - first, completely rebuild the application tier machine. Make sure everything is configured before we even start the installation, everything has access and permissions. Then try installing again. Second option could be to try and install the reporting services on the application tier. This would probably slow it down a bit. And another option is to try and make Microsoft fix the problem for us. We'll make a decision tomorrow.

by . Also posted on my website

Tuesday, April 7, 2009

Full process of scanning the document and extracting data

Now the full process of scanning the document and extracting data from it is mostly finished. It does not look too complicated now, unlike it was just a few weeks earlier.

First, I create the instances of a class that will keep the extracted data (AIT3MDocument) and the class that will communicate with the scanner (AIT3MDocumentScanner) through a ReaderClass object. I explained it in some more detail in one of the earlier blog entries.

AIT3MDocument document = new AIT3MDocument();
AIT3MDocumentScanner docScanner = new AIT3MDocumentScanner();
document = docScanner.ScanDocument(_Filepath, useOCRCheckBox.Checked);

documentReader = new ReaderClass();
documentItem = new DocumentItemClass();

Next, I will extract some data that lets me identify what kind of document is in the scanner.

string documentID = documentReader.RetrieveTextItem(DOCUMENT_ID);
string documentSpecific = documentReader.RetrieveTextItem(DOCUMENT_SPECIFIC);
string mrzDocumentType = documentReader.RetrieveTextItem(MRZ_DOCUMENT_TYPE);


I can extract data such as the english spelling of the person's name through the ReaderClass object.

string name = documentReader.RetrieveTextItem(NAME);

If, for example, the document was identified as a Hong Kong ID, I know that it has a special sequence of digits on it which can be extracted too.

string nameAsNum = documentReader.RetrieveTextItem(MRZ_NAME_AS_NUMBER_1);

This sequence of digits encodes the chinese spelling of the person's name. To be able to extract that spelling was the only reason I spent all that time and effort writing my glorious wrapper for the unmanaged dll.

So, after I found out the type of the document and the sequence of digits that encodes the name, I can call the proper function:

if (!String.IsNullOrEmpty(nameAsNum))
{
TSSLWrapper.RECO_DATA recoData = new TSSLWrapper.RECO_DATA();
IntPtr ptr = TSSLWrapper.SDKCreate();
bool res = TSSLWrapper.CcnOCRsdk_HKID(ptr, nameAsNum, out recoData);

if (res)
{
document.FirstNameExtended = recoData.FirstName;
document.LastNameExtended = recoData.Surname;
}
TSSLWrapper.SDKDelete(ptr);
}

and observe the result.

Now I just populate the names into the proper fields and my job is done.

This is the simplest example. Some other types of documents I work with do not contain a sequence of digits, but rather operate with the image directly. In this case a path to the saved image has to be provided, but the output is more or less similar anyway, so there is not much difference from the development point of view.

by . Also posted on my website

Wednesday, April 1, 2009

Calling an unmanaged C++ dll from C# managed code.

I finished the task of calling the unmanaged dll functions from the C# application today. After I found out that my solution described before does not work, I spent almost two days on the task. Looking at what I've done, I can see that the whole solution takes maybe 2-3 dozen lines of code. And while I understand that lines of code is not the best way to measure productivity, I could have done it in half a day if I always chose the right way to do things. Why did it take me that long? Well, there are some reasons.

  • I have not touched C++ in the last 8 years, so every little complaint from the linker or compiler was a challenge.
  • I could not find a complete solution for my case, so I had to compile a few together: learn how to write a dll in C++, learn how to write a wrapper class, learn how to call the wrapper class from the managed code etc.
  • There are numerous solutions on the internet, but choosing the right one is the problem. Some of the approaches I tried led me nowhere but took some time.

Now, when I'm done with this bucket of excuses, here is how I solved my particular case:

First of all, I had to understand that the straightforward DllImport solution does not work. I came across a comment that suggested that when a class member has to be called from a dll, a managed wrapper dll has to be written to allow access to the class members. From the C++ sample code I had I could see that yes, a class member is called.

CcnOCRsdk *ocr;
CString code;
RECO_DATA data;
GetDlgItemText(IDC_CODE,code);
char _code[200];
WideCharToMultiByte(CP_UTF8, 0, code, -1, (char *)_code, 200, NULL, NULL);
ocr->convertHKID_Name(_code,&data)

So, I looked at this example of creating a C++ dll

Walkthrough: Creating and Using a Dynamic Link Library

and at this example of writing a wrapper class

Calling Managed Code from Unmanaged Code and vice-versa

and made my first attempt at writing a wrapper. The wrapper in my initial solution exported the member function of a wrapped class. It worked to the point where the function was being called, and then threw the 'AccessViolationException'. Here I got stuck again.

Next thing to understand was that I have to export the whole class, including the constructor. To get access to the member of the class, I would have to return a pointer to the instance of the unmanaged class, and then pass this pointer to the function, that exports the member of the unmanaged class (I hope I'm describing it properly, but I'm not completely sure). The answers to this question pointed me to the right direction.

using a class defined in a c++ dll in c# code

After that, it was really simple, because I only had to apply my DllImport skills. So here we go:

First step, create a C++ DLL project in Visual Studio 2005.

  • Start Visual Studio
  • From the File menu, select New and then Project….
  • From the Project types pane, under Visual C++, select Win32.
  • From the Templates pane, select Win32 Console Application.
  • Choose a name for the project and enter it in the Name field. Choose a name for the solution, and enter it in the Solution Name field.
  • Press OK to start the Win32 application wizard. From the Overview page of the Win32 Application Wizard dialog, press Next.
  • From the Application Settings page of the Win32 Application Wizard, under Application type, select DLL if it is available or Console application if DLL is not available.
  • From the Application Settings page of the Win32 Application Wizard, under Additional options, select Empty project.
  • Press Finish to create the project.

There were some setting I had to change for my project in Project->Properties:

  • Under General->Project Defaults, set Use of MFC to 'Use MFC in a Shared DLL' as the unmanaged code was using MFC
  • Under General->Project Defaults set Common Language Runtime support to 'Common Language Runtime Supportj(/clr)'
  • Under Linker->Input->Additional Dependencies enter the name of the lib file for the unmanaged dll.

Second step, write a wrapper class for the unmanaged dll.

This is how the unmanaged class looks like:

class CNOCRSDK_API CcnOCRsdk {
public:
CcnOCRsdk(void);
bool convertHKID_Name(char *code,RECO_DATA *o_data); //hkid
//more member functions
~CcnOCRsdk();
private:
RECT *regionList;
RECT *chRect,*enRect;
//more member functions
};

This is the wrapper class I wrote, that exports the class constructor and one of the member functions:

include "stdafx.h"
#using
#include "cnOCRsdk.h"

using namespace System::Runtime::InteropServices;
using namespace System;

namespace TSSL
{
public class __declspec(dllexport) Wrapper
{
public:
CcnOCRsdk* SDKCreate()
{
return new CcnOCRsdk();
}

bool CcnOCRsdk_HKID(CcnOCRsdk* pSDK, char *code, RECO_DATA *o_data)
{
return pSDK->convertHKID_Name(code, o_data);
}

void SDKDelete(CcnOCRsdk* pSDK)
{
delete pSDK;
}
};
}

The __declspec(dllexport) at the class level exports all public class member in the dll. If it was applied on the member level, it would only export the member it was applied to.

Third step, run the Dumpbin utility and find out what are the 'mangled' names of the functions exported by the dll.

          1    0 00001240 ??4Wrapper@TSSL@@QAEAAV01@ABV01@@Z = __t2m@??4Wrapper@TSSL@@QAEAAV01@ABV01@@Z ([T2M] public: class TSSL::Wrapper & __thiscall TSSL::Wrapper::operator=(class TSSL::Wrapper const &))
2 1 00001220 ?CcnOCRsdk_HKID@Wrapper@TSSL@@QAE_NPAVCcnOCRsdk@@PADPAURECO_DATA@@@Z = __t2m@?CcnOCRsdk_HKID@Wrapper@TSSL@@QAE_NPAVCcnOCRsdk@@PADPAURECO_DATA@@@Z ([T2M] public: bool __thiscall TSSL::Wrapper::CcnOCRsdk_HKID(class CcnOCRsdk *,char *,struct RECO_DATA *))
3 2 00001200 ?SDKCreate@Wrapper@TSSL@@QAEPAVCcnOCRsdk@@XZ = ?SDKCreate@Wrapper@TSSL@@QAEPAVCcnOCRsdk@@XZ (public: class CcnOCRsdk * __thiscall TSSL::Wrapper::SDKCreate(void))
4 3 00001410 ?SDKDelete@Wrapper@TSSL@@QAEXPAVCcnOCRsdk@@@Z = ?SDKDelete@Wrapper@TSSL@@QAEXPAVCcnOCRsdk@@@Z (public: void __thiscall TSSL::Wrapper::SDKDelete(class CcnOCRsdk *))

Fourth step, write another wrapper, now in C#, which will define the DllImports for the unmanaged functions and the structures required to call them

public class TSSLWrapper
{
[StructLayoutAttribute(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct RECO_DATA
{
/// wchar_t[200]
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 200)]
public string FirstName;
/// wchar_t[200]
[MarshalAsAttribute(UnmanagedType.ByValTStr, SizeConst = 200)]
public string Surname;
}

[DllImport(@"TSSL.dll", EntryPoint = "?SDKCreate@Wrapper@TSSL@@QAEPAVCcnOCRsdk@@XZ")]
public static extern IntPtr SDKCreate();
[DllImport(@"TSSL.dll", EntryPoint = "?CcnOCRsdk_HKID@Wrapper@TSSL@@QAE_NPAVCcnOCRsdk@@PADPAURECO_DATA@@@Z")]
public static extern bool CcnOCRsdk_HKID(IntPtr ptr, string num, out RECO_DATA o_data);
}

RECO_DATA is the structure I have to send from C# to C++. Note how the SDKCreate returns the IntPtr which is, in my understanding, a pointer to the instance of the unmanaged class. To call a member function of this class, I pass this pointer to the function.

And fifth and last step, is to call the C# wrapper class from the C# application and enjoy the results.

TSSLWrapper.RECO_DATA recoData = new TSSLWrapper.RECO_DATA();
string num = "262125355174";
IntPtr ptr = TSSLWrapper.SDKCreate();
bool res = TSSLWrapper.CcnOCRsdk_HKID(ptr, num, out recoData);

(To make the solution complete, I should also wrap the destructor and call it after I don't need the class any more, I'll do it soon)

Here we go, my longest post ever has arrived.

by . Also posted on my website

Sunday, March 22, 2009

Copy Constructor

It's strange, but I did not have to deal with copy constructors in C# until today. Here is the problem: in the application I'm working on, a user can retrieve customer's details from the database, change some values on several separate tabs, and then choose to save or discard changes. Now, discarding did not work properly till today. The problem is that the old values were not saved anywhere, so no way to access them was implemented.

Two approaches came to my mind: First, to reload data from the database, and second, to keep a 'backup' copy of the customer. Since an extra trip to the database takes several seconds, the 'backup' approach was chosen.

Now, when the customer details are retrieved from the database, they are immediately copied into the 'backup' customer object. Obviously, if you try to create a copy like this

Customer cust = SelectCustomerFromDB(id);
Customer cust2 = cust;

You will end up with two references to the same object. Now, if you change anything in 'cust' object, your 'cust2' is useless, cause it references the same object and will reflect the changes. This MSDN page gives a brief idea of the proper way to create a copy of the object

How to: Write a Copy Constructor (C# Programming Guide)

The above sample would now look more like this

class Customer
{
private string name;

// Copy constructor.
public Customer(Customer previousCustomer)
{
name = previousCustomer.name;
}

...
}

...

Customer cust = SelectCustomerFromDB(id);
Customer cust2 = new Customer(cust);

What if your customer has a list of names though?

class Customer
{
private List names;

// Copy constructor.
public Customer(Customer previousCustomer)
{
names = previousCustomer.names;
}

...
}

...

Customer cust = SelectCustomerFromDB(id);
Customer cust2 = new Customer(cust);

Now try changing something in cust.names. Ooops, the cust2.names reflected the changes. Unfortunately, the article linked above only shows the approach to copy the object with members of value types. Now, if you have members of reference types in your class, you would have to copy each one properly.

For the List type, for example, the following approach would work:

class Customer
{
private List names;

// Copy constructor.
public Customer(Customer previousCustomer)
{
names = new List(previousCustomer.names.ToArray());
}

...
}

I came across a few smart ways to implement the copy constructor so that you would not have to specify each class field separately, like in this exapmle

An Intelligent Copy Constructor In C# Using Reflection

public MyClass( MyClass rhs )
{
// get all the fields in the class
FieldInfo[] fields_of_class = this.GetType().GetFields(
BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance );

// copy each value over to 'this'
foreach( FieldInfo fi in fields_of_class )
{
fi.SetValue( this, fi.GetValue( rhs ) );
}
}

This is great, because if you add a new field to the class and forget to add it to a copy constructor, your code will compile without warnings but you will get troubles sooner or later.

Unfortunately, this approach also works for value type fields only so I could not benefit from it. I could not quickly find a way to use this approach to reference type class members so I'll leave it for myself as a TODO task.

by . Also posted on my website

Thursday, March 19, 2009

DllImport Adventures

More fun with the scanner. Now we want to get the first name and last name in chinese from the certain types of passports. There is a dll written in C++ which supposedly provides this functionality. The function in this dll takes a path to an image as a parameter and returns a structure that contains first name and last name.

Here is the C++ signature of the function


bool recoCHN_P_Name(char *imgPath,RECO_DATA *o_data);

Here is the RECO_DATA struct

struct RECO_DATA{
wchar_t FirstName[200];
wchar_t Surname[200];
};

Now I have to call the C++ method from my C# code. The 'DllImport' thingy comes to mind. There are heaps of tutorials on that, for example

Call Unmanaged Code. Part 1 - Simple DLLImport

So I quickly come up with the first version:

This is the 'wrapper' class

public class cnOCRsdk
{
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct RECO_DATA{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst=200)]
public string FirstName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 200)]
public string Surname;
}

[DllImport(@"cnOCRsdk.dll", EntryPoint="recoCHN_P_Name")]
public static extern bool recoCHN_P_Name(byte[] imgPath, RECO_DATA o_data);
}

This is the call to the function in the wrapper class

cnOCRsdk.RECO_DATA recoData = new cnOCRsdk.RECO_DATA();

string path = @"C:\WINDOWS\twain_32\twainrgb.bmp";

System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
byte[] bytes = encoding.GetBytes(path);

bool res = cnOCRsdk.recoCHN_P_Name(bytes, recoData);

Unfortunately, this gives me a "Unable to find an entry point named 'recoCHN_P_Name' in DLL 'cnOCRsdk.dll'." error. My first guess was that I am converting types from C++ to C# incorrectly. I asked for some help

Unable to find an entry point named [function] in dll

I got a suggestion to use the 'dumpbin' tool

DUMPBIN

and also a link to the page that explained C++ name mangling

C++ Name Mangling/Demangling

my function had a mangled name of '?recoCHN_P_Name@CcnOCRsdk@@QAE_NPADPAURECO_DATA@@@Z', but how did that help me?

Anyway, I came across another article on calling C++ from C#

Inheriting From a Native C++ Class in C#

and noticed an interesting thing in it:

[DllImport("cppexp.dll", EntryPoint = "?M1@CSimpleClass@@QAEXXZ", CallingConvention = CallingConvention.ThisCall)]
private static extern void _M1(__CSimpleClass* ths);

this guy is using the mangled function name as an entry point! Why shouldn't I try this approach?

Long story short, the 'unable to find entry point' error went away and after a few more tweaks I made this work. Unfortunately, the function always returns me false so far, which is probably because all the images I try are 'bad', but at least it does not break anymore.

public class cnOCRsdk
{
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct RECO_DATA
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 200)]
public string FirstName;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 200)]
public string Surname;
}

[DllImport(@"cnOCRsdk.dll", EntryPoint="?recoCHN_P_Name@CcnOCRsdk@@QAE_NPADPAURECO_DATA@@@Z")]
public static extern bool recoCHN_P_Name(ref string imgPath, ref RECO_DATA o_data);
}
cnOCRsdk.RECO_DATA recoData = new cnOCRsdk.RECO_DATA();
recoData.FirstName = new string(new char[200]);
recoData.Surname = new string(new char[200]);

string path = @"C:\WINDOWS\twain_32\twainrgb.bmp";

bool res = cnOCRsdk.recoCHN_P_Name(ref path, ref recoData);
by . Also posted on my website