SQL XML Bulk Load (SQLXMLBULKLOADLib) is STA

This is a reminder to myself the next time I have problems getting the SQLXMLBULKLOAD library to work. According to several articles online, the bulk load capabilities Microsoft provides through this library only work in a single threaded environment. You can read a bit more about it over at sqlxml.org.

The basic gist is this. When you want to run the bulk load utility in another thread, specify to that thread that it should run as a single threaded apartment, like this:

System.Threading.Thread.CurrentThread.ApartmentState =
System.Threading.ApartmentState.STA;

You can also specify it when you create a thread by doing this:

System.Threading.Thread bulkLoaderThread
= new Thread( new ThreadStart( BulkLoadProc) );
bulkLoaderThread.ApartmentState = System.Threading.ApartmentState.STA;
bulkLoaderThread.Start();

For anyone who has never used this library before you may want to take a look at it. It provides a great way to take large amounts of XML and load them into a database. The usage of the library is as simple as instantiating the object, setting a few parameters, and then calling Execute() which takes the path to your schema file that provides mapping between XML fields and database fields, and a the path to the actual XML file you want to load.The library is limited in that it doesn’t provide any sort of record by record reporting or event hooking, however, it is a great tool for when you are sure you have valid XML going in (e.g. it expects you’ve already validated the XML and data integrity).

SQL XML 3.0 SP3 Download

Resetting a .NET Installation

Recently I had to re-install an application I had written using C# on a Windows® 2000 server. The server had been restored after a crash from a system backup. The issue was that the registry had been corrupted and many keys had been destroyed altogether.

Once I got onto the server I realized that there were several components I use that needed re-installed. I decided to start with Windows® Update. While there I realized that the .NET framework 1.1 had not been updated. I went ahead with that update and rebooted. When the system came back online, I went back into Windows® Update and realized it was again reporting that the .NET framework 1.1 still needed installed.

To make a long story short, it turns out that I needed to force Windows® update to see the .NET framework as really uninstalled. I started with trying to install using the dotnetfx.exe redistributable file and then removing that, but that didn’t work. In the end it boiled down to removing one file. This file is called mscoree.dll. It is located in ‘C:\WINDOWS\system32’. I simply moved the file from this location to a temporary folder (in case it broke something and I needed to put it back). Once this was done, Windows update saw it as “really” uninstalled and actually re-installed everything.

So, should you find yourself needing to re-install the .NET framework after a system recovery, remember that the easiest way is to simple move the mscoree.dll file and you’ll save yourself a lot of time.

Note: I’m not sure if this is the same with .NET framework 2.0. You’ll have to try it out and let me know should you need to do this on a system that uses .NET 2.0.

Microsoft Love/Hate or Love to Hate

For a long time I have seen Microsoft as being evil in tactics but good in what they provide for developers. I use Microsoft’s tools to do my job and find that the majority of my experience doing so is positive. This is why I consider that I have a love/hate relationship with them.

The other day however, I was working and got notified that I needed to install an update from Microsoft, so I went ahead and had it do the install. Keep in mind that I was in the middle of doing something when I was notified. When the install had finished it asked me if I wanted to restart now or restart later. I clicked “Restart Later” and believed that would be the end of it. Within five or ten minutes I was getting notified again with the exact same question. “Restart Now” or “Restart Later”. This was getting very irritating.

Now, if you find yourself asking, “why didn’t you just restart?”, well then you’ve never been in my situation before. I was running and debugging code and trying to fix a problem with my work. When my mind gets into it, it is counter productive to pull away and have to wait. My mind gets on other things and I don’t tend to return to my work with the same vigor as I had before

After several hours of being notified every couple minutes, I realized I needed to download a large amount of data, so I started that process. I had to leave for a while and figured when I came back the download would be finished.

Bwah haa haa haaa haaaa!!! (I can hear Steve Ballmer laughing out loud).

There’s going to be none of that!! You think you have control over your own computer? Well, you don’t. And apparently Microsoft thinks you shouldn’t. When I returned to my computer later, it had restarted without getting the ok from me!!!! What the *&%#$!!!

I suppose part of the problem is that Microsoft doesn’t like getting blamed for security problems and so they enforce that you restart and accept the update, but this is just ridiculous. At least give me a third option on the reminder dialog that says “I will restart my computer as soon as I’m done with my work, I promise.” next to the “Restart Now”, and “Restart Later” buttons. C’mon Microsoft, I love the dev tools, but this crap makes me love to hate you!

C# and the ‘ref’ Keyword

Lately I’ve been thinking about the nuances of the C# language. A friend of mine and I were talking about the merits of C# as compared to Java. Of course, I don’t want to open that can of worms here, however I did realize that I appreciate small details in the C# language that are not present in Java. One of those details is the keywords used in method parameter lists. These include ‘ref’, ‘in’, and ‘out’. The reason these are significant is that in both C# and Java, how are parameters passed? Well we’ve all been told that, of course, parameters are passed by reference and so we go along happily with this information because most of the time for all intents and purposes it is true. But the deeper truth of the matter is that while references are passed, the are actually passed by value.

Here’s is some code to prove it:

public class Person
{
    protected string name;
    public string Name 
    {
        set { this.name = value; }
        get { return this.name;}
    }
    protected string email;
    public string Email 
    {
        set { this.email= value; }
        get { return this.email;}
    }
    public override string ToString()
    {
        return this.name + ", " + this.email;
    }
}

public static void main ( string[] args )
{
    Person p = new Person();
    p.Name = "Matt Long";
    p.Email = "matt.long@matthew-long.com";
    System.Console.WriteLine( p.ToString() );

    ChangeMyMembers( p, "Bob Smith", "bob@yyyyy.com" );
    System.Console.WriteLine( p.ToString() );

    ChangeMe( p, "Wendy Johnson", "wendy@yyyyy.com" );
    System.Console.WriteLine( p.ToString() );

}
public static void 
    ChangeMyMembers( Person p, string name, string email )
{
    p.Name = name;
    p.Email = email;
}

public static void 
    ChangeMe( Person p, string name, string email )
{
    p = new Person();
    p.Name = name;
    p.Email = email;
}

If you were to run this program, what would you expect the output to be? Here is the actual output:

Matt Long, matt.long@matthew-long.com
Bob Smith, bob@yyyyy.com
Bob Smith, bob@yyyyy.com

Is that what you expected? Well, since this whole post is about the parameter keywords, I would expect your answer is no. But what happened?

When we passed the Person reference to the ChangeMe() method we passed a copy of the reference. That copy is now only available in the method scope. If we change that reference, it won’t change the original–only that local copy. So when we call p = new Person(); we have assigned that local copy of the reference to point to an actual place in memory, however, it goes away as soon as we leave the method.

Now, to my knowledge there is nothing you can do to address (no pun intended) this behavior in Java directly. In C# however, enter the keyword ‘ref’. If we simply change the ChangeMe() method signature to look like this:

public static void
    ChangeMe( ref Person p, string name, string email )

And then when we call ChangeMe(), we also use the ref keyword like this:

ChangeMe( ref p, "Wendy Johnson", "wendy@yyyyy.com" );

We now get the output we expected:

Matt Long, matt.long@matthew-long.com
Bob Smith, bob@yyyyy.com
Wendy Johnson, wendy@yyyyy.com

I am not by my comments here going to enter into a “which language is better” holy war, however, the parameter keywords in C# are a nice feature. Here is an excercise for the reader. Can you implement a swap method in Java? If so, how? How would you do it in C#?

If you need some pointers for the Java way, you can start here: http://www.javaworld.com/javaworld/javaqa/2000-05/03-qa-0526-pass.html. Then do an Internet search. (Here’s a little hint: you would have to encapsulate the parameters in a class that can be de-referenced in the swap method.)

If you want to learn more about the nuances of C#, a good way to do so is to read some interview questions. Take a look at these here.

Mac OS X86 Project

For a long time I have loved the Macintosh and the Mac OS platform. It has just been difficult for me to ever justify owning a Mac because the work I do is all Microsoft Windows based.

Recently there has been a flurry of posts over at Digg and other technology news sites that discuss how to install the Mac OS on Intel hardware. This, of course, has been made possible first and foremost by Apple, when they released development computers to premium members of the Apple Developer Network, and then ultimately by hacker types in the community at large who were able to circumvent the security technology that was intended to keep users from installing the OS on any arbitrary Intel based hardware and then making available a disc image of the installation.

After reading several posts, I decided to try and see if I was able to get the Mac OS running on my own Dell laptop. The main site with information about the MacOS X86 port is of course http://www.osx86project.org/ , however, there are others. I found the necessary files and created the install disc. I then purchased a secondary hard drive and bay for my laptop and ran the install last night. It worked very well. Out of the box I had networking support and audio support.

Software such as most of the iLife suite has to run using a technology named Rosetta which provides an emulation layer. This makes it quite inefficient to use most of those applications since they are already resource hogs without the emulation, so I am finding that I can’t really see using it as a permanent setup. I will purchase one of the MacIntel computers when they are available, but this has been a great way to get to know the Mac OS.

I also plan to start learning how to write code for the Mac when I get some free time. This is a great way to do it without having to make the initial investment yet. The time will come for that and when it does, I’ll be ready to start building applications for the excellent Mac OS X platform.

C# Memory Pinning: Convert a byte array to a data structure

C# Memory Pinning: Convert a byte array to a data structure. Have you ever wanted to simply typecast a byte array to a data structure and then use that data structure like you used to be able to do in C? Well, C# provides a facility to accomplish this.

This solution is several months old to me now, but I feel it is something very worthwhile for C# programmers to know.

Have you ever wanted to simply typecast a byte array to a data structure and then use that data structure like you used to be able to do in C? Well, C# provides a facility to accomplish this. Several months ago I needed to take a byte array 64 bytes long and extract different fields from it. At first it seemed that the only solution was to convert the byte array to a string and then use substrings to get what I needed. To add insult to injury, I then had to take each of the substrings and convert them into the data types that I wanted and then assign the members of my data structre to each of those. Something ain’t right about that!!

Now, being the stubborn programmer that I am, I was not satisfied with this solution as I used to have the ability to just typecast a byte array into whatever structure I wanted in C and dangit newer languages such as C# shouldn’t remove features. They should enhance existing ones along with introducing new ones.

Enter memory pinning. Now, I can’t take credit for finding this on my own. It was a tough find and I was only most of the way there when a co-worker of mine found the answer. Here is a code snippet to show how it’s done:

public static object RawDataToObject(ref byte[] rawData,
		Type overlayType) 
{ 
	object result = null;




	GCHandle pinnedRawData = GCHandle.Alloc(rawData,
		GCHandleType.Pinned);
	try
	{
	
		// Get the address of the data array
		IntPtr pinnedRawDataPtr = 
			pinnedRawData.AddrOfPinnedObject(); 

		// overlay the data type on top of the raw data
		result = Marshal.PtrToStructure(
			pinnedRawDataPtr ,
			overlayType);
	}
	finally
	{
		// must explicitly release
		pinnedRawData.Free(); 
	} 

	return result;
} 

Becaues the garbage collector will normally have the ability to change the location of where we have our object allocated in memory, we have to actually pin it by calling GCHandle.Alloc() passing it a handle type of GCHandleType.Pinned.

In the application I was working on I knew that the byte array would always be the same size. If, however, it had been a variable size, then we would have to ensure that the size of the overlay type is the same as the length of the byte array.

Now that we know how to actually do the conversion from byte array to data type, we should probably look at the data types themselves. The data types and their members have to be explicitly set to their correct sizes in the code. The following is an example structure:

[StructLayout(LayoutKind.Sequential, Size=64, Pack=1,
		CharSet=CharSet.Ansi)]
public class MyDataStructure
{
	[ MarshalAs( UnmanagedType.ByValArray, SizeConst=40 )]
	public byte[] name;
    
	public byte eyeColor;
	public byte hairColor;
	public Int16 age; // Two byte integer
	public Int16 height;
	public Int16 weight;
	public Int16 noOfPets;
	public byte sign;

	[ MarshalAs( UnmanagedType.ByValArray, SizeConst=3 )]
	public byte[] birthDate; //yymmdd

	[ MarshalAs( UnmanagedType.ByValArray, SizeConst=10 )]
	public byte[] unused;
	
}

There are several items that need to be explained here. First off, our structure is actually a class. Both classes and structs can be used. Classes just need to be treated as structs and include no methods. Next, notice our StructLayout attribute. Here are the meanings of each of the fields used in this attribute:

  • LayoutKind.Sequential: Tells the compiler that the fields in the structure should be allocated in the same order they are laid out in code
  • Size: This is the total size of the structure. When this is set, you must ensure that the fields all add up to this number.
  • Pack: This is the packing size that the compiler should use to allocate memory for the structure. The default is 8, so if you left this blank, Pack would would be set to 8. In this case, however, we have set the pack to 1, ensuring that memory for the structure is all allocated together. Pack is only used when the LayoutKind is set to Seqential.
  • CharSet: Indicates how the data fields should be marshaled.

Next, look at each of the fields in the data structure. We are representing three different data types in this particular code snippet. They include a single byte a two byte integer and byte arrays which are normally used for strings. To indicate that a field should be marshaled as an array, we have to use the MarshalAs attribute specifying the size the array should be in the attribute rather than in the delcaration code.

Now, when we put this all together, it looks something like this:

// We assume GetNextRecord retuns the 64 byte array
byte[] arrayOfBytes = GetNextRecord();
MyDataStructure mds = (MyDataStructure)RawDataToObject( 
		ref arrayOfBytes, typeof( MyDataStructure ) );

// Now we can reference the data structure
// however we would like
string name = ASCIIEncoding.ASCII.GetString( mds.name );
string age = mds.age.ToString();
...

This solution works great if you have to read in a fixed length data record from either a binary file or even some hardware device. It’s a very cool technique (if you ask me).