Reorganize a Visual Studio Project

I have been working on this project for a month. When I started, I didn't know anything about it and was inexperienced in .NET development. As a result, I created a project file structure that I do not like and will not work as the effort, and team, grows. This week is the week to get a code repository up and to create a testing/demo web server. That means it is also time to organize the files correctly.

What I want to do is simple. I want to rename my projects from their idiosyncratic starter names to, and I quote, UserInterface, DatabaseModel, TestProject. I want to move them out of their default directory and into one that is named after the project and is structured so that I can manage other things. The initial structure is to be, SourceCode (home to the three named above), SharedSoftware (home to nHibernate, mbUnit, etc), and ManagementFiles (home to stuff that arguably should not be in source control, especially Virtual Studio control files).

This would be, in my old world, about two hours work, and that would include reconfiguring Apache to point at the new directories. In my new world, I have just completed the chore and it has taken a full day, a good eight hours. 

Among the reasons for this is the fact that, in true Microsoft fashion, they make no provisions for user flexibility. Specifically, they knit VS projects together with a "solution" file (.sln) that is hard-coded into the file system. It cannot be moved, nor can the code it manages. Worse, it insists on storing it in the middle of the source code. Anyone else that downloads the project is stuck with your preferences. It appears to combine the very worst combination of traits. Make catastrophic by the fact that Microsoft has provided no tool to edit this thing or to relocate the project. I have tried one million things. It is the dumbest thing I have ever heard.

But, I did devise a way to do it (including the relocation of the .slnfiles) and here is the story...

Reprise, the goals:

1) rename projects
2) create new directory structure (with .sln outside SourceCode)
3) put existing code into it
4) make it work correctly with Visual Studio

And an idiosyncracy: I prefer to divide the projects into separate solutions. The more usual way is to have one "solution" containing several projects. I prefer this because it allows me to have a separate, complete window for each solution. This is not possible for projects within a solution. Consequenlty, the three projects listed above, will each become a separate solution, each in its own directory. (Where needed, they refer to each other by reference.) This means that there is a .sln for each of them. My solution is the ManagementFiles directory inside of which I have directory, SolutionFiles, and inside that, one called, tqwhite. This is so that other people can store their own solution files without stomping on mine.

How to do it:

1) back up!!

2) create your new directory structure (inside ManagementFiles, create the SolutionFiles directory but not ones for each sub-project, VS will do that later)

3) distribute your source code into it; remove the .sln and .suo, keep the .csproj

4) open visual studio; if you moved, not copied, your source, when you select each solution, it will tell you it's lost and ask to remove it; say yes

For each sub-project:

5) sub-project, create a new project (for me, it's an ASP.NET MVC Empty Project). It will ask where to put it. The correct answer is NOT "SourceCode", it goes in ManagementFiles, in the place where you want the .sln for this sub-project to reside.

6) choose the new project info, right-click and Remove it.

7) go to File->Add->Existing Project; choose the .csproj file for your source code (in the new sub-project directory); it will ask for a name, use the new one that you like

8) build the project

When you quit the project and return, you should see your new projects and they should work. If you're as lucky as me, one of the three will. I ended redoing it several times to get them all going.

You might be able to use, File->New->Project from Existing Code to add your sub-projects to the newly created (renamed and relocated) .sln files. I did it for the javascript project because I lost its .csproj.

Finally, when you move your stuff to SharedSoftware, you are going to have to reassign your projects' references.

Good GUID

I have implemented the good kind of guid, uudi.comb. It required that I change my industry supplied data type from "string" to "System.guid". This does not appear to have any adverse consequences.

Yay!

I Take It Back

Upon further experimentation... 1) if I tell the <many-to-one> to be not-null="false", it creates a correct schema that works. This new schema demonstrates that I was also wrong about the foreign key constraint. 2) With the new schema, the foreign key constraint doesn't need to be removed

Many to Many with Data in the Join Table

When I wrote the last post, I had gotten nHibernate to create objects. They were specified to be in a many-to-many relationship and, though I can't remember the exact details of that moment, the objects were being created but the relationship was not. Since then, I have been really struggling to create that relationship.
The biggest thing that I have learned is that many-to-many is no longer fashionable and, I think, appropriately so. It is replaced by symmetrical one-to-many associations with a single relationship table. That is, a person visits many bars, and a bar has many visitors, but they really have a codependent relationship with each other. Which is to say, a person has many codependent relationships that help him be drunk and the bar has many codependent relationships that help it contribute to the delinquency of the culture.
For me, the nicest thing about this is that it removes the behind the scenes magic. It also makes use of that joining table. After all, it's not like there is some special kind of table for this. It's a table with data and making it a relationship rather than something invisible makes it more useful by giving us some semantics that allow us to store data about the relationship. The drunken person might want to note the number of blackout experiences enjoyed in each codependent relationship. The bar might want to record how many times police were involved for any particular codependency.
When I worked out how to use nHibernate <bag> elements and the <many-to-one> element and which entity they refer to (simple but slippery), I got an error message, "Cannot insert the value NULL into column." The column referred to is the foreign key in the join table. It's as if nH tries to create the join table first, before it knows the private keys of the other two records. Since I am using MSSQL, it allows you to specify whether a field is nullable and, nHibernate is really keen on using that when it generates a schema.

I communicated with the nice people on the google group (If you want to see the details, look at the technical conversation here) and tried many things. Inverse="true" did no good. Using <ibag> neither worked nor made sense. I tried using <set> just in case (couldn't figure out the type of the object property for that and it makes no sense because there can be duplicates and no sequence is implied). I tried using interlocking many-to-many. I tried changing the order of creation and saving in the app. Nothing mattered.

The only way I made any progress was to tell the database that the field is nullable but then, it complained that I was violating a "foreign key constraint". Sure enough, reading the generated schema, nHibernate adds this to the database, too. A foreign key constraint basically tells the database what two table/column pairs relate to each other and will not allow that to be screwed up. Good idea, unless you are using an ORM that creates the objects first and subsequently updates the foreign keys. I suspect that the update is when the violation occurs. I was far enough into it then that I knew there was no way I was going to convince nH to do it differently. I edited the schema again and Voila! Two concrete objects and a join table with correct foreign keys.

Tough luck on the database constraints.

Finally, as I write this, I am thinking about that guid. It's a long string and it's used as a key. I had chosen an nHibernate generator type that turned the guid into a string (uuid.hex) and can see that there is nothing that can be alphabetized about them. This is worrisome. I looked at the other guid generator schemes in nH (here) and found one that had a link to the rationale behind it. First, the article confirmed my suspicions. In simple tests, a guid is takes thirty times longer than an integer key. Big problem. This guys new guid generator makes for good, local commonality in the beginning characters. This reduces the penalty from thirty to .1, a 300x reduction. Unfortunately, using this method (guid.comb) gives me a type conversion violation in C#. I guess I have more work to do.

nHibernate is getting there

Phew! This is a hard thing to do.

I have done two major things lately. First, I made it so that I can generate database schema based on nHibernate mappings as well as generate update scripts that will revise the database to match changes in the nH mappings. This latter is one of the things I am most excited about. I can't tell you how annoying I have found it over the years to try to keep code and database schemas in sync. 

The other is that I have managed to write data using nHibernate. I haven't got all functions shaken down yet, but it's getting close.

The main trick for schemas was getting mapping to work. The nH docs are not very instructive, especially to a C# newbie. I had the devil of a time figuring out (and eventually just asked and got the answer; I wish I was rich enough to just pay someone so I didn't feel obligated to make a good faith effort to find out on my own) how to deal with object properties that are objects themselves. In this case, I had a property called name, that was a BaseName type with it's own properties of First, Last, etc. Turns out that the <component> element is the one to use. Once I heard that, the example of how to use it was for a Name object. 

The other thing that was hard was to deal with many-to-many. There are obvious provisions for this in the nH docs but it isn't really explained. The main thing is that I had to add a property to my class for the elements of the related collection. That is, if Widgets and Goodies are in a many-to-many relationship (maybe I should have chosen People and Bars, many People visit each Bar has many People visiting it) then you will, at some point need to assign one to the other, eg,

firstWidget.barList.add(someBar);
firstWidget.barList.add(anotherBar);

I had to learn how to create firstWidget.barList. This is probably easy for experienced C# people. For me, I had to started learning about enumerable types. List<Widget> (and List<Goodie>) work for schema generation. when I tried to write data, I got, "Unable to cast object of type." Ouch.

Turns out that nH, behind the scenes, changes the type of that grouping property (actually, I used the <bag> element) to its own kind of enumerable type. "List" is actually a type based on the interface, "IList". I don't thoroughly understand this typing business, but nH can't change out an actual type but it can assign a type to a property specified with an interface.

Bottom line, I used IList<Widget> and IList<Goodie> and Voila! I got me some database writing.

But there's more!

These are smaller things that I knew about, but that came to bite me anyway. First, because of the whole lazy loading thing, properties need to be "virtual". This was easy to debug because the error message was very explicit. In my case, I am working with an industry xsd-based type system so I got to use me some serious regular expression mojo to make it happen.

My text editor of choice is BBedit. I needed to change all the public properties from "public PropertyName" to "public virtual PropertyName". Unfortunately, the classes are public as well (public partial class BlahBlah{}) so I had to exclude the public classes from the substitution. Eventually I found someone that had constructed a conditional regex pattern that did the job:

     (?(?=^(the_only_string_you_dont_want_to_match)$)^$|.*)

Since it was constructed to deal with an entire line, I changed it to:

     public +(?(?=(partial)) |.*)

and replaced it with

     public virtual \1

I have run into the other major issue several times and so, when I hit it again, it was easy. The xml mapping files, class.hbm.xml, property needs its output directory to be set as an Embedded Resource. If they are, things work. If not, "No persister for ...".

In a closely related point, remember to set your xml file property Build Action to "Copy Always."

Onward!!

C# Controller Parameter

I am astonished to tell you that the incoming variable to a controller in a default ASP.NET MVC setup apparently must be named "id".

WTF?

Someone, please correct me on this. I just did a sequence of experiments that appear to prove it and it resurrects my innate distaste for MS technology.

mbUnit

In the process of shaking down nHibernate, I've learned a bunch about mbUnit. The main thing is that it really wants to be driven from it's management partner, Icarus. Test files will run using Visual Studio's built-in testing framework, but, you basically cannot see any output from the test except the pass/fail notation. That makes it very, very difficult to debug.

Icarus is a standalone program. You link it to the assembly dll for your testing project and it shows you the list of tests. When you run them, anything that you write to the console (Console.WriteLine();) comes out in the report as do normal sorts of runtime errors.

I'm happy to say that it is very good at seeing the inside of the target system that is being tested. nHibernate has been difficult. One of the things I didn't trust was that the tests were able to see all the files and nuances so I verified. Simply adding a reference to the target projectName.dll does wonders.

XSD to C#, Woo Hoo!

There is an industry standard data model made by some gigantic multi-year committee. When I learned about it, I got all excited. These people had designed my data structure!! Woo Hoo!

Then I discovered that it's - literally - 600 pages. Fortunately, there is an xsd file. Even better, Microsoft has a utility in its developer kit (a separate download, "Power Toys") called xsd.exe. It converts an xsd file into some sort of data object class definitions. Considering all the [attributes] they stack into this file, it must have some serious .NET mojo. I don't know how to work that yet. But, I grabbed it into my Macintosh, cleaned the gigantic amount of .NET cruft and converted it into an outline program so that I can organize the class definitions. 

My thought is to use the outliner format to select the subset of data elements I want to play with and then take it and convert it into javascript class definitions (ah, the wonders of bbedit and regular expressions). The same data definitions for both sides of my AJAX fence. That sounds like a great thing. I'm so excited.

nHibernate Works!

This has been among my most grueling experiences. Since Wednesday, nHibernate has been showing a "Security Exception" error message. I have been fighting it ever since with absolutely no progress. It has been brutal. I read and experimented, read more, tried something else. Did it again. Again. Again. Again. Again. Again. Weds and Thurs, I went home exhausted. I wrote on technical bulletin boards and got some great ideas from colleagues. There technical details are in this thread on the nhusers group on google. It basically started working for no real reason at the end of Friday. I do not know what fixed the problem. I care, but only a tiny, little bit. Working is working.

I have learned about trust levels in IIS7. There are five of them. The lower three will not allow debugging. The upper two give security exceptions to nHibernate unless it gets to late Friday afternoon and then they don't anymore. I also learned about the 'unblock' feature that prevents files loaded off the internet from being used inadvertently. Rather than asking you if you want to allow a web-sourced file to be used when the file is needed, Windows 7 requires you to select each file, right-click to Property and choose unblock. There is no option to cause it to propagate to enclosed files. Very inconvenient. Anyway, I had to do that for each of the nHibernate .dll files.

Things I got about nHibernate that were harder to find than they should have been:

You need to specify the assembly. I don't think there was a <mapping assembly="AssemblyName"/> attribute in the sample configuration file. It can be specified there or as a string parameter on an addAssembly() (or something like that) when the nHibernate session is created. I couldn't get the latter to work myself. 

This is common sense, but the data definition object class name has to either be the same as the name of the table (in a database with a name that matches the one in the database login string in .cfg). If the names are different for some reason, you can us a table attribute (<class name="AssemblyName.Services.BaseDataObjects.categorytBase, AssemblyName" table="categoryBase">) in class element in the mapping file (.hbm). 

I just went back, hours later, and took a look. It works all the way through to the database. Sweet.

nHibernate Emergency

I've spent the last two days trying to get my first bit of nHibernate 
to work. I am in "No Persister" hell. I have done everything and can't make it work.

Please, for the love of all that is holy, help me!! 

If you know anything about nHibernate, please visit this page at google groups, where I detail my pain and save my bacon.