The Framework Issue Expands

With Sproutcore gone, Javascript MVC is next on my list. It's a cleaner looking package that doesn't appear to try to do so much (Sproutcore had a million kinds of controls and other stuff to go along with it). I watched their introductory videos (this is a great new trend in software) and think it might be ok.

But, while I was at it, I looked around for other information about the topic and ran into the guys at Yahoo who are doing YUI, or more specifically the next version, YUI3. This is not a Javascript MVC replacement. It's more of a jQuery targeted at serious application development. jQuery can, and is, used by plenty of serious people, but here is another level of complexity. I am more familiar with YUI's renegade child, EXT JS. From what I can see, they are similar in intent.

EXT JS was amazing. It allowed one to, in a very, very short time, create highly capable tables of data with a lot of tasty controls and features that just came straight from the framework. It was very, very complicated to learn, but the guys that did so (I was a novice), were able to crank up some serious business. It is my understanding that YUI was a precursor to EXT JS and is striving toward much the same goal.

Listening to the YUI3 people doing a presentation to some sort of user group, I was impressed by their intention to make YUI work in a way that is very consistent with the basic ideas of the DOM. Since this is the foundation of every web browser presentation layer and, thus, every application, that seems like a really good thing. Further, the main thing I thought about EXT JS was that it felt like none of my previous knowledge really helped me master it.

The other thing they talked about, that I had forgotten about and that I liked very, very much, was the ability to create custom events. It's nice to have "onclick", etc, but sometimes I want to have an "onStupidUser" event that allows me to do something outside the normal flow. I used this a lot in my work with EXT JS. The YUI guys talked about some good improvements upon the theme.

What's it all mean? Well, two things. First, it reminded me that I may not be satisfied with only jQuery. Second, it reminded me that I may not be satisfied with only jQuery and Javascript MVC. It seems fully possible that I will want to add YUI3 or EXT JS to the kit. That would push jQuery back to the role of user interface sugar maker, good for decorations, etc, but leaving the heavy lifting of data display and interaction to one of the more formalized user interface systems.

But, it's a series of decisions for another day.

Sproutcore Bites the Dust

When I wrote my inaugural empty <div> Ajax application last summer, I figured out some things. One is that the View and Controller layers of MVC will eventually be entirely written in Javascript. The other is that writing an application in Javascript is like any other language, it really needs some sort of structure to avoid code that is difficult to understand or change.

Though I suppose there might be other structures that are as good (maybe), I find MVC to be a natural and pleasing way to organize code. The "separation of concerns" is about right. I can usually find a place that seems right for whatever bit I want to add to a site and, even more importantly, I can find and understand it later. 

Javascript makes life even more difficult because of it's strongly event-driven nature. One needs to separate the (in jQuery jargon) $.ready() code from things you want to run immediately. There are event handlers and a ton of other functions and objects that are needed to handle callbacks on them. AJAX ends up complicating it further with a whole other set of events and functions needed to process data as it comes and goes.  It is a mess.

So, I searched for Javascript MVC. Happily, I came up with a nice handful of options. Among them, I had heard of Sproutcore. It has been adopted by Apple for it's javascript apps and gets a lot of nice conversation among the smart kids on the web. I read about others and settled on it and a project named, appropriately enough, Javascript MVC as the likely candidates. 

I spent much of yesterday working on Sproutcore and have eliminated it from my competition. I am pained by how complicated and foreign it is. I watched and read several instruction things about it and, it's not like any javascript I've ever seen. I mean, I recognize the language constructs, but the conduct of development in it is way too much like .NET before .NET MVC. Lots of magical things to configure. I might have stuck with it except for it's other major resemblance to .NET: not only doesn't it develop like other web languages, it doesn't deploy like other javascript thingies.

To install Sproutcore, I first had to install Ruby/Webrick. I love the idea of Ruby, but having to add a third language to my life at this moment is a troubling addition to the already overwhelming complication of my technology. Then, it turns out that in development, you can't view its function in the normal way. To look at it, you need to use Webrick, a small, configurable (and cool) web serving tool. I didn't bother to truly comprehend, but it appears that Webrick executes some Ruby scripts to reorganize the Sproutcore system in a way that is servable. I made it work and couldn't figure out anything about what it was doing. I learned that there is a special build step to package it up for deployment on an external server. I don't know how you can trust how it's going to behave or what to do to debug and modify after. I do not like it.

I researched it for probably longer than I should, trying to figure out what I could do to connect it to my existing C# tools. That is, after all, where the model layer will be. Though Sproutcore has facillities that supposedly help it do that (that's the point, after all), the problem of doing so in a .NET context are just huge. Add the whole Ruby/Webrick deal and, fuhgedaboudit.

AJAX Ho!!

ASP.NET MVC is so reasonable that it's not much of a challenge. So, it's on to AJAX. I decided to implement my previous example, select a category and show the details, but, of course, in AJAX.

There was a bunch of fairly straightforward stuff. I had to figure out how to work master pages and how to place text into it. Then I had to figure out how to link to jQuery (included in the .NET MVC installation). It's stored in the Scripts directory. I wanted to put them in the same directory as the view (Views/Home/js) but, and I can't figure out why, it wouldn't serve from there. For the time being, I created an App/Home directory in the Scripts folder. 

Next, I made a Linq to Sql mapping. I basically did the same procedure I documented below and it worked right away. I made a structural decision based on that cool paper I read and wrote about to put the database access code (the Linq stuff) outside the Models. That is, I decided that the Models are business objects and database access is a Service, which is now a directory in my project.

Being enamored with that dropdown binding article from last week, I decided to try my first abstract type. It worked just like in the article and, while I was at it, caused me to an a Services/Interfaces directory for the interface and type class. 

The I learned about the Json() operator. Then I got my first AJAX data out to the page. This was very fun.

The only downside was that Visual Studio doesn't necessarily build the project when it changes. If I forget, then the changes I saved in the code files are not represented in the .dll file (or whatever it is) and the changes don't show in the browser. I have to figure that there is some way to make it recompile the changed bits.

Next step is to retrieve an entire category with details. In fact, it was easy. This Linq stuff is very nice. I got me an object that works right away but then the nightmare began.

The first big problem was that I had two classes with the same name. That was bad. Then I changed the Linq one to refer to both tables (CategoryDetail.cs) and life was better. But then I ran into the real problem.

The Json() operator couldn't serialize my object. It told me that it was finding a circular reference. I tried everything. Eventually, I hit the right google idea and found a page that explained that the Linq objects are the problem.

The object I was trying to convert to JSON was a Category. It's properties are id, name, and a list of Students. It turns out that the Student object has an id, a name and a Category. Category refers to Student. Student refers back to Category. If ever there was anything that is circular, this is it.

The article explains that I could tell the object to treat the detail list as "internal" and thus, not "public" and so, not part of the serialize process. I did it and the error went away. Unfortunately, there were no students either. Eventually, I wrote a routine that extracts the students into a list which can then be serialized.

Then I remember about the "internal" thingy. I change it to public and it breaks. Of course, it does. It's got a Category object inside and those have Detail objects in them. I wonder how far down it goes. It's weird and sort of stupid.

Fine. It works. I have to figure out a new solution in the future because I definitely want to be able to pass an entire category object containing details out to JSON and then get it back for persistence. Maybe that's why I will love nHibernate more that Linq.

Two things: First, the Linq query stuff is very confusing.
Second, I had trouble making a class constructor. It is not supposed to return a value so I did something like:

public void CategoryDetail() { ... }

and got an error telling me I couldn't have a method the same name as its enclosing entity. ?????

Turns out that the "void" is the moral equivalent of saying, "Yes, this does return a value and it's nothing." The formulation that worked is:

public CategoryDetail() { ... }

No void. So that's it.

ASP.NET MVC

I am thrilled with ASP.NET MVC. Why? Because it ditches almost all of the horrible .NET crapola. For the most part, those screwy server controls are gone. The ViewState idea, out the window. Code-behind files and APP_CODE are also in the past. .NET MVC is an MVC framework and works like the rest of the internet. This is a very good thing.

It's actually been a little harder to move forward with it because of this. With the old stuff, there were a ton of things that I couldn't figure out that provided questions to be answered, problems to be understood, etc. With MVC, it all looks like any other MVC framework. After having put some code in each part of the system to see it work, there's not too much. Of course, there are always a few things to do. It will be interesting to find out what's new to learn.

Unable to Reflect Upon Objects by Namespace Designation

Note, I don't say it can't be done. It definitely can. I see it all the time in popup menus in Visual Studio. I type something that refers to "System.Web.UI.HtmlControls" and, poof, there's a menu that tells me all the options and what they mean.

Unfortunately, I am unable to do so with any form of reflection I have been able to devise. On top of all the stuff I did yesterday, I tried so many variations on the theme that I can't even remember them. No matter what, I could not convert the string "System.Web.UI.HtmlControls" into a list of methods, properties or anything else. I'm going to have to move on.

I did take a little side trip into Javascript today. That was very fun. Familiar is nice.

Reflection

My next major task is to implement ASP.NET MVC and figure out how it works. However, I have come to understand that reflection is incredibly useful in figuring out how things work so I have digressed. 

From the first reflection tutorial I tried:

The First Experiment

This code:

rosters_studentsDataContext gridData = new rosters_studentsDataContext()

Type testType = gridData.GetType();
 Response.Write("testType=" + testType);

Yields:

testType=rosters_studentsDataContext          

And does not requires a "using namespace" reference.

The Second Experiment

This code:

            MethodInfo testMethodInfo = testType.GetMethod("helloWorld");
            Response.Write("<BR>testMethodInfo=" + testMethodInfo);

Yields:

testMethodInfo=System.String helloWorld(System.String) 

But does require adding a namespace directive:

using System.Reflection;

Also note that the object against which the GetMethod is applied is the Type object produced by the first step, testType.

The Third Experiment

This code:
            string[] invokeData = new string[] {"reflection invoke"};
            var testResult= testMethodInfo.Invoke(gridData, invokeData);
            Response.Write("<br>invocation result="+testResult);

Yields:

invocation result=reflection invoke says, Hello World 

Note that 1) invoke is applied to the method that was extracted from the type, 2) an instance of the class was passed to invoke and 3) the test data has to be an array (I tried every simpler version). (ps, As you can see from the second experiment, the definition of helloWorld() is that it takes a string but, invoke needs an array to pop it off of.)

Moving right along to a different person's view of a reflection tutorial, I try these things:

The Fourth Experiment

This code:

            ConstructorInfo[] info = testType.GetConstructors();
            foreach (ConstructorInfo cf in info) { Response.Write("<br>Constructor=" + cf); }

Yields:

Constructor=Void .ctor()
Constructor=Void .ctor(System.String)
Constructor=Void .ctor(System.Data.IDbConnection)
Constructor=Void .ctor(System.String, System.Data.Linq.Mapping.MappingSource)
Constructor=Void .ctor(System.Data.IDbConnection, System.Data.Linq.Mapping.MappingSource) 

SInce I am using a LINQ generated class, this is sensible though it does make me realize that the idea of a constructor is different in C# than in PHP. Note that this is executed against the Type object, testType.

The Fifth Experiment

This code:

            MethodInfo[] info2 = testType.GetMethods();
            foreach (MethodInfo cf in info2) { Response.Write("<br>Method=" + cf); }

Yields:

Method=System.String helloWorld(System.String)
Method=System.Data.Linq.Table`1[roster] get_rosters()
Method=System.Data.Linq.Table`1[student] get_students()
Method=Void Dispose()
Method=System.Data.Common.DbConnection get_Connection()
... and another jillion more like them

Again, the LINQ generated class has a lot of stuff in it. Note that my helloWorld() method shows up nicely though, at the beginning where I put it. It's also interesting that there are many of these that do not appear in the class definition file, eg, Dispose(). The class is opened with a lot of "using" declarations and then, "public partial class rosters_studentsDataContext : System.Data.Linq.DataContext". I'm guessing that a lot of those are in that DataContext stuff but, it won't let me open it. Perhaps that will be another day's reflection experiment.

And to put a Bow on it, I wrap up with the obvious, but absolutely necessary

This code:

            PropertyInfo[] info3 = testType.GetProperties();
            foreach (PropertyInfo cf in info3) { Response.Write("<br>Property=" + cf); }

Yields:

Property=System.String tqTestVar
Property=System.Data.Linq.Table`1[roster] rosters
Property=System.Data.Linq.Table`1[student] students
Property=System.Data.Common.DbConnection Connection
Property=System.Data.Common.DbTransaction Transaction  
... and another jillion more

OK, so the bottom line is this:

Given the addition of System.Reflection to an assembly (another thing whose definition I need to nail down), you can crack open the class and operate on it. There are methods to extract the components of classes and methods to activate them (I only tried invoke() today but there are more, I'm sure).

Tomorrow, I think I will try to figure out how to set a property and how to reflect upon System.Data.Linq.DataContext.

Summary: Category/Detail Demo Page

Assuming you have a database and connection working in Visual Studio.

1) Create tables (don't forget a primary key for each and a foreign key column, category_id is a good one).

2) Set the foreign key relationship by making a database diagram and dragging the categories.id field onto the detail.category_id field.

3) Over on App_Code, right-click to add item, choose Linq to Sql. Drag the two tables into the resulting design surface. They should link to each other reflecting the foreign key relationship. categories_items is a good name to save under and you should see categories_items.dbml appear in App_Code.

4) Create test data (use Show Table Data and type over the word null in the emptiness). Remember to organize the foreign keys.

5) Make new page (click on site header, right-click to New Item, choose Web Form, categoryDetailDemo is a nice name)

6) Open the new page. click on design at the bottom of the window.

7) Drag a RadioButtonList from the Toolbox. Select configure data source from the little menu tab that shows on the linq block. Choose auto-postback.

8) Drag a gridview from the Toolbox. Don't configure it. This will be filled from the code-behind file (categoryDetailDemo.aspx.cs) in it's page_load method.

9) Open that code behind file. Insert code that resembles the following into the page_load method:

        var incomingId = 0;
        var view = RadioButtonList1.SelectedValue;

        if (RadioButtonList1.SelectedValue == "")
        {
            incomingId = -1; //causes nothing to show when page is entered
        }
        else
        {
            incomingId = Convert.ToInt32(RadioButtonList1.SelectedValue); //turns out that MSSQL uses (typed) queries, not strings
        }
        using (categories_itemsDataContext gridData = new categories_itemsDataContext())
        {
            var itemList = from item in gridData.items
                              where item.category_id == incomingId
                              select new { item.name };
            GridView1.DataSource = itemList;
            GridView1.DataBind();

        }

For me, this produces a functioning page. Clicking on a radio button shows items from that category.

I now pronounce myself an adequate beginner.

Killer Article Leads to Killer Learning Experience

Among my many shortcomings is this: I am old and learned to program in the procedural days. Worse, I learned to program for the web with PHP 3.0, long before anyone had devised any structure or protocol for how the web should work. Consequently, I am a johnny-come-lately to most of the cool stuff that has been figured out about making websites (and most other kinds of modern software).

In recent years, I have adopted some object-oriented techniques and read a lot. I pretty much understand the modern world. However, as much as I know about the trees, I really cannot very easily see the forest. I can play my recital piece, but it's not great music.

So, I have endeavoured to change that. Even before this .NET experience, I spent several months writing an object-oriented, MVC framework for my personal projects. It's very nice in a lot of ways (I have gone for simplicity and it's good) but, it has not turned me into an OO maven.

C# is changing that. Not only is it a very OO kind of language, but it's also freed me to think OO thoughts from the beginning of the project. It is also serving as a source of examples and giving me the opportunity to really think through a lot of things.

To wit...

Last night I got stuck on something (can't remember what) and, as always, googled it. I came up with a page entitled, "Three Ways to Bind a DropDownList with LINQ and C# ASP.NET". 

It starts with the obvious way, natural to .NET. It creates both a display object and a database access object in the .aspx file. The guy explains that this is the equivalent of an MVC view file (I paraphrase very broadly here) and that makes for a nasty confusion of architecture and deeply limits the potential for code-reuse and, for that matter, opportunities for more refined data presentation.

So, he suggests a somewhat better way. He extracts the data access code and puts it into the controller. This is passed (in that annoying, automatic .NET kind of way) to the view file where the display control has been told what fields to extract. He notes that this is better, but if the names of the fields in the database ever change, every page that has this dropdown will have to be revised. It also makes for cumbersome, un-reusable code. He's also unhappy because it uses a database object, not a business object, and that it violates the ideal separation of code functions (those field names in the view are plenty of evidence for this).

Which brings him to his "enterprise" quality answer, which I love and studied and learned immensely from.

What he did, is to make it so that display object only knows what sort of objects it wants to represent. The control basically says, "I am ListNumberOne and I am a drop-down list displaying GoodObjects." When his controller loads the page, it has a line of code that says, in effect, "PopulateList(ListNumberOne)".

PopulateList examines the list object passed to it (reflection is my next major topic) and finds out that it is supposed to display GoodObjects. It asks GoodObjects for a list suitable to be applied to a display object and applies it.

So far, neither of these things knows any more than it needs to. The display object knows what sort of things it wants to show, but nothing about how to get them. The controller only knows that it has to populate ListNumberOne, but nothing about how it's done.

The next part is where he provides the so-called "business object" that will make him happier about how things work. This business object does know 1) how to get the data (from a LINQ object defined in Visual Studio) and 2) what information is needed to populate the display object (provided by a simple class whose only business is to represent an appropriate data type).

Probably it's main coolness is that it does some cool stuff to grab the type of GoodObject so that it can be instantiated and asked for the appropriate data type.

This article uses a tone of esoteric (at least to me) C# constructs and thus provided me a comprehensible context to learn about them:

1) Attributes, ie, code constructs that look like [attributeName].  These are things that add behavior to whatever you put them next to. I'm not entirely clear about the internals, but the basic premise is that you write a class that does something you like, mention it as an attribute adjacent to the class you want that behavior added to, and Voila!, you've got some new behavior. I'm not certain, but I think this may be their way of doing composition in the class definition code. It also seems to offer a whiff of Aspect Oriented Programming. You can added crosscutting behaviors to almost anything. (I first saw, and was unable to understand, attributes when I ran into Postsharp which implements a sort of AOP this way.)

2) Generic classes, eg, Systems.Collections.Generic.List<T>. Turns out that this is the equivalent of a utility library for C#. 

3) Specifying types for generics, eg, List<NameValueType> (the "appropriate data type" mentioned above). Turns out that lots of things have generic types and that you can specify what they are supposed to be this way. I have to say, though, that I'm still a little unclear about the benefits of strong typing.

4) That you can have several methods of the same name for a class that are, apparently, distinguished by their parameters, ie, if you pass it one type, it will select the item of that name with that as a specified parameter; pass it a different type and it will choose the other one. Maybe this is why strong typing is good. Not really feeling it yet.

5) The [serializable] attribute. This allows you to pass the object to a variety of serializers (binary, xml, etc) that store objects in serialized form.

6) A practical and believable example of why it's good to have an interface class. In this case, it allows future programmers to be confident that their "lightweight business object" will have the ability to generate the "appropriate data type" for populating drop down lists (and many other things).

Phew!!

And it's only early afternoon.

Cha-Ching!

So, I did it. I made it so that I could show a bunch of radio buttons reflecting categories in the database. It uses the auto-postback feature of the radiolist thingy so that when you click on a radio, it auto-reloads the page.

Hey, it's not much, but if I had the patience to tell you about all of the hassles (values in sql-type queries are strongly typed, where clause comparisons use '==', not '=', etc), you'd be proud.

Progress != Perfection

The new project does a lot of good things. Unfortunately, it didn't make it so that my website could connect to the database. 

Google lead me to a page where a person asked the exact right question (Praise Google!). As I suspected, I had set up my database to use Windows authentication. That means that the tools could access the database as tqwhite, but the website could not. It runs as IUSR (I think) and that has no way of logging in so that MSSQL can inherit its privileges.
Turns out that one can use the Manager to change the security mode to allow both Windows authentication and the usual SQL user model. Reading further, I found that MSSQL ships with a user "sa" that is disabled by choosing Windows authentication. Blah blah.
Bottom line, I the thread I was reading also told me about another page [see update below] that explains how to deal with this. Basically, it's right-click on the database in the Manager and change Security. Then restart the server and execute these two queries:
 
ALTER LOGIN sa ENABLE
 ALTER LOGIN sa WITH PASSWORD = 'VeryStrongPasswordHere'

Obviously, you should choose your own password.
But then, I need to make it so that the website knows about these credentials. I right-clicked my butt off on the database connection and couldn't do it. Eventually, I chose properties and found that the connection string was:
 
Data Source=QUINDOWS;Initial Catalog=tmpTest;Integrated Security=True

I tried to edit this but it wouldn't let me. I'm sure I'll figure it out, but instead I edited the generated code file (temporarily). In my sandbox it's App_Code/DataClasses.designer.cs. I found the code that accesses a static class to get the string and changed that part to:
 
Server= QUINDOWS;Database= tmpTest;integrated security=sspi;Uid=sa;Pwd= VeryStrongPasswordHere;Trusted_Connection=no

Save, Run, Scream Eureka!!
 
I got a data grid.
 
Obviously, the next step is to figure out how to implement the new authentication using the tool, but at least I know that it will work when I find it.
 
UPDATE: I'm not sure if I'm stupid or some context sensitive thing changed. However, on the Server Explorer where the database connection resides, I was able to right-click and select Modify Connection. I changed it to SQL authentication and entered the info. I undid the damage I had done by editing the generated code (had to delete and recreate). It works.

UPDATE 6/29/10: As you might have found out, the page that explained how to set the security mode has become password protected. I found new info HERE. It says to open SQL Server Management Studio, right-click on the server item at the top of the object list at the left. Then, choose Properties, then Security. You will see a section on Server authentication. Choose SQL Server and Windows Authentication Mode. Life will be good. You can then add the users using SQL commands as above.