Installing Javascript MVC

Some of this was learned during my Sproutcore effort but, since I am writing this series of posts post-facto, here is the overall understanding.

First, Visual Studio is a Microsoft product and, as such, sucks. I know, I know, many developers consider it to be wonderful. Intellisense, the code completion and context-sensitive documentation function is great. It's tight integration with IIS is very helpful, as is its integration with MS SQL.  But it also is cumbersome and bloated. Worse, it doesn't have Eclipse's ability to expand an editor to fill the whole window and then shrink again. That means that one is forever forced to look at code through a little, tiny window. They say that VS 2010 will have multiple-monitor support. Maybe then it won't suck.

Anyway, that is only the beginning of the suckage. I want to add Javascript MVC to my site. Actually, I really want it to be a separate project. I'd actually like it to be a separate website. I'd like to have it be the main website and I'd like to have the C# stuff, aka, the Model Layer, be connected only by AJAX calls that return JSON. I want no other interaction between them.

It turns out that Javascript won't allow you to access a different domain for security reasons. Good thinking, but it means that I need to have another angle. Since Javascript MVC is the user interface, I would like it to present itself when a user goes to mydomain.com. That says that the Model Layer would be good at say, mydomain.com/modellayer, or whatever. Javascript would let me access it. Life is good.

I tried doing it with a Virtual Directory. No dice. I don't know why, but it said it needs to be an application. Google google. Turns out that there is an option, right next to the virtual directory on the popup to add an application. Cool Beans. Another step on the road to happiness.

Actually, I have to be honest. I spent a whole bunch of time cursing IIS. I got a message that said it coudn't show the directory. Nothing I did affected the display. Since I assume that IIS is doing invisible, mysterious crap, I thought that it needed something special to see the files in the folder, especially since there is no index.php analog in this project. I learned a great deal about how website.config works and global.asax.cs but no cigar. Then I got the idea to turn on Directory Listing and see if I could click on something.

I was pointing to the wrong directory!!!! Visual Studio in its infinite wisdom has lots of directories named 'sandbox'. I pointed at the one enclosing the good one. Fixed and works.

One other thing. I tried, during this thrashing, to place Javascript MVC into a directory inside my sandbox project. I never could figure out a way to do it. "Add Existing Item" doesn't work. Neither does adding a directory full of stuff to the file system inside it. Nothing. Never did get it to work, but that's good. I settled on a better approach. mydomain.com and mydomain.com/data is just what I want.

The Framework Issue Expands

With Sproutcore gone, Javascript MVC is next on my list. It's a cleaner looking package that doesn't appear to try to do so much (Sproutcore had a million kinds of controls and other stuff to go along with it). I watched their introductory videos (this is a great new trend in software) and think it might be ok.

But, while I was at it, I looked around for other information about the topic and ran into the guys at Yahoo who are doing YUI, or more specifically the next version, YUI3. This is not a Javascript MVC replacement. It's more of a jQuery targeted at serious application development. jQuery can, and is, used by plenty of serious people, but here is another level of complexity. I am more familiar with YUI's renegade child, EXT JS. From what I can see, they are similar in intent.

EXT JS was amazing. It allowed one to, in a very, very short time, create highly capable tables of data with a lot of tasty controls and features that just came straight from the framework. It was very, very complicated to learn, but the guys that did so (I was a novice), were able to crank up some serious business. It is my understanding that YUI was a precursor to EXT JS and is striving toward much the same goal.

Listening to the YUI3 people doing a presentation to some sort of user group, I was impressed by their intention to make YUI work in a way that is very consistent with the basic ideas of the DOM. Since this is the foundation of every web browser presentation layer and, thus, every application, that seems like a really good thing. Further, the main thing I thought about EXT JS was that it felt like none of my previous knowledge really helped me master it.

The other thing they talked about, that I had forgotten about and that I liked very, very much, was the ability to create custom events. It's nice to have "onclick", etc, but sometimes I want to have an "onStupidUser" event that allows me to do something outside the normal flow. I used this a lot in my work with EXT JS. The YUI guys talked about some good improvements upon the theme.

What's it all mean? Well, two things. First, it reminded me that I may not be satisfied with only jQuery. Second, it reminded me that I may not be satisfied with only jQuery and Javascript MVC. It seems fully possible that I will want to add YUI3 or EXT JS to the kit. That would push jQuery back to the role of user interface sugar maker, good for decorations, etc, but leaving the heavy lifting of data display and interaction to one of the more formalized user interface systems.

But, it's a series of decisions for another day.

Sproutcore Bites the Dust

When I wrote my inaugural empty <div> Ajax application last summer, I figured out some things. One is that the View and Controller layers of MVC will eventually be entirely written in Javascript. The other is that writing an application in Javascript is like any other language, it really needs some sort of structure to avoid code that is difficult to understand or change.

Though I suppose there might be other structures that are as good (maybe), I find MVC to be a natural and pleasing way to organize code. The "separation of concerns" is about right. I can usually find a place that seems right for whatever bit I want to add to a site and, even more importantly, I can find and understand it later. 

Javascript makes life even more difficult because of it's strongly event-driven nature. One needs to separate the (in jQuery jargon) $.ready() code from things you want to run immediately. There are event handlers and a ton of other functions and objects that are needed to handle callbacks on them. AJAX ends up complicating it further with a whole other set of events and functions needed to process data as it comes and goes.  It is a mess.

So, I searched for Javascript MVC. Happily, I came up with a nice handful of options. Among them, I had heard of Sproutcore. It has been adopted by Apple for it's javascript apps and gets a lot of nice conversation among the smart kids on the web. I read about others and settled on it and a project named, appropriately enough, Javascript MVC as the likely candidates. 

I spent much of yesterday working on Sproutcore and have eliminated it from my competition. I am pained by how complicated and foreign it is. I watched and read several instruction things about it and, it's not like any javascript I've ever seen. I mean, I recognize the language constructs, but the conduct of development in it is way too much like .NET before .NET MVC. Lots of magical things to configure. I might have stuck with it except for it's other major resemblance to .NET: not only doesn't it develop like other web languages, it doesn't deploy like other javascript thingies.

To install Sproutcore, I first had to install Ruby/Webrick. I love the idea of Ruby, but having to add a third language to my life at this moment is a troubling addition to the already overwhelming complication of my technology. Then, it turns out that in development, you can't view its function in the normal way. To look at it, you need to use Webrick, a small, configurable (and cool) web serving tool. I didn't bother to truly comprehend, but it appears that Webrick executes some Ruby scripts to reorganize the Sproutcore system in a way that is servable. I made it work and couldn't figure out anything about what it was doing. I learned that there is a special build step to package it up for deployment on an external server. I don't know how you can trust how it's going to behave or what to do to debug and modify after. I do not like it.

I researched it for probably longer than I should, trying to figure out what I could do to connect it to my existing C# tools. That is, after all, where the model layer will be. Though Sproutcore has facillities that supposedly help it do that (that's the point, after all), the problem of doing so in a .NET context are just huge. Add the whole Ruby/Webrick deal and, fuhgedaboudit.

AJAX Ho!!

ASP.NET MVC is so reasonable that it's not much of a challenge. So, it's on to AJAX. I decided to implement my previous example, select a category and show the details, but, of course, in AJAX.

There was a bunch of fairly straightforward stuff. I had to figure out how to work master pages and how to place text into it. Then I had to figure out how to link to jQuery (included in the .NET MVC installation). It's stored in the Scripts directory. I wanted to put them in the same directory as the view (Views/Home/js) but, and I can't figure out why, it wouldn't serve from there. For the time being, I created an App/Home directory in the Scripts folder. 

Next, I made a Linq to Sql mapping. I basically did the same procedure I documented below and it worked right away. I made a structural decision based on that cool paper I read and wrote about to put the database access code (the Linq stuff) outside the Models. That is, I decided that the Models are business objects and database access is a Service, which is now a directory in my project.

Being enamored with that dropdown binding article from last week, I decided to try my first abstract type. It worked just like in the article and, while I was at it, caused me to an a Services/Interfaces directory for the interface and type class. 

The I learned about the Json() operator. Then I got my first AJAX data out to the page. This was very fun.

The only downside was that Visual Studio doesn't necessarily build the project when it changes. If I forget, then the changes I saved in the code files are not represented in the .dll file (or whatever it is) and the changes don't show in the browser. I have to figure that there is some way to make it recompile the changed bits.

Next step is to retrieve an entire category with details. In fact, it was easy. This Linq stuff is very nice. I got me an object that works right away but then the nightmare began.

The first big problem was that I had two classes with the same name. That was bad. Then I changed the Linq one to refer to both tables (CategoryDetail.cs) and life was better. But then I ran into the real problem.

The Json() operator couldn't serialize my object. It told me that it was finding a circular reference. I tried everything. Eventually, I hit the right google idea and found a page that explained that the Linq objects are the problem.

The object I was trying to convert to JSON was a Category. It's properties are id, name, and a list of Students. It turns out that the Student object has an id, a name and a Category. Category refers to Student. Student refers back to Category. If ever there was anything that is circular, this is it.

The article explains that I could tell the object to treat the detail list as "internal" and thus, not "public" and so, not part of the serialize process. I did it and the error went away. Unfortunately, there were no students either. Eventually, I wrote a routine that extracts the students into a list which can then be serialized.

Then I remember about the "internal" thingy. I change it to public and it breaks. Of course, it does. It's got a Category object inside and those have Detail objects in them. I wonder how far down it goes. It's weird and sort of stupid.

Fine. It works. I have to figure out a new solution in the future because I definitely want to be able to pass an entire category object containing details out to JSON and then get it back for persistence. Maybe that's why I will love nHibernate more that Linq.

Two things: First, the Linq query stuff is very confusing.
Second, I had trouble making a class constructor. It is not supposed to return a value so I did something like:

public void CategoryDetail() { ... }

and got an error telling me I couldn't have a method the same name as its enclosing entity. ?????

Turns out that the "void" is the moral equivalent of saying, "Yes, this does return a value and it's nothing." The formulation that worked is:

public CategoryDetail() { ... }

No void. So that's it.

ASP.NET MVC

I am thrilled with ASP.NET MVC. Why? Because it ditches almost all of the horrible .NET crapola. For the most part, those screwy server controls are gone. The ViewState idea, out the window. Code-behind files and APP_CODE are also in the past. .NET MVC is an MVC framework and works like the rest of the internet. This is a very good thing.

It's actually been a little harder to move forward with it because of this. With the old stuff, there were a ton of things that I couldn't figure out that provided questions to be answered, problems to be understood, etc. With MVC, it all looks like any other MVC framework. After having put some code in each part of the system to see it work, there's not too much. Of course, there are always a few things to do. It will be interesting to find out what's new to learn.

Unable to Reflect Upon Objects by Namespace Designation

Note, I don't say it can't be done. It definitely can. I see it all the time in popup menus in Visual Studio. I type something that refers to "System.Web.UI.HtmlControls" and, poof, there's a menu that tells me all the options and what they mean.

Unfortunately, I am unable to do so with any form of reflection I have been able to devise. On top of all the stuff I did yesterday, I tried so many variations on the theme that I can't even remember them. No matter what, I could not convert the string "System.Web.UI.HtmlControls" into a list of methods, properties or anything else. I'm going to have to move on.

I did take a little side trip into Javascript today. That was very fun. Familiar is nice.

Reflection

My next major task is to implement ASP.NET MVC and figure out how it works. However, I have come to understand that reflection is incredibly useful in figuring out how things work so I have digressed. 

From the first reflection tutorial I tried:

The First Experiment

This code:

rosters_studentsDataContext gridData = new rosters_studentsDataContext()

Type testType = gridData.GetType();
 Response.Write("testType=" + testType);

Yields:

testType=rosters_studentsDataContext          

And does not requires a "using namespace" reference.

The Second Experiment

This code:

            MethodInfo testMethodInfo = testType.GetMethod("helloWorld");
            Response.Write("<BR>testMethodInfo=" + testMethodInfo);

Yields:

testMethodInfo=System.String helloWorld(System.String) 

But does require adding a namespace directive:

using System.Reflection;

Also note that the object against which the GetMethod is applied is the Type object produced by the first step, testType.

The Third Experiment

This code:
            string[] invokeData = new string[] {"reflection invoke"};
            var testResult= testMethodInfo.Invoke(gridData, invokeData);
            Response.Write("<br>invocation result="+testResult);

Yields:

invocation result=reflection invoke says, Hello World 

Note that 1) invoke is applied to the method that was extracted from the type, 2) an instance of the class was passed to invoke and 3) the test data has to be an array (I tried every simpler version). (ps, As you can see from the second experiment, the definition of helloWorld() is that it takes a string but, invoke needs an array to pop it off of.)

Moving right along to a different person's view of a reflection tutorial, I try these things:

The Fourth Experiment

This code:

            ConstructorInfo[] info = testType.GetConstructors();
            foreach (ConstructorInfo cf in info) { Response.Write("<br>Constructor=" + cf); }

Yields:

Constructor=Void .ctor()
Constructor=Void .ctor(System.String)
Constructor=Void .ctor(System.Data.IDbConnection)
Constructor=Void .ctor(System.String, System.Data.Linq.Mapping.MappingSource)
Constructor=Void .ctor(System.Data.IDbConnection, System.Data.Linq.Mapping.MappingSource) 

SInce I am using a LINQ generated class, this is sensible though it does make me realize that the idea of a constructor is different in C# than in PHP. Note that this is executed against the Type object, testType.

The Fifth Experiment

This code:

            MethodInfo[] info2 = testType.GetMethods();
            foreach (MethodInfo cf in info2) { Response.Write("<br>Method=" + cf); }

Yields:

Method=System.String helloWorld(System.String)
Method=System.Data.Linq.Table`1[roster] get_rosters()
Method=System.Data.Linq.Table`1[student] get_students()
Method=Void Dispose()
Method=System.Data.Common.DbConnection get_Connection()
... and another jillion more like them

Again, the LINQ generated class has a lot of stuff in it. Note that my helloWorld() method shows up nicely though, at the beginning where I put it. It's also interesting that there are many of these that do not appear in the class definition file, eg, Dispose(). The class is opened with a lot of "using" declarations and then, "public partial class rosters_studentsDataContext : System.Data.Linq.DataContext". I'm guessing that a lot of those are in that DataContext stuff but, it won't let me open it. Perhaps that will be another day's reflection experiment.

And to put a Bow on it, I wrap up with the obvious, but absolutely necessary

This code:

            PropertyInfo[] info3 = testType.GetProperties();
            foreach (PropertyInfo cf in info3) { Response.Write("<br>Property=" + cf); }

Yields:

Property=System.String tqTestVar
Property=System.Data.Linq.Table`1[roster] rosters
Property=System.Data.Linq.Table`1[student] students
Property=System.Data.Common.DbConnection Connection
Property=System.Data.Common.DbTransaction Transaction  
... and another jillion more

OK, so the bottom line is this:

Given the addition of System.Reflection to an assembly (another thing whose definition I need to nail down), you can crack open the class and operate on it. There are methods to extract the components of classes and methods to activate them (I only tried invoke() today but there are more, I'm sure).

Tomorrow, I think I will try to figure out how to set a property and how to reflect upon System.Data.Linq.DataContext.

Summary: Category/Detail Demo Page

Assuming you have a database and connection working in Visual Studio.

1) Create tables (don't forget a primary key for each and a foreign key column, category_id is a good one).

2) Set the foreign key relationship by making a database diagram and dragging the categories.id field onto the detail.category_id field.

3) Over on App_Code, right-click to add item, choose Linq to Sql. Drag the two tables into the resulting design surface. They should link to each other reflecting the foreign key relationship. categories_items is a good name to save under and you should see categories_items.dbml appear in App_Code.

4) Create test data (use Show Table Data and type over the word null in the emptiness). Remember to organize the foreign keys.

5) Make new page (click on site header, right-click to New Item, choose Web Form, categoryDetailDemo is a nice name)

6) Open the new page. click on design at the bottom of the window.

7) Drag a RadioButtonList from the Toolbox. Select configure data source from the little menu tab that shows on the linq block. Choose auto-postback.

8) Drag a gridview from the Toolbox. Don't configure it. This will be filled from the code-behind file (categoryDetailDemo.aspx.cs) in it's page_load method.

9) Open that code behind file. Insert code that resembles the following into the page_load method:

        var incomingId = 0;
        var view = RadioButtonList1.SelectedValue;

        if (RadioButtonList1.SelectedValue == "")
        {
            incomingId = -1; //causes nothing to show when page is entered
        }
        else
        {
            incomingId = Convert.ToInt32(RadioButtonList1.SelectedValue); //turns out that MSSQL uses (typed) queries, not strings
        }
        using (categories_itemsDataContext gridData = new categories_itemsDataContext())
        {
            var itemList = from item in gridData.items
                              where item.category_id == incomingId
                              select new { item.name };
            GridView1.DataSource = itemList;
            GridView1.DataBind();

        }

For me, this produces a functioning page. Clicking on a radio button shows items from that category.

I now pronounce myself an adequate beginner.

Killer Article Leads to Killer Learning Experience

Among my many shortcomings is this: I am old and learned to program in the procedural days. Worse, I learned to program for the web with PHP 3.0, long before anyone had devised any structure or protocol for how the web should work. Consequently, I am a johnny-come-lately to most of the cool stuff that has been figured out about making websites (and most other kinds of modern software).

In recent years, I have adopted some object-oriented techniques and read a lot. I pretty much understand the modern world. However, as much as I know about the trees, I really cannot very easily see the forest. I can play my recital piece, but it's not great music.

So, I have endeavoured to change that. Even before this .NET experience, I spent several months writing an object-oriented, MVC framework for my personal projects. It's very nice in a lot of ways (I have gone for simplicity and it's good) but, it has not turned me into an OO maven.

C# is changing that. Not only is it a very OO kind of language, but it's also freed me to think OO thoughts from the beginning of the project. It is also serving as a source of examples and giving me the opportunity to really think through a lot of things.

To wit...

Last night I got stuck on something (can't remember what) and, as always, googled it. I came up with a page entitled, "Three Ways to Bind a DropDownList with LINQ and C# ASP.NET". 

It starts with the obvious way, natural to .NET. It creates both a display object and a database access object in the .aspx file. The guy explains that this is the equivalent of an MVC view file (I paraphrase very broadly here) and that makes for a nasty confusion of architecture and deeply limits the potential for code-reuse and, for that matter, opportunities for more refined data presentation.

So, he suggests a somewhat better way. He extracts the data access code and puts it into the controller. This is passed (in that annoying, automatic .NET kind of way) to the view file where the display control has been told what fields to extract. He notes that this is better, but if the names of the fields in the database ever change, every page that has this dropdown will have to be revised. It also makes for cumbersome, un-reusable code. He's also unhappy because it uses a database object, not a business object, and that it violates the ideal separation of code functions (those field names in the view are plenty of evidence for this).

Which brings him to his "enterprise" quality answer, which I love and studied and learned immensely from.

What he did, is to make it so that display object only knows what sort of objects it wants to represent. The control basically says, "I am ListNumberOne and I am a drop-down list displaying GoodObjects." When his controller loads the page, it has a line of code that says, in effect, "PopulateList(ListNumberOne)".

PopulateList examines the list object passed to it (reflection is my next major topic) and finds out that it is supposed to display GoodObjects. It asks GoodObjects for a list suitable to be applied to a display object and applies it.

So far, neither of these things knows any more than it needs to. The display object knows what sort of things it wants to show, but nothing about how to get them. The controller only knows that it has to populate ListNumberOne, but nothing about how it's done.

The next part is where he provides the so-called "business object" that will make him happier about how things work. This business object does know 1) how to get the data (from a LINQ object defined in Visual Studio) and 2) what information is needed to populate the display object (provided by a simple class whose only business is to represent an appropriate data type).

Probably it's main coolness is that it does some cool stuff to grab the type of GoodObject so that it can be instantiated and asked for the appropriate data type.

This article uses a tone of esoteric (at least to me) C# constructs and thus provided me a comprehensible context to learn about them:

1) Attributes, ie, code constructs that look like [attributeName].  These are things that add behavior to whatever you put them next to. I'm not entirely clear about the internals, but the basic premise is that you write a class that does something you like, mention it as an attribute adjacent to the class you want that behavior added to, and Voila!, you've got some new behavior. I'm not certain, but I think this may be their way of doing composition in the class definition code. It also seems to offer a whiff of Aspect Oriented Programming. You can added crosscutting behaviors to almost anything. (I first saw, and was unable to understand, attributes when I ran into Postsharp which implements a sort of AOP this way.)

2) Generic classes, eg, Systems.Collections.Generic.List<T>. Turns out that this is the equivalent of a utility library for C#. 

3) Specifying types for generics, eg, List<NameValueType> (the "appropriate data type" mentioned above). Turns out that lots of things have generic types and that you can specify what they are supposed to be this way. I have to say, though, that I'm still a little unclear about the benefits of strong typing.

4) That you can have several methods of the same name for a class that are, apparently, distinguished by their parameters, ie, if you pass it one type, it will select the item of that name with that as a specified parameter; pass it a different type and it will choose the other one. Maybe this is why strong typing is good. Not really feeling it yet.

5) The [serializable] attribute. This allows you to pass the object to a variety of serializers (binary, xml, etc) that store objects in serialized form.

6) A practical and believable example of why it's good to have an interface class. In this case, it allows future programmers to be confident that their "lightweight business object" will have the ability to generate the "appropriate data type" for populating drop down lists (and many other things).

Phew!!

And it's only early afternoon.

Cha-Ching!

So, I did it. I made it so that I could show a bunch of radio buttons reflecting categories in the database. It uses the auto-postback feature of the radiolist thingy so that when you click on a radio, it auto-reloads the page.

Hey, it's not much, but if I had the patience to tell you about all of the hassles (values in sql-type queries are strongly typed, where clause comparisons use '==', not '=', etc), you'd be proud.