sftp/scp fails at connection, but ssh is OK

The brilliant people at Server Grove just helped me solve a nasty problem.

My SFTP access to my Linux (Ubuntu) web server was working fine. Then one night, I was working and it died in the middle of a session. I tried everything I could think of to no avail. 

The next day it worked again for a few minutes. Intermittent sucks even more. Then it died again.

Among the things I note: other user accounts worked fine for SFTP and SSH into the problem account worked correctly.

What I didn't note is that a change I made to .bashrc had a syntax error. I didn't notice this because 1) I always have .bashrc tell me things so the error message was lost in a few lines of other stuff and 2) the error caused no problems.

Turns out that latter statement was wrong. SFTP, as you may know, uses SSH for access. When I SSH into that account, I had .bashrc remind me of some things. SFTP apparently knows how to turn those off and, thus, work ok. It does not turn off error messages.

So, when the erroneous .bashrc was moved out of the directory, SFTP worked. When I fixed it, it worked.

So, if SFTP (or SCP or any other SSH-based utility) fails while SSH is still working, look to .bashrc. I don't know if it's only error messages that cause trouble but they definitely do.

And thanks, Server Grove, for helping me with this. I've always been very happy with their service. Their prices are reasonable and systems good. Going the extra mile to help me is icing on the cake.

A New Mission for the Darkside

My occupation is to create rich internet applications that access business databases. There is no advertising. Search engines are irrelevant. I make tools for business people to enter, examine and update data for various purposes. The results are usually supposed to resemble the dedicated client applications of yesteryear with immediate feedback for errors or lookup data, dynamic entry forms that change in response to user choices, etc. If this sounds like the modern world of AJAX websites, it should. Over the last couple of years, that has become my specialty.

It's not easy, though. As far as the web has come for rich internet applications, there still aren't tools that make this simple. In my old days, I had a gui application that allowed me to drag a data field onto a canvas, open a control panel that allowed me to connect it to a database field or calculation, specify the focus and blur behaviors, and trigger all sorts of events. Even then it was complex but, now that it is done in Javascript, talking to a server in PHP or C#, and using HTML/CSS for presentation, it's a bear.

It turns out that my real world is rarely a 'green field' application. With one exception, I have not been able to design the database. There are incumbent client applications that people are using to address the database. My mission is to, more or less, port certain functions to the internet based on an existing database server. 

I come to this with a bias in favor of browser-based applications. I don't have to worry about IE6. My clients are not allowed to use that. I don't care about search engines. Nobody without a password is ever allowed to use my work. All of my users have Javascript. At most, I support the previous two versions of the main programs and I'm not very fussy about that. Almost everyone I work with uses Safari, Firefox or Chrome. Those update automatically so, again with some exceptions, if it works for me, it works for others. Graceful degradation? Nope. It's my way or the highway and that's the way my clients like it.

I have settled on a simple architectural plan. The server serves data. The user interface is entirely in the browser. Except for the <script> tag that calls the javascript application, my web servers send absolutely no html. AJAX requests ask for data. That returns in JSON. The only exception to this is reports and exports. I have apps that generate a file as a side effect and return, in JSON, a way to retrieve it.

Because of this, my server code is rudimentary. Given that I am mostly working with existing database apps, complex data retrieval and manipulation is already done. My server code accesses that and puts it into JSON. This isn't always trivial, but often is. I do use an MVC framework (.NET MVC or ZEND these days) but it's mostly overkill. There are, for example, only a couple view templates. One for JSON that is used all over the place and another for database manipulation (it does a post to the controller of my choice for debugging).

My main focus is on the UI code in Javascript. For this I also use an MVC framework, JavascriptMVC. It provides numerous utilities that allow me to organize my code into comprehensible files and folders. It also includes good utilities for communicating among classes. It's a pretty sweet product.

I will be talking, for the moment, mostly about two applications. One is a tiny sales contact data entry app. The other is a huge gradebook application for school districts. These represent the extremes of my world. Often, I will pioneer a technique in the small one and then use it in the big one.

As it was when I was learning Windows and C#, this is my lab notebook. Though I am not a novice at this Javascript game as I was with those, I still keep learning things and need a place to write it down.

Sharing Windows 7 files

I run Windows 7 on a Macintosh using Parallels 5. It is an incredibly good thing. It has a display mode called Crystal that entirely hides the Windows desktop and controls. The only thing you see are windows. They appear in the Dock like good little windows. Parallels has settings that allow you to map your command key functions to control key functions so well that I basically never use the actual control key. It does flawless copy/paste and file dragging. Except for the unescapable details (close box on top left, menus inside the window, crappy UI detailsI), it is easy to forget which is which. So much so that I chose a pink colored theme for my Windows windows so that I could distinguish them in Expose.

Anyway, that's all been wonderful but, I still am stuck with the crappy UI details and Visual Studio. Visual Studio is reputed to be great and, I think it is. When I am working on .NET/C# things, it's "Intellisense" is nothing short of wonderful. It knows everything about what I want to do. The "Go To Definition" operation is a dream come true. There are some drawbacks but the benefits are well worth the trouble.

When working on Javascript, not so much. It doesn't fold code well. It is sluggish. It has the annoying UI defects. All the drawbacks and no benefits. I have long wanted to work in my trusty, old bbedit. It's a very clean text editor. No Intellisense but great regular expressions. It's fast as can be. It has all the UI perfection that Apple can offer. It is good.

So, I decided to figure out how to share the javascript project directory so that I can use the tool I prefer and that is the main point of this post.

It wasn't easy. It isn't intuitive. It requires doing things that make no sense. Fortunately for you, I boiled it down into a few simple steps:

1) Navigate to the folder you want to share. Right-click and choose Properties-> Sharing Tab->Advanced Sharing. It will let you type in a name. This is the name that will appear when you log into the sharing server you are about to create. Click Add and Close. For you who know about such things, I believe that you have just created a Mount Point. While you are still looking at the Sharing tab, not the Network path. You can select and copy it if you want. You will need to know the first part, the server name, later.

2) In the Windows control panel, choose Network And Sharing Center.
-- You'll see a little house that says Network next to it. If you have not already done this (and why are you reading this if you have?), it will probably say Work Network underneath that. Whatever it says, click it. You will be presented with three options. I tried Home and Work. Home is the one that worked best.

-- It will show you another page that lets you select your libraries. I don't have or want any of the things it shows so I clicked them Off. (In an early iteration, I selected Documents. It did no good.) Then click Finish.

--It will offer you a random password. I never found a use for it and it is easy to change (Network and Sharing Center->Choose homegroup and sharing options (middle bottom)->change the password. But, as I said, I never found a place where I could type it.

-- Also, since I am working on a virtual machine, I do not keep a password on my Windows login. When I tried to access the new server from my Macintosh with my userId and (blank) password, it said the password had expired. To workaround this, I went to Network and Sharing Center->Change advanced sharing settings (left column - great UI!, not) and clicked "Turn of password protected sharing". If you read carefully, this allows guest access to the shared folder. Since I am, technically, visible on my office network, I also clicked "Turn off network discovery" to reduce my visibility.

3) Back on the folder you want to share, right-click and, about a third of the way down is a selection called Sharing. Do this and choose, homegroup (read/write).

4) On that same tired folder, right-click->properties->Security. Click Edit, then Add. There you will see a place where you can Enter an Object Name. Enter "everyone" (without the quotes). Click Check Names. It should capitalize the word. Then OK. You will see the list of "Group or user names." Make suer 'everyone' is selected and, in the panel below, choose Full Control. Then OK to get out of that property panel.

Back on your Macintosh, go to the find, do command-k (Go->Connect to server). Enter smb\\SERVERNAME (the one I told you to notice in step one). This should present you with a dialog box for your userID and password. If you have a password, use this. Else, choose guest. The system should present you with a list of mount points, the one you made and one called User. Choose the one you made.

I made it so that this pointed at my javascript project. I created a bbedit project for it and have been editing nicely since. Interestingly, if I have editors open on both sides (Visual Studio and bbedit) looking at the same file, it takes care of the potential conflict. If I change it on the Mac side and save, when I click on Visual Studio, it tells me it changed and asks if I want to load the changed version. If I change it on the VS side, bbedit just changes the file I am seeing.

Cool stuff.

IE 8 is no better

Today I spent a long time debugging an error in IE8 that said "Object doesn't support this property or method."  It was in some testing code. The line was:


This works perfectly in Firefox.

The solution? You're gonna love this. "hello" is a reserved word in IE. Change it to:



Oh, but that's not all. IE8 has it's own definition of window.setTimeout(). 

The rest of the world defines it thus:

setTimeout(function, milliseconds, arguments);

The 'arguments' are passed to the function when the 'function' is called after the timeout.


setTimeout(function, milliseconds, language);

Language? It very nicely explains that you can't pass arguments to the callback function. It doesn't explain why it doesn't do this obvious thing that everyone else does.


timeoutProxy= function(func, milliseconds, scope, args) {

    if ($.browser.msie == true) {
            setTimeout(function() {
                        func(scope, args) 
            }, milliseconds);
        else {
            setTimeout(func, 1000, scope, args);


timeoutProxy(setAcceptClicks, 500, this, {});


Getting back to JavascriptMVC

Having decided on a basic navigation tool, I needed to get it from demo-land to actual implementation.

First thing, I had to decide a million things about the actual structure and this is hard for me. I have three full levels of control. The topmost basically comes down to a set of departments, each having a set of functional roles that each require some set of tools. Last time I was using an MVC framework, the navigation was separate from the mvc setup. This didn't seem right so I made a navigation controller. This, of course, means that I have to figure out inter-controller communications. I also have to figure out an addressing structure.

Remember, this is javascript. There are no real URLs. There is exactly one html file and any url goes to it. What I want to do is make it so that URLs look normal and are meaningful. If I can't find a way to update the URL in the menu bar, I will go for a specific bookmark user interface widget where the user will be able to copy a link that will work. JavascriptMVC has some function that purports to do this. It looks like it relies on the URL anchor (#). I'll change that if I can. I want perfectly clean URLs.

It wasn't too hard (ha!). I made a class for navigation elements, set a click listener for them. I use the id of the control to tell me the 'path' which I will use as if it came from the original page entry using a bookmarked URL.

The URL is structured as http://domain.com/Department/Role/Tool/arg1/arg2. And here I find another issue. The usual MVC URL structure is http://domain.com/Controller/Action/args. That suggests that the Department field would correspond to the controller.
Eventually, I decided to count from the other end. Considering the last two to be the Controller/Action and that makes more sense to me. So, I name the controllers, "DeptnameRolenameController" and it will have, among other things, some methods called "Toolname". The real use for the department is to name a DIV that will serve as a canvas for it's controllers and to change the toolbar that is displayed.

My current problem is that I can't figure out how to communicate to my controllers. The current structure is that the navigation controller receives a navigation click. It figures out what controller and action needs activation. Presently, I issue a 'navigationClick' event. All of my appropriate controllers listen for it, check the path that is sent with it and do what comes naturally, ie, ignore it or act on it. This has the advantage that I can have non-navigation things pay attention to the event, logging comes to mind.

The downside is that I've had some trouble making sure that all events get processed in the right order when there are several listeners. It also seems a little wasteful to have all controllers listening. I'd like to be able to simply address the controller.action() but I can't.

I fully believe that I am missing something fundamental but have not been able to figure it out. I have tried everything I can think of. Fortunately, publish/subscribe works.

User Interface

Things have been a little vague lately. At the end of last week and Monday, I finally got clean Ajax communication twixt hither and yon. JSON.NET, it turns out, does a fine job of sending the object. I haven't taken the time to understand the algorithm exactly but it contains data deeply enough to support my current requirement. As I get past the demo phase, I'm sure I will revisit the topic. I actually made the old demo, category list that clicks to reveal a list of details, work. It made for a good boundary.

So, I turn my mind to user interface. I have an incumbent application with a particularly old-school user interface. It works surprisingly well but it's looks stodgy. I figure I need to find an approach that does two main things: organize a lot of complexity into some sort of workspaces or toolsets or something, and have some snap so that this thing sells once it's complete.

I actually spent a couple of days reading about user interface. I am amazed at how little real discussion there is. There's Jacob Nielsen, of course. He's really the main game if you are interested in learning things about user interface. In two days, the only practical advice was his. His 2008 best apps on the web list included the observation that toolbars has won. He had been reluctant before, but a number of the best apps had made good use of them.

I looked around for some info on that idea and found that the world is surprisingly short of straightforward user interface frameworks. Seems like imagination in the jquery world ended at with tabs and widgets. Eventually though, I found two toolbar packages. Both were pretty nice. One, however, was abandoned. The other had a nice thing it calls a backstage. A menu in the upper left that opens an overlay page structured into a two level heirarchical menu and an information pane (though I haven't been able to get the info pane to work yet). You can check it out HERE.

Actually, that's V2.0. The guy has something else he wants to do with it but hasn't gotten around to it. He says, no repository, just grab the code.

That's cool with me and I did. 

Custom Error Page IIS7

Just spent a happy hour or so debugging misleading error messages to find out that Microsoft requires an obscure second action to activate custom error pages in addition to setting a custom error page.

To serve a custom page on an error (in my case, 404), you go to the IIS control panel, select the site and double click on the icon marked Error Pages.

This will give you a nice list of the usual error messages. You can double click on one of them and, using the handy browse button, select your new error page.

Then you can go to a web browser and induce an error (in my case, typing the domain with a gibberish page name). You may be rewarded with a 500 error message that says that you cannot have an absolute file path in website.config, or some such (sorry I didn't get it exactly). You will look at the path name, read other path names, and eventually change the "C:" to "%SystemDrive%". This may reward you with more complicated error messages.

Eventually, it will give you a 404. Hooray, that's what I was looking for. Except that it's the standard page. You may spend more time trying to figure out how you misrepresented the file name. You may notice the Feature Settings button and try it. Putting the file path into its Custom Page field won't work either.

It turns out that you need to click the radio in the Feature Settings that says, "Custom Page." Then it will pay attention to the fact that you entered a custom page already.

Do I sound bitter?

nHibernate and JSON.NET

This is a problem with LINQ to SQL as well, or maybe it's just a problem with the .NET JSON converter. Whichever, the problem is that the retrieval from database to an object graph produces data that, when given to Json(), produces an error claiming that it has discovered recursion. Of course, it's right. A Relationship object links to a Patron object that contains a list of relationships (one to many) each of which links back to the Patron which has a list of Relationships.

My initial plan was to write something that prunes the object before sending it to Json(). This is a tricky bit of work and the general case still eludes me. But, along the way, a coworker told me about JSON.NET, a converter that is claimed to be vastly superior. So far, it is. At first, it produces the same error. However, it has a setting, "ReferenceLoopHandling.ignore," that makes it go away.

I'm not sure if the resulting data structure is complete (there is evidence that it prunes a little too much) or if it has more settings to make it work better. Still, it's a start.

Sometimes Things Work

I'm working toward a routine that prunes nHibernate object graphs (that's what the cool kids call them) so that they can be serialized for JSON transmission without getting an error because it's recursive. In the process, I've been experimenting with ways of handling objects retrieved by NH. Bottom line, it is easy and works the way one would hope. In particular, I have retrieved a triple object, people, bars and relationship from the people side, then changed the data on some of the relationship records. A simple flush() and the correct records get back to the database with changes in place. The amount of not having to write a bunch of save routines I am enjoying is marvelous. Similarly, I made a routine to populate the database and it works nicely, too. FYI

IIS7 Static Variables Refer to the Entire Server Context

I just had a conversation that saved my bacon BIGTIME!!

Static variables are shared across the entire server. If you declare a static variable in your web application and have ten thousand people using it, when one of them does something that sets the variable, it changes it for everyone. There does not appear to be a way to create any sort of global variable that is not also universal.

The workaround is to use the http session object instead (though this is a crime, in my opinion).

DataType ShouldBeStatic = (DataType) System.Web.HttpContext.Current.Session["ShouldBeStatic"];

What I can't believe is that this isn't emblazoned all of the universe when you google ".NET tutorial". There is nothing like it in the LAMP world. I can't imagine what sort of problems I would have suffered if I had not learned this and then the problems of going through an entire application to fix it. Bullet Dodged!