rsync error: error in rsync protocol data stream (code 12)

The internet failed to tell me:

This error can result from not having one of the directories present. Yes, I know that rsync creates lots of directories very nicely. Not all of them.

To wit:

rsync someDirectory someUser@1.1.1.1:/home/someUser/system/code

Gave the the data stream error until I created the directory 'system'.

I can't imagine why. The only thing that is distinctive about 'system' is that it is also ~/system, ie, at the top of the user's directory.



Windows is Bad, Example #billion

I often note that whenever Microsoft has a decision to make, they unerringly choose the worst alternative. Most of these are so small that I forget about them but today, I ran into one that is so perfect, I want to remember it forever.

As with my Macintosh, one way to rename a file is to click once directly on the text of the name. It will turn into a text entry and select the name, awaiting the new file name.

Occasionally, I want to append something to the name, eg, BACKUP, HOLD, TMP, DISCARD, etc. So that I can use the real file name for something new without having to lose access to the old file.

On my Macintosh, I open the name for editing and touch the right arrow. This moves the cursor to the end of the selection and lets me keep typing.

On my Windows machine, it moves it past the end of the selection and then past the period so that I am changing the file extension. While it's not impossible that this is what I want (on my Mac, select-all, right-arrow, delete delete delete), it's relatively rare.

Which is to say, Windows forces me to do extra work to accomplish the most common function. Like always. It also increases the likelihood of error.

How much does Windows suck? Just sayin'.

Access Web.config from Classic ASP

I am in pain. The things I do for money. Visual Basic in asp. Kill me now!

I am handed this app with the URL of a JSON endpoint hard coded into it. That might not be horrible except that this is not a one-shot application. This is something we sell to a lot of people and the endpoint needs to point to the customer's domain or that nasty cross browser stuff will bite you.

After sneering about the laziness, I set about looking for the equivalent of getEnv() or process.env.varName or any of the sane ways that other language systems provide access to configuration data.

Nope. This is Microsoft after all. Not only can't I find said simple 'read the config' function, I can't really find a straight answer. (The fact that I didn't know to google "Classic ASP" didn't help. - Hey! Stop laughing. I still am not sure what the language is called. vbscript, visual basic, asp... WTF? I hate Microsoft apps. I'm only doing this because nobody else in my shop could figure out why it was broken.)

Of course, StackOverflow eventually came to the rescue, sort of. A nice person named Connor worked through this problem. It didn't work for me right away but, thanks. It saved my bacon. I present it here so I can find it again if I am ever again unable to escape a Classic ASP task, and so that it has more good google keywords. This is copied directly out of my app. It works.


'****************************** GetConfigValue *******************************
' Purpose:      Utility function to get value from a configuration file.
' Conditions:   CONFIG_FILE_PATH must be refer to a valid XML file
' Input:        sectionName - a section in the file, eg, appSettings
'               attrName - refers to the "key" attribute of an entry
' Output:       A string containing the value of the appropriate entry
'**********************************************************************************

CONFIG_FILE_PATH="Web.config" 'if no qualifier, refers to this directory. can point elsewhere.
Function GetConfigValue(sectionName, attrName)
    Dim oXML, oNode, oChild, oAttr, dsn
    Set oXML=Server.CreateObject("Microsoft.XMLDOM")
    oXML.Async = "false"
    oXML.Load(Server.MapPath(CONFIG_FILE_PATH))
    Set oNode = oXML.GetElementsByTagName(sectionName).Item(0)
    Set oChild = oNode.GetElementsByTagName("add")
    ' Get the first match
    For Each oAttr in oChild
        If  oAttr.getAttribute("key") = attrName then
            dsn = oAttr.getAttribute("value")
            GetConfigValue = dsn
            Exit Function
        End If
    Next
End Function

settingValue = GetConfigValue("appSettings", "someKeyName")
Response.Write(settingValue)


ps, here's Connor's post: http://stackoverflow.com/questions/28960446/having-classic-asp-read-in-a-key-from-appsettings-in-a-web-config-file/34779392#34779392



why use "console.log.bind(console)"

Oddly, I never saw this construct much until recently (late 2015). I suppose that's because the problem it solves wasn't common before promises arrived on the scene and people wanted to pass console.log around for debugging purposes. When it did start showing up, I was perplexed. I am 100% comfortable with the details of the bind() function and why to use it. I could not, for the life of me, understand why it was needed here.

The answer:

It's not needed. I tried to get the behavior discussed below in a few contexts and it appears that console.log has been revised so that this isn't needed. Or, maybe it is. I didn't do a comprehensive check of the entire world.

Well, then, wtf?

Turns out that, in some previous world (perhaps some browsers today that I don't have the patience to experiment with), console.log() contains a reference to it's own this which is lost in a simple function assignment. IE...

If you, for some reason, want to pass the function console.log around, eg,

var xxx=console.log

xxx('message');

It won't work. The console.s log() method refers to this.something that isn't a part of it's new, post assignment 'this' (ie, the global scope, which is 'this' when nothing else is applicable).

That is, if you could magically add console.dir(this) to console.log, you would get something in this spirit...

{

log:function,

somethingElse:godKnowsWhat

}

But, if you were magically able to get that same console.dir(this) after the assignment to xxx, it would produce something like this...

{

//global scope items that do not include anything name somethingElse

}

The construct in question specifies the value of this for the newly assigned xxx

var xxx=console.log.bind(console);

That is, it will be bound to its original 'this', (ie, this===console) and it will be able to find whatever somethingElse it needs.

Mongo Mongoose Objects Weird Properties Can't Access

Yep. I'm all about search engine titles this week.

In my new Mongo/mongoose world, I've exposed myself ot a ton of information, tutorials, explanations, examples, you name it and there is a HUGE thing that no one mentions.

Perhaps I'm an idiot or maybe the fact that I work entirely alone means that I am not privy to common knowledge so common that no one needs to say it but, I'm saying it now:

Mongoose does not create nice Javascript objects. When you do a find, you get something that will console.dir() and look like something you recognize. It is a false appearance. It is some nasty Mongo thing. Eg,...

I had an object that, viewed with my general purpose display program, looks like this:

Doesn't look like it should be a problem. Addressing it as...

var item=recipient[0][0];

recipient[0] gave me the first array element, an object.

recipient[0][0] attempted to to address the object containing the email address and came back undefined.

Nothing I could do gave me the email address. It was insane.

Turns out that I could google the answer pretty easily once I realized what was going on but only after tearing my hair out because no one mentioned that mongoose produces bizarre data structures.

The answer is (drum roll, please)...

.lean()

as in...

query.lean().exec(wrapCallback(callback)); //this produces a plain, Javascript object

as compared to

query.exec(wrapCallback(callback)); //this produces a mongoose object from hell

Obviously, if you are doing your processing in a model, you might really like the benefits of using the mongoose data type. I bet there are a TON of them but, if you are trying to construct a test, as I was, or send it to a web page, like I plan to do, you are going to want something that acts like a civilized citizen, not some rogue toxic waste factory.

But, I wouldn't have minded at all if somewhere in the mongoose quick start and documentation they had said, "The output of find() must be accessed with mongoose utilities unless you add the lean() function to convert it to plain Javascript."

Maybe you got to this note before you actually ruined yourself. Or, maybe you, like everyone else on earth, already know.


Mongo Mongoose Simple Tutorial Quick Start Code Example

How's that for a search engine friendly title? It's a combination of the things I tried as I started my experience with Mongo/mongoose that did not give me what I wanted. What I did get were all sorts of useful explanations and advice. What I did not get is a simple bit of working code to satisfy my Hello World needs.

The code below includes the lesson, 'use createConnection(), not connect()'. The former allows you to access the database from separate model classes without having to pass a connection object around. In the depths of the mongoose forums, the fancy guys admit that they prefer createConnection() so it's ok.

It also has my curried default test callback. I had a problem when I tested the example that required it so I left it in so you can solve whatever screws it up for you more easily.

I had two problems you might want to know about:

You will notice that the boilerplate has a peculiar model name (userTqTest). I built this example inside my real, working project. It turns out that Mongo does not forget old model definition stuff. I haven't quite worked out the details but the bottom line is that when I declared this model as User and saved, I got errors based on the validation in my previous, real model. That is, the mongoose model does not supersede the one inside Mongo itself. I got over it by, first, dropping the collection. Then, since I don't want you to have to drop your User collection, I changed it to a name that is very unlikely to collide with you database.

Second is one that you probably will notice instantly but, because I was focused so much on showing the simple syntax, I didn't think about right away is that these calls to save, get and delete are asynchronous. They are triggered in the right order but they are happening in whatever order Mongo takes care of it.

I didn't want to complicate the example with async or some other method to sequence the calls, so I didn't. This code definitely works if you run the three functions one at a time but give unpredictable results otherwise. My problem was that the delete fired before the save was complete leaving the record to give me a duplicate key error on the second pass.

Enjoy.

========================================

        var mongoConnection = mongoose.createConnection("mongodb://localhost:27017/test");
        mongoConnection.on('error', function(err) {
            throw ("user code says, mongoose failed")
        });

        var userSchema = new Schema({
            userName: {
                type: String,
                unique: true
            } /*obviously you'd have more properties*/
        });

        var userAccessor = mongoConnection.model('UserTqTest', userSchema);

        var saveUser = function(user, callback) {
            /*you'd probably do some validation even though the mongoose will also validate
            you'd also wrap stuff in try/catch but that's not the point of this example.*/

            var newUser = new userAccessor(user);
            newUser.save(callback('saveUser' + '/' + user.userName));
        }

        var getUser = function(user, callback) {
            var query = userAccessor.findOne(user);
            query.exec(callback('getUser'));

        }

        var deleteUser = function(user, callback) {
            userAccessor.remove(user, callback('deleteUser'));
        }

        var callback = function(callingName) {
            return function(err, result) {
                console.log(callingName + '============');
                console.dir(err);
                console.dir(result);
                console.log('=====');
            }
        }

        saveUser({
            userName: 'tqwhite'
        }, callback);


        getUser({
            userName: 'tqwhite'
        }, callback);


        deleteUser({
            userName: 'tqwhite'
        }, callback);




   


Running Multiple Versions of NodeJS on One Server

In the modern age, node installation is handled by the 'n', eg,

sudo n stable #the current stable version

or,

sudo n 0.10.40 # to install v0.10.40

Once you have installed a version, it never goes away. You can switch back and forth between versions instantly. That means that you can easily alternate between two (or more) versions if you want.

But, you might (as I did) have an app that must run on an old version but you otherwise want to use a modern version. That is, you want two different versions active at once.

This can be done by referring to the node binary with a fully qualified path. (Remember, when you type "node", your shell is simply giving you /usr/local/bin/node.)

The program 'n' allows you to find the path to the binary (assuming that you have previously installed it), ie,

n bin 0.10.40

->/usr/local/n/versions/node/0.10.40/bin/node

You can then type

/usr/local/n/versions/node/0.10.40/bin/node SOME_OLD_APP.js

into your shell or put it into your initialization script or whatever.

I create separate users for each of the apps I have running on my server. There is only one that requires the old version of node. I want to use the latest stable version for everything else so, I have done "n stable". For the user account where I run the old app, I change my .bash_alias file to include...

alias node="/usr/local/n/versions/node/0.10.40/bin/node"

Then I can type

node SOME_OLD_APP.js

and get good results, or,

node --version

->v0.10.40

But when I log in as the user for a different app, I get the latest stable version.

Obviously, you can do this for as many node versions as you want but, if you need a lot, you might want to think about why (for me, it is Meteor, which only runs on old node).


UPDATE:

I have had occasion since I wrote this to install a new server. I was reminded that the utility 'n' does not come automatically with a NodeJS install. After you install NodeJS (which also installs npm), type

sudo npm install n -g

and the you can verify it by typing

n --help

and you will get all kinds of good info.

Color and Other Format for Fargo.io

Dave Winer is the guru of outlining. He is the person who moved outlines from English class onto the computer in the eighties. He's done a lot of other things, too. But, his development tool Frontier was a game changer for me. I eventually had to leave it behind because my brain needs color. I need code coloring when I'm programming and I need color cues to work productively with an outliner (everything else, too; my emails alway have colored sections).

His newest outline, Fargo, is very cool (at http://fargo.io). It runs in a web browser and saves your files into your Dropbox. Awesome. Unfortunately, it's also black and white and serif type face. I don't like that and, honestly, I have a hard time working with black type if there is very much of it. I tried it out. Liked it. I continued to use Omni Outliner. (If I could get Frontier to use Omni's formatting, I would be so happy.)

Lately I've been working on a very outline intensive project and now I want to work on it with other people. But, it really needs to be an active outline with expanding and collapsing sections.

I discovered that I could export the outline from Omni into OPML, the basic data structure Dave uses for Fargo. I copied the OPML into the Dropbox file Fargo uses and, boom!, I had Fargo functionality.

To my eye, it was ugly but it turns out that Dave did a BEAUTIFUL THING. He made it so that you can execute Javascript from the outliner. No kidding. You just type some JS into the outline, hit cmd-/ and it runs.

Here's what I did:

$('body').prepend("<script src='http://static.tqwhite.org/iepProject/formatFargo.js'>");


That's right, I loaded a chunk of JS from my static server. That code changes this:



to look like this..



It took a fair amount of reverse engineering to figure it out but, it works like a charm.


Here's the code:

(I think the colorized picture is easier to read.)


And here it is if you want to do your own colorizing:


var colorize = function() {
    $('.concord .concord-node .concord-wrapper .concord-text').css({'font-family': 'sans-serif'});
    $('.concord-level-1-text').css({'color': 'black'});
    $('.concord-level-2-text').css({'color': '#664F58'});
    $('.concord-level-3-text').css({'color': '#456D72'});
    $('.concord-level-4-text').css({'color': '#AD9470'});
    $('.concord-level-5-text').css({'color': '#D3AF74'});
    $('.concord-level-6-text').css({'color': '#90967E'});

    $('.concord-level-7-text').css({'color': '#778'});
    $('.concord-level-8-text').css({'color': '#788'});

    $('.concord .concord-node > .concord-wrapper').css({'background': 'white'});
    $('.concord .concord-node.selected > .concord-wrapper').css({'background': 'rgb(245,250, 250)'});
    $('.concord .concord-node.selected').find('li .concord-wrapper').css('background', 'rgb(245,250, 250)')
}

$('body').bind('keyup', colorize);
$('body').bind('click', colorize);

colorize();

document.styleSheets[0].insertRule(".selected { background:rgb(245,250, 250); }", 0);
document.styleSheets[0].insertRule(".selected div { background:rgb(245,250, 250); }", 0);
document.styleSheets[0].insertRule(".selected div { color:normal; }", 0);
document.styleSheets[0].insertRule(".selected i { background:rgb(245,250, 250); }", 0);







Reinstalling NodeJS and npm

Recently, I upgraded to the latest NodeJS/npm and npm stopped working. It turns out that there was a problem with the OSX installer.

After painful amounts of googling, I found that some had solved it by "tracking down" the node and npm files, removing them and then reinstalling with the node distribution download (dmg) from nodejs.org.

I tracked down the files. Do this:


sudo rm -rf /usr/local/bin/node

sudo rm /usr/local/bin/npm

sudo rm -rf /usr/local/lib/node_modules/npm


And then hit up http://nodejs.org for a new installer. You will have a fresh working installation.


PS, the problem with npm was this:

When I typed

npm init

to start up a node module, I got errors that included

Error: Cannot find module 'github-url-from-git'

Turns out that basically everything I did with npm except --version was broken in this way.

The 'delete before reinstalling' process listed above fixed it.







Visual Studio 2015, .NET 5 rc1, dnu restore, asp.net missing (I can't believe it either)

It's been a half dozen years since I started a new project in Visual Studio. I was a little excited at the prospect. I like learning things and I know a lot about almost all the rest of the internet development topics.

I looked up the latest stuff and it turns out that we have a new Visual Studio and a new .NET that have taken a lot of good lessons from the rest of the world of web development. .NET 5 is out of beta and into Release Candidate 1. That's good enough for me. I expect the bugs will be small.

Wait. It's Microsoft and everything they do is stupid.

Problem Zero (which I won't detail)

I actually went through fits trying to get it all installed and looking good, but, having done that, I create a new project: ASP Web Application/ASP.NET 5 Web Application.

Problem One:

I have to do this twice. I keep code on an external drive that the file dialog navigates to as //psf.stuff... . It tells me I that "UNC paths are not supported." The second time, I typed (not navigate) the volume letter, "X:", and it worked.

Problem Two:

I build the solution. I get a bazillion (well, 204) errors. The first one tells me that "The type or namespace name 'Identity' does not exist in the namespace "Microsoft.AspNet'". Another, "The type or namesspace name 'AspNet' does not exist in the namespace 'Microsoft'". Can you imagine?

The project listed in the first error says "TestProject.DNX 4.5.1, TestProject.DNX Core 5.0" (obviously, the 'TestProject' is my project name). For the second one, it's "DNX 4.5.1" only.

I try using Nuget to add "Identity.Core" and it changes things. I screw around with that for awhile as new missing references appear until I start getting messages telling me that I have duplicate definitions. This is truly awful. (Did I mention that Microsoft always does it stupid? The package manager doesn't make sure the references are correct? Really?)

Problem Three:

I start over and this time I decide that I'm working toward .NET 5.0 so to hell with 4.5.1. I edit the project.json file and remove it. Build takes forever and I pretty much expect everything will blow up but instead, I get a message, "Dependencies in project.json were modified. Please run "dnu restore" to generate a new lock file."

This feels like progress. I right-click on the project, choose Open Command Line and type "dnu restore". It works. I return to VS and build again. It instantly repeats the exact same message. I delete the lock file and restore it. Same thing. A complete, stupid dead end.

THE SOLUTION (and a lesson is the complete depth of Microsoft stupidity)

I reverse the order of the references to 4.5.1 and 5.0 so that 5.0 comes first. IE,

I change from this:



to this:



The build succeeds promptly. Clicking the IIS Express button opens a web browser and shows me the scaffold web page.

This has taken me over 2.5 hours. Certainly, my inexperience with this technology made it slower. Someone better might have done it more quickly. However, this is the scaffold. This is the part that supposed to save time. This is an epic fail on Microsoft's part. I mean, they put the dependencies in the scaffold in the wrong order!!

The good news is that, if I got enough Google-friendly text in this page, you might have found it well before 2.5 hours elapsed.

Of course, that just means you need to endure Microsoft's next awful surprise. Good luck. I know I need some.