Общо показвания

февруари 05, 2012

Developing frontend with node js and cloud9

Not so long ago I was describing how disguised I am with the whole nodejs and cloud9 story, how it was working bad, how it does not support 'MY' use case and so on.

Well as with most things I tend to re-evaluate my position from time to time, just to be sure and sometimes to check if things are progressing.

The last few days I was starting an spin-off of my Tornado UI project ( an Open Source interface for Tornado IPTV devices, read more ). The new code is loosely based on the existing one, re-using some of the components, but still is a chance for fresh start and evaluation of new options in code organization and structuring.

Part of the code for the previous project was ported to closure-library to allow it to cleanly compile with closure compiler. I can gladly tell that it was a success, all tests of the converted code passed and it is available online for anyone to use it in a closure based project. It covers completely the communication layer for Tornado, but does not provide any UI. It assumes the closure developer would use closure techniques for the UI as well and I doubt my ability to fully utilize closure paradigm for UI components.

The effort to use closure library was an enlightening in a way, allowing me to free myself from Crockford's mantra for privacy in the code. Meanwhile I have read tons of posts and articles about patterns, speed, reuse as well as package management, what Harmony will bring, what should we do until then and so on and so on, basically re-educating myself on the matter of serious JavaScript.

One big problem for me was the fact that I am not actually using any of the power of the 'abstraction' the libraries provide (i.e. Jquery, MooTools, closure and so on), when developing for a known environment one does not need abstractions and polyfils (well, mostly does not, if you have seen the code for TUI, you'd notice that bind is not part of the Function prototype). It just bloats the code and rarely enforces useful patterns ( unlike closure, mootools and jquery allow you to code in any style possible, even with no style).

Another problem was that I wanted to be able to test the code on a test server or at least in a console with cli, instead of loading it in browser every time. At the time I started the project there was no clean way to run the same code without any change on node and in the browser. Or at least I could not find it.

This week I started the new project, it aims to bring new features to the Tornado line of device, at least two of which will be co-developed with the in-house development team at the company. It will also be able to work on any android device (2.2+). This time I was determined to use some CLI testing for the code to speed up development and bug catching.

I went looking for a suitable tool again. First I was thinking about closure library and found nclosure. Unfortunately it seem to be not updated lately and does not work with node 0.6+. Then I tried JSClass, it require shift in package management, but still is manageable. Unfortunately the dependency tracking and loading is not working as I expected, maybe it is me, that cannot figure it out, (even though I figured out closure and requirejs), but after two attempts I gave it up.

The last was amdefine. It requires node 0.6+ but apart from that works very well and suits me perfectly. Very good explanation on how to use it could be found on their github page, linked above.

Lastly I wanted to try cloud9 again. I downloaded the latest source from github and it work very nicely with node 0.4. Unpleasantly for me it does not work with node 0.6, while amdefine requires it, which makes it impossible to use nodejs as test run and cloud9 locally. The only possibility remained hosted version of cloud9. Recently they announced support for running your code on node 0.4 or node 0.6, configurable from your IDE. Month ago they also featured npm command line interface, so I figured I should try it.

A working setup follows:

On c9.io open your project and add requirejs and amdefine:

npm install requirejs
npm install amdefine

In your project, configure the test runners, one for node and one for browser. Node:

var requirejs = require('requirejs');

requirejs.config({
    baseUrl: __dirname + '/javascript',
    nodeRequire: require
});

//Use eyes-console in node js for prettier output

requirejs(['tests', 'debug/eyes-console'], function(tests, SimpleConsole) {
//    Enable logging in node
    (new SimpleConsole()).enable();
    tests.run();
});

Browser:

require.config({
    baseUrl: "tablet/javascript"
});
require([
    'tests',
    'debug/simple-console'
], function(tests, SimpleConsole) {
    (new SimpleConsole()).enable();
    tests.run();
});

Note: the past are relative and fit my project, those may vary for yours. 

The browser runner requires html file as well, however the node test runner can be run as easily as starting a node process with the file. Remember to set the run process to use node 0.6.

Conclusion: Cloud9 ide is a viable option for developing and testing your code, even if you are developing only a front-end application. However not all approaches for code organization work, currently I have managed to make only AMD with requireJS. I expect the future to bring more variety in the options.

As a side note, Eclipse Orion has much better git UI (as a real UI, not only cli), however it is still in a very early development and on top it does not support running node instance, so you cannot really test your JS code in it.

декември 23, 2011

TUI status update

The Tornado UI project is now more or less in stable phase, there are some small issues here and there, but the transport layer is stable and the HTML UI is complete. Also the closure-library link for the transport layer is ready.

As of today the develop branch from the last 2 months is merged in the master, which means that it is useful as it is, no need for special setup. You can just host it anywhere and point your Tornado device to the start URL (edit the M55/M70 settings and set 'Start URL' and 'Use start URL' to true).

The one piece still missing is the picture browser, we are working to resolve the memory restriction making web-kit to crash after a while with large files. Once this is ready you will have a complete interface, fully customizable via CSS only and very easy to extend with JavaScript.

The code is available on GitHub.

The documentation is progressing well, for now it covers only the transport layer and its abstraction layers, but the plan is for it to include the UI elements as well for easier use of the source as SDK for your own AV application or even your games.

The binding for closure library will be released soon, we are still clearing some compiler warnings in strict mode with type checking. Stay tuned. Using the closure compiler brings some new features, like compilation for a particular device, cutting out large portions of the code base and making the application faster. We are also working on some nifty features like app cache and local storage for even faster loading as well as some advanced visualization features, that will run on the newer devices and will bring the cutting edge presentation layer one expects from a modern web application.

декември 22, 2011

Списание Дъга

Ако сте над 25 годишни и сте от мъжки пол (пък даже и от женски да сте..) сигурно си спомняте списание "Дъга". На мен ми беше любимото. Нещо като социалистическа версия на "комиксите". Интересното при него беше, че освен разказите съдържаше и игри и статии. Също друго, много интересно нещо бе, че различните истории се рисуваха в различен стил, което развива артистичните усещание у младежта. Нещо такова.

Та така де, всички броеве можете да разгледате или свалите от тук: списание Дъга

What is better than Glee?

Short answer: Mary Lou by Eytan Fox. 


Why? Because it treats real life and real problems and not high school imaginary drama with bunch of adults in their mids pretending to be high-school students. Yup. 

And the music is better. I like Svika Pick! And you like it too, if you like Harel Skaat. 

Attention (Anti Semites): the movie is from Israel and is in Hebrew
Attention (Anti Gays): the movie contains some homosexual imagery and transvestite imagery - nothing profane though. 

декември 16, 2011

Developing JavaScript in Vim

Vim is great for coding, anyone can tell you that. It is customizable and extendable beyond anything I have encountered. However this is not seen as s plus immediately by JavaScript developers.

In this post I will try to summarize my experience with Vim for JavaScript programming and in particular with closure tools and node.

First of course you will need Vim 7.x. You will also need exuberant ctags or gjstags. The former is preferred though (explained later) and a JavaScript hinter/linter (jsl, jshint, jslint or gjslint).

Now for the vim plugins: you will need syntactic, taglist, nerd three and writeJSDOCcommen.

ctags is used to annotate your code and while it does not have the fancy options of Aptana Studio or Eclipse (like annotation for variables in the scope of a method or the properties of an object enumerated in the Object annotation as well as properties on its prototype all arranged in a nice tree view) it has something I often fine more useful - the fact that it is able to append the annotations so when you have opened 10 files and you see only one in your editor you still see all annotations for all open files. This is especially useful when working with code you do not know well (like the vast closure library for example, did I mentioned that I struggled with a tool to help me getting better knowledge of closure).

To use ctags effectively in vim I would suggest taglist. I use the following option setting in my vimrc:

let tlist_javascript_settings = 'javascript;f:functions;c:classes;m:methods;p:properties;v:globals'


This is - I want everything annotated in JavaScript files. I also use this shortcut:

nmap :TlistToggle


In ftplugin/javascript.vim I also have:

nnoremap   :call WriteJSDocComment()

This is not extremely useful, but a time saver. This was for code annotation, not lets begin with code checking.  I used to go with node js and jshint, but the truth is it does not make sense to install node js just to run hinting on it - after all it is marketed as server side JavaScript. Hinting is actually better done with syntactic.

By default it will look for any linter installed, I tried gjslint (the closure instrumentation linter) and jsl. The former tend to be too much closire library centric so probably the later would be better if you are not developing with closure in mind. Also the gjslint executable is much slower on larger files.


The last is NERD_tree which is not JavaScript must have, but it helps a lot see and browse your files as you do in any other IDE.

What I was not able to find is help system that is able to read JSDoc annotation for JavaScript and hint the usage. If anyone knows about such, please let me know.

PS: I often switch OSes, Looks and feels, IDEs and editors, browsers etc. I like diversity. However to this day I have not found the perfect JavaScript editor. Every one out there has a feature that I like but the others lack and each one lacks multiple features I like. Looks like finding the perfect tool is a never ending quest.

октомври 14, 2011

What Is Wrong About 17259 Lines Of Code, Q&A

Web Reflection: What Is Wrong About 17259 Lines Of Code

I can answer and it is pretty simple and they have answered already by the way, you seem to just failed to read it: They have the resource and the will. Which means they can put 30 or 100 developers into something for months and even years and pay them to do something that might fail epically and still they would not mind! While the smaller companies and single developers cannot invest months into something that might fail or just be useless.

On the 'why' - it is just an experiment. The high expectations are to be the browsers that 'runs it native' and maybe they believe it will give chrome and chromeos some better user acceptance. Or maybe they really believe it is a step forward.

Anyway, just let them be. AFAIK very little number of companies are using the other attempts: GWT, Closure library, Go and so on. If we leave it alone it might become just one more project of them.

октомври 03, 2011

Node madness

I may or main not know - I make living as javaScript developer. As such I invested time and energy even beyond my 8/5 work schedule - I often fund myself involved in on-line activities related to augmenting my skills in the language I support my family with.

This said, it is explicable that I as excited when nodejs came around. I was not exactly one of the first adopters and around he time I tried it there were connect and early form of express and other middle ware and frameworks that ease the server side development. And I was relieved! Because frankly who wants to reinvent the wheel.

Yeah, node promoted itself big time in the JavaScript community, promising you the ability to write your logic once and run it anywhere, the server and the client alike. There are even companies that struggled for almost two years to reveal a large scale product based on node (like nodejitsu and cloud9). Interesting and useful (mind you!) things came to life - Socket alternative (like commet for python) at your disposal that you can code for in the language you already know well, for example. I for one was struggling once with Comet!

This however must come to an end. Why? Well, the thing is node might be great for some subset of application cases, I for one have never had one of those however. Node still does not have great SQL linkage (unlike any other server side language you still cannot reliably use stored procedure in the SQL server and have reliable return values in your JavaScript, I my practice we had to modify the procedures to have only one SELECT to be able to parse the results correct in mysql-node.

And today I read this article that says it well : node js is cancer. Why? Because it is over praised and use for "everything", while its niche is in fact pretty small. The fact that get to use language on the server that you already know might be tempting, but the thing is - it is not necessarily appropriate for everything. And while you applications logic start requiring ultra dynamic server side + client side mix of JavaScript it will never be!  I mention that because there are good projects out there that allow you to call server side JavaScript objects and method and this sounds very tempting to me, as on the server I can rely heavily on standards (V8 for example) this is mostly not the case in browser environments. On the other side this is somewhat simplified work around for cases where server side calls are performed and executed in other languages - i.e. the client side does not need to care about the server side implementation, which IMO makes this somewhat pointless, unless you need to write the server side logic as well.

My disclaimer: I have release exactly one web project completed entirely in node.js environment. And to tell you the truth - very simple one. And I gave it all up. Why? Because this is simply bull shit! I had to re-create in JavaScript functionality that was build and ready to use for years in pretty much any other server side language. And on top of that I had to learn new tools for server side deployment, 'forever', node set-up, npm set up and other things just to get this one web site going. And mind you  - the idea and implementation there was easy! The most simple thing, sending emails, took me a day and a half to start working properly. Not because I don't know the language, just because out of the 8 implementations in npm only one was able to do what it is supposed to to - everything else made the main server dislike us. Try this in any other language, be it java, python or php or really anything server side and you will see - it will just work.

So you see, I am not against node.js. I am against using it as general purpose server implementation.

Yet another thing I have to confess: I use node applications to JSLint my client side code, I use jade executable to compile my client side templates and I use uglify JS to minify my code. I sometimes also use requirejs with node to minify and compile the client side code before deployment. But notice - those are only tools. I would never use those on server side application.

How I see the future? I think most useful it would be to just make node work as library on top of apache like the python interpreter is. And use it only where needed!

октомври 01, 2011

Project shift evaluation (Target: Closure library)

I have given another though on closure library + compiler + templates.

I have downloaded the source, the compiler and template utilities. The source is really well documented.

Unfortunately I have issues with the fact that the compiler understands only certain subset of the JavaScript language, which means that I as developer need to restrain my work to those limitations for the only purpose to be able to compile the code to a usable size once development is completed.

I have tried to compile some of code as it is - it mostly did not work:
if (!window.console.group) {...}
resulted in:
if (!window.console.a){..} 
which is actually pointless. To avoid this I have to write:
if(!window.console['group']){..}
which is then compiler to
if(!window.console.group){...}

I find this exercise to be tedious for the developer; if a statement is valid JavaScript compilation should NOT break it.

I likes the name spaces idea nice, however the module pattern is equally nice to me.

Now about the templates. Our design guys make the prototypes in plain html. It seem fairly easy for a developer to translate this in closure template. However the fact that I need to use it with closure compiler throws it under the bus. Indeed the generated JavaScript is fast and compact, but fast and compact javascript is generate for jade templates also (and probably other templates for the node platform, those can be 'browserified'. The overhead of translating the HTML markup to Jade markup is not that bug as the Jade is fairly simple.

As of the compilation always valid code with minification is produced by uglify.

I was hoping for some more advances when dealing with UI, for example loading the css, the template and the logic of widget on demand. Nothing like that, at least not at first glance was found in Closure.  Seems we will continue handling the css manually and loading the modules with our JavaScript module implementation.

Before anyone gets this personally and start explaining the numerous advantages of Closure, let me tell you, I even went as far as reading "Closure: The definitive guide". I did not found the answer to our problem there. So it might be the greatest of them all, it still relies on coding style, which we do not have and there is no currently plan to port 100000 lines of code to another coding style so that closure compiler can handle it. Maybe one day when it can handle any code we will move on.

Conclusion: The main motivation behind this attempt with closure was the fact it makes maintainability much simpler. For example if we rely on Google to update the core and keep the API, we can assure the longevity of our code.Our code base grew enormous for the last year and we want to support only that, forgetting about browser inconsistencies, we would want to NOT back-port new versions of libraries, we do not like libraries that alter/augment native and host objects and we want either the namespace pattern or the module pattern. We want to be able to minify. Still we have not found all those in one project.

Our current approach is:
  • requirejs for module loading
  • jade for templating
  • css for styling - no processing
  • simple build process that minifies all scripts and then runs the requirejs optimizer. We minify all scripts because most of them are dynamically loaded and the optimizer cannot see them. 
  • documentation is built by an in-house developed script reading the source (comment based as with YUIdoc, however much simpler).
  • development environments are created with scripts that produces the base of the library, the project structures and a test server with local and remote debug server that compiles the templates on the fly.
  • deployment is created with script and is complemented by a build from the development 
  • versioning is done with git flow