The promise of Touch

I rarely write about series in this blog, because I find it kind of boring to praise (or not) a production based solely on your own experience with it.

However as my TV time is diminished lately (and is threatening to approach zero) I had to choose wisely what to watch.

When the premiere of Touch aired several months before it was actually scheduled to begin, the first episode was kind of unusual, exciting, intriguing, promising... Who was to guess right then and there that this same story will repeat over and over and over again with each end every episode and nothing else will ever happen, not even a hint for actual line of events moving forward while the season progresses.

There is of course the possibility for speculation. Maybe this phone travelling around the world will have some meaning at the end, but I would not hold my breath (Lost anyone?). What else? Maybe those Japanese    girls will have something to do with the whole thing? Who knows.. (yeah, Lost, Heroes and so on - you expect things to matter, but the scrip writers think otherwise.

The take away: don't expect too much of those 'sci-fi' flicks. Most of them are disappointing anyway.

The JavaScript ecosystem problem

I tend to follow the news in the JavaScript field because it is related to my work. What I notice lately is that every developer out there is making attempts to popularize his/her favorite framework and/or tool.

Here is my objection to all this: Currently the situation is this - the ecosystem of javascript tools and libraries is fragmented. The pieces are so small and so incompatible that it makes it impossible to build a large web application without actually spend months in development time trying to make the pieces work with each other.

There is jQuery, which unfortunately is now much more than a tool to hide the browser quirks. It now provides pub/sub, promises and what not. There is also a whole lot of 'plug-ins' for jQuery. But those require different version of jQuery. So you might need more than one jQuery.

Then there is 'modular' code, AMD. One being the new dojo (1.7), which is a pretty big modular library, but it is not really compatible with other AD loaders. There is RequireJS, which is also popular because of its 'optimizer', but it is only that - loader, and does not provide any library for browser quirks hiding, so the developer is left with no library at all. jQuery is popular choice in this case.

There is closure library and closure compiler, however those are completely unusable with code that was not specially designed to work with them.

There is prototype, there is backbone there is underscore... every single one of those has its fan base and none of theme follows a code pattern that will allow a developer to just use the needed functionality without any additional hassle.

This of course leads to the decision enforcement: the developer has to make decision which technology to use (some 'ninjas' even go as far as creating their own set of tools/mini libraries and use those, but this is time consuming and error prone and the library takes additional work to be supported and renewed). It does not matter what technology is selected, every single one that is popular has its benefits. What is more worrisome is the following: How much tie will it take for the used tool/library to become incompatible/obsolete?

Lets say that currently I find the closure tools to bring the best combination of browser quirks hiding, performance and code optimization quality. As every single tool out there it imposes a set of rules I need to follow in order to make my code work with it. Notice that __ALL__ tools do this. For example jQuery also requires the whole library to be loaded before I load the  plugin. It also requires a global variable to be exposed ($ or jquery). Most libraries count on a global variable to be available with a certain name. Closure makes exception in this, but imposes much more rules. So let's say that I create a super useful UI component, that is based on closure library. If you want to use it you have two choices: a) use closure library/tools or b) rewrite the functionality with your fave library. The last one is the more popular choice. One and same widget/component is currently being re-written over and over again with little augmentation to the actual functionality. LightWindow anyone? It is available as stand alone, jquery plugin, mootools plugin, prototype plugin, probably as closure library component as well. Can you imagine how many useful development hours have been wasted for this single component to be ported to so many tools/libraries?

Lets continue our in-mind exploration! I finish my component, I write tests for it and and I am happy with it, it goes to production. Then one day, lets say 2 years later I need it in another project. Unfortunately this new project does not use closure tools, because, let's say requirejs optimizer has been rewritten to use a new AST parser that is able to rewrite and minify javascript more effectively and faster than closure compiler and does not impose such strict rules on the code style. And (I will imagine this one) there is great browser abstraction library that is AMD and require js compatible, and maybe even IE is now standards compliant and I don't really need that much abstraction. So I don't really need all the closure library code. I can only hope that someone has already rewritten my code as an AMD module. But because it was closure compiled most probably no one did that ( here is a -1 for closure compiled code, you might think that you protect your code but what you are doing is actually making more work for yourself for later times, just because you did not allow someone else to do it for you), I need to do it manually. How possible is that scenario. Very much possible.

The problem this small example demonstrates is the lack of stability in the defined interfaces. Yesterday everything was jquery. Today people tend to look at other direction, very soon something else might be the big thing. Today people are talking about large code bases and large web applications. AMD and Closure tools are the most talked in those, while I think extJS is also a great example.

Looking at other languages the interfaces remain much longer untouched. In JavaScript code written 2 years ago is now deprecated because the engines change and optimization are found.. code written 10 years ago is actually not running anymore in the new browsers. I have a large code base of windowing system in the browser (similar to extJS) at my hand and honestly I do not know what to do with it...

What should we do?

I don't know.  My best bet right now would be AMD. But as we all know there should be modules in next version of the javascript language. Which means that the code will still be unusable few years from now. Or it will require legacy tooling, which is even worse than rewriting it. Imagine a situation where the quirks are actually in your code instead of in the host environment and you need shims for the code itself to assure its proper working.

I think this was on the Dart team's mind when the new language was born. And yet I am not sure that it is the answer. Maybe time will show.

But in the mean time I think the developers should work toward more compatible and replaceable code. Code where you do not require a module, but instead you require a functionality. Or an interface. Lets say I want to use the query selector interface for the DOM. Most modern browsers support it natively, but some that are still popular and significant percentage of people use do not. In that case the current solution is to use a library, that has selector engine implementation. jQuery has one, mootools also has one, dojo also has its own. Unfortunately not two of those provide the same interface. Mootools separated the selector engine and it can be used stand alone, closure library use dojo's implementation as a namespace, but yet those are not hot swappable. For example I cannot just require the functionality? A robust solution would be such as: the developer requires the functionality and the tool, based on the environment it runs in provides either a complete selector engine (like Slick for example) or a thin wrapper around the native implementation, should the host supports it. This is possible in the current state of affairs, but what about all the differences in the browsers?

This problem causes lots of time being spend in evaluating the endless possible solution to one and same problem. Which I find insane! And that is why I think I will have to move on on another field. Because this one, the web development, has taken too much talking and thinking and not so many action....

One more thing on javascript minification

There seem to be some very troubling misunderstanding that arises when one go on the hunt for compression technology for javascript: most comparison is made on two false assumptions.

1. Closure compiler should be used in simple mode
2. Code size measurement should be done when using either libraries not written in the required for advanced optimization style or using own application code.

Here is why those two are fallacies!

I will start with number two - unused code from libraries and utilities.
In advanced mode the compiler makes something no other compression tool out there is able to do (including uglifyjs, as confirmed by Mihai Bazon this is even not in their scope): the compiler is able to determine the types as if it was run-time and thus strip code paths that were never utilized.

Most code is written to be application agnostic and thus contains lots and lots of methods and functions and variables that are never used in an application. All those contribute to slow execution because they a) need to be parsed and then b) evaluated

Here is an example code

Car = function(color) {
  this.color_ = color;
  this.mileage_ = 0;
};

Car.prototype.drive = function( direction ) {
  this.mileage_ ++;
};

var n = new Car('black');
console.log(n.color_);

UglifyJS currently is completely unable to identify the fact that, should this be considered the whole project code, the drive method and the mileage_ property are never used. Even more, when posted this on the discussion group I гot pointed out that considering the following additional code:

function driveCar(c) { c.drive() }

uglify is not able to determine the type of the argument for the driveCar function, thus, is unable to know that the drive method is used (or unused in case c is not actually instance of Car) and thus it is unable to determine if the drive method should be removed or not.

I was also pointed out that this kind of code modifications are outside the scope of the project. I can understand and accept that. But consider how much one could save from a DOM access library if all code related to ajax is stripped, should ajax is not required, or promise related, or pub/sub related code can be removed from the resulting build should it not be used.I would argue that while this might not seem like a lot, getting built code that contains only the instruction that you actually use is much better than containing all possible instructions in all used pieces of code.

Now for number one: Closure is compared to UglifyJS in simple mode.
Well, of course it is slower, it is written in java! And of course closure cannot compile all your libraries, it is written to compile applications, not libraries for third party use ( however at google they use it for this too!). Using the compiler in advanced mode will most often break your code, this is why there are some code style rules to be followed should the compiler is to be used in that mode. The gains are significant, but the friction is so much, most developers just do not bother. For one, you cannot just stop any library in your project path and expect it to work. This is a big show stopper for most developers. Then you also need to write your code in google's funny way of name spaces. You are forced to use only a few subset of JS patterns ( no matter how memory and efficiency optimized those are - limits are limits).

So next time you evaluate the two options, please make note of the difference between minification and 'compilation'. While it is not a real compilation, the code changes made are much greater than method and property renaming.

JavaScript optimizations

I have been working the last year and a half on large in-browser applications and the last few months I have been looking into automated method of code optimization. The two most significant options out there currently are Closure compiler and Uglify-js.

I have used both and the following text is about my experience.

I will start with the few per-conditions: The code is modularized, i.e. different components are written as modules in separate files already and not all components should be loaded for the application to run. The application should be able to load additional dependencies/components as reaction to user actions or other events. The code should be minified  if possible and concatenated into small number of files, the lower the number of requests - the better.

Starting with code base of 1.8MB the task at hand was to squeeze it to something that can be loaded via the Internet on a low performance STB device. The application is the main device interface, so it should indicate activity to the UI asap.

The code was initially developer with requirejs so making a build of it with node requirejs module was not very difficult. The code needed some tweaking but in one day work the whole project compiled to 279KB. The resulting javascript file is only one (i.e. monolithic build) and once downloaded was able to start the UI in 2.79 seconds (as opposite to 9 seconds when un-compiled, due to more file loading mostly). This however, combined with the network lag of downloading 280KB  resulted in a black screen for almost 6 seconds and was not fast enough for a home entertainment equipment.  What we needed was 'load indication' module, that loads first and then indicate the load progress for the next few seconds.

Next we decided to try and use closure compiler. The compiler however forces the developer to use limited set of javascript patterns and it took a few days to stip the incompatible code or replace it with one that is. Some modules were not ported, for example the settings module. We used the closure-script to ease the development cycle. The result was impressive. The compiler is able to inline lot of the code ( for example property getters/setters are inlined, which lowers the scope creation load) and the tool we used is able to include the needed files per modules and modules are defined with namespaces. One caveat is that all modules need additional code for initialization, which we needed to write in addition to the logic code we already had. Another caveat (or one can think of it as advantage) is the renaming of method names, which means that you get much smaller size of the final built but you loose the names. The structure of your code might change beyond recognition, but the payoff often worth it. However the additional code that need to be written and the required changes should your code base is written without the compiler in mind can make the process long and tedious.

Finally we decided to use the requirejs whole project build capabilities. Once again it required some getting used to it and a bit testing, but we managed to separate the monolithic build to 4 modules. While testing the closure compiler we really liked the defines ( i.e. variables that can be defined at compile time and allow the compiler to strip code that is branched out based on the value of the defines) and we wanted to used it in the project. It was a nice surprise for us the fact that uglify-js supports defines as well. Both uglify and closure compiler require special syntax for this to work but the overhead is very small compared to the benefits.  We use it now to strip the debug modules and debug statements from the build. As requirejs works in node we were able to produce a small development server that can run the build process based on the query string submitted with the index request in less than 15 minutes. This frees us from the java virtual machine (as opposed to the closure tools). Uglify also makes its magic faster compared to closure compiler. Uglify however does not do function inlining for methods (getters/setters) nor does overwrite method names ( as this is not safe transformation in the context of non-restrictive javascript style ), it also does not make namespace squashing, which means that you need to make some extra effort to make sure your code does not use excessive scope creation/nesting.

The final thought on this project are as follow:

1. You need to first decide which compiler you would you as both require different code style to work properly. 
2. You need to go an extra step to make sure the code is structured suitably for modularization in both compilers.
3. Do not forget that it is simply a tool, there are more than one ways to get to the destination, do not get obsessed!

As a last point I would suggest closure tools for projects that want to work across all browsers and have the latest abstractions (for example they already have abstraction for the transitionend event as it differs in firefox and chrome) and you do not want to collect the abstractions yourself. Closure library can save you lot of time for this.

On the other hand if you are targeting only newer browsers you can go with the faster and easier to use uglify-js/requirejs.

Tests running in both browser and nodejs

Following my post about developing on cloud9 and running your tests in both node.js and browser, this post will show how I had to make some sacrifices to allow proper tests constructions and support for assert testing that works on both node and browser.

First - the screenshots!

Tests run in browsers

Same tests run in node

The 'test runner' (debug name designated to 'AllTest' in the screenshot) is exactly the same file (see the post mention in the beginning of this one), the only difference is the run script: we need to use run-in-node.js and run-in-browser.js respectively, which is pretty easy to maintain.

I spend a good part of the Sunday trying to figure out a way to use a test framework that works exactly the same in both the browser and node.js and on top of that can work just fine with requirejs (which I use on the server and on the browser with amdefine and rjs on the server). I have tried several but I either couldn't make it work with requireJS or run the same tests in both environments. The closest thing to what I imagined a test suite would be was Vows. Unfortunately it runs only in node and the code depends on process, file and other objects not found in the list of host objects in the browser.

Never the less, the way one structure the tests in Vows was what I was looking for. The only thing I could do in this case was 'take it and make one for yourself'. Finding assert library that works in both node and browser was easy. Next I created a simplified version of Vows. It is _NOT_ API compatible and it lacks large portion of the features offered in Vows (If you are developing for node.js I suggest using Vows - really!), like nested topics for example.

However for my needs I was satisfied: it supported sync and async topics (thanks Vows for the idea), the topic can be a function or any object, the tests should return its result as boolean.

A simple asynchronous test looks like this:

    vow.addTests({
        topic: function() {
          var callback = this.callback;
            next(function () {
                callback(null);
            });
        },
        'test async assert': function( shoudBeNull ) {
            return assert.isNull( shoudBeNull );
        },
        'test async assert failing': function( willBeNull ) {
            return assert.isTrue( willBeNull );
        }
    });

All topics that are functions are run and the tests are invoked asynchronously. I have also implemented this same thing in a sync manner and it was much slower, so maybe the people that dislike the 'callback spaghetti node code' are not right after all?

Example for sync test ( i.e. tests invoked directly with the topic result ):

vow.addTests({

    topic: (function() {return null;})(),

    'sync test': function(shouldBeNull) {

        return assert.isNull( shouldBeNull );

    }

)};

If you are looking for simple test running suite that works in both browser and nodejs, take a look at it. I am sure there are many improvements to be desired, but I really needed it for my use case mostly and it covers it.

The code is part of the tablet branch of the 'Tornado UI' project and can be found on GitHub.

Happy hacking.

ACTA и българите

В България всичко е наопъки. Накъдето и да погледнеш, никога нищо не си е на мястото и никога нищо не е наред. В американските филми се питат "всичко наред ли е", в нашите условия се питаме "има ли нещо, което е наред".

Та така и с протестите срещу ACTA. Някакви хора разпространяват от трибуната на що годе популярния си блог (а вероятно и още много, просто спрях да чета БГ блогове, защото ми писна да чета глупост след глупост, този го оставих, защото се списва от чужбина) идеята да се противопоставим на АКТА чрез постване на снимка в фейсбука си.

Сега, аз вярно че фейсбук нямам, като бях малък нямах измислен приятел, сега не са ми нужни 700 такива. Но не това е важното. Важното е, че все още не става ясно как твоя снимка в твоя профил как даряваш кръв фактически (за разлика от имагинерно) влияе на ратифицирането (или не) на АКТА. Мисля, че е ясно, че управниците ни не четат ТВОЯ фейсбук.

Един от "интелигентните" коментари е( перефразирам ): "Какво не е ясно - показваш, че да споделяш не е лошо."

Така е. Сигурно въпросният господин споделя и жена си с нуждаещите се. Вероятно е така, щом толкова е 'за' споделянето. Но дори и той лично да споделя половинката си с който там има нужда или пък чувства близък и достоен за споделяне, не става ясно как това влияе на властите ни, които в крайна сметка взимат решенията вместо нас. 'Щото прочем и да споделя жена си, в законовата нормативна база това си е прелюбодеяние. Нищо, че за него лично е хубаво нещо споделянето. Но да оставим този екземплят.

Насочваме се към "организираните" протести.

Предвид климатичните условия в нашата бедна република (дневна темепратура -10 и надолу), по мои груби убеждения на уречените места ще се явят има няма 10% от заявилите желанието си. Останалите ще протестират от комфорта на отопления си дом или офис с очи широко вперени в монитора, белким се почувстват на мястото на събитието зачитайки някой и друг "туит". Останалите ще се почувстваме на мястото на събитието само ако някоя национална телевизия няма коя бедстваща община да ни покаже та заеме преден пост сред протестиращите/премръзващите.

В крайна сметка и протестите не виждам какво могат да направят специфично в нашата държава. Даже това, което се чудя на нашата държава е как има толкова много 'задник'. Как е възможно някой да се е надупил на толкова много страни едновременно. На мутрите, на Русия, на САЩ, на ЕС, на не знам си кой още.

На протестиращи ще кажа: вместо да си губите времето да протестирате в БГ, особено пък извън столицата, вземете се вдигнете да идете до някоя европейска държава, в която задника на държавния апарат не е изцяло насочен към бизнес интересите на САЩ, ами поне частично, ей така, с един градус обсег, и към поданиците и. Там протестирайте, белким там не мине ратификацията, а пък те нашите по напъна отзад ще усетят дали трябва или не трябва и какво трябва да направят.

Това е.

Developing frontend with node js and cloud9

Not so long ago I was describing how disguised I am with the whole nodejs and cloud9 story, how it was working bad, how it does not support 'MY' use case and so on.

Well as with most things I tend to re-evaluate my position from time to time, just to be sure and sometimes to check if things are progressing.

The last few days I was starting an spin-off of my Tornado UI project ( an Open Source interface for Tornado IPTV devices, read more ). The new code is loosely based on the existing one, re-using some of the components, but still is a chance for fresh start and evaluation of new options in code organization and structuring.

Part of the code for the previous project was ported to closure-library to allow it to cleanly compile with closure compiler. I can gladly tell that it was a success, all tests of the converted code passed and it is available online for anyone to use it in a closure based project. It covers completely the communication layer for Tornado, but does not provide any UI. It assumes the closure developer would use closure techniques for the UI as well and I doubt my ability to fully utilize closure paradigm for UI components.

The effort to use closure library was an enlightening in a way, allowing me to free myself from Crockford's mantra for privacy in the code. Meanwhile I have read tons of posts and articles about patterns, speed, reuse as well as package management, what Harmony will bring, what should we do until then and so on and so on, basically re-educating myself on the matter of serious JavaScript.

One big problem for me was the fact that I am not actually using any of the power of the 'abstraction' the libraries provide (i.e. Jquery, MooTools, closure and so on), when developing for a known environment one does not need abstractions and polyfils (well, mostly does not, if you have seen the code for TUI, you'd notice that bind is not part of the Function prototype). It just bloats the code and rarely enforces useful patterns ( unlike closure, mootools and jquery allow you to code in any style possible, even with no style).

Another problem was that I wanted to be able to test the code on a test server or at least in a console with cli, instead of loading it in browser every time. At the time I started the project there was no clean way to run the same code without any change on node and in the browser. Or at least I could not find it.

This week I started the new project, it aims to bring new features to the Tornado line of device, at least two of which will be co-developed with the in-house development team at the company. It will also be able to work on any android device (2.2+). This time I was determined to use some CLI testing for the code to speed up development and bug catching.

I went looking for a suitable tool again. First I was thinking about closure library and found nclosure. Unfortunately it seem to be not updated lately and does not work with node 0.6+. Then I tried JSClass, it require shift in package management, but still is manageable. Unfortunately the dependency tracking and loading is not working as I expected, maybe it is me, that cannot figure it out, (even though I figured out closure and requirejs), but after two attempts I gave it up.

The last was amdefine. It requires node 0.6+ but apart from that works very well and suits me perfectly. Very good explanation on how to use it could be found on their github page, linked above.

Lastly I wanted to try cloud9 again. I downloaded the latest source from github and it work very nicely with node 0.4. Unpleasantly for me it does not work with node 0.6, while amdefine requires it, which makes it impossible to use nodejs as test run and cloud9 locally. The only possibility remained hosted version of cloud9. Recently they announced support for running your code on node 0.4 or node 0.6, configurable from your IDE. Month ago they also featured npm command line interface, so I figured I should try it.

A working setup follows:

On c9.io open your project and add requirejs and amdefine:

npm install requirejs

npm install amdefine

In your project, configure the test runners, one for node and one for browser. Node:

var requirejs = require('requirejs');

requirejs.config({
    baseUrl: __dirname + '/javascript',
    nodeRequire: require
});

//Use eyes-console in node js for prettier output

requirejs(['tests', 'debug/eyes-console'], function(tests, SimpleConsole) {

//    Enable logging in node
    (new SimpleConsole()).enable();
    tests.run();
});

Browser:

require.config({
    baseUrl: "tablet/javascript"
});
require([
    'tests',
    'debug/simple-console'
], function(tests, SimpleConsole) {
    (new SimpleConsole()).enable();
    tests.run();
});

Note: the past are relative and fit my project, those may vary for yours. 

The browser runner requires html file as well, however the node test runner can be run as easily as starting a node process with the file. Remember to set the run process to use node 0.6.

Conclusion: Cloud9 ide is a viable option for developing and testing your code, even if you are developing only a front-end application. However not all approaches for code organization work, currently I have managed to make only AMD with requireJS. I expect the future to bring more variety in the options.

As a side note, Eclipse Orion has much better git UI (as a real UI, not only cli), however it is still in a very early development and on top it does not support running node instance, so you cannot really test your JS code in it.

TUI status update

The Tornado UI project is now more or less in stable phase, there are some small issues here and there, but the transport layer is stable and the HTML UI is complete. Also the closure-library link for the transport layer is ready.

As of today the develop branch from the last 2 months is merged in the master, which means that it is useful as it is, no need for special setup. You can just host it anywhere and point your Tornado device to the start URL (edit the M55/M70 settings and set 'Start URL' and 'Use start URL' to true).

The one piece still missing is the picture browser, we are working to resolve the memory restriction making web-kit to crash after a while with large files. Once this is ready you will have a complete interface, fully customizable via CSS only and very easy to extend with JavaScript.

The code is available on GitHub.

The documentation is progressing well, for now it covers only the transport layer and its abstraction layers, but the plan is for it to include the UI elements as well for easier use of the source as SDK for your own AV application or even your games.

The binding for closure library will be released soon, we are still clearing some compiler warnings in strict mode with type checking. Stay tuned. Using the closure compiler brings some new features, like compilation for a particular device, cutting out large portions of the code base and making the application faster. We are also working on some nifty features like app cache and local storage for even faster loading as well as some advanced visualization features, that will run on the newer devices and will bring the cutting edge presentation layer one expects from a modern web application.

Списание Дъга

Ако сте над 25 годишни и сте от мъжки пол (пък даже и от женски да сте..) сигурно си спомняте списание "Дъга". На мен ми беше любимото. Нещо като социалистическа версия на "комиксите". Интересното при него беше, че освен разказите съдържаше и игри и статии. Също друго, много интересно нещо бе, че различните истории се рисуваха в различен стил, което развива артистичните усещание у младежта. Нещо такова.

Та така де, всички броеве можете да разгледате или свалите от тук: списание Дъга

What is better than Glee?

Short answer: Mary Lou by Eytan Fox. 

Why? Because it treats real life and real problems and not high school imaginary drama with bunch of adults in their mids pretending to be high-school students. Yup. 

And the music is better. I like Svika Pick! And you like it too, if you like Harel Skaat. 

Attention (Anti Semites): the movie is from Israel and is in Hebrew
Attention (Anti Gays): the movie contains some homosexual imagery and transvestite imagery - nothing profane though.