The JavaScript ecosystem problem

I tend to follow the news in the JavaScript field because it is related to my work. What I notice lately is that every developer out there is making attempts to popularize his/her favorite framework and/or tool.

Here is my objection to all this: Currently the situation is this - the ecosystem of javascript tools and libraries is fragmented. The pieces are so small and so incompatible that it makes it impossible to build a large web application without actually spend months in development time trying to make the pieces work with each other.

There is jQuery, which unfortunately is now much more than a tool to hide the browser quirks. It now provides pub/sub, promises and what not. There is also a whole lot of 'plug-ins' for jQuery. But those require different version of jQuery. So you might need more than one jQuery.

Then there is 'modular' code, AMD. One being the new dojo (1.7), which is a pretty big modular library, but it is not really compatible with other AD loaders. There is RequireJS, which is also popular because of its 'optimizer', but it is only that - loader, and does not provide any library for browser quirks hiding, so the developer is left with no library at all. jQuery is popular choice in this case.

There is closure library and closure compiler, however those are completely unusable with code that was not specially designed to work with them.

There is prototype, there is backbone there is underscore... every single one of those has its fan base and none of theme follows a code pattern that will allow a developer to just use the needed functionality without any additional hassle.

This of course leads to the decision enforcement: the developer has to make decision which technology to use (some 'ninjas' even go as far as creating their own set of tools/mini libraries and use those, but this is time consuming and error prone and the library takes additional work to be supported and renewed). It does not matter what technology is selected, every single one that is popular has its benefits. What is more worrisome is the following: How much tie will it take for the used tool/library to become incompatible/obsolete?

Lets say that currently I find the closure tools to bring the best combination of browser quirks hiding, performance and code optimization quality. As every single tool out there it imposes a set of rules I need to follow in order to make my code work with it. Notice that __ALL__ tools do this. For example jQuery also requires the whole library to be loaded before I load the  plugin. It also requires a global variable to be exposed ($ or jquery). Most libraries count on a global variable to be available with a certain name. Closure makes exception in this, but imposes much more rules. So let's say that I create a super useful UI component, that is based on closure library. If you want to use it you have two choices: a) use closure library/tools or b) rewrite the functionality with your fave library. The last one is the more popular choice. One and same widget/component is currently being re-written over and over again with little augmentation to the actual functionality. LightWindow anyone? It is available as stand alone, jquery plugin, mootools plugin, prototype plugin, probably as closure library component as well. Can you imagine how many useful development hours have been wasted for this single component to be ported to so many tools/libraries?

Lets continue our in-mind exploration! I finish my component, I write tests for it and and I am happy with it, it goes to production. Then one day, lets say 2 years later I need it in another project. Unfortunately this new project does not use closure tools, because, let's say requirejs optimizer has been rewritten to use a new AST parser that is able to rewrite and minify javascript more effectively and faster than closure compiler and does not impose such strict rules on the code style. And (I will imagine this one) there is great browser abstraction library that is AMD and require js compatible, and maybe even IE is now standards compliant and I don't really need that much abstraction. So I don't really need all the closure library code. I can only hope that someone has already rewritten my code as an AMD module. But because it was closure compiled most probably no one did that ( here is a -1 for closure compiled code, you might think that you protect your code but what you are doing is actually making more work for yourself for later times, just because you did not allow someone else to do it for you), I need to do it manually. How possible is that scenario. Very much possible.

The problem this small example demonstrates is the lack of stability in the defined interfaces. Yesterday everything was jquery. Today people tend to look at other direction, very soon something else might be the big thing. Today people are talking about large code bases and large web applications. AMD and Closure tools are the most talked in those, while I think extJS is also a great example.

Looking at other languages the interfaces remain much longer untouched. In JavaScript code written 2 years ago is now deprecated because the engines change and optimization are found.. code written 10 years ago is actually not running anymore in the new browsers. I have a large code base of windowing system in the browser (similar to extJS) at my hand and honestly I do not know what to do with it...

What should we do?

I don't know.  My best bet right now would be AMD. But as we all know there should be modules in next version of the javascript language. Which means that the code will still be unusable few years from now. Or it will require legacy tooling, which is even worse than rewriting it. Imagine a situation where the quirks are actually in your code instead of in the host environment and you need shims for the code itself to assure its proper working.

I think this was on the Dart team's mind when the new language was born. And yet I am not sure that it is the answer. Maybe time will show.

But in the mean time I think the developers should work toward more compatible and replaceable code. Code where you do not require a module, but instead you require a functionality. Or an interface. Lets say I want to use the query selector interface for the DOM. Most modern browsers support it natively, but some that are still popular and significant percentage of people use do not. In that case the current solution is to use a library, that has selector engine implementation. jQuery has one, mootools also has one, dojo also has its own. Unfortunately not two of those provide the same interface. Mootools separated the selector engine and it can be used stand alone, closure library use dojo's implementation as a namespace, but yet those are not hot swappable. For example I cannot just require the functionality? A robust solution would be such as: the developer requires the functionality and the tool, based on the environment it runs in provides either a complete selector engine (like Slick for example) or a thin wrapper around the native implementation, should the host supports it. This is possible in the current state of affairs, but what about all the differences in the browsers?

This problem causes lots of time being spend in evaluating the endless possible solution to one and same problem. Which I find insane! And that is why I think I will have to move on on another field. Because this one, the web development, has taken too much talking and thinking and not so many action....