Decoupling components for frontend development.

Introduction: It appears the era of monolithic UI libraries is coming to an end.

It has been irritating for years: you go deep into a project for months just to realize that you need a certain type of UI component that is not available for your set of libraries but is readily available for another (for example it is written for Mootools, but you use jQuery or it is in Dojo but you are actually using jQuery).

So before you even start you have to have in mind all the components you might need. Then o and hunt for a library that has them all or at least most of them. You then have to spend time and learn their component idioms (i.e. how a new component is created or composed from existing ones). This is one of the hardest things to grasp in a new library in addition to its build model.

Build model is another opinion in a library that you can't (or can but hardly) change. Some libraries (smaller ones) do not provide a build model and you are on your own, however if a build model is selected you have to stick to it. If the library uses AMD you have to stick to AMD. If the library uses CommonJS you ave to stick to browserify or a similar one. The primary goal being dependency management, it often includes concatenation and minification steps.

The problem often overseen is that you can achieve greater responsiveness if you separate initialization code from the rest of your application. RequireJS supports building your application with build units allowing such set ups. Closure compiler also allows you to separate your code into loadable modules. However with browserify you are on your own. You have to manage the dependencies and the build step if you elect to use one.

A new concept recently introduced by TJHolowwaychuck is components: The idea is that you could develop a reusable component and put it in its own repository on github or on a private git service and then reuse it from within another component or in an application. The components could contain css, JavaScript and HTML.

I find the idea great, however the execution and the design are not so great. The following problems stick out pretty fast:

• while component management is made easy with this project, you are required to use only one final file in your application, so modularization and modular loading is not possible yet.
• while the builder used in the project provides a hooking mechanism it is unusable unless you wither alter the builder itself locally to attach your own hooks (i.e. ones that are not provided by default) or you have to fork the component npm package to make it require augmented builder package where you have extended the base builder. This makes the project really hard to augment in transparent manner and locks you into constant chasing with the original. 

On the other hand the naming convention used and the plumbing added to make ti work makes the project a real pearl in the dust: no more strange and ugly names (as in the author's example you could use 'tip' for your project name, no more tippie, tipsy, tips, tiping etc) thanks to the repo/project naming convention. The names are disambiguated automatically and this works great.

The even greater thing about this is that dependencies are managed automatically for you and transparently and that you can concentrate on a smaller, testable units of code. This IMO encourages decoupling (similar to AMD) but removes the extra complexity of AMD (the paths for example could be a real hell if you try to mix several repositories). Fattening out the directory tree was a great win for this project. 

Conclusion: One thing we should get used to in the coming months and years is that monolithic application model (an application that consists of only one pre-build file) is going away. Shadow DOM, Web components, JavaScript modules etc are making steps forward and the sooner we understand the speed and memory implications of those the better we can implement them in your projects. Until now - use decoupled code as much as possible.

Tree shaking and should you care about it.

Introduction: Tree shaking is the process in which unreached parts of the code in your application are removed. 

Description: So now we have at least 4 JavaScript compilers / minifiers out there and all of them are pretty much feature complete and stable. They support source maps (very important now days) and they do a decent job as squeezing those large (sometimes over a megabyte) chunks of JavaScript code into something that you can actually serve to your users without making them wait long enough and thus loosing them.

While in some parts of Europe having 80MB optical down link in your home for 10 euro/month is pretty common, there are places on this Earth that still consider even a single megabyte per second to be a really fast, pretty much inaccessible luxury. So if your content takes under a second to load on an 80MB link, it might take a whole minute in another location. Not that most content is the JavaScript, often pictures take a larger portion of the traffic (and oh my god, stop with those ultra large "retina ready" images!). However the perceived performance of your application is mostly affected by the JavaScript and it being minified is only one side of the coin.

Of course you should try and write optimized code. Of course you should avoid unneeded repaints.  Of course you should throttle repeating actions. This is mostly that you should look for as a developer working on your code. What you should not look for is how many properties are you attaching to your function's prototype. Or how deep is your inheritance chain. Or how to avoid name collision.

Now. lets talk about tree shaking. This is how developers call the action of stripping our code that is never used in the application, should the tree shaking procedure is presented with the whole application code. So basically you give all of your code and it returns the same code, but with the unneeded parts removed. If you are mostly writing application code, you might argue that all your code is used and that is the truth most often. However if you are relying on a library, do you use all of it? If you inherits from an UI class and you want to use only a small subset of methods then tree shaking is what you want. It reduces file size and it reduces parse time.

Now, there are two instruments out there that can make tree shaking: closure compiler and uglifyjs.

I will start with uglify JS: if you have code as this:

if (false) {
// do somthing
}

 then uglifyjs is smart enought to remove it.  However uglify is designed to minify your code, stripping unneeded spaces and reduce the length of the variable names. It is not designed to make application level tree shaking.

The other available tool is closure compiler with its advanced mode: it is able to determine all the code paths that are not executed, assuming you input all of your code. However the compiler assumes a subset of JavaScript that most developers find to be too restrictive. So basically if you use a library that is not conformant with the closure style you cannot use it in a build, which means that you have to either use extern files or use bracket notation (which is more tedious) in your code.

Here is a small truth: closure compiler is perpetually used in large application like gmail, because gmail introduces an average of 10% code change in a month. This is a product google is still actively developing and augmenting and in their case using app cache of other techniques to allow the browser to not make those request is not possible. If your applications can get away with the same version of a file just go with it.

Conclusion: The whole idea of 'building world' is designed and introduced for projects that match Google's workflow (iterative augmentation of a product on a regular basis). It is not necessarily the best thing for your project. On its own tree shaking is a good thing, but the code restrictions are too great to ignore. If possible you should investigate other modern solutions for speeding up code loading.

Browser wars?

It is old news but still: Chrome is taking up share from Firefox.

I, being Linux user for at least 10 years, think that the Chrome 'speed' and 'performance' hype is way overestimated.

Maybe because I use Linux, or maybe because I do not like making hazardous trash too much, change my laptop only every 6 years: every 6 years I spend tons of money on a top of its class laptop. It is usually 'business' class (I have no idea what this means except the extended warranty, but anyways), which basically leaves me with a machine that is not that performance capable, but more durable and with a little more squeezed battery life.

Now about Firefox/Chrome. First of all I notice the following with Chrome:

1. much slower from first start to being capable of showing anything (have not measure it, but at least 3-4 seconds and 1 more to colorize the applications (meaning really ready, not just showing the main window)). 
2. much more CPU intensive when loading content: it does not really bother me how much CPU it uses, but if I scroll on a long page and click on a link with the middle button to open in a background tab the scrolling experience greatly degrades until the new tab is loaded. Maybe this is why Chrome does not allow setting to always open links in background when opening a new tab? Besides it is supposed to use all cores of my CPU, so how could Firefox do it with only one process and chrome cannot do it with all?
3. tab switch is much more prone to "redraws": after I have been on a tab and witch to another and go back to the first one I see re-flows happening. This is not really a problem, but just wondering, eating like twice the memory and CPU compared to Firefox why not just do the re-flows in the background whenever those are needed? And no, the pages are not using the visibility API:)
4. Sync does work, but in a very strange manner: Lets say I have to use a PC at work and I sync with my google account. It takes hours to have everything synced, some applications never sync (so basically now I have different set of plugins (for example the google chat in tabbar for chrome) and different set of applications (for example google play) on three different computers. And no, those will never sync. Theme syncs after hours too, sometimes days. I have had a case where my theme was different on 3 different computers for 3 days. Eventually it syncs, but ... I also has cases where the sync gives up at some point, it gives no error or any kind but you can clearly see that new bookmarks never arrive to your other instances. While this is not directly chrome's fault, being able to sync everything is considered part of the browser now days.
5. I cannot only tell about the wrongs: Chrome has the best developer tools ever! Not only they are feature rich (much much more than those of Firefox) but also they work much much faster than anything else available. It would be foolish not to use them instead of other tools.

 So basically from browsing / surfing point of view I find Firefox to be much more friendly than Chrome, not only RAM and CPU wise, but also perception wise: Firefox shows up and is ready to be used much faster, also is able to retain much more tabs without issues and also is able to store those tabs for long time (I often use the ability to store tabs for viewing later in another tab group (using panorama) and only check them out when I have the time - the browser performance is not affected by this at all, not is the start up time, try the same with chrome...). On the other hand I develop large web applications and I would not be as effective without top class developer tools, so I have to use chrome.

I think it is worth mentioning that on mac OS X the picture was slightly different: I have briefly had access to a brand new AirBook. I have initially installed Firefox, but the performance was really bad. I have also installed Chrome and the performance was much much better: the start time was again a bit longer, but once it is running it was much smoother and faster than Firefox. Also Chrome was using features of the OS while Firefox ignored them (like back/forward with swipe) or full screen the OSX way. However I am not such an Apple fan and sold the laptop after a month of trying to get used to it. The point is the performance is much different on another OS, so it is possible that Chrome is better on Windows as well, but on Linux it is much better really.

On one side Google could at least try to make Chrome as good on Linux as on the other platforms (but probably will not, because Linux has such a small share on the desktop). Firefox on the other hand could try to pick up those OS specific features a little bit, but mostly they should come up with an useful debugging tools. I can tell you, working with the current tools really feels like hell, I cannot do anything with those, when I want to check out how a feature is working in a page I encounter, I have to open chrome and load the page in it.

I wonder if anyone else has similar experience with the browsers on Linux? I also wonder how is Chrome comparing to Firefox on Windows machines. Not that I am going to use one any time soon, but still.

Butter smooth timeline for your video/audio project (canvas drawing)

Conceived as a pet project for editing TV schedule for small IPTV providers, the project proved to be an interesting challenge.

For starter we looked at some other implementations and liked this one: it comes with lots of examples and proved to be interesting enough to work with. However we quickly realized that the code does not scale well at all. All the examples feature very short videos and we needed a visual time representation for at least 24 hours period. Basically the code would have drawn each second on the canvas no matter how long the period is and thus was working progressively slower with larger periods of time being used. It also binds directly to a slider / scroller and thus was updating on value change, instead of when the browser has a drawing frame schedule.

Never the less the code was a good example of a timeline view executed with canvas and we decided to use its concepts.

The first thing to do was to make it closure compiler friendly, this was easy. We also stripped the binding with the scales / scrollers and instead exposed methods to set the values on the class regardless of the means used for the user interface.

Then we started looking for optimization opportunities. We notices that if we draw all the seconds in the period on the canvas we quickly turn the time lines (scale separators) into a blank line - this is because all the drawing is always performed: each second's position is determined and drawn on the canvas. As you can imagine for 24 hours period those draws were often from thousands to tens of thousands. We decided it would be more wise to separate the whole period into dividable frames and use those as steps when jumping from one time position to the next instead of iterating on every second of the period.

Our first solution was fine, but iterating over the same amount of time using different steps has two side effects: number one is related to how canvas use the dimensions internally. Basically for best results you would want to set the canvas size to the actual size of the view. Also when drawing single pixel wide lines you would want to shift the line to a half a pixel (i.e if you want to draw on pixel one you need to perform a stroke at position 0.5 and 1 pixel wide). If you draw on anything else anti-aliasing kicks in and what you see is not as crisp as you might hoped for. Because we implemented this half pixel shifting as the first thing in our code the defect introduced with the multiple walk trough the scale was not visible: basically we were drawing on the same pixel multiple times. Still - the code was much faster now, because at this point we did not draw on every second but instead on every time the next time point would stand at least few pixels away from the previous. Considering the capabilities of the modern monitors this would mean one would draw not more than few thousand lines, more often than never less than one thousand, regardless of the time frame to be visualized. The second problem was to manage the time stamps on the top - managing meaningful time stamps required much more code.

Using the shifting to a whole pixel drawing technique, we also need to consider this shifting when drawing the separate program items and that proved to be not so trivial, because while using steps in drawing the scale, using those when drawing precise time is not perfect. Instead of putting too much effort in aligning the pixels, we instead decided to investigate another alternative: let the canvas use fractions when drawing but keep the reduced drawing calls by using time steps larger than 1.

This however allowed us to see the defect introduced on the previous step: going on top of a fraction of pixels does make a difference in the visualization  and because we draw the different steps sizes in different length the resulting image was very poor. Basically the drawing is performed on the same fraction (i.e. 0.232536) but visually you get a wider line because you have drawn twice on it.

At this point we were not happy with neither pixel alignment results (too difficult to shift everything to a full number, too much calculations) nor step cycling results (poor picture quality).

During the weekend I had the idea of removing the cycles of drawing with different step and instead use only one step but instead calculate the step offset to the predefined steps. If modulus division matches 0 then we are on a certain step (i.e. full hour if we divide to 3600). Also one thing to notice was that we do not actually need to calculate the X position to draw on every step: because the step is determined only once per redraw we can also determine the difference between two adjacent steps use it increment the X position without actually calculating X for a certain second in time. As a final optimization I suggested also to calculate the first visible second and align it to the step (so we skip cycling over x positions that are out of view) So basically you calculate the X for the 0 second in the period, then for the first step value and using the offset to 0 (the start of the visible area) you have the initial visible second and from it you calculate the first draw-able X value. From then on you only increment the second to draw by the time step and the x value for it by the difference between two values.

Now the code behaved perfectly, even on Atom powered laptops with Chrome browser. However in Firefox things were not that brilliant. There was noticeable lag on updates on desktop Core processor, let alone a netbook.

Next optimization we had to perform was to decouple the drawing of the canvas from the scale and scroll updates. This is easily done and well documented technique (just look up "using request animation frame" in google). Basically every time we have value updates we set a dirty flag and call the RAF. It calls itself until the dirty flag is turned off.

Now that was a butter smooth canvas visualization.

What was accomplished: unlimited time frame to visualize, unlimited items in the schedule, decoupled value updates (i.e. you can set them on batches, it will not hurt performance), smooth animation and great performance on all modern browsers.

What we learned: drawing on the canvas is expensive, draw as little as possible, calculations with floating number are also expensive, reduce those to the possible minimum, use RAF to limit the drawing operation to the browser optimized frame rate.

A demo application will be available soon, as well as the code related to the time line will be on GitHub once the project is complete.

Non-sense all around the place...

Following the last post I would like to express what or who was the motivation for it and how badly one can be misled by authority.

At this address you can find a long post trying to argue that the closure compiler in advanced mode is a bad idea. Posts from the same author has been trowing crap at the fan for some time now, but this last post was really the tipping point for me. Basically the behavior exhibited is all over the Internet now since the JavaScript language  became so important and widely spread.

First rule: attack something that does not match your project criteria or something you have never used or something that you even do not understand. The author states that he has never used the compiler in advanced mode and judging by his examples of how bad the advanced mode is has also never tried to understand the concepts in the compiler.

Second rule: make a full of yourself by talking about optimizations you have actually never applied. The author talks about you should manually remove pieces of your code that are not reached in the application - this would be probably the most insane thing to do, but lets assume we want to do it. Lets say we have a calendar widget that can do really anything calendar related. And then we only use it as static calendar in our app. All the methods and properties attached to the constructor are never reached - like 80% of the code is unreachable. Lets say we have a large application that uses at least 20 such widgets. Lets say the code accumulates to 2+MB. Now imagine you want to make this application accessible to people with slow network connection and old software, like people still living with XP of worse (windows 2000 for example). Waaaay to go removing the code that is not reached. Really!

Third rule: attach importance to things that are really not possible with the technology you are attacking but make it sound like it is essential to use/have them. ES6? ES5? Polluting the global scope? Just wrap the compiled code in immediately invoked function and move on! It is as simple as specifying the wrap text at the command line. Object.defineProperty - combined with the wrapper I do not see how one could mess with your code, really, plus the access is controlled by the directives. So basically those are overlapping. Which one is used depends on the developer's preferences. Different syntax? Not really, one can either use externs or stop assigning meaning to the names.

Forth rule: Just go plainly mad and barf all over! "Not compatible with requirejs"??? Really? That is a thing now? So basically we are supposed to believe that AMD is now the only way to scale an application? I have seen only one (ONE) large application that uses AMD - cloud9 IDE. That's it. Everything else I have encountered was a regular size application with some AMD inside, mostly just to be on the cool side. The compiler also has modules by the way. But the author wouldn't care because those are not AMD compatible.

Bonus rule: Talk about security. Just to make sure you will be taken seriously by the masses starving for JavaScript enlightenment, do not omit the opportunity to talk about security - whatever it means in your context.  Basically the thing with deleting your code is really applicable to anything - really! Wrapping it in a closure would solve this one as well, but of course the author is looking for problems, not for solutions. How your code is treated (and removed) by the compiler is well documented and many posts have been written to explain that advanced optimization is not for everyone. But of course if security has been compromised then yes - this compiler mode must be a crap. Evidently this is why google's web products fail pretty much all the time - because of the advanced mode and if they just switch to AMD and simple mode it would be much better. Hopefully someone from Google is reading the author's blog and will take a note and start action on the issue....

When it comes to awful JavaScript

The worse thing about JavaScript is not eval, with or this. The worse thing is that people take a higher ranking development position in a top shop company and pretend they rule the universe and they know it all. But wait, there is even worse - everybody else in the JavaScript universe almost religiously start to believe every word those people say.

The end point? Be careful who you trust, be careful what a benchmark is actually tracking, avoid micro optimization especially integrating it just because 'know it all' says so. Basically - be skeptical and peer review:)