JS Compilers

Decided to see how much arithmetic operations can be optimized out by the FE accessible compilers (GCC, Dart).

Assuming we have a list of list of numbers that for a reason unimportant to the presented case we want to represent as arithmetic operations:

var list = [[0, 360/2], [0, 360/3, 360/3*2],... [0,... 180+40+(30*2)]];

On top of that we want to do some more calculations on each of these numbers, lets assume angles and rotate -90 degree, but keep all values positive.

var rotate = (num) => { num < 90 ? num + 270  : num - 90 };
var process = list => list.map(_ => _.map(_ => rotate(_));

Lets also say that we have some code that later on randomly can access any of the values, thus any compiler should:

1. retain all values
2. possibly calculate them on compile time*

* because either the code never changes any of the values (i.e.only console.logs them) or because we annotated the code for all items to be immutable and because we are using 'whole app compilers' - this is part of what they do, right, remove unnesessary things. 

At this point I would expect the resulting code to look something like this:

var a = [[270,90],....[270,...]];

This is not the case with either Google Closure compiler nor Dart2JS compiler. 

Interestingly GCC did calculate the initial numbers (before rotation) but only if the result is an integer, if the result was a floating point number it was left as it is (i.e. 360/7 was lest as is). The call to the process method was left intact as well as the logical operand in rotate.

Dart2JS did something similar, but it calculated all initial numbers including the ones resulting in floating point numbers. However calls to both functions was left in the resulting code. 

Still I was sure there has to be a way to tell the compilers that those numbers should all be pre-calculated and  used directly by the VM instead of running calculations on startup. (Yes, I know how fast JS is with simple arithmetic and that several numbers will not slow down the start of the app, but this is just an experiment).

Next step was to try and take out the logical operator from the  rotating function:

var rotate = n => n - 90; 

This will allow negative numbers to go inside the list but for our use case it was okay. Mind you, however that this can significantly change the  behavior in other cases, so it might not be appropriate to do in your case.

Still the result was not what I expected in both GCC and Dart2JS.

I was determined to force the compiler to inline the results... somehow.

Next step was to wrap each number in a direct call to the original rotate function. Yes, no one would write the code like this, but little regexp and voila:

var list = [[rotate(0), rotate(360/2)], .... [.... rotate(180+40+(30*2))]];

Still, no luck in GCC numbers resulting in integers were calculated, but wrapped in the minified identifier for the rotatte function:

var a = [[a(0), a(180)]...];

Same with Dart2JS as before, all numbers calculated before the rotation including floats.

Finally I tried with the simplified (no logical operators) rotate function and GCC gives us this:

var a = [[-90,90],... [...225]];

Note that we still had to compromise with allowing negative values. Of course still preserving the non-integers as original arithmeric expression. It finally hit me - GCC optimizes for size, not for speed or anything else. 360/7 is much shorter than 51.42857142857143

Dart2JS continues to insist that we call a function to make the calculation with minus 90...

We have come a long way in FE with adding compilers and transpilers of all sorts, however we still cannot expect miracles .. or in this case consise understanding of our intent when it comes to little 3d grade arithmetics.

Next step would be to see how does this compile in Rust to target web assembly!

Малко истини в книгите за лични финанси

Да започна с това какво мотивира този пост.

Преди няколко седмици попаднах на профила на млад български предприемач. Зарадвах се на успеха на един съвсем съвсем млад човек и доста време прекарах разглеждайки какво е споделил за себе си и пътя си.

Направи ми впечатление, че в една от статиите си споделя няколко книги, които са му помогнали да се развие като личност и да извърви по-бързо пътя към успеха. Една от тези книги беше "The richest man in Babylon" на Джордж Клейсън. Не бях чел книгата и реших да отделя време и да видя какво пък толкова може да помогне една книга.

Самият Даниел споделя, че посланието на книгата е простичко: харчи по-малко от колкото изкарваш.

В книгата обаче навсякъде се натъртва на още две неща: 1) отделяй една десета от онова което изкарваш и го инвестирай разумно и 2) ако имаш вече натрупани дългове отделяй 2/10 от онова което изкарваш за да погасяваш дълговете.

Стана ми интересно, какво всъщност наистина може да се постигне ако се следва този съвет буквално, по начинът по който е предаден, включително и с примери от началото на 20-ти век (предполагаме че фантастичното действие представляващо по-големия обем в книгата се развива няколко хиляди години преди новата ера).

Бидейки един виден прагматик, направих табличката, която може да бъде видяна по-долу (Линк към табличката за по-удобно разглеждане). В нея са заложени няколко интересни неща:

• данните са от 2000 година насам, за да избегнем труса с деноминацията
• използвали сме средната работна заплата така, както е докладвана от националния статистически институт
• разработени са и двата варианта (с дългове и без такива)

Разяснения:

• ако сме следвали логиката посочена в книгата и сме спестявали 1/10 от заплатата си, към днешна дата би следвало да имаме спестени 10,868.40 лв. 
• също по съвет в книгата, ако сме спестявали така посочената 1/10 и сме инвестирали разумно (заложил съм 5% дивидент на годишна база при месечни вноски 1/10 от заработеното) днес бихме разполагали 15,320.00 лв
• ако приемем че сме задлъжняли и през цялото време отделяме 2/10 от доходите за разплащане към тези задължения, то за периода бихме "върнали" 21,736.80 лв.
• ако през целия период отделяме 3/10 от парите (защото нямаме задължения, но се чувствае някак мотивирани да пестим повече) бихме имали 32,605.20 лв, а ако сме ги инвестирали разумно* - 45,959.99 лв

В "най-добрия" вариант (спестяваме 3/10 или 30% и инвестираме) сумата не изглежда голяма наистина, особено на фона на инфлацията, която се шири из нашите краища от доста време. От друга страна обаче кумулативното инвестиране на 30% от припечеленото за 16 години резултира в 42%, грубо погледнато за 16 години имаме една разлика от 12%.

Струва ли си?

За да преценим дали нещо си "струва" е необходимо да го сравним с алтернативни варианти. Един такъв вариант е да сме харчили всичко, което изкараме и да имаме 0.00 лв или минус някаква сума.

Друг вариант би било да ползваме кредитна услуга (например ипотечен кредит, фирмен кредит и тн) при който вариант нещата разчитаме така, че приходите възникващи на база инвестирания заем да покриват разходите по същия този заем и да генерират достатъчно разлика, за да има остатък (печалба).

Оставям на будния читател да си състави табличка за алтернативен подход, например "покупка на жилище за отдаване под наем през 2000 година", като цените на жилищата, кактои цените на наемите можете да получите като справка от всяка агенция, която оперира все още и е оперирала и тогава.

Истината е много проста: това, което изглежда като решение и може би е работило в древен Вавилон, днес е по-скоро консервативен вариант по няколко причини.

Вече знаем, че спестяващите биват наказвани от монетарната система след 1974 година (тоест необвъзразността на валутата с материална стойност), знаем също така, че ако искаме да имаме повече трябва да увеличаваме постоянно приходите си, най-добре по пасивен начин, защото в крайна сметка силите ни за работа не се увеличават с годините. Знаем и че монополизацията на кредитните системи е извършена с цел от една страна контрол над мнозинството и от друга поддържане на икономиката активна.

Взимайки предвид всички тези неща, моето мнение е, че съветите дадени в книгата е по-скоро неприложими ако търсеният ефект е онзи, който подобни съвети са дадени преди 4-6 хиляди години по-рано.

Дали ще изберете да ги следвате зависи само от Вас самите.

Послеслов: един много интересен съвет има даден в същата тази книга: не доверявай парите си на съвета на жената на месаря или на грънчаря. Съвет искай и взимай само от онези, които се занимават с правенето/наптрупването на пари.

Web worker, RAF, nextTick - what is there to it?

Let's start with the premise: we want to know the details on how a combination of a web worker (i.e. event based communication with a separate thread in JS) can be used to process data and communicate with the main thread in regards of asynchronous job scheduling.

What is what

Request Animation Frame - from MDN

nextTick - an API for scheduling work after the event loop is emptied. We use an emulation using iframes so basically we get message broker in frames covered instead of the actual setImmediate as it only works in IE10+.

Web worker - single threaded JS execution environment that is a separate process from the main thread (i.e. it does not share anything with it and scheduling execution in it is not related to the scope of async programming from JS developer's perspective - we cannot influence when things are happening there but we have the guarantee that those things that happen follow the same rules: async, single thread)

So what are test results showing

• responses from web worker always come in order (assuming no async programming is done there)
• the timing of the responses varies (i.e. run the test several times and see how sometimes 1, 2 or 3 responses arrive before the RAF or nextTick execution kicks in)
• nextTick kicks in after the first RAF

If you want to review the results make sure to: a) run the test several times and b) test in different browsers.

The code is here.

Conclusion: it is not trivial to drive animations and data processing from a worker instance if you depend on input for it.

Reducing time to first paint

This post is about the new techniques recently discussed on events related to web app for mobile: how to make our web application behave more like a native one and reduce friction for our users and meet their greatest expectation.

In this post I will discuss how we managed to reduce the time to first paint in an app we already had developed. We will be measuring several things:

• time to first meaningful paint
• time to app readiness
• time to first paint on first visit
• time to app readiness on first visit

Why there is a difference? First visit is important for several reasons: even with service workers being readily available in Android one still needs to consider those first visits to the app which can happen at any time and any network condition, forcing the user to wait is not going to be very good for any conversion strategy.

Secondary visits, especially for devices that to not have the service worker enabled (or in case we decided to not use it for our app for some reason) can also be potentially problematic for several reasons: busted browser cache, new version of the app etc.

Lets start with some numbers!

The app is consisting of a single html file, a processed (built) css file and a javascript (also built).

index.html - 566 bytes

app.build.css - 50702 bytes

app.build.js - 188824 bytes

What were the measurements on load time with this initial state:

First visit:

• time to first paint: 850ms
• time to readiness: 1000ms

Secondary visit:

• time to first paint: 15ms
• time to readiness: 1000ms

Its not that bad, but this is a relatively small application and also we use landline network to load the app.

The one problem is that in first visit the user is staring at blank screen for almost a second, loading on slower networks would only exaggerate that and can make the user give up on our app.

How can be possibly reduce this time to first paint on first visit?

We can attempt and utilize a technique described as 'fake' first paint: provide only a skeleton on the actual app view and how it would be looking like when ready and provide it as soon as possible (possibly in the initial HTML with CSS only for it). 

This sounds okay, but the fact is we already have the application built and we did not wanted to invest too much, also the additional markup and styling would have to be maintained separately and updated every once in a while as the application evolves. 

We decided to go with another approach: separate the code that is responsible for the initial html structure into a module to be loaded first, let it execute (i.e. unblock the event loop) and only then load the rest of that app that will 'decorate' the pre-rendered html and make the application ready for the user.

How does this look like in the context of a mid-sized closure tool driven application?

Several steps were taken:

• create a new namespace and render the main app template in it, it should be doing only that - requiring only the template and dom utilities
• create the HTML loading module that invokes the namespace from previous step and asynchronously wait for the next event loop tick to start loading the application logic module
• create the application logic module which should simply invoke your app (basically your main entry point from static (non modular) version
• make sure your main app logic accepts a new option to tell it that it should decorate a preexisting html structure as opposite of rendering itself

An example:

//Step 1
app.html.rootElement = goog.dom.htmlToDocumentFragment(
    app.template.App({
      user: app.settings.RUN_AS_USER
    }).getContent());

//Step 2
html_init = function() {
  // Add thr whole view to the document.
  document.body.appendChild(app.html.rootElement);

  // Wait for the rendering to kick off.
  setTimeout(function() {
    // Configure modules.
    var mm = goog.module.ModuleManager.getInstance();
    var ml = new goog.module.ModuleLoader();
    mm.setLoader(ml);
    mm.setAllModuleInfo(goog.global['MODULE_INFO']);
    mm.setModuleUris(goog.global['MODULE_URIS']);

    // Tell module manager that the html module is loaded
    goog.module.ModuleManager.getInstance().setLoaded('html');
    // start loading tha app module.
    goog.module.ModuleManager.getInstance().execOnLoad('app',
        goog.functions.NULL);
  }, 10);
};


// Fireoff things immediately.
html_init();

// Step 3
app_init = function() {
  // Instanciate the app controller.
  (new app.Main(true));
  // Tell the module manager that the app has been loaded.
  goog.module.ModuleManager.getInstance().setLoaded('app');
};

app_init();

// Step 4
app.Main = goog.defineClass(pstj.control.Control, {
  /**
   * @constructor
   * @param {boolean=} opt_useDecorate
   */
  constructor: function(opt_useDecorate) {
    pstj.control.Control.call(this);
    /**
     * @type {boolean}
     * @private
     */
    this.useDecorate = !!opt_useDecorate;
    this.init();
  },
...
});

As you can see the change is trivial, but it allows us to do two things: reuse the actual application template (might it be not functional yet, but it looks exactly like the final result) and thus its automatically kept in sync with the application as it evolves and two: reduce the initial load size (the size of bytes needed by the app to reach first paint that resembles the app view for the user).

What are the new results (after this change):

First visit:

• time to first paint: 400ms
• time to app readiness: 1100ms

Secondary visit:

• time to first paint: 15ms
• time to app readiness: 1100ms

Not that bad for 20 lines of code: we managed to reduce the time to paint on first visit (or cache miss) by a half with a simple change of order of execution. 

Now one question arises: why not simply do it inside the main build: render the view first as a template, then free the event loop and then decorate the application logic on top of it. 

Yes, this was initially the intent, that is why we went with the option to tell the main app entry point to use rendering or decoration (step 4 in the example), but since closure compiler is so good with modules we decided to start using this technique to be ready for the growth of the application and instead of adding feature after feature to the main build simply module-base new features and load them on demand, but not before the user needs them. This will allow us to have a 10 times bigger application but still keep the initial load size (and times) the same. 

As you might have notices the separation of the built JS binary to two modules had a cost: ~100ms delay until application readiness, you should b aware of that and carefully examine the trade-offs when deciding if you should use this approach or instead go for the one described in the beginning and simply use a dummy scaffold in the HTML. In our case we wanted to stay as close to the existing code as possible.

How does this change result in code size?

index.html - 782 bytes (because we included the module info in the html, it could be instead pre-pended to the first module)

app.build.css - 50702 bytes

module_html.js - 70231 bytes

modules_app.js - 118707 bytes 

As you can see the size impact is insignificant. Interesting side effect since the change we made for the modules is that the initial module (module_html.js) does not change significantly or all if the templates do not change (for example if we add new views but use the rendering path instead of decoration one the views logic and its structure will still be in the later loaded modules thus not impacting the initial load).

As a final though: working with modules in closure is fun but is not as intuitive as one might expect it. Also there is no guided module separation and for this reason the developer has to be an expert on how closure dependencies work and how to arrange the files in such a way as to avoid direct dependency and instead add dependencies and code indirectly. In more modern solutions (like Dart for example) requiring a piece of code (or a whole library that is) lazily / on demand is as simple as adding a modifier to the import and whenever you need make sure to load it first

import 'myasynccode.dart' deferred as myasync;
// later on in my code
main() {
  // Wait for user to request that piece of functionality
  await myasync.loadLibrary();
  // call code from library
  myasyncMethod();
}

Never the less closure can be used successfully to manage async code loading and thus help us with the task of greeting our newly come users as fast as possible with the best product we can offer.

Happy coding!

The real price of going ChromeOS all the way

Chromebooks to this school, chromebooks for that school...

It is being repeated over and over again, reviews are being written almost every week about a new chromebook (which is by the way exactly the same as the one from previous week, including chips, size, weight, battery life, keyboard, performance and so on), tons of user testimonials and praises are being liked and re-shared ... it is getting a bit old, isn't it?

Chromebooks are here to stay and with the new strategy to teach kids on them they are more than probable to remain part of the picture if not dominant.

However one important question is rarely if ever asked: what is the real price of using chromeOS as a replacement device (not as primary one, and not as a complimentary one, but as a sole keyboard equipped device)?

In the plethora of available devices like phones, tablets, TVs and now watches, one almost feel like there is no real need to have a PC or a tablet. Most people either already have one and use it in the rare cases where one of their other devices cannot do the job or buy new one(s) for one single reason: work!

But let's define how a laptop or a PC is used for work! If one's company is already embracing the cloud chances are they already use one or several services and pretty much all the office work can be done from a browser, thus chromeOS is capable of handling this. However there are several large classes or work related tasks that cannot be done without a specialized software: CAD/CAM, 3D production, Electrical and other type of engineering, and... software development. 

The last is possibly the most common use case. For years now companies are trying to create businesses that offer an online solution for those kind of work related tasks, but the adoption is slow and currently - very low. The concerns are numerous and come from different sides: security, speed, availability, reliability, management etc. The reality is that while those services are well intentioned and legit business ideas, the execution is often subpar and even if it is at good level with the user target the cost if overwhelmingly steeper compared to the cost of ownership. 

So how do things really stack together for a regular Joe developer for web? I will present a simple example - one that is of course based on experience (mine and those of friends of mine who have switched) and is by no means presentable as evidence, but I think it comes close to at least people in similar situation. 

So, Joe is working as a freelance developer, web stuff, JavaScript, PHP, some HTML and CSS. However from time to time he has to deal with Java, Python and other technologies. Joe knows that there are at least two possible solutions for his needs: a MacBook or a ChromeBook. The projected lifespan of the laptop would be 5 years.

MacBook:

Price (ownership): 1299$

External ssd disk (for backups): 199$

Text editor/IDE: 70-100$

ChromeBook:

Price (ownership): 199$-399$

Storage (auto-backups included by vendor): 2 years free, 1.99/mo for 3 years = 71.64$

IDE: 19.00/mo = 1440$

At this point we have to make some very important clarifications. The online IDE in the example is Cloud9, I personally find it to be the best for my needs and I have recommended it to my peers and this is possibly the reason why they use it as well. The pricing there is 'pay as you go' so basically it could be much cheaper than that, but it is worth noting that with the MacBook you get much more RAM and processing power than what you get on Cloud9 for almost the same amount of money. Also I have included an external disk for the MacBook case as one need to backup, we all know that, some of use learned it the hard way. With cloud storage this is being taken care of for us. 

I have not included other stuff like music or movies for several reasons: it is unclear to me how the cost of ownership compares to the cost of streaming, also I am not aware if purchased once the song becomes yours forever and can you copy it over unlimited times etc. I have also not included software like budgeting tools etc because I see lots of people migrating to online tools for this and not for other reason but being able to access their data from their mobile devices like phones and tablets. I would consider those expenses to be similar in both cases.

As seen from the above example if you are working as web developer the cost at the end of the period would be higher for the ChromeBook. That's okay if we apply some basic knowledge of finances and make sure the money spent in a period of time have different present value than the same amount if spent today, so lets keep it simple and say that those are almost equal (1).

Now let's consider a different price. I call it emotional price. People tend to love their stuff: gadgets are especially lover and Apple's products are at the top of loved possession. With a chromebook you simply do not care. I used to be proud of my computer - business class, slick, thin, wonderful. But on its own it is really nothing more than a way to show of as the components inside are doing the exactly same job as any other laptop. With the chromebook I really do not care. I know no one will notice it because it looks exactly the same as any other. Samsung attempted to make the chromebook look fancy and they failed. Being liberated from this emotional dependency and more over being unburdened with its security and safety was really something very new to me. 

Of course I try to keep it clean and working, but when my partner drops it do I freak out? Nope. Because I have no attachment to it. I know I can use another one, on the cheap end, and continue where I left of. Years ago Google put accent on that with the video ads for the first chromebook. Back then I did not even understood it. Now I do!

But wait, there is even another price - ecological. We all tend to say that because those devices are very cheap they are basically disposable. But in fact we do not use them as such. We use them as regular laptops, we tend to try to not throw them at walls and drop them, slip them in the bath full of water and so on, we don't care about them but we still use them as intended. So the projected life, while maybe not 5 years, is at least good 3 to 4 years. Of course Apple says that they do recycle and their laptops are indeed one of the best when it comes to energy efficiency. But think of this in a different angle: while your laptop sleeps for at least 14 hours every 24 hours, the cloud never sleeps. Which means that while you are not doing anything on it, someone else is using that processing power for something else. Cumulatively this reduces the material needed to satisfy everyone's computing needs globally. Think about the processor: yes, macbooks are really powerful, but how often do you load it at 100%, let alone load all the cores? Does it cumulate to more than 30 minutes daily? Now consider the cloud where you IDE is a virtualized machine and one single CPU is running yours and possible many more users. Electricity? If you run your MacBook at 100% load all the time it will not last 10 hours on battery, on the other hand cloud providers are in the rush to become greener and greener in ways we as individuals could not accomplish even if we wanted: cooling from the see, energy from the sun and wind etc. 

"But all the traffic I am making using those online apps, does that not require energy". Well yes, it does, but you are doing it anyways. Besides the amount of bytes to watch a movie online and to download it and watch it locally is the same. The amount of bytes you will exchange in a month of using an online IDE is lower than the amount of bytes you will exchnage to download a source locally and upgrade your IDE. 

Choosing a ChromeBook might be on par financially with buying a latest model MacBook, but the emotional and ecological costs are much lower in the case of ChromeBook.

And yes, it is certainly not for everyone, especially as a only device. But depending on your needs it could be reasonable choice and a more responsible one as well. 

Many years ago I have written about how you could be a responsible person in global planetary context and switch to Linux to keep the baby seal alive. Today I am saying this: be a responsible person and chose the greener alternative if you can!

Evaluating jsaction library.

From time to time one starts wondering how are the some bigger web applications written, what technology has been used and from the properties of the product attempt to make decision if such technology could be useful for one's workflow.

This week I decided to take a look at jsaction - somewhat small library used in the making of Google+. If you happen to not use G+ (and if you believe the Internets - the chances are you are not, unless you are a Google employee) the web app features fluid design and is very capable on mobile phone (under chrome) - works really great and stays responsive and fast even after using it for a while.

The library itself has an interesting proposition: separate the event registration from the event handling and in this way allow for some interesting situations:

• render server side and send content to be rendered as HTML
• send script that basically does nothing else but to queue events 
• additionally send 'handler' script that contains all the app logic 

As expected this results in HTML that is fast to load and works without waiting for the actual application logic to arrive over the wire. All handlers are registered on the body element and once the application logic is loaded the events are being replayed and dispatched to the designated handlers. 

The handlers themselves are described in the HTML directly via custom attribute (jsaction, thus the name of the library). The system is very similar to the event delegation pattern, but the actual handler code is determined from the html attribute instead of being declared in the code. This means that the handlers do not care much about where the event comes from in the DOM tree, which is kind of powerful.

The library supports custom events as well, unfortunately those cannot be declared in the HTML with the actual custom type of the event, instead the '_custom' type is always used and the actual type is hidden deep in the detail property of he event. In fact there is a global define flag that allows you to use the custom action name in the jsaction property: you can test it like this

var CLOSURE_UNCOMPILED_DEFINES = {
    'jsaction.EventContract.CUSTOM_EVENT_SUPPORT': true
}

Additionally the event's artificial propagation is stopped once a matching handler is found.

This poses and interesting limitation: you cannot use the same combination of event name and handler in a nested tree as the first match will stop the flow. This however can also be controller by a global flag, take a look at eventcontract.js

The library is small and does not contain any 'gotchas' as far as I can tell. Big part of the code is wrappers around IE and other specific cases (mouseenter/mouselave, mac specifics etc). Another big chunk is static description of the library specific syntax. 

In what case would it be useful? 

I could easily imagine pages where certain actions must be available on user input (a la jQuery 'sprinkling' magic) and you would like the application logic to be kept on only one place. In this scenario you would need code that 'finds' elements on the page and add handlers to them (usually in the form of $('whatever').bind(...)).

jsaction allows you to skip the last part and instead of imperatively bind the actual logic to specific elements that JS needs to find in the DOM, one can simply add a property to the needed elements that will automatically bind the correct elements to the correct actions and corresponding handler.

It is important to notice that the library requires closure library and closure compiler (the latter one is optional, but almost inescapable for production), but this is only an implementation detail. The idea of the library could be useful outside the scope of closure tools and reminisce of the inline handler definitions we used back in the 90s, only in this case we have the namespaces to separate the different controls and the custom events. 

In the context of larger apps I believe the concept is becoming too complex to handle as the JS developer and the template developer need to coordinate on what is exposed and where (as the controls can be separated into smaller containers and not necessarily being document level, this is the events are being listened for always on the body, but the resolution of handlers can be scoped to container element, allowing for 'widgets' like boundaries).

For smaller projects I think it could be very useful especially to decouple the application logic from the specific endpoints in the DOM from where it should be triggered. 

Conclusion

Is this library for me?

Probably not, unless you already have experience with closure tools and you do not want to go deeper in the UI frameworks forest and instead you need simple and clean solution to attach application logic to specific points in the view. Note that all default events are supported (so mouse, keyboard, touch) and you can make it as complicated as needed (G+ is using it, and it's not simple), but unless you really want to render mostly server side and not deal with finding elements or constructing DOM on the client or you have very fluid design and you cannot keep up with HTML changes in case you have a large project it might be beneficial to consider other options before electing to go with jsaction. 

RPC package generator for closure library / compiler

Update: published package and demo app.

Recently I have been using the rpc package for Dart, which in turn provides a very useful capability: it supports discovery document generation and stub file generation for consuming the API.

However my use case is a little bit different and Dart is not yet there when it comes to easy JS integration (i.e. consuming Dart code from external JS file).

My fronted is written entirely in JavaScript with closure library and closure compiler compatible style. Taking a note from the idea of automatically generated code I wanted to generate API specific rpc consuming package based on any discovery document, generated code to be 100% compatible with the compiler and the existing code in the library.

The effort took 3 days, but I am happy to say that it now works for what I have as a small test API. It is very possible that I missed something or there is an edge case I did not see, so apply cautiousness if you decide to use it.

What does it support?

On top of the standard expected things like DTO classes and rpc method implementations the following extras are included:

• integration with the xhr package (goog.labs.net.xhr) for clean promise based interface
• integration with the assertion library and checks on incoming data when goog.DEBUG is true
• dto instances that optionally extend a base class to support event emission when changed
• implements 'toJSON' in order to support easier serialization
• full type information preserved
• closure style guide rules are honored in the resulting code

The main idea was that I wanted the code to be as clean and as useful as if written by hand.

The produced file contains two packages namespaced after the API - one for the DTO classes and one for the rpc call. Resources are nested inside the rpc package.

The whole thing was designed as part of my pstj library. The code is here.

To run it with your project install all dependencies as described in the readme, then use this command:

node path/to/pstjlib/nodejs/discovery --uri http://some/discovery/endpoint --output-path path/to/store/rpc.js

As usual the code is closure compatible and can be run in the browser as well (see here).

I hope it could be useful to someone else as well.

List data optimizations experiment.

First and foremost it seems like a very important thing to tell that this test is entirely synthetic: it does not represent any significant information that you should consider when writing your applications. 

This said, if you are interested in the use case I will share it at the end of the post.

Last week I had a discussion with a colleague about memory, garbage collection and CPU utilization for list-like data structures - basically we wanted to check a theoretical postulate deducted from the knowledge about how JavaScript VM works and some assumptions about the speed it operates on different objects types.

The riddle was as follows: given a list of objects representing points on a drawing board, consider possible implementation as to achieve speed and low memory footprint.

Two competing ideas were considered initially: Plain JavaScript arrays and linked list implementation as JavaScript objects with 'next' property.

Additionally we wanted to see what is the overhead of having garbage collection involved, for that reason we have implemented four versions of the test code:

1. Using array, recreating the array at each pass and recreate the objects in the array each time.
2. Using linked list, recreating the items in the list each time and linking them, de-reference the head of the list when cleaning.
3. Using array but taking the items from a pool and returning them before de-referencing  the array on cleanup.
4. Using linked list and taking items from a pool and returning them before de-referencing the first item on the list.

To make it even more interesting we decided to implement the same thing in JavaScript and a language compiled to JavaScript, in our case Dart.

Then we had to come up with a value that is way out of the actual use case, but seemed like could be large enough to have a measurable performance implications. In our case we tried with 100BLN - however this simply crashed Chrome. After some experimentation we settles on 1MLN.

To simulate real world case we separated the creation and destruction of the data by binding it to buttons in the HTML,this was because we wanted to prevent any hidden optimization that could be performed if the scenario was run without pauses between or equal pauses. I am not aware if such optimization even exist, but I just wanted to be sure. Using setTimeout was not an option as I wanted to keep the code as clean as possible. Before running each test we were making a full refresh and force clean of memory, then we measured 6-10 creations and destruction and then we repeated the test and took the average time and memory of the execution.

The computer the test were run on is Intel(R) Core(TM) i5-3330 CPU @ 3.00GHz, 8GB mem, Fedora Linux (with all latest updates), Chrome 44.0.2403.130 in incognito window.

The results

## JavaScript

1) 210ms total time (75MB fluctuation)
2) 140ms total time (55MB fluctuation)
3) 70ms total time (23MB on 1 fold, 45MB on 2 folds)
4) 17ms total time (2K on 1 fold)

## Dart

1) 290ms total time (74MB fluctuation)
2) 130ms total time (56MB fluctuation)
3) 100ms total time (28MB on 1 fold, 55MB on 2 fold)
4) 29ms total time (1.5K fluctuation)

The first number of each measurement is the total time it took to execute the test function (i.e. without the handler of the click event of the button), the second one is the amount of memory the heap had to grow and subsequently clean for each run. Where 2 folds are used - the memory was cleaned every two runs, the first number is the amount it grew for single run. The last test does not trigger garbage collection for the small number of tests so it only grew.

As a side note: the time to clean things up in the case of a pool is almost the same as the time it takes to get items out of the pool, but it is not included in the total time as it happens asynchronously. 

Because we had feedback on the idea even before deciding to actually implement it I will share some of the feedback:

• arrays are very fast and you can use them everywhere, no need to optimize further
• garbage collection is very fast for new generation objects so you do not need pools
• arrays do not trigger garbage collection is you clean them with 'array.length = 0'
• iteration over arrays is as fast as accessing properties on an object so you do not need linked list

Because we were never involved in the process of optimizing for garbage collected languages, let alone V8 it was safe to assume that we might be wrong and any of the above statement are true.

The first test was no surprise: 1 million objects created and de-referenced on each run was expected to be memory costly.

What took us by surprise was that a list with 1 million items on it actually took  ~20MB of memory. Second test proved that having an array to contain all the items with such a big number of items was a significant overhead. At this stage we have also discovered that the 'array.length = 0' trick was no better than simply create a new array: arr = [].

There was simply no difference when comparing things in the context of 1 mln items.

The third test was interesting because it allowed us to limit the memory fluctuation using items from a pool and basically allowed us to measure much better how much memory the array actually consumes. The amount of memory used by the pool remains constant as it does not change the whole time. 

As you might notice the time of creating an array and crating objects is identical to the time of using only the objects (linked list) and only the array (when using pool). This was logical and expected.

The final test recreates destroys the linked list entirely from the pool. We made sure to clean the 'next' property correctly when returning items to the pool. At this point we almost made it to the point of making it into time for a RAF, but not entirely. The code will take longer on a lower specs computer of course. At this point I was really surprised, for some reason most people I talked about this with (me including) expect that adding an item to an array should not be so costly. 

I am almost sure that this is not the case for smaller arrays.

Potential use case

The reason I wanted to use the lowest memory solution was because of the context the data will be used in: drawing and animating on large canvas. I wanted to be sure that adding to the list and iterating over it is as efficient as possible as to avoid any pauses in the animation and drawing.

Dart 

The reason I wanted to implement the same test in Dart is that it is marketed as a solution for large scale applications that perform just as well as JavaScript and is scale-able and more suitable for large apps and teams. 

As one can see from the results there is a different between the two implementation, while I tried to follow the exact same semantics as close as possible. One difference that can be noticed in the dev tools is that the call stack is 3 levels deeper, however the measurement state that all those nested invocation have the same time share. I am not sure if this might be a good reason for the slower execution.

A side note for the Dart to JS compilation: it was done from the command line with the following transformation flags:

- $dart2js:

    checked: false

    minify: true

    commandLineOptions: ['--trust-type-annotations', '--trust-primitives']

To my knowledge this should produce the smallest, fasted code possible. 

Surely Dart VM could have beaten these results, but it is not in discussion anymore, JavaScript is the way to go.  What worries me a little bit is if dart code when transpiled to JS will ever run as fast 'native' JavaScript. Of course this test is totally superficial and should be taken with a ton of salt, but still, it is one of the simplest things to do in computing (working with lists). Is it possible that small, insignificant slow-downs here and there in Dart could make a whole large application slower than a similar one written in JS? This is for the Dart developers to answer.

Conclusion

Test are fun. We had a hypothesis and we wanted to see what will happen. We did. Should we always prefer linked list over arrays (where possible, as indexing is much slower as well as random access)? Of course not, arrays are great! 

But from time to time you might be in doubt how things work and if you correctly understand the execution of your code and the implication it has. In that case just write a test. Even if it is flowed, even if it is unrealistic - write a test and try to understand more about your target platform through it! 

Co-authored with LexMihaylov

Request Animation Frame for high performance applications

Disclaimer: This is a recent document I wrote for the technical blog run by a SaaS company operating in the USA. I got the permission to post here as well and this is what I am doing. 
Disclaimer 2: The code is based on the implementation of the same approach as found in the open source closure library.



As per recent publication on optimizing the drawing operations on a canvas element one of the recommendations is to use a single RAF instance and put your callbacks that should be working right before the next drawing to be done by the browser.

In a less demanding application a simplistic solution based on a queue should be sufficient to bundle the callbacks to run when the browser is ready to draw, this however does not solves the main bottleneck of the RAF approach in the first place: the callbacks might be causing a lot of layout requests and trashing without knowing about each other's internals and thus make the frame (the pre-drawing execution) much slower than it needs to be.

Another possible problem with the default approach (i.e. calling

function myCallback {

// potentially close over the variables in the current scope

requestAnimationFrame(myCallback);

}

inside the body of the callback) is the creation of closures on each execution. When multiple such occurrences are live in the application this might end up creating too much memory allocations every N-th frame and thus promoting the garbage collector run. This is a potential problem because several generations of callbacks have already been passed and the result is that generative garbage collection might not be effective enough and your application might end up spending > 40ms for middle sized apps in GC phase while inside a RAF callback. As expected this leads to a delay in the call for the next frame and user perceives this as frame being dropped.

To mitigate those pitfalls we can devise an approach that has the following characteristics:

• make it easy to schedule work for the next frame
• make it easy to only do work once per animation frame, even if more than one event fires that triggers the same work
• make it easy to separate and do work in two phases to avoid repeated style recalculations caused by interleaved reads and writes
• avoid closures creation per schedule operation

To achieve this we need to be able to:

• create the callback only once and call it without using closures
• create the callback in such a way as to allow ordered execution in two contexts: read from DOM and write to DOM for each callback

Because the callback is now separated in two phases we also need an object to represent 'read' state and to be passed to the 'write' state. For example we want to read the offsetHeight of an element and we want to perform some calculations with it. After that we might need to update the style / positioning of an element based on the performed calculations.

As already known the fastest way to perform a task is to not perform it at all. Same principle applies to animating with RAF: if you can avoid a job - avoid it. An extension to this principle is to use 'pre-calculation' - that is to use the time between the triggering of a new animated interaction and actually start the animation to pre calculate anything you might need as values while the animation is running. An example for this is pre calculating dragging thresholds that might trigger an action: instead of calculating the potential next threshold while performing the dragging you can pre-calculate all thresholds in the visible area and use them to compare values while animating the object the user is dragging.  This will also avoid more memory allocations while animation is being run. For example if you create an array of threshold values it is better than to create a new value after each threshold. Basically think of this code path as critical and make the code as static as possible: pre-allocate all memory you might need (i.e. new array with the exact length you expect to use in any calculation involved), pre-generate the actual animation code.

Now lets take a look at one possible implementation of such approach: https://gist.github.com/pstjvn/f0197e09381eb346160b

What we did is define a single function that will be available in the global scope and can be used to create callbacks for event handling that can be used as regular handlers for events and still work in sync with the browser's drawing operations.

Lets see an example:

var el = document.body;

var task = window.createTask({

 measure: function(ts, state) {

   // Record if the document is scrolled and how much.

   state.scrollTop = this.scrollTop;

 },

 mutate: function(ts, state) {

   if (state.scrollTop == 0) {

     // remove header shadow

     el.classList.remove('scrolled');

   } else {

     el.classList.add('scrolled');

     // add header shadow to indicate that there is scrolled content

   }

 }

}, el);

// Will ignore parameters passes.

el.addEventListener('scroll', task);

While contrived, this example demonstrates the power of this approach: a single function is created once for each RAF tasks and is reused as a regular event handler. Internally the work is synchronized with the browser drawing scheduler and layout trashing is prevented assuming you avoid mixing the measure and mutate operations and correctly separate them in the corresponding functions. Existing implementation do check your calls for correct use of only measuring in measure phase and only mutations in mutation phase, but ultimately it is developer's responsibility to use the tool as per its design.

What can be improved in this example? One might add new property to the state that keeps the last state and only assign classes when there is actually change. In this case this is neglectable as we know that modern browsers do avoid re-layouting when 'classList' is used and no change is detected, but might be a potential gain in other use cases.

An optional improvement in the implementation is to allow the creation of the task to also accept an instance that has a certain shape and thus avoid garbage while restructuring the state in the animations. For example:

function MyState() {

 this.scrollTop = 0;

 this.wasZeroPreviousTime = false;

}

Now one can create state instance when creating the task and have completely static task representation with state.

Conclusion

When developing large and highly interactive web application with JavaScript one might often be tempted to take the short road and write code in an intuitive way in order to accomplish programming tasks faster. JavaScript is notorious for allowing the developers enough flexibility to do just that. However flexibility often comes at a cost and while the code is valid and runs fast on your desktop computer, one need to consider the implications in mobile devices.

Finally, if you already have an application and you see performance penalties instead of blindly rewriting your code always measure and deduct where the potential for improvement is and only then start refactoring.

The approach presented in this article is a tool and not a complete solution for all your intensive animations, but is a good one and should be considered when applicable.

The confessions of a chromebook user

Here is the deal - I am sick with the Chromebook!

I know I will get flamed even after that first line, not because I have no right to be mad at those machines and not because they are perfect and I am wrong; from that first line it will be because of the new fan-boy bandwagon that has been rolling recently - the Chromebook / ChromeOS core fans.

It is understandable - after so many years in the grip of Microsoft and Windows, after so many years of Linux being the idiot child in the bunch, after so many OS disappearances (even today I have read about someone testing Solaris... what?). And then there is OSX as well, the brave saviour of us all that never was - on its latest incarnations OSX is so buggy that I often hear the once 'windows' branded joke - "it needs rocket fuel to run at normal speed" being repeated over and over again, now that 8GB of RAM is the minimum required to have installed if you want to browse the web in more than one tab or actually do some work on your computer. 

And there it is - the sunshine, the little gem that Google decided to gift to the world for unknown or at least unclear reasons: ChromeOS - a stripped down version of Linux that runs it's own graphics stack and has Chrome on top of it.

I can whine and whine for days about the issues with Linux and even more - months - for the stupidity collection labelled as OSX, but this is a post about ChromeOS.

I knew I was not going to get out of this without whining at the end, but there it was - cheap (okay okay - affordable) and promising to never bother me with updates, viruses, hardware incompatibilities etc. laptop...

What I did not expect was that even their own (Google) services will be impossibly slow to use on those machines in less than 1 year! And don't start with 'powerwash' or 'try in an incognito mode' - this is bull shit! I want to have adblock and still be able to use the damn site and I would NEVER use youtube if I have to watch an ad before each fucking video there!

Google+ is now so slow I can actually see the rerendering when new data is loaded. Spreadsheets - I can actually take a small nap when the initial load is going. Hangouts - I actually need to kill it from time to time because I consider half a gigabyte of RAM used by a chat service to be obscene!


Was this whole thing a plan to 'revive' the dying PC market? Or are we too stupid to not remember the rise and fall of the 'netbook'?

In any way ChromeOS might be gaining 10-20 points in some benchmarks now and then but those devs at google are making sure that each week the apps become more and more demanding even if they do not present new features (like for example using paper elements in drive - WTF!) - at the end of the full year of ownership you end up with an almost completely unusable device - its too slow to handle new apps, its unsellable (unless the buyer is a fanboy who does not care if the device can actually be used for something), its untweakable (at least with old windows laptops you can sell to linux nerds and they can use it but who would buy a device with 16GB or storage, its not 1999 any more - the last time I was able to sell a PC with only 4 GBs of storage).

So for 1/3 of the price you actually get 1/3 of the usable lifetime and 1/3 of the functionality of a regular laptop.

I guess one gets what one pais for.

So I have only me to blame for my stupidity. Once I had a thinkpad - latest business model, ultra portable (its funny how almost all laptops today are more portable than the then ultra portable) - I paid premium. I have made a mistake.

Now I have a chromebook - I have paid really small amount and again I made a mistake.

I guess Buddha was right - the true path is in the middle.

And if you are wondering - yes, I ave tried crouton - on paper it sounds great, have you tried it running for more than a day - all the storage is eaten by the swap created for it. ALL OF IT! It is a complete crap! Even if I leave o the background doing nothing it still eats up all the (oh so large) free space on the ssd.

Also I am sick of not being able to play AC3 sounds. I cannot chose what people send me.

If after all this you still consider buying a Chromebook as a second laptop ask yourself this: why do you need a second laptop? You get bored with the first one and you need something else to sit on your lap?

At least this is what I feel right now. Might be the chromebook, might be the pain from the oral surgery I had today. But I will definitely not buy another chromebook in the foreseeable future.