Wednesday, November 12, 2014

Bacon Blog is Back!

Been a while since my last post on this blog. And it's not that stuff hasn't been happening around Bacon.js. It's more that I haven't had time to write and/or have kept the bar a bit high. But now it's time for a change.

I'm gonna throw a post here weekly from now on, with short posts that take less than 30 minutes to write. Let's see what happens.

I just made 2 releases of Bacon.js today. And in total, there's been 32 minor releases since 0.7.0 that my previous post covered. A lot of bugs have been fixed and performance improvements have been made. So if you're for some reason using Bacon.js version < 0.7.32, you should update! In case you're interested in a detailed list of changes, have a look at RELEASE-NOTES.md!

Anyway, let me briefly introduce the API changes since 0.7.0.

Error handling improvements (#310)

Added Bacon.retry for retrying a background request a few times before giving up and producing an Error event. Here's @mileskin's original example with JQuery Ajax.
Bacon.retry
  source: -> Bacon.fromPromise($.ajax(...))
  retries: 4
  interval: ({retriesDone}) -> (1 + retriesDone) * 5000
  isRetryable: (response) -> response.status isnt 401
Also added flatMapError for flatMapping errors.

flatMapWithConcurrencyLimit, flatMapConcat, bufferingThrottle, holdWhen (#324)

The bufferingThrottle method allows you to throttle events with a buffer. So, when a regular throttle discards excess events, this one will buffer them.

The flatMapWithConcurrencyLimit method is a variant of flatMap that limits the number of concurrently active child streams, so that you can, for instance limit the number of concurrent AJAX requests to a certain maximum by automatically queuing the excess requests.

The flatMapConcat method is like the previous, but with concurrency limit of 1.

The holdWhen method can be used like a valve, to buffer events when a Property holds the value true, and release all buffered events when the value changes to false.

Support "on/off" events in Bacon.fromEventTarget (#461)

Some libs use the "on" and "off" methods for registering/removing event listeners. Now it's easy to integrate into those libs too. So, for instance, if you use CodeMirror and want a stream of code changes in the editor, you can now

    Bacon.fromEventTarget(codeMirror, "change")

That's it for now. Will come back next week. Suggestions for topics?

Saturday, December 21, 2013

Bacon.js 0.7.0 is out!

This is the stuff I've been slowly working on in my spare time. I'm so glad I finally got this version released, since it's been in a kind of a 99% ready state for quite a long time, and I absolutely hate to have any "almost ready" code rotting in a codebase. Gotta ship it! 
And the timing is of course perfect too, so you can consider this my $holiday_occurring_in_late_december present to you.

Glitch-free FRP #272

The main goal of the 0.7.0 release is to make FRP with Bacon.js essentially glitch-free. And this is the most invasive change into Bacon.js internals for quite a while. Also something that I many times thought impossible to solve with the brain capacity available to me.
Let's start with the problem that I'm trying to solve here. A glitch in FRP is a temporary violation of an invariant that, with bacon.js at least, occurs when simultaneous events are involved. And just to be precise, my definition for simultaneous events is events that have the same root event. So, for instance, a stream a and its descendant a.map((x) -> x * 2) produce simultaneous events by this definition, while a = Bacon.later(1) and b = Bacon.later(1) do not.
The exec summary of this improvement is that 0.7.0 is more reliable.
But if you're intrested in the details, read on! For instance, if you use Bacon.js 0.6.x and run this
a = Bacon.sequentially(10, [1, 2])
b = a.map((x) -> x * 2)
c = Bacon.combineAsArray([a, b])
c.log()
You'll get
[ 1, 2 ]
[ 2, 2 ]
[ 2, 4 ]
where the pair [2,2] is a glitch that occurs because c is updated first by the changed value of a and the the changed value ofb. In glitch-free FRP you should get just
[ 1, 2 ]
[ 2, 4 ]
Since 0.4.0, Bacon.js has featured "atomic updates", but that has beed quite limited, because they only apply to a graph of Properties. The above example fails in pre-0.7.0 because of this limitation. In this example, the problem could be worked-around by making a a property as in
a = Bacon.sequentially(10, [1, 2]).toProperty()
But there are more complex cases that are harder to work-around. Like if you have a complex graph of Properties that you combine using combineWith(a, b, f), the pre-0.7.0 system doesn't guarantee that your combinator function f won't be called for some invalid intermediate pairs of values when the sources a and b occur simultaneously.
And then there are takeUntil and skipUntil. Previously the functioning of these combinators has depended on subscription order. For example, the correct functioning of a.takeUntil(b) in case of simultaneous events from sources a and b has depended on that the event from b gets in first. And it did in most cases but not always. There's a new test case (look for the word "evil") that fails for pre-0.7.0 versions.
How did I solve this? Simply using a dependency-graph. Now all Observables have a list of "deps". For instance, if you run
Bacon = require("../src/Bacon")

a = Bacon.constant("a")
b = Bacon.constant("b")
c = Bacon.combineAsArray([a, b])

console.log("deps of c:", c.deps().map((s) -> s.toString()))
You'll get
deps of c: [ 'Bacon.constant(a)', 'Bacon.constant(b)' ]
So now, using this new dependency information, it's possible to hold the output of any combined Observable until all of its deps are ready. And that's how it works. Well, except when you add a new subscriber while bacon.js is currently flushing events. To make that work flawlessly too, I had to shave another yak. But it's shaved now and all is well.

Observable.toString #265

As you might have spotted above, there's now a rather nice toString method for all Observables. The output of this method resembles very closely to the code that was used to create the Observable. For instance
Bacon.combineTemplate({name:Bacon.constant("Bacon"), version: Bacon.constant("0.7")}).toString()
will output
Bacon.combineTemplate({name:Bacon.constant(Bacon),version:Bacon.constant(0.7)})'
Also, I've added a name method as @rassie suggested in #273 and implemented .inspect() as an alias to .toString() so that Node.js will show your Observables in a nice manner in the console.
So now you can do this in Node.js.
> B.once("1").name("one")
one
> B.once(1).name("stuff").take(1)
stuff.take(1)

Support for analysis/debugging/profiling tools #273

These are some experimental features that I included in 0.7.0 for preview.
I'm talking about improvements that will enable the implementation of development tools that can, for instance, visualize the event network and event flow going on. The improvements include
observable.deps()
Returns an array of dependencies that the Observable has. For instance, for a.map(function() {}), the deps would return [a]. This method returns the "visible" dependencies only, skipping internal details. This method is thus suitable for visualization tools.
Internally, many combinator functions depend on other combinators to create intermediate Observables that the result will actually depend on. The deps method will skip these internal dependencies.
observable.toString()
Returns a nice textual presentation of the stream. Covered in #265.
observable.internalDeps()
Returns the true dependencies of the observable, including the intermediate "hidden" Observables. This method is for Bacon.js internal purposes but could be useful for debugging/analysis tools as well.
observable.desc()
Returns a structured version of what toString returns. The structured description is an object that contains the fields context,method and args. For example, for Bacon.fromArray([1,2,3]).desc() you'd get
{ context: Bacon, method: "fromArray", args: [[1,2,3]] }
Bacon.spy(f)
Adds your function as a "spy" that will get notified on all new Observables. This will allow a visualization/analytis tool to spy on all Bacon activity. Current implementation just calls your function, but maybe, it should allow you to wrap all created Observables in your own wrapper to make spying even easier? This would enable AOFRP (aspect oriented functional reactive programming) lol.
All of this stuff is included in 0.7.0 but only as experimental features subject to change later. The problem is that spy alone is not enough for visualizing the event network. To observe the lifecycle and events passing through all the Observables, we need to add a way to observe without forcing a subscription. Currently, all you can do is to call subscribe to watch all the events, but that will also prevent the stream from being disposed when it has no "real" subscribers. Nasty details to be be found in #273.
So, even though my intention was to "officially" include #273 into 0.7.0 too, it proved a bit more involved than it seemed at first, so we'll get back to it later.
Anyways, enjoy the new version of Bacon.js and please let me know what you think!
Cheers!
@raimohanska

Wednesday, December 4, 2013

Wrapping Things in Bacon

I've got a lot of questions along the lines of "can I integrate Bacon with X". And the answer is, of course, Yes You Can. Assuming X is something with a Javascript API that is. In fact, for many values of X, there are ready-made solutions.
  • JQuery events are supported out-of-the box
  • With Bacon.JQuery you get more, including AJAX and two-way bindings
  • Node.js style callbacks, with (error, result) parameters, are supported with Bacon.fromNodeCallback
  • General unary callbacks are supported with Bacon.fromCallback
  • There's probably a plenty of wrappers I haven't listed here. Let's put them on the Wiki, shall we?
In case X doesn't fall into any of these categories, you may have to roll your own. And that's not hard either. UsingBacon.fromBinder, you should be able to plug into any data source quite easily. In this blog posting, I'll show some examples of just that.
You might want to take a look at Bacon.js readme for documentation and reference.

Example 1: Timer

Let's start with a simple example. Suppose you want to create a stream that produces timestamp events each second. Easy!
Using Bacon.interval, you'll get a stream that constantly produces a value. Then you can use map to convert the values into timestamps.
Bacon.interval(1000).map(function() { return new Date().getTime() })
Using Bacon.fromPoll, you can have Bacon call your function each 1000 milliseconds, and produce the current timestamp on each call.
Bacon.fromPoll(1000, function() { return new Date().getTime() })
So, clearly Using Bacon.fromBinder is an overkill here, but if you want to learn to roll your own streams, this might be a nice example:
var timer = Bacon.fromBinder(function(sink) {
    var id = setInterval(function() {
        sink(new Date().getTime())
    }, 1000)
    return function() {
        clearInterval(id)
    }
})
timer.take(5).onValue(function(value) {
    $("#events").append($("<li>").text(value))
})
So,
  • you call Bacon.fromBinder and you provide your own "subscribe" function
  • there there you register to your underlying data source. In the example, setInterval.
  • when you get data from your data source, you push it to the provided "sink" function. In the example, you push the current timestamp
  • from your "subscribe" function you return another function that cleans up. In this example, you'll call clearInterval

Example 2: Hammer.js

Hammer.js is a library for handling multi-touch gesture events. Just to prove my point, I created a fiddle where I introduce a "hammerStream" function that wraps any Hammer.js event into an EventStream:
function hammerStream(element, event) {
    return Bacon.fromBinder(function(sink) {
        function push() {
            sink("hammer time!")
        }
        Hammer(element).on(event, push)
        return function() {
            Hammer(element).off(event, push)
        }
    })
}
It's exactly the same thing as with the above example. In my "subscribe" function, I register an event handler to Hammer.js. In this event handler I push a value "hammer time!" to the stream. I return a function that will de-register the hammer event handler.

More examples

You're not probably surprised at the fact that all the included wrappers and generators (including $.asEventStream,Bacon.fromNodeCallbackBacon.intervalBacon.fromPoll etc) are implemented on top of Bacon.fromBinder. So, for more examples, just dive into the Bacon.js codebase itself.

Sunday, February 3, 2013

Chicken, Egg and Bacon.js


FRP is easy when you can define your "event network" first and then just watch it make profit. For instance, you may define EventStreams and Properties based on DOM events, compose them, do some AJAX and show the result in the UI, just like we did in the Bacon.js Tutorial parts II & III.
Sooner or later, though, you'll run into the "chicken and egg" problem: you need to base your streams on something that will change on the fly. So, you cannot just define your streams based on existing DOM objects, because the DOM will change.
For example, if you want to implement the quite well-known TodoMVC application with Bacon.js, you'll have to solve this problem. I suggest you take a look at TodoMVC yourself, but here's a brief description of the problem. Let's start with a screenshot.
TodoMVC
The TodoMVC application is a simple task manager; you add new tasks, mark them complete, edit them and delete them. You have the All, Active and Completed views. Stuff like that.
Assuming you've read the previous log posts and done some Bacon.js coding, you'd hack most of the app together in a couple of hours, save for the tough part. The tough part is the dynamic view, say TodoListView, which lists the Todos that match the current filter (All/Active/Completed). In fact, the only tough part of the view is that the view also allows you to edit and delete the tasks.
Now let's start with the assumption that we want to define a Property, say todosProperty that captures the whole data model of the application. The value of this Property will be an Array of Todos, i.e. something like this:
[ 
  { id: 1, title: "Buy Bacon", completed: false },
  { id: 2, title: "Buy Eggs", completed: false }
]
This model would make it very easy to re-render the list whenever the property changes. Also it would be easy to show only completed or active task by applying a filter:
var activeTodos = todosProperty
  .map(function(todos) { return _.where({ completed: false })})
The number of active tasks could be derived like this:
var activeTaskCount = activeTodos.map(".length")
To simplify a bit, the todosProperty itself would now depend on item additions only, so we could define the property in a very FRP'ish way:
var newTodos; // stream of new Todo items
var todosProperty = addedItems.scan([], function(todos, todo) {
  return todos.concat([todo])
})
In real life, there are more factors involved, but let's stick to this assumption for now.

If the list was not editable

If the list items were not editable, it would be easy to render the list based on the todosProperty:
var todosProperty; // a Property containing an Array of Todos
var listElement; // jQuery element
todosProperty.onValue(function(todos) {
  listElement.children().remove()
  _.each(todos, function(todo) {
    var todoElement = renderTodoView(todo)
    listElement.append(todoElement)
  })
})
We just redraw the list each time anything changes. The list items are not editable so we don't have to capture any data from them. Just render and forget.

If the view was static

On the other hand, if view was static, i.e. the set of displayed items was not changing, things would be quite easy too. Now you could capture the edits for the Todo items with something like this:
var todos; // list of todos
var listElement; // jQuery element
var properties = _.map(todos, function(todo) {
  // call renderer function, returns a jQuery element
  var todoElement = renderTodoView(todo) 
  listElement.append(todoElement)
  // Use Bacon.UI helpers to create Properties for the "title" and "completed" fields
  return Bacon.combineTemplate({
    title: Bacon.UI.textFieldValue(todoElement.find(".edit"), todo.title)
    completed: Bacon.UI.checkBoxValue(todoElement.find(".toggle", todo.completed)
  })
})
// now we convert this list of Properties to a single property
// each value of which is an Array of Properties
var todosProperty = Bacon.combineTemplate(properties)
Nice and easy. Render Todos once, collect editor values into a single Property using a couple of combineTemplate calls and we've re-defined the todosProperty.

But it's dynamic and editable

So it happens to be the case that the value of todosProperty depends on user's interaction with the DOM elements corresponding each Todo item on the list. And these DOM elements will be added and removed based on changes to the todosProperty. Classic chicken-egg.
The point of this blog post is to present some of the known solutions to this problem.

Event delegation (a.k.a live bindings)

One way around that chicken-egg problem with dynamically-created views is to use event delegation; bind event handlers to a containing element, and use a specific selector as the second argument to asEventStream. In TodoMVC, we could try something like
var itemEdits = listElement.asEventStream("keyup", ".edit")
This would give us all keyup events from child elements of listElement, matching the .edit selector. Practically we'd get events when the user edits any of the Todo titles on the list.
Now we could re-define todosProperty to take these events into account. But before that we'd have to take care of something: we need to include some identifier into the events in order to know which Todo was actually edited. One way to do this would be to include a data-todo-id attribute to the text input element. This wouldn't be a big deal especially if we used a templating lib like Handlebars.
Assuming the text input element had this data attribute, we could continue like
var itemEdits = listElement.asEventStream("keyup", ".edit")
  .map(function(event) {
    var textField = $(event.target)
    return {
      id: textField.attr("data-todo-id"),
      newTitle: textField.val()
    }
  })
Which would give us a stream of all item edits. For example
{ id: 1, newTitle: "Buy a lot of bacon" }
Re-implementing the todosProperty could now be done in a number of different ways. For some reason, I like to model this by constructing a number of streams containing modifications to the list, where the modifications are actually functions that take the previous list of Todos and return the modified list. So we might say
var itemModifications = itemEdits.map(function(edit) {
  return function(todos) {
    return _.map(todos, function(todo) {
      if (todo.id == edit.id) {
        return _.extend(_.clone(todo), { title: edit.newTitle})
      } else {
        return todo
      }
    })
  }
})
This converts the stream of edits into a stream of functions that will transform the list of todos by changing the title of a single todo. We'd also have to convert the stream of new Todos into this form, but I'll leave that as an excercise for you. The todosPropertywould now look like
var todosProperty = itemModifications.merge(itemAdditions)
  .scan([], function(todos, modification) {
    return modification(todos)
  })
So, for some cases, we can come by the chicken-egg problem by using jQuery smartly. But at least I find this a bit inelegant and not very scalable; speaking in MVC terms, you have to create your Views before you create your Model because the latter depends on the former.

Apply some MVC

At this point I'm struggling not to bake in some smart-ass analogy involving chickens, eggs and bacon. And speaking of MVC, there's another way to get by the chicken-egg problem.
So far we've kept trying to define todoProperty in the traditional FRP way, which is, by composing from a fixed set of source streams. This is elegant to some point. But sometimes it may be more practical and arguably more elegant to make a switch from pull to push. And use Bacon.Bus.
Instead of defining all of the event sources at once, we can introduce a Bus, which allows us to explicitly push() events to the stream. It also allows us to plug() in streams after the creation of todosProperty.
So let's take a step towards a more traditional MVC architecture and use a stateful model object to represent the list of Todos. The UI can tell this model to change using methods like addTodoremoveTodo and modifyTodo.
We'll use a variable, todos in the list model to hold the current list of Todos.
function TodoListModel() {
  var todos = []
  var changes = new Bacon.Bus()
  this.todoProperty = changes.toProperty(todos)

  function update(modification) {
    todos = modification(todos)
    this.changes.push(todos)
  }

  this.addTodo = function(newTodo) {
    update(function(todos) {
      return todos.concat([newTodo])
    })
  }
  this.modifyTodo = function(updatedTodo) {
    update(function(todos) { 
      return _.map(todos, function(todo) {
        return todo.id === updatedTodo.id ? updatedTodo : todo
      })
    })
  }
}
This model object now exposes the field todoProperty which can be used like in the earlier examples. You might notice that I'm using the exact same functions for the actual list modifications too. The idea is to expose the required set of imperative-style mutator functions that will modify the model's state and push the new list of Todos to the Bus object changes. The exposed todoPropertynow reflects the current state of the model, which is the latest list pushed to changes.
Now it's easy to derive Properties like activeTodoscompletedTodosactiveTodoCount etc in the FRP way.
To summarize, we use imperative style for changing state, but expose the state as FRP EventStreams and Properties. One might say that we gain the best of both worlds.
This approach may get easier by using, for instance, an actual Backbone model and expose its state as EventStreams and Properties, as in the Backbone-Bacon TodoMVC list model by Jarno Keskikangas. He's even put a Backbone.EventStreams library on Github. In his own words,
Bacon gives superpowers to Backbone controllers.
Amen.

One step back towards FRP

If you feel uncomfortable with variables (I do), you might want to revise the MVC solution above. Instead of using a variable, we'll usescan:
function TodoListModel() {
  // A Bus of modification functions
  var modifications = new Bacon.Bus()
  this.todoProperty = modifications.scan([], function(todos, modification) {
    return modification(todos)
  })

  function update(modification) {
    this.modifications.push(modification)
  }

  // no changes to the rest...
}
So that's the same scan approach as in the pure FRP solution.

Yet another step towards FRP

This is the way I actually implemented TodoMVC. And submitted a Pull Request too.
Instead of exposing imperative mutators in the model, you can expose corresponding Bus objects. This allows you to plug-in source streams to the model. The model would look like this:
function TodoListModel() {
  function mapTodos(f) { return function (todos) { return _.map(todos, f) } }

  function modifyTodo(updatedTodo) {
    return mapTodos(function (todo) {
      return todo.id === updatedTodo.id ? updatedTodo : todo
    })
  }
  function addTodo(newTodo) {
    return function (todos) {
      return todos.concat([newTodo])
    }
  }

  this.todoAdded = new Bacon.Bus()
  this.todoModified = new Bacon.Bus()

  var modifications = this.todoAdded.map(addTodo)
                      .merge(this.todoModified.map(modifyTodo))

  this.todoProperty = modifications.scan([], function (todos, modification) {
    return modification(todos)
  })
}
It the View code you'll plug-in streams to the exposed Buses todoAdded and todoModified. For instance, this is how you might implement a simplified view responsible of rendering a single Todo row.
function TodoView(todo, listModel) {
  this.element = render(todo) // use Handlebars or whatnot
  var titleChanges = this.element.find(".edit").asEventStream("keyup")
    .map(".target").map($).map(".val")
  var modifications = titleChanges.map(function(title) {
    return { id: todo.id, title: title, completed: todo.completed }
  })
  listModel.todoModified.plug(modifications)
}
One advantage of this approach, compared to the previous one with the imperative mutators is that when you expose the busestodoAdded and todoModified you're also exposing the same things as streams. So now your View components can subscribe to these streams to react to Todo additions and modifications!
There's a little catch here too: you should make sure you apply an "end condition" to the input streams you plug into the model Buses. If you don't do this, the Bus will have references to the streams even after the corresponding UI components have been removed. In practise, you'll probably remove the UI components based on an event in some EventStream, so you can use this same stream for ending the input streams. For example, in TodoView you might have a deleted stream that signals the deletion of this particular Todo. Just add .takeUntil(deleted) to your modifications stream and it will end on deletion:
function TodoView(todo, listModel) {
  // .. begins as before ..
  var deleted = listModel.todoProperty.changes.filter(function(todos) {
    return _.where({ id: todo.id}).length == 0
  })
  deleted.onValue(function() { this.element.remove() })
  listModel.todoModified.plug(modifications.takeUntil(deleted))
}
You may have a look at the full Bacon.js TodoMVC implementation for reference. I'm sorry for the tab indentation. That's the standard for TodoMVC. Also, don't expect the implementation to be exactly the same as in any of the above examples. But it definitely is based on a TodoListModel that exposes Buses and streams to the Views.

Conclusion

We've actually covered two related problems in the pure FRP approach, both of which stem from having to introduce some Views before the Model because the Model depends on the Views.
  1. The Chicken-Egg Problem - your Model depends on your Views which depend on the Model
  2. The Coupling Problem - you have to Introduce some View components before the Model, so you cannot properly modularize your application
The first problem in isolation can be in some cases solved by using techniques like jQuery event delegation. But to address the problem in the bigger picture, you need to apply architectural solutions, such as introducing a Model object with either mutator functions or pluggable Bus objects.