# deferred.el #
[](https://travis-ci.org/kiwanami/emacs-deferred)
[](https://coveralls.io/r/kiwanami/emacs-deferred)
[](http://melpa.org/#/deferred)
[](http://stable.melpa.org/#/deferred)
[](https://github.com/kiwanami/emacs-deferred/tags)
[](http://www.gnu.org/licenses/gpl-3.0.html)
`deferred.el` provides facilities to manage asynchronous tasks.
The API and implementations were translated from
[JSDeferred](https://github.com/cho45/jsdeferred "JSDeferred") (by cho45) and
[Mochikit.Async](http://mochikit.com/doc/html/MochiKit/Async.html
"Mochikit.Async") (by Bob Ippolito) in JavaScript.
*(note the README for `concurrent` is [here in the same repo](./README-concurrent.markdown))*
## Installation ##
You can install deferred.el from [MELPA](http://melpa.org) by package.el.
## Sample codes ##
You can find following sample codes in `deferred-sample.el`.
Executing `eval-last-sexp` (C-x C-e), you can try those codes.
### Basic usage ###
This is a basic deferred chain. This code puts some outputs into
message buffer, and then require a number from minibuffer.
Chain:
```el
(deferred:$
(deferred:next
(lambda () (message "deferred start")))
(deferred:nextc it
(lambda ()
(message "chain 1")
1))
(deferred:nextc it
(lambda (x)
(message "chain 2 : %s" x)))
(deferred:nextc it
(lambda ()
(read-minibuffer "Input a number: ")))
(deferred:nextc it
(lambda (x)
(message "Got the number : %i" x)))
(deferred:error it
(lambda (err)
(message "Wrong input : %s" err))))
```
* This s-exp returns immediately.
* Asynchronous tasks start subsequently.
* The macro `deferred:$` chains deferred objects.
* The anaphoric variable `it` holds a deferred object in the previous line.
* The next deferred task receives the value that is returned by the previous deferred one.
* Inputting a wrong value, such as alphabets, this s-exp raises an error. The error is caught by the errorback function defined by `deferred:error`.
### Timer ###
After evaluating this s-exp and waiting for 1 second, a message is shown in the minibuffer.
Timer:
```el
(deferred:$
(deferred:wait 1000) ; 1000msec
(deferred:nextc it
(lambda (x)
(message "Timer sample! : %s msec" x))))
```
* The next deferred task subsequent to deferred:wait receives the actual elapse time in millisecond.
### Commands and Sub-process ###
This s-exp inserts the result that is performed by the command `ls -la`. (This s-exp may not run in windows. Try `dir` command.)
Command process:
```el
(deferred:$
(deferred:process "ls" "-la")
(deferred:nextc it
(lambda (x) (insert x))))
```
* This s-exp hardly blocks Emacs because of asynchronous mechanisms.
### HTTP GET : Text ###
This s-exp inserts a text from http://www.gnu.org asynchronously. (You can clear the result with undo command.)
HTTP GET:
```el
(require 'url)
(deferred:$
(deferred:url-retrieve "http://www.gnu.org")
(deferred:nextc it
(lambda (buf)
(insert (with-current-buffer buf (buffer-string)))
(kill-buffer buf))))
```
### HTTP Get : Image ###
This s-exp inserts an image from google asynchronously.
Get an image:
```el
(deferred:$
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png")
(deferred:nextc it
(lambda (buf)
(insert-image
(create-image
(let ((data (with-current-buffer buf (buffer-string))))
(substring data (+ (string-match "\n\n" data) 2)))
'png t))
(kill-buffer buf))))
```
### Parallel ###
This s-exp retrieves two images from google concurrently and wait for the both results. Then, the file sizes of the images are inserted the current buffer.
Parallel deferred:
```el
(deferred:$
(deferred:parallel
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/intl/en_com/images/srpr/logo1w.png"))
(lambda ()
(deferred:url-retrieve "http://www.google.co.jp/images/srpr/nav_logo14.png")))
(deferred:nextc it
(lambda (buffers)
(cl-loop for i in buffers
do
(insert
(format
"size: %s\n"
(with-current-buffer i (length (buffer-string)))))
(kill-buffer i)))))
```
* The function `deferred:parallel` runs asynchronous tasks concurrently.
* The function wait for all results, regardless normal or abnormal. Then, the subsequent tasks are executed.
* The next task receives a list of the results.
* The order of the results is corresponding to one of the argument.
* Giving an alist of tasks as the argument, the results alist is returned.
### Deferred Combination : try-catch-finally ###
This s-exp executes following tasks:
* Getting an image by wget command,
* Resizing the image by convert command in ImageMagick,
* Insert the re-sized image into the current buffer.
You can construct the control structure of deferred tasks, like try-catch-finally in Java.
Get an image by wget and resize by ImageMagick:
```el
(deferred:$
;; try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") 'jpeg nil)))))
;; catch
(deferred:error it ;
(lambda (err)
(insert "Can not get a image! : " err)))
;; finally
(deferred:nextc it
(lambda ()
(deferred:parallel
(lambda () (delete-file "a.jpg"))
(lambda () (delete-file "b.jpg")))))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))
```
* In this case, the deferred tasks are statically connected.
Here is an another sample code for try-catch-finally blocks. This is simpler than above code because of the `deferred:try' macro. (Note: They bring the same results practically, but are not perfectly identical. The `finally` task may not be called because of asynchrony.)
Try-catch-finally:
```el
(deferred:$
(deferred:try
(deferred:$
(deferred:process "wget" "-O" "a.jpg" "http://www.gnu.org/software/emacs/tour/images/splash.png")
(deferred:nextc it
(lambda () (deferred:process "convert" "a.jpg" "-resize" "100x100" "jpg:b.jpg")))
(deferred:nextc it
(lambda ()
(clear-image-cache)
(insert-image (create-image (expand-file-name "b.jpg") `jpeg nil)))))
:catch
(lambda (err) (insert "Can not get a image! : " err))
:finally
(lambda ()
(delete-file "a.jpg")
(delete-file "b.jpg")))
(deferred:nextc it
(lambda (x) (message ">> %s" x))))
```
### Timeout ###
Although a long time command is executed (3 second sleeping), the task is rejected by timeout for 1 second.
The function `deferred:earlier` also runs asynchronous tasks concurrently, however, the next deferred task receives the first result. The other results and tasks will be rejected (canceled or ignored).
Timeout Process:
```el
(deferred:$
(deferred:earlier
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'")
(deferred:$
(deferred:wait 1000) ; timeout msec
(deferred:nextc it (lambda () "canceled!"))))
(deferred:nextc it
(lambda (x) (insert x))))
```
* Changing longer timeout for `deferred:wait`, the next task receives a result of the command.
* When a task finishes abnormally, the task is ignored.
* When all tasks finishes abnormally, the next task receives nil.
* The functions `deferred:parallel` and `deferred:earlier` may be corresponding to `and` and `or`, respectively.
Here is an another sample code for timeout, employing `deferred:timeout` macro.
Timeout macro:
```el
(deferred:$
(deferred:timeout
1000 "canceled!"
(deferred:process "sh" "-c" "sleep 3 | echo 'hello!'"))
(deferred:nextc it
(lambda (x) (insert x))))
```
Note that the `deferred:timeout` and `deferred:earlier` just rejects the task result and does not stop the running task chains. Please see the document for `deferred:cancel`.
### Loop and Animation ###
This s-exp plays an animation at the cursor position for few seconds. Then, you can move cursor freely, because the animation does not block Emacs.
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. (In this case, the interrupt task is dynamically connected.)
Employing a recursive structure of deferred tasks, you can construct a deferred loop.
It may seem the multi-thread in Emacs Lisp.
Loop and animation:
```el
(let ((count 0) (anm "-/|\\-")
(end 50) (pos (point))
(wait-time 50))
(deferred:$
(deferred:next
(lambda (x) (message "Animation started.")))
(deferred:nextc it
(deferred:lambda (x)
(save-excursion
(when (< 0 count)
(goto-char pos) (delete-char 1))
(insert (char-to-string
(aref anm (% count (length anm))))))
(if (> end (cl-incf count)) ; return nil to stop this loop
(deferred:nextc (deferred:wait wait-time) self)))) ; return the deferred
(deferred:nextc it
(lambda (x)
(save-excursion
(goto-char pos) (delete-char 1))
(message "Animation finished.")))))
```
* `deferred:lambda` is an anaphoric macro in which `self` refers itself. It is convenient to construct a recursive structure.
### Wrapping asynchronous function ###
Let's say you have an asynchronous function which takes a callback. For example, dbus.el, xml-rpc.el and websocket.el has such kind of asynchronous APIs. To use such libraries with deferred.el, you can make an unregistered deferred object using `deferred:new` and then start the deferred callback queue using `deferred:callback-post` in the callback given to the asynchronous function. If the asynchronous function supports "errorback", you can use `deferred:errorback-post` to pass the error information to the following callback queue.
In the following example, `run-at-time` is used as an example for the asynchronous function. Deferred.el already has `deferred:wait` for this purpose so that you don't need the following code if you want to use `run-at-time`.
```el
(deferred:$
(deferred:next
(lambda ()
(message "1")
1))
(deferred:nextc it
(lambda (x)
(let ((d (deferred:new #'identity)))
(run-at-time 0 nil (lambda (x)
;; Start the following callback queue now.
(deferred:callback-post d x))
x)
;; Return the unregistered (not yet started) callback
;; queue, so that the following queue will wait until it
;; is started.
d)))
;; You can connect deferred callback queues
(deferred:nextc it
(lambda (x)
(message "%s" (1+ x)))))
```
## API ##
### Functions ###
#### Basic functions ####
* deferred:next (callback)
* Arguments
* callback: a function with zero or one argument
* Return
* a deferred object
* Return a deferred object that wrap the given callback function. Then, put the deferred object into the execution queue to run asynchronously.
* Namely, run the given function asynchronously.
* deferred:nextc (d callback)
* Arguments
* d: a deferred object
* callback: a function with zero or one argument
* Return
* a deferred object
* Return a deferred object that wrap the given callback function. Then, connect the created deferred object with the given deferred object.
* Namely, add the given function to the previous deferred object.
* deferred:error (d errorback)
* Arguments
* d: a deferred object
* errorback: a function with zero or one argument
* Return
* a deferred object
* Return a deferred object that wrap the given function as errorback. Then, connect the created deferred object with the given deferred object.
* Namely, the given function catches the error occurred in the previous task.
* If this function does not throw an error, the subsequent callback functions are executed.
* deferred:cancel (d)
* Arguments
* d: a deferred object
* Return
* the given deferred object (invalidated)
* Invalidate the given deferred object.
* Because this function modifies the deferred object, one can not used the given deferred instance again.
* This function just cancels the given deferred instance, not the whole deferred chain. In the current deferred implementation, a message of cancellation can not propagate to chained deferred objects because the chain is built by the singly linked list. If the deferred chains may be canceled on your code, you should care the side-effect tasks.
* deferred:watch (d callback)
* Arguments
* d: deferred object
* callback: a function with zero or one argument
* Return
* a deferred object
* Create a deferred object with watch task and connect it to the given deferred object.
* The watch task CALLBACK can not affect deferred chains with return values.
* This function is used in following purposes, simulation of try-finally block in asynchronous tasks, monitoring of progress of deferred tasks.
* deferred:wait (msec)
* Arguments
* msec: a number (millisecond)
* Return
* a deferred object
* Return a deferred object that will be called after the specified millisecond.
* The subsequent deferred task receives the actual elapse time in millisecond.
* deferred:$
* Arguments / more than one deferred forms
* Return / the last deferred object
* An anaphoric macro chains deferred objects.
* The anaphoric variable `it` holds a deferred object in the previous line.
#### Utility functions ####
* deferred:loop (number-or-list callback)
* Arguments
* number-or-list: an integer or a list
* callback: a function with zero or one argument
* Return
* a deferred object
* Return a deferred object that iterates the function for the specified times.
* The function receives the count number that begins zero.
* If a list is given, not a number, the function visits each elements in the list like `mapc`.
* deferred:parallel (list-or-alist)
* Arguments
* list-or-alist:
* more than one deferred objects or a list of functions
* an alist consist of cons cells with a symbol and a deferred object or a function
* Return
* a deferred object
* Return a deferred object that executes given functions in parallel and wait for all callback values.
* The subsequent deferred task receives a list of the results. The order of the results is corresponding to one of the argument.
* Giving an alist of tasks as the argument, the results alist is returned.
* If the parallel task throws an error, the error object is passed as a result.
* deferred:earlier (list-or-alist)
* Arguments
* list-or-alist:
* more than one deferred objects or a list of functions
* an alist consist of cons cells with a symbol and a deferred object or a function
* Return
* a deferred object
* Return a deferred object that executes given functions in parallel and wait for the first callback value.
* The other tasks are rejected. (See the document for `deferred:cancel`)
* Giving an alist of tasks as the argument, a cons cell is returned as a result.
* When a task finishes abnormally, the task is ignored.
* When all tasks finishes abnormally, the next task receives nil. That is, no errorback function is called.
#### Wrapper functions ####
* deferred:call (function args...)
* Arguments
* function: a function
* args: arguments (variable length)
* Return
* a deferred object
* a wrapper of the function `funcall`
* deferred:apply (function args)
* Arguments
* function: a function
* args: a list of arguments
* Return
* a deferred object
* a wrapper of the function `apply`
* deferred:process (command args...) / deferred:process-shell (command args...)
* Arguments
* command: command to execute
* args: command arguments (variable length)
* Return
* a deferred object
* Execute a command asynchronously. These functions are wrappers of `start-process` and `start-process-shell-command`.
* The subsequent deferred task receives the stdout and stderr from the command as a string.
* deferred:process-buffer (command args...) / deferred:process-shell-buffer (command args...)
* Arguments
* command: command to execute
* args: command arguments (variable length)
* Return
* a deferred object
* Execute a command asynchronously. These functions are wrappers of `start-process` and `start-process-shell-command`.
* The subsequent deferred task receives the stdout and stderr from the command as a buffer.
* The following tasks are responsible to kill the buffer.
* deferred:wait-idle (msec)
* Arguments
* msec: a number (millisecond)
* Return
* a deferred object
* Return a deferred object that will be called when Emacs has been idle for the specified millisecond.
* The subsequent deferred task receives the elapse time in millisecond.
* deferred:url-retrieve (url [cbargs])
* Arguments
* url: URL to get
* cbargs: callback argument (optional)
* Return
* a deferred object
* A wrapper function of `url-retrieve` in the `url` package.
* The subsequent deferred task receives the content as a buffer.
* The following tasks are responsible to kill the buffer.
* [experimental] deferred:url-get (url [params])
* Arguments
* url: URL to get
* params: alist of parameters
* Return
* a deferred object
* [experimental] deferred:url-post (url [params])
* Arguments
* url: URL to get
* params: alist of parameters
* Return
* a deferred object
#### Primitive functions ####
* deferred:new ([callback])
* Arguments
* callback: a function with zero or one argument (optional)
* Return
* a deferred object
* Create a deferred object
* The created deferred object is never called until someone call the function `deferred:callback` or `deferred:errorback`.
* Using this object, a deferred chain can pause to wait for other events. (See the source for `deferred:wait`.)
* deferred:succeed ([value])
* Arguments
* value: a value (optional)
* Return
* a deferred object
* Create a deferred object that has been called the callback function.
* When a deferred task is connected, the subsequent task will be executed immediately (synchronously).
* deferred:fail ([error])
* Arguments
* error: an error value (optional)
* Return
* a deferred object
* Create a deferred object that has been called the errorback function.
* When a deferred task is connected, the subsequent task will be executed immediately (synchronously).
* deferred:callback (d [value])
* Arguments
* d: a deferred object
* value: a value (optional)
* Return
* a deferred object or a result value
* Start executing the deferred tasks. The first task is executed synchronously.
* deferred:callback-post (d [value])
* Arguments
* d: a deferred object
* value: a value (optional)
* Return
* a deferred object or a result value
* Start executing the deferred tasks. The first task is executed asynchronously.
* deferred:errorback (d [error])
* Arguments
* d: a deferred object
* error: an error value (optional)
* Return
* a deferred object or a result value
* Start executing the deferred tasks from errorback. The first task is executed synchronously.
* deferred:errorback-post (d [error])
* Arguments
* d: a deferred object
* error: an error value (optional)
* Return
* a deferred object or a result value
* Start executing the deferred tasks from errorback. The first task is executed asynchronously.
### Utility Macros ###
* deferred:try (d &key catch finally)
* Arguments
* d: deferred object
* catch: [keyword argument] A function that is called when an error is occurred during tasks `d`. (This function is expanded as an argument of `deferred:error`.)
* finally: [keyword argument] A function that is called when tasks `d` finishes whether in success or failure. (This function is expanded as an argument of deferred:watch.)
* Return
* a deferred object
* Try-catch-finally macro. This macro simulates the try-catch-finally block asynchronously.
* Because of asynchrony, this macro does not ensure that the `finally` task should be called.
* This macro is implemented by `deferred:error` and `deferred:watch`.
* deferred:timeout (msec timeout-form d)
* Arguments
* msec: a number
* timeout-form: sexp-form
* d: a deferred object
* Return
* a deferred object
* Time out macro on a deferred task `d`.
* If the deferred task `d` does not complete within `timeout-msec`, this macro rejects the deferred task and return the `timeout-form`. (See the document for `deferred:cancel`)
* This macro is implemented by `deferred:earlier` and `deferred:wait`.
* deferred:process...
* deferred:processc (d command args...)
* deferred:process-bufferc (d command args...)
* deferred:process-shellc (d command args...)
* deferred:process-shell-bufferc (d command args...)
* Arguments
* d: a deferred object
* command: command to execute
* args: command arguments (variable length)
* Return
* a deferred object
* This macro wraps the deferred:process function in deferred:nextc and connect the given deferred task.
### Execution and Connection ###
#### Firing ####
Some deferred functions can fire a deferred chain implicitly. Following functions register a deferred object with the execution queue to run asynchronously.
* next
* wait
* loop
* parallel
* earlier
* call, apply
* process
* url-retrieve, url-get, url-post
The deferred tasks those are created by `deferred:new` are never called. Using this object, a deferred chain can pause to wait for other events. (See the source for `deferred:wait`.)
One can fire the chain before connecting. That is, deferred objects wait for connecting the subsequent task holding the result value. The functions `deferred:succeed` and `deferred:fail` create those waiting objects.
#### Static connection ####
The `static connection (statically connected)` is a connection between deferred tasks on the source code.
This is a basic usage for the deferred chain.
The static connection is almost equivalent to ordinary callback notation as an argument in the function declarations. The deferred notation is easy to read and write better than the callback one, because the sequence of asynchronous tasks can be written by the deferred notation straightforward.
#### Dynamic Connection ####
Returning a deferred object in the deferred tasks, the returned task is executed before the next deferred one that is statically connected on the source code. This is the `dynamic connection (dynamically connected)`.
Employing a recursive structure of deferred tasks, you can construct higher level control structures, such as loop.
## Discussion ##
Some discussions of writing deferred codes.
### Using lexical scope ###
Using the lexical scope macro, such as `let`, the deferred tasks defined by lambdas can access local variables.
`let` Ex.:
```el
(let ((a (point)))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(goto-char a)
(insert "here!")))))
```
If you write a code of deferred tasks without lexical scope macros, you should be careful with the scopes of each variables.
### Excursion (Current status) ###
The `excursion` functions those hold the current status with the s-exp form, such as `save-execursion` or `with-current-buffer`, are not valid in the deferred tasks, because of execution asynchronously.
Wrong Ex.:
```el
(with-current-buffer (get-buffer "*Message*")
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(insert "Time: %s " x) ; `insert` may not be in the *Message* buffer!
))))
```
In this case, using lexical scope macros to access the buffer variable, you can change the buffer in the deferred task.
Corrected:
```el
(let ((buf (get-buffer "*Message*")))
(deferred:$
(deferred:wait 1000)
(deferred:nextc it
(lambda (x)
(with-current-buffer buf ; Set buffer in the asynchronous task.
(insert "Time: %s " x))))))
```
### Be aware of return values ###
However the dynamic connection is a powerful feature, sometimes it causes bugs of the wrong execution order, because of returning not intended deferred objects.
Then, you should watch the return values of the deferred tasks not to cause an unexpected dynamic connection.
### Debugging ###
The debugging of asynchronous tasks is difficult. Of course, you can use debugger for deferred tasks, but asynchronous tasks cause some troubles, such as interruptions of your debugging and timing gap of simultaneous deferred tasks. Therefore, logging is a safe debugging to observe the tasks correctly, for example, using the `message` function and making custom application log buffer.
If deferred tasks fall into an infinite loop unexpectedly (but Emacs may not freeze), calling the command `deferred:clear-queue`, you can stop the deferred tasks immediately.
If the errors occurred in deferred tasks are caught by no errorback functions, finally the deferred framework catches it and reports to the message buffer. Because the implementation of the framework uses a `condition-case` form, the debugger can not catch the signals normally. If you want to debug the errors in the deferred tasks with the debug-on-error mechanism, set the variable `deferred:debug-on-signal` non-nil.
Wrapping a deferred task in the function `deferred:sync!`, you can wait for the result of the task synchronously. However, the wrapper function should be used for test or debug purpose, because the synchronous waiting is not exact.
### Using macros ###
Writing deferred tasks with `deferred.el`, you may write a lot of `deferred:nextc` and `lambda` to define tasks. Defining a macro, you may write codes shortly. The test code `test-deferred.el` uses many macros to shorten test codes.
On the other hand, using macros to hide `lambda`, it is difficult to realize when the deferred codes are evaluated. That is why `deferred.el` does not provide lot of convenient macros. If you use macros, be careful evaluation timing of deferred forms.
### Introduction for deferred ###
Following documents are good introduction to deferred.
* [Introduction to JSDeferred](http://cho45.stfuawsc.com/jsdeferred/doc/intro.en.html "Introduction to JSDeferred")
* [JSDeferred site](http://cho45.stfuawsc.com/jsdeferred/ "JSDeferred site")
* * * * *
(C) 2010-2016 SAKURAI Masashi All rights reserved.
m.sakurai at kiwanami.net