= Larch

Larch is a tool to copy messages from one IMAP server to another quickly and
safely. It's smart enough not to copy messages that already exist on the
destination and robust enough to deal with interruptions caused by flaky
connections or misbehaving servers.

Larch is particularly well-suited for copying email to, from, or between Gmail
accounts.

*Author*::    Ryan Grove (mailto:ryan@wonko.com)
*Version*::   1.1.2 (2013-01-24)
*Copyright*:: Copyright (c) 2013 Ryan Grove. All rights reserved.
*License*::   GPL 2.0 (http://opensource.org/licenses/gpl-2.0.php)
*Website*::   http://github.com/rgrove/larch

== Installation

Latest stable release:

  gem install larch

Latest development version:

  gem install larch --pre

== Usage

  larch [config section] [options]
  larch --from <uri> --to <uri> [options]

  Server Options:
           --from, -f <s>:   URI of the source IMAP server
    --from-folder, -F <s>:   Source folder to copy from (default: INBOX)
      --from-pass, -p <s>:   Source server password (default: prompt)
      --from-user, -u <s>:   Source server username (default: prompt)
             --to, -t <s>:   URI of the destination IMAP server
      --to-folder, -T <s>:   Destination folder to copy to (default: INBOX)
        --to-pass, -P <s>:   Destination server password (default: prompt)
        --to-user, -U <s>:   Destination server username (default: prompt)

  Copy Options:
                --all, -a:   Copy all folders recursively
     --all-subscribed, -s:   Copy all subscribed folders recursively
             --delete, -d:   Delete messages from the source after copying
                             them, or if they already exist at the destination
           --exclude <s+>:   List of mailbox names/patterns that shouldn't be
                             copied
       --exclude-file <s>:   Filename containing mailbox names/patterns that
                             shouldn't be copied
            --expunge, -x:   Expunge deleted messages from the source
         --sync-flags, -S:   Sync message flags from the source to the
                             destination for messages that already exist at the
                             destination

  General Options:
         --config, -c <s>:   Specify a non-default config file to use (default:
                             ~/.larch/config.yaml)
           --database <s>:   Specify a non-default message database to use
                             (default: ~/.larch/larch.db)
            --dry-run, -n:   Don't actually make any changes
        --max-retries <i>:   Maximum number of times to retry after a
                             recoverable error (default: 3)
       --no-create-folder:   Don't create destination folders that don't
                             already exist
          --ssl-certs <s>:   Path to a trusted certificate bundle to use to
                             verify server SSL certificates
             --ssl-verify:   Verify server SSL certificates
      --verbosity, -V <s>:   Output verbosity: debug, info, warn, error, or
                             fatal (default: info)
            --version, -v:   Print version and exit
               --help, -h:   Show this message

== Usage Examples

Larch is run from the command line. The following examples demonstrate how to
run Larch using only command line arguments, but you may also place these
options in a config file and run Larch without any arguments if you prefer. See
the "Configuration" section below for more details.

For an overview of all available options, run:

  larch -h

At a minimum, you must <b>specify a source server and a destination server</b>
in the form of IMAP URIs:

  larch --from imap://mail.example.com --to imap://imap.gmail.com

Larch will prompt you for the necessary usernames and passwords, then sync the
contents of the source's +INBOX+ folder to the destination's INBOX folder.

To <b>connect using SSL</b>, specify a URI beginning with <tt>imaps://</tt>:

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com

If you'd like to <b>sync a specific folder</b> other than +INBOX+, specify the
source and destination folders using <tt>--from-folder</tt> and
<tt>--to-folder</tt>. Folder names containing spaces must be enclosed in quotes:

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com \
    --from-folder 'Sent Mail' --to-folder 'Sent Mail'

To <b>sync all folders</b>, use the <tt>--all</tt> option (or
<tt>--all-subscribed</tt> if you only want to <b>sync subscribed folders</b>):

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all

By default Larch will create folders on the destination server if they don't
already exist. To prevent this, add the <tt>--no-create-folder</tt> option:

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
    --no-create-folder

You can <b>prevent Larch from syncing one or more folders</b> by using the
<tt>--exclude</tt> option, which accepts multiple arguments:

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
    --exclude Spam Trash Drafts "[Gmail]/*"

If your exclusion list is long or complex, create a text file with one exclusion
pattern per line and tell Larch to load it with the <tt>--exclude-file</tt>
option:

  larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
    --exclude-file exclude.txt

The wildcard characters <tt>*</tt> and <tt>?</tt> are supported in exclusion
lists. You may also use a regular expression by enclosing a pattern in
forward slashes, so the previous example could be achieved with the
pattern <tt>/(Spam|Trash|Drafts|\[Gmail\]\/.*)/</tt>

== Configuration

While it's possible to control Larch entirely from the command line, this can be
inconvenient if you need to specify a lot of options or if you run Larch
frequently and can't always remember which options to use. Using a configuration
file can simplify things.

By default, Larch looks for a config file at <tt>~/.larch/config.yaml</tt> and
uses it if found. You may specify a custom config file using the
<tt>--config</tt> command line option.

The Larch configuration file is a simple YAML[http://yaml.org/] file that may
contain multiple sections, each with a different set of options, as well as a
special +default+ section. The options in the +default+ section will be used
unless they're overridden either in another config section or on the command
line.

=== Example

Here's a sample Larch config file:

  default:
    all-subscribed: true # Copy all subscribed folders by default

  # Copy mail from Gmail to my server, excluding stuff I don't want.
  gmail to my server:
    from: imaps://imap.gmail.com
    from-user: example
    from-pass: secret

    to: imaps://mail.example.com
    to-user: example
    to-pass: secret

    exclude:
      - "[Gmail]/Sent Mail"
      - "[Gmail]/Spam"
      - "[Gmail]/Trash"

  # Copy mail from my INBOX to Gmail's INBOX
  my inbox to gmail inbox:
    all-subscribed: false

    from: imaps://mail.example.com
    from-folder: INBOX
    from-user: example
    from-pass: secret

    to: imaps://imap.gmail.com
    to-folder: INBOX
    to-user: example
    to-pass: secret

This file contains three sections. The options from +default+ will be used in
all other sections as well unless they're overridden.

To specify which config section you want Larch to use, just pass its name on the
command line (use quotes if the name contains spaces):

  larch 'gmail to my server'

If you specify additional command line options, they'll override options in the
config file:

  larch 'gmail to my server' --from-user anotheruser

Running Larch with no command line arguments will cause the +default+ section
to be used. With the example above, this will result in an error since the
+default+ section doesn't contain the required +from+ and +to+ options, but if
you only need to use Larch with a single configuration, you could use the
+default+ section for everything and save yourself some typing on the command
line.

== Server Compatibility

Larch should work well with any server that properly supports
IMAP4rev1[http://tools.ietf.org/html/rfc3501], and does its best to get along
with servers that have buggy, unreliable, or incomplete IMAP implementations.

Larch has been tested on and is known to work well with the following IMAP
servers:

* Dovecot
* Gmail
* Microsoft Exchange 2003

The following servers are known to work, but with caveats:

* Yahoo! Mail

The following servers do not work well with Larch:

* BlitzMail - Buggy server implementation; fails to properly quote or escape
  some IMAP responses, which can cause Net::IMAP to hang waiting for a
  terminating character that will never arrive.

=== Gmail Quirks

Gmail's IMAP implementation is quirky. Larch does its best to work around these
quirks whenever possible, but here are a few things to watch out for:

==== "Some messages could not be FETCHed" error

This error indicates that a message on Gmail is corrupt, and Gmail itself is
unable to read it. The message will continue to show up in the mailbox, but all
attempts to access it via IMAP, POP, or the Gmail web interface will result in
errors. Larch will try to skip these messages and continue processing others
if possible.

It's not clear how this corruption occurs or exactly what kind of corruption
causes these errors, although in every case I'm aware of, the corrupt message
has originated outside of Gmail (Gmail itself does not corrupt the message).
There is currently no known solution for this problem apart from deleting the
corrupted messages.

==== Folder names cannot contain leading or trailing whitespace

Most IMAP servers allow folder names to contain leading and trailing whitespace,
such as "  folder  ". Gmail does not. When copying folders to Gmail, Larch will
automatically remove leading and trailing whitespace in folder names to prevent
errors.

=== Yahoo! Mail Quirks

Yahoo! doesn't officially support IMAP access for general usage, but Larch is
able to connect to imap.mail.yahoo.com and imap-ssl.mail.yahoo.com by using a
fairly well-known trick. That said, as with anything tricky, there are caveats.

==== No hierarchical folders

Similar to Gmail, Yahoo! Mail doesn't allow hierarchical (nested) folders. If
you try to copy a folder hierarchy to Yahoo!, it will work, but you'll end up
with a set of folders named "folder" and "folder.subfolder" rather than seeing
"subfolder" as an actual subfolder of "folder".

==== No custom flags

Yahoo! Mail IMAP doesn't support custom message flags, such as the tags and
junk/not junk flags used by Thunderbird. When transferring messages with custom
flags to a Yahoo! Mail IMAP account, the custom flags will be lost.

==== Here there be dragons

Larch's support for Yahoo! Mail is very new and very lightly tested. Given its
newness and the fact that Yahoo!'s IMAP gateway isn't official, there are likely
to be other quirks we're not yet aware of. There's also no guarantee that Yahoo!
won't shut down its IMAP gateway, deprecate the trick Larch uses to connect, or
just outright block Larch. Use at your own risk.

== Known Issues

* Larch uses Ruby's Net::IMAP standard library for all IMAP operations. While
  Net::IMAP is generally a very solid library, it contains a bug that can
  cause a deadlock to occur if a connection drops unexpectedly (either due to
  network issues or because the server closed the connection without warning)
  when the server has already begun sending a response and Net::IMAP is
  waiting to receive more data. If this happens, Net::IMAP will continue waiting
  forever without passing control back to Larch, and you will need to manually
  kill and restart Larch.

  Net::IMAP in Ruby 1.8 has also been known to hang when it can't parse a server
  response, either because the response itself is malformed or because of a
  bug in Net::IMAP's parser. This is rare, but it happens. Unfortunately there's
  nothing Larch can do about this.

* The Ruby package on Debian, Ubuntu, and some other Debian-based Linux
  distributions doesn't include the OpenSSL standard library. If you see an
  error like <tt>uninitialized constant Larch::IMAP::OpenSSL (NameError)</tt>
  when running Larch, you may need to install the <tt>libopenssl-ruby</tt>
  package. Please feel free to complain to the maintainer of your distribution's
  Ruby packages.

== Support

The Larch mailing list is the best place for questions, comments, and discussion
about Larch. You can join the list or view the archives at
http://groups.google.com/group/larch

First-time senders to the list are moderated to prevent spam, so there may be a
delay before your first message shows up.

== Contributors

Larch was created and is maintained by Ryan Grove <ryan@wonko.com>.

The following lovely people have also contributed to Larch:

* Torey Heinz <torey@ihswebdesign.com>
* Edgardo Hames <ehames@gmail.com>
* Andrew Hobson <ahobson@damballa.com>
* Justin Mazzi <hh@mailheist.com>

== Credit

The Larch::IMAP class borrows heavily from Sup[http://sup.rubyforge.org] by
William Morgan, the source code of which should be required reading if you're
doing anything with IMAP in Ruby.

Larch uses the excellent Trollop[http://trollop.rubyforge.org] command-line
option parser (also by William Morgan) and the
HighLine[http://highline.rubyforge.org] command-line IO library (by James Edward
Gray II).

== License

Copyright (c) 2013 Ryan Grove <ryan@wonko.com>

Licensed under the GNU General Public License version 2.0.

This program is free software; you can redistribute it and/or modify it under
the terms of version 2.0 of the GNU General Public License as published by the
Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.  See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
this program; if not, visit http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt
or write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330,
Boston, MA 02111-1307 USA.