Commit 937f163c9f4e522278b2bfe8495c011079736ec7 - ledger-autosync

New upstream version 1.0.1 Antonio Terceiro 4 years ago

35 changed file(s) with 1674 addition(s) and 1314 deletion(s). Raw diff Collapse all Expand all

-0

.flake8 less more

	0	[flake8]
	1	max-line-length=100⏎

-0

.gitignore less more

	0	venv/
	1	*.pyc
	2	*.egg
	3	.eggs
	4	build/
	5	dist/
	6	ledger_autosync.egg-info

-0

.hgignore less more

	0	venv/
	1	.*pyc
	2	build/
	3	dist/
	4	ledger_autosync.egg-info⏎

+15

-0

.travis.yml less more

	0	dist: xenial
	1	language: python
	2	python:
	3	- "3.5"
	4	- "3.6"
	5	- "3.7"
	6	before_install:
	7	- sudo add-apt-repository ppa:mbudde/ledger -y
	8	- sudo apt-get update
	9	- sudo apt-get install ledger
	10	# - sudo apt-get install hledger
	11	# - cp /usr/lib/python2.7/dist-packages/ledger.so `pip show pip \|grep Location \|awk '{print $2}'`
	12	install: "pip install -r requirements.txt"
	13	# command to run tests
	14	script: nosetests -a generic -a ledger

+59

-0

CHANGELOG.md less more

	0	## Unreleased
	1	- Migrated to python3
	2	- Add --reverse option to print txns in reverse
	3	- Better error handling on OFX server error
	4	- Add --shorten-account, --hardcode-account options
	5	improve user privacy
	6	- Add --payee-format argument
	7	- Move ofxid metadata to correct posting
	8	- Misc bugfixes
	9
	10	## Version 0.3.5
	11	- Disable default usage of python bindings
	12	- Change to using 65 spaces to align txns (per ledger-mode)
	13	- Improve .ledgerrc parsing
	14	- Better error messages
	15	- Add basic plugin system
	16	- Misc bugfixes
	17
	18	## Version 0.3.4
	19	- Packaging fixes
	20
	21	## Version 0.3.3
	22	- Fix problem building on ubuntu trusty
	23
	24	## Version 0.3.2
	25	- Fix problem with certain characters in transaction id
	26
	27	## Version 0.3.0
	28	- Support CSV files (Mint, Paypal and Amazon flavors)
	29	- Uses ticker symbol by for currencies, not CUSIP (You will need to change
	30	previous transactions which used the CUSIP so they work with new transactions)
	31	- Dividends will now be formatted correctly
	32	- Fuzzy payee matching
	33
	34	## Version 0.2.5
	35	- Support advanced investment transactions
	36	- Upgrade to ofxparse 0.15
	37
	38	## Version 0.2.4
	39	- Add `--unknown-account` argument
	40
	41	## Version 0.2.2
	42	- Better support for strange OFX, payee characters
	43
	44	## Version 0.2.1
	45	- Support ledger python API
	46
	47	## Version 0.2.0
	48	- Improved hledger support
	49
	50	## Version 0.1.4
	51	- Balance assertions
	52	- Initial balance
	53
	54	## Version 0.1.3
	55	- Reverse transactions
	56
	57	## Version 0.1.0
	58	- Initial release

-345

~~PKG-INFO~~ less more

0		Metadata-Version: 1.1
1		Name: ledger-autosync
2		Version: 0.3.5
3		Summary: Automatically sync your bank's data with ledger
4		Home-page: https://gitlab.com/egh/ledger-autosync
5		Author: Erik Hetzner
6		Author-email: egh@e6h.org
7		License: GPLv3
8		Description: ledger-autosync
9		===============
10
11		ledger-autosync is a program to pull down transactions from your bank
12		and create `ledger <http://ledger-cli.org/>`__ transactions for them. It
13		is designed to only create transactions that are not already present in
14		your ledger files (that is, deduplicate transactions). This should make
15		it comparable to some of the automated synchronization features
16		available in products like GnuCash, Mint, etc. In fact, ledger-autosync
17		performs OFX import and synchronization better than all the alternatives
18		I have seen.
19
20		Features
21		--------
22
23		- supports `ledger <http://ledger-cli.org/>`__ 3 and
24		`hledger <http://hledger.org/>`__
25		- like ledger, ledger-autosync will never modify your files directly
26		- interactive banking setup via
27		`ofxclient <https://github.com/captin411/ofxclient>`__
28		- multiple banks and accounts
29		- support for non-US currencies
30		- support for 401k and investment accounts
31
32		- tracks investments by share, not dollar value
33		- support for complex transaction types, including transfers, buys,
34		sells, etc.
35
36		- import of downloaded OFX files, for banks not supporting automatic
37		download
38		- import of downloaded CSV files from Paypal, Amazon and Mint
39
40		Platforms
41		---------
42
43		ledger-autosync is developed on Linux with ledger 3 and python 2.7; it has been
44		tested on Windows (although it will run slower) and should run on OS X. It
45		requires ledger 3 or hledger, but it should run faster with ledger, because it
46		will not need to start a command to check every transaction.
47
48		Quickstart
49		----------
50
51		Installation
52		~~~~~~~~~~~~
53
54		If you are on Debian or Ubuntu, an (older) version of ledger-autosync
55		should be available for installation. Try:
56
57		::
58
59		$ sudo apt-get install ledger-autosync
60
61		If you use pip, you can install the latest released version:
62
63		::
64
65		$ pip install ledger-autosync
66
67		You can also install from source, if you have downloaded the source:
68
69		::
70
71		$ python setup.py install
72
73		You may need to install the following libraries (on debian/ubuntu):
74
75		::
76
77		$ sudo apt-get install libffi-dev libpython-dev libssl-dev libxml2-dev python-pip libxslt-dev
78
79		Running
80		~~~~~~~
81
82		Once you have ledger-autosync installed, you can download an OFX file
83		from your bank and run ledger-autosync against it:
84
85		::
86
87		$ ledger-autosync download.ofx
88
89		This should print a number of transactions to stdout. If you add these
90		transactions to your default ledger file (whatever is read when you run
91		``ledger`` without arguments), you should find that if you run
92		ledger-autosync again, it should print no transactions. This is because
93		of the deduplicating feature: only new transactions should be printed
94		for insertion into your ledger files.
95
96		Using the ofx protocol for automatic download
97		~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98
99		ledger-autosync also supports using the OFX protocol to automatically
100		connect to banks and download data. You can use the ofxclient program
101		(which should have been installed with ledger-autosync) to set up
102		banking:
103
104		::
105
106		$ ofxclient
107
108		When you have added your institution, quit ofxclient.
109
110		(At least one user has reported being signed up for a pay service by
111		setting up OFX direct connect. Although this seems unusual, please be
112		aware of this.)
113
114		Edit the generated ``~/ofxclient.ini`` file. Change the ``description``
115		field of your accounts to the name used in ledger. Optionally, move the
116		``~/ofxclient.ini`` file to your ``~/.config`` directory.
117
118		Run:
119
120		::
121
122		ledger-autosync
123
124		This will download a maximum of 90 days previous activity from your
125		accounts. The output will be in ledger format and printed to stdout. Add
126		this output to your ledger file. When that is done, you can call:
127
128		::
129
130		ledger-autosync
131
132		again, and it should print nothing to stdout, because you already have
133		those transactions in your ledger.
134
135		Syncing a file
136		--------------
137
138		Some banks allow users to download OFX files, but do not support
139		fetching via the OFX protocol. If you have an OFX file, you can convert
140		to ledger:
141
142		::
143
144		ledger-autosync /path/to/file.ofx
145
146		This will print unknown transactions in the file to stdout in the same
147		way as ordinary sync. If the transaction is already in your ledger, it
148		will be ignored.
149
150		How it works
151		------------
152
153		ledger-autosync stores a unique identifier, (for OFX files, this is a
154		unique ID provided by your institution for each transaction), as
155		metadata in each transaction. When syncing with your bank, it will check
156		if the transaction exists by running the ledger or hledger command. If
157		the transaction exists, it does nothing. If it does not exist, the
158		transaction is printed to stdout.
159
160		Syncing a CSV file
161		------------------
162
163		If you have a CSV file, you may also be able to import it using a recent
164		(installed via source) version of ledger-autosync. ledger-autosync can
165		currently process CSV files as provided by Paypal, Amazon, or Mint. You
166		can process the CSV file as follows:
167
168		::
169
170		ledger-autosync /path/to/file.csv -a Assets:Paypal
171
172		With Amazon and Paypal CSV files, each row includes a unique identifier,
173		so ledger-autosync will be able to deduplicate against any previously
174		imported entries in your ledger files.
175
176		With Mint, a unique identifier based on the data in the row is generated
177		and stored. If future downloads contain identical rows, they will be
178		deduplicated. This method is probably not as robust as a method based on
179		unique ids, but Mint does not provide a unique id, and it should be
180		better than nothing. It is likely to generate false negatives:
181		transactions that seem new, but are in fact old. It will not generate
182		false negatives: transactions that are not generated because they seem
183		old.
184
185		If you are a developer, you should fine it easy enough to add a new CSV
186		format to ledger-autosync. See, for example, the ``MintConverter`` class
187		in the ``ledgerautosync/converter.py`` file in this repository.
188
189		Assertions
190		----------
191
192		If you supply the ``--assertions`` flag, ledger-autosync will also print
193		out valid ledger assertions based on your bank balances at the time of
194		the sync. These otherwise empty transactions tell ledger that your
195		balance should be something at a given time, and if not, ledger will
196		fail with an error.
197
198		401k and investment accounts
199		----------------------------
200
201		If you have a 401k account, ledger-autosync can help you to track the
202		state of it. You will need OFX files (or an OFX protocol connection as
203		set up by ofxclient) provided by your 401k.
204
205		In general, your 401k account will consist of buy transactions,
206		transfers and reinvestments. The type will be printed in the payee line
207		after a colon (``:``)
208
209		The buy transactions are your contributions to the 401k. These will be
210		printed as follows:
211
212		::
213
214		2016/01/29 401k: buymf
215		; ofxid: 1234
216		Assets:Retirement:401k 1.12345 FOOBAR @ $123.123456
217		Income:Salary -$138.32
218
219		This means that you bought (contributed) $138.32 worth of FOOBAR (your
220		investment fund) at the price of $123.123456. The money to buy the
221		investment came from your income. In ledger-autosync, the
222		``Assets:Retirement:401k`` account is the one specified using the
223		``--account`` command line, or configured in your ``ofxclient.ini``. The
224		``Income:Salary`` is specified by the ``--unknown-account`` option.
225
226		If the transaction is a “transfer” transaction, this usually means
227		either a fee or a change in your investment option:
228
229		::
230
231		2014/06/30 401k: transfer: out
232		; ofxid: 1234
233		Assets:Retirement:401k -1.61374 FOOBAR @ $123.123456
234		Transfer $198.69
235
236		You will need to examine your statements to determine if this was a fee
237		or a real transfer back into your 401k.
238
239		Another type of transaction is a “reinvest” transaction:
240
241		::
242
243		2014/06/30 401k: reinvest
244		; ofxid: 1234
245		Assets:Retirement:401k 0.060702 FOOBAR @ $123.123456
246		Income:Interest -$7.47
247
248		This probably indicates a reinvestment of dividends. ledger-autosync
249		will print ``Income:Interest`` as the other account.
250
251		resync
252		------
253
254		By default, ledger-autosync will process transactions backwards, and
255		stop when it sees a transaction that is already in ledger. To force it
256		to process all transactions up to the ``--max`` days back in time
257		(default: 90), use the ``--resync`` option. This can be useful when
258		increasing the ``--max`` option. For instance, if you previously
259		synchronized 90 days and now want to get 180 days of transactions,
260		ledger-autosync would stop before going back to 180 days without the
261		``--resync`` option.
262
263		python bindings
264		---------------
265
266		If the ledger python bindings are available, ledger-autosync can use them if you
267		pass in the ``--python`` argument.Note, however, they can be buggy, which is why
268		they are disabled by default
269
270		Plugin support (Experimental)
271		-----------------------------
272
273		ledger-autosync has experimental support for plugins. By placing python files a
274		directory named ``~/.config/ledger-autosync/plugins/`` it should be possible to
275		automatically load python files from there. This allows you to extend the csv
276		converters with your own code. For example, given the input CSV file:
277
278		::
279
280		"Date","Name","Amount","Balance"
281		"11/30/2016","Dividend","$1.06","$1,000“
282
283		The following converter in the file ``~/.config/ledger-autosync/plugins/my.py``:
284
285		::
286
287		from ledgerautosync.converter import CsvConverter, Posting, Transaction, Amount
288		import datetime
289		import re
290
291		class SomeConverter(CsvConverter):
292		FIELDSET = set(["Date", "Name", Amount", "Balance"])
293
294		def __init__(self, args, *kwargs):
295		super(SomeConverter, self).__init__(args, *kwargs)
296
297		def convert(self, row):
298		md = re.match(r"^(\(?)\$([0-9,\.]+)", row['Amount'])
299		amount = md.group(2).replace(",", "")
300		if md.group(1) == "(":
301		reverse = True
302		else:
303		reverse = False
304		if reverse:
305		account = 'expenses'
306		else:
307		account = 'income'
308		return Transaction(
309		date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
310		payee=row['Name'],
311		postings=[Posting(self.name, Amount(amount, '$', reverse=reverse)),
312		Posting(account, Amount(amount, '$', reverse=not(reverse)))])
313
314		Running ``ledger-autosync file.csv -a assets:bank`` will generate:
315
316		::
317
318		2016/11/30 Dividend
319		assets:bank $1.06
320		income -$1.06
321
322		For more examples, see
323		https://gitlab.com/egh/ledger-autosync/blob/master/ledgerautosync/converter.py#L421
324
325		Testing
326		-------
327
328		ledger-autosync uses nose for tests. To test, run nosetests in the
329		project directory. This will test the ledger, hledger and ledger-python
330		interfaces. To test a single interface, use nosetests -a hledger. To
331		test the generic code, use nosetests -a generic. To test both, use
332		nosetests -a generic -a hledger. For some reason nosetests -a '!hledger'
333		will not work.
334
335		Keywords: ledger accounting
336		Platform: UNKNOWN
337		Classifier: Development Status :: 5 - Production/Stable
338		Classifier: Intended Audience :: End Users/Desktop
339		Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
340		Classifier: Operating System :: OS Independent
341		Classifier: Programming Language :: Python :: 2.7
342		Classifier: Topic :: Office/Business :: Financial :: Accounting
343		Classifier: Topic :: Office/Business :: Financial :: Investment
344		Classifier: Topic :: Office/Business :: Financial

+154

-92

README.rst less more

0	0	ledger-autosync
1	1	===============
2	2
	3	.. image:: https://travis-ci.org/egh/ledger-autosync.svg?branch=master
	4	:target: https://travis-ci.org/egh/ledger-autosync
	5
3	6	ledger-autosync is a program to pull down transactions from your bank
4		and create `ledger <http://ledger-cli.org/>`__ transactions for them. It
5		is designed to only create transactions that are not already present in
6		your ledger files (that is, deduplicate transactions). This should make
7		it comparable to some of the automated synchronization features
8		available in products like GnuCash, Mint, etc. In fact, ledger-autosync
9		performs OFX import and synchronization better than all the alternatives
10		I have seen.
	7	and create `ledger <http://ledger-cli.org/>`__ transactions for them.
	8	It is designed to only create transactions that are not already
	9	present in your ledger files (that is, it will deduplicate
	10	transactions). This should make it comparable to some of the automated
	11	synchronization features available in products like GnuCash, Mint,
	12	etc. In fact, ledger-autosync performs OFX import and synchronization
	13	better than all the alternatives I have seen.
	14
	15	News
	16	----
	17
	18	v1.0.0
	19	~~~~~~
	20
	21	Versions of ledger-autosync before 1.0.0 printed the ofxid in a
	22	slightly incorrect position. This should not effect usage of the
	23	program, but if you would like to correct the error, see below for
	24	more details.
11	25
12	26	Features
13	27	--------

28	42	- import of downloaded OFX files, for banks not supporting automatic
29	43	download
30	44	- import of downloaded CSV files from Paypal, Amazon and Mint
	45	- any CSV file can be supported via plugins
31	46
32	47	Platforms
33	48	---------
34	49
35		ledger-autosync is developed on Linux with ledger 3 and python 2.7; it has been
36		tested on Windows (although it will run slower) and should run on OS X. It
37		requires ledger 3 or hledger, but it should run faster with ledger, because it
38		will not need to start a command to check every transaction.
	50	ledger-autosync is developed on Linux with ledger 3 and python 3; it
	51	has been tested on Windows (although it will run slower) and should
	52	run on OS X. It requires ledger 3 or hledger, but it should run faster
	53	with ledger, because it will not need to start a command to check
	54	every transaction.
	55
39	56
40	57	Quickstart
41	58	----------

79	96	$ ledger-autosync download.ofx
80	97
81	98	This should print a number of transactions to stdout. If you add these
82		transactions to your default ledger file (whatever is read when you run
83		``ledger`` without arguments), you should find that if you run
84		ledger-autosync again, it should print no transactions. This is because
85		of the deduplicating feature: only new transactions should be printed
86		for insertion into your ledger files.
	99	transactions to your default ledger file (whatever is read when you
	100	run ``ledger`` without arguments), you should find that if you run
	101	ledger-autosync again, it should print no transactions. This is
	102	because of the deduplicating feature: only new transactions will be
	103	printed for insertion into your ledger files.
87	104
88	105	Using the ofx protocol for automatic download
89	106	~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

103	120	setting up OFX direct connect. Although this seems unusual, please be
104	121	aware of this.)
105	122
106		Edit the generated ``~/ofxclient.ini`` file. Change the ``description``
107		field of your accounts to the name used in ledger. Optionally, move the
108		``~/ofxclient.ini`` file to your ``~/.config`` directory.
	123	Edit the generated ``~/ofxclient.ini`` file. Change the
	124	``description`` field of your accounts to the name used in ledger.
	125	Optionally, move the ``~/ofxclient.ini`` file to your ``~/.config``
	126	directory.
109	127
110	128	Run:
111	129

114	132	ledger-autosync
115	133
116	134	This will download a maximum of 90 days previous activity from your
117		accounts. The output will be in ledger format and printed to stdout. Add
118		this output to your ledger file. When that is done, you can call:
	135	accounts. The output will be in ledger format and printed to stdout.
	136	Add this output to your ledger file. When that is done, you can call:
119	137
120	138	::
121	139

124	142	again, and it should print nothing to stdout, because you already have
125	143	those transactions in your ledger.
126	144
127		Syncing a file
128		--------------
129
130		Some banks allow users to download OFX files, but do not support
131		fetching via the OFX protocol. If you have an OFX file, you can convert
132		to ledger:
133
134		::
135
136		ledger-autosync /path/to/file.ofx
137
138		This will print unknown transactions in the file to stdout in the same
139		way as ordinary sync. If the transaction is already in your ledger, it
140		will be ignored.
141
142	145	How it works
143	146	------------
144	147
145		ledger-autosync stores a unique identifier, (for OFX files, this is a
146		unique ID provided by your institution for each transaction), as
147		metadata in each transaction. When syncing with your bank, it will check
148		if the transaction exists by running the ledger or hledger command. If
149		the transaction exists, it does nothing. If it does not exist, the
150		transaction is printed to stdout.
	148	ledger-autosync stores a unique identifier as metadata with each
	149	transaction. (For OFX files, this is a unique ID provided by your
	150	institution for each transaction.) When syncing with your bank, it
	151	will check if the transaction exists by running the ledger or hledger
	152	command. If the transaction exists, it does nothing. If it does not
	153	exist, the transaction is printed to stdout.
	154
	155	ofxid/csvid metadata tag
	156	~~~~~~~~~~~~~~~~~~~~~~~~
	157
	158	ledger-autosync stores a metatag with every posting that it outputs to
	159	support deduplication. This metadata tag is either ``ofxid`` (for OFX
	160	imports) or ``csvid`` for CSV imports.
	161
	162	Pre-1.0.0 versions of ledger-autosync put this metadata tag in a
	163	slightly incorrect place, associating the metadata tag with the
	164	transaction itself, and not simply one posting. This should not effect
	165	the usage of ledger-autosync, but if you would like to correct your
	166	ledger files, there is a small python script ``fix_ofxid.py`` included
	167	with ledger-autosync. It can be run as:
	168
	169	::
	170
	171	python fix_ofxid.py <input file>
	172
	173	and will print a corrected file to stdout.
151	174
152	175	Syncing a CSV file
153	176	------------------
154	177
155		If you have a CSV file, you may also be able to import it using a recent
156		(installed via source) version of ledger-autosync. ledger-autosync can
157		currently process CSV files as provided by Paypal, Amazon, or Mint. You
158		can process the CSV file as follows:
	178	If you have a CSV file, you may also be able to import it using a
	179	recent (installed via source) version of ledger-autosync.
	180	ledger-autosync can currently process CSV files as provided by Paypal,
	181	Amazon, or Mint. You can process the CSV file as follows:
159	182
160	183	::
161	184
162	185	ledger-autosync /path/to/file.csv -a Assets:Paypal
163	186
164		With Amazon and Paypal CSV files, each row includes a unique identifier,
165		so ledger-autosync will be able to deduplicate against any previously
166		imported entries in your ledger files.
167
168		With Mint, a unique identifier based on the data in the row is generated
169		and stored. If future downloads contain identical rows, they will be
170		deduplicated. This method is probably not as robust as a method based on
171		unique ids, but Mint does not provide a unique id, and it should be
172		better than nothing. It is likely to generate false negatives:
173		transactions that seem new, but are in fact old. It will not generate
174		false negatives: transactions that are not generated because they seem
175		old.
176
177		If you are a developer, you should fine it easy enough to add a new CSV
178		format to ledger-autosync. See, for example, the ``MintConverter`` class
179		in the ``ledgerautosync/converter.py`` file in this repository.
	187	With Amazon and Paypal CSV files, each row includes a unique
	188	identifier, so ledger-autosync will be able to deduplicate against any
	189	previously imported entries in your ledger files.
	190
	191	With Mint, a unique identifier based on the data in the row is
	192	generated and stored. If future downloads contain identical rows, they
	193	will be deduplicated. This method is probably not as robust as a
	194	method based on unique ids, but Mint does not provide a unique id, and
	195	it should be better than nothing. It is likely to generate false
	196	negatives: transactions that seem new, but are in fact old. It will
	197	not generate false positives: transactions that are not generated
	198	because they seem old.
	199
	200	If you are a developer, you should fine it easy enough to add a new
	201	CSV format to ledger-autosync. See, for example, the ``MintConverter``
	202	class in the ``ledgerautosync/converter.py`` file in this repository.
	203	See below for how to add these as plugins.
180	204
181	205	Assertions
182	206	----------
183	207
184		If you supply the ``--assertions`` flag, ledger-autosync will also print
185		out valid ledger assertions based on your bank balances at the time of
186		the sync. These otherwise empty transactions tell ledger that your
187		balance should be something at a given time, and if not, ledger will
188		fail with an error.
	208	If you supply the ``--assertions`` flag, ledger-autosync will also
	209	print out valid ledger assertions based on your bank balances at the
	210	time of the sync. These otherwise empty transactions tell ledger that
	211	your balance should be something at a given time, and if not, ledger
	212	will fail with an error.
189	213
190	214	401k and investment accounts
191	215	----------------------------

195	219	set up by ofxclient) provided by your 401k.
196	220
197	221	In general, your 401k account will consist of buy transactions,
198		transfers and reinvestments. The type will be printed in the payee line
199		after a colon (``:``)
	222	transfers and reinvestments. The type will be printed in the payee
	223	line after a colon (``:``)
200	224
201	225	The buy transactions are your contributions to the 401k. These will be
202	226	printed as follows:

204	228	::
205	229
206	230	2016/01/29 401k: buymf
	231	Assets:Retirement:401k 1.12345 FOOBAR @ $123.123456
207	232	; ofxid: 1234
208		Assets:Retirement:401k 1.12345 FOOBAR @ $123.123456
209	233	Income:Salary -$138.32
210	234
211	235	This means that you bought (contributed) $138.32 worth of FOOBAR (your
212	236	investment fund) at the price of $123.123456. The money to buy the
213	237	investment came from your income. In ledger-autosync, the
214	238	``Assets:Retirement:401k`` account is the one specified using the
215		``--account`` command line, or configured in your ``ofxclient.ini``. The
216		``Income:Salary`` is specified by the ``--unknown-account`` option.
	239	``--account`` command line, or configured in your ``ofxclient.ini``.
	240	The ``Income:Salary`` is specified by the ``--unknown-account``
	241	option.
217	242
218	243	If the transaction is a “transfer” transaction, this usually means
219	244	either a fee or a change in your investment option:

221	246	::
222	247
223	248	2014/06/30 401k: transfer: out
	249	Assets:Retirement:401k -1.61374 FOOBAR @ $123.123456
224	250	; ofxid: 1234
225		Assets:Retirement:401k -1.61374 FOOBAR @ $123.123456
226	251	Transfer $198.69
227	252
228		You will need to examine your statements to determine if this was a fee
229		or a real transfer back into your 401k.
	253	You will need to examine your statements to determine if this was a
	254	fee or a real transfer back into your 401k.
230	255
231	256	Another type of transaction is a “reinvest” transaction:
232	257
233	258	::
234	259
235	260	2014/06/30 401k: reinvest
	261	Assets:Retirement:401k 0.060702 FOOBAR @ $123.123456
236	262	; ofxid: 1234
237		Assets:Retirement:401k 0.060702 FOOBAR @ $123.123456
238	263	Income:Interest -$7.47
239	264
240	265	This probably indicates a reinvestment of dividends. ledger-autosync

252	277	ledger-autosync would stop before going back to 180 days without the
253	278	``--resync`` option.
254	279
	280	payee format
	281	------------
	282
	283	By default, ledger-autosync attempts to generate a decent payee line
	284	(the information that follows the date in a ledger transaction).
	285	Unfortunately, because of differences in preference and in the format
	286	of OFX files, it is not always possible to generate the user’s
	287	preferred payee format. ledger-autosync supports a ``payee-format``
	288	option that can be used to generate your preferred payee line. This
	289	option is of the format ``Text {memo}``, where ``memo`` is a
	290	substitution based on the value of the transaction. Available
	291	substitutions are ``memo``, ``payee``, ``txntype``, ``account`` and
	292	``tferaction``. For example:
	293
	294	::
	295
	296	$ ledger-autosync --payee-format "Memo: {memo}"
	297	2011/03/31 Memo: DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	298
	299	This option is also available for CSV conversion. For CSV files, you
	300	can substitution any of the values of the rows in the CSV file by
	301	name. For instance, for Paypal files:
	302
	303	::
	304
	305	$ ledger-autosync --payee-format "{Name} ({To Email Address})" -a Paypal paypal.csv
	306	2016/06/04 Jane Doe (someone@example.net)
	307
255	308	python bindings
256	309	---------------
257	310
258		If the ledger python bindings are available, ledger-autosync can use them if you
259		pass in the ``--python`` argument.Note, however, they can be buggy, which is why
260		they are disabled by default
261
262		Plugin support (Experimental)
263		-----------------------------
264
265		ledger-autosync has experimental support for plugins. By placing python files a
266		directory named ``~/.config/ledger-autosync/plugins/`` it should be possible to
267		automatically load python files from there. This allows you to extend the csv
268		converters with your own code. For example, given the input CSV file:
	311	If the ledger python bindings are available, ledger-autosync can use
	312	them if you pass in the ``--python`` argument. Note, however, they can
	313	be buggy, which is why they are disabled by default
	314
	315	Plugin support
	316	--------------
	317
	318	ledger-autosync has support for plugins. By placing python files a
	319	directory named ``~/.config/ledger-autosync/plugins/`` it should be
	320	possible to automatically load python files from there. You may place
	321	``CsvCconverter`` subclasses here, which will be selected based on the
	322	columns in the CSV file being parsed and the FIELDSET of the CSV
	323	converters. You may also place a single ``OfxConverter`` in the plugin
	324	directory, which will be used in place of the stock ``OfxConverter``.
	325
	326	Below is an example CSV converter, starting with the input CSV file:
269	327
270	328	::
271	329

313	371
314	372	For more examples, see
315	373	https://gitlab.com/egh/ledger-autosync/blob/master/ledgerautosync/converter.py#L421
	374	or the `example plugins directory <examples/plugins>`_.
	375
	376	If you develop a converter that you think will be generally
	377	useful, please consider submitting a pull request.
316	378
317	379	Testing
318	380	-------
319	381
320	382	ledger-autosync uses nose for tests. To test, run nosetests in the
321		project directory. This will test the ledger, hledger and ledger-python
322		interfaces. To test a single interface, use nosetests -a hledger. To
323		test the generic code, use nosetests -a generic. To test both, use
324		nosetests -a generic -a hledger. For some reason nosetests -a '!hledger'
325		will not work.
	383	project directory. This will test the ledger, hledger and
	384	ledger-python interfaces. To test a single interface, use nosetests -a
	385	hledger. To test the generic code, use nosetests -a generic. To test
	386	both, use nosetests -a generic -a hledger. For some reason
	387	nosetests -a '!hledger' will not work.

+24

-0

examples/plugins/firstdirect.py less more

	0	# ledger-autosync plugin for CSV files from First Direct, a UK bank.
	1	# The currency is fixed to GBP for that reason.
	2
	3	from ledgerautosync.converter import CsvConverter, Posting, Transaction, Amount
	4	import datetime
	5	import re
	6
	7	class SomeConverter(CsvConverter):
	8	FIELDSET = set(["Date","Description","Amount","Balance"])
	9
	10	def __init__(self, args, *kwargs):
	11	super(SomeConverter, self).__init__(args, *kwargs)
	12
	13	def convert(self, row):
	14	amount = row['Amount']
	15	if amount.startswith('-'):
	16	reverse = True
	17	else:
	18	reverse = False
	19	return Transaction(
	20	date=datetime.datetime.strptime(row['Date'], "%d/%m/%Y"),
	21	payee=row['Description'],
	22	postings=[Posting(self.name, Amount(amount, 'GBP', reverse=reverse)),
	23	Posting(self.unknownaccount, Amount(amount, 'GBP', reverse=not(reverse)))])

+27

-0

fix_ofxid.py less more

	0	#!/usr/bin/python
	1	import re
	2	import sys
	3
	4	first_line = False
	5	ofxline = None
	6
	7	with open(sys.argv[1]) as f:
	8	for line in f.readlines():
	9	md = re.match(r"^(19\|20)[0-9][0-9]", line)
	10	if md is not None:
	11	# Mark the next line as the first line in a txn
	12	first_line = True
	13	else:
	14	if first_line:
	15	first_line = False
	16	# Check if there is an ofxid on this line
	17	md = re.match(r"^\s+; ofxid:", line)
	18	if md is not None:
	19	ofxline = line
	20	continue
	21	# In every case except the one above where we call next, print the line
	22	sys.stdout.write(line)
	23	# We had a misplaced ofxid last, print it now
	24	if ofxline:
	25	sys.stdout.write(ofxline)
	26	ofxline = None

-0

fixtures/amazon2.csv less more

	0	Order Date,Order ID,Title,Category,ASIN/ISBN,UNSPSC Code,Website,Release Date,Condition,Seller,Seller Credentials,List Price Per Unit,Purchase Price Per Unit,Quantity,Payment Instrument Type,Purchase Order Number,PO Line Number,Ordering Customer Email,Shipment Date,Shipping Address Name,Shipping Address Street 1,Shipping Address Street 2,Shipping Address City,Shipping Address State,Shipping Address Zip,Order Status,Carrier Name & Tracking Number,Item Subtotal,Item Subtotal Tax,Item Total,Tax Exemption Applied,Tax Exemption Type,Exemption Opt-Out,Buyer Name,Currency,Group Name
	1	06/05/17,111-1111111-1111111,"Test "" double quote",Kitchen,"A00AA1A11A",,Amazon.com,,new,,,$0.00,$9.99,1,"MasterCard - 1234",,,me@example.com,06/06/17,Me,123 Main St,,Springfield,CA,00000-0000,Shipped,AMZN_US(TBA000000000000),$9.99,$0.00,$9.99,,,,Me,USD,

+66

-0

fixtures/checking-payee-match.ofx less more

	0	OFXHEADER:100
	1	DATA:OFXSGML
	2	VERSION:102
	3	SECURITY:NONE
	4	ENCODING:USASCII
	5	CHARSET:1252
	6	COMPRESSION:NONE
	7	OLDFILEUID:NONE
	8	NEWFILEUID:NONE
	9
	10	<OFX>
	11	<SIGNONMSGSRSV1>
	12	<SONRS>
	13	<STATUS>
	14	<CODE>0
	15	<SEVERITY>INFO
	16	</STATUS>
	17	<DTSERVER>20130525225731.258
	18	<LANGUAGE>ENG
	19	<DTPROFUP>20050531060000.000
	20	<FI>
	21	<ORG>FAKE
	22	<FID>1101
	23	</FI>
	24	<INTU.BID>51123
	25	<INTU.USERID>9774652
	26	</SONRS>
	27	</SIGNONMSGSRSV1>
	28	<BANKMSGSRSV1>
	29	<STMTTRNRS>
	30	<TRNUID>0
	31	<STATUS>
	32	<CODE>0
	33	<SEVERITY>INFO
	34	</STATUS>
	35	<STMTRS>
	36	<CURDEF>USD
	37	<BANKACCTFROM>
	38	<BANKID>5472369148
	39	<ACCTID>1452687~7
	40	<ACCTTYPE>CHECKING
	41	</BANKACCTFROM>
	42	<BANKTRANLIST>
	43	<DTSTART>20000101070000.000
	44	<DTEND>20130525060000.000
	45	<STMTTRN>
	46	<TRNTYPE>DEBIT
	47	<DTPOSTED>20110331120000.000
	48	<TRNAMT>-0.01
	49	<FITID>0000489
	50	<NAME>Payment to MATCH PAYEE and so on and so forth
	51	<MEMO>
	52	</STMTTRN>
	53	</BANKTRANLIST>
	54	<LEDGERBAL>
	55	<BALAMT>100.99
	56	<DTASOF>20130525225731.258
	57	</LEDGERBAL>
	58	<AVAILBAL>
	59	<BALAMT>75.99
	60	<DTASOF>20130525225731.258
	61	</AVAILBAL>
	62	</STMTRS>
	63	</STMTTRNRS>
	64	</BANKMSGSRSV1>
	65	</OFX>⏎

-0

fixtures/checking.lgr less more

49	49	2011/03/31 PAYEE TEST"QUOTE
50	50	Assets:Foo $0.01
51	51	Income:Bar -$0.01
	52
	53	2011/03/31 Match Payee
	54	; AutosyncPayee: Payment to MATCH PAYEE and so on and so forth
	55	Assets:Foo -$0.01
	56	Expenses:Bar $0.01

+72

-0

fixtures/comments.ofx less more

	0	OFXHEADER:100
	1	DATA:OFXSGML
	2	VERSION:102
	3	SECURITY:NONE
	4	ENCODING:USASCII
	5	CHARSET:1252
	6	COMPRESSION:NONE
	7	OLDFILEUID:NONE
	8	NEWFILEUID:NONE
	9
	10	<OFX>
	11	<SIGNONMSGSRSV1>
	12	<SONRS>
	13	<STATUS>
	14	<CODE>0
	15	<SEVERITY>INFO
	16	</STATUS>
	17	<DTSERVER>20130525225731.258
	18	<LANGUAGE>ENG
	19	<DTPROFUP>20050531060000.000
	20	<FI>
	21	<ORG>FAKE
	22	<FID>1101
	23	</FI>
	24	<INTU.BID>51123
	25	<INTU.USERID>9774652
	26	</SONRS>
	27	</SIGNONMSGSRSV1>
	28	<BANKMSGSRSV1>
	29	<STMTTRNRS>
	30	<TRNUID>0
	31	<STATUS>
	32	<CODE>0
	33	<SEVERITY>INFO
	34	</STATUS>
	35	<STMTRS>
	36	<CURDEF>USD
	37	<BANKACCTFROM>
	38	<BANKID>5472369148
	39	<ACCTID>1452687~7
	40	<ACCTTYPE>CHECKING
	41	</BANKACCTFROM>
	42	<BANKTRANLIST>
	43	<DTSTART>20000101070000.000
	44	<DTEND>20161027160000.000
	45	<STMTTRN>
	46	<TRNTYPE>CREDIT
	47	<DTPOSTED>20161027160000.000
	48	<TRNAMT>123.45
	49	<FITID>FITID20161027123.45ABCDE
	50	<NAME>AMAZON MKTPLACE PMTS AMZN.COM/BI
	51	</STMTTRN>
	52	<STMTTRN>
	53	<TRNTYPE>DEBIT
	54	<DTPOSTED>20161027160000.000
	55	<TRNAMT>-0
	56	<FITID>FITID20161027-0.0ABCDE
	57	<NAME>XXXEXTRASTUFFXXX
	58	</STMTTRN>
	59	</BANKTRANLIST>
	60	<LEDGERBAL>
	61	<BALAMT>100.99
	62	<DTASOF>20161027160000.000
	63	</LEDGERBAL>
	64	<AVAILBAL>
	65	<BALAMT>75.99
	66	<DTASOF>20161027160000.000
	67	</AVAILBAL>
	68	</STMTRS>
	69	</STMTTRNRS>
	70	</BANKMSGSRSV1>
	71	</OFX>

-1

fixtures/paypal.csv less more

0	0	Date, Time, Time Zone, Name, Type, Status, Currency, Gross, Fee, Net, From Email Address, To Email Address, Transaction ID, Counterparty Status, Shipping Address, Address Status, Item Title, Item ID, Shipping and Handling Amount, Insurance Amount, Sales Tax, Option 1 Name, Option 1 Value, Option 2 Name, Option 2 Value, Auction Site, Buyer ID, Item URL, Closing Date, Escrow Id, Invoice Id, Reference Txn ID, Invoice Number, Custom Number, Receipt ID, Balance, Contact Phone Number,
1	1	"6/4/2016","10:46:49","PDT","Jane Doe","Recurring Payment Sent","Completed","USD","-20.00","0.00","-20.00","me@example.com","someone@example.net","XYZ1","Verified","John Doe, 123 Main St, Springfield, XX 00000, United States","Confirmed","My Friend","friend","","","","","","","","","","","","","","","","","","0.00","",
2		"6/4/2016","10:46:49","PDT","Debit Card","Charge From Debit Card","Completed","USD","20.00","0.00","20.00","","","XYZ2","","","","","","","","","","","","","","","","","","","XYZ3","","","","20.00","",
	2	"6/4/2016","10:46:49","PDT","Debit Card","Charge From Debit Card","Completed","USD","1,120.00","0.00","1,120.00","","","XYZ2","","","","","","","","","","","","","","","","","","","XYZ3","","","","20.00","",

-0

fixtures/paypal_alternate.csv less more

	0	Date, Time, Time Zone, Name, Type, Status, Amount, Receipt ID, Balance,
	1	"12/31/2016","23:59:59","PST","Some User","Payment Sent","Completed","-12.34","","0.00",
	2	"12/31/2016","23:59:59","PST","Bank Account","Add Funds from a Bank Account","Completed","12.34","","12.34",

-345

~~ledger_autosync.egg-info/PKG-INFO~~ less more

0		Metadata-Version: 1.1
1		Name: ledger-autosync
2		Version: 0.3.5
3		Summary: Automatically sync your bank's data with ledger
4		Home-page: https://gitlab.com/egh/ledger-autosync
5		Author: Erik Hetzner
6		Author-email: egh@e6h.org
7		License: GPLv3
8		Description: ledger-autosync
9		===============
10
11		ledger-autosync is a program to pull down transactions from your bank
12		and create `ledger <http://ledger-cli.org/>`__ transactions for them. It
13		is designed to only create transactions that are not already present in
14		your ledger files (that is, deduplicate transactions). This should make
15		it comparable to some of the automated synchronization features
16		available in products like GnuCash, Mint, etc. In fact, ledger-autosync
17		performs OFX import and synchronization better than all the alternatives
18		I have seen.
19
20		Features
21		--------
22
23		- supports `ledger <http://ledger-cli.org/>`__ 3 and
24		`hledger <http://hledger.org/>`__
25		- like ledger, ledger-autosync will never modify your files directly
26		- interactive banking setup via
27		`ofxclient <https://github.com/captin411/ofxclient>`__
28		- multiple banks and accounts
29		- support for non-US currencies
30		- support for 401k and investment accounts
31
32		- tracks investments by share, not dollar value
33		- support for complex transaction types, including transfers, buys,
34		sells, etc.
35
36		- import of downloaded OFX files, for banks not supporting automatic
37		download
38		- import of downloaded CSV files from Paypal, Amazon and Mint
39
40		Platforms
41		---------
42
43		ledger-autosync is developed on Linux with ledger 3 and python 2.7; it has been
44		tested on Windows (although it will run slower) and should run on OS X. It
45		requires ledger 3 or hledger, but it should run faster with ledger, because it
46		will not need to start a command to check every transaction.
47
48		Quickstart
49		----------
50
51		Installation
52		~~~~~~~~~~~~
53
54		If you are on Debian or Ubuntu, an (older) version of ledger-autosync
55		should be available for installation. Try:
56
57		::
58
59		$ sudo apt-get install ledger-autosync
60
61		If you use pip, you can install the latest released version:
62
63		::
64
65		$ pip install ledger-autosync
66
67		You can also install from source, if you have downloaded the source:
68
69		::
70
71		$ python setup.py install
72
73		You may need to install the following libraries (on debian/ubuntu):
74
75		::
76
77		$ sudo apt-get install libffi-dev libpython-dev libssl-dev libxml2-dev python-pip libxslt-dev
78
79		Running
80		~~~~~~~
81
82		Once you have ledger-autosync installed, you can download an OFX file
83		from your bank and run ledger-autosync against it:
84
85		::
86
87		$ ledger-autosync download.ofx
88
89		This should print a number of transactions to stdout. If you add these
90		transactions to your default ledger file (whatever is read when you run
91		``ledger`` without arguments), you should find that if you run
92		ledger-autosync again, it should print no transactions. This is because
93		of the deduplicating feature: only new transactions should be printed
94		for insertion into your ledger files.
95
96		Using the ofx protocol for automatic download
97		~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98
99		ledger-autosync also supports using the OFX protocol to automatically
100		connect to banks and download data. You can use the ofxclient program
101		(which should have been installed with ledger-autosync) to set up
102		banking:
103
104		::
105
106		$ ofxclient
107
108		When you have added your institution, quit ofxclient.
109
110		(At least one user has reported being signed up for a pay service by
111		setting up OFX direct connect. Although this seems unusual, please be
112		aware of this.)
113
114		Edit the generated ``~/ofxclient.ini`` file. Change the ``description``
115		field of your accounts to the name used in ledger. Optionally, move the
116		``~/ofxclient.ini`` file to your ``~/.config`` directory.
117
118		Run:
119
120		::
121
122		ledger-autosync
123
124		This will download a maximum of 90 days previous activity from your
125		accounts. The output will be in ledger format and printed to stdout. Add
126		this output to your ledger file. When that is done, you can call:
127
128		::
129
130		ledger-autosync
131
132		again, and it should print nothing to stdout, because you already have
133		those transactions in your ledger.
134
135		Syncing a file
136		--------------
137
138		Some banks allow users to download OFX files, but do not support
139		fetching via the OFX protocol. If you have an OFX file, you can convert
140		to ledger:
141
142		::
143
144		ledger-autosync /path/to/file.ofx
145
146		This will print unknown transactions in the file to stdout in the same
147		way as ordinary sync. If the transaction is already in your ledger, it
148		will be ignored.
149
150		How it works
151		------------
152
153		ledger-autosync stores a unique identifier, (for OFX files, this is a
154		unique ID provided by your institution for each transaction), as
155		metadata in each transaction. When syncing with your bank, it will check
156		if the transaction exists by running the ledger or hledger command. If
157		the transaction exists, it does nothing. If it does not exist, the
158		transaction is printed to stdout.
159
160		Syncing a CSV file
161		------------------
162
163		If you have a CSV file, you may also be able to import it using a recent
164		(installed via source) version of ledger-autosync. ledger-autosync can
165		currently process CSV files as provided by Paypal, Amazon, or Mint. You
166		can process the CSV file as follows:
167
168		::
169
170		ledger-autosync /path/to/file.csv -a Assets:Paypal
171
172		With Amazon and Paypal CSV files, each row includes a unique identifier,
173		so ledger-autosync will be able to deduplicate against any previously
174		imported entries in your ledger files.
175
176		With Mint, a unique identifier based on the data in the row is generated
177		and stored. If future downloads contain identical rows, they will be
178		deduplicated. This method is probably not as robust as a method based on
179		unique ids, but Mint does not provide a unique id, and it should be
180		better than nothing. It is likely to generate false negatives:
181		transactions that seem new, but are in fact old. It will not generate
182		false negatives: transactions that are not generated because they seem
183		old.
184
185		If you are a developer, you should fine it easy enough to add a new CSV
186		format to ledger-autosync. See, for example, the ``MintConverter`` class
187		in the ``ledgerautosync/converter.py`` file in this repository.
188
189		Assertions
190		----------
191
192		If you supply the ``--assertions`` flag, ledger-autosync will also print
193		out valid ledger assertions based on your bank balances at the time of
194		the sync. These otherwise empty transactions tell ledger that your
195		balance should be something at a given time, and if not, ledger will
196		fail with an error.
197
198		401k and investment accounts
199		----------------------------
200
201		If you have a 401k account, ledger-autosync can help you to track the
202		state of it. You will need OFX files (or an OFX protocol connection as
203		set up by ofxclient) provided by your 401k.
204
205		In general, your 401k account will consist of buy transactions,
206		transfers and reinvestments. The type will be printed in the payee line
207		after a colon (``:``)
208
209		The buy transactions are your contributions to the 401k. These will be
210		printed as follows:
211
212		::
213
214		2016/01/29 401k: buymf
215		; ofxid: 1234
216		Assets:Retirement:401k 1.12345 FOOBAR @ $123.123456
217		Income:Salary -$138.32
218
219		This means that you bought (contributed) $138.32 worth of FOOBAR (your
220		investment fund) at the price of $123.123456. The money to buy the
221		investment came from your income. In ledger-autosync, the
222		``Assets:Retirement:401k`` account is the one specified using the
223		``--account`` command line, or configured in your ``ofxclient.ini``. The
224		``Income:Salary`` is specified by the ``--unknown-account`` option.
225
226		If the transaction is a “transfer” transaction, this usually means
227		either a fee or a change in your investment option:
228
229		::
230
231		2014/06/30 401k: transfer: out
232		; ofxid: 1234
233		Assets:Retirement:401k -1.61374 FOOBAR @ $123.123456
234		Transfer $198.69
235
236		You will need to examine your statements to determine if this was a fee
237		or a real transfer back into your 401k.
238
239		Another type of transaction is a “reinvest” transaction:
240
241		::
242
243		2014/06/30 401k: reinvest
244		; ofxid: 1234
245		Assets:Retirement:401k 0.060702 FOOBAR @ $123.123456
246		Income:Interest -$7.47
247
248		This probably indicates a reinvestment of dividends. ledger-autosync
249		will print ``Income:Interest`` as the other account.
250
251		resync
252		------
253
254		By default, ledger-autosync will process transactions backwards, and
255		stop when it sees a transaction that is already in ledger. To force it
256		to process all transactions up to the ``--max`` days back in time
257		(default: 90), use the ``--resync`` option. This can be useful when
258		increasing the ``--max`` option. For instance, if you previously
259		synchronized 90 days and now want to get 180 days of transactions,
260		ledger-autosync would stop before going back to 180 days without the
261		``--resync`` option.
262
263		python bindings
264		---------------
265
266		If the ledger python bindings are available, ledger-autosync can use them if you
267		pass in the ``--python`` argument.Note, however, they can be buggy, which is why
268		they are disabled by default
269
270		Plugin support (Experimental)
271		-----------------------------
272
273		ledger-autosync has experimental support for plugins. By placing python files a
274		directory named ``~/.config/ledger-autosync/plugins/`` it should be possible to
275		automatically load python files from there. This allows you to extend the csv
276		converters with your own code. For example, given the input CSV file:
277
278		::
279
280		"Date","Name","Amount","Balance"
281		"11/30/2016","Dividend","$1.06","$1,000“
282
283		The following converter in the file ``~/.config/ledger-autosync/plugins/my.py``:
284
285		::
286
287		from ledgerautosync.converter import CsvConverter, Posting, Transaction, Amount
288		import datetime
289		import re
290
291		class SomeConverter(CsvConverter):
292		FIELDSET = set(["Date", "Name", Amount", "Balance"])
293
294		def __init__(self, args, *kwargs):
295		super(SomeConverter, self).__init__(args, *kwargs)
296
297		def convert(self, row):
298		md = re.match(r"^(\(?)\$([0-9,\.]+)", row['Amount'])
299		amount = md.group(2).replace(",", "")
300		if md.group(1) == "(":
301		reverse = True
302		else:
303		reverse = False
304		if reverse:
305		account = 'expenses'
306		else:
307		account = 'income'
308		return Transaction(
309		date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
310		payee=row['Name'],
311		postings=[Posting(self.name, Amount(amount, '$', reverse=reverse)),
312		Posting(account, Amount(amount, '$', reverse=not(reverse)))])
313
314		Running ``ledger-autosync file.csv -a assets:bank`` will generate:
315
316		::
317
318		2016/11/30 Dividend
319		assets:bank $1.06
320		income -$1.06
321
322		For more examples, see
323		https://gitlab.com/egh/ledger-autosync/blob/master/ledgerautosync/converter.py#L421
324
325		Testing
326		-------
327
328		ledger-autosync uses nose for tests. To test, run nosetests in the
329		project directory. This will test the ledger, hledger and ledger-python
330		interfaces. To test a single interface, use nosetests -a hledger. To
331		test the generic code, use nosetests -a generic. To test both, use
332		nosetests -a generic -a hledger. For some reason nosetests -a '!hledger'
333		will not work.
334
335		Keywords: ledger accounting
336		Platform: UNKNOWN
337		Classifier: Development Status :: 5 - Production/Stable
338		Classifier: Intended Audience :: End Users/Desktop
339		Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
340		Classifier: Operating System :: OS Independent
341		Classifier: Programming Language :: Python :: 2.7
342		Classifier: Topic :: Office/Business :: Financial :: Accounting
343		Classifier: Topic :: Office/Business :: Financial :: Investment
344		Classifier: Topic :: Office/Business :: Financial

-41

~~ledger_autosync.egg-info/SOURCES.txt~~ less more

0		LICENSE
1		README.rst
2		setup.py
3		fixtures/amazon.csv
4		fixtures/apostrophe.ofx
5		fixtures/checking-dynamic-account.lgr
6		fixtures/checking-partial.lgr
7		fixtures/checking.lgr
8		fixtures/checking.ofx
9		fixtures/checking_order.ofx
10		fixtures/cusip.ofx
11		fixtures/empty.lgr
12		fixtures/fidelity-one-dtsettle.ofx
13		fixtures/fidelity.ofx
14		fixtures/income.ofx
15		fixtures/investment_401k.ofx
16		fixtures/mint.csv
17		fixtures/multiple.lgr
18		fixtures/no-institution.ofx
19		fixtures/ofxclient.ini
20		fixtures/paypal.csv
21		fixtures/paypal.lgr
22		ledger_autosync.egg-info/PKG-INFO
23		ledger_autosync.egg-info/SOURCES.txt
24		ledger_autosync.egg-info/dependency_links.txt
25		ledger_autosync.egg-info/entry_points.txt
26		ledger_autosync.egg-info/requires.txt
27		ledger_autosync.egg-info/top_level.txt
28		ledgerautosync/__init__.py
29		ledgerautosync/cli.py
30		ledgerautosync/converter.py
31		ledgerautosync/ledgerwrap.py
32		ledgerautosync/plugins.py
33		ledgerautosync/sync.py
34		tests/__init__.py
35		tests/test_cli.py
36		tests/test_converter.py
37		tests/test_ledger.py
38		tests/test_ofx_formatter.py
39		tests/test_sync.py
40		tests/test_weird_ofx.py⏎

-1

~~ledger_autosync.egg-info/dependency_links.txt~~ less more

-4

~~ledger_autosync.egg-info/entry_points.txt~~ less more

0		[console_scripts]
1		hledger-autosync = ledgerautosync.cli:run
2		ledger-autosync = ledgerautosync.cli:run
3

-9

~~ledger_autosync.egg-info/requires.txt~~ less more

0		setuptools>=26
1		ofxclient
2		ofxparse>=0.14
3		BeautifulSoup4
4		fuzzywuzzy
5
6		[test]
7		nose>=1.0
8		mock

-1

~~ledger_autosync.egg-info/top_level.txt~~ less more

ledgerautosync

-0

ledgerautosync/__init__.py less more

23	23	def __str__(self):
24	24	return repr(self.value)
25	25
	26
26	27	class LedgerAutosyncException(Exception):
27	28	def __init__(self, value):
28	29	self.value = value

+148

-47

ledgerautosync/cli.py less more

18	18	# along with ledger-autosync. If not, see
19	19	# <http://www.gnu.org/licenses/>.
20	20
21		from __future__ import absolute_import
	21
22	22	from ofxclient.config import OfxConfig
23	23	import argparse
24		import csv
25		from ledgerautosync import EmptyInstitutionException, LedgerAutosyncException
26		from ledgerautosync.converter import OfxConverter, CsvConverter, AUTOSYNC_INITIAL, \
27		ALL_AUTOSYNC_INITIAL
	24	from ledgerautosync import LedgerAutosyncException
	25	from ledgerautosync.converter import OfxConverter, AUTOSYNC_INITIAL, \
	26	ALL_AUTOSYNC_INITIAL, UNKNOWN_BANK_ACCOUNT
28	27	from ledgerautosync.converter import SecurityList
29	28	from ledgerautosync.sync import OfxSynchronizer, CsvSynchronizer
30	29	from ledgerautosync.ledgerwrap import mk_ledger, Ledger, HLedger, LedgerPython

69	68	if (not(ledger.check_transaction_by_id
70	69	("ofxid", converter.mk_ofxid(AUTOSYNC_INITIAL))) and
71	70	not(ledger.check_transaction_by_id("ofxid", ALL_AUTOSYNC_INITIAL))):
72		print converter.format_initial_balance(ofx.account.statement)
	71	print(converter.format_initial_balance(ofx.account.statement))
73	72	for txn in txns:
74		print converter.convert(txn).format(args.indent)
	73	print(converter.convert(txn).format(args.indent))
75	74	if args.assertions:
76		print converter.format_balance(ofx.account.statement)
	75	print(converter.format_balance(ofx.account.statement))
77	76
78	77	# if OFX has positions use these to obtain commodity prices
79	78	# and print "P" records to provide dated/timed valuations

81	80	# not your position (e.g. # shares), even though this is in the OFX record
82	81	if hasattr(ofx.account.statement, 'positions'):
83	82	for pos in ofx.account.statement.positions:
84		print converter.format_position(pos)
	83	print(converter.format_position(pos))
	84
	85	def make_ofx_converter(account,
	86	name,
	87	ledger,
	88	indent,
	89	fid,
	90	unknownaccount,
	91	payee_format,
	92	hardcodeaccount,
	93	shortenaccount,
	94	security_list):
	95	klasses = OfxConverter.__subclasses__()
	96	if len(klasses) > 1:
	97	raise Exception("I found more than 1 OfxConverter subclass, but only "
	98	"know how to handle 1. Remove extra subclasses from "
	99	"the plugin directory")
	100	elif len(klasses) == 1:
	101	return klasses[0](account=account,
	102	name=name,
	103	ledger=ledger,
	104	indent=indent,
	105	fid=fid,
	106	unknownaccount=unknownaccount,
	107	payee_format=payee_format,
	108	hardcodeaccount=hardcodeaccount,
	109	shortenaccount=shortenaccount,
	110	security_list=security_list)
	111	else:
	112	return OfxConverter(account=account,
	113	name=name,
	114	ledger=ledger,
	115	indent=indent,
	116	fid=fid,
	117	unknownaccount=unknownaccount,
	118	payee_format=payee_format,
	119	hardcodeaccount=hardcodeaccount,
	120	shortenaccount=shortenaccount,
	121	security_list=security_list)
85	122
86	123	def sync(ledger, accounts, args):
87		sync = OfxSynchronizer(ledger)
	124	sync = OfxSynchronizer(ledger, shortenaccount=args.shortenaccount)
88	125	for acct in accounts:
89	126	try:
90	127	(ofx, txns) = sync.get_new_txns(acct, resync=args.resync,
91	128	max_days=args.max)
92	129	if ofx is not None:
93		converter = OfxConverter(ofx=ofx,
94		name=acct.description,
95		ledger=ledger,
96		indent=args.indent,
97		unknownaccount=args.unknownaccount)
	130	converter = make_ofx_converter(account=ofx.account,
	131	name=acct.description,
	132	ledger=ledger,
	133	indent=args.indent,
	134	fid=None,
	135	unknownaccount=args.unknownaccount,
	136	payee_format=args.payee_format,
	137	hardcodeaccount=None,
	138	shortenaccount=args.shortenaccount,
	139	security_list=SecurityList(ofx))
98	140	print_results(converter, ofx, ledger, txns, args)
99	141	except KeyboardInterrupt:
100	142	raise
101		except:
102		sys.stderr.write("Caught exception processing %s" %
	143	except BaseException:
	144	sys.stderr.write("Caught exception processing %s\n" %
103	145	(acct.description))
104	146	traceback.print_exc(file=sys.stderr)
105	147
106	148
107	149	def import_ofx(ledger, args):
108		sync = OfxSynchronizer(ledger)
109		(ofx, txns) = sync.parse_file(args.PATH)
	150	sync = OfxSynchronizer(ledger, hardcodeaccount=args.hardcodeaccount,
	151	shortenaccount=args.shortenaccount)
	152	ofx = OfxSynchronizer.parse_file(args.PATH)
	153	txns = sync.filter(
	154	ofx.account.statement.transactions,
	155	ofx.account.account_id)
110	156	accountname = args.account
111	157	if accountname is None:
112	158	if ofx.account.institution is not None:
113	159	accountname = "%s:%s" % (ofx.account.institution.organization,
114	160	ofx.account.account_id)
115	161	else:
116		raise EmptyInstitutionException("Institution provided by OFX is \
117		empty and no accountname supplied!")
118
119		converter = OfxConverter(ofx=ofx,
120		name=accountname,
121		ledger=ledger,
122		indent=args.indent,
123		fid=args.fid,
124		unknownaccount=args.unknownaccount)
125
	162	accountname = UNKNOWN_BANK_ACCOUNT
	163
	164	# build SecurityList (including indexing by CUSIP and ticker symbol)
	165	security_list = SecurityList(ofx)
	166
	167	converter = make_ofx_converter(account=ofx.account,
	168	name=accountname,
	169	ledger=ledger,
	170	indent=args.indent,
	171	fid=args.fid,
	172	unknownaccount=args.unknownaccount,
	173	payee_format=args.payee_format,
	174	hardcodeaccount=args.hardcodeaccount,
	175	shortenaccount=args.shortenaccount,
	176	security_list=security_list)
126	177	print_results(converter, ofx, ledger, txns, args)
127	178
128	179
129	180	def import_csv(ledger, args):
130	181	if args.account is None:
131		raise Exception("When importing a CSV file, you must specify an account name.")
132		sync = CsvSynchronizer(ledger)
133		accountname = args.account
134		for txn in sync.parse_file(args.PATH, accountname=args.account):
135		print txn.format(args.indent)
	182	raise Exception(
	183	"When importing a CSV file, you must specify an account name.")
	184	sync = CsvSynchronizer(ledger, payee_format=args.payee_format)
	185	txns = sync.parse_file(args.PATH, accountname=args.account,
	186	unknownaccount=args.unknownaccount)
	187	if args.reverse:
	188	txns = reversed(txns)
	189	for txn in txns:
	190	print(txn.format(args.indent, args.assertions))
	191
136	192
137	193	def load_plugins(config_dir):
138	194	plugin_dir = os.path.join(config_dir, 'ledger-autosync', 'plugins')
139	195	if os.path.isdir(plugin_dir):
140		for plugin in filter(re.compile('.py$', re.IGNORECASE).search, os.listdir(plugin_dir)):
	196	for plugin in filter(
	197	re.compile(
	198	'.py$',
	199	re.IGNORECASE).search,
	200	os.listdir(plugin_dir)):
141	201	# Quiet loader
142		import ledgerautosync.plugins
	202	import ledgerautosync.plugins # noqa: F401
143	203	path = os.path.join(plugin_dir, plugin)
144		imp.load_source('ledgerautosync.plugins.%s'%(os.path.splitext(plugin)[0]), path)
	204	imp.load_source(
	205	'ledgerautosync.plugins.%s' %
	206	(os.path.splitext(plugin)[0]), path)
	207
145	208
146	209	def run(args=None, config=None):
147	210	if args is None:

159	222	if importing from file, set account name for import')
160	223	parser.add_argument('-l', '--ledger', type=str, default=None,
161	224	help='specify ledger file to READ for syncing')
162		parser.add_argument('-L', dest='no_ledger', action='store_true', default=False,
163		help='do not de-duplicate against a ledger file')
	225	parser.add_argument(
	226	'-L',
	227	'--no-ledger',
	228	dest='no_ledger',
	229	action='store_true',
	230	default=False,
	231	help='do not de-duplicate against a ledger file')
164	232	parser.add_argument('-i', '--indent', type=int, default=4,
165	233	help='number of spaces to use for indentation')
166	234	parser.add_argument('--initial', action='store_true', default=False,

168	236	parser.add_argument('--fid', type=int, default=None,
169	237	help='pass in fid value for OFX files that do not \
170	238	supply it')
	239	parser.add_argument(
	240	'--hardcode-account',
	241	type=str,
	242	default=None,
	243	dest='hardcodeaccount',
	244	help='pass in hardcoded account number for OFX files \
	245	to maintain ledger files without real account numbers')
	246	parser.add_argument(
	247	'--shorten-account',
	248	default=False,
	249	action='store_true',
	250	dest='shortenaccount',
	251	help='shorten all account numbers to last 4 digits \
	252	to maintain ledger files without full account numbers')
171	253	parser.add_argument('--unknown-account', type=str, dest='unknownaccount',
172	254	default=None,
173	255	help='specify account name to use when one can\'t be \

178	260	help='enable debug logging')
179	261	parser.add_argument('--hledger', action='store_true', default=False,
180	262	help='force use of hledger (on by default if invoked \
181		as hledger-autosync)')
	263	as hledger-autosync)')
	264	parser.add_argument(
	265	'--payee-format',
	266	type=str,
	267	default=None,
	268	dest='payee_format',
	269	help="""Format string to use for generating the payee line. Substitutions
	270	can be written using {memo}, {payee}, {txntype}, {account} or
	271	{tferaction} for OFX. If the input file is a CSV file,
	272	substitutions are written using the CSV file column names
	273	between {}.""")
182	274	parser.add_argument('--python', action='store_true', default=False,
183	275	help='use the ledger python interface')
184	276	parser.add_argument('--slow', action='store_true', default=False,

188	280	help='display which version of ledger (cli), hledger, \
189	281	or ledger (python) will be used by ledger-autosync to check for previous \
190	282	transactions')
	283	parser.add_argument('--reverse', action='store_true', default=False,
	284	help='print CSV transactions in reverse order')
	285	parser.add_argument('-o', '--ofxconfig', type=str, default=None,
	286	help='specify config file for ofxclient')
191	287	args = parser.parse_args(args)
192	288	if sys.argv[0][-16:] == "hledger-autosync":
193	289	args.hledger = True
194	290
195	291	ledger_file = None
196	292	if args.ledger and args.no_ledger:
197		raise LedgerAutosyncException('You cannot specify a ledger file and -L')
	293	raise LedgerAutosyncException(
	294	'You cannot specify a ledger file and -L')
198	295	elif args.ledger:
199	296	ledger_file = args.ledger
200	297	else:

205	302	if ledger_file is None:
206	303	sys.stderr.write("LEDGER_FILE environment variable not set, and no \
207	304	.ledgerrc file found, and -l argument was not supplied: running with deduplication disabled. \
208		All transactions will be printed!")
	305	All transactions will be printed!\n")
209	306	ledger = None
210	307	elif args.no_ledger:
211	308	ledger = None

220	317
221	318	if args.which:
222	319	sys.stderr.write("ledger-autosync is using ")
223		if type(ledger) == Ledger:
	320	if isinstance(ledger, Ledger):
224	321	sys.stderr.write("ledger (cli)\n")
225		elif type(ledger) == HLedger:
	322	elif isinstance(ledger, HLedger):
226	323	sys.stderr.write("hledger\n")
227		elif type(ledger) == LedgerPython:
	324	elif isinstance(ledger, LedgerPython):
228	325	sys.stderr.write("ledger.so (python)\n")
229	326	exit()
230	327

236	333
237	334	if args.PATH is None:
238	335	if config is None:
239		config_file = os.path.join(config_dir, 'ofxclient.ini')
	336	if args.ofxconfig is None:
	337	config_file = os.path.join(config_dir, 'ofxclient.ini')
	338	else:
	339	config_file = args.ofxconfig
240	340	if (os.path.exists(config_file)):
241	341	config = OfxConfig(file_name=config_file)
242	342	else:

247	347	if acct.description == args.account]
248	348	sync(ledger, accounts, args)
249	349	else:
250		_, file_extension = os.path.splitext(args.PATH)
	350	_, file_extension = os.path.splitext(args.PATH.lower())
251	351	if file_extension == '.csv':
252	352	import_csv(ledger, args)
253	353	else:
254	354	import_ofx(ledger, args)
255	355
	356
256	357	if __name__ == '__main__':
257	358	run()

+422

-151

ledgerautosync/converter.py less more

16	16	# along with ledger-autosync. If not, see
17	17	# <http://www.gnu.org/licenses/>.
18	18
19		from __future__ import absolute_import
	19
20	20	from decimal import Decimal
21	21	import re
22	22	from ofxparse.ofxparse import Transaction as OfxTransaction, InvestmentTransaction

26	26
27	27	AUTOSYNC_INITIAL = "autosync_initial"
28	28	ALL_AUTOSYNC_INITIAL = "all.%s" % (AUTOSYNC_INITIAL)
	29	UNKNOWN_BANK_ACCOUNT = "Assets:Unknown"
	30
	31
	32	class EasyEquality(object):
	33	def __eq__(self, other):
	34	if isinstance(other, self.__class__):
	35	return self.__dict__ == other.__dict__
	36	else:
	37	return False
	38
	39	def __ne__(self, other):
	40	return not self.__eq__(other)
	41
29	42
30	43	class SecurityList(object):
31	44	"""

39	52	It is iterable, and also provides lookup table (LUT) functionality
40	53	provides __next__() for Py3
41	54	"""
42		def __init__(self, securities):
	55
	56	def __init__(self, ofx):
	57	securities = []
	58	if hasattr(ofx, 'security_list') and ofx.security_list is not None:
	59	securities = ofx.security_list
	60
43	61	self.cusip_lut = dict()
44	62	self.ticker_lut = dict()
45	63
46	64	self._iter = iter(securities)
47	65	self.securities = securities
48		if len(securities) == 0: return
	66	if len(securities) == 0:
	67	return
49	68
50	69	# index
51	70	for sec in securities:
52	71	# unfortunately OFXparse does not currently implement
53	72	# security.uniqueid_type so I am presuming here
54		if sec.uniqueid: self.cusip_lut[sec.uniqueid] = sec
55		if sec.ticker: self.ticker_lut[sec.ticker] = sec
	73	if sec.uniqueid:
	74	self.cusip_lut[sec.uniqueid] = sec
	75	if sec.ticker:
	76	self.ticker_lut[sec.ticker] = sec
56	77	# This indexing strategy (whereby I index the object instead of
57	78	# the inverse value (e.g. ticker symbol) directly has a flaw
58	79	# in that an OFX file could define a security list section and

61	82	def __iter__(self):
62	83	return self
63	84
64		def __next__(self): # Py3 iterable
65		return next(self._iter)
66
67		def next(self): # Python 2
	85	def __next__(self):
68	86	return next(self._iter)
69	87
70	88	def __len__(self):

75	93	# I'll have no idea if what I am seeing is a CUSISP
76	94	# unless I look it up specifically as a CUSIP (and it exists)
77	95	def find_cusip(self, cusip):
78		if cusip in self.cusip_lut: return self.cusip_lut[cusip]
79		else: return None
	96	if cusip in self.cusip_lut:
	97	return self.cusip_lut[cusip]
	98	else:
	99	return None
80	100
81	101	def find_ticker(self, ticker):
82		if ticker in self.ticker_lut: return self.ticker_lut[ticker]
83		else: return None
	102	if ticker in self.ticker_lut:
	103	return self.ticker_lut[ticker]
	104	else:
	105	return None
84	106
85	107
86	108	class Transaction(object):
87		def __init__(self, date, payee, postings, cleared=False, metadata={}, aux_date=None):
	109	def __init__(
	110	self,
	111	date,
	112	payee,
	113	postings,
	114	cleared=False,
	115	metadata={},
	116	aux_date=None):
88	117	self.date = date
89	118	self.aux_date = aux_date
90	119	self.payee = payee

92	121	self.metadata = metadata
93	122	self.cleared = cleared
94	123
95		def format(self, indent=4):
	124	def format(self, indent=4, assertions=True):
96	125	retval = ""
97	126	cleared_str = " "
98	127	if self.cleared:
99	128	cleared_str = " * "
100	129	aux_date_str = ""
101	130	if self.aux_date is not None:
102		aux_date_str = "=%s"%(self.aux_date.strftime("%Y/%m/%d"))
103		retval += "%s%s%s%s\n"%(self.date.strftime("%Y/%m/%d"), aux_date_str, cleared_str, self.payee)
104		for k,v in self.metadata.iteritems():
105		retval += "%s; %s: %s\n" % (" "*indent, k, v)
	131	aux_date_str = "=%s" % (self.aux_date.strftime("%Y/%m/%d"))
	132	retval += "%s%s%s%s\n" % (self.date.strftime("%Y/%m/%d"),
	133	aux_date_str, cleared_str, self.payee)
	134	for k in sorted(self.metadata.keys()):
	135	retval += "%s; %s: %s\n" % (" " * indent, k, self.metadata[k])
106	136	for posting in self.postings:
107		retval += posting.format(indent)
	137	retval += posting.format(indent, assertions)
108	138	return retval
109	139
	140
110	141	class Posting(object):
111		def __init__(self, account, amount, asserted=None, unit_price=None):
	142	def __init__(
	143	self,
	144	account,
	145	amount,
	146	asserted=None,
	147	unit_price=None,
	148	metadata={}):
112	149	self.account = account
113	150	self.amount = amount
114	151	self.asserted = asserted
115	152	self.unit_price = unit_price
116
117		def format(self, indent=4):
118		space_count = 65 - indent - len(self.account) - len(self.amount.format())
	153	self.metadata = metadata
	154
	155	def format(self, indent=4, assertions=True):
	156	space_count = 65 - indent - \
	157	len(self.account) - len(self.amount.format())
119	158	if space_count < 2:
120	159	space_count = 2
121		retval = "%s%s%s%s" % (
122		" " * indent, self.account, " "*space_count, self.amount.format())
123		if self.asserted is not None:
124		retval = "%s = %s"%(retval, self.asserted.format())
	160	retval = "%s%s%s%s" % (" " * indent,
	161	self.account,
	162	" " * space_count,
	163	self.amount.format())
	164	if assertions and self.asserted is not None:
	165	retval = "%s = %s" % (retval, self.asserted.format())
125	166	if self.unit_price is not None:
126		retval = "%s @ %s"%(retval, self.unit_price.format())
127		return "%s\n"%(retval)
128
129		class Amount(object):
	167	retval = "%s @ %s" % (retval, self.unit_price.format())
	168	retval += "\n"
	169	for k in sorted(self.metadata.keys()):
	170	retval += "%s; %s: %s\n" % (" " * indent, k, self.metadata[k])
	171	return retval
	172
	173	def clone_inverted(self, account, asserted=None, metadata={}):
	174	return Posting(account,
	175	self.amount.clone_inverted(),
	176	asserted=asserted,
	177	unit_price=self.unit_price,
	178	metadata=metadata)
	179
	180	class Amount(EasyEquality):
130	181	def __init__(self, number, currency, reverse=False, unlimited=False):
131	182	self.number = Decimal(number)
132	183	self.reverse = reverse

155	206	# USD comes after
156	207	return "%s%s %s" % (prefix, number, currency)
157	208
	209	def clone_inverted(self):
	210	return Amount(self.number,
	211	self.currency,
	212	reverse=not(self.reverse),
	213	unlimited=self.unlimited)
158	214
159	215	class Converter(object):
160	216	@staticmethod

164	220	replace(' ', '_').\
165	221	replace('@', '_').\
166	222	replace('*', '_').\
	223	replace('+', '_').\
167	224	replace('[', '_').\
168	225	replace(']', '_')
169	226
170		def __init__(self, ledger=None, unknownaccount=None, currency='$', indent=4):
	227	def __init__(
	228	self,
	229	ledger=None,
	230	unknownaccount=None,
	231	currency='$',
	232	indent=4,
	233	payee_format=None):
171	234	self.lgr = ledger
172	235	self.indent = indent
173	236	self.unknownaccount = unknownaccount
174	237	self.currency = currency.upper()
	238	self.payee_format = payee_format
175	239	if self.currency == "USD":
176	240	self.currency = "$"
177	241

187	251
188	252
189	253	class OfxConverter(Converter):
190		def __init__(self, ofx, name, indent=4, ledger=None, fid=None,
191		unknownaccount=None):
	254	def __init__(
	255	self,
	256	account,
	257	name,
	258	indent=4,
	259	ledger=None,
	260	fid=None,
	261	unknownaccount=None,
	262	payee_format=None,
	263	hardcodeaccount=None,
	264	shortenaccount=False,
	265	security_list=SecurityList(
	266	[])):
192	267	super(OfxConverter, self).__init__(ledger=ledger,
193	268	indent=indent,
194	269	unknownaccount=unknownaccount,
195		currency=ofx.account.statement.currency)
196		self.acctid = ofx.account.account_id
197		# build SecurityList (including indexing by CUSIP and ticker symbol)
198		if hasattr(ofx, 'security_list') and ofx.security_list is not None:
199		self.security_list = SecurityList(ofx.security_list)
200		else:
201		self.security_list = SecurityList([])
	270	currency=account.statement.currency,
	271	payee_format=payee_format)
	272	self.real_acctid = account.account_id
	273	if hardcodeaccount is not None:
	274	self.acctid = hardcodeaccount
	275	elif shortenaccount:
	276	self.acctid = account.account_id[-4:]
	277	else:
	278	self.acctid = account.account_id
	279	self.payee_format = payee_format
	280	self.security_list = security_list
202	281
203	282	if fid is not None:
204	283	self.fid = fid
205	284	else:
206		if ofx.account.institution is None:
	285	if account.institution is None:
207	286	raise EmptyInstitutionException(
208	287	"Institution provided by OFX is empty and no fid supplied!")
209	288	else:
210		self.fid = ofx.account.institution.fid
	289	self.fid = account.institution.fid
211	290	self.name = name
212	291
213	292	def mk_ofxid(self, txnid):
	293	if self.acctid != self.real_acctid:
	294	# Some banks insert the bank account number into the transaction ID
	295	# We will do this to properly hide the account number for privacy
	296	# reasons
	297	txnid = txnid.replace(self.real_acctid, self.acctid)
214	298	return Converter.clean_id("%s.%s.%s" % (self.fid, self.acctid, txnid))
215	299
216	300	def format_payee(self, txn):
217		payee = None
218		memo = None
219		if (hasattr(txn, 'payee')):
	301	payee = ""
	302	if (hasattr(txn, 'payee') and txn.payee is not None):
220	303	payee = txn.payee
221		if (hasattr(txn, 'memo')):
	304	memo = ""
	305	if (hasattr(txn, 'memo') and txn.memo is not None):
222	306	memo = txn.memo
223
224		if (payee is None or payee == '') and (memo is None or memo == ''):
225		retval = "%s: %s"%(self.name, txn.type)
226		if txn.type == 'transfer' and hasattr(txn, 'tferaction'):
227		retval += ": %s"%(txn.tferaction.lower())
228		return retval
229		if (payee is None or payee == '') or txn.memo.startswith(payee):
230		return memo
231		elif (memo is None or memo == '') or payee.startswith(memo):
232		return payee
233		else:
234		return "%s %s" % (payee, memo)
	307	txntype = ""
	308	if (hasattr(txn, 'type') and txn.type is not None):
	309	txntype = txn.type
	310	tferaction = ""
	311	if (hasattr(txn, 'tferaction') and txn.tferaction is not None):
	312	tferaction = txn.tferaction.lower()
	313
	314	if payee != "" and self.lgr is not None:
	315	payee = self.lgr.get_autosync_payee(payee, self.name)
	316
	317	payee_format = self.payee_format
	318
	319	if payee_format is None:
	320	# Default when not provided
	321	payee_format = "{payee} {memo}"
	322
	323	# Alternate for when payee and memo are blank, sometimes true for
	324	# investment accounts.
	325	if (payee == "") and (memo == ""):
	326	payee_format = "{account}: {txntype}"
	327	if tferaction != "":
	328	payee_format += ": {tferaction}"
	329
	330	# Sometimes memo/payee are simply longer versions of the other.
	331	if memo.startswith(payee):
	332	payee = ""
	333	if payee.startswith(memo):
	334	memo = ""
	335
	336	return payee_format.format(
	337	payee=payee,
	338	memo=memo,
	339	txntype=txntype,
	340	account=self.name,
	341	tferaction=tferaction).strip()
235	342
236	343	def format_balance(self, statement):
237	344	# Get date. Ensure the date is a date-like object.
238	345	if (hasattr(statement, 'balance_date') and
239		hasattr(statement.balance_date, 'strftime')):
	346	hasattr(statement.balance_date, 'strftime')):
240	347	date = statement.balance_date
241	348	elif (hasattr(statement, 'end_date') and
242	349	hasattr(statement.end_date, 'strftime')):

262	369	initbal = statement.balance
263	370	for txn in statement.transactions:
264	371	initbal -= txn.amount
	372
	373	posting = Posting(
	374	self.name,
	375	Amount(
	376	initbal,
	377	currency=self.currency),
	378	metadata={
	379	"ofxid": self.mk_ofxid(AUTOSYNC_INITIAL)})
	380
265	381	return Transaction(
266	382	date=statement.start_date,
267	383	payee="--Autosync Initial Balance",
268	384	cleared=True,
269	385	postings=[
270		Posting(
271		self.name,
272		Amount(initbal, currency=self.currency)).format(self.indent),
273		Posting(
274		"Assets:Equity",
275		Amount(initbal, currency=self.currency, reverse=True)).format(self.indent)
276		],
277		metadata={ "ofxid": self.mk_ofxid(AUTOSYNC_INITIAL) }
278		).format(self.indent)
	386	posting,
	387	posting.clone_inverted("Assets:Equity").format(self.indent)],
	388	).format(
	389	self.indent)
279	390	else:
280	391	return ""
281	392

294	405	"""
295	406
296	407	ofxid = self.mk_ofxid(txn.id)
	408	metadata = {}
	409	posting_metadata = {"ofxid": ofxid}
297	410
298	411	if isinstance(txn, OfxTransaction):
	412	posting = Posting(self.name,
	413	Amount(txn.amount, self.currency),
	414	metadata=posting_metadata)
299	415	return Transaction(
300	416	date=txn.date,
301	417	payee=self.format_payee(txn),
302		metadata={"ofxid": ofxid},
303	418	postings=[
304		Posting(
305		self.name,
306		Amount(txn.amount, self.currency)
307		),
308		Posting(
309		self.mk_dynamic_account(self.format_payee(txn), exclude=self.name),
310		Amount(txn.amount, self.currency, reverse=True)
311		)]
312		)
	419	posting,
	420	posting.clone_inverted(
	421	self.mk_dynamic_account(self.format_payee(txn),
	422	exclude=self.name))])
313	423	elif isinstance(txn, InvestmentTransaction):
314	424	acct1 = self.name
315	425	acct2 = self.name

317	427	posting1 = None
318	428	posting2 = None
319	429
320		metadata = {"ofxid": ofxid}
321
322	430	security = self.maybe_get_ticker(txn.security)
323	431
324		if isinstance(txn.type, basestring):
	432	if isinstance(txn.type, str):
325	433	# recent versions of ofxparse
326	434	if re.match('^(buy\|sell)', txn.type):
327	435	acct2 = self.unknownaccount or 'Assets:Unknown'

336	444	# type: income, income_type: DIV
337	445	# TODO: determine how dividend income is listed from other institutions
338	446	# income/DIV transactions do not involve buying or selling a security
339		# so their postings need special handling compared to others
	447	# so their postings need special handling compared to
	448	# others
340	449	metadata['dividend_from'] = security
341	450	acct2 = 'Income:Dividends'
342		posting1 = Posting( acct1,
343		Amount(txn.total, self.currency))
344		posting2 = Posting( acct2,
345		Amount(txn.total, self.currency, reverse=True ))
	451	posting1 = Posting(acct1,
	452	Amount(txn.total, self.currency),
	453	metadata=posting_metadata)
	454	posting2 = posting1.clone_inverted(acct2)
346	455	else:
347	456	# ???
348	457	pass

366	475	# income/DIV already defined above;
367	476	# this block defines all other posting types
368	477	if posting1 is None and posting2 is None:
369		posting1 = Posting(acct1,
370		Amount(txn.units, security, unlimited=True),
371		unit_price=Amount(txn.unit_price, self.currency, unlimited=True))
372		posting2 = Posting(acct2,
373		Amount(txn.units * txn.unit_price, self.currency, reverse=True))
	478	posting1 = Posting(
	479	acct1,
	480	Amount(
	481	txn.units,
	482	security,
	483	unlimited=True),
	484	unit_price=Amount(
	485	txn.unit_price,
	486	self.currency,
	487	unlimited=True),
	488	metadata=posting_metadata)
	489	posting2 = Posting(
	490	acct2,
	491	Amount(
	492	txn.units *
	493	txn.unit_price,
	494	self.currency,
	495	reverse=True))
374	496	else:
375	497	# Previously defined if type:income income_type/DIV
376	498	pass
377	499
378	500	return Transaction(
379	501	date=txn.tradeDate,
380		aux_date=txn.settleDate,
	502	aux_date=aux_date,
381	503	payee=self.format_payee(txn),
382	504	metadata=metadata,
383		postings=[ posting1, posting2 ]
	505	postings=[posting1, posting2]
384	506	)
385	507
386	508	def format_position(self, pos):
387	509	if hasattr(pos, 'date') and hasattr(pos, 'security') and \
388	510	hasattr(pos, 'unit_price'):
389	511	dateStr = pos.date.strftime("%Y/%m/%d %H:%M:%S")
390		return "P %s %s %s\n" % (dateStr, self.maybe_get_ticker(pos.security), pos.unit_price)
	512	return "P %s %s %s\n" % (
	513	dateStr, self.maybe_get_ticker(pos.security), pos.unit_price)
391	514
392	515
393	516	class CsvConverter(Converter):
394	517	@staticmethod
395		def make_converter(csv, name=None, **kwargs):
396		fieldset = set(csv.fieldnames)
397		for klass in CsvConverter.__subclasses__():
	518	def make_converter(fieldset, dialect, name=None, **kwargs):
	519	for klass in CsvConverter.descendants():
398	520	if klass.FIELDSET <= fieldset:
399		return klass(csv, name=name, **kwargs)
	521	return klass(dialect, name=name, **kwargs)
400	522	# Found no class, bail
401	523	raise Exception('Cannot determine CSV type')
	524
	525	@classmethod
	526	def descendants(cls):
	527	retval = cls.__subclasses__()
	528	for cls2 in cls.__subclasses__():
	529	retval.extend(cls2.descendants())
	530	return retval
402	531
403	532	# By default, return an MD5 of the key-value pairs in the row.
404	533	# If a better ID is available, should be overridden.
405	534	def get_csv_id(self, row):
406	535	h = hashlib.md5()
407	536	for key in sorted(row.keys()):
408		h.update("%s=%s\n"%(key, row[key]))
	537	h.update(("%s=%s\n" % (key, row[key])).encode('utf-8'))
409	538	return h.hexdigest()
410	539
411		def __init__(self, csv, name=None, indent=4, ledger=None, unknownaccount=None):
	540	def __init__(
	541	self,
	542	dialect,
	543	name=None,
	544	indent=4,
	545	ledger=None,
	546	unknownaccount=None,
	547	payee_format=None):
412	548	super(CsvConverter, self).__init__(
413		ledger=ledger,
414	549	indent=indent,
415		unknownaccount=unknownaccount)
	550	unknownaccount=unknownaccount,
	551	payee_format=payee_format)
416	552	self.name = name
417		self.csv = csv
	553	self.dialect = dialect
	554
	555	def format_payee(self, row):
	556	return re.sub(r"\s+", " ",
	557	self.payee_format.format(**row).strip())
418	558
419	559
420	560	class PaypalConverter(CsvConverter):
421		FIELDSET = set(['Currency', 'Date', 'Gross', 'Item Title', 'Name', 'Net', 'Status', 'To Email Address', 'Transaction ID', 'Type'])
	561	FIELDSET = {
	562	'Currency',
	563	'Date',
	564	'Gross',
	565	'Item Title',
	566	'Name',
	567	'Net',
	568	'Status',
	569	'To Email Address',
	570	'Transaction ID',
	571	'Type'}
422	572
423	573	def __init__(self, args, *kwargs):
424	574	super(PaypalConverter, self).__init__(args, *kwargs)
	575	if self.payee_format is None:
	576	self.payee_format = \
	577	"{Name} {To Email Address} {Item Title} ID: {Transaction ID}, {Type}"
425	578
426	579	def get_csv_id(self, row):
427		return "paypal.%s"%(Converter.clean_id(row['Transaction ID']))
	580	return "paypal.%s" % (Converter.clean_id(row['Transaction ID']))
428	581
429	582	def convert(self, row):
430		if (((row['Status'] != "Completed") and (row['Status'] != "Refunded") and (row['Status'] != "Reversed")) or (row['Type'] == "Shopping Cart Item")):
	583	if (((row['Status'] != "Completed") and (row['Status'] != "Refunded") and (
	584	row['Status'] != "Reversed")) or (row['Type'] == "Shopping Cart Item")):
431	585	return ""
432	586	else:
433	587	currency = row['Currency']
434		if row['Type'] == "Add Funds from a Bank Account" or row['Type'] == "Charge From Debit Card":
435		postings=[
436		Posting(
437		self.name,
438		Amount(Decimal(row['Net']), currency)
439		),
440		Posting(
441		"Transfer:Paypal",
442		Amount(Decimal(row['Net']), currency, reverse=True)
443		)]
	588	posting_metadata = {"csvid": self.get_csv_id(row)}
	589	net = Decimal(row['Net'].replace(',', ''))
	590	gross = Decimal(row['Gross'].replace(',', ''))
	591
	592	if row['Type'] == "Add Funds from a Bank Account" or \
	593	row['Type'] == "Charge From Debit Card":
	594	posting = Posting(self.name,
	595	Amount(net, currency),
	596	metadata=posting_metadata)
	597	postings = [posting,
	598	posting.clone_inverted("Transfer:Paypal")]
444	599	else:
445		postings=[
446		Posting(
447		self.name,
448		Amount(Decimal(row['Gross']), currency)
449		),
450		Posting(
451		# TODO Our payees are breaking the payee search in mk_dynamic_account
452		"Expenses:Misc", #self.mk_dynamic_account(payee, exclude=self.name),
453		Amount(Decimal(row['Gross']), currency, reverse=True)
454		)]
	600	posting = Posting(self.name,
	601	Amount(gross, currency),
	602	metadata=posting_metadata)
	603	postings = [
	604	posting,
	605	# TODO Our payees are breaking the payee search in
	606	# mk_dynamic_account
	607	# self.mk_dynamic_account(payee, exclude=self.name),
	608	posting.clone_inverted("Expenses:Misc")]
455	609	return Transaction(
456	610	date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
457		payee=re.sub(
458		r"\s+", " ",
459		"%s %s %s ID: %s, %s"%(row['Name'], row['To Email Address'], row['Item Title'], row['Transaction ID'], row['Type'])),
460		metadata={"csvid": self.get_csv_id(row)},
	611	payee=self.format_payee(row),
461	612	postings=postings)
462	613
	614	# Apparently Paypal has another CSV
	615
	616
	617	class PaypalAlternateConverter(CsvConverter):
	618	FIELDSET = {"Date", "Name", "Type", "Status", "Amount"}
	619
	620	def __init__(self, args, *kwargs):
	621	super(PaypalAlternateConverter, self).__init__(args, *kwargs)
	622	if self.payee_format is None:
	623	self.payee_format = "{Name}: {Type}"
	624
	625	def mk_amount(self, row, reverse=False):
	626	currency = '$'
	627	if 'Currency' in row:
	628	currency = row['Currency']
	629	return Amount(
	630	Decimal(
	631	re.sub(
	632	r"\$",
	633	"",
	634	row['Amount'])),
	635	currency,
	636	reverse=reverse)
	637
	638	def convert(self, row):
	639	if (((row['Status'] != "Completed") and (row['Status'] != "Refunded") and (
	640	row['Status'] != "Reversed")) or (row['Type'] == "Shopping Cart Item")):
	641	return ""
	642	else:
	643	posting_metadata = {"csvid": self.get_csv_id(row)}
	644	posting = Posting(self.name,
	645	self.mk_amount(row),
	646	metadata=posting_metadata)
	647	if row['Type'] == "Add Funds from a Bank Account" \
	648	or row['Type'] == "Charge From Debit Card":
	649
	650	posting2_account="Transfer:Paypal"
	651	else:
	652	posting2_account="Expenses:Misc"
	653	return Transaction(
	654	date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
	655	payee=self.format_payee(row),
	656	postings=[posting,posting.clone_inverted(posting2_account)])
	657
463	658
464	659	class AmazonConverter(CsvConverter):
465		FIELDSET = set(['Currency', 'Title', 'Order Date', 'Order ID'])
	660	FIELDSET = {'Currency', 'Title', 'Order Date', 'Order ID'}
466	661
467	662	def __init__(self, args, *kwargs):
468	663	super(AmazonConverter, self).__init__(args, *kwargs)
	664	self.dialect.doublequote = True
469	665
470	666	def mk_amount(self, row, reverse=False):
471	667	currency = row['Currency']
472		if currency == "USD": currency = "$"
473		return Amount(Decimal(re.sub(r"\$", "", row['Item Total'])), currency, reverse=reverse)
	668	if currency == "USD":
	669	currency = "$"
	670	return Amount(
	671	Decimal(
	672	re.sub(
	673	r"\$",
	674	"",
	675	row['Item Total'])),
	676	currency,
	677	reverse=reverse)
474	678
475	679	def get_csv_id(self, row):
476		return "amazon.%s"%(Converter.clean_id(row['Order ID']))
	680	return "amazon.%s" % (Converter.clean_id(row['Order ID']))
477	681
478	682	def convert(self, row):
	683	posting = Posting(
	684	self.name,
	685	self.mk_amount(row),
	686	metadata={
	687	"url": "https://www.amazon.com/gp/css/summary/print.html/ref=od_aui_print_invoice?ie=UTF8&orderID=%s" % # noqa E501
	688	(row['Order ID']),
	689	"csvid": self.get_csv_id(row)})
	690
479	691	return Transaction(
480		date=datetime.datetime.strptime(row['Order Date'], "%m/%d/%y"),
	692	date=datetime.datetime.strptime(
	693	row['Order Date'],
	694	"%m/%d/%y"),
481	695	payee=row['Title'],
482		metadata={
483		"url": "https://www.amazon.com/gp/css/summary/print.html/ref=od_aui_print_invoice?ie=UTF8&orderID=%s"%(row['Order ID']),
484		"csvid": self.get_csv_id(row)},
485		postings=[
486		Posting(self.name, self.mk_amount(row)),
487		Posting("Expenses:Misc", self.mk_amount(row, reverse=True))
488		])
	696	postings=[posting,
	697	posting.clone_inverted("Expenses:Misc")])
	698
489	699
490	700	class MintConverter(CsvConverter):
491		FIELDSET = set(['Date', 'Amount', 'Description', 'Account Name', 'Category', 'Transaction Type'])
	701	FIELDSET = {
	702	'Date',
	703	'Amount',
	704	'Description',
	705	'Account Name',
	706	'Category',
	707	'Transaction Type'}
492	708
493	709	def __init__(self, args, *kwargs):
494	710	super(MintConverter, self).__init__(args, *kwargs)

501	717	if account is None:
502	718	account = row['Account Name']
503	719	postings = []
	720	posting_metadata = {"csvid": "mint.%s" % (self.get_csv_id(row))}
504	721	if (row['Transaction Type'] == 'credit'):
505		postings = [Posting(account, self.mk_amount(row, reverse=True)),
506		Posting(row['Category'], self.mk_amount(row))]
507		else:
508		postings = [Posting(account, self.mk_amount(row)),
509		Posting("Expenses:%s"%(row['Category']), self.mk_amount(row, reverse=True))]
	722	posting = Posting(account,
	723	self.mk_amount(row, reverse=True),
	724	metadata=posting_metadata)
	725	postings = [posting,
	726	posting.clone_inverted(row['Category'])]
	727	else:
	728	posting = Posting(account,
	729	self.mk_amount(row),
	730	metadata=posting_metadata)
	731	postings = [posting,
	732	posting.clone_inverted("Expenses:%s" % (row['Category']))]
510	733
511	734	return Transaction(
512	735	date=datetime.datetime.strptime(row['Date'], "%m/%d/%Y"),
513		metadata={"csvid": "mint.%s"%(self.get_csv_id(row))},
514	736	payee=row['Description'],
515	737	postings=postings)
	738
	739	# Simple.com
	740	class SimpleConverter(CsvConverter):
	741	FIELDSET = {
	742	"Date",
	743	"Recorded at",
	744	"Scheduled for",
	745	"Amount",
	746	"Activity",
	747	"Pending",
	748	"Raw description",
	749	"Description",
	750	"Category folder",
	751	"Category",
	752	"Street address",
	753	"City",
	754	"State",
	755	"Zip",
	756	"Latitude",
	757	"Longitude",
	758	"Memo"}
	759
	760	def __init__(self, args, *kwargs):
	761	super(SimpleConverter, self).__init__(args, *kwargs)
	762
	763	def convert(self, row):
	764	amount = abs(float(row['Amount']))
	765	reverse = row['Amount'][0] == '-'
	766
	767	if reverse:
	768	account = "Expenses:%s" % (row['Category'])
	769	else:
	770	account = "Income:%s" % (row['Category'])
	771
	772	posting_metadata = {
	773	"csvid": "simple.%s" % (self.get_csv_id(row)),
	774	"raw_description": row['Raw description'],
	775	"activity_type" : row['Activity'],
	776	}
	777	if row['Memo']:
	778	posting_metadata["memo"] = row["Memo"]
	779
	780	return Transaction(
	781	date = datetime.datetime.strptime(row['Date'], "%Y/%m/%d"),
	782	payee = row['Description'],
	783	postings = [Posting(self.name, Amount(amount, '$', reverse), metadata = posting_metadata),
	784	Posting(account, Amount(amount, '$', reverse = not(reverse)))
	785	]
	786	)

+46

-29

ledgerautosync/ledgerwrap.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18
19	19	import csv
20	20	import os
21	21	import re

23	23	import subprocess
24	24	from subprocess import Popen, PIPE
25	25	from threading import Thread
26		from Queue import Queue, Empty
	26	from queue import Queue, Empty
27	27	from ledgerautosync.converter import Converter
28	28	import logging
29		from fuzzywuzzy import process
30
31
32		csv.register_dialect('ledger', delimiter=',', quoting=csv.QUOTE_ALL, escapechar="\\")
	29
	30
	31	csv.register_dialect(
	32	'ledger',
	33	delimiter=',',
	34	quoting=csv.QUOTE_ALL,
	35	escapechar="\\")
	36
33	37
34	38	def mk_ledger(ledger_file):
35	39	if Ledger.available():

37	41	elif HLedger.available():
38	42	return HLedger(ledger_file)
39	43	elif LedgerPython.available():
40		# string_read=True works around http://bugs.ledger-cli.org/show_bug.cgi?id=973
	44	# string_read=True works around
	45	# http://bugs.ledger-cli.org/show_bug.cgi?id=973
41	46	return LedgerPython(ledger_file, string_read=True)
42	47	else:
43	48	raise Exception("Neither ledger 3 nor hledger found!")

53	58	return s
54	59	return [clean_str(s) for s in a]
55	60
56		@staticmethod
57		def clean_payee(s):
58		s = s.replace('%', '')
59		s = s.replace('/', '\/')
60		s = s.replace("'", "")
61		return s
62
63	61	# Return True if this ledgerlike interface is available
64	62	@staticmethod
65	63	def available():

81	79	self.load_payees()
82	80	return self.filter_accounts(self.payees.get(payee, []), exclude)
83	81
84		def get_fuzzy_account_by_payee(self, payee, exclude):
85		self.load_payees()
86		fuzzed_payee = process.extractOne(payee, self.payees)[0]
87		return self.filter_accounts([fuzzed_payee], exclude)
88
89	82	def __init__(self):
90	83	self.payees = None
91	84
	85
92	86	class Ledger(MetaLedger):
93	87	@staticmethod
94	88	def available():
95	89	return ((distutils.spawn.find_executable('ledger') is not None) and
96		(Popen(["ledger", "--version"], stdout=PIPE).
	90	(Popen(["ledger", "--version"], stdout=PIPE, universal_newlines=True).
97	91	communicate()[0]).startswith("Ledger 3"))
98	92
99	93	def __init__(self, ledger_file=None, no_pipe=True):

117	111	self.args += ["-f", ledger_file]
118	112	if self.use_pipe:
119	113	self.p = Popen(self.args, bufsize=1, stdin=PIPE, stdout=PIPE,
120		close_fds=True)
	114	universal_newlines=True, close_fds=True)
121	115	self.q = Queue()
122		self.t = Thread(target=enqueue_output, args=(self.p.stdout, self.q))
	116	self.t = Thread(
	117	target=enqueue_output, args=(
	118	self.p.stdout, self.q))
123	119	self.t.daemon = True # thread dies with the program
124	120	self.t.start()
125	121	# read output until prompt

156	152	cmd = self.args + ["csv"] + cmd
157	153	if os.name == 'nt':
158	154	cmd = MetaLedger.windows_clean(cmd)
159		return csv.reader(subprocess.check_output(cmd).splitlines(), dialect='ledger')
160
	155	return csv.reader(
	156	subprocess.check_output(
	157	cmd,
	158	universal_newlines=True).splitlines(),
	159	dialect='ledger')
161	160
162	161	def check_transaction_by_id(self, key, value):
163	162	q = ["-E", "meta", "%s=%s" % (key, Converter.clean_id(value))]
164	163	try:
165		self.run(q).next()
	164	next(self.run(q))
166	165	return True
167	166	except StopIteration:
168	167	return False

174	173	for line in r:
175	174	self.add_payee(line[2], line[3])
176	175
	176	def get_autosync_payee(self, payee, account):
	177	q = [account, "--last", "1", "--format", "%(quoted(payee))\n",
	178	"--limit", 'tag("AutosyncPayee") == "%s"' % (payee)]
	179	r = self.run(q)
	180	try:
	181	return next(r)[0]
	182	except StopIteration:
	183	return payee
	184
177	185
178	186	class LedgerPython(MetaLedger):
179	187	@staticmethod
180	188	def available():
181	189	try:
182		import ledger
	190	import ledger # noqa: F401
183	191	return True
184	192	except ImportError:
185	193	return False

208	216	self.payees = {}
209	217	for xact in self.journal:
210	218	for post in xact.posts():
211		self.add_payee(xact.payee, post.reported_account().fullname())
	219	self.add_payee(
	220	xact.payee, post.reported_account().fullname())
212	221
213	222	def check_transaction_by_id(self, key, value):
214	223	q = self.journal.query("-E meta %s=\"%s\"" %
215	224	(key, Converter.clean_id(value)))
216	225	return len(q) > 0
	226
	227	def get_autosync_payee(self, payee, account):
	228	logging.error("payee lookup not implemented for LedgerPython, using raw payee")
	229	return payee
217	230
218	231
219	232	class HLedger(MetaLedger):

242	255	if os.name == 'nt':
243	256	cmd = MetaLedger.windows_clean(cmd)
244	257	logging.debug(" ".join(cmd))
245		return subprocess.check_output(cmd)
	258	return subprocess.check_output(cmd, universal_newlines=True)
246	259
247	260	def check_transaction_by_id(self, key, value):
248	261	cmd = ["reg", "tag:%s=%s" % (key, Converter.clean_id(value))]

253	266	self.payees = {}
254	267	cmd = ["reg", "-O", "csv"]
255	268	r = csv.DictReader(self.run(cmd).splitlines())
256		headers = r.next()
	269	next(r) # skip headers
257	270	for line in r:
258	271	self.add_payee(line['description'], line['account'])
	272
	273	def get_autosync_payee(self, payee, account):
	274	logging.error("payee lookup not implemented for HLedger, using raw payee")
	275	return payee

+86

-33

ledgerautosync/sync.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18
19	19	from ofxparse import OfxParser
20	20	from ledgerautosync.converter import CsvConverter
21		from ofxparse.ofxparse import InvestmentTransaction
	21	from ofxparse import OfxParserException
22	22	import logging
23	23	import csv
	24	import codecs
	25
24	26
25	27	class Synchronizer(object):
26	28	def __init__(self, lgr):
27	29	self.lgr = lgr
28	30
	31
29	32	class OfxSynchronizer(Synchronizer):
30		def __init__(self, lgr):
	33	def __init__(self, lgr, hardcodeaccount=None, shortenaccount=None):
	34	self.hardcodeaccount = hardcodeaccount
	35	self.shortenaccount = shortenaccount
31	36	super(OfxSynchronizer, self).__init__(lgr)
32	37
33		def parse_file(self, path, accountname=None):
34		ofx = OfxParser.parse(file(path))
35		return (ofx, self.filter(ofx))
	38	@staticmethod
	39	def parse_file(path):
	40	return OfxParser.parse(open(path))
36	41
37	42	def is_txn_synced(self, acctid, txn):
38	43	if self.lgr is None:

40	45	# All transactions are considered "synced" in this case.
41	46	return False
42	47	else:
43		ofxid = "%s.%s" % (acctid, txn.id)
	48	acctid_to_use = acctid
	49	txnid_to_use = txn.id
	50	if self.hardcodeaccount:
	51	acctid_to_use = self.hardcodeaccount
	52	txnid_to_use = txnid_to_use.replace(acctid, acctid_to_use)
	53	elif self.shortenaccount:
	54	acctid_to_use = acctid[-4:]
	55	txnid_to_use = txnid_to_use.replace(acctid, acctid_to_use)
	56	ofxid = "%s.%s" % (acctid_to_use, txnid_to_use)
44	57	return self.lgr.check_transaction_by_id("ofxid", ofxid)
45	58
46	59	# Filter out comment transactions. These have an amount of 0 and the same

62	75	retval.append(txn)
63	76	return retval
64	77
65		def filter(self, ofx):
66		def extract_sort_key(txn):
67		if hasattr(txn, 'tradeDate'):
68		return txn.tradeDate
69		elif hasattr(txn, 'date'):
70		return txn.date
71		elif hasattr(txn, 'settleDate'):
72		return txn.settleDate
73		return None
74		txns = ofx.account.statement.transactions
	78	@staticmethod
	79	def extract_sort_key(txn):
	80	if hasattr(txn, 'tradeDate'):
	81	return txn.tradeDate
	82	elif hasattr(txn, 'date'):
	83	return txn.date
	84	elif hasattr(txn, 'settleDate'):
	85	return txn.settleDate
	86	return None
	87
	88	def filter(self, txns, acctid):
75	89	if len(txns) == 0:
76	90	sorted_txns = txns
77	91	else:
78		sorted_txns = sorted(txns, key=extract_sort_key)
79		acctid = ofx.account.account_id
	92	sorted_txns = sorted(txns, key=OfxSynchronizer.extract_sort_key)
80	93	retval = [txn for txn in sorted_txns
81	94	if not(self.is_txn_synced(acctid, txn))]
82	95	return self.filter_comment_txns(retval)

94	107	raw = acct.download(days=days)
95	108
96	109	if raw.read() == 'Server error occured. Received HttpStatusCode of 400':
97		raise Exception("Error connecting to account %s"%(acct.description))
	110	raise Exception(
	111	"Error connecting to account %s" %
	112	(acct.description))
98	113	raw.seek(0)
99		ofx = OfxParser.parse(raw)
	114	ofx = None
	115	try:
	116	ofx = OfxParser.parse(raw)
	117	except OfxParserException as ex:
	118	if ex.message == 'The ofx file is empty!':
	119	return (ofx, [])
	120	else:
	121	raise ex
	122	if ofx.signon is not None:
	123	if ofx.signon.severity == 'ERROR':
	124	raise Exception(
	125	"Error returned from server for %s: %s" %
	126	(acct.description, ofx.signon.message))
100	127	if not(hasattr(ofx, 'account')):
101	128	# some banks return this for no txns
102	129	if (days >= max_days):

113	140	last_txns_len = 0
114	141	else:
115	142	txns = ofx.account.statement.transactions
116		new_txns = self.filter(ofx)
	143	new_txns = self.filter(txns, ofx.account.account_id)
117	144	logging.debug("txns: %d" % (len(txns)))
118	145	logging.debug("new txns: %d" % (len(new_txns)))
119	146	if ((len(txns) > 0) and (last_txns_len == len(txns))):
120	147	# not getting more txns than last time; we have
121	148	# reached the beginning
122		logging.debug("Not getting more txns than last time, done.")
	149	logging.debug(
	150	"Not getting more txns than last time, done.")
123	151	return (ofx, new_txns)
124	152	elif (len(txns) > len(new_txns)) or (days >= max_days):
125	153	# got more txns than were new or hit max_days, we've

139	167
140	168
141	169	class CsvSynchronizer(Synchronizer):
142		def __init__(self, lgr):
	170	def __init__(self, lgr, payee_format=None):
143	171	super(CsvSynchronizer, self).__init__(lgr)
	172	self.payee_format = payee_format
	173
	174	def is_row_synced(self, converter, row):
	175	if self.lgr is None:
	176	# User called with --no-ledger
	177	# All transactions are considered "synced" in this case.
	178	return False
	179	else:
	180	return self.lgr.check_transaction_by_id(
	181	"csvid", converter.get_csv_id(row))
144	182
145	183	def parse_file(self, path, accountname=None, unknownaccount=None):
146		with open(path, 'rb') as f:
	184	with open(path) as f:
	185	has_bom = f.read(3) == codecs.BOM_UTF8
	186	if not(has_bom):
	187	f.seek(0)
	188	else:
	189	f.seek(3)
147	190	dialect = csv.Sniffer().sniff(f.read(1024))
148		f.seek(0)
	191	if not(has_bom):
	192	f.seek(0)
	193	else:
	194	f.seek(3)
149	195	dialect.skipinitialspace = True
150	196	reader = csv.DictReader(f, dialect=dialect)
151	197	converter = CsvConverter.make_converter(
152		reader,
	198	set(reader.fieldnames),
	199	dialect,
153	200	name=accountname,
154		ledger=self.lgr,
155		unknownaccount=unknownaccount)
156		return [converter.convert(row)
157		for row in reader
158		if not(self.lgr.check_transaction_by_id(
159		"csvid", converter.get_csv_id(row)))]
	201	unknownaccount=unknownaccount,
	202	payee_format=self.payee_format)
	203	# Create a new reader in case the converter modified the dialect
	204	if not(has_bom):
	205	f.seek(0)
	206	else:
	207	f.seek(3)
	208	reader = csv.DictReader(f, dialect=dialect)
	209	return [
	210	converter.convert(row) for row in reader if not(
	211	self.is_row_synced(
	212	converter, row))]

+18

-0

requirements.txt less more

	0	asn1crypto==0.24.0
	1	beautifulsoup4==4.7.1
	2	cffi==1.12.2
	3	cryptography==2.6.1
	4	entrypoints==0.3
	5	jeepney==0.4
	6	keyring==19.0.1
	7	lxml==4.3.3
	8	mock==2.0.0
	9	nose==1.3.7
	10	ofxclient==2.0.3
	11	ofxhome==0.3.3
	12	ofxparse==0.20
	13	pbr==5.1.3
	14	pycparser==2.19
	15	SecretStorage==3.1.1
	16	six==1.12.0
	17	soupsieve==1.9.1

-5

~~setup.cfg~~ less more

0		[egg_info]
1		tag_build =
2		tag_date = 0
3		tag_svn_revision = 0
4

-7

setup.py less more

11	11
12	12	setup(
13	13	name='ledger-autosync',
14		version="0.3.5",
	14	version="1.0.1",
15	15	description="Automatically sync your bank's data with ledger",
16	16	long_description=long_description,
17	17	author='Erik Hetzner',

22	22	classifiers=[
23	23	'Development Status :: 5 - Production/Stable',
24	24	'Intended Audience :: End Users/Desktop',
25		'License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)',
	25	'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
26	26	'Operating System :: OS Independent',
27		'Programming Language :: Python :: 2.7',
	27	'Programming Language :: Python :: 3.5',
28	28	'Topic :: Office/Business :: Financial :: Accounting',
29	29	'Topic :: Office/Business :: Financial :: Investment',
30	30	'Topic :: Office/Business :: Financial'

37	37	install_requires=[
38	38	'setuptools>=26',
39	39	'ofxclient',
40		'ofxparse>=0.14',
41		'BeautifulSoup4',
42		'fuzzywuzzy'
	40	'ofxparse'
43	41	],
44	42
45	43	extras_require={

53	51	]
54	52	},
55	53
56		test_suite = 'nose.collector'
	54	test_suite='nose.collector'
57	55	)

+37

-12

tests/test_cli.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18	# flake8: noqa E501
	19
19	20	from ledgerautosync import LedgerAutosyncException
20	21	from ledgerautosync.cli import run, find_ledger_file
21	22	from ledgerautosync.ledgerwrap import Ledger, LedgerPython, HLedger
22	23	from ofxclient.config import OfxConfig
23	24	import os.path
24	25	import tempfile
25		import sys
26		from StringIO import StringIO
	26	from io import StringIO
27	27
28	28	from unittest import TestCase
29	29	from mock import Mock, call, patch
30	30	from nose.plugins.attrib import attr
31	31	from nose.tools import raises
32	32
	33
33	34	class CliTest():
34	35	def test_run(self):
35	36	config = OfxConfig(os.path.join('fixtures', 'ofxclient.ini'))
36	37	acct = config.accounts()[0]
37	38	acct.download = Mock(side_effect=lambda args, *kwargs:
38		file(os.path.join('fixtures', 'checking.ofx')))
	39	open(os.path.join('fixtures', 'checking.ofx')))
39	40	config.accounts = Mock(return_value=[acct])
40	41	run(['-l', os.path.join('fixtures', 'empty.lgr')], config)
41	42	acct.download.assert_has_calls([call(days=7), call(days=14)])

43	44
44	45	def test_run_csv_file(self):
45	46	config = OfxConfig(os.path.join('fixtures', 'ofxclient.ini'))
46		run(['-a', 'Paypal', '-l', os.path.join('fixtures', 'empty.lgr'), os.path.join('fixtures', 'paypal.csv')], config)
	47	run(['-a', 'Paypal', '-l', os.path.join('fixtures', 'empty.lgr'),
	48	os.path.join('fixtures', 'paypal.csv')], config)
47	49
48	50	def test_filter_account(self):
49	51	config = OfxConfig(os.path.join('fixtures', 'ofxclient.ini'))

52	54	bar = next(acct for acct in config.accounts()
53	55	if acct.description == 'Assets:Checking:Bar')
54	56	foo.download = Mock(side_effect=lambda args, *kwargs:
55		file(os.path.join('fixtures', 'checking.ofx')))
	57	open(os.path.join('fixtures', 'checking.ofx')))
56	58	bar.download = Mock()
57	59	config.accounts = Mock(return_value=[foo, bar])
58	60	run(['-l', os.path.join('fixtures', 'checking.lgr'),

62	64
63	65	def test_find_ledger_path(self):
64	66	os.environ["LEDGER_FILE"] = "/tmp/foo"
65		self.assertEqual(find_ledger_file(), "/tmp/foo", "Should use LEDGER_FILE to find ledger path.")
	67	self.assertEqual(
	68	find_ledger_file(),
	69	"/tmp/foo",
	70	"Should use LEDGER_FILE to find ledger path.")
66	71
67	72	(f, tmprcpath) = tempfile.mkstemp(".ledgerrc")
68		os.close(f) # Who wants to deal with low-level file descriptors?
	73	os.close(f) # Who wants to deal with low-level file descriptors?
69	74	with open(tmprcpath, 'w') as f:
70	75	f.write("--bar foo\n")
71	76	f.write("--file /tmp/bar\n")
72	77	f.write("--foo bar\n")
73		self.assertEqual(find_ledger_file(tmprcpath), "/tmp/foo", "Should prefer LEDGER_FILE to --file arg in ledgerrc")
	78	self.assertEqual(
	79	find_ledger_file(tmprcpath),
	80	"/tmp/foo",
	81	"Should prefer LEDGER_FILE to --file arg in ledgerrc")
74	82	del os.environ["LEDGER_FILE"]
75		self.assertEqual(find_ledger_file(tmprcpath), "/tmp/bar", "Should parse ledgerrc")
	83	self.assertEqual(
	84	find_ledger_file(tmprcpath),
	85	"/tmp/bar",
	86	"Should parse ledgerrc")
76	87	os.unlink(tmprcpath)
77	88
78	89	@raises(LedgerAutosyncException)

81	92	run(['-l', os.path.join('fixtures', 'checking.lgr'),
82	93	'-L'], config)
83	94
	95	def test_format_payee(self):
	96	with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
	97	run([os.path.join('fixtures', 'paypal.csv'), '-a',
	98	'Assets:Foo', '--payee-format', 'GROSS:{Gross}', '-L'])
	99	self.assertRegex(mock_stdout.getvalue(), r"GROSS:-20\.00")
	100
	101	# def test_multi_account(self):
	102	# with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
	103	# run([os.path.join('fixtures', 'multi_account.ofx'), '-a', 'Assets:Foo'])
	104	# self.assertRegexpMatches(mock_stdout.getvalue(), r"GROSS:-20\.00")
	105
84	106	def test_no_ledger(self):
85	107	config = OfxConfig(os.path.join('fixtures', 'ofxclient.ini'))
86	108	acct = config.accounts()[0]
87	109	acct.download = Mock(side_effect=lambda args, *kwargs:
88		file(os.path.join('fixtures', 'checking.ofx')))
	110	open(os.path.join('fixtures', 'checking.ofx')))
89	111	config.accounts = Mock(return_value=[acct])
90	112	with patch('ledgerautosync.cli.find_ledger_file', return_value=None):
91	113	with patch('sys.stderr', new_callable=StringIO) as mock_stdout:
92	114	run([], config)
93		self.assertEquals(mock_stdout.getvalue(), 'LEDGER_FILE environment variable not set, and no .ledgerrc file found, and -l argument was not supplied: running with deduplication disabled. All transactions will be printed!')
	115	self.assertEqual(
	116	mock_stdout.getvalue(),
	117	'LEDGER_FILE environment variable not set, and no .ledgerrc file found, and -l argument was not supplied: running with deduplication disabled. All transactions will be printed!\n')
	118
94	119
95	120	@attr('hledger')
96	121	class TestCliHledger(TestCase, CliTest):

+92

-40

tests/test_converter.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
19		from ledgerautosync.converter import Converter, CsvConverter, AmazonConverter, MintConverter, PaypalConverter, Amount, Posting
	18	# flake8: noqa E501
	19
	20	from ledgerautosync.converter import CsvConverter, AmazonConverter, MintConverter, \
	21	PaypalConverter, PaypalAlternateConverter, Amount, Posting
20	22	from decimal import Decimal
21	23	import hashlib
22	24	import csv

28	30	@attr('generic')
29	31	class TestPosting(LedgerTestCase):
30	32	def test_format(self):
31		self.assertRegexpMatches(
	33	self.assertEqualLedgerPosting(
32	34	Posting(
33	35	"Foo",
34		Amount(Decimal("10.00"), "$")
	36	Amount(Decimal("10.00"), "$"),
	37	metadata={'foo': 'bar'}
35	38	).format(indent=2),
36		r'^ Foo.*$')
	39	" Foo $10.00\n ; foo: bar\n")
37	40
38	41
39	42	@attr('generic')

69	72	def test_get_csv_id(self):
70	73	converter = CsvConverter(None)
71	74	h = {'foo': 'bar', 'bar': 'foo'}
72		self.assertEqual(converter.get_csv_id(h),
73		hashlib.md5("bar=foo\nfoo=bar\n").hexdigest())
	75	self.assertEqual(converter.get_csv_id(h), hashlib.md5(
	76	"bar=foo\nfoo=bar\n".encode('utf-8')).hexdigest())
	77
	78
	79	class CsvConverterTestCase(LedgerTestCase):
	80	def make_converter(self, f, name=None):
	81	dialect = csv.Sniffer().sniff(f.read(1024))
	82	f.seek(0)
	83	dialect.skipinitialspace = True
	84	reader = csv.DictReader(f, dialect=dialect)
	85	converter = CsvConverter.make_converter(
	86	set(reader.fieldnames), dialect, name)
	87	f.seek(0)
	88	reader = csv.DictReader(f, dialect=dialect)
	89	return (reader, converter)
74	90
75	91
76	92	@attr('generic')
77		class TestPaypalConverter(LedgerTestCase):
	93	class TestPaypalConverter(CsvConverterTestCase):
78	94	def test_format(self):
79		with open('fixtures/paypal.csv', 'rb') as f:
80		dialect = csv.Sniffer().sniff(f.read(1024))
81		f.seek(0)
82		dialect.skipinitialspace = True
83		reader = csv.DictReader(f, dialect=dialect)
84		converter = CsvConverter.make_converter(reader, name='Foo')
	95	with open('fixtures/paypal.csv') as f:
	96	(reader, converter) = self.make_converter(f, 'Foo')
85	97	self.assertEqual(type(converter), PaypalConverter)
86	98	self.assertEqual(
87		converter.convert(reader.next()).format(),
	99	converter.convert(
	100	next(reader)).format(),
88	101	"""2016/06/04 Jane Doe someone@example.net My Friend ID: XYZ1, Recurring Payment Sent
	102	Foo -20.00 USD
89	103	; csvid: paypal.XYZ1
90		Foo -20.00 USD
91	104	Expenses:Misc 20.00 USD
92	105	""")
93	106	self.assertEqual(
94		converter.convert(reader.next()).format(),
	107	converter.convert(next(reader)).format(),
95	108	"""2016/06/04 Debit Card ID: XYZ2, Charge From Debit Card
	109	Foo 1120.00 USD
96	110	; csvid: paypal.XYZ2
97		Foo 20.00 USD
98		Transfer:Paypal -20.00 USD
	111	Transfer:Paypal -1120.00 USD
99	112	""")
100	113
	114
101	115	@attr('generic')
102		class TestAmazonConverter(LedgerTestCase):
	116	class TestPaypalAlternateConverter(CsvConverterTestCase):
103	117	def test_format(self):
104		with open('fixtures/amazon.csv', 'rb') as f:
105		dialect = csv.Sniffer().sniff(f.read(1024))
106		f.seek(0)
107		dialect.skipinitialspace = True
108		reader = csv.DictReader(f, dialect=dialect)
109		converter = CsvConverter.make_converter(reader, name='Foo')
	118	with open('fixtures/paypal_alternate.csv') as f:
	119	(reader, converter) = self.make_converter(f, 'Foo')
	120	self.assertEqual(type(converter), PaypalAlternateConverter)
	121	self.assertEqual(
	122	converter.convert(next(reader)).format(),
	123	"""2016/12/31 Some User: Payment Sent
	124	Foo -$12.34
	125	; csvid: 1209a7bb0d17276248d463b71a6a8b8c
	126	Expenses:Misc $12.34
	127	""")
	128	self.assertEqual(
	129	converter.convert(next(reader)).format(),
	130	"""2016/12/31 Bank Account: Add Funds from a Bank Account
	131	Foo $12.34
	132	; csvid: 581e62da71bab74c7ce61854c2b6b6a5
	133	Transfer:Paypal -$12.34
	134	""")
	135
	136	def test_mk_amount(self):
	137	converter = PaypalAlternateConverter(None)
	138	row = {"Currency": "USD", "Amount": "12.34"}
	139	self.assertEqual(
	140	converter.mk_amount(row), Amount(
	141	Decimal('12.34'), "USD"))
	142
	143
	144	@attr('generic')
	145	class TestAmazonConverter(CsvConverterTestCase):
	146	def test_format(self):
	147	with open('fixtures/amazon.csv') as f:
	148	(reader, converter) = self.make_converter(f, 'Foo')
110	149	self.assertEqual(type(converter), AmazonConverter)
111	150	self.assertEqual(
112		converter.convert(reader.next()).format(),
	151	converter.convert(next(reader)).format(),
113	152	"""2016/01/29 Best Soap Ever
	153	Foo $21.90
	154	; csvid: amazon.123-4567890-1234567
114	155	; url: https://www.amazon.com/gp/css/summary/print.html/ref=od_aui_print_invoice?ie=UTF8&orderID=123-4567890-1234567
115		; csvid: amazon.123-4567890-1234567
116		Foo $21.90
117	156	Expenses:Misc -$21.90
118	157	""")
119	158
	159
120	160	@attr('generic')
121		class TestMintConverter(LedgerTestCase):
	161	class TestAmazonConverter2(CsvConverterTestCase):
122	162	def test_format(self):
123		with open('fixtures/mint.csv', 'rb') as f:
124		dialect = csv.Sniffer().sniff(f.read(1024))
125		f.seek(0)
126		dialect.skipinitialspace = True
127		reader = csv.DictReader(f, dialect=dialect)
128		converter = CsvConverter.make_converter(reader)
	163	with open('fixtures/amazon2.csv') as f:
	164	(reader, converter) = self.make_converter(f, 'Foo')
	165	self.assertEqual(type(converter), AmazonConverter)
	166	self.assertEqual(
	167	converter.convert(next(reader)).format(),
	168	"""2017/06/05 Test " double quote
	169	Foo $9.99
	170	; csvid: amazon.111-1111111-1111111
	171	; url: https://www.amazon.com/gp/css/summary/print.html/ref=od_aui_print_invoice?ie=UTF8&orderID=111-1111111-1111111
	172	Expenses:Misc -$9.99
	173	""")
	174
	175
	176	@attr('generic')
	177	class TestMintConverter(CsvConverterTestCase):
	178	def test_format(self):
	179	with open('fixtures/mint.csv') as f:
	180	(reader, converter) = self.make_converter(f)
129	181	self.assertEqual(type(converter), MintConverter)
130	182	self.assertEqual(
131		converter.convert(reader.next()).format(),
	183	converter.convert(next(reader)).format(),
132	184	"""2016/08/02 Amazon
	185	1234 $29.99
133	186	; csvid: mint.a7c028a73d76956453dab634e8e5bdc1
134		1234 $29.99
135	187	Expenses:Shopping -$29.99
136	188	""")
137	189	self.assertEqual(
138		converter.convert(reader.next()).format(),
	190	converter.convert(next(reader)).format(),
139	191	"""2016/06/02 Autopay Rautopay Auto
	192	1234 -$123.45
140	193	; csvid: mint.a404e70594502dd62bfc6f15d80b7cd7
141		1234 -$123.45
142	194	Credit Card Payment $123.45
143	195	""")

+33

-13

tests/test_ledger.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18
19	19	from ledgerautosync.ledgerwrap import Ledger, HLedger, LedgerPython
20	20	from nose.plugins.attrib import attr
21	21	from unittest import TestCase

23	23	import os.path
24	24	import tempfile
25	25
	26
26	27	class LedgerTest(object):
27	28	ledger_path = os.path.join('fixtures', 'checking.lgr')
28		dynamic_ledger_path = os.path.join('fixtures', 'checking-dynamic-account.lgr')
	29	dynamic_ledger_path = os.path.join(
	30	'fixtures', 'checking-dynamic-account.lgr')
29	31
30	32	def check_transaction(self):
31		self.assertTrue(self.lgr.check_transaction_by_id("ofxid", "1101.1452687~7.0000486"))
	33	self.assertTrue(
	34	self.lgr.check_transaction_by_id(
	35	"ofxid", "1101.1452687~7.0000486"))
32	36
33	37	def test_nonexistent_transaction(self):
34	38	self.assertFalse(self.lgr.check_transaction_by_id("ofxid", "FOO"))

37	41	self.assertTrue(self.lgr.check_transaction_by_id("ofxid", "empty"))
38	42
39	43	def test_get_account_by_payee(self):
40		account = self.lgr.get_account_by_payee("AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )", exclude="Assets:Foo")
	44	account = self.lgr.get_account_by_payee(
	45	"AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )",
	46	exclude="Assets:Foo")
41	47	self.assertEqual(account, "Expenses:Bar")
42	48
43	49	def test_get_ambiguous_account_by_payee(self):
44		account = self.dynamic_lgr.get_account_by_payee("Generic", exclude="Assets:Foo")
	50	account = self.dynamic_lgr.get_account_by_payee(
	51	"Generic", exclude="Assets:Foo")
45	52	# shoud use the latest
46	53	self.assertEqual(account, "Expenses:Bar")
47	54

54	61	'PAYEE TEST"QUOTE',
55	62	'PAYEE TEST.PERIOD']
56	63	for payee in payees:
57		self.assertNotEqual(self.lgr.get_account_by_payee(payee, ['Assets:Foo']), None,
58		msg="Did not find %s in %s" % (payee, self.lgr))
	64	self.assertNotEqual(
	65	self.lgr.get_account_by_payee(
	66	payee, ['Assets:Foo']), None, msg="Did not find %s in %s" %
	67	(payee, self.lgr))
59	68
60	69	def test_ofx_id_quoting(self):
61		self.assertEqual(self.lgr.check_transaction_by_id("ofxid", "1/2"), True,
62		msg="Did not find 1/2 in %s" % (self.lgr))
	70	self.assertEqual(
	71	self.lgr.check_transaction_by_id(
	72	"ofxid",
	73	"1/2"),
	74	True,
	75	msg="Did not find 1/2 in %s" %
	76	(self.lgr))
63	77
64	78	def test_load_payees(self):
65	79	self.lgr.load_payees()
66		self.assertEqual(self.lgr.payees['PAYEE TEST:COLON'], ['Assets:Foo', 'Income:Bar'])
	80	self.assertEqual(
	81	self.lgr.payees['PAYEE TEST:COLON'], [
	82	'Assets:Foo', 'Income:Bar'])
	83
67	84
68	85	@attr('hledger')
69	86	class TestHledger(TestCase, LedgerTest):
70	87	def setUp(self):
71	88	self.lgr = HLedger(self.ledger_path)
72	89	self.dynamic_lgr = HLedger(self.dynamic_ledger_path)
	90
73	91
74	92	@attr('ledger')
75	93	class TestLedger(LedgerTest, TestCase):

79	97
80	98	def test_args_only(self):
81	99	(f, tmprcpath) = tempfile.mkstemp(".ledgerrc")
82		os.close(f) # Who wants to deal with low-level file descriptors?
83		# Create an init file that will narrow the test data to a period that contains no trasnactions
	100	os.close(f) # Who wants to deal with low-level file descriptors?
	101	# Create an init file that will narrow the test data to a period that
	102	# contains no trasnactions
84	103	with open(tmprcpath, 'w') as f:
85	104	f.write("--period 2012")
86	105	# If the command returns no trasnactions, as we would expect if we
87	106	# parsed the init file, then this will throw an exception.
88		self.lgr.run([""]).next()
	107	next(self.lgr.run([""]))
89	108	os.unlink(tmprcpath)
	109
90	110
91	111	@attr('ledger-python')
92	112	class TestLedgerPython(TestCase, LedgerTest):

+274

-121

tests/test_ofx_formatter.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
19		from ledgerautosync.converter import OfxConverter
	18	# flake8: noqa E501
	19
	20	from ledgerautosync.converter import OfxConverter, SecurityList
20	21	from ledgerautosync.ledgerwrap import Ledger
21	22	import os.path
22		from decimal import Decimal
23	23
24	24	from ofxparse import OfxParser
25	25

30	30	@attr('generic')
31	31	class TestOfxConverter(LedgerTestCase):
32	32	def test_checking(self):
33		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
34		converter = OfxConverter(ofx=ofx, name="Foo")
35		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[0]).format(),
36		"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	33	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	34	converter = OfxConverter(account=ofx.account, name="Foo")
	35	self.assertEqualLedgerPosting(
	36	converter.convert(
	37	ofx.account.statement.transactions[0]).format(),
	38	"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	39	Foo $0.01
37	40	; ofxid: 1101.1452687~7.0000486
	41	Expenses:Misc -$0.01
	42	""")
	43	self.assertEqualLedgerPosting(
	44	converter.convert(
	45	ofx.account.statement.transactions[1]).format(),
	46	"""2011/04/05 AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )
	47	Foo -$34.51
	48	; ofxid: 1101.1452687~7.0000487
	49	Expenses:Misc $34.51
	50	""")
	51
	52	self.assertEqualLedgerPosting(
	53	converter.convert(
	54	ofx.account.statement.transactions[2]).format(),
	55	"""2011/04/07 RETURNED CHECK FEE, CHECK # 319 FOR $45.33 ON 04/07/11
	56	Foo -$25.00
	57	; ofxid: 1101.1452687~7.0000488
	58	Expenses:Misc $25.00
	59	""")
	60
	61	def test_indent(self):
	62	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	63	converter = OfxConverter(account=ofx.account, name="Foo", indent=4)
	64	# testing indent, so do not use the string collapsing version of assert
	65	self.assertEqual(
	66	converter.convert(
	67	ofx.account.statement.transactions[0]).format(),
	68	"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	69	Foo $0.01
	70	; ofxid: 1101.1452687~7.0000486
	71	Expenses:Misc -$0.01
	72	""")
	73
	74	def test_shortenaccount(self):
	75	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	76	converter = OfxConverter(
	77	account=ofx.account,
	78	name="Foo",
	79	indent=4,
	80	shortenaccount=True)
	81	# testing indent, so do not use the string collapsing version of assert
	82	self.assertEqual(
	83	converter.convert(
	84	ofx.account.statement.transactions[0]).format(),
	85	"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	86	Foo $0.01
	87	; ofxid: 1101.87~7.0000486
	88	Expenses:Misc -$0.01
	89	""")
	90
	91	def test_hardcodeaccount(self):
	92	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	93	converter = OfxConverter(
	94	account=ofx.account,
	95	name="Foo",
	96	indent=4,
	97	hardcodeaccount="9999")
	98	# testing indent, so do not use the string collapsing version of assert
	99	self.assertEqual(
	100	converter.convert(
	101	ofx.account.statement.transactions[0]).format(),
	102	"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
	103	Foo $0.01
	104	; ofxid: 1101.9999.0000486
	105	Expenses:Misc -$0.01
	106	""")
	107
	108	def test_investments(self):
	109	ofx = OfxParser.parse(open(os.path.join('fixtures', 'fidelity.ofx')))
	110	converter = OfxConverter(
	111	account=ofx.account,
	112	name="Foo",
	113	security_list=SecurityList(ofx))
	114	self.assertEqualLedgerPosting(
	115	converter.convert(
	116	ofx.account.statement.transactions[0]).format(),
	117	"""2012/07/20 YOU BOUGHT
	118	Foo 100.00000 INTC @ $25.635000000
	119	; ofxid: 7776.01234567890.0123456789020201120120720
	120	Assets:Unknown -$2563.50
	121	""")
	122	# test no payee/memo
	123	self.assertEqualLedgerPosting(
	124	converter.convert(
	125	ofx.account.statement.transactions[1]).format(),
	126	"""2012/07/27 Foo: buystock
	127	Foo 128.00000 SDRL @ $39.390900000
	128	; ofxid: 7776.01234567890.0123456789020901120120727
	129	Assets:Unknown -$5042.04
	130	""")
	131
	132	def test_dynamic_account(self):
	133	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	134	ledger = Ledger(
	135	os.path.join(
	136	'fixtures',
	137	'checking-dynamic-account.lgr'))
	138	converter = OfxConverter(
	139	account=ofx.account,
	140	name="Assets:Foo",
	141	ledger=ledger)
	142	self.assertEqualLedgerPosting(
	143	converter.convert(
	144	ofx.account.statement.transactions[1]).format(),
	145	"""2011/04/05 AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )
	146	Assets:Foo -$34.51
	147	; ofxid: 1101.1452687~7.0000487
	148	Expenses:Bar $34.51
	149	""")
	150
	151	def test_balance_assertion(self):
	152	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	153	ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
	154	converter = OfxConverter(
	155	account=ofx.account,
	156	name="Assets:Foo",
	157	ledger=ledger)
	158	self.assertEqualLedgerPosting(
	159	converter.format_balance(
	160	ofx.account.statement),
	161	"""2013/05/25 * --Autosync Balance Assertion
	162	Assets:Foo $0.00 = $100.99
	163	""")
	164
	165	def test_initial_balance(self):
	166	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	167	ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
	168	converter = OfxConverter(
	169	account=ofx.account,
	170	name="Assets:Foo",
	171	ledger=ledger)
	172	self.assertEqualLedgerPosting(
	173	converter.format_initial_balance(
	174	ofx.account.statement),
	175	"""2000/01/01 * --Autosync Initial Balance
	176	Assets:Foo $160.49
	177	; ofxid: 1101.1452687~7.autosync_initial
	178	Assets:Equity -$160.49
	179	""")
	180
	181	def test_unknownaccount(self):
	182	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	183	converter = OfxConverter(account=ofx.account, name="Foo",
	184	unknownaccount='Expenses:Unknown')
	185	self.assertEqualLedgerPosting(
	186	converter.convert(
	187	ofx.account.statement.transactions[0]).format(),
	188	"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
38	189	Foo $0.01
39		Expenses:Misc -$0.01
40		""")
41		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[1]).format(),
42		"""2011/04/05 AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )
43		; ofxid: 1101.1452687~7.0000487
44		Foo -$34.51
45		Expenses:Misc $34.51
46		""")
47
48		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[2]).format(),
49		"""2011/04/07 RETURNED CHECK FEE, CHECK # 319 FOR $45.33 ON 04/07/11
50		; ofxid: 1101.1452687~7.0000488
51		Foo -$25.00
52		Expenses:Misc $25.00
53		""")
54
55		def test_indent(self):
56		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
57		converter = OfxConverter(ofx=ofx, name="Foo", indent=4)
58		# testing indent, so do not use the string collapsing version of assert
59		self.assertEqual(converter.convert(ofx.account.statement.transactions[0]).format(),
60		"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
61		; ofxid: 1101.1452687~7.0000486
62		Foo $0.01
63		Expenses:Misc -$0.01
64		""")
65
66		def test_investments(self):
67		ofx = OfxParser.parse(file(os.path.join('fixtures', 'fidelity.ofx')))
68		converter = OfxConverter(ofx=ofx, name="Foo")
69		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[0]).format(),
70		"""2012/07/20 YOU BOUGHT
	190	; ofxid: 1101.1452687~7.0000486
	191	Expenses:Unknown -$0.01
	192	""")
	193
	194	def test_quote_commodity(self):
	195	ofx = OfxParser.parse(open(os.path.join('fixtures', 'fidelity.ofx')))
	196	converter = OfxConverter(
	197	account=ofx.account,
	198	name="Foo",
	199	security_list=SecurityList(ofx))
	200	self.assertEqualLedgerPosting(
	201	converter.convert(
	202	ofx.account.statement.transactions[0]).format(),
	203	"""2012/07/20 YOU BOUGHT
	204	Foo 100.00000 INTC @ $25.635000000
71	205	; ofxid: 7776.01234567890.0123456789020201120120720
72		Foo 100.00000 INTC @ $25.635000000
73		Assets:Unknown -$2563.50
74		""")
75		# test no payee/memo
76		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[1]).format(),
77		"""2012/07/27 Foo: buystock
78		; ofxid: 7776.01234567890.0123456789020901120120727
79		Foo 128.00000 SDRL @ $39.390900000
80		Assets:Unknown -$5042.04
81		""")
82
83		def test_dynamic_account(self):
84		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
85		ledger = Ledger(os.path.join('fixtures', 'checking-dynamic-account.lgr'))
86		converter = OfxConverter(ofx=ofx, name="Assets:Foo", ledger=ledger)
87		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[1]).format(),
88		"""2011/04/05 AUTOMATIC WITHDRAWAL, ELECTRIC BILL WEB(S )
89		; ofxid: 1101.1452687~7.0000487
90		Assets:Foo -$34.51
91		Expenses:Bar $34.51
92		""")
93
94		def test_balance_assertion(self):
95		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
96		ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
97		converter = OfxConverter(ofx=ofx, name="Assets:Foo", ledger=ledger)
98		self.assertEqualLedgerPosting(converter.format_balance(ofx.account.statement),
99		"""2013/05/25 * --Autosync Balance Assertion
100		Assets:Foo $0.00 = $100.99
101		""")
102
103		def test_initial_balance(self):
104		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
105		ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
106		converter = OfxConverter(ofx=ofx, name="Assets:Foo", ledger=ledger)
107		self.assertEqualLedgerPosting(converter.format_initial_balance(ofx.account.statement),
108		"""2000/01/01 * --Autosync Initial Balance
109		; ofxid: 1101.1452687~7.autosync_initial
110		Assets:Foo $160.49
111		Assets:Equity -$160.49
112		""")
113
114		def test_unknownaccount(self):
115		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
116		converter = OfxConverter(ofx=ofx, name="Foo",
117		unknownaccount='Expenses:Unknown')
118		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[0]).format(),
119		"""2011/03/31 DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%
120		; ofxid: 1101.1452687~7.0000486
121		Foo $0.01
122		Expenses:Unknown -$0.01
123		""")
124
125		def test_quote_commodity(self):
126		ofx = OfxParser.parse(file(os.path.join('fixtures', 'fidelity.ofx')))
127		converter = OfxConverter(ofx=ofx, name="Foo")
128		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[0]).format(),
129		"""2012/07/20 YOU BOUGHT
130		; ofxid: 7776.01234567890.0123456789020201120120720
131		Foo 100.00000 INTC @ $25.635000000
132	206	Assets:Unknown -$2563.50
133	207	""")
134	208
135	209	# Check that <TRANSFER> txns are parsed.
136	210	def test_transfer_txn(self):
137		ofx = OfxParser.parse(file(os.path.join('fixtures', 'investment_401k.ofx')))
138		converter = OfxConverter(ofx=ofx, name="Foo",
	211	ofx = OfxParser.parse(
	212	open(
	213	os.path.join(
	214	'fixtures',
	215	'investment_401k.ofx')))
	216	converter = OfxConverter(account=ofx.account, name="Foo",
139	217	unknownaccount='Expenses:Unknown')
140	218	if len(ofx.account.statement.transactions) > 2:
141	219	# older versions of ofxparse would skip these transactions
142	220	if hasattr(ofx.account.statement.transactions[2], 'tferaction'):
143	221	# unmerged pull request
144		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[2]).format(),
145		"""2014/06/30 Foo: transfer: out
	222	self.assertEqualLedgerPosting(
	223	converter.convert(
	224	ofx.account.statement.transactions[2]).format(),
	225	"""2014/06/30 Foo: transfer: out
	226	Foo -9.060702 BAZ @ $21.928764
146	227	; ofxid: 1234.12345678.123456-01.3
	228	Transfer $198.69
	229	""")
	230	else:
	231	self.assertEqualLedgerPosting(
	232	converter.convert(
	233	ofx.account.statement.transactions[2]).format(),
	234	"""2014/06/30 Foo: transfer
147	235	Foo -9.060702 BAZ @ $21.928764
	236	; ofxid: 1234.12345678.123456-01.3
148	237	Transfer $198.69
149	238	""")
150		else:
151		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[2]).format(),
152		"""2014/06/30 Foo: transfer
153		; ofxid: 1234.12345678.123456-01.3
154		Foo -9.060702 BAZ @ $21.928764
155		Transfer $198.69
156		""")
157
158	239
159	240	def test_position(self):
160		ofx = OfxParser.parse(file(os.path.join('fixtures', 'cusip.ofx')))
161		converter = OfxConverter(ofx=ofx, name="Foo", indent=4,
162		unknownaccount='Expenses:Unknown')
163		self.assertEqual(converter.format_position(ofx.account.statement.positions[0]),
164		"""P 2016/10/08 07:30:08 SHSAX 47.8600000
	241	ofx = OfxParser.parse(open(os.path.join('fixtures', 'cusip.ofx')))
	242	converter = OfxConverter(account=ofx.account, name="Foo", indent=4,
	243	unknownaccount='Expenses:Unknown',
	244	security_list=SecurityList(ofx))
	245	self.assertEqual(
	246	converter.format_position(
	247	ofx.account.statement.positions[0]),
	248	"""P 2016/10/08 07:30:08 SHSAX 47.8600000
165	249	""")
166	250
167	251	def test_dividend(self):
168		ofx = OfxParser.parse(file(os.path.join('fixtures', 'income.ofx')))
169		converter = OfxConverter(ofx=ofx, name="Foo")
170		self.assertEqualLedgerPosting(converter.convert(ofx.account.statement.transactions[0]).format(),
171		"""2016/10/12 DIVIDEND RECEIVED
	252	ofx = OfxParser.parse(open(os.path.join('fixtures', 'income.ofx')))
	253	converter = OfxConverter(account=ofx.account, name="Foo")
	254	self.assertEqualLedgerPosting(
	255	converter.convert(
	256	ofx.account.statement.transactions[0]).format(),
	257	"""2016/10/12 DIVIDEND RECEIVED
172	258	; dividend_from: cusip_redacted
	259	Foo $1234.56
173	260	; ofxid: 1234.12345678.123456-01.redacted
174		Foo $1234.56
175	261	Income:Dividends -$1234.56
176	262	""")
	263
	264	def test_checking_custom_payee(self):
	265	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
	266	converter = OfxConverter(
	267	account=ofx.account,
	268	name="Foo",
	269	payee_format="{memo}")
	270	self.assertEqual(
	271	converter.format_payee(
	272	ofx.account.statement.transactions[0]),
	273	'DIVIDEND EARNED FOR PERIOD OF 03/01/2011 THROUGH 03/31/2011 ANNUAL PERCENTAGE YIELD EARNED IS 0.05%')
	274	converter = OfxConverter(
	275	account=ofx.account,
	276	name="Foo",
	277	payee_format="{payee}")
	278	self.assertEqual(
	279	converter.format_payee(ofx.account.statement.transactions[0]),
	280	'DIVIDEND EARNED FOR PERIOD OF 03')
	281	converter = OfxConverter(
	282	account=ofx.account,
	283	name="Foo",
	284	payee_format="{account}")
	285	self.assertEqual(
	286	converter.format_payee(ofx.account.statement.transactions[0]),
	287	'Foo')
	288	converter = OfxConverter(
	289	account=ofx.account,
	290	name="Foo",
	291	payee_format=" {account} ")
	292	self.assertEqual(
	293	converter.format_payee(ofx.account.statement.transactions[0]),
	294	'Foo')
	295
	296	def test_investments_custom_payee(self):
	297	ofx = OfxParser.parse(
	298	open(
	299	os.path.join(
	300	'fixtures',
	301	'investment_401k.ofx')))
	302	converter = OfxConverter(
	303	account=ofx.account,
	304	name="Foo",
	305	payee_format="{txntype}")
	306	self.assertEqual(
	307	converter.format_payee(ofx.account.statement.transactions[1]),
	308	'transfer')
	309	converter = OfxConverter(
	310	account=ofx.account,
	311	name="Foo",
	312	payee_format="{tferaction}")
	313	self.assertEqual(
	314	converter.format_payee(ofx.account.statement.transactions[1]),
	315	'in')
	316
	317	def test_payee_match(self):
	318	ofx = OfxParser.parse(
	319	open(os.path.join('fixtures', 'checking-payee-match.ofx')))
	320	ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
	321	converter = OfxConverter(account=ofx.account, name="Foo", ledger=ledger)
	322	self.assertEqualLedgerPosting(
	323	converter.convert(
	324	ofx.account.statement.transactions[0]).format(),
	325	"""2011/03/31 Match Payee
	326	Foo -$0.01
	327	; ofxid: 1101.1452687~7.0000489
	328	Expenses:Bar $0.01
	329	""")

+49

-11

tests/test_sync.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18
19	19	import os
20	20	import os.path
21	21	from ofxparse import OfxParser

26	26	from nose.plugins.attrib import attr
27	27	from mock import Mock
28	28
	29
29	30	@attr('generic')
30	31	class TestOfxSync(TestCase):
31	32	def test_fresh_sync(self):
32	33	ledger = Ledger(os.path.join('fixtures', 'empty.lgr'))
33	34	sync = OfxSynchronizer(ledger)
34		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking.ofx')))
	35	ofx = OfxParser.parse(open(os.path.join('fixtures', 'checking.ofx')))
35	36	txns1 = ofx.account.statement.transactions
36		txns2 = sync.filter(ofx)
	37	txns2 = sync.filter(txns1, ofx.account.account_id)
37	38	self.assertEqual(txns1, txns2)
38	39
39	40	def test_sync_order(self):
40	41	ledger = Ledger(os.path.join('fixtures', 'empty.lgr'))
41	42	sync = OfxSynchronizer(ledger)
42		ofx = OfxParser.parse(file(os.path.join('fixtures', 'checking_order.ofx')))
43		txns = sync.filter(ofx)
	43	ofx = OfxParser.parse(
	44	open(
	45	os.path.join(
	46	'fixtures',
	47	'checking_order.ofx')))
	48	txns = sync.filter(
	49	ofx.account.statement.transactions,
	50	ofx.account.account_id)
44	51	self.assertTrue(txns[0].date < txns[1].date and
45	52	txns[1].date < txns[2].date)
46	53
47	54	def test_fully_synced(self):
48	55	ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
49	56	sync = OfxSynchronizer(ledger)
50		(ofx, txns) = sync.parse_file(os.path.join('fixtures', 'checking.ofx'))
	57	ofx = OfxSynchronizer.parse_file(
	58	os.path.join('fixtures', 'checking.ofx'))
	59	txns = sync.filter(
	60	ofx.account.statement.transactions,
	61	ofx.account.account_id)
51	62	self.assertEqual(txns, [])
52	63
53	64	def test_partial_sync(self):
54	65	ledger = Ledger(os.path.join('fixtures', 'checking-partial.lgr'))
55	66	sync = OfxSynchronizer(ledger)
56		(ofx, txns) = sync.parse_file(os.path.join('fixtures', 'checking.ofx'))
	67	ofx = OfxSynchronizer.parse_file(
	68	os.path.join('fixtures', 'checking.ofx'))
	69	txns = sync.filter(
	70	ofx.account.statement.transactions,
	71	ofx.account.account_id)
57	72	self.assertEqual(len(txns), 1)
58	73
59	74	def test_no_new_txns(self):
60	75	ledger = Ledger(os.path.join('fixtures', 'checking.lgr'))
61	76	acct = Mock()
62		acct.download = Mock(return_value=file(os.path.join('fixtures', 'checking.ofx')))
	77	acct.download = Mock(
	78	return_value=open(
	79	os.path.join(
	80	'fixtures',
	81	'checking.ofx')))
63	82	sync = OfxSynchronizer(ledger)
64	83	self.assertEqual(len(sync.get_new_txns(acct, 7, 7)[1]), 0)
65	84
66	85	def test_all_new_txns(self):
67	86	ledger = Ledger(os.path.join('fixtures', 'empty.lgr'))
68	87	acct = Mock()
69		acct.download = Mock(return_value=file(os.path.join('fixtures', 'checking.ofx')))
	88	acct.download = Mock(
	89	return_value=open(
	90	os.path.join(
	91	'fixtures',
	92	'checking.ofx')))
70	93	sync = OfxSynchronizer(ledger)
71	94	self.assertEqual(len(sync.get_new_txns(acct, 7, 7)[1]), 3)
72	95
73	96	def test_comment_txns(self):
74	97	ledger = Ledger(os.path.join('fixtures', 'empty.lgr'))
75	98	sync = OfxSynchronizer(ledger)
76		(ofx, txns) = sync.parse_file(os.path.join('fixtures', 'comments.ofx'))
	99	ofx = OfxSynchronizer.parse_file(
	100	os.path.join('fixtures', 'comments.ofx'))
	101	txns = sync.filter(
	102	ofx.account.statement.transactions,
	103	ofx.account.account_id)
77	104	self.assertEqual(len(txns), 1)
78	105
79	106	def test_sync_no_ledger(self):
80	107	acct = Mock()
81		acct.download = Mock(return_value=file(os.path.join('fixtures', 'checking.ofx')))
	108	acct.download = Mock(
	109	return_value=open(
	110	os.path.join(
	111	'fixtures',
	112	'checking.ofx')))
82	113	sync = OfxSynchronizer(None)
83	114	self.assertEqual(len(sync.get_new_txns(acct, 7, 7)[1]), 3)
	115
84	116
85	117	@attr('generic')
86	118	class TestCsvSync(TestCase):

91	123	2, len(sync.parse_file(
92	124	os.path.join('fixtures', 'paypal.csv'))))
93	125
	126	def test_sync_no_ledger(self):
	127	sync = CsvSynchronizer(None)
	128	self.assertEqual(
	129	2, len(sync.parse_file(
	130	os.path.join('fixtures', 'paypal.csv'))))
	131
94	132	def test_partial_sync(self):
95	133	ledger = Ledger(os.path.join('fixtures', 'paypal.lgr'))
96	134	sync = CsvSynchronizer(ledger)

+21

-6

tests/test_weird_ofx.py less more

15	15	# along with ledger-autosync. If not, see
16	16	# <http://www.gnu.org/licenses/>.
17	17
18		from __future__ import absolute_import
	18
19	19	from ledgerautosync.cli import run
20	20	from ledgerautosync.converter import OfxConverter
21	21	from ledgerautosync.ledgerwrap import Ledger, HLedger, LedgerPython

39	39
40	40	def test_no_institution(self):
41	41	ofxpath = os.path.join('fixtures', 'no-institution.ofx')
42		OfxSynchronizer(self.lgr).parse_file(ofxpath)
	42	sync = OfxSynchronizer(self.lgr)
	43	ofx = OfxSynchronizer.parse_file(ofxpath)
	44	txns = sync.filter(
	45	ofx.account.statement.transactions,
	46	ofx.account.account_id)
	47	self.assertEqual(len(txns), 3)
43	48
44	49	@raises(EmptyInstitutionException)
45	50	def test_no_institution_no_accountname(self):
46	51	ofxpath = os.path.join('fixtures', 'no-institution.ofx')
47		(ofx, txns) = OfxSynchronizer(self.lgr).parse_file(ofxpath)
48		OfxConverter(ofx, name=None)
	52	ofx = OfxSynchronizer.parse_file(ofxpath)
	53	OfxConverter(account=ofx.account, name=None)
49	54
50	55	def test_apostrophe(self):
51	56	ofxpath = os.path.join('fixtures', 'apostrophe.ofx')
52		OfxSynchronizer(self.lgr).parse_file(ofxpath)
	57	ofx = OfxSynchronizer.parse_file(ofxpath)
	58	sync = OfxSynchronizer(self.lgr)
	59	txns = sync.filter(
	60	ofx.account.statement.transactions,
	61	ofx.account.account_id)
	62	self.assertEqual(len(txns), 1)
53	63
54	64	def test_one_settleDate(self):
55	65	ofxpath = os.path.join('fixtures', 'fidelity-one-dtsettle.ofx')
56		OfxSynchronizer(self.lgr).parse_file(ofxpath)
	66	ofx = OfxSynchronizer.parse_file(ofxpath)
	67	sync = OfxSynchronizer(self.lgr)
	68	txns = sync.filter(
	69	ofx.account.statement.transactions,
	70	ofx.account.account_id)
	71	self.assertEqual(len(txns), 17)
57	72
58	73
59	74	@attr('hledger')