Commit 85da301afdd8e8083b6eb379c1a4936a66509996 - feed2imap

+1

-162

ChangeLog less more

0		Feed2Imap 1.0 (18/04/2010)
1		==========================
2		* Removed patch from Haegar as it's no longer needed with the new
3		rubyimap.rb (see discussion in https://gna.org/bugs/?13977)
4		* Support writing to maildirs instead of through IMAP
5		* Use Message-ID instead of X-CacheIndex
6		* Do not use acme.com
7		* Update rubyimap.rb
8		* Provide a way to disable SSL certification verification when
9		connecting to IMAPS accounts
10
11		Feed2Imap 0.9.4 (27/07/2009)
12		============================
13		* Warn (INFO level, so only displayed with feed2imap -v) if fetching a
14		feed takes more than 10s, as this might cause massive delays
15		in feed2imap run times.
16		* Fix encoding of email headers
17		* Only include images once when used several times in the same item
18		* New version of Net::Imap
19		* Use Message-Id instead of X-CacheIndex
20		* Fix MIME formatting when including images
21		* Require ruby-feedparser 0.7, better email formatting
22		* Made it possible to re-use substrings in targets
23		* Fix buffering problem with filters
24		* Added a patch from Haegar to fix problem with dovecot 1.2.1
25
26		Feed2Imap 0.9.3 (23/07/2008)
27		============================
28		* Check the return code from external commands, and warn if != 0. Fixes
29		Gna bug #10516.
30		* Support for including images in the mails (see include-images config
31		option). Based on a patch by Pavel Avgustinov, and with help from
32		Arnt Gulbrandsen.
33		* Dropped rubymail_patch.rb
34		* Added option to wrap text output. Thanks to Maxime Petazzoni for
35		the patch.
36		* When updating an email, remove the Recent flag.
37		* You need to use ruby-feedparser 0.6 or greater with that release.
38
39		Feed2Imap 0.9.2 (28/10/2007)
40		============================
41		* resynchronized rubyimap.rb with stdlib, and fixed send! -> send.
42		* upload items in reverse order, to upload the older first
43		Closes Gna bug #8986. Thanks go do Rial Juan for the patch.
44		* Don't allow more than 16 fetchers to run at the same time.
45		16 should be a reasonable default for everybody.
46		Closes Gna #9032.
47		* Reduce the default HTTP timeout to 30s.
48		* Don't update the cache if uploading items failed (should avoid
49		missing some items).
50		* Safer cache writing. Should avoid some cache corruption problems.
51		* Now exits when we receive an IMAP error, instead of trying to recover. We
52		shouldn't receive IMAP errors anyway. Closes Debian #405070.
53		* Fixed content-type-encoding in HTML emails. Reported by Arnt Gulbrandse.
54
55		Feed2Imap 0.9.1 (15/05/2007)
56		============================
57		* Fixed bug with folder creation.
58
59		Feed2Imap 0.9 (15/05/2007)
60		============================
61		* Folder creation moved to upload. This should make feed2imap run
62		slightly faster.
63		* Per-feed dumpdir option (helps debugging)
64		* Now uses Content-Transfer-Encoding: 8bit (thanks Arnt Gulbrandsen
65		<arnt@gulbrandsen.priv.no>)
66		* Now supports Snowscripts, using the 'execurl' and 'filter' config
67		keywords. For more information, see the example configuration file.
68		* Slightly better option parsing. Thanks to Paul van Tilburg for the
69		patch.
70		* A debug mode was added, and the normal mode was improved, so it is
71		no longer necessary to redirect feed2imap output to /dev/null:
72		transient errors are only reported after they have happened a
73		certain number of times (default 10).
74		* An ignore-hash option was added for feeds whose content change all
75		the time.
76
77		Feed2Imap 0.8 (28/06/2006)
78		============================
79		* Uses the http_proxy environment variable to determine the proxy server
80		if available. (fixes gna bug #5820, all credits go to Boyd Adamson
81		<boyd-adamson@usa.net>)
82		* Fixes flocking on Solaris (fixes gna bug #5819). Again, all credits go to
83		Boyd Adamson <boyd-adamson@usa.net>.
84		* Rewrite of the "find updated and new items" code. It should work much better
85		now. Also, a debug-updated configuration variable was added to make it
86		easier to debug those issues.
87		* New always-new flag in the config file to consider all items as new (for
88		feeds where items are wrongly marked as updated, e.g mediawiki feeds).
89		See example configuration file for more information (fixes Debian bug
90		#366878).
91		* When disconnecting from the IMAP server, don't display an exception in
92		non-verbose mode if the "connection is reset by peer" (fixes Debian bug
93		#367282).
94		* Handling of exceptions in needMIME (fixes gna bug #5872).
95
96		Feed2Imap 0.7 (17/02/2006)
97		============================
98		* Fixes the IMAPS disconnection problem (patch provided by Gael Utard
99		<gael.utard@laposte.net>) (fixes gna bug #2178).
100		* Fixes some issues regarding parallel fetching of feeds.
101		* Now displays the feed creator as sender of emails. (fixes gna bug #5043).
102		* Don't display the password in error messages (fixes debian bug #350370).
103		* Upload mail with the Item's time, not the upload's time (fixes debian
104		bug #350371).
105
106		Feed2Imap 0.6 (11/01/2006)
107		============================
108		* Moved the RSS/Atom parser to a separate library (ruby-feedparser).
109		* Locks the Cache file to avoid concurrent instances of feed2imap.
110		* Issue a warning if the config file is world readable.
111		* Fixed a small bug in Atom feeds parsing.
112		* Fix another bug related to escaped HTML.
113		* Minor fixes.
114
115		Feed2Imap 0.5 (19/09/2005)
116		============================
117		* Fixed a parser problem with items with multiple children in the description.
118		* Mails were upload with \n only, but \r\n are needed.
119		* Feed2Imap can now work without libopenssl.
120		* Fixed a bug in the HTML2Text converter with <a> tags without href.
121		* Reserved characters (eg @) can now be included in the login, password or
122		folder. You just need to escape them.
123		* Feed2Imap is now included in Debian (package name: feed2imap).
124		* Much better handling of feeds with escaped HTML (LinuxFR for example).
125
126		Feed2Imap 0.4 (25/07/2005)
127		============================
128		* now available as a Debian package.
129		* added manpages for everything.
130		* added min-frequency and disable config options. Added doc in example config.
131		* You can now use WordPress's feed:http://something urls in feed2imaprc.
132		* Switched to a real SGML parser for the text version.
133		* Much better output for the text version of emails.
134		* New feed2imap-cleaner to remove old mails seen but not flagged
135		* Feed2Imap version number wasn't displayed in the User-Agent
136		* Better exception handling when parsing errors occur
137		* added feed2imap's RSS feed to the default feeds in the config file
138
139		Feed2Imap 0.3 (04/06/2005)
140		============================
141		* New releases are now advertised using a RSS feed
142		* Cleaner way to manage duplicate IDs (#1773)
143		* Fixed a problem with pseudo-duplicate entries from Mediawiki
144		* Fixed a problem with updated items being seen as updated at each update.
145		* Fixed a problem when the disconnection from the IMAP server failed.
146		reported by Ludovic Gomez <ludogomez@chez.com>
147
148		Feed2Imap 0.2 (30/04/2005)
149		============================
150		* Fixed a problem with feeds with strange caching bugs (old items going away
151		and coming back)
152		* The text version is now encoded in iso-8859-1 instead of utf-8.
153		* The subject is now MIME-encoded in utf-8. It works with mutt & evo.
154		* No longer overwrite mail flags (Read, Important,..) when updating an item.
155		* HTTP fetching is now multithreaded and is much faster (about 300%).
156		* Fetching over HTTPS works.
157		* HTTP fetching is fully unit-tested.
158
159		Feed2Imap 0.1 (02/04/2005)
160		==========================
161		* first public release.
	0	See the git history

+1

-1

README less more

3	3
4	4	Currently, all the information is provided on
5	5
6		http://home.gna.org/feed2imap
	6	https://github.com/feed2imap/feed2imap
7	7
8	8	Copyright (c) 2005-2010 Lucas Nussbaum <lucas@lucas-nussbaum.net>
9	9

+9

-1

Rakefile less more

2	2	require 'rake/packagetask'
3	3	require 'rake'
4	4	require 'find'
	5	require_relative "lib/feed2imap/version"
5	6
6	7	task :default => [:test]
7	8
8	9	PKG_NAME = 'feed2imap'
9		PKG_VERSION = `ruby -Ilib -rfeed2imap/feed2imap -e 'print F2I_VERSION'`
	10	PKG_VERSION = Feed2Imap::VERSION
10	11	PKG_FILES = [ 'ChangeLog', 'README', 'COPYING', 'setup.rb', 'Rakefile']
11	12	Find.find('bin/', 'lib/', 'test/', 'data/') do \|f\|
12	13	if FileTest.directory?(f) and f =~ /\.svn/

64	65	rescue LoadError
65	66	puts "Will not generate gem."
66	67	end
	68
	69	task :release => :repackage do
	70	sh 'git', 'tag', 'v' + PKG_VERSION
	71	sh 'git', 'push'
	72	sh 'git', 'push', '--tags'
	73	sh 'gem', 'push', "pkg/#{PKG_NAME}/#{PKG_VERSION}.gem"
	74	end

+1

-1

bin/feed2imap less more

42	42	end
43	43
44	44	if version
45		puts "Feed2Imap v.#{F2I_VERSION}"
	45	puts "Feed2Imap v.#{Feed2Imap::VERSION}"
46	46	else
47	47	Feed2Imap::new(verbose, cacherebuild, configf)
48	48	end

+32

-0

feed2imap.gemspec less more

	0	#########################################################
	1	# This file has been automatically generated by gem2tgz #
	2	#########################################################
	3	# -- encoding: utf-8 --
	4	# stub: feed2imap 1.2.6 ruby lib
	5
	6	Gem::Specification.new do \|s\|
	7	s.name = "feed2imap".freeze
	8	s.version = "1.2.6"
	9
	10	s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
	11	s.require_paths = ["lib".freeze]
	12	s.authors = ["Lucas Nussbaum".freeze]
	13	s.date = "2021-01-14"
	14	s.description = "RSS/Atom feed aggregator".freeze
	15	s.executables = ["feed2imap".freeze, "feed2imap-cleaner".freeze, "feed2imap-dumpconfig".freeze, "feed2imap-opmlimport".freeze]
	16	s.files = ["COPYING".freeze, "ChangeLog".freeze, "README".freeze, "Rakefile".freeze, "bin/feed2imap".freeze, "bin/feed2imap-cleaner".freeze, "bin/feed2imap-dumpconfig".freeze, "bin/feed2imap-opmlimport".freeze, "data/doc/feed2imap/examples/feed2imaprc".freeze, "data/man/man1/feed2imap-cleaner.1".freeze, "data/man/man1/feed2imap-dumpconfig.1".freeze, "data/man/man1/feed2imap-opmlimport.1".freeze, "data/man/man1/feed2imap.1".freeze, "data/man/man5/feed2imaprc.5".freeze, "lib/feed2imap.rb".freeze, "lib/feed2imap/cache.rb".freeze, "lib/feed2imap/config.rb".freeze, "lib/feed2imap/feed2imap.rb".freeze, "lib/feed2imap/html2text-parser.rb".freeze, "lib/feed2imap/httpfetcher.rb".freeze, "lib/feed2imap/imap.rb".freeze, "lib/feed2imap/itemtomail.rb".freeze, "lib/feed2imap/maildir.rb".freeze, "lib/feed2imap/rexml_patch.rb".freeze, "lib/feed2imap/sgml-parser.rb".freeze, "lib/feed2imap/version.rb".freeze, "setup.rb".freeze, "test/maildir/cur/1376317520.15784_1.debian:2,S".freeze, "test/maildir/cur/1376317520.15789_1.debian:2,S".freeze, "test/maildir/cur/1376319137.17850_1.debian:2,".freeze, "test/maildir/cur/1376320022.18396_5.debian:2,FS".freeze, "test/maildir/new/1376320099.18396_7.debian".freeze, "test/tc_cache.rb".freeze, "test/tc_config.rb".freeze, "test/tc_httpfetcher.rb".freeze, "test/tc_maildir.rb".freeze]
	17	s.rubygems_version = "2.5.2.1".freeze
	18	s.summary = "RSS/Atom feed aggregator".freeze
	19
	20	if s.respond_to? :specification_version then
	21	s.specification_version = 4
	22
	23	if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
	24	s.add_runtime_dependency(%q<ruby-feedparser>.freeze, [">= 0.9"])
	25	else
	26	s.add_dependency(%q<ruby-feedparser>.freeze, [">= 0.9"])
	27	end
	28	else
	29	s.add_dependency(%q<ruby-feedparser>.freeze, [">= 0.9"])
	30	end
	31	end

+2

-1

lib/feed2imap/config.rb less more

16	16	Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17	17	=end
18	18
	19	require 'cgi'
19	20	require 'yaml'
20	21	require 'uri'
21	22	require 'feed2imap/imap'

70	71	f['name'] = f['name'].to_s
71	72	if f['disable'].nil?
72	73	uri = URI::parse(Array(f['target']).join(''))
73		path = URI::unescape(uri.path)
	74	path = CGI::unescape(uri.path)
74	75	if uri.scheme == 'maildir'
75	76	@feeds.push(ConfigFeed::new(f, maildir_account, path, self))
76	77	else

+3

-4

lib/feed2imap/feed2imap.rb less more

16	16	Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
17	17	=end
18	18
19		# Feed2Imap version
20		F2I_VERSION = '1.2.5'
21	19	F2I_WARNFETCHTIME = 10
22	20
	21	require 'feed2imap/version'
23	22	require 'feed2imap/config'
24	23	require 'feed2imap/cache'
25	24	require 'feed2imap/httpfetcher'

31	30
32	31	class Feed2Imap
33	32	def Feed2Imap.version
34		return F2I_VERSION
	33	return Feed2Imap::VERSION
35	34	end
36	35
37	36	def initialize(verbose, cacherebuild, configfile)

44	43	else
45	44	@logger.level = Logger::WARN
46	45	end
47		@logger.info("Feed2Imap V.#{F2I_VERSION} started")
	46	@logger.info("Feed2Imap V.#{Feed2Imap::VERSION} started")
48	47	# reading config
49	48	@logger.info('Reading configuration file ...')
50	49	if not File::exist?(configfile)

+2

-1

lib/feed2imap/imap.rb less more

22	22	require 'openssl'
23	23	rescue LoadError
24	24	end
	25	require 'cgi'
25	26	require 'uri'
26	27
27	28	# This class is a container of IMAP accounts.

76	77	# use given port if port given
77	78	port = uri.port if uri.port
78	79	@connection = Net::IMAP::new(uri.host, port, usessl, nil, !@@no_ssl_verify)
79		user, password = URI::unescape(uri.userinfo).split(':',2)
	80	user, password = CGI::unescape(uri.userinfo).split(':',2)
80	81	@connection.login(user, password)
81	82	self
82	83	end

+4

-19

lib/feed2imap/itemtomail.rb less more

24	24	require 'feedparser'
25	25	require 'feedparser/text-output'
26	26	require 'feedparser/html-output'
	27	require 'feed2imap/version'
27	28	require 'base64'
28	29	require 'rmail'
29	30	require 'digest/md5'

64	65	else
65	66	message.header['Date'] = item.date.rfc2822
66	67	end
67		message.header['X-Feed2Imap-Version'] = F2I_VERSION if defined?(F2I_VERSION)
	68	message.header['X-Feed2Imap-Version'] = Feed2Imap::VERSION
68	69	message.header['Message-Id'] = id
69	70	message.header['X-F2IStatus'] = "Updated" if updated
70	71	# treat subject. Might need MIME encoding.

99	100	# $2 contains url, $3 the image name, $4 the image extension
100	101	begin
101	102	fetcher = HTTPFetcher.new
102		image = Base64.encode64(fetcher.fetch($2, Time.at(0)).chomp) + "\n"
103		cid = "#{Digest::MD5.hexdigest($2)}@#{config.hostname}"
104		if not cids.include?(cid)
105		cids << cid
106		imgpart = RMail::Message.new
107		imgpart.header.set('Content-ID', "<#{cid}>")
108		type = $4
109		type = 'jpeg' if type.downcase == 'jpg' # hack hack hack
110		imgpart.header.set('Content-Type', "image/#{type}", 'name' => $3)
111		imgpart.header.set('Content-Disposition', 'attachment', 'filename' => $3)
112		imgpart.header.set('Content-Transfer-Encoding', 'base64')
113		imgpart.body = image
114		imgs << imgpart
115		end
116		# now to specify what to replace with
117		newtag = "#{$1}src=\"cid:#{cid}\"#{$5}"
118		#print "#{cid}: Replacing '#{$&}' with '#{newtag}'...\n"
119		newtag
	103	image = Base64.encode64(fetcher.fetch($2, Time.at(0)).chomp)
	104	"#{$1}src=\"data:image/#{$4};base64,#{image}\"#{$5}"
120	105	rescue
121	106	#print "Error while fetching image #{$2}: #{$!}...\n"
122	107	$& # don't modify on exception

+1

-2

lib/feed2imap/maildir.rb less more

38	38	dir = folder_dir(folder)
39	39	guarantee_maildir(dir)
40	40	mail_files = find_mails(dir, idx)
41		flags = nil
42	41	if mail_files.length > 0
43	42	# get the info from the first result and delete everything
44	43	info = maildir_file_info(mail_files[0])

122	121	File.link tmp_path, new_path
123	122	stored = true
124	123	ensure
125		File.unlink tmp_path if File.exists? tmp_path
	124	File.unlink tmp_path if File.exist? tmp_path
126	125	end
127	126	end
128	127

+3

-0

lib/feed2imap/version.rb less more

	0	class Feed2Imap
	1	VERSION = "1.2.6"
	2	end