Import upstream version 7.1+git20111204.671ccf7, md5 36e44d338af4bc3b2a41de689b5200b6
Debian Janitor
4 years ago
0 | build |
45 | 45 | unshield |
46 | 46 | |
47 | 47 | rar archives |
48 | unrar | |
48 | unar or unrar | |
49 | ||
50 | arj archives | |
51 | arj | |
49 | 52 | |
50 | 53 | Files compressed with gzip or compress |
51 | 54 | zcat |
58 | 61 | |
59 | 62 | Files compressed with xz |
60 | 63 | xzcat |
64 | ||
65 | Files compressed with lrzip | |
66 | lrzcat | |
67 | ||
68 | Files compressed with lzip | |
69 | lzip | |
61 | 70 | |
62 | 71 | Installation |
63 | 72 | ------------ |
0 | 0 | Changes in dtrx |
1 | 1 | =============== |
2 | ||
3 | Version 7.2 | |
4 | ----------- | |
5 | ||
6 | Thanks to Ville Skyllä, who contributed most of the new features and | |
7 | enhancements in this release. | |
8 | ||
9 | New features | |
10 | ~~~~~~~~~~~~ | |
11 | ||
12 | * dtrx now supports the arj archive, lrzip encoding, and several specific | |
13 | file extensions. | |
14 | ||
15 | * If unar is available, dtrx can try to use it to extract rar archives. | |
16 | ||
17 | Bug fixes | |
18 | ~~~~~~~~~ | |
19 | ||
20 | * dtrx will get correct file magic information for archives it's reading | |
21 | through symbolic links. | |
22 | ||
23 | * File listings for rar archives now include full paths. | |
24 | ||
25 | Development changes | |
26 | ~~~~~~~~~~~~~~~~~~~ | |
27 | ||
28 | * dtrx development is now `hosted on Gitorious`_. | |
29 | ||
30 | .. _hosted on Gitorious: http://gitorious.org/dtrx | |
31 | ||
32 | * The test script can run specific tests, and has improved output on | |
33 | interactive terminals. | |
2 | 34 | |
3 | 35 | Version 7.1 |
4 | 36 | ----------- |
0 | Metadata-Version: 1.0 | |
1 | Name: dtrx | |
2 | Version: 7.1 | |
3 | Summary: Script to intelligently extract multiple archive types | |
4 | Home-page: http://www.brettcsmith.org/2007/dtrx/ | |
5 | Author: Brett Smith | |
6 | Author-email: brettcsmith@brettcsmith.org | |
7 | License: GNU General Public License, version 3 or later | |
8 | Download-URL: http://www.brettcsmith.org/2007/dtrx/ | |
9 | Description: dtrx extracts archives in a number of different | |
10 | formats; it currently supports tar, zip (including self-extracting | |
11 | .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield | |
12 | files. It can also decompress files compressed with gzip, bzip2, | |
13 | lzma, xz, or compress. | |
14 | ||
15 | In addition to providing one command to handle many different archive | |
16 | types, dtrx also aids the user by extracting contents consistently. | |
17 | By default, everything will be written to a dedicated directory | |
18 | that's named after the archive. dtrx will also change the | |
19 | permissions to ensure that the owner can read and write all those | |
20 | files. | |
21 | Platform: UNKNOWN | |
22 | Classifier: Development Status :: 5 - Production/Stable | |
23 | Classifier: Environment :: Console | |
24 | Classifier: Intended Audience :: End Users/Desktop | |
25 | Classifier: Intended Audience :: System Administrators | |
26 | Classifier: License :: OSI Approved :: GNU General Public License (GPL) | |
27 | Classifier: Natural Language :: English | |
28 | Classifier: Operating System :: POSIX | |
29 | Classifier: Programming Language :: Python | |
30 | Classifier: Topic :: Utilities |
40 | 40 | |
41 | 41 | dtrx extracts archives in a number of different formats; it currently |
42 | 42 | supports tar, zip (including self-extracting .exe files), cpio, rpm, deb, |
43 | gem, 7z, cab, rar, lzh, and InstallShield files. It can also decompress | |
44 | files compressed with gzip, bzip2, lzma, xz, or compress. | |
43 | gem, 7z, cab, rar, lzh, arj, and InstallShield files. It can also decompress | |
44 | files compressed with gzip, bzip2, lzma, xz, lrzip, lzip, or compress. | |
45 | 45 | |
46 | 46 | In addition to providing one command to handle many different archive |
47 | 47 | types, dtrx also aids the user by extracting contents consistently. By |
0 | -*- text -*- | |
1 | ||
2 | To do: | |
3 | ||
4 | Things which I have a use case/anti-use case for: | |
5 | * Support pisi packages (http://paketler.pardus.org.tr/pardus-2007/) | |
6 | * Steal ideas from <http://martin.ankerl.com/files/e>. | |
7 | * More consistently raise and handle exceptions. | |
8 | ||
9 | Things that are generally good: | |
10 | * Better tests. | |
11 | * Better error messages. | |
12 | ||
13 | Things I think might be good but can't prove: | |
14 | * Use zipfile instead of the zip commands. | |
15 | * Processing from stdin. | |
16 | * shar support. | |
17 | * --expert mode: prompts don't show an explanation of what the options are, | |
18 | unless you ask with ?. |
0 | From Peter.Kelemen@gmail.com Fri Sep 19 18:51:00 2008 | |
1 | Return-Path: <Peter.Kelemen@gmail.com> | |
2 | Date: Sat, 20 Sep 2008 00:51:17 +0200 | |
3 | From: KELEMEN Peter <Peter.Kelemen@gmail.com> | |
4 | To: Brett Smith <brettcsmith@brettcsmith.org> | |
5 | Subject: [PATCH] dtrx: RAR support | |
6 | Message-ID: <20080919225117.GA993@kaylee> | |
7 | MIME-Version: 1.0 | |
8 | Content-Type: text/plain; charset=us-ascii | |
9 | Content-Disposition: inline | |
10 | User-Agent: Mutt/1.5.17+20080114 (2008-01-14) | |
11 | Status: RO | |
12 | X-Status: A | |
13 | Content-Length: 1589 | |
14 | Lines: 48 | |
15 | ||
16 | Brett, | |
17 | ||
18 | The following patch adds RAR support for dtrx. | |
19 | ||
20 | HTH, | |
21 | Peter | |
22 | ||
23 | diff -r ececf7836546 scripts/dtrx | |
24 | --- a/scripts/dtrx Tue Sep 16 21:51:59 2008 -0400 | |
25 | +++ b/scripts/dtrx Sat Sep 20 00:50:16 2008 +0200 | |
26 | @@ -515,6 +515,25 @@ class ShieldExtractor(NoPipeExtractor): | |
27 | return result | |
28 | ||
29 | ||
30 | +class RarExtractor(NoPipeExtractor): | |
31 | + file_type = 'RAR archive' | |
32 | + extract_command = ['unrar', 'x'] | |
33 | + list_command = ['unrar', 'l'] | |
34 | + border_re = re.compile('^-+$') | |
35 | + | |
36 | + def get_filenames(self): | |
37 | + inside = False | |
38 | + for line in NoPipeExtractor.get_filenames(self): | |
39 | + if self.border_re.match(line): | |
40 | + if inside: | |
41 | + break | |
42 | + else: | |
43 | + inside = True | |
44 | + elif inside: | |
45 | + yield line.split(' ')[1] | |
46 | + self.archive.close() | |
47 | + | |
48 | + | |
49 | class BaseHandler(object): | |
50 | def __init__(self, extractor, options): | |
51 | self.extractor = extractor | |
52 | @@ -775,6 +794,10 @@ class ExtractorBuilder(object): | |
53 | 'mimetypes': ('x-cab',), | |
54 | 'extensions': ('cab',), | |
55 | 'magic': ('Microsoft Cabinet Archive',)}, | |
56 | + 'rar': {'extractor': RarExtractor, | |
57 | + 'mimetypes': ('rar',), | |
58 | + 'extensions': ('rar'), | |
59 | + 'magic': ('RAR archive')}, | |
60 | 'shield': {'extractor': ShieldExtractor, | |
61 | 'mimetypes': ('x-cab',), | |
62 | 'extensions': ('cab', 'hdr'), | |
63 | ||
64 | ||
65 | From Peter.Kelemen@gmail.com Wed Sep 24 09:42:18 2008 | |
66 | X-Original-To: me@brettcsmith.org | |
67 | Date: Wed, 24 Sep 2008 15:31:18 +0200 | |
68 | From: KELEMEN Peter <Peter.Kelemen@gmail.com> | |
69 | To: Brett Smith <me@brettcsmith.org> | |
70 | Subject: Re: [PATCH] dtrx: RAR support | |
71 | Message-ID: <20080924133118.GG8943@kyra> | |
72 | References: <20080919225117.GA993@kaylee> <20080921180522.GB15446@canonical.org> | |
73 | MIME-Version: 1.0 | |
74 | Content-Type: text/plain; charset=utf-8 | |
75 | Content-Disposition: inline | |
76 | Content-Transfer-Encoding: 8bit | |
77 | In-Reply-To: <20080921180522.GB15446@canonical.org> | |
78 | User-Agent: Mutt/1.5.17+20080114 (2008-01-14) | |
79 | Status: RO | |
80 | Content-Length: 684 | |
81 | Lines: 25 | |
82 | ||
83 | * Brett Smith (me@brettcsmith.org) [20080921 14:05]: | |
84 | ||
85 | > Thanks very much for this; it looks great. | |
86 | ||
87 | You're welcome. | |
88 | ||
89 | > Can you please confirm for me that it's licensed under GPLv3 or | |
90 | > any later version? A simple "yes" will do. :) | |
91 | ||
92 | Sure, GPLv3 is OK. | |
93 | ||
94 | > Also, do you want me to add any copyright notice(s) for your | |
95 | > contribution? | |
96 | ||
97 | I'm ambivalent, I trust your judgement. Should you choose to do | |
98 | so, please use my Peter.Kelemen@gmail.com address. | |
99 | ||
100 | Thanks, | |
101 | Peter | |
102 | ||
103 | -- | |
104 | .+'''+. .+'''+. .+'''+. .+'''+. .+'' | |
105 | Kelemen Péter / \ / \ Peter.Kelemen@gmail.com | |
106 | .+' `+...+' `+...+' `+...+' `+...+' | |
107 | ||
108 | ||
109 | From - Thu Dec 01 18:53:16 2011 | |
110 | Return-Path: <ville.skytta@iki.fi> | |
111 | X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on llewellyn | |
112 | X-Spam-Level: | |
113 | X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,SPF_PASS | |
114 | autolearn=ham version=3.3.1 | |
115 | X-Original-To: me@brettcsmith.org | |
116 | Delivered-To: me@brettcsmith.org | |
117 | Received: from filtteri1.pp.htv.fi (filtteri1.pp.htv.fi [213.243.153.184]) | |
118 | by brettcsmith.org (Postfix) with ESMTP id A51BA67A6 | |
119 | for <me@brettcsmith.org>; Thu, 1 Dec 2011 02:22:59 -0500 (EST) | |
120 | Received: from localhost (localhost [127.0.0.1]) | |
121 | by filtteri1.pp.htv.fi (Postfix) with ESMTP id CC02018B3AA | |
122 | for <me@brettcsmith.org>; Thu, 1 Dec 2011 09:22:58 +0200 (EET) | |
123 | X-Virus-Scanned: Debian amavisd-new at pp.htv.fi | |
124 | Received: from smtp5.welho.com ([213.243.153.39]) | |
125 | by localhost (filtteri1.pp.htv.fi [213.243.153.184]) (amavisd-new, port 10024) | |
126 | with ESMTP id nX7Bob2-Ku0p for <me@brettcsmith.org>; | |
127 | Thu, 1 Dec 2011 09:22:56 +0200 (EET) | |
128 | Received: from viper.bobcat.mine.nu (cs181085020.pp.htv.fi [82.181.85.20]) | |
129 | (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) | |
130 | (No client certificate requested) | |
131 | by smtp5.welho.com (Postfix) with ESMTPS id 50F675BC002 | |
132 | for <me@brettcsmith.org>; Thu, 1 Dec 2011 09:22:56 +0200 (EET) | |
133 | Message-ID: <4ED72B50.60000@iki.fi> | |
134 | Date: Thu, 01 Dec 2011 09:22:56 +0200 | |
135 | From: =?ISO-8859-1?Q?Ville_Skytt=E4?= <ville.skytta@iki.fi> | |
136 | User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Fedora/3.1.16-1.fc14 Thunderbird/3.1.16 | |
137 | MIME-Version: 1.0 | |
138 | To: Brett Smith <me@brettcsmith.org> | |
139 | Subject: Re: [PATCH 0 of 3] Some trivial dtrx patches | |
140 | References: <patchbomb.1322320204@viper.bobcat.mine.nu> <4ED459D2.4090703@brettcsmith.org> <4ED52917.900@iki.fi> <4ED5B081.4040505@brettcsmith.org> <4ED68074.70500@iki.fi> <4ED6C398.8050101@brettcsmith.org> | |
141 | In-Reply-To: <4ED6C398.8050101@brettcsmith.org> | |
142 | X-Enigmail-Version: 1.1.2 | |
143 | Content-Type: text/plain; charset=ISO-8859-1 | |
144 | Content-Transfer-Encoding: 8bit | |
145 | ||
146 | On 12/01/2011 02:00 AM, Brett Smith wrote: | |
147 | > On 11/30/2011 02:13 PM, Ville Skyttä wrote: | |
148 | >> BTW, you mentioned earlier that you were applying some of my initial | |
149 | >> trival patches but I don't see them in the public hg repo yet, did you | |
150 | >> find a problem with them? | |
151 | > | |
152 | > No, no problem. That's really weird -- they started showing up at | |
153 | > <http://brettcsmith.org/2007/dtrx/dtrx/> as soon as I pushed them, and | |
154 | > when I clone that repository with hg they show up in the log too. How | |
155 | > are you checking this? | |
156 | ||
157 | hg pull -u. But I see the two first ones are indeed in the repository, | |
158 | so it's possible that I've managed to mess up something locally. No | |
159 | problem, I'll probably just throw away my earlier clone. | |
160 | ||
161 | > I'm happy with the second round of patches too, but I meant to ask | |
162 | > before I apply them: since they're more substantive, can I go ahead and | |
163 | > add your copyright line to the source code? And can you please just | |
164 | > confirm that you're okay with this code being distributed under GPLv3 or | |
165 | > later? | |
166 | ||
167 | Yes, go ahead, and GPLv3+ is fine. (That means that I may need to | |
168 | switch from GPLv2+ to GPLv3+ for some other utilities I have that may be | |
169 | using dtrx in the future, but that shouldn't be a problem.) | |
170 | ||
171 |
3 | 3 | # dtrx -- Intelligently extract various archive types. |
4 | 4 | # Copyright © 2006-2011 Brett Smith <brettcsmith@brettcsmith.org> |
5 | 5 | # Copyright © 2008 Peter Kelemen <Peter.Kelemen@gmail.com> |
6 | # Copyright © 2011 Ville Skyttä <ville.skytta@iki.fi> | |
6 | 7 | # |
7 | 8 | # This program is free software; you can redistribute it and/or modify it |
8 | 9 | # under the terms of the GNU General Public License as published by the |
48 | 49 | VERSION_BANNER = """dtrx version %s |
49 | 50 | Copyright © 2006-2011 Brett Smith <brettcsmith@brettcsmith.org> |
50 | 51 | Copyright © 2008 Peter Kelemen <Peter.Kelemen@gmail.com> |
52 | Copyright © 2011 Ville Skyttä <ville.skytta@iki.fi> | |
51 | 53 | |
52 | 54 | This program is free software; you can redistribute it and/or modify it |
53 | 55 | under the terms of the GNU General Public License as published by the |
82 | 84 | mimetypes.encodings_map.setdefault('.lzma', 'lzma') |
83 | 85 | mimetypes.encodings_map.setdefault('.xz', 'xz') |
84 | 86 | mimetypes.encodings_map.setdefault('.lz', 'lzip') |
87 | mimetypes.encodings_map.setdefault('.lrz', 'lrzip') | |
85 | 88 | mimetypes.types_map.setdefault('.gem', 'application/x-ruby-gem') |
86 | 89 | |
87 | 90 | logger = logging.getLogger('dtrx-log') |
140 | 143 | |
141 | 144 | class BaseExtractor(object): |
142 | 145 | decoders = {'bzip2': ['bzcat'], 'gzip': ['zcat'], 'compress': ['zcat'], |
143 | 'lzma': ['lzcat'], 'xz': ['xzcat'], 'lzip': ['lzip', '-cd']} | |
146 | 'lzma': ['lzcat'], 'xz': ['xzcat'], 'lzip': ['lzip', '-cd'], | |
147 | 'lrzip': ['lrzcat', '-q'], 'lrz': ['lrzcat', '-q']} | |
144 | 148 | name_checker = DirectoryChecker |
145 | 149 | |
146 | 150 | def __init__(self, filename, encoding): |
567 | 571 | border_re = re.compile(r'^[-\+]+$') |
568 | 572 | |
569 | 573 | def get_filenames(self): |
570 | fn_index = None | |
571 | 574 | filenames = NoPipeExtractor.get_filenames(self) |
572 | 575 | for line in filenames: |
573 | 576 | if self.border_re.match(line): |
607 | 610 | class RarExtractor(NoPipeExtractor): |
608 | 611 | file_type = 'RAR archive' |
609 | 612 | extract_command = ['unrar', 'x'] |
610 | list_command = ['unrar', 'l'] | |
613 | list_command = ['unrar', 'v'] | |
611 | 614 | border_re = re.compile('^-+$') |
612 | 615 | |
613 | 616 | def get_filenames(self): |
614 | 617 | inside = False |
618 | isfile = True | |
615 | 619 | for line in NoPipeExtractor.get_filenames(self): |
616 | 620 | if self.border_re.match(line): |
617 | 621 | if inside: |
619 | 623 | else: |
620 | 624 | inside = True |
621 | 625 | elif inside: |
622 | yield line.split(' ')[1] | |
626 | if isfile: | |
627 | yield line.strip() | |
628 | isfile = not isfile | |
629 | self.archive.close() | |
630 | ||
631 | ||
632 | class UnarchiverExtractor(NoPipeExtractor): | |
633 | file_type = 'RAR archive' | |
634 | extract_command = ['unar', '-D'] | |
635 | list_command = ['lsar'] | |
636 | ||
637 | def get_filenames(self): | |
638 | output = NoPipeExtractor.get_filenames(self) | |
639 | output.next() | |
640 | for line in output: | |
641 | end_index = line.rfind('(') | |
642 | yield line[:end_index].strip() | |
643 | ||
644 | ||
645 | class ArjExtractor(NoPipeExtractor): | |
646 | file_type = 'ARJ archive' | |
647 | extract_command = ['arj', 'x', '-y'] | |
648 | list_command = ['arj', 'v'] | |
649 | prefix_re = re.compile(r'^\d+\)\s+') | |
650 | ||
651 | def get_filenames(self): | |
652 | for line in NoPipeExtractor.get_filenames(self): | |
653 | match = self.prefix_re.match(line) | |
654 | if match: | |
655 | yield line[match.end():] | |
623 | 656 | self.archive.close() |
624 | 657 | |
625 | 658 | |
888 | 921 | 'magic': ('POSIX tar archive',)}, |
889 | 922 | 'zip': {'extractors': (ZipExtractor, SevenExtractor), |
890 | 923 | 'mimetypes': ('zip',), |
891 | 'extensions': ('zip',), | |
924 | 'extensions': ('zip', 'jar', 'epub', 'xpi'), | |
892 | 925 | 'magic': ('(Zip|ZIP self-extracting) archive',)}, |
893 | 926 | 'lzh': {'extractors': (LZHExtractor,), |
894 | 927 | 'mimetypes': ('x-lzh', 'x-lzh-compressed'), |
895 | 928 | 'extensions': ('lzh', 'lha'), |
896 | 'magic': ('LHa [\d\.\?]+ archive',)}, | |
929 | 'magic': (r'LHa [\d\.\?]+ archive',)}, | |
897 | 930 | 'rpm': {'extractors': (RPMExtractor,), |
898 | 931 | 'mimetypes': ('x-redhat-package-manager', 'x-rpm'), |
899 | 932 | 'extensions': ('rpm',), |
919 | 952 | 'mimetypes': ('x-cab',), |
920 | 953 | 'extensions': ('cab',), |
921 | 954 | 'magic': ('Microsoft Cabinet Archive',)}, |
922 | 'rar': {'extractors': (RarExtractor,), | |
955 | 'rar': {'extractors': (RarExtractor, UnarchiverExtractor), | |
923 | 956 | 'mimetypes': ('rar',), |
924 | 957 | 'extensions': ('rar',), |
925 | 958 | 'magic': ('RAR archive',)}, |
959 | 'arj': {'extractors': (ArjExtractor,), | |
960 | 'mimetypes': ('arj',), | |
961 | 'extensions': ('arj',), | |
962 | 'magic': ('ARJ archive',)}, | |
926 | 963 | 'shield': {'extractors': (ShieldExtractor,), |
927 | 964 | 'mimetypes': ('x-cab',), |
928 | 965 | 'extensions': ('cab', 'hdr'), |
950 | 987 | for mapping in (('tar', 'bzip2', 'tar.bz2', 'tbz2', 'tb2', 'tbz'), |
951 | 988 | ('tar', 'gzip', 'tar.gz', 'tgz'), |
952 | 989 | ('tar', 'lzma', 'tar.lzma', 'tlz'), |
953 | ('tar', 'xz', 'tar.xz'), | |
990 | ('tar', 'xz', 'tar.xz', 'txz'), | |
954 | 991 | ('tar', 'lz', 'tar.lz'), |
955 | 992 | ('tar', 'compress', 'tar.Z', 'taz'), |
993 | ('tar', 'lrz', 'tar.lrz'), | |
956 | 994 | ('compress', 'gzip', 'Z', 'gz'), |
957 | 995 | ('compress', 'bzip2', 'bz2'), |
958 | 996 | ('compress', 'lzma', 'lzma'), |
959 | ('compress', 'xz', 'xz')): | |
997 | ('compress', 'xz', 'xz'), | |
998 | ('compress', 'lrzip', 'lrz')): | |
960 | 999 | for extension in mapping[2:]: |
961 | 1000 | extension_map.setdefault(extension, []).append(mapping[:2]) |
962 | 1001 | |
965 | 1004 | ('gzip', 'gzip compressed'), |
966 | 1005 | ('lzma', 'LZMA compressed'), |
967 | 1006 | ('lzip', 'lzip compressed'), |
1007 | ('lrzip', 'LRZIP compressed'), | |
968 | 1008 | ('xz', 'xz compressed')): |
969 | 1009 | for pattern in mapping[1:]: |
970 | 1010 | magic_encoding_map[re.compile(pattern)] = mapping[0] |
1019 | 1059 | magic_map_matches = classmethod(magic_map_matches) |
1020 | 1060 | |
1021 | 1061 | def try_by_magic(cls, filename): |
1022 | process = subprocess.Popen(['file', '-z', filename], | |
1062 | process = subprocess.Popen(['file', '-zL', filename], | |
1023 | 1063 | stdout=subprocess.PIPE) |
1024 | 1064 | status = process.wait() |
1025 | 1065 | if status != 0: |
21 | 21 | 'Topic :: Utilities'], |
22 | 22 | long_description = """dtrx extracts archives in a number of different |
23 | 23 | formats; it currently supports tar, zip (including self-extracting |
24 | .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield | |
25 | files. It can also decompress files compressed with gzip, bzip2, | |
26 | lzma, xz, or compress. | |
24 | .exe files), cpio, rpm, deb, gem, 7z, cab, rar, lzh, arj, and | |
25 | InstallShield files. It can also decompress files compressed with gzip, | |
26 | bzip2, lzma, xz, lrzip, lzip, or compress. | |
27 | 27 | |
28 | 28 | In addition to providing one command to handle many different archive |
29 | 29 | types, dtrx also aids the user by extracting contents consistently. |
16 | 16 | # You should have received a copy of the GNU General Public License along |
17 | 17 | # with this program; if not, see <http://www.gnu.org/licenses/>. |
18 | 18 | |
19 | import fcntl | |
19 | 20 | import os |
20 | 21 | import re |
22 | import struct | |
21 | 23 | import subprocess |
22 | import yaml | |
23 | 24 | import sys |
24 | 25 | import tempfile |
26 | import termios | |
27 | import yaml | |
25 | 28 | |
26 | 29 | try: |
27 | 30 | set |
39 | 42 | DTRX_SCRIPT = os.path.realpath('../scripts/dtrx') |
40 | 43 | SHELL_CMD = ['sh', '-se'] |
41 | 44 | ROOT_DIR = os.path.realpath(os.curdir) |
42 | OUTCOMES = ['error', 'failed', 'passed'] | |
45 | NUM_TESTS = 0 | |
43 | 46 | |
44 | 47 | class ExtractorTestError(Exception): |
45 | 48 | pass |
46 | 49 | |
47 | 50 | |
51 | class StatusWriter(object): | |
52 | def __init__(self): | |
53 | try: | |
54 | size = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, | |
55 | struct.pack("HHHH", 0, 0, 0, 0)) | |
56 | except IOError: | |
57 | self.show = self.show_file | |
58 | else: | |
59 | self.width = struct.unpack("HHHH", size)[1] - 1 | |
60 | self.last_width = self.width | |
61 | self.show = self.show_term | |
62 | ||
63 | def show_term(self, message): | |
64 | sys.stdout.write(message.ljust(self.last_width) + "\r") | |
65 | sys.stdout.flush() | |
66 | self.last_width = max(self.width, len(message)) | |
67 | ||
68 | def show_file(self, message): | |
69 | if message: | |
70 | print message | |
71 | ||
72 | def clear(self): | |
73 | self.show("") | |
74 | ||
75 | ||
48 | 76 | class ExtractorTest(object): |
77 | status_writer = StatusWriter() | |
78 | ||
49 | 79 | def __init__(self, **kwargs): |
80 | global NUM_TESTS | |
81 | NUM_TESTS += 1 | |
82 | self.test_num = NUM_TESTS | |
50 | 83 | setattr(self, 'name', kwargs['name']) |
51 | 84 | setattr(self, 'options', kwargs.get('options', '-n').split()) |
52 | 85 | setattr(self, 'filenames', kwargs.get('filenames', '').split()) |
86 | 119 | directory_hint = '../' |
87 | 120 | else: |
88 | 121 | directory_hint = '' |
89 | self.start_proc(SHELL_CMD + [directory_hint], commands) | |
122 | self.start_proc(SHELL_CMD + [directory_hint], commands).wait() | |
90 | 123 | |
91 | 124 | def get_shell_results(self): |
92 | 125 | self.run_script('prerun') |
122 | 155 | raise ExtractorTestError("cleanup exited with status code %s" % |
123 | 156 | (status,)) |
124 | 157 | |
125 | def show_status(self, status, message=None): | |
126 | raw_status = status.lower() | |
127 | if raw_status != 'passed': | |
128 | self.outbuffer.seek(0, 0) | |
129 | sys.stdout.write(self.outbuffer.read(-1)) | |
158 | def show_pass(self): | |
159 | self.status_writer.show("Passed %i/%i: %s" % | |
160 | (self.test_num, NUM_TESTS, self.name)) | |
161 | return 'passed' | |
162 | ||
163 | def show_report(self, status, message=None): | |
164 | self.status_writer.clear() | |
165 | self.outbuffer.seek(0, 0) | |
166 | sys.stdout.write(self.outbuffer.read(-1)) | |
130 | 167 | if message is None: |
131 | 168 | last_part = '' |
132 | 169 | else: |
133 | 170 | last_part = ': %s' % (message,) |
134 | print "%7s: %s%s" % (status, self.name, last_part) | |
135 | return raw_status | |
171 | print "%s: %s%s\n" % (status, self.name, last_part) | |
172 | return status.lower() | |
136 | 173 | |
137 | 174 | def compare_results(self, actual): |
138 | 175 | posttest_result = self.get_posttest_result() |
144 | 181 | print >>self.outbuffer, '\n'.join(expected.difference(actual)) |
145 | 182 | print >>self.outbuffer, "Only in actual results:" |
146 | 183 | print >>self.outbuffer, '\n'.join(actual.difference(expected)) |
147 | return self.show_status('FAILED') | |
184 | return self.show_report('FAILED') | |
148 | 185 | elif posttest_result != 0: |
149 | 186 | print >>self.outbuffer, "Posttest gave status code", posttest_result |
150 | return self.show_status('FAILED') | |
151 | return self.show_status('Passed') | |
187 | return self.show_report('FAILED') | |
188 | return self.show_pass() | |
152 | 189 | |
153 | 190 | def have_error_mismatch(self, status): |
154 | 191 | if self.error and (status == 0): |
182 | 219 | problem = (self.have_error_mismatch(status) or |
183 | 220 | self.check_output(output) or self.grep_output(output)) |
184 | 221 | if problem: |
185 | return self.show_status('FAILED', problem) | |
222 | return self.show_report('FAILED', problem) | |
186 | 223 | if self.baseline is not None: |
187 | 224 | return self.compare_results(actual) |
188 | 225 | else: |
189 | 226 | self.clean() |
190 | return self.show_status('Passed') | |
227 | return self.show_pass() | |
191 | 228 | |
192 | 229 | def run(self): |
193 | 230 | self.outbuffer = tempfile.TemporaryFile() |
197 | 234 | try: |
198 | 235 | result = self.check_results() |
199 | 236 | except ExtractorTestError, error: |
200 | result = self.show_status('ERROR', error) | |
237 | result = self.show_report('ERROR', error) | |
201 | 238 | self.outbuffer.close() |
202 | 239 | if self.directory: |
203 | 240 | os.chdir(ROOT_DIR) |
206 | 243 | return result |
207 | 244 | |
208 | 245 | |
209 | test_db = open('tests.yml') | |
210 | test_data = yaml.load(test_db.read(-1)) | |
211 | test_db.close() | |
212 | tests = [ExtractorTest(**data) for data in test_data] | |
213 | for original_data in test_data: | |
214 | if (original_data.has_key('directory') or | |
215 | (not original_data.has_key('baseline'))): | |
216 | continue | |
217 | data = original_data.copy() | |
218 | data['name'] += ' in ..' | |
219 | data['directory'] = 'inside-dir' | |
220 | data['filenames'] = ' '.join(['../%s' % filename for filename in | |
221 | data.get('filenames', '').split()]) | |
222 | tests.append(ExtractorTest(**data)) | |
223 | results = [test.run() for test in tests] | |
224 | counts = {} | |
225 | for outcome in OUTCOMES: | |
226 | counts[outcome] = 0 | |
227 | for result in results: | |
228 | counts[result] += 1 | |
229 | print " Totals:", ', '.join(["%s %s" % (counts[key], key) for key in OUTCOMES]) | |
246 | class TestsRunner(object): | |
247 | outcomes = ['error', 'failed', 'passed'] | |
248 | ||
249 | def __init__(self): | |
250 | test_db = open('tests.yml') | |
251 | self.test_data = yaml.load(test_db.read(-1)) | |
252 | test_db.close() | |
253 | self.name_regexps = [re.compile(s) for s in sys.argv[1:]] | |
254 | self.tests = [ExtractorTest(**data) for data in self.test_data | |
255 | if self.wanted_test(data)] | |
256 | self.add_subdir_tests() | |
257 | ||
258 | def wanted_test(self, data): | |
259 | if not self.name_regexps: | |
260 | return True | |
261 | return filter(None, [r.search(data['name']) for r in self.name_regexps]) | |
262 | ||
263 | def add_subdir_tests(self): | |
264 | for odata in self.test_data: | |
265 | if ((not self.wanted_test(odata)) or odata.has_key('directory') or | |
266 | (not odata.has_key('baseline'))): | |
267 | continue | |
268 | data = odata.copy() | |
269 | data['name'] += ' in ..' | |
270 | data['directory'] = 'inside-dir' | |
271 | data['filenames'] = ' '.join(['../%s' % filename for filename in | |
272 | data.get('filenames', '').split()]) | |
273 | self.tests.append(ExtractorTest(**data)) | |
274 | ||
275 | def run(self): | |
276 | results = {} | |
277 | for outcome in self.outcomes: | |
278 | results[outcome] = 0 | |
279 | for test in self.tests: | |
280 | results[test.run()] += 1 | |
281 | if self.tests: | |
282 | self.tests[-1].status_writer.clear() | |
283 | print "Totals:", ', '.join(["%s %s" % (results[key], key) | |
284 | for key in self.outcomes]) | |
285 | return (results["error"] + results["failed"]) == 0 | |
286 | ||
287 | ||
288 | runner = TestsRunner() | |
289 | if not runner.run(): | |
290 | sys.exit(1) |
Binary diff not shown
Binary diff not shown
Binary diff not shown
Binary diff not shown
0 | # tests.yml -- Whole-program comparison tests for dtrx | |
1 | # Copyright © 2006-2011 Brett Smith <brettcsmith@brettcsmith.org> | |
2 | # Copyright © 2011 Ville Skyttä <ville.skytta@iki.fi> | |
3 | # | |
4 | # This program is free software; you can redistribute it and/or modify it | |
5 | # under the terms of the GNU General Public License as published by the | |
6 | # Free Software Foundation; either version 3 of the License, or (at your | |
7 | # option) any later version. | |
8 | # | |
9 | # This program is distributed in the hope that it will be useful, but | |
10 | # WITHOUT ANY WARRANTY; without even the implied warranty of | |
11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General | |
12 | # Public License for more details. | |
13 | # | |
14 | # You should have received a copy of the GNU General Public License along | |
15 | # with this program; if not, see <http://www.gnu.org/licenses/>. | |
16 | ||
0 | 17 | - name: basic .tar |
1 | 18 | filenames: test-1.23.tar |
2 | 19 | baseline: | |
13 | 30 | mkdir test-1.23 |
14 | 31 | cd test-1.23 |
15 | 32 | tar -jxf ../$1 |
33 | ||
34 | - name: basic .tar.lrz | |
35 | filenames: test-1.23.tar.lrz | |
36 | baseline: | | |
37 | lrzcat $1 | tar -xf - | |
16 | 38 | |
17 | 39 | - name: basic .zip |
18 | 40 | filenames: test-1.23.zip |
64 | 86 | baseline: | |
65 | 87 | cpio -i --make-directories <$1 |
66 | 88 | antigrep: blocks? |
89 | ||
90 | - name: basic .rar | |
91 | filenames: test-1.23.rar | |
92 | baseline: | | |
93 | mkdir test-1.23 | |
94 | cd test-1.23 | |
95 | unar -D ../$1 || unrar x ../$1 | |
96 | ||
97 | - name: basic .arj | |
98 | filenames: test-1.23.arj | |
99 | baseline: | | |
100 | mkdir test-1.23 | |
101 | cd test-1.23 | |
102 | arj x -y ../$1 | |
67 | 103 | |
68 | 104 | - name: .deb metadata |
69 | 105 | filenames: test-1.23_all.deb |
128 | 164 | posttest: | |
129 | 165 | exec [ "$(cat test-text)" = "hi" ] |
130 | 166 | |
167 | - name: decompressing lrzip, not interactive | |
168 | directory: inside-dir | |
169 | filenames: ../test-text.lrz | |
170 | options: "" | |
171 | antigrep: "." | |
172 | baseline: | | |
173 | lrzcat $1 >test-text | |
174 | posttest: | | |
175 | exec [ "$(cat test-text)" = "hi" ] | |
176 | ||
131 | 177 | - name: decompressing lzip, not interactive |
132 | 178 | directory: inside-dir |
133 | 179 | filenames: ../test-text.lz |
254 | 300 | 1/2/3 |
255 | 301 | a/ |
256 | 302 | a/b |
303 | foobar | |
304 | ||
305 | - name: list contents of .arj | |
306 | options: -n -l | |
307 | filenames: test-1.23.arj | |
308 | output: | | |
309 | a/b | |
310 | 1/2/3 | |
257 | 311 | foobar |
258 | 312 | |
259 | 313 | - name: list contents of .cpio |
0 | body { padding: 0 3em; } | |
1 | h1, h2, h3 { margin-left: -1em; } | |
2 | span.pname { font-family: monospace; } | |
3 | pre { margin-left: 2em; } |
0 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" | |
1 | "http://www.w3.org/TR/html4/strict.dtd"> | |
2 | ||
3 | <html><head><title>dtrx: Intelligent archive extraction</title> | |
4 | <link rel="stylesheet" href="common.css"> | |
5 | <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> | |
6 | </head> | |
7 | <body> | |
8 | <h1>dtrx: Intelligent archive extraction</h1> | |
9 | ||
10 | <h2>Introduction</h2> | |
11 | ||
12 | <p><span class="pname">dtrx</span> stands for “Do The Right | |
13 | Extraction.” It's a tool for Unix-like systems that takes all the | |
14 | hassle out of extracting archives. Here's an example of how you use | |
15 | it:</p> | |
16 | ||
17 | <pre>$ dtrx linux-3.0.1.tar.bz2</pre> | |
18 | ||
19 | <p>That's basically the same thing as:</p> | |
20 | ||
21 | <pre>$ tar -jxf linux-3.0.1.tar.bz2</pre> | |
22 | ||
23 | <p>But there's more to it than that. You know those really annoying files | |
24 | that don't put everything in a dedicated directory, and have the | |
25 | permissions all wrong?</p> | |
26 | ||
27 | <pre>$ tar -zvxf random-tarball.tar.gz | |
28 | foo | |
29 | bar | |
30 | data/ | |
31 | data/text | |
32 | $ cd data/ | |
33 | cd: permission denied: data</pre> | |
34 | ||
35 | <p><span class="pname">dtrx</span> takes care of all those problems for | |
36 | you, too:</p> | |
37 | ||
38 | <pre>$ dtrx random-tarball.tar.gz | |
39 | $ cd random-tarball/data | |
40 | $ cat text | |
41 | This all works properly.</pre> | |
42 | ||
43 | <p><span class="pname">dtrx</span> is simple and powerful. Just use the | |
44 | same command for all your archive files, and they'll never frustrate you | |
45 | again.</p> | |
46 | ||
47 | <h2>Features</h2> | |
48 | ||
49 | <ul> | |
50 | ||
51 | <li><strong>Handles many archive types</strong>: You only need to remember | |
52 | one simple command to extract | |
53 | ||
54 | <span class="pname">tar</span>, | |
55 | <span class="pname">zip</span>, | |
56 | <span class="pname">cpio</span>, | |
57 | <span class="pname">deb</span>, | |
58 | <span class="pname">rpm</span>, | |
59 | <span class="pname">gem</span>, | |
60 | <span class="pname">7z</span>, | |
61 | <span class="pname">cab</span>, | |
62 | <span class="pname">lzh</span>, | |
63 | <span class="pname">rar</span>, | |
64 | <span class="pname">arj</span>, | |
65 | <span class="pname">gz</span>, | |
66 | <span class="pname">bz2</span>, | |
67 | <span class="pname">lzma</span>, | |
68 | <span class="pname">xz</span>, | |
69 | <span class="pname">lrzip</span>, | |
70 | <span class="pname">lzip</span>, | |
71 | and many kinds of | |
72 | <span class="pname">exe</span> files, including Microsoft Cabinet archives, | |
73 | InstallShield archives, and self-extracting <span class="pname">zip</span> | |
74 | files. | |
75 | ||
76 | If they have any extra compression, like <span class="pname">tar.bz2</span> | |
77 | files, <span class="pname">dtrx</span> will take care of that for you, | |
78 | too.</li> | |
79 | ||
80 | <li><strong>Keeps everything organized</strong>: <span | |
81 | class="pname">dtrx</span> will make sure that archives are extracted into | |
82 | their own dedicated directories.</li> | |
83 | ||
84 | <li><strong>Sane permissions</strong>: <span class="pname">dtrx</span> makes | |
85 | sure you can read and write all the files you just extracted, while leaving | |
86 | the rest of the permissions intact.</li> | |
87 | ||
88 | <li><strong>Recursive extraction</strong>: <span class="pname">dtrx</span> can | |
89 | find archives inside the archive and extract those too.</li> | |
90 | ||
91 | </ul> | |
92 | ||
93 | <h2>Download</h2> | |
94 | ||
95 | <p><a href="dtrx-7.1.tar.gz">Download <span class="pname">dtrx</span> | |
96 | 7.1</a>. The SHA1 checksum for this file | |
97 | is <tt>05cfe705a04a8b84571b0a5647cd2648720791a4</tt>. Improvements in this | |
98 | release include:</p> | |
99 | ||
100 | <ul> | |
101 | ||
102 | <li>Support for LZH archives.</li> | |
103 | <li>Minor bug fixes in handling recursive extraction and empty | |
104 | archives.</li> | |
105 | ||
106 | </ul> | |
107 | ||
108 | <p>If you would like to try the latest development version—or maybe do some | |
109 | work on it yourself—you can check out the | |
110 | project's <a href="http://git-scm.com/">Git</a> | |
111 | repository. A | |
112 | <a href="http://gitorious.org/dtrx">web repository</a> is | |
113 | available, or you can just run:</p> | |
114 | ||
115 | <pre>$ git clone git://gitorious.org/dtrx/dtrx.git</pre> | |
116 | ||
117 | <h2>Requirements</h2> | |
118 | ||
119 | <p>If you have Python 2.4 or greater, this should work out of the box. If | |
120 | you're stuck on Python 2.3, you can use this if you install | |
121 | the <a href="http://www.lysator.liu.se/~astrand/popen5/">subprocess | |
122 | module</a>. You'll need the usual tools for the archive types you want to | |
123 | extract: for example, if you're extracting <span class="pname">zip</span> | |
124 | files, you'll need <span class="pname">zipinfo</span> | |
125 | and <span class="pname">unzip</span>. See the <tt>INSTALL</tt> file included | |
126 | with <span class="pname">dtrx</span> for a complete list of necessary | |
127 | utilities.</p> | |
128 | ||
129 | <h2>Installation</h2> | |
130 | ||
131 | <p>You can just put <span class="pname">scripts/dtrx</span> wherever is | |
132 | convenient for you, but if you want to install the program system-wide, you | |
133 | can also run the following command as root or equivalent:</p> | |
134 | ||
135 | <pre>python setup.py install --prefix=/usr/local</pre> | |
136 | ||
137 | <p>See the included <tt>INSTALL</tt> file for more information.</p> | |
138 | ||
139 | </body> | |
140 | </html> |